Off-road road recognition method based on a transformer network

Abstract

This invention discloses an off-road road recognition method based on Transformer networks, aiming to solve the problems of low accuracy, poor adaptability, and difficulty in adapting to complex scenarios in existing off-road road recognition methods. The method includes: collecting off-road road data through multiple sensors, obtaining depth images through mapping matrix transformation and interpolation; constructing a customized dataset containing RGB and depth images, and enhancing feature recognition through data augmentation techniques; building a model that integrates feature modules, a Transformer encoder, and a lightweight MLP decoder, replacing positional encoding with 3×3 convolutions, optimizing the attenuation ratio of the self-attention mechanism, and training the model with appropriate parameters. This invention improves the accuracy and robustness of target recognition in complex off-road environments through multi-sensor fusion and model structure optimization, solving the accuracy attenuation problem caused by resolution differences, and balancing recognition efficiency and deployment feasibility. It can be widely applied to scenarios such as unmanned vehicles and special-purpose vehicles, and has high engineering value.

Description

Technical Field

[0001] This invention relates to the field of intelligent sensing technology for tracked vehicles, specifically to a road recognition method for off-road vehicles based on Transformer networks, which aims to improve the accuracy and real-time performance of road recognition for off-road vehicles in complex off-road environments. Background Technology

[0002] In complex and ever-changing off-road scenarios, off-road vehicles require accurate road perception information to ensure driving safety and efficiency. Traditional road recognition methods mainly rely on image information acquired by visual sensors, using image processing techniques to identify road areas. However, road conditions in off-road environments are complex and diverse, making it difficult for traditional methods to fully cover all situations, and they are also challenging to debug. With the development of deep learning technology, road recognition methods based on semantic segmentation have demonstrated higher recognition accuracy and robustness due to their ability to automatically extract image features and perform pixel-level classification. Summary of the Invention

[0003] To address the aforementioned issues, this invention provides an off-road road recognition method based on Transformer networks. This method improves the accuracy and robustness of target recognition in complex off-road environments through multi-sensor fusion and model structure optimization, solves the problem of accuracy attenuation caused by resolution differences, and balances recognition efficiency and deployment feasibility. It can be widely applied to scenarios such as unmanned vehicles and special operation vehicles, and has high engineering value.

[0004] To solve the above problems, the technical solution adopted by the present invention is as follows: A method for off-road road recognition based on Transformer networks includes the following steps: Step 1: Off-road road data collection: Using a data collection platform equipped with a color camera, LiDAR, IMU accelerometer sensor and GPS navigation system, the system travels on various off-road roads and records sensor road data packets using the ROS system framework; Save point cloud data at preset time intervals, extract the color image with the smallest time difference based on the timestamp, and obtain the color image, the corresponding point cloud and the joint calibration result; Step 2: Data preprocessing: Using the Labelme tool, manually plot points from the outside in to outline the closed contour, and label the passable area of ​​the off-road road, rock obstacles, tree obstacles, and road background; The three-dimensional coordinates and reflection intensity information of the point cloud data are converted into a bin format file. The mapping matrix is ​​used to realize the mutual conversion between the lidar coordinate system and the RGB image coordinate system, and the three-dimensional point cloud data is mapped into a point cloud mapping image of the same size as the RGB image. The point cloud mapping image is completed using the two-dimensional linear interpolation method in OpenCV and the griddata function of the SciPy library to obtain the depth image. Step 3: Construction of Off-Road Scene Road Dataset: Save the labeled images in Pascal voc2007 dataset format, and linearly overlay them with the original images to form the RGB image portion of the dataset; Perform the preprocessing operation in step 2 on the point cloud data corresponding to the RGB images to obtain depth images; combine the RGB images and depth images into a dataset with a total capacity of 1025 images, divide it into a training set of 700 images and a validation set of 325 images, and uniformly adjust the resolution to 1280×720. Step 4: Building and training the off-road road recognition model: Constructing a network model containing a feature fusion module, an encoder, and a decoder; The feature fusion module fuses RGB image features and depth image features through channel stitching and convolution operations; The encoder employs a multi-layer Transformer module, introduces a self-attention mechanism that sets the attenuation ratio R, and replaces the position encoding module with a 3×3 convolution module; The decoder employs a lightweight multilayer perceptron, and achieves semantic segmentation through channel dimension transformation, spatial dimension transformation, feature concatenation and channel compression, and classification prediction. The PyTorch framework was used to train the model based on the dataset. During training, the input image size was adjusted to 1152×640, and data augmentation and standardization were employed to set the training parameters.

[0005] Preferably, the mapping matrix expression in step two is:

[0006] Where M is a (3,4)-dimensional mapping matrix, I is a (3,3)-dimensional camera intrinsic parameter matrix, R is a (3,3)-dimensional rotation matrix from the lidar coordinate system to the camera coordinate system, and T is a (3,1)-dimensional translation matrix from the lidar coordinate system to the camera coordinate system. All parameters are derived from the joint calibration results of the camera and lidar.

[0007] Preferably, the RGB colors of the labels in step two are defined as: passable area (0,128,0), stone (255,255,0), tree (0,0,128), and road background (128,0,0).

[0008] Preferably, the attenuation ratio R of the multi-layer Transformer module in step four is set to [64, 16, 4, 1] in sequence.

[0009] Preferably, the parameters for model training in step four are set as follows: the optimizer is AdamW, the number of iterations is 60,000, the initial learning rate is 0.001, the learning rate decay factor is 0.01, the number of learning rate decay iterations is 1,000, the number of model save interval iterations is 2,000, and the training device is an NVIDIA GeForce RTX3090 GPU.

[0010] Preferably, the data augmentation methods in step four include random scaling, random cropping, random adjustment of hue, random adjustment of brightness, and random adjustment of saturation, which are used to enhance the feature recognition of the input image.

[0011] Preferably, the fusion process of the feature fusion module in step four is as follows: first, the RGB features extracted from the RGB image are... Depth features extracted from depth images Perform channel splicing operation Then perform a 3×3 convolution operation on the concatenated feature map. , obtain fusion features The fusion expression is: .

[0012] Preferably, the specific implementation process of the lightweight multilayer perceptron in step four is as follows: (1) Perform channel dimension transformation on multi-scale features to unify the channel dimensions of all features:

[0013] (2) Perform spatial dimension transformation on the multi-scale features after unifying the channel dimensions to unify the spatial dimensions of all features:

[0014] (3) Features with uniform size and dimensions are spliced ​​together, and the spliced ​​features are then subjected to channel compression:

[0015] (4) Perform classification prediction on the compressed features and output pixel-level classification results: .

[0016] Preferably, the specific way to replace the position encoding module with the 3×3 convolution module in step four is as follows: embed the 3×3 convolution module into the feedforward network, utilize the translation invariance of the semantic segmentation task to dynamically express position information, and avoid the decrease in accuracy caused by the difference in resolution between the test image and the training dataset.

[0017] The present invention also provides an off-road road recognition system based on Transformer network, including a data acquisition module, a data preprocessing module, a dataset construction module, and a model training and recognition module; The data acquisition module includes a color camera, a lidar, an IMU accelerometer, a GPS navigation system, and a ROS data recording unit, used to perform the operation in step one; The data preprocessing module is used to perform the operation in step two; The dataset construction module is used to perform the operation in step three; The model training and recognition module is used to perform the operation in step four.

[0018] The beneficial effects of this invention are as follows: 1. This method employs a multi-sensor fusion scheme combining a color camera and LiDAR. A mapping matrix enables precise coordinate system transformation, converting 3D point cloud data into depth images. This is combined with RGB image features to form complementary information. The feature fusion module enhances feature correlation through channel stitching and 3×3 convolution operations, effectively compensating for the limitations of single sensors in complex off-road environments (such as changing lighting and obstacle occlusion). It can accurately distinguish traversable areas, rocks, trees, and other targets, significantly reducing the false positive rate. Furthermore, a customized dataset containing 1025 labeled samples, coupled with multi-dimensional data augmentation techniques, further enhances the model's adaptability to different off-road conditions.

[0019] 2. The encoder adopts a self-attention mechanism with multi-layer Transformer modules and a decay rate gradient setting, which can capture global feature associations and focus on local details. It innovatively uses a 3×3 convolution module to replace the traditional position encoding, and dynamically expresses position information based on the translation invariance of semantic segmentation. This completely solves the problem of accuracy decay caused by the difference in resolution between the test image and the training dataset in the existing technology, so that the model can be adapted to image inputs from different acquisition devices and different scenarios, and the generalization ability is significantly improved.

[0020] 3. The decoder adopts a lightweight multilayer perceptron. Through optimization designs such as dimensional unification and feature compression, it significantly reduces the model parameter size and computational overhead while ensuring semantic segmentation accuracy. With the AdamW optimizer and reasonable training parameter settings, the model can be trained efficiently on an NVIDIA GeForce RTX3090 GPU. After training, the model has a fast inference speed, which can meet the real-time recognition requirements in off-road scenarios and provide technical support for the field deployment of equipment such as unmanned vehicles and special operation vehicles.

[0021] 4. A complete closed-loop technical solution is formed, encompassing multi-sensor data acquisition, standardized preprocessing, customized dataset construction, model building, and training. The accompanying recognition system features clearly defined modules, with the data acquisition module integrating multiple sensors and a ROS recording unit, allowing direct application to various off-road data acquisition scenarios without additional modifications. The method and system work together to meet both algorithm verification in laboratory environments and the complex needs of actual off-road operations, demonstrating broad application scenarios and high engineering value and promotional significance. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of off-road road data collection according to the present invention; Figure 2 This is a schematic diagram of the RGB portion of the image in this invention; Figure 3 This is a schematic diagram of the image depth portion of the present invention; Figure 4 This is a schematic diagram of the SegNet network architecture of the present invention; Figure 5 This is a schematic diagram of the overall network framework of the present invention; Figure 6 This is a schematic diagram of the fusion method of the present invention; Figure 7 This is a schematic diagram illustrating the semantic segmentation results of the present invention; Figure 8 This is a schematic diagram of the identification process of the present invention. Detailed Implementation

[0023] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0024] Reference Figure 1-8 A method for off-road road recognition based on Transformer networks includes the following steps: Step 1: Off-road scenario road data collection The data acquisition platform is equipped with a color camera, LiDAR, IMU accelerometer, and a GPS navigation system. After joint calibration of the camera and LiDAR, it travels on various off-road terrains and uses the ROS system framework to record road data packets obtained from all sensors.

[0025] To extract the corresponding color images and point clouds from the collected continuous time data packets, the point cloud data is first saved at certain time intervals. Then, the color image with the smallest time difference is extracted according to the timestamp. The final data includes the color image, the corresponding point cloud, and the joint calibration results. The off-road scene road acquisition results are as follows: Figure 1 As shown.

[0026] Step 2: Data Preprocessing The goal of off-road environment recognition is primarily to identify traversable areas and static obstacles on off-road terrain. Traversable areas refer to the regions where vehicles can drive normally in off-road environments, while static obstacles mainly include rocks and trees. The goal of off-road type classification is to categorize the diverse road types found on off-road terrain. The road type identification results significantly influence the selection of the road drag coefficient, thereby affecting the vehicle's steering control stability.

[0027] To address the challenge of locating road boundaries and obstacles in unstructured off-road environments, existing solutions primarily fall into two categories: object recognition using bounding boxes to delineate targets, and image segmentation classifying all pixels in the image. Since the shapes of passable areas and obstacles in off-road environments are highly random and cannot be effectively identified using bounding boxes, this study employs semantic segmentation for information annotation. Labelme is used as the annotation tool, which manually plots points sequentially to outline closed contours and then annotates them, following an outside-to-inside principle. This yields labels for passable off-road areas, rock obstacles, and the road background, which are then matched and fused with the collected point cloud data.

[0028] Before fusion processing, the acquired point cloud data needs to be format-converted using a semantic segmentation neural network, converting the 3D coordinates and reflection intensity information in the point cloud data into a BIN format file. The key to matching the point cloud data with the RGB image lies in the coordinate system transformation, specifically converting the LiDAR coordinate system to the RGB image coordinate system and vice versa. This transformation can be achieved using the following mapping matrix, referencing previous research:

[0029] The definitions of each parameter are shown in Table 1. All parameters are based on the joint calibration results in the previous study

[44] . The specific methods for obtaining them will not be repeated here. The mapping matrix can map the three-dimensional point cloud data onto the two-dimensional image data. The point cloud data in front of the vehicle can be sized and projected to obtain a point cloud mapping image of the same size as the RGB image.

[0030] Table 1 Parameter Meaning

[0031] To make neural network training more efficient, the more accurate the input data, the better. Therefore, the obtained point cloud mapping image needs to be further supplemented. The two-dimensional linear interpolation method in OpenCV is used for image completion. The griddata function in the SciPy library of OpenCV is used. This function has the function of performing two-dimensional linear interpolation on randomly scattered pixels. This method can obtain more detailed depth images for the complex and variable road conditions in off-road environments.

[0032] Step 2: Building the Off-Road Scene Road Dataset The labeled images, annotated using the Labelme tool, were saved in Pascal voc2007 dataset format, as shown in Table 2. The labels for the four target objects have different colors, such as... Figure 2 As shown, the generated label images are linearly superimposed on the original images to obtain the RGB image portion of the off-road road recognition dataset, which contains 1025 RGB images. The point cloud data of these 1025 original images are then subjected to the aforementioned preprocessing operations to obtain the following... Figure 3 The depth images shown complete the construction of the off-road road environment recognition dataset, which has a total capacity of 1025 images, including 700 images for training and 325 images for validation, with all resolutions adjusted to 1280×720.

[0033] Table 2 Data Label Statistics

[0034] Step 4: Building the Off-Road Recognition Model First, deep learning methods are used to fuse RGB and depth images in the dataset, and then this fusion is used as input to the network model. Existing research shows that convolutional neural networks can be used to fuse RGB and depth images to achieve RGBD semantic segmentation. Semantic segmentation classifies all pixels in an image into different regions, thus labeling different objects. The final result corresponds to all pixels in a three-dimensional tensor, which requires upsampling to make the feature map size consistent with the original image size. Figure 4 As shown, the encoder-decoder structure mentioned in the above research—the SegNet network—indexes positional information during the upsampling process, thereby solving the problem of feature loss during downsampling, resulting in better segmentation performance, and has become a commonly used structure for semantic segmentation networks.

[0035] This network model was built using ideas from SegNet, and the overall network framework is as follows: Figure 5As shown, the feature fusion module first fuses the RGB image and the depth image using the method of feature map channel concatenation, such as... Figure 6 As shown, the feature maps from the RGB image and depth image branches are first concatenated by channels and then fused using convolution to obtain the fused feature map, as expressed below:

[0036] in , , These represent RGB features, depth features, and fusion features, respectively. This represents the convolution operation. This indicates a channel splicing operation.

[0037] After fusing features from RGB and depth images, these images are used as input to the network model. To achieve faster speed and higher accuracy while avoiding interference, the encoder employs a multi-layer Transformer module, enabling the recognition and extraction of both high-resolution shallow features and low-resolution fine features. Adding a self-attention mechanism to the Transformer module significantly improves the capture of temporal input features; however, this mechanism greatly impacts the overall complexity of the network. The traditional self-attention mechanism works as follows:

[0038] in The dimension is The computational complexity is The computational complexity is closely related to the size of the input image, and becomes excessively high for high-resolution images. To address this issue, a decay ratio R is introduced, and a fully connected layer is used to reduce the computational complexity. The specific implementation process is as follows:

[0039]

[0040] First, a reshape operation is used to change the dimensions of K from [N, C] to [N / R, C*R]. Then, a fully connected layer is used to further change it to [N / R, C]. At this point, the computational complexity becomes... In this multi-layer Transformer module, the attenuation ratio R is set to [64, 16, 4, 1] in sequence.

[0041] Considering that the original positional encoding module is fixed during model training, if the resolution of the test image is different from that of the training dataset, the positional encoding module will perform linear interpolation, which will lead to a decrease in accuracy. However, semantic segmentation is a pixel-level classification task, and the impact of positional encoding on pixel-level classification is negligible. Furthermore, semantic segmentation is translation-invariant for classification and recognition tasks, meaning that changes in image scale and angle do not affect the output results. Therefore, a 3×3 convolutional module is used to replace the positional encoding module in the feedforward network to achieve dynamic representation of positional information.

[0042] Given that the layered Transformer encoder has a larger receptive field than the traditional CNN encoder, a lightweight multilayer perceptron (MLP) is used as the decoder. This avoids redundant computations and reduces problems such as manual design and high computational requirements. Good results can be achieved with minimal computation. The specific process mainly consists of four steps: (1) Perform channel dimension transformation on multi-scale features to unify the dimensions:

[0043] (2) Perform spatial dimension transformation on multi-scale features to unify dimensions:

[0044] (3) Feature splicing and channel compression:

[0045] (4) Prediction based on classification:

[0046] The specific parameters of the encoder in this network are shown in Table 3, where K is the kernel, S is the stride, P is the padding, C is the dimension, R is the scaling ratio of the self-attention mechanism, N is the number of heads in the self-attention mechanism, E is the inflation rate of the feedforward network layer, and L is the number of encoders in the Transformer.

[0047] Table 3 Encoder Parameters

[0048] The advantages of the invention are further illustrated by examining the actual model training results. The PyTorch framework was used to train the network model, starting with the off-road road recognition model. Due to limitations in computer memory for network training, all input images were uniformly resized to 1152×640. Random scaling, cropping, and randomization of hue, brightness, and saturation were employed to make the input image features more prominent. Furthermore, it was found that standardizing the input image data could accelerate the network training convergence speed. Specific training parameters are shown in Table 4, and the training device used was an NVIDIA GeForce RTX3090 GPU.

[0049] Table 4. Parameters related to the training of the off-road road recognition model

[0050] The evaluation metrics for pixel-based multi-class classification problems mainly include accuracy (ACC), recall, intersection over union (IoU), mean intersection over union (MIoU), Kappa coefficient, and Dice coefficient. These metrics are inseparable from the basic confusion matrix, the specific form of which is shown in Table 5. Table 5 Confusion Matrix

[0051] The calculation formulas for each evaluation indicator are as follows: The meanings of all parameters are shown in Table 6.

[0052] Table 6 Parameter Meaning

[0053] The off-road detection model showed good convergence on the training set, and the evaluation results on the validation set are shown in Tables 7 and 8.

[0054] Table 7 Model Indicators

[0055] Table 8 Evaluation results of the off-road testing model (%)

[0056] In terms of accuracy, the model performs better in semantic segmentation of passable areas, achieving over 80% accuracy in recognizing large rocks, trees, and backgrounds. Multi-class recognition accuracy exceeds 90%, and the average intersection-over-union ratio (IoU), Kappa coefficient, and Dice coefficient all exceed 80%, indicating a high overall detection and recognition capability for off-road terrain. Although the obstacle recognition accuracy did not reach 90%, its recall rate was around 80% and IoU exceeded 60%, indicating that the model can detect static obstacles. However, it struggles to accurately detect the size of small or significantly occluded static obstacles such as rocks and trees. This is because each image in the training set contains passable areas but not necessarily obstacles like rocks and trees; the edge shapes of obstacles are more randomized than passable areas; and the overall proportion of obstacles is much smaller than that of passable areas. Figure 7 As shown, considering both the validation set examples and practical application requirements, the segmentation and recognition performance of this off-road road detection model is acceptable.

[0057] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying off-road terrain based on Transformer networks, characterized in that, Includes the following steps: Step 1: Off-road road data collection: Using a data collection platform equipped with a color camera, LiDAR, IMU accelerometer sensor and GPS navigation system, the system travels on various off-road roads and records sensor road data packets using the ROS system framework; Save point cloud data at preset time intervals, extract the color image with the smallest time difference based on the timestamp, and obtain the color image, the corresponding point cloud and the joint calibration result; Step 2: Data preprocessing: Using the Labelme tool, manually plot points from the outside in to outline the closed contour, and label the passable area of ​​the off-road road, rock obstacles, tree obstacles, and road background; The three-dimensional coordinates and reflection intensity information of the point cloud data are converted into a bin format file. The mapping matrix is ​​used to realize the mutual conversion between the lidar coordinate system and the RGB image coordinate system, and the three-dimensional point cloud data is mapped into a point cloud mapping image of the same size as the RGB image. The point cloud mapping image is completed using the two-dimensional linear interpolation method in OpenCV and the griddata function of the SciPy library to obtain the depth image. Step 3: Construction of Off-Road Scene Road Dataset: Save the labeled images in Pascal voc2007 dataset format, and linearly overlay them with the original images to form the RGB image portion of the dataset; Perform the preprocessing operation in step 2 on the point cloud data corresponding to the RGB images to obtain depth images; combine the RGB images and depth images into a dataset with a total capacity of 1025 images, divide it into a training set of 700 images and a validation set of 325 images, and uniformly adjust the resolution to 1280×720. Step 4: Building and training the off-road road recognition model: Constructing a network model containing a feature fusion module, an encoder, and a decoder; The feature fusion module fuses RGB image features and depth image features through channel stitching and convolution operations; The encoder employs a multi-layer Transformer module, introduces a self-attention mechanism that sets the attenuation ratio R, and replaces the position encoding module with a 3×3 convolution module; The decoder employs a lightweight multilayer perceptron, and achieves semantic segmentation through channel dimension transformation, spatial dimension transformation, feature concatenation and channel compression, and classification prediction. The PyTorch framework was used to train the model based on the dataset. During training, the input image size was adjusted to 1152×640, and data augmentation and standardization were employed to set the training parameters.

2. The off-road road recognition method based on Transformer networks according to claim 1, characterized in that, The mapping matrix expression mentioned in step two is: ， Where M is a (3,4)-dimensional mapping matrix, I is a (3,3)-dimensional camera intrinsic parameter matrix, R is a (3,3)-dimensional rotation matrix from the lidar coordinate system to the camera coordinate system, and T is a (3,1)-dimensional translation matrix from the lidar coordinate system to the camera coordinate system. All parameters are derived from the joint calibration results of the camera and lidar.

3. The off-road road recognition method based on Transformer networks according to claim 1, characterized in that, In step two, the RGB colors of the labels are defined as follows: passable area (0,128,0), stone (255,255,0), tree (0,0,128), and road background (128,0,0).

4. The off-road road recognition method based on Transformer networks according to claim 1, characterized in that, In step four, the attenuation ratio R of the multi-layer Transformer module is set to [64, 16, 4, 1] in sequence.

5. The off-road road recognition method based on Transformer networks according to claim 1, characterized in that, In step four, the parameters for model training are set as follows: the optimizer is AdamW, the number of iterations is 60,000, the initial learning rate is 0.001, the learning rate decay factor is 0.01, the number of learning rate decay iterations is 1,000, the number of model save interval iterations is 2,000, and the training device is an NVIDIA GeForce RTX3090 GPU.

6. The off-road road recognition method based on Transformer networks according to claim 1, characterized in that, The data augmentation methods described in step four include random scaling, random cropping, random hue adjustment, random brightness adjustment, and random saturation adjustment, which are used to enhance the feature recognition of the input image.

7. The off-road road recognition method based on Transformer networks according to claim 1, characterized in that, The fusion process of the feature fusion module in step four is as follows: first, the RGB features extracted from the RGB image are... Depth features extracted from depth images Perform channel splicing operation Then perform a 3×3 convolution operation on the concatenated feature map. , obtain fusion features The fusion expression is: 。 8. The off-road road recognition method based on Transformer networks according to claim 1, characterized in that, The specific implementation process of the lightweight multilayer perceptron described in step four is as follows: (1) Perform channel dimension transformation on multi-scale features to unify the channel dimensions of all features: ， (2) Perform spatial dimension transformation on the multi-scale features after unifying the channel dimensions to unify the spatial dimensions of all features: ， (3) Features with uniform size and dimensions are spliced ​​together, and the spliced ​​features are then subjected to channel compression: ， (4) Perform classification prediction on the compressed features and output pixel-level classification results: 。 9. The off-road road recognition method based on Transformer networks according to claim 1, characterized in that, The specific way to replace the position encoding module with the 3×3 convolution module in step four is as follows: embed the 3×3 convolution module into the feedforward network, utilize the translation invariance of the semantic segmentation task to dynamically express the position information, and avoid the decrease in accuracy caused by the difference in resolution between the test image and the training dataset.

10. The off-road road recognition method based on Transformer networks according to claim 1, characterized in that, The identification method is implemented through an identification system, which includes a data acquisition module, a data preprocessing module, a dataset construction module, and a model training and identification module. The data acquisition module includes a color camera, a lidar, an IMU accelerometer, a GPS navigation system, and a ROS data recording unit, and is used to perform the operation of step one of claim 1; The data preprocessing module is used to perform the operation in step two of claim 1; The dataset construction module is used to perform the operation in step three of claim 1; The model training and recognition module is used to perform the operation in step four of claim 1.

Images