An ai-based terrain recognition system

Abstract

This invention discloses an AI-based terrain recognition system, relating to the field of image processing technology. It generates a material semantic probability field and a spatial geometric depth matrix from the input image and concatenates them into a feature map. High-confusion regions are segmented a second time to update the material semantic probability field. Spatial geometric indicators are extracted and combined with the updated probability field to generate terrain label conclusions. Mathematical transformations and inferences are performed on the input image to obtain a pixel-level uncertainty entropy field, ultimately generating a terrain accessibility cost map. This invention reduces the probability of misclassification in confusing regions such as wet mud and shadows through a dual error correction mechanism of conflict determination and physical verification; it introduces spatial geometric indicators and performs reverse re-determination to improve the stability of terrain label conclusions; and it generates an uncertainty entropy field through active perturbation, enabling the cost map to form a more conservative risk response in high-uncertainty regions, thereby improving the reliability of terrain recognition and accessibility assessment for mobile platforms in complex environments.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an AI-based terrain recognition system. Background Technology

[0002] In recent years, when mobile platforms navigate in off-road, construction site, and unstructured environments, they typically need to generate terrain accessibility cost maps based on the identification results of the terrain ahead to support path planning and motion control. Terrain recognition tasks generally use input images as the primary source of perceptual data, obtain terrain labeling conclusions through image understanding, and further combine geometric factors such as slope and undulation to estimate accessibility risks.

[0003] In existing technologies, image-based terrain recognition methods often employ semantic segmentation networks to directly output pixel-level category results from the input image, or output the probabilities of each category and then determine the pixel-level label by maximizing the probability. These methods are prone to confusing the terrain ontology category set with the environmental noise category set under conditions of varying lighting, shadow coverage, overexposure of highlights, and reflection interference. For example, wet mud and shadow categories, or water and reflection categories, have similar brightness and color distributions, leading to localized incorrect clustering in the material semantic probability field and thus affecting the stability of the terrain label conclusions. Some methods correct misjudgments through rule-based post-processing, but rules are often highly scene-dependent and difficult to maintain consistent performance under different terrain and lighting conditions.

[0004] On the other hand, to improve the reliability of terrain recognition, some solutions incorporate spatial geometric information, such as using binocular cameras or laser ranging to obtain depth, and then combining this with slope or roughness to determine drivability. These solutions are limited in terms of hardware cost, calibration complexity, deployment space, and environmental adaptability. Furthermore, when there are resolution differences or coordinate misalignments between the spatial geometric depth matrix and the semantic output, the semantic and geometric boundaries are prone to shift, leading to inaccurate fusion results. Even when using monocular depth estimation to obtain the spatial geometric depth matrix, without effective data alignment and consistency verification mechanisms, inconsistencies between semantic judgments and geometric morphology may still occur. For example, assigning excessive confidence to labels such as gravel type in areas with steep slopes could lead to an underestimation of drivability risks.

[0005] Furthermore, existing technologies often rely primarily on terrain label conclusions when outputting terrain accessibility cost maps, lacking a quantitative expression of the uncertainty in model inference. When the input image changes due to variations in exposure, white balance, and light source incident angle, the semantic output may fluctuate significantly. If the system does not generate a pixel-level uncertainty entropy field and incorporate it into the cost calculation, the terrain accessibility cost map may still provide overly optimistic cost estimates in high-uncertainty areas, which is detrimental to the safe decision-making of mobile platforms in complex environments. Summary of the Invention

[0006] The technical problem solved by this invention is that existing technologies struggle to stably generate accurate material semantic probability fields in scenarios with similar terrain appearances and environmental noise interference such as shadows, highlights, reflections, and tire tracks, using only input images. They also struggle to effectively align and consistently fuse the material semantic probability field with the spatial geometric depth matrix in the same pixel coordinate system, to perform targeted error correction on highly confused regions, and to conduct physical verification and reverse judgment of terrain label conclusions based on spatial geometric indicators. Furthermore, they struggle to quantify the pixel-level uncertainty of inference results under changes in exposure, white balance, and light source incident angle and incorporate it into the construction of the terrain accessibility cost map. Consequently, the reliability of terrain label conclusions and terrain accessibility cost maps is insufficient in complex environments.

[0007] To address the aforementioned technical problems, the present invention provides the following technical solution: an AI-based terrain recognition system, comprising: The data alignment module processes the input image using a concept cue segmentation operator to obtain a material semantic probability field, processes the input image using a monocular depth operator to obtain a spatial geometric depth matrix, and then aligns and stitches the data to obtain a feature map. The conflict determination module identifies highly confusing regions and performs secondary segmentation to obtain an updated material semantic probability field. The physical verification module extracts spatial geometric indicators based on the spatial geometric depth matrix channels in the feature map and generates initial terrain label conclusions by combining them with the updated material semantic probability field. The response diagnostic module performs mathematical transformations and pixel-level terrain label inference on the input image to obtain a pixel-level uncertainty entropy field. Based on the pixel-level uncertainty entropy field, spatial geometric indicators, and initial terrain label conclusions, it generates a terrain accessibility cost map.

[0008] As a preferred embodiment of the AI-based terrain recognition system described in this invention, the data alignment module includes a semantic unit, a geometric unit, and a tensor alignment unit. The semantic unit is used to process the input image through a concept cue segmentation operator to obtain a material semantic probability field; The material semantic probability field includes a set of terrain ontology categories and a set of environmental noise categories. The set of terrain ontology categories includes grassland, bare soil, gravel, wet mud, rock surface, water body, and sand. The set of environmental noise categories includes shadow, highlight, reflection, and ruts. The geometric unit is used to process the input image using a monocular depth operator to obtain a spatial geometric depth matrix; The tensor alignment unit is used to read the homogeneous coordinates of the material semantic probability field, and to perform a resampling homogenization transformation on the spatial geometric depth matrix based on the homogeneous coordinates. The spatial geometric depth matrix after the resampling homogenization transformation is then concatenated with the material semantic probability field in the pixel channel dimension to obtain the feature map.

[0009] As a preferred embodiment of the AI-based terrain recognition system described in this invention, the conflict determination module includes an overlapping scanning unit, a block segmentation unit, and a secondary segmentation unit. The overlapping scanning unit is used to calculate the cross-union ratio (CUI) between different types of masks in the material semantic probability field channel of the feature map. When the CUI is greater than the first threshold, a conflict marker is attached and the region is identified as a high confusion pair. The block segmentation unit is used to extract the boundaries of highly confused regions and segment out local high-dimensional image blocks. The secondary segmentation unit is used to generate hard negative sample prompts based on the contextual features of local high-dimensional image patches, and inputs the local high-dimensional image patches and hard negative sample prompts into the concept prompt segmentation operator to obtain the updated material semantic probability field.

[0010] As a preferred embodiment of the AI-based terrain recognition system described in this invention, the processing logic for calculating the intersection-union ratio (IU / U) and the mounting conflict markers includes: Separate wet mud category mask and shadow category mask from the material semantic probability field; Extract the first grid coordinates corresponding to non-zero pixels in the wet mud category mask and the second grid coordinates corresponding to non-zero pixels in the shadow category mask, and take the intersection of the first grid coordinates and the second grid coordinates to obtain the overlapping coordinate region; The total number of pixels in the overlapping coordinate region is taken as the total number of intersection pixels. The total number of pixels in the union of the wet mud category mask and the shadow category mask is also counted. The total number of intersection pixels is divided by the total number of union pixels to obtain the cross-union ratio. Determine whether the intersection-union ratio (IU) is greater than the first threshold. When the crossover ratio is greater than the first threshold, the overlapping coordinate region is assigned a binary conflict status bit of 1, thus completing the mounting of the conflict marker.

[0011] As a preferred embodiment of the AI-based terrain recognition system described in this invention, the physical verification module includes an index extraction unit, a boundary matching unit, and a frequency domain calculation unit. The index extraction unit is used to perform local window gradient difference calculation on the spatial geometric depth matrix channel in the feature map to obtain the spatial geometric index of each pixel. The spatial geometric index includes local tilt angle, surface curvature and normal vector. The boundary matching unit is used to determine the maximum value of the probability of each category in the updated material semantic probability field, obtain the initial terrain label conclusion corresponding to each pixel, perform threshold matching judgment between the local tilt angle and the initial terrain label conclusion to obtain the physical mismatch signal, and perform confidence reduction operation on the updated material semantic probability field. The frequency domain calculation unit is used to divide the region corresponding to the physical mismatch signal into a two-dimensional sliding analysis window, extract the local height residual of each pixel point in the two-dimensional sliding analysis window relative to the spatial geometric depth matrix, and obtain the height field residual. The height field residuals are transformed to the frequency domain using a two-dimensional fast Fourier transform, and the spatial geometric power spectrum feature vector is calculated. Based on the spatial geometric power spectrum feature vector, the initial terrain label conclusion is reversed.

[0012] In a preferred embodiment of the AI-based terrain recognition system described in this invention, the threshold matching determination processing logic is as follows: Read the probability of the gravel category and the local tilt angle in the updated material semantic probability field. ; When the probability of the gravel category is greater than the second threshold and the local dip angle is... When a physical mismatch signal is obtained, the probability of the gravel category of that pixel is reduced to a preset baseline value.

[0013] As a preferred embodiment of the AI-based terrain recognition system described in this invention, the reverse judgment processing logic is as follows: The height field residual of each pixel within the two-dimensional sliding analysis window is extracted from the spatial geometric depth matrix. A two-dimensional fast Fourier transform is performed on the height field residual to obtain the spatial geometric power spectrum feature vector. Extract the energy distribution value and DC component of the spatial geometric power spectrum feature vector in the low-frequency and high-frequency ranges. When the energy distribution value in the low-frequency range is greater than the fourth threshold and the DC component is greater than the third threshold, determine that the surface corresponding to the two-dimensional sliding analysis window is a smooth surface. Then, reverse the initial terrain label conclusion corresponding to the gravel category to the initial terrain label conclusion corresponding to the rock surface category.

[0014] As a preferred embodiment of the AI-based terrain recognition system described in this invention, the response diagnosis module includes an active perturbation unit, an entropy field calculation unit, and a decision cascade unit. The active perturbation unit is used to perform mathematical transformations on the input image to obtain a set of virtual terrain maps; The set of virtual terrain maps includes underexposed virtual terrain maps, overexposed virtual terrain maps, warm color temperature shifted virtual terrain maps, cool color temperature shifted virtual terrain maps, and light source incident angle rotation virtual terrain maps. The entropy field calculation unit is used to perform pixel-level terrain label inference on the virtual terrain map set to obtain a pixel-level uncertainty entropy field. The decision cascade unit is used to perform cascaded fusion calculations on pixel-level uncertainty entropy field, spatial geometric indicators and terrain label conclusions to obtain a terrain accessibility cost map.

[0015] As a preferred embodiment of the AI-based terrain recognition system described in this invention, the processing logic for mathematical transformation of the input image includes: The input image is subjected to gamma nonlinear stretching using the first gamma coefficient and the second gamma coefficient respectively to obtain underexposed virtual terrain map and overexposed virtual terrain map; The input image is transformed by applying warm color temperature parameters and cool color temperature parameters respectively to obtain a warm color temperature offset virtual terrain map and a cool color temperature offset virtual terrain map. Using the relighting operator, the brightness and darkness features in the input image are reprojected in three dimensions according to the preset incident angle of the simulated light source, resulting in a virtual terrain map rotated by the incident angle of the light source.

[0016] As a preferred embodiment of the AI-based terrain recognition system described in this invention, the processing logic for the terrain accessibility cost map is as follows: The terrain label results are mapped to a pixel-level base cost matrix, and a geometric cost matrix is ​​constructed based on spatial geometric indicators. The base cost matrix and the geometric cost matrix are multiplied point-to-point to obtain the initial cost matrix. The entropy value corresponding to each pixel in the pixel-level uncertainty entropy field is extracted as a weight coefficient. The weight coefficient is multiplied by the matrix element of the corresponding coordinate in the initial cost matrix to obtain the terrain accessibility cost map.

[0017] The beneficial effects of this invention are as follows: This invention innovatively constructs a dual error correction mechanism based on conflict determination and physical verification, which can accurately locate and correct highly confused regions. Without increasing the overhead of repeated inference across the entire image, it reduces the probability of misclassification in complex scenarios such as wet mud and shadow categories, water and reflection categories, and improves the boundary accuracy and stability of the material semantic probability field. By introducing a spatial geometric depth matrix channel and extracting spatial geometric indicators such as local tilt angle, surface curvature, and normal vectors, the terrain label conclusions simultaneously satisfy the consistency of appearance semantics and spatial geometric morphology, thereby reducing texture loss and reflection. Systematic misjudgments under light and shadow interference conditions are addressed, improving the stability and interpretability of terrain labeling conclusions in complex terrain environments. Frequency domain calculations are performed on physically mismatched areas, and reverse judgments are executed in conjunction with spatial geometric power spectrum feature vectors, enhancing the terrain category identification capability under complex terrain morphology and improving the reliability of terrain recognition results in complex environments. By actively perturbing to generate a pixel-level uncertainty entropy field and participating in terrain accessibility cost map fusion, the cost output forms a more conservative risk response for high uncertainty areas, thereby improving the reliability and security of mobile platform terrain accessibility assessment in complex environments. Attached Figure Description

[0018] Figure 1 A schematic diagram of the basic process of an AI-based terrain recognition system provided in one embodiment of the present invention; Figure 2 Flowchart for the data alignment module; Figure 3 Here is the flowchart for the conflict determination module; Figure 4 Here is the flowchart for the physical verification module; Figure 5 Flowchart for the response diagnostic module. Detailed Implementation

[0019] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0020] Example, refer to Figure 1 This provides an AI-based terrain recognition system, including: The data alignment module processes the input image using a concept cue segmentation operator to obtain a material semantic probability field, processes the input image using a monocular depth operator to obtain a spatial geometric depth matrix, and then aligns and stitches the data to obtain a feature map. The conflict determination module identifies highly confusing regions and performs secondary segmentation to obtain an updated material semantic probability field. The physical verification module extracts spatial geometric indicators based on the spatial geometric depth matrix channels in the feature map and generates initial terrain label conclusions by combining them with the updated material semantic probability field. The response diagnostic module performs mathematical transformations and pixel-level terrain label inference on the input image to obtain a pixel-level uncertainty entropy field. Based on the pixel-level uncertainty entropy field, spatial geometric indicators, and initial terrain label conclusions, it generates a terrain accessibility cost map.

[0021] In practice, the data alignment module includes semantic units, geometric units, and tensor alignment units; Semantic units are used to process the input image through a concept cue segmentation operator to obtain the material semantic probability field; The material semantic probability field includes a set of terrain ontology categories and a set of environmental noise categories. The set of terrain ontology categories includes grassland, bare soil, gravel, wet mud, rock surface, water body, and sand. The set of environmental noise categories includes shadow, highlight, reflection, and ruts. Geometric units are used to process the input image using monocular depth operators to obtain a spatial geometric depth matrix; Tensor alignment units are used to read the homogeneous coordinates of the material semantic probability field and perform a resampling homogenization transformation on the spatial geometry depth matrix based on the homogeneous coordinates. The spatial geometry depth matrix after the resampling homogenization transformation is then concatenated with the material semantic probability field in the pixel channel dimension to obtain the feature map.

[0022] Specifically, the input image is acquired in real time by a monocular camera mounted on a mobile platform. The camera's imaging optical axis is oriented towards the direction of travel and covers the surface area to be identified. The camera uses global shutter imaging and outputs a color input image. The resolution of the input image is set between 640×480 and 1920×1080. Too low a resolution will smooth out the texture differences between gravel and bare soil categories, thus reducing the distinguishing ability of the material semantic probability field. Too high a resolution will significantly increase the inference latency of the concept cue segmentation operator and the monocular depth operator and reduce the system's real-time performance. Before entering the data alignment module, the input image is normalized, linearly scaling the pixel values ​​to the range of 0 to 1. Radial distortion is corrected based on camera intrinsic parameters to improve the stability of subsequent homogeneous coordinate alignment.

[0023] The concept cue segmentation operator is a cue-driven semantic segmentation model, consisting of an image encoder, a text encoder, a cue fusion decoder, and a mask output head. The image encoder employs a visual Transformer structure with 12 layers, each with a hidden state dimension of 768, and 12 multi-head self-attention heads. The input image is divided into 16×16 image blocks, mapped to sequence features, and then encoded by 12 Transformer layers to obtain the image feature sequence. The text encoder also employs a Transformer structure with 12 layers, each with a hidden state dimension of 512, and 8 multi-head self-attention heads. The text input is a sequence of concept cue words corresponding to a set of terrain ontology categories and a set of environmental noise categories. The length of the concept cue word sequence is set to 10 units. A shorter sequence will result in insufficient description of environmental noise categories such as shadows and reflections, while a longer sequence will introduce redundant semantics, thus reducing the separability between categories. The cue fusion decoder employs a cross-attention structure with four layers, each with a hidden state dimension of 512 and eight multi-heads. It uses image feature sequences as keys and values, and text feature sequences as queries to guide the matching of image features and output category-related spatial features. The mask output head consists of three convolutional upsampling modules, each with a 3×3 kernel size and 256, 128, and 64 channels respectively. A 1×1 convolution layer maps the channels to the number of categories, ultimately obtaining the category logits for each pixel in the categories of grass, bare soil, gravel, wet mud, rock surface, water, sand, shadow, highlight, reflection, and ruts. The material semantic probability field is obtained by softmax normalization, and the spatial size of the output tensor is consistent with the input image, with 11 channels, each representing the pixel-level probability distribution of the corresponding category. The training of the concept cue segmentation operator adopts a two-stage approach. The first stage involves cross-modal alignment pre-training of the image encoder and text encoder. The training data consists of surface scene images and corresponding concept cues. The optimization objective uses contrastive learning loss to improve the correlation between concept cues and image region features. The second stage involves pixel-level segmentation fine-tuning. The training data consists of surface images with pixel-level annotations, and the annotation categories are consistent with the 11 categories of the material semantic probability field. The loss function in the fine-tuning stage consists of a weighted average of pixel-level cross-entropy loss and Dice loss. The cross-entropy loss weight coefficient is set to 0.7, and the Dice loss weight coefficient is set to 0.3. The cross-entropy loss weight coefficient is used to emphasize multi-class competition and improve overall convergence stability, while the Dice loss weight coefficient is used to improve the overlap consistency of small target areas and improve the connectivity of fine line structures. The weighted average of the two can balance overall accuracy and detail integrity. The optimizer used is AdamW, with a learning rate of 1×10⁻⁶. -4 The weight decay is set to 1×10. -2The number of training rounds was set to 50 to strike a balance between convergence speed and the risk of overfitting.

[0024] The monocular depth operator is a monocular depth estimation model with an encoder-decoder structure. The encoder is a residual convolutional network with 34 layers, containing four downsampling stages. The number of output channels in each stage is 64, 128, 256, and 512 respectively, with a downsampling factor of 32, thus obtaining deep features with global context. The decoder contains four upsampling stages, each consisting of a deconvolutional layer and two 3×3 convolutional layers. It is connected to the encoder's features at the same scale via skip connections to recover edge details. The decoder output is processed by a 1×1 convolution to obtain single-channel depth logits. The depth values ​​are mapped to a 0-1 range using a sigmoid function and then linearly scaled to the depth range of the spatial geometric depth matrix. The upper limit of the depth range is set to 40m. Terrain accessibility assessment of mobile platforms typically focuses on the near-to-mid distance surface. An excessively large upper limit of depth will reduce near-field depth resolution, while an excessively small upper limit of depth will cause depth saturation at distant locations and affect slope judgment. The monocular depth operator is trained using a supervised training method. The supervision signals come from samples with ground-proof depth values, which are generated offline by a high-precision ranging device with the same viewpoint and registered with the input image to form training pairs. The training loss consists of a weighted average of the logarithmic depth L1 loss and the gradient smoothing loss. The L1 loss weight is set to 0.8 to ensure absolute depth accuracy, while the gradient smoothing loss weight is set to 0.2 to suppress noise and maintain structural continuity, thus making subsequent gradient difference calculations based on the spatial geometric depth matrix more stable. The optimizer used is AdamW, with a learning rate of 5×10⁻⁶. -5 The batch size is set to 16 to adapt to different computing power platforms.

[0025] Each pixel position in the material semantic probability field is represented by horizontal and vertical coordinates in a two-dimensional pixel coordinate system. The tensor alignment unit expands this two-dimensional coordinate system into a three-dimensional homogeneous coordinate vector, with the third dimension fixed at 1. Since the concept hint segmentation operator and the monocular depth operator may use different input resolutions and internal downsampling rates, the spatial dimensions of the material semantic probability field and the spatial geometric depth matrix may be inconsistent. The tensor alignment unit constructs a 3×3 homogeneous transformation matrix from the material semantic probability field coordinate system to the spatial geometric depth matrix coordinate system. This homogeneous transformation matrix adopts an affine form that includes horizontal and vertical scale factors. The horizontal scale factor is the ratio of the depth matrix width minus 1 to the probability field width minus 1, and the vertical scale factor is the ratio of the depth matrix height minus 1 to the probability field height minus 1, thereby ensuring consistent corner mapping and reducing overall geometric offset. The tensor alignment unit multiplies each homogeneous coordinate vector by the homogeneous transformation matrix to obtain continuous sampling coordinates in the depth matrix. Then, it uses the first two dimensions of these continuous sampling coordinates as resampling indices and employs bilinear interpolation to extract values ​​from the spatial geometric depth matrix, generating a resampling result with the same spatial dimensions as the material semantic probability field. This completes the resampling homogenization transformation. Compared to nearest-neighbor interpolation, bilinear interpolation reduces step artifacts and makes depth variations at boundaries more continuous, thereby improving the stability of subsequent spatial geometric index extraction. The material semantic probability field is a 3D tensor of height × width × 11, and the spatial geometric depth matrix after resampling homogenization is a 3D tensor of height × width × 1, both having the same height and width. The tensor alignment unit appends the spatial geometric depth matrix to the material semantic probability field along the channel dimension, forming a feature map of height × width × 12, where the first 11 channels are probability channels for each category, and the last channel is the spatial geometric depth matrix channel. This channel stitching method ensures that the semantic and geometric information of each pixel position are aligned at the same pixel coordinates, which facilitates subsequent modules to jointly process the material semantic probability field and spatial geometric depth matrix at the same coordinate position.

[0026] In practice, the conflict determination module includes an overlapping scanning unit, a block segmentation unit, and a secondary segmentation unit. The overlapping scanning unit is used to calculate the cross-union ratio (CUP) between different types of masks in the material semantic probability field channel of the feature map. When the CUP is greater than the first threshold, a conflict marker is attached and the region is identified as a high confusion pair. The block segmentation unit is used to extract the boundaries of highly confused regions and segment them into local high-dimensional image blocks. The secondary segmentation unit is used to generate hard negative sample prompts based on the contextual features of local high-dimensional image patches, and inputs the local high-dimensional image patches and hard negative sample prompts into the concept prompt segmentation operator to obtain the updated material semantic probability field.

[0027] Specifically, different category masks are generated by the overlapping scanning unit based on the material semantic probability field. The material semantic probability field includes 11 probability channels: grass, bare soil, gravel, wet mud, rock surface, water, sand, shadow, highlight, reflection, and ruts. The overlapping scanning unit thresholds each probability channel to obtain a binary mask, setting pixels with a probability greater than or equal to 0.5 in that category channel to 1 and setting the remaining pixels to 0, thus forming the corresponding masks for different categories. This thresholding achieves a balance between boundary sensitivity and low-confidence noise suppression, ensuring that the cross-union ratio (CUI) value primarily reflects the actual category competition region rather than the probability tail overlap.

[0028] The overlapping scanning unit performs pairwise combinations of masks of different types and calculates the cross-union ratio (CUI). The CUI is the ratio of the total number of intersection pixels to the total number of union pixels. The total number of intersection pixels is the number of pixels in both types of masks that are simultaneously 1 at the same pixel coordinate, and the total number of union pixels is the number of pixels in both types of masks that are at least 1 at the same pixel coordinate. A first threshold is used to determine the degree of significant overlap. In this embodiment, the first threshold is specifically set to 0.25. When the value is less than the first threshold, slight overlap caused by image noise and low-confidence pixels at the boundaries will frequently trigger conflict markers, causing the secondary segmentation unit to be over-called and increasing inference latency. When the value is greater than the first threshold, truly high-confusion pairs, especially narrow boundary bands, are easily missed, preventing secondary segmentation from correcting errors. This allows for the coverage of major high-confusion pairs while reducing false triggers, thus achieving higher error correction benefits in confused regions with lower computational overhead. When the CUI is greater than the first threshold, the overlapping scanning unit attaches conflict markers to the overlapping pixel coordinate areas of the two types of masks and identifies them as high-confusion pairs.

[0029] The boundary extraction process involves performing 8-neighborhood connected component analysis on the binary map formed by conflict markers to obtain the connected regions of each high-confusion pair. Then, the outer boundary pixel set is extracted for each connected region, and the minimum bounding rectangle of this outer boundary pixel set is calculated as the segmentation box. To introduce neighborhood information to support subsequent hard negative sample prompt word generation, the block segmentation unit expands the segmentation box. In this embodiment, the expansion pixel width is set to 32. If the expansion pixel width is less than 32, the neighborhood texture and lighting information are insufficient, which is not conducive to distinguishing between confusion cases such as shadow and wet mud categories. If the expansion pixel width is greater than 32, too much irrelevant background will be introduced, weakening the category contrast of local regions and reducing the targeting of secondary segmentation. In this embodiment, the local high-dimensional image block is a local input image block cropped from the input image according to the expanded segmentation box. It maintains the original pixel resolution and contains the high-confusion pair regions and their neighborhood content.

[0030] The contextual features consist of a set of features within a local high-dimensional image patch and its neighborhood. These features include color statistical features, texture features, and edge features. Color statistical features are obtained by calculating the mean and standard deviation of the luminance components of the local high-dimensional image patch, and further statistically analyzing the proportion of bright pixels to characterize local over-brightness interference in the highlight and reflective categories. Texture features are obtained by calculating the local variance and the mean gradient magnitude of the local grayscale image, used to characterize the high-frequency texture difference between the gravel category and the relatively low-frequency texture difference between the bare soil and wet mud categories. Edge features are obtained by calculating the proportion of strong edge pixels using Sobel gradients and calculating the concentration of the gradient direction histogram to characterize the directional edge structure of the rut category. The secondary segmentation unit selects the category name most likely to cause interference from the environmental noise category set as hard negative sample prompts based on the above contextual features, and supplements the competing category names by combining the material semantic probability field with the category competition relationship within the high-confusion region. In the high-confusion region, the average probability of each of the 11 category channels is calculated. The top two category names with the highest average probability are selected as competing pairs, and the second-ranked category name is added to the hard negative sample cue words to exert stronger suppression on the main competing categories in the secondary segmentation. In this embodiment, the number of hard negative sample cue words is set to 2, which can cover the main environmental noise interference while suppressing the strongest competing category, avoiding the distraction of the cue attention of the concept cue segmentation operator due to the large number of cue words.

[0031] The generation process of hard negative sample prompts involves the secondary segmentation unit first performing candidate matching on the environmental noise category set based on contextual features, and then determining the category prompts to be suppressed based on the category competition relationship within the high-confusion pair region in the local block of the material semantic probability field. When the color statistical features show a significant decrease in overall brightness and insignificant edge features, the secondary segmentation unit uses the category name corresponding to the shadow category as the hard negative sample prompt; when the color statistical features show a significant increase in the proportion of bright pixels and low local variance of texture features, the secondary segmentation unit uses the category names corresponding to the highlight and reflection categories as hard negative sample prompts; when the concentration of the principal direction of edge features increases significantly and is spatially continuous in a band-like manner, the secondary segmentation unit uses the category name corresponding to the rut category as the hard negative sample prompt. Simultaneously, the secondary segmentation unit reads the top two category probabilities within the high-confusion pair region as conflict category pairs and adds the non-target conflict category names to the hard negative sample prompts to enhance the distinction between categories during secondary segmentation. In this embodiment, the number of hard negative sample prompts is set to 2. If there are fewer than 2, the coverage of complex interference will be insufficient. If there are more than 2, the attention of the text encoder will be distracted, resulting in a decrease in the expression intensity of the target category prompts.

[0032] Compared to traditional terrain recognition methods that directly output category results after one-time semantic segmentation, this scheme actively locates highly confusing pairs where different category masks significantly overlap using overlapping scanning units and intersection-union ratio (IU) values. It triggers block segmentation and secondary segmentation only for these highly confusing pairs, thus avoiding the computational overhead of repeatedly reasoning across the entire input image. Compared to traditional methods that use fixed post-processing rules for global correction of misjudgments, this scheme generates hard negative sample prompts based on the contextual features of local high-dimensional image blocks in the secondary segmentation unit, and then drives the concept prompt segmentation operator again. This allows the model to explicitly address and suppress the interference of environmental noise category sets on terrain ontology category sets within a local scope, thereby reducing the misjudgment probability of highly confusing pairs such as wet mud and shadow categories, as well as water and reflective categories. Furthermore, it improves the boundary accuracy and stability of the material semantic probability field under complex lighting and surface disturbance conditions.

[0033] In specific implementation, the processing logic for calculating the intersection-union ratio and the mount conflict flag includes: Separate wet mud category mask and shadow category mask from the material semantic probability field; Extract the first grid coordinates corresponding to non-zero pixels in the wet mud category mask and the second grid coordinates corresponding to non-zero pixels in the shadow category mask, and take the intersection of the first grid coordinates and the second grid coordinates to obtain the overlapping coordinate region; The total number of pixels in the overlapping coordinate region is taken as the total number of intersection pixels. The total number of pixels in the union of the wet mud category mask and the shadow category mask is also counted. The total number of intersection pixels is divided by the total number of union pixels to obtain the cross-union ratio. Determine whether the intersection-union ratio (IU) is greater than the first threshold. When the crossover ratio is greater than the first threshold, the overlapping coordinate region is assigned a binary conflict status bit of 1, thus completing the mounting of the conflict marker.

[0034] Specifically, the method for separating the wet mud category mask and the shadow category mask from the material semantic probability field is to perform thresholding processing on the wet mud category channel and the shadow category channel in the material semantic probability field to generate binary masks. Pixels with a probability greater than or equal to 0.5 are set to 1, and the remaining pixels are set to 0, thus obtaining the wet mud category mask and the shadow category mask. Here, non-zero pixels refer to pixels with a value of 1 in the mask, thus ensuring that the subsequent extraction of the first grid coordinates and the second grid coordinates is unique and corresponds one-to-one with the pixel coordinates in the material semantic probability field. In this embodiment, the first grid coordinates and the second grid coordinates adopt the same pixel coordinate system as the input image. The grid coordinates consist of a horizontal coordinate u and a vertical coordinate v, where u ranges from 0 to width minus 1, and v ranges from 0 to height minus 1. The extraction method is to traverse the mask and write all pixel coordinates with a value of 1 into the corresponding coordinate set. The method of obtaining the overlapping coordinate region by taking the intersection of the first grid coordinate and the second grid coordinate is to perform a pixel-by-pixel AND operation on the wet mud category mask and the shadow category mask under the same pixel coordinate to obtain the overlapping mask. The set of pixel coordinates with a value of 1 in the overlapping mask is the overlapping coordinate region. The total number of intersection pixels is the number of pixels with a value of 1 in the overlapping mask. The total number of union pixels is obtained by performing a pixel-by-pixel OR operation on the wet mud category mask and the shadow category mask to obtain the union mask, and then counting the number of pixels with a value of 1 in the union mask. The intersection-union ratio is the total number of intersection pixels divided by the total number of union pixels. This method can maintain sufficient sensitivity to the true overlap of wet mud category and shadow category on the boundary band, while suppressing the occasional overlap triggering caused by probabilistic tail noise, thereby achieving a balance between error correction benefits and secondary segmentation triggering frequency. When the crossover ratio is greater than the first threshold, the overlapping coordinate region is assigned a binary conflict state bit of 1. The binary conflict state bit is stored in a binary matrix of the same size as the material semantic probability field and shares pixel coordinates with the feature map, thereby completing the mounting of the conflict marker and providing direct input for the boundary extraction of the block segmentation unit.

[0035] In practice, the physical verification module includes an index extraction unit, a boundary matching unit, and a frequency domain calculation unit. The index extraction unit is used to perform local window gradient difference calculation on the spatial geometric depth matrix channel in the feature map to obtain the spatial geometric index of each pixel. The spatial geometric index includes local tilt angle, surface curvature and normal vector. The boundary matching unit is used to determine the maximum value of the probability of each category in the updated material semantic probability field, obtain the initial terrain label conclusion corresponding to each pixel, perform threshold matching judgment between the local tilt angle and the initial terrain label conclusion, obtain the physical mismatch signal, and perform confidence reduction operation on the updated material semantic probability field. The frequency domain calculation unit is used to divide the region corresponding to the physical mismatch signal into a two-dimensional sliding analysis window, extract the local height residual of each pixel point in the two-dimensional sliding analysis window relative to the spatial geometric depth matrix, and obtain the height field residual. The height field residuals are transformed to the frequency domain using a two-dimensional fast Fourier transform, and the spatial geometric power spectrum feature vector is calculated. Based on the spatial geometric power spectrum feature vector, the initial terrain label conclusion is reversed.

[0036] It should be noted that the index extraction unit performs local window gradient difference calculation on the spatial geometric depth matrix channel to obtain the spatial geometric index. In this embodiment, let the spatial geometric depth matrix at pixel coordinates be... The depth value at that location is The local window size is set to 5×5. A window that is too small will amplify depth noise, causing local tilt jitter; a window that is too large will cross the true boundary, resulting in excessive smoothing of surface curvature. For any pixel coordinate... The depth of the pixel in the spatial geometric depth matrix is ​​taken as The lateral and longitudinal gradients are calculated using central difference, with the lateral gradient denoted as . The longitudinal gradient is denoted as The calculation formula is as follows: ; ; in, This is the row index of the input image, and its value ranges from 0 to the height of the input image minus 1. The column index of the input image and its value ranges from 0 to the width of the input image minus 1. The spatial geometric depth matrix at pixel coordinates is Depth value at that location, The spatial geometric depth matrix at pixel coordinates is Depth value at that location, The spatial geometric depth matrix at pixel coordinates is Depth value at that location, The spatial geometric depth matrix at pixel coordinates is The depth value at that location.

[0037] In the above formula, 2 represents the step size coefficient of the central difference, used to convert the depth change between two adjacent pixel positions into an approximate local depth gradient value at the current pixel. Both the lateral and longitudinal gradients are calculated in the pixel coordinate system based on the spatial geometric depth matrix. Their values ​​characterize the relative change trend of the spatial geometric depth matrix within the local neighborhood, rather than the absolute slope value in the actual geographic coordinate system. The local tilt angle is calculated jointly by the lateral and longitudinal gradients, reflecting the intensity and degree of undulation of the surface morphology within the local neighborhood, thus serving as a spatial geometric indicator in subsequent physical verification processing. Since the spatial geometric depth matrix and the material semantic probability field have been pixel-level aligned, the local tilt angle, surface curvature, and normal vector are all calculated in a unified pixel coordinate system and participate in the consistency verification of the terrain label conclusions. When an index out-of-bounds error occurs at the boundary pixel, this embodiment uses a mirroring method to supplement the required depth value to maintain gradient continuity and avoid abnormal gradient values ​​in the boundary area due to differential calculation, which could affect the stability of the spatial geometric indicators. The local tilt angle is denoted as... The calculation formula is as follows: ; in, For arctangent operation, For square root operations, Used to characterize the relative undulation intensity and geometric change of the spatial geometric depth matrix in the local neighborhood; normal vector The calculation formula is: ; in, Used to characterize the orientation of the Earth's surface; In this embodiment, the surface curvature is approximated using the second-order difference Laplace approximation, and the calculation formula is as follows: ; ; ; in, It is a second-order difference in the transverse direction. It is a vertical second-order difference. In pixel coordinates The surface curvature at that point. The larger the absolute value, the more obvious the local deformation; For each pixel, the category with the highest probability among 11 category channels is selected as the initial terrain label conclusion. Then, a threshold matching judgment is performed between the local tilt angle and the initial terrain label conclusion to obtain a physical mismatch signal. When the local tilt angle of a pixel is greater than the threshold corresponding to its initial terrain label conclusion, the physical mismatch signal of that pixel is set to 1; otherwise, it is set to 0. In this embodiment, the local tilt angle thresholds corresponding to each category are: 25° for grassland, 30° for bare soil, 35° for gravel, 20° for wet mud, 45° for rock surface, 5° for water body, 25° for sand, 30° for shadow, 30° for highlight, 30° for reflective, and 15° for ruts. These thresholds are determined based on the surface morphology characteristics of each category and the sample statistical results. When the local tilt angle of the category corresponding to the terrain label conclusion exceeds its corresponding local tilt angle threshold, it is considered that there is a physical inconsistency between that category and the spatial geometry, thereby triggering the physical mismatch signal.

[0038] The boundary matching unit performs a confidence reduction operation on pixels with a physical mismatch signal of 1. The processing logic is as follows: multiply the probability of the channel corresponding to the initial terrain label conclusion of the pixel by the confidence attenuation coefficient and renormalize the 11 category channels. In this embodiment, the confidence attenuation coefficient is 0.4. The smaller the confidence attenuation coefficient, the stronger the suppression. It can maintain the continuity of the probability field while suppressing physically inconsistent labels and avoid large-area probability abrupt changes.

[0039] For pixels with a physical mismatch signal of 1, 8-neighborhood connected component analysis is performed to obtain multiple physical mismatch regions. Then, the bounding rectangle of each physical mismatch region is calculated, and a two-dimensional sliding analysis window is created within the coverage area of ​​this bounding rectangle using a sliding method with a fixed window size. In this embodiment, the size of the two-dimensional sliding analysis window is 32×32. A window size that is too small will result in insufficient frequency domain resolution, leading to unstable power spectral characteristics; a window size that is too large will easily include various surface features, reducing local discriminative power. The sliding step size is 16 in this embodiment. A smaller step size improves the coverage integrity of the physical mismatch regions but increases computational load; a larger step size reduces computational load but may miss narrow-band physical mismatch regions. The height field residual is calculated by averaging the depth of all pixels within the window of the spatial geometric depth matrix within each two-dimensional sliding analysis window to obtain a local mean. The height field residual is then obtained by subtracting this local mean from the depth value of each pixel within the window, thereby removing the overall height bias and highlighting local undulations, facilitating subsequent frequency domain analysis.

[0040] The frequency domain calculation unit performs a two-dimensional discrete Fourier transform on the height field residual matrix within each two-dimensional sliding analysis window and obtains the frequency domain matrix using a fast implementation. Each element of the frequency domain matrix corresponds to a horizontal frequency index and a vertical frequency index. The power spectrum is calculated from the frequency domain matrix by taking the amplitude of each element and squaring it. To form a fixed-dimensional spatial geometric power spectrum feature vector, in this embodiment, the frequency plane corresponding to the power spectrum matrix is ​​divided into 8 frequency bands according to radial frequency bands, and the power spectrum values ​​in each frequency band are summed to obtain the energy of the 8 frequency bands. At the same time, the DC component at the zero-frequency position of the power spectrum matrix is ​​extracted as the DC energy. Then, the energy of the 8 frequency bands is added to the DC energy to obtain the total energy, and each frequency band energy is divided by the total energy to obtain 8 normalized energy components. Then, the DC energy is divided by the total energy to obtain the normalized DC component. These 9 normalized components are arranged in order to form the spatial geometric power spectrum feature vector.

[0041] Compared to traditional terrain recognition methods that rely solely on the maximum value of the material semantic probability field for judgment, this scheme introduces a spatial geometric depth matrix channel into the physical verification module and extracts local dip angle, surface curvature, and normal vector as spatial geometric indicators. It obtains physical mismatch signals and reduces confidence through threshold matching, and then uses the spatial geometric power spectrum feature vector obtained by two-dimensional fast Fourier transform to perform reverse judgment. This ensures that the terrain label conclusions simultaneously satisfy the consistency of appearance semantics and geometric morphology, thereby reducing appearance misjudgments caused by shadow interference, reflection interference, and texture loss, and improving the stability and interpretability of terrain label conclusions under complex surface conditions.

[0042] In practice, the threshold matching determination logic is as follows: Read the probability of the gravel category and the local tilt angle in the updated material semantic probability field. ; When the probability of the gravel category is greater than the second threshold and the local dip angle is... When a physical mismatch signal is obtained, the probability of the gravel category of that pixel is reduced to a preset baseline value.

[0043] Specifically, the second threshold is set to 0.6 in this embodiment. This physically constrains only the gravel category decision given by the concept cue segmentation operator with high confidence, avoiding excessive perturbation of the probability field caused by repeatedly suppressing the natural fluctuations of low-confidence pixels. The preset baseline value is set to 0.15 in this embodiment, suppressing the gravel category probability to a small but non-zero level. This makes it easier for the pixel to be replaced by other categories that better conform to spatial geometric indicators during subsequent maximum value decisions or reverse re-determinations, while maintaining probability continuity and avoiding abrupt probability changes in the neighborhood caused by directly setting it to zero.

[0044] The processing logic for reducing the probability of the gravel category to a preset baseline value is as follows: when the probability of the gravel category is greater than a second threshold and the local tilt angle is... When the angle is greater than 35°, the physical mismatch signal is set to 1, and the probability of the gravel category channel of that pixel is replaced with a preset baseline value. Then, the probabilities of the 11 category channels of that pixel are renormalized so that the sum of the probabilities of the 11 category channels after normalization is 1. This ensures that the updated material semantic probability field is still a valid probability field and can continue to be used by the frequency domain calculation unit. In this embodiment, the local dip angle threshold corresponding to the gravel category is set to 35°. The reason is that gravel surfaces usually have certain accumulation stability angle characteristics. When the local dip angle continues to increase, the degree of deviation between its spatial geometry and the statistical characteristics of ordinary gravel surfaces gradually increases. Through statistical analysis of spatial geometric indicators of gravel areas in training and validation samples, it was found that when the local dip angle exceeds 35°, the confusion probability between gravel categories and terrains such as rock surface categories and steep slope bare rock categories increases significantly. Therefore, 35° is used as the physical verification trigger threshold to identify possible inconsistencies between the material semantic probability field and spatial geometry.

[0045] In practice, the logic for reverse judgment is as follows: The height field residual of each pixel within the two-dimensional sliding analysis window is extracted from the spatial geometric depth matrix. A two-dimensional fast Fourier transform is performed on the height field residual to obtain the spatial geometric power spectrum feature vector. Extract the energy distribution value and DC component of the spatial geometric power spectrum feature vector in the low-frequency and high-frequency ranges. When the energy distribution value in the low-frequency range is greater than the fourth threshold and the DC component is greater than the third threshold, determine that the surface corresponding to the two-dimensional sliding analysis window is a smooth surface. Then, reverse the initial terrain label conclusion corresponding to the gravel category to the initial terrain label conclusion corresponding to the rock surface category.

[0046] Specifically, the energy distribution value in the low-frequency range is defined as the sum of the normalized energy components corresponding to the first few radial frequency bands in the spatial geometric power spectrum feature vector; the energy distribution value in the high-frequency range is defined as the sum of the normalized energy components corresponding to the last few radial frequency bands in the spatial geometric power spectrum feature vector; and the DC component is defined as the normalized energy component of the DC component in the spatial geometric power spectrum feature vector. The first three frequency bands are taken for the low-frequency range, and the last three frequency bands are taken for the high-frequency range. Smooth surfaces typically exhibit a dominance of low-frequency energy and a larger DC component in the power spectrum, while rough surfaces corresponding to gravel typically exhibit a higher proportion of high-frequency energy. The third threshold is set to 0.2 to constrain the proportion of the DC component in the spatial geometric power spectrum feature vector; the fourth threshold is set to 0.6 to constrain the proportion of energy in the low-frequency range of the spatial geometric power spectrum feature vector. These values ​​are determined after performing window-level power spectrum statistics on smooth surface samples and gravel surface samples, ensuring a balance between the false trigger rate and the missed trigger rate when judging smooth surfaces. When the frequency domain calculation unit performs reverse reclassification, the pixel selection range is limited to pixels within the two-dimensional sliding analysis window whose initial terrain label conclusion is gravel and which have triggered a physical mismatch signal. This forms a closed loop with the physical mismatch condition, avoiding indiscriminate reclassification of non-gravel pixels and introducing new errors. When the same pixel falls into multiple two-dimensional sliding analysis windows and the smooth surface judgment results are inconsistent, the number of windows that simultaneously meet the conditions of low-frequency energy ratio being greater than the fourth threshold and DC component ratio being greater than the third threshold is used as the count basis, and the judgment result with the dominant count is used as the final judgment result for that pixel. When the count cannot form a dominant count, the window with the larger sum of the overthreshold margin of low-frequency energy ratio relative to the fourth threshold and the overthreshold margin of DC component ratio relative to the third threshold is used as the final judgment result to ensure the stability of the reverse reclassification result.

[0047] In practice, the response diagnosis module includes an active disturbance unit, an entropy field calculation unit, and a decision cascade unit. The active perturbation unit is used to perform mathematical transformations on the input image to obtain a set of virtual terrain maps; The collection of virtual terrain maps includes underexposed virtual terrain maps, overexposed virtual terrain maps, warm color temperature shifted virtual terrain maps, cool color temperature shifted virtual terrain maps, and light source incident angle rotated virtual terrain maps; The entropy field calculation unit is used to perform pixel-level terrain label inference on the virtual terrain map set to obtain a pixel-level uncertain entropy field. The decision cascade unit is used to perform cascaded fusion calculations on pixel-level uncertainty entropy field, spatial geometric indicators and terrain label conclusions to obtain terrain accessibility cost map.

[0048] Specifically, when the entropy field calculation unit performs pixel-level terrain label inference on the virtual terrain map set, it first ensures that the underexposed virtual terrain map, overexposed virtual terrain map, warm color temperature shifted virtual terrain map, cool color temperature shifted virtual terrain map, and light source incident angle rotated virtual terrain map have the same resolution and the same pixel coordinate system as the input image, so that the same pixel coordinate corresponds one-to-one among the five virtual terrain maps. The entropy field calculation unit sequentially inputs the five virtual terrain maps into the concept hint segmentation operator to obtain the corresponding material semantic probability field, and performs maximum value judgment at each pixel coordinate for each material semantic probability field. It selects the category with the highest probability among grassland, bare soil, gravel, wet mud, rock surface, water body, sand, shadow, highlight, reflection, and ruts as the pixel-level terrain label inference result for that virtual terrain map at that pixel. Subsequently, the entropy field calculation unit statistically analyzes the probability distribution of the inference results for each pixel across five virtual terrain maps, with each virtual terrain map outputting a probability vector for the corresponding pixel across 11 terrain categories. The probability results from the five virtual terrain maps are then averaged and fused to obtain the fused probability distribution for the pixel across the 11 terrain categories. Based on this fused probability distribution, the uncertainty entropy value for the pixel is calculated as the pixel value of the pixel-level uncertainty entropy field, where the entropy value calculation is based on the 11 terrain category space. To avoid instability in logarithmic operations when the probability is 0, a 1×10⁻⁶ modulo operation is added to each category probability value in the fused probability distribution during the entropy value calculation process. -6 The smoothing term.

[0049] In specific implementation, the processing logic for mathematical transformation of the input image includes: The input image is subjected to gamma nonlinear stretching using the first gamma coefficient and the second gamma coefficient respectively to obtain underexposed virtual terrain map and overexposed virtual terrain map; The input image is transformed by applying warm color temperature parameters and cool color temperature parameters respectively to obtain a warm color temperature offset virtual terrain map and a cool color temperature offset virtual terrain map. Using the relighting operator, the brightness and darkness features in the input image are reprojected in three dimensions according to the preset incident angle of the simulated light source, resulting in a virtual terrain map rotated by the incident angle of the light source.

[0050] Specifically, the processing logic for gamma nonlinear stretching of the input image is as follows: when performing gamma nonlinear stretching on the input image, a power function transformation is performed on each pixel channel of the input image, and the transformed pixel value is set as the gamma coefficient power of the input pixel value. In this embodiment, the first gamma coefficient is set to 1.4 to generate an underexposed virtual terrain map; the second gamma coefficient is set to 0.7 to generate an overexposed virtual terrain map. By simultaneously simulating underexposed and overexposed scenarios, the system can evaluate the stability of the terrain label conclusions under different exposure conditions.

[0051] When performing color temperature matrix transformation on the input image, in this embodiment, 3000K is selected as the warm color temperature parameter to obtain a warm color temperature offset virtual terrain map; 9000K is selected as the cool color temperature parameter to obtain a cool color temperature offset virtual terrain map. By simultaneously simulating warm and cool light environments, the system can assess the sensitivity of terrain label conclusions to white balance changes.

[0052] The processing logic for color temperature matrix transformation of the input image is as follows: channel gain is applied to the red, green, and blue channels of the input image, and normalization is performed respectively. The target color temperature range is set to 3000K to 9000K, with 6500K as the reference color temperature. 3000K is used to generate a warm color temperature offset virtual terrain map to simulate the white balance change of relatively enhanced red channel under warm light conditions; 9000K is used to generate a cool color temperature offset virtual terrain map to simulate the white balance change of relatively enhanced blue channel under cool light conditions.

[0053] This method covers the color temperature variations of common indoor and outdoor light sources and avoids color distortion caused by extreme color temperatures, which could affect the stability of the concept cues segmentation operator. In this embodiment, channel gain is obtained by using a lookup table and interpolation to establish a mapping relationship between color temperature and channel gain. The gains of the red, green, and blue channels are pre-calibrated for discrete color temperature points and stored in the mapping table. When the input target color temperature falls between two adjacent points, linear interpolation is used to obtain the corresponding channel gain. After channel transformation, the red, green, and blue channels are normalized to a uniform scale, scaling them by the same proportion so that the maximum value of any of the three channels does not exceed 1. This normalization operation maintains the relative proportions between channels, does not change the image hue information, and only constrains the overall dynamic range, thus avoiding numerical overflow that could affect the stability of the concept cues segmentation operator. When the target color temperature is 3000K, a warm color temperature offset virtual terrain map is obtained; when the target color temperature is 9000K, a cool color temperature offset virtual terrain map is obtained, thus forming virtual terrain maps under different color temperature conditions and maintaining numerical stability.

[0054] The mapping table is shown in Table 1; Table 1

[0055] The relighting operator performs 3D reprojection of the brightness and darkness features in the input image based on a preset simulated light source incident angle to obtain a virtual terrain map rotated by the light source incident angle. The processing logic is as follows: First, the spatial geometric depth matrix is ​​read, and the depth value of each pixel is back-projected to the 3D point cloud coordinates in the camera coordinate system in combination with the camera intrinsic parameters, thereby establishing the correspondence between pixels and 3D geometry. Then, the normal vector of each pixel is calculated based on the depth difference in the pixel neighborhood of the spatial geometric depth matrix, and the normal vector represents the local orientation of the ground surface at that pixel. In this embodiment, the relighting operator uses the Lambert lighting model to calculate the brightness and darkness terms. The brightness and darkness terms are determined by the dot product of the normal vector and the light source direction, and negative values ​​are truncated to 0, thereby obtaining the pixel illumination intensity under the preset simulated light source incident angle. The preset simulated light source incident angle is a fixed value, the pitch angle is set to 50 degrees, and the azimuth angle is set to 120 degrees, where the reference direction is the camera optical axis direction and is used as the zero azimuth reference direction. To achieve the rotation of the light source incident angle, the initial light source direction is first taken as the camera's optical axis. This light source direction is then rotated 120 degrees around the camera's vertical axis to obtain the target azimuth direction. Next, this light source direction is rotated around the camera's lateral axis to a pitch angle of 50 degrees to obtain the final light source direction. Subsequently, the brightness intensity at each pixel is calculated using the relighting operator based on the normal vector obtained from the spatial geometric depth matrix and the final light source direction. The brightness components of the input image are then remapped according to the proportion of this brightness intensity change, while maintaining the relative proportions of the red, green, and blue channels. This results in a virtual terrain map with rotated light source incident angle, ensuring that the brightness variations in this virtual terrain map are consistent with the spatial geometric depth matrix and used for calculating the pixel-level uncertainty entropy field.

[0056] In practice, the processing logic for the terrain accessibility cost map is as follows: The terrain label results are mapped to a pixel-level base cost matrix, and a geometric cost matrix is ​​constructed based on spatial geometric indicators. The base cost matrix and the geometric cost matrix are multiplied point-to-point to obtain the initial cost matrix. The entropy value corresponding to each pixel in the pixel-level uncertainty entropy field is extracted as a weight coefficient. The weight coefficient is multiplied by the matrix element of the corresponding coordinate in the initial cost matrix to obtain the terrain accessibility cost map.

[0057] Specifically, when mapping terrain label conclusions to a pixel-level base cost matrix, the terrain label conclusion for each pixel is read and a value is assigned by looking up a table. The base cost value ranges from 0 to 1, with a larger value indicating a less traversable path. In this embodiment, the values ​​are: grassland 0.25, bare soil 0.30, gravel 0.45, wet mud 0.85, rock surface 0.40, water body 1.00, sandy land 0.75, shadow 0.55, highlight 0.55, reflective 0.60, and ruts 0.65, thus obtaining the pixel-level base cost matrix. When constructing the geometric cost matrix based on spatial geometric indicators, only local dip angle and surface curvature are used. First, the local dip angle is mapped to a slope cost component, with 0 for a local dip angle less than or equal to 10 degrees and 1 for a local dip angle greater than or equal to 45 degrees. The remaining values ​​are linearly proportional, with the local dip angle minus 10 and then divided by 35. Next, the absolute value of surface curvature is mapped to an undulation cost component, with the value being the absolute value of surface curvature divided by 0.08 and truncated to 0 to 1. Finally, the geometric cost matrix is ​​1 plus 0.7 times the slope cost component and 0.3 times the undulation cost component. Here, 0.7 and 0.3 are the fusion weight coefficients of the geometric cost matrix. 0.7 is used to characterize the dominant contribution of local dip angle in geometric risk assessment, and 0.3 is used to characterize the auxiliary contribution of surface curvature in geometric risk assessment. The above values ​​are determined by calibrating the spatial geometric indicators and actual accessibility results of typical surface samples, so that the geometric cost matrix maintains higher sensitivity to slope changes and simultaneously reflects the access instability caused by local undulations. The initial cost matrix is ​​obtained by multiplying the pixel-level basic cost matrix and the geometric cost matrix pixel by pixel.

[0058] When extracting the entropy value corresponding to each pixel in the pixel-level uncertainty entropy field as a weighting coefficient, the entropy value is first normalized to between 0 and 1. The normalization denominator is set to the maximum entropy value of 1.6094. The maximum entropy value is set to the natural logarithm of 5 because the virtual terrain map set contains 5 virtual terrain maps, and the maximum uncertainty in the pixel-level terrain label inference result occurs when the 5 virtual terrain maps give different categories. The weighting coefficient is set to 1 plus 0.8 times the normalized entropy value, where 0.8 represents the amplification of uncertainty risk, meaning that the greater the uncertainty, the higher the passage cost of that pixel to make the planning more conservative. The weighting coefficient is multiplied pixel by pixel with the initial cost matrix to obtain the terrain accessibility cost map.

[0059] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0060] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the protection scope of the present invention.

Claims

1. An AI-based terrain recognition system, characterized in that, include: The data alignment module processes the input image using a concept cue segmentation operator to obtain a material semantic probability field, processes the input image using a monocular depth operator to obtain a spatial geometric depth matrix, and then aligns and stitches the data to obtain a feature map. The conflict determination module identifies highly confusing regions and performs secondary segmentation to obtain an updated material semantic probability field. The physical verification module extracts spatial geometric indicators based on the spatial geometric depth matrix channels in the feature map and generates initial terrain label conclusions by combining them with the updated material semantic probability field. The response diagnostic module performs mathematical transformations and pixel-level terrain label inference on the input image to obtain a pixel-level uncertainty entropy field. Based on the pixel-level uncertainty entropy field, spatial geometric indicators, and initial terrain label conclusions, it generates a terrain accessibility cost map.

2. The AI-based terrain recognition system as described in claim 1, characterized in that, The data alignment module includes semantic units, geometric units, and tensor alignment units; The semantic unit is used to process the input image through a concept cue segmentation operator to obtain a material semantic probability field; The material semantic probability field includes a set of terrain ontology categories and a set of environmental noise categories. The set of terrain ontology categories includes grassland, bare soil, gravel, wet mud, rock surface, water body, and sand. The set of environmental noise categories includes shadow, highlight, reflection, and ruts. The geometric unit is used to process the input image using a monocular depth operator to obtain a spatial geometric depth matrix; The tensor alignment unit is used to read the homogeneous coordinates of the material semantic probability field, and to perform a resampling homogenization transformation on the spatial geometric depth matrix based on the homogeneous coordinates. The spatial geometric depth matrix after the resampling homogenization transformation is then concatenated with the material semantic probability field in the pixel channel dimension to obtain the feature map.

3. The AI-based terrain recognition system as described in claim 2, characterized in that, The conflict determination module includes an overlapping scanning unit, a block segmentation unit, and a secondary segmentation unit; The overlapping scanning unit is used to calculate the cross-union ratio (CUI) between different types of masks in the material semantic probability field channel of the feature map. When the CUI is greater than the first threshold, a conflict marker is attached and the region is identified as a high confusion pair. The block segmentation unit is used to extract the boundaries of highly confused regions and segment out local high-dimensional image blocks. The secondary segmentation unit is used to generate hard negative sample prompts based on the contextual features of local high-dimensional image patches, and inputs the local high-dimensional image patches and hard negative sample prompts into the concept prompt segmentation operator to obtain the updated material semantic probability field.

4. The AI-based terrain recognition system as described in claim 3, characterized in that, The logic for calculating the intersection-union ratio and the mount conflict flag includes: Separate wet mud category mask and shadow category mask from the material semantic probability field; Extract the first grid coordinates corresponding to non-zero pixels in the wet mud category mask and the second grid coordinates corresponding to non-zero pixels in the shadow category mask, and take the intersection of the first grid coordinates and the second grid coordinates to obtain the overlapping coordinate region; The total number of pixels in the overlapping coordinate region is taken as the total number of intersection pixels. The total number of pixels in the union of the wet mud category mask and the shadow category mask is also counted. The total number of intersection pixels is divided by the total number of union pixels to obtain the cross-union ratio. Determine whether the intersection-union ratio (IU) is greater than the first threshold. When the crossover ratio is greater than the first threshold, the overlapping coordinate region is assigned a binary conflict status bit of 1, thus completing the mounting of the conflict marker.

5. The AI-based terrain recognition system as described in claim 4, characterized in that, The physical verification module includes an index extraction unit, a boundary matching unit, and a frequency domain calculation unit. The index extraction unit is used to perform local window gradient difference calculation on the spatial geometric depth matrix channel in the feature map to obtain the spatial geometric index of each pixel. The spatial geometric index includes local tilt angle, surface curvature and normal vector. The boundary matching unit is used to determine the maximum value of the probability of each category in the updated material semantic probability field, obtain the initial terrain label conclusion corresponding to each pixel, perform threshold matching judgment between the local tilt angle and the initial terrain label conclusion to obtain the physical mismatch signal, and perform confidence reduction operation on the updated material semantic probability field. The frequency domain calculation unit is used to divide the region corresponding to the physical mismatch signal into a two-dimensional sliding analysis window, extract the local height residual of each pixel point in the two-dimensional sliding analysis window relative to the spatial geometric depth matrix, and obtain the height field residual. The height field residuals are transformed to the frequency domain using a two-dimensional fast Fourier transform, and the spatial geometric power spectrum feature vector is calculated. Based on the spatial geometric power spectrum feature vector, the initial terrain label conclusion is reversed.

6. The AI-based terrain recognition system as described in claim 5, characterized in that, The threshold matching determination logic is as follows: Read the probability of the gravel category and the local tilt angle in the updated material semantic probability field. ; When the probability of the gravel category is greater than the second threshold and the local dip angle is... When a physical mismatch signal is obtained, the probability of the gravel category of that pixel is reduced to a preset baseline value.

7. The AI-based terrain recognition system as described in claim 6, characterized in that, The logic for handling reverse judgment is as follows: The height field residual of each pixel within the two-dimensional sliding analysis window is extracted from the spatial geometric depth matrix. A two-dimensional fast Fourier transform is performed on the height field residual to obtain the spatial geometric power spectrum feature vector. Extract the energy distribution value and DC component of the spatial geometric power spectrum feature vector in the low-frequency and high-frequency ranges. When the energy distribution value in the low-frequency range is greater than the fourth threshold and the DC component is greater than the third threshold, determine that the surface corresponding to the two-dimensional sliding analysis window is a smooth surface. Then, reverse the initial terrain label conclusion corresponding to the gravel category to the initial terrain label conclusion corresponding to the rock surface category.

8. The AI-based terrain recognition system as described in claim 7, characterized in that, The response diagnosis module includes an active disturbance unit, an entropy field calculation unit, and a decision cascade unit. The active perturbation unit is used to perform mathematical transformations on the input image to obtain a set of virtual terrain maps; The set of virtual terrain maps includes underexposed virtual terrain maps, overexposed virtual terrain maps, warm color temperature shifted virtual terrain maps, cool color temperature shifted virtual terrain maps, and light source incident angle rotation virtual terrain maps. The entropy field calculation unit is used to perform pixel-level terrain label inference on the virtual terrain map set to obtain a pixel-level uncertainty entropy field. The decision cascade unit is used to perform cascaded fusion calculations on pixel-level uncertainty entropy field, spatial geometric indicators and terrain label conclusions to obtain a terrain accessibility cost map.

9. The AI-based terrain recognition system as described in claim 8, characterized in that, The processing logic for performing mathematical transformations on the input image includes: The input image is subjected to gamma nonlinear stretching using the first gamma coefficient and the second gamma coefficient respectively to obtain underexposed virtual terrain map and overexposed virtual terrain map; The input image is transformed by applying warm color temperature parameters and cool color temperature parameters respectively to obtain a warm color temperature offset virtual terrain map and a cool color temperature offset virtual terrain map. Using the relighting operator, the brightness and darkness features in the input image are reprojected in three dimensions according to the preset incident angle of the simulated light source, resulting in a virtual terrain map rotated by the incident angle of the light source.

10. The AI-based terrain recognition system as described in claim 9, characterized in that, The processing logic for the terrain accessibility cost map is as follows: The terrain label results are mapped to a pixel-level base cost matrix, and a geometric cost matrix is ​​constructed based on spatial geometric indicators. The base cost matrix and the geometric cost matrix are multiplied point-to-point to obtain the initial cost matrix. The entropy value corresponding to each pixel in the pixel-level uncertainty entropy field is extracted as a weight coefficient. The weight coefficient is multiplied by the matrix element of the corresponding coordinate in the initial cost matrix to obtain the terrain accessibility cost map.

Images