An off-road complex road condition recognition method and system based on image recognition

Abstract

This invention discloses an image recognition-based method for identifying complex off-road conditions, relating to the field of off-road complex road condition technology. The method involves obtaining a high dynamic range input image; outputting a multi-scale geometric feature tensor; generating a global fusion feature tensor using an evidence pyramid fusion module with multi-scale uncertainty estimates as dynamic weights; obtaining pixel-level evidence parameter tensors and combining them with prior parameters to generate a Dirichlet distribution parameter tensor; obtaining a temporally consistent semantic mapping and a temporally consistent uncertainty mapping; generating a drivable area safety mask; and outputting drivable path constraint information to the vehicle chassis control system to execute deceleration control, path replanning, or safety takeover commands. This invention transforms chassis control decision-making from a single semantic-driven approach to a semantic-uncertainty collaborative approach, significantly improving the robustness and safety redundancy of path planning and control decisions in complex off-road environments.

Description

Technical Field

[0001] This invention relates to the field of off-road complex road condition technology, and in particular to an off-road complex road condition recognition method based on image recognition. Background Technology

[0002] With the development of autonomous driving technology, environmental perception systems are gradually expanding from structured road scenarios to unstructured off-road and complex road conditions. In structured road environments, road boundaries are clear and semantic categories are stable, allowing traditional convolutional neural network-based semantic segmentation methods to accurately identify lanes and detect obstacles. However, in off-road and complex road conditions, the terrain exhibits high irregularity and uncertainty, resulting in numerous blurred boundaries and non-uniform texture distributions in images. Existing feature extraction methods based on regular convolutional structures, due to their use of fixed-shape local receptive fields, struggle to adapt to the spatial variations of complex terrain and easily introduce background noise in boundary regions, leading to incomplete representation of terrain topology and consequently affecting the accurate segmentation of traversable areas.

[0003] Most existing semantic segmentation models are based on the closed-set assumption. In complex off-road environments, there is a large amount of unseen or out-of-distribution data in real-world scenarios. Existing methods usually normalize the class probabilities using the Softmax function. When faced with an unknown class, they still force the output of a certain known class, resulting in an overconfident erroneous judgment. During off-road driving, this can cause vehicles to misidentify impassable areas as passable areas, leading to chassis collisions or the risk of getting stuck.

[0004] Existing technologies for handling uncertainty mostly focus on the post-processing stage or simple confidence threshold judgment, lacking a mechanism for systematically modeling and propagating uncertainty within the model. This makes it difficult to effectively distinguish between uncertainty caused by data noise and uncertainty caused by insufficient model understanding, resulting in compromised robustness and security in complex environments. In terms of temporal consistency processing, existing methods often rely on simple inter-frame smoothing or filtering strategies, which struggle to maintain the stability of semantic and uncertainty information under conditions of severe off-road vehicle vibration and frequent changes in perspective.

[0005] When applying perception results to vehicle control decisions, existing technologies typically rely solely on a single semantic segmentation result to construct drivable areas, lacking effective mechanisms to suppress uncertain areas and failing to proactively avoid high-risk areas, thus posing safety hazards in path planning and chassis control. Summary of the Invention

[0006] One objective of this invention is to propose an image recognition-based method for identifying complex off-road conditions. This invention transforms chassis control decisions from being driven by a single semantic meaning to being driven by semantic-uncertainty collaboration, significantly improving the robustness and safety redundancy of path planning and control decisions in complex off-road environments.

[0007] An image recognition-based method for identifying complex off-road conditions according to an embodiment of the present invention includes: The RGB image of the off-road scene is acquired by the imaging device installed at the front of the off-road vehicle and preprocessed to obtain a high dynamic range input image. The high dynamic range input image is input into the InternImage feature extraction network, which includes uncertainty-guided deformable convolutional layers. Based on the splicing tensor of the uncertainty mapping of the previous frame and the shallow features of the current frame, the sampling offset and weight magnitude of the deformable convolution are adaptively adjusted, and a multi-scale geometric feature tensor is output. Cross-scale residual fusion is performed on the multi-scale geometric feature tensor, and a global fused feature tensor is generated by an evidence pyramid fusion module with multi-scale uncertainty estimates as dynamic weights. The global fusion feature tensor is input into the evidence inference head, and the pixel-level evidence parameter tensor is obtained by using non-negative mapping. The Dirichlet distribution parameter tensor is then generated by combining the prior parameters. Based on the Dirichlet distribution parameter tensor, the semantic expectation map and the pixel-level uncertainty map are calculated. The semantic expectation map and the pixel-level uncertainty map are then subjected to convolutional gated recurrent unit temporal filtering to obtain the temporally consistent semantic map and the temporally consistent uncertainty map. The temporally consistent semantic mapping and the temporally consistent uncertainty mapping are projected onto the bird's-eye view coordinate system to construct an uncertainty suppression mask. Projected pixels with uncertainty exceeding a preset threshold are rewritten as impassable labels to generate a safe mask for drivable areas. The drivable path constraint information is calculated based on the safety mask of the drivable area, and the drivable path constraint information is output to the vehicle chassis control system to execute deceleration control, path replanning, or safety takeover commands.

[0008] Optionally, the preprocessing includes gamma correction, histogram equalization, and multi-exposure stitching based on Poisson fusion.

[0009] Optionally, the step of inputting the high dynamic range input image into the InternImage feature extraction network, which includes uncertainty-guided deformable convolutional layers, includes: The high dynamic range input image of frame t is input into the shallow convolutional mapping unit of the InternImage feature extraction network to obtain the shallow features of the current frame. For the initial time, the uncertainty mapping of frame t-1 is initialized to a preset no-priority tensor; for non-initial time, spatial alignment of the uncertainty mapping of frame t-1 is performed based on the self-motion information of the off-road vehicle between two adjacent frames; the spatially aligned uncertainty mapping of frame t-1 adjacent to frame t is adjusted to the same spatial resolution as the shallow features of the current frame through a scaling operator to obtain a resampled uncertainty tensor; the resampled uncertainty tensor is concatenated with the shallow features of the current frame along the channel dimension to obtain a concatenated tensor. The spliced ​​tensor is input into the corresponding hierarchical adaptation unit in the InternImage feature extraction network, and spatial and channel scale transformations are performed on the spliced ​​tensor to obtain the hierarchical guiding tensor. The hierarchical guiding tensor is input into the offset generation branch and the magnitude generation branch in the InternImage feature extraction network to obtain the sampling offset tensor and the weight magnitude tensor. The input feature tensor of the corresponding scale level is used as the convolution input of the uncertainty-guided deformable convolutional layer. The sampling offset tensor is used to adaptively adjust the position of each sampling point in the regular convolution sampling grid, and the weight amplitude tensor is used to adaptively adjust the convolution response intensity corresponding to each sampling point to obtain the output feature tensor. The output feature tensors at each scale level are aggregated to obtain a multi-scale geometric feature tensor.

[0010] Optionally, the cross-scale residual fusion of multi-scale geometric feature tensors includes: Perform channel-unified mapping on the output feature tensor of each scale level in the multi-scale geometric feature tensor to obtain a scale-level aligned feature tensor. The alignment feature tensors at each scale level are mapped to non-negative evidence tensors, and the corresponding multi-scale uncertainty estimate is calculated based on each non-negative evidence tensor. Based on the multi-scale uncertainty estimates corresponding to each scale level, calculate the dynamic weights corresponding to each scale level. Based on the dynamic weights corresponding to each scale level, cross-scale residual fusion is performed on the aligned feature tensors of each scale level to obtain the fused feature tensors of each scale level. The fused feature tensors at each scale level are transformed to a unified output spatial resolution and then weighted and aggregated according to the dynamic weights corresponding to each scale level to generate a global fused feature tensor.

[0011] Optionally, the step of inputting the global fusion feature tensor into the evidence inference head and obtaining pixel-level evidence parameter tensors using non-negative mapping includes: Input the global fusion feature tensor into the evidence inference head, perform pixel-by-pixel category mapping on the global fusion feature tensor, and obtain the pixel-level original category response tensor; The pixel-level terrain transition intensity is calculated based on the pixel-level original class response tensor, and the pixel-level transition suppression coefficient is calculated based on the pixel-level terrain transition intensity. Perform a nonnegative mapping on the pixel-level original class response tensor, and combine the class competition relationship with the pixel-level transition suppression coefficient to obtain the pixel-level evidence parameter tensor; Pixel-level local consistency coefficients and pixel-level class tendency coefficients are calculated based on pixel-level evidence parameter tensors, and pixel-level adaptive prior parameter tensors are generated by combining prior parameters. The pixel-level evidence parameter tensor and the pixel-level adaptive prior parameter tensor are superimposed position by position and category by category to generate the Dirichlet distribution parameter tensor.

[0012] Optionally, the step of calculating the semantic expectation mapping and pixel-level uncertainty mapping based on the Dirichlet distribution parameter tensor includes: Calculate the semantic expectation mapping based on the Dirichlet distribution parameter tensor; Calculate pixel-level uncertainty mapping based on the Dirichlet distribution parameter tensor; Temporally filtered convolutional gated recurrent units are applied to the semantic expectation mapping to obtain a temporally consistent semantic mapping. A temporally consistent uncertainty map is obtained by performing convolutional gated recurrent unit temporal filtering on the pixel-level uncertainty map.

[0013] Optionally, the step of projecting the temporally consistent semantic map and the temporally consistent uncertainty map onto the bird's-eye view coordinate system to construct an uncertainty suppression mask includes: Projecting the temporally consistent semantic map onto the bird's-eye view coordinate system yields the bird's-eye view semantic map; Projecting the time-consistent uncertainty map onto the bird's-eye view coordinate system yields the bird's-eye view uncertainty map; Construct an uncertainty suppression mask based on a bird's-eye view uncertainty mapping; Bird's-eye view semantic labels are determined based on the bird's-eye view semantic mapping, and an uncertainty suppression mask is invoked. Bird's-eye view semantic labels with uncertainty exceeding a preset threshold are rewritten as impassable labels to obtain a safe semantic label mapping and generate a safe mask for drivable areas.

[0014] Optionally, the step of calculating drivable path constraint information based on the drivable area safety mask and outputting the drivable path constraint information to the vehicle chassis control system includes: Based on the drivable area safety mask, determine the drivable connectivity area safety mask that connects to the current position of the off-road vehicle; Based on the vehicle projection reference position, vehicle orientation, and safety mask of the drivable connected area, construct the forward drivable constraint area; Calculate drivable path constraint information based on the drivable connected area safety mask and the forward drivable constraint area; Based on the drivable path constraint information, generate the corresponding judgment results for deceleration control command, path replanning command, and safe takeover command; The drivable path constraint information is output to the vehicle chassis control system to execute deceleration control, path replanning, or safety takeover commands.

[0015] Optionally, an image recognition-based off-road complex road condition recognition system is provided for executing an image recognition-based off-road complex road condition recognition method, comprising: The image acquisition and preprocessing module is used to acquire and preprocess RGB images of off-road scenes captured by an imaging device installed at the front of the off-road vehicle to obtain high dynamic range input images. The feature extraction module is used to input high dynamic range input images into the InternImage feature extraction network, which includes uncertainty-guided deformable convolutional layers. Based on the splicing tensor of the uncertainty mapping of the previous frame and the shallow features of the current frame, the module adaptively adjusts the sampling offset and weight magnitude of the deformable convolution and outputs a multi-scale geometric feature tensor. The cross-scale fusion module is used to perform cross-scale residual fusion on multi-scale geometric feature tensors. The global fusion feature tensor is generated by the evidence pyramid fusion module with multi-scale uncertainty estimates as dynamic weights. The evidence reasoning module is used to input the global fusion feature tensor into the evidence reasoning head, obtain the pixel-level evidence parameter tensor using non-negative mapping, and generate the Dirichlet distribution parameter tensor by combining the prior parameters. The uncertainty quantization and temporal filtering module is used to calculate the semantic expectation map and pixel-level uncertainty map based on the Dirichlet distribution parameter tensor, and to perform convolutional gated recurrent unit temporal filtering on the semantic expectation map and pixel-level uncertainty map to obtain temporally consistent semantic map and temporally consistent uncertainty map. The spatial projection and safety constraint generation module is used to project the temporally consistent semantic mapping and the temporally consistent uncertainty mapping onto the bird's-eye view coordinate system, construct an uncertainty suppression mask, rewrite the projected pixels with uncertainty exceeding a preset threshold as impassable labels, and generate a drivable area safety mask. The path constraint and chassis control module is used to calculate drivable path constraint information based on the safety mask of the drivable area, and output the drivable path constraint information to the vehicle chassis control system to execute deceleration control, path replanning or safety takeover commands.

[0016] The beneficial effects of this invention are: (1) This invention constructs an InternImage feature extraction mechanism based on uncertainty guidance, which concatenates the uncertainty mapping of the previous frame with the shallow features of the current frame and uses it to modulate the sampling offset and weight amplitude of deformable convolution, so that the convolution sampling position can be adaptively distributed along the real texture boundary of the off-road terrain. In the irregular boundary scene of the junction of mud and water pit, and the mixed area of ​​gravel and grass, it effectively suppresses the background noise interference caused by regular convolution, significantly improves the boundary preservation ability in complex terrain areas, and makes the boundary extraction of passable areas more in line with the real terrain topology. The boundary consistency index is significantly improved on typical off-road datasets, thereby avoiding the chassis misjudgment problem caused by boundary deviation.

[0017] (2) This invention constructs a Dirichlet distribution parameter generation mechanism based on evidence reasoning, introduces pixel-level terrain transition intensity, category competition relationship and local consistency constraint, and makes a structured improvement on the traditional evidence deep learning model for off-road scenarios. The pixel-level evidence not only reflects the confidence level of the category, but also can characterize the ambiguity of terrain boundaries and spatial consistency features. Through the improved mechanism, when facing obstacles outside the distribution or unknown terrain, the model no longer forces the output of high confidence category results, but generates a high uncertainty response, thereby significantly suppressing the phenomenon of overconfidence misjudgment. The misjudgment rate is significantly reduced in the unknown obstacle recognition task, and the safety perception capability in the off-road environment is improved.

[0018] (3) This invention constructs a dual-stream temporal filtering mechanism of semantic expectation mapping and pixel-level uncertainty mapping, and combines uncertainty suppression and path constraint calculation in the bird's-eye view coordinate system to realize closed-loop coupling optimization from perception results to vehicle control decisions. The convolutional gated loop unit performs synchronous temporal modeling of semantic and uncertainty information, effectively eliminating inter-frame jitter caused by the bumps of off-road vehicles. The bird's-eye view rewriting mechanism based on uncertainty suppression can actively reconstruct high-risk areas into impassable areas. The path constraint information is constructed by combining connectivity constraints, boundary clearance distance and forward coverage, so that the chassis control decision is transformed from single semantic drive to semantic-uncertainty collaborative drive, which significantly improves the robustness and safety redundancy of path planning and control decisions in complex off-road environments. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of an image recognition-based method for identifying complex off-road conditions proposed in this invention; Figure 2This is a block diagram of the InternImage feature extraction network with uncertainty-guided deformable convolutional layers in an image recognition-based off-road complex road condition recognition method proposed in this invention. Detailed Implementation

[0020] Example 1: Reference Figures 1-2 A method for identifying complex off-road terrain based on image recognition, comprising: The RGB image of the off-road scene is acquired by the imaging device installed at the front of the off-road vehicle and preprocessed to obtain a high dynamic range input image. In this embodiment, preprocessing includes gamma correction, histogram equalization, and multi-exposure stitching based on Poisson fusion.

[0021] The high dynamic range input image is input into the InternImage feature extraction network, which includes uncertainty-guided deformable convolutional layers. Based on the splicing tensor of the uncertainty mapping of the previous frame and the shallow features of the current frame, the sampling offset and weight magnitude of the deformable convolution are adaptively adjusted, and a multi-scale geometric feature tensor is output. In this embodiment, the high dynamic range input image is input into the InternImage feature extraction network, which includes an uncertainty-guided deformable convolutional layer, including: The high dynamic range input image of frame t is input into the shallow convolutional mapping unit of the InternImage feature extraction network to obtain the shallow features of the current frame. In Example 1, the shallow convolutional mapping unit of the InternImage feature extraction network consists of an initial convolutional mapping layer, a normalization layer, a nonlinear activation layer, and a downsampling convolutional mapping layer connected in sequence.

[0022] The initial convolutional mapping layer is used to perform local receptive field mapping on the high dynamic range input image to extract the initial response features corresponding to the surface texture, boundary morphology and reflective areas of the off-road scene. The channel distribution of the initial response features is adjusted by the normalization processing layer. The nonlinear activation layer is used to perform nonlinear response enhancement on the initial response features after channel distribution adjustment. The spatial resolution compression and channel dimension expansion of the features after nonlinear response enhancement are performed by the downsampling convolutional mapping layer to obtain the shallow features of the current frame, which are used to represent the shallow texture response, shallow edge response and shallow brightness gradient response of the off-road scene at the current moment.

[0023] The shallow features of the current frame are used as the input feature tensor of the first scale level of the InternImage feature extraction network. The shallow features of the current frame form a dual-branch flow in the computation graph: on the one hand, they are used to perform channel concatenation with the resampled uncertainty tensor, and on the other hand, they are directly used as the input feature tensor of the first scale level of the InternImage feature extraction network.

[0024] For the initial time, the uncertainty mapping of frame t-1 is initialized to a preset no-priority tensor; for non-initial time, spatial alignment of the uncertainty mapping of frame t-1 is performed based on the self-motion information of the off-road vehicle between two adjacent frames; the spatially aligned uncertainty mapping of frame t-1 adjacent to frame t is adjusted to the same spatial resolution as the shallow features of the current frame through a scaling operator to obtain a resampled uncertainty tensor; the resampled uncertainty tensor is concatenated with the shallow features of the current frame along the channel dimension to obtain a concatenated tensor. In Example 1, the initialization cold start and motion compensation determination of the execution time-series computation graph are performed as follows: when t=1, the uncertainty mapping of the (t-1)th frame is initialized as a zero tensor, driving the current frame network to perform baseline feature extraction without prior history; when t>1, the pose self-motion parameters or image optical flow field of the off-road vehicle from time t-1 to time t are obtained, and the coordinate system distortion transformation is performed on the uncertainty mapping of the (t-1)th frame based on the self-motion parameters or image optical flow field to eliminate the physical space misalignment between the previous and next frames caused by vehicle bumps and translations.

[0025] The scaling operator performs position mapping and interpolation on the uncertainty values ​​of each pixel position in the spatially aligned uncertainty mapping of the (t-1)th frame according to the spatial size of the shallow features of the current frame, so that each pixel position after scaling corresponds one-to-one with each spatial position of the shallow features of the current frame, thus obtaining the resampled uncertainty tensor.

[0026] The resampled uncertainty tensor and the shallow features of the current frame are stacked in the same spatial position according to the channel dimension, so that the shallow texture response, shallow edge response and shallow brightness gradient response at each spatial position and the uncertainty value at the corresponding position together form a joint feature representation of the same position, thus obtaining the spliced ​​tensor.

[0027] The spliced ​​tensor is input into the corresponding hierarchical adaptation unit in the InternImage feature extraction network, and spatial and channel scale transformations are performed on the spliced ​​tensor to obtain the hierarchical guiding tensor. In Example 1, the hierarchical adaptation unit is used to perform spatial scale transformation and channel scale transformation on the spliced ​​tensor according to the spatial resolution and number of channels of the input feature tensor of the corresponding scale level. For the l-th scale level, the hierarchical adaptation unit with the spliced ​​tensor input set at the input end of the l-th scale level downsamples the spliced ​​tensor according to the spatial resolution of the input feature tensor of the l-th scale level, and performs channel mapping on the spliced ​​tensor according to the number of channels of the input feature tensor of the l-th scale level, so as to obtain the hierarchical guiding tensor of the l-th scale level that matches the input feature tensor of the l-th scale level in terms of spatial resolution and number of channels.

[0028] The hierarchical guiding tensor is input into the offset generation branch and the magnitude generation branch in the InternImage feature extraction network to obtain the sampling offset tensor and the weight magnitude tensor. In Example 1, the offset generation branch is a convolutional prediction branch used to generate the two-dimensional displacement corresponding to each sampling position based on the hierarchical guiding tensor of the l-th scale level, and the amplitude generation branch is a convolutional prediction branch used to generate the weighted amplitude modulation coefficients corresponding to each sampling position based on the hierarchical guiding tensor of the l-th scale level.

[0029] The offset generation branch performs a convolution mapping on the hierarchical guide tensor of the l-th scale level and outputs the sampling offset tensor of the l-th scale level. The magnitude generation branch performs a convolution mapping on the hierarchical guide tensor of the l-th scale level and applies numerical constraints to the convolution mapping result, outputting the weight magnitude tensor of the l-th scale level.

[0030] The input feature tensor of the corresponding scale level is used as the convolution input of the uncertainty-guided deformable convolutional layer. The sampling offset tensor is used to adaptively adjust the position of each sampling point in the regular convolution sampling grid, and the weight amplitude tensor is used to adaptively adjust the convolution response intensity corresponding to each sampling point to obtain the output feature tensor. In Example 1, for each target sampling center position coordinate on the feature map of the l-th scale level, based on the sampling offset tensor of the l-th scale level, a sampling offset vector corresponding to the target sampling center position coordinate is assigned to each sampling point in the regular convolution sampling grid, and the sampling offset vector is superimposed on the reference displacement of the sampling point relative to the target sampling center position coordinate to obtain the actual sampling position corresponding to the sampling point.

[0031] Based on the actual sampling location, the feature response value corresponding to the sampling point is extracted from the input feature tensor of the first scale level. Based on the weight amplitude tensor, the corresponding weight amplitude modulation coefficient is assigned to each sampling point under the coordinates of the target sampling center position. The feature response value corresponding to the sampling point, the convolution kernel weight, and the weight amplitude modulation coefficient are multiplied. The product results corresponding to all sampling points under the coordinates of the target sampling center position are accumulated to obtain the corresponding output feature response value. The above process is repeated for all target sampling center position coordinates on the feature map of the l-th scale level to obtain the output feature tensor of the l-th scale level.

[0032] ; in, Indicates the first The coordinates of the output feature tensor at each scale level at the target sampling center position Output characteristic response value at that location, This indicates the time sequence number corresponding to the current frame. This indicates the scale level number in the InternImage feature extraction network. Indicates the first The target sampling center coordinates on the feature map at each scale level Indicates the first The channel number corresponding to the input feature tensor at each scale level. Indicates the first The total number of channels in the input feature tensor at each scale level. This represents the sampling point index in the regular convolution sampling grid. Indicates the first The total number of sampling points corresponding to a single convolutional kernel in each scale level Indicates the first Each scale level, the first The first channel, the first Coordinates of each sampling point at the target sampling center The weight amplitude modulation coefficient at that point is a specific element value in the weight amplitude tensor. Indicates the first Each scale level, the first The first channel, the first The convolution kernel weights corresponding to each sampling point Indicates the first Each scale level, the first The feature response values ​​of each channel input feature at the corresponding spatial location. Represents the coordinates of the target sampling center. At that location, according to the reference displacement With sampling offset vector The jointly determined actual sampling location, from the first Each scale level, the first Feature response values ​​extracted from the input features of each channel. The first element in the regular convolution sampling grid represents the... The coordinates of each sampling point relative to the target sampling center The reference displacement, Indicates the first Each scale level, the first The first channel, the first Coordinates of each sampling point at the target sampling center The sampling offset vector at that location.

[0033] The output feature tensors at each scale level are aggregated to obtain a multi-scale geometric feature tensor.

[0034] In Example 1, the multi-scale geometric feature tensor is composed of the output feature tensors of the first scale level, the second scale level, and up to the Lth scale level, which are jointly output by the InternImage feature extraction network. It is used to represent the local detailed geometric information and large-scale terrain structure information of the off-road scene at the current moment.

[0035] The improved InternImage feature extraction network in this implementation uses InternImage as its backbone. The front end includes shallow convolutional mapping units, the middle section includes spatial alignment and resampling operators incorporating vehicle motion information and channel stitching nodes, and the back end connects to a hierarchical adaptation unit, two parallel convolutional prediction branches (offset and amplitude generation branches), and an uncertainty-guided deformable convolutional core layer. In terms of data flow and format transmission, the input three-channel high dynamic range image is downsampled by shallow units and expanded into a high-dimensional shallow feature tensor with rich spatial semantics. This high-dimensional shallow feature tensor... The computation graph forms a dual-track parallel flow. One track directly serves as the input feature tensor of the first layer of the backbone network to continue feature evolution. The other track is strictly aligned and stacked with the one-dimensional cognitive uncertainty tensor of the previous frame after pose distortion compensation and resolution interpolation (forming a spliced ​​tensor with deep coupling between features and uncertainty). The spliced ​​tensor is matched with the number of channels through spatial downsampling at the corresponding level and is transformed by the two prediction branches into a pixel-level two-dimensional sampling offset tensor and a one-dimensional weight amplitude modulation tensor subject to nonlinear constraints. This modulates the physical extraction coordinates and weight intensity of the feature response values ​​in DCNv3 and aggregates to output a multi-scale geometric feature tensor.

[0036] Compared with existing conventional visual base models and standard deformable convolutional networks, the improved InternImage feature extraction network substantially breaks the one-way open-loop paradigm that relies solely on the static features of the current frame to calculate the sampling offset. It introduces a cross-frame Dirichlet cognitive uncertainty closed-loop feedback control based on vehicle self-motion compensation and cold start mechanism. The fundamental significance and advantage of the improved underlying structure lies in giving the operator true spatial crisis memory and active focusing capabilities. This enables the improved InternImage feature extraction network to not only overcome the spatiotemporal feature misalignment caused by severe off-road bumps, but also to automatically drive the underlying receptive field to perform extremely irregular dense sampling deformation on blurred, high-risk, or unknown unstructured terrain in historical frames, just like human vision. This completely eliminates the risk of overconfidence and misjudgment caused by the traditional closed-world visual model's forced recognition of rare obstacles.

[0037] Cross-scale residual fusion is performed on the multi-scale geometric feature tensor, and a global fused feature tensor is generated by an evidence pyramid fusion module with multi-scale uncertainty estimates as dynamic weights. In this embodiment, cross-scale residual fusion of multi-scale geometric feature tensors includes: Perform channel-unified mapping on the output feature tensor of each scale level in the multi-scale geometric feature tensor to obtain a scale-level aligned feature tensor. In Example 1, for each scale level output feature tensor in the multi-scale geometric feature tensor, while maintaining the spatial resolution, a convolutional mapping with a kernel size of 1×1 is used to linearly combine and bias-superimpose the feature responses of all channels, so that each scale level output feature tensor is mapped to the same channel dimension, resulting in a scale-level aligned feature tensor.

[0038] The alignment feature tensors at each scale level are mapped to non-negative evidence tensors, and the corresponding multi-scale uncertainty estimate is calculated based on each non-negative evidence tensor. In Example 1, Softplus nonnegative activation mapping is performed on the aligned feature tensor of each scale level to remove negative responses and obtain the nonnegative evidence tensor corresponding to the scale level; nonnegative evidence statistics are performed on the spatial dimension of the nonnegative evidence tensor to obtain the evidence mean corresponding to the scale level; then the constant stability term is added to the evidence mean corresponding to the scale level, and the reciprocal of the addition result is taken to obtain the multiscale uncertainty estimate corresponding to the scale level.

[0039] Based on the multi-scale uncertainty estimates corresponding to each scale level, calculate the dynamic weights corresponding to each scale level. In Example 1, the multi-scale uncertainty estimates corresponding to each scale level are negatively exponentially mapped to obtain the unnormalized weights corresponding to each scale level; then, the unnormalized weights corresponding to each scale level are divided by the sum of the unnormalized weights corresponding to all scale levels to obtain the dynamic weights corresponding to each scale level.

[0040] Based on the dynamic weights corresponding to each scale level, cross-scale residual fusion is performed on the aligned feature tensors of each scale level to obtain the fused feature tensors of each scale level. In Example 1, for any target scale level, the alignment feature tensors of the other scale levels are first transformed to the same spatial resolution as the alignment feature tensor of the target scale level through bilinear interpolation to obtain the cross-scale alignment feature tensor relative to the target scale level. Each cross-scale alignment feature tensor is multiplied by the dynamic weight of the corresponding scale level, and the product results are accumulated to obtain the residual fusion tensor corresponding to the target scale level. The residual fusion tensor corresponding to the target scale level is added to the alignment feature tensor of the target scale level position by position to obtain the scale level fusion feature tensor corresponding to the target scale level.

[0041] The fused feature tensors at each scale level are transformed to a unified output spatial resolution and then weighted and aggregated according to the dynamic weights corresponding to each scale level to generate a global fused feature tensor. In Example 1, the fused feature tensors at each scale level are first mapped to coordinates according to the spatial resolution of the first scale level, and the feature responses at adjacent positions are weighted and summed based on the bilinear interpolation weight function to obtain the fused feature tensors at a unified resolution at each scale level. Then, the fused feature tensors at a unified resolution at each scale level are multiplied by the dynamic weights of the corresponding scale level and weighted and summed to obtain the global fused feature tensor.

[0042] The global fusion feature tensor is input into the evidence inference head, and the pixel-level evidence parameter tensor is obtained by using non-negative mapping. The Dirichlet distribution parameter tensor is then generated by combining the prior parameters. In this embodiment, the global fusion feature tensor is input into the evidence inference head, and pixel-level evidence parameter tensors are obtained using non-negative mapping, including: Input the global fusion feature tensor into the evidence inference head, perform pixel-by-pixel category mapping on the global fusion feature tensor, and obtain the pixel-level original category response tensor; In Example 1, while maintaining the spatial resolution, a convolutional mapping with a kernel size of 1×1 is used to linearly combine and bias-superimpose the feature responses of all channels on the global fusion feature tensor. This allows the global fusion feature tensor to be mapped to the same category dimension as the total number of categories for recognizing complex off-road conditions at each spatial location, resulting in a pixel-level original category response tensor.

[0043] The pixel-level terrain transition intensity is calculated based on the pixel-level original class response tensor, and the pixel-level transition suppression coefficient is calculated based on the pixel-level terrain transition intensity. In Example 1, for each spatial location and each off-road complex road condition recognition category, the response difference between the spatial location and its adjacent spatial locations on the pixel-level original category response tensor is calculated, and the response differences corresponding to all off-road complex road condition recognition categories are averaged to obtain the pixel-level terrain transition intensity corresponding to the spatial location; then, negative exponential mapping is performed on the pixel-level terrain transition intensity corresponding to the spatial location to obtain the pixel-level transition suppression coefficient corresponding to that spatial location.

[0044] Perform a nonnegative mapping on the pixel-level original class response tensor, and combine the class competition relationship with the pixel-level transition suppression coefficient to obtain the pixel-level evidence parameter tensor; In Example 1, the pixel-level original category response tensor is subjected to Softplus non-negative activation mapping at each spatial location and at each off-road complex road condition recognition category to obtain the basic non-negative evidence value.

[0045] The class competition coefficient for a particular off-road complex road condition recognition category at the current spatial location is calculated by comparing the base non-negative evidence value corresponding to the average base non-negative evidence values ​​corresponding to the other off-road complex road condition recognition categories. in, Indicates the spatial location of the current frame. Place, No. The category competition coefficients corresponding to the categories of complex off-road terrain identification; Indicates the spatial location of the current frame. Place, No. The basic non-negative evidence value corresponding to each category of complex off-road terrain identification; This represents a positive, stable term used to prevent the denominator of the category competition coefficient from being zero. This indicates the total number of categories identified for complex off-road terrain. In the process of calculating the category competition coefficient, except for the first... The serial numbers of the remaining off-road complex road condition recognition categories, excluding the existing off-road complex road condition recognition categories; Indicates the spatial location of the current frame. Place, No. The basic non-negative evidence value corresponding to each category of complex off-road terrain identification; Indicates the time sequence number corresponding to the current frame; This indicates the category number for identifying complex off-road terrain. Indicates the position number of the current frame in the spatial width direction; This indicates the position number of the current frame in the spatial height direction.

[0046] The pixel-level evidence parameter tensor is obtained by multiplying the basic non-negative evidence value, the category competition coefficient, and the pixel-level transition suppression coefficient position by position and category by category.

[0047] Pixel-level local consistency coefficients and pixel-level class tendency coefficients are calculated based on pixel-level evidence parameter tensors, and pixel-level adaptive prior parameter tensors are generated by combining prior parameters. In Example 1, a local neighborhood is constructed centered on the current spatial location. The pixel-level evidence parameter values ​​for all spatial locations within the local neighborhood across all off-road complex road condition recognition categories are statistically analyzed against the pixel-level evidence parameter values ​​corresponding to the current spatial location. The statistical results of these differences are then subjected to negative exponential mapping to obtain the pixel-level local consistency coefficient. in, Indicates the spatial location of the current frame. Pixel-level local consistency coefficient at the location; : Indicates the time sequence number corresponding to the current frame; Indicates the spatial coordinates of the current frame; Represents the natural exponential function; This represents the consistency adjustment coefficient, which is used to modulate the degree of influence of evidence differences within a local neighborhood on the pixel-level local consistency coefficient. Indicates the current spatial location The total number of spatial locations contained in the local neighborhood constructed around the center; This indicates the total number of categories identified for complex off-road terrain. Indicates spatial location in the current frame A local neighborhood constructed around a central point; Indicates adjacent spatial locations within a local neighborhood; Indicates local neighborhood Accumulate all adjacent spatial positions within the range; This indicates that all categories of complex off-road terrain identification are accumulated; Indicates the spatial location of the current frame. Place, No. Pixel-level evidence parameter values ​​corresponding to each category of complex off-road terrain identification; Indicates the adjacent spatial positions of the current frame in the local neighborhood. Place, No. Pixel-level evidence parameter values ​​corresponding to each category of complex off-road terrain identification; This indicates the current spatial location and its neighboring spatial locations in the local neighborhood at the 1st position. The absolute value of pixel-level evidence parameter differences in each category of off-road complex road condition recognition; It represents the average evidence difference between the current spatial location and all adjacent spatial locations within the local neighborhood across all categories of off-road complex road condition identification.

[0048] Divide the pixel-level evidence parameter value corresponding to a certain off-road complex road condition recognition category at the current spatial location by the sum of the pixel-level evidence parameter values ​​corresponding to all off-road complex road condition recognition categories at the current spatial location to obtain the pixel-level category tendency coefficient corresponding to the off-road complex road condition recognition category: in, Indicates the spatial location of the current frame. Place, No. Pixel-level category tendency coefficients corresponding to each category of complex off-road terrain recognition; This represents a positive stabilizing term used to prevent the denominator of the pixel-level class bias coefficient from being zero; Indicates the spatial location of the current frame. The sum of pixel-level evidence parameter values ​​corresponding to all categories of complex off-road terrain recognition.

[0049] The prior parameters corresponding to the classification of complex off-road conditions are jointly modulated with the pixel-level local consistency coefficient and the pixel-level class bias coefficient to obtain the pixel-level adaptive prior parameter tensor: in, Indicates the spatial location of the current frame. Place, No. Pixel-level adaptive prior parameter values ​​corresponding to each category of complex off-road terrain recognition; Indicates the first Prior parameters corresponding to each category of complex off-road terrain identification; This represents the prior adjustment coefficient used to adjust the magnitude of the enhancement of the prior parameters by the pixel-level local consistency coefficient and the pixel-level class bias coefficient.

[0050] The pixel-level evidence parameter tensor and the pixel-level adaptive prior parameter tensor are superimposed position by position and category by category to generate the Dirichlet distribution parameter tensor.

[0051] In Example 1, for each spatial location and each off-road complex road condition recognition category, the pixel-level evidence parameter value in the pixel-level evidence parameter tensor and the pixel-level adaptive prior parameter value in the pixel-level adaptive prior parameter tensor are added position by position and category by category to obtain the Dirichlet distribution parameter value for the corresponding spatial location and the corresponding off-road complex road condition recognition category. The Dirichlet distribution parameter values ​​for all spatial locations and all off-road complex road condition recognition categories are then combined to form the Dirichlet distribution parameter tensor.

[0052] Based on the Dirichlet distribution parameter tensor, the semantic expectation map and the pixel-level uncertainty map are calculated. The semantic expectation map and the pixel-level uncertainty map are then subjected to convolutional gated recurrent unit temporal filtering to obtain the temporally consistent semantic map and the temporally consistent uncertainty map. In this embodiment, the semantic expectation mapping and pixel-level uncertainty mapping are calculated based on the Dirichlet distribution parameter tensor, including: Calculate the semantic expectation mapping based on the Dirichlet distribution parameter tensor; In Example 1, for each spatial location and each off-road complex road condition recognition category, the Dirichlet distribution parameter value corresponding to the off-road complex road condition recognition category at the spatial location is divided by the sum of the Dirichlet distribution parameter values ​​corresponding to all off-road complex road condition recognition categories at the spatial location to obtain the semantic expectation value corresponding to the off-road complex road condition recognition category at the spatial location. The semantic expectation values ​​at all spatial locations and all off-road complex road condition recognition categories are then combined to form a semantic expectation map.

[0053] Calculate pixel-level uncertainty mapping based on the Dirichlet distribution parameter tensor; In Example 1, for each spatial location, the total number of off-road complex road condition recognition categories is divided by the sum of the Dirichlet distribution parameter values ​​corresponding to all off-road complex road condition recognition categories at the spatial location to obtain the pixel-level uncertainty value corresponding to the spatial location, and the pixel-level uncertainty values ​​at all spatial locations are combined to form a pixel-level uncertainty map.

[0054] Temporally filtered convolutional gated recurrent units are applied to the semantic expectation mapping to obtain a temporally consistent semantic mapping. In Example 1, the pose self-motion information of the off-road vehicle between two adjacent frames is obtained. Based on the pose self-motion information, the semantic expectation hidden state of the previous frame is spatially aligned to obtain the spatially aligned semantic expectation hidden state. The semantic expectation mapping of the current frame and the spatially aligned semantic expectation hidden state are concatenated in the channel dimension. The semantic expectation update gate tensor, semantic expectation reset gate tensor, and semantic expectation candidate hidden state tensor are calculated using convolutional gated recurrent units. Based on the semantic expectation update gate tensor, semantic expectation candidate hidden state tensor, and the spatially aligned semantic expectation hidden state of the previous frame, the semantic expectation hidden state tensor of the current frame is obtained. The semantic expectation hidden state tensor of the current frame is used as the temporally consistent semantic mapping.

[0055] Specifically, for the initial time step, the semantic expectation hidden state of the previous frame is initialized to an all-zero tensor; for non-initial time steps, the semantic expectation hidden state of the previous frame is set to the semantic expectation hidden state tensor output by the temporal filtering of the convolutional gated recurrent unit of the previous frame.

[0056] A temporally consistent uncertainty mapping is obtained by performing convolutional gated recurrent unit temporal filtering on the pixel-level uncertainty mapping. In Example 1, the pose self-motion information of the off-road vehicle between two adjacent frames is obtained. Based on the pose self-motion information, spatial alignment is performed on the uncertain hidden state of the previous frame to obtain the spatially aligned uncertain hidden state. The pixel-level uncertainty mapping of the current frame and the spatially aligned uncertain hidden state are concatenated in the channel dimension. The uncertainty update gate tensor, uncertainty reset gate tensor, and uncertainty candidate hidden state tensor are calculated using a convolutional gated recurrent unit. Based on the uncertainty update gate tensor, the uncertainty candidate hidden state tensor, and the spatially aligned uncertain hidden state of the previous frame, the uncertain hidden state tensor of the current frame is obtained. The uncertain hidden state tensor of the current frame is used as a temporally consistent uncertainty mapping.

[0057] Specifically, for the initial time step, the uncertainty hidden state of the previous frame is initialized to a tensor with all zeros; for non-initial time steps, the uncertainty hidden state of the previous frame is set to the uncertainty hidden state tensor output by the temporal filtering of the convolutional gated recurrent unit in the previous frame.

[0058] The temporally consistent semantic mapping and the temporally consistent uncertainty mapping are projected onto the bird's-eye view coordinate system to construct an uncertainty suppression mask. Projected pixels with uncertainty exceeding a preset threshold are rewritten as impassable labels to generate a safe mask for drivable areas. In this embodiment, the temporally consistent semantic mapping and the temporally consistent uncertainty mapping are projected onto the bird's-eye view coordinate system to construct an uncertainty suppression mask, including: Projecting the temporally consistent semantic map onto the bird's-eye view coordinate system yields the bird's-eye view semantic map; In Example 1, a dynamic projection mapping relationship from the image coordinate system to the bird's-eye view coordinate system is established based on the intrinsic and extrinsic calibration results of the imaging device and the local adaptive reference surface constructed by combining the prior knowledge of off-road terrain elevation. For all bird's-eye view spatial locations in the bird's-eye view coordinate system, their bird's-eye view semantic values ​​are initialized to empty sets or unknown category semantic values. The temporal consistent semantic values ​​corresponding to each spatial location and each off-road complex road condition identification category in the temporal consistent semantic mapping are projected to the corresponding bird's-eye view spatial location in the bird's-eye view coordinate system. The maximum value aggregation operator is used to aggregate the temporal consistent semantic values ​​corresponding to the same off-road complex road condition identification category projected to the same bird's-eye view spatial location to obtain the bird's-eye view semantic value corresponding to the off-road complex road condition identification category at the bird's-eye view spatial location. The bird's-eye view semantic mapping is composed of the bird's-eye view semantic values ​​corresponding to all off-road complex road condition identification categories at all bird's-eye view spatial locations.

[0059] Projecting the time-consistent uncertainty map onto the bird's-eye view coordinate system yields the bird's-eye view uncertainty map; In Example 1, the temporal consistency uncertainty map is projected onto the bird's-eye view coordinate system according to the dynamic projection mapping relationship from the image coordinate system to the bird's-eye view coordinate system. For bird's-eye view spatial locations that have not received any projection values, the corresponding bird's-eye view uncertainty value is assigned a preset global maximum uncertainty penalty constant. For bird's-eye view spatial locations that have received at least one projection value, the maximum value aggregation operator is used to aggregate the temporal consistency uncertainty values ​​that fall into the same bird's-eye view spatial location after projection, to obtain the bird's-eye view uncertainty value corresponding to the bird's-eye view spatial location, and the bird's-eye view uncertainty values ​​at all bird's-eye view spatial locations are combined to form the bird's-eye view uncertainty map.

[0060] Construct an uncertainty suppression mask based on a bird's-eye view uncertainty mapping; In Example 1, the bird's-eye view uncertainty value at the bird's-eye view spatial location is compared with a preset threshold. Bird's-eye view spatial locations with a bird's-eye view uncertainty value greater than the preset threshold are marked as uncertainty suppression locations; bird's-eye view spatial locations with a bird's-eye view uncertainty value less than or equal to the preset threshold are marked as non-uncertainty suppression locations; and the marking results of all bird's-eye view spatial locations constitute an uncertainty suppression mask.

[0061] Bird's-eye view semantic labels are determined based on the bird's-eye view semantic mapping, and an uncertainty suppression mask is invoked. Bird's-eye view semantic labels with uncertainty exceeding a preset threshold are rewritten as impassable labels to obtain a safe semantic label mapping and generate a safe mask for drivable areas.

[0062] In Example 1, for each bird's-eye view spatial location, the off-road complex road condition identification category with the largest bird's-eye view semantic value among all off-road complex road condition identification categories is determined as the bird's-eye view semantic label; for bird's-eye view spatial locations marked as uncertainty suppression locations in the uncertainty suppression mask, the corresponding bird's-eye view semantic labels are rewritten as impassable labels; for bird's-eye view spatial locations marked as non-uncertainty suppression locations in the uncertainty suppression mask, the corresponding bird's-eye view semantic labels remain unchanged, thus obtaining a safe semantic label mapping.

[0063] For each bird's-eye view location, determine whether the corresponding safety semantic label belongs to the preset set of drivable off-road complex road condition recognition categories; if it belongs, mark the bird's-eye view location as a drivable location; if it does not belong, mark the bird's-eye view location as a non-drivable location; combine the marking results of all bird's-eye view locations to form a drivable area safety mask.

[0064] The drivable path constraint information is calculated based on the safety mask of the drivable area, and the drivable path constraint information is output to the vehicle chassis control system to execute deceleration control, path replanning, or safety takeover commands.

[0065] In this embodiment, the drivable path constraint information is calculated based on the drivable area safety mask, and the drivable path constraint information is output to the vehicle chassis control system, including: Based on the drivable area safety mask, determine the drivable connectivity area safety mask that connects to the current position of the off-road vehicle; In Example 1, the vehicle projection reference position of the off-road vehicle in the bird's-eye view coordinate system is obtained. Based on the vehicle projection reference position and the drivable area safety mask, the vehicle projection reference position is used as the starting seed point. Using the spatial neighborhood connectivity analysis algorithm, topological connectivity diffusion is performed only along the bird's-eye view spatial positions marked as drivable positions in the drivable area safety mask, and cross-domain diffusion to non-drivable positions is blocked, resulting in a set of bird's-eye view spatial positions that are topologically connected to the starting seed point. The bird's-eye view spatial positions belonging to the set of bird's-eye view spatial positions are marked as connected drivable positions, and the remaining bird's-eye view spatial positions are marked as non-connected drivable positions. The marking results on all bird's-eye view spatial positions are used to form the drivable connected area safety mask.

[0066] Based on the vehicle projection reference position, vehicle orientation, and safety mask of the drivable connected area, construct the forward drivable constraint area; In Example 1, the vehicle orientation of the off-road vehicle in the current frame is obtained. A local forward constraint coordinate system is established with the vehicle projection reference position as the origin and the vehicle orientation as the longitudinal positive direction. The longitudinal projection distance and lateral projection distance of each bird's-eye view spatial position relative to the vehicle projection reference position are calculated. Bird's-eye view spatial positions whose longitudinal projection distance is within the preset forward constraint length range and whose lateral projection distance is within the preset lateral constraint width range are determined as candidate bird's-eye view spatial positions in the forward drivable constraint area. The forward drivable constraint area is composed of all candidate bird's-eye view spatial positions.

[0067] Calculate drivable path constraint information based on the drivable connected area safety mask and the forward drivable constraint area; In Example 1, for each bird's-eye view location, the minimum spatial distance from the bird's-eye view location to the non-connected drivable location is calculated to obtain the boundary clearance distance field. Based on the safety mask of the forward drivable constraint region and the drivable connected region, the forward connected drivable region is extracted, and the current steering angle of the off-road vehicle is obtained. Based on the current steering angle and the vehicle kinematic model, the expected driving curvature of the off-road vehicle is calculated, and the expected driving centerline of the vehicle in the forward connected drivable region is extracted along the expected driving curvature. The maximum value among the longitudinal projection distances corresponding to all bird's-eye view locations in the forward connected drivable region is determined as the maximum longitudinal projection distance. The distance values ​​corresponding to each bird's-eye view location on the expected driving centerline of the vehicle in the boundary clearance distance field are compared, and the minimum value is determined as the minimum boundary clearance distance. The number of connected drivable locations in the forward drivable constraint region is divided by the number of all bird's-eye view locations in the forward drivable constraint region to obtain the forward connectivity coverage. The maximum longitudinal projection distance, the minimum boundary clearance distance, and the forward connectivity coverage constitute the drivable path constraint information.

[0068] Based on the drivable path constraint information, generate the corresponding judgment results for deceleration control command, path replanning command, and safe takeover command; In Example 1, the current speed of the off-road vehicle and the current off-road surface adhesion coefficient fed back by the vehicle chassis system are obtained. Based on the maximum longitudinal projection distance, minimum boundary clearance distance, and forward connectivity coverage, the upper limit of the permissible speed is calculated. Based on the vehicle kinematic model, the maximum longitudinal projection distance is correlated with the real-time dynamic braking distance corresponding to the current off-road surface adhesion coefficient to determine the upper limit of the longitudinal braking speed. The upper limit of the clearance constraint speed is determined based on the minimum boundary clearance distance. The upper limit of the coverage constraint speed is determined based on the forward connectivity coverage and the preset maximum permissible speed. The minimum value among the preset maximum permissible speed, the upper limit of the longitudinal braking speed, the upper limit of the clearance constraint speed, and the upper limit of the coverage constraint speed is determined as the upper limit of the permissible speed. The current speed of the off-road vehicle is compared with the upper limit of the permissible speed to generate the judgment result corresponding to the deceleration control command. The maximum longitudinal projection distance, minimum boundary clearance distance, and forward connectivity coverage are compared with the path replanning threshold group and the safety takeover threshold group, respectively, to generate the judgment result corresponding to the path replanning command and the judgment result corresponding to the safety takeover command.

[0069] The drivable path constraint information is output to the vehicle chassis control system to execute deceleration control, path replanning, or safety takeover commands.

[0070] In this embodiment, an image recognition-based off-road complex road condition recognition system is used to execute an image recognition-based off-road complex road condition recognition method, including: The image acquisition and preprocessing module is used to acquire and preprocess RGB images of off-road scenes captured by an imaging device installed at the front of the off-road vehicle to obtain high dynamic range input images. The feature extraction module is used to input high dynamic range input images into the InternImage feature extraction network, which includes uncertainty-guided deformable convolutional layers. Based on the splicing tensor of the uncertainty mapping of the previous frame and the shallow features of the current frame, the module adaptively adjusts the sampling offset and weight magnitude of the deformable convolution and outputs a multi-scale geometric feature tensor. The cross-scale fusion module is used to perform cross-scale residual fusion on multi-scale geometric feature tensors. The global fusion feature tensor is generated by the evidence pyramid fusion module with multi-scale uncertainty estimates as dynamic weights. The evidence reasoning module is used to input the global fusion feature tensor into the evidence reasoning head, obtain the pixel-level evidence parameter tensor using non-negative mapping, and generate the Dirichlet distribution parameter tensor by combining the prior parameters. The uncertainty quantization and temporal filtering module is used to calculate the semantic expectation map and pixel-level uncertainty map based on the Dirichlet distribution parameter tensor, and to perform convolutional gated recurrent unit temporal filtering on the semantic expectation map and pixel-level uncertainty map to obtain temporally consistent semantic map and temporally consistent uncertainty map. The spatial projection and safety constraint generation module is used to project the temporally consistent semantic mapping and the temporally consistent uncertainty mapping onto the bird's-eye view coordinate system, construct an uncertainty suppression mask, rewrite the projected pixels with uncertainty exceeding a preset threshold as impassable labels, and generate a drivable area safety mask. The path constraint and chassis control module is used to calculate drivable path constraint information based on the safety mask of the drivable area, and output the drivable path constraint information to the vehicle chassis control system to execute deceleration control, path replanning or safety takeover commands.

[0071] Example 2: During an autonomous driving test of an off-road vehicle, an imaging device deployed at the front of the vehicle continuously acquired RGB images of the off-road scene at a fixed frame rate. The image resolution was 1280×720, and 20 frames were acquired per second. The original acquired images showed significant brightness unevenness, with approximately 32% of the pixel areas in the low-brightness range (grayscale value less than 50) and approximately 18% in the high-reflectivity range (grayscale value greater than 220). After the system performed preprocessing on the original images, the effective contrast of the high dynamic range input image was increased to 1.67 times that of the original image, the detail recovery rate in dark areas reached 91.3%, and the saturated pixel ratio in reflective areas decreased from 18% to 4.6%.

[0072] In a given processing cycle, when the system processes the t-th frame image, it inputs the high dynamic range input image into the InternImage feature extraction network. The current frame's shallow feature tensor has 64 channels and a spatial size of 320×180; the uncertainty mapping from the previous frame has a high uncertainty region (uncertainty value greater than 0.6) in space. The system resamples the uncertainty mapping to the same spatial resolution as the shallow features and then performs channel concatenation to form a concatenated tensor with 65 channels.

[0073] During deformable convolution calculations, for the boundary region between mud and puddles (approximately 9.8% of the image area), the average offset of the generated sampling offset vector is 1.37 pixels. Compared to the fixed sampling positions of regular convolution, the sampling point distribution more closely matches the real boundary. In the gravel region (approximately 12.4% of the image area), the average sampling offset is 0.52 pixels, indicating that the sampling area remains relatively stable. The weight amplitude modulation coefficient averages 0.43 in high-uncertainty regions and 1.12 in low-uncertainty regions, effectively suppressing the feature responses in high-risk areas. The output multi-scale geometric feature tensor contains four scale levels with spatial resolutions of 320×180, 160×90, 80×45, and 40×23.

[0074] In the cross-scale fusion stage, the system calculates the evidence mean after performing channel-unified mapping on features at each scale level. The evidence mean for scale 1 is 2.84, for scale 2 it is 2.12, for scale 3 it is 1.36, and for scale 4 it is 0.79, with corresponding uncertainty estimates of 0.35, 0.47, 0.73, and 1.21, respectively. After negative exponential normalization, the dynamic weights are 0.41, 0.31, 0.18, and 0.10, respectively. The average signal-to-noise ratio of the fused global feature tensor increases from 17.2 dB in the original multi-scale features to 24.6 dB.

[0075] During the evidence reasoning phase, the system performs pixel-by-pixel category mapping on the global fused feature tensor to obtain pixel-level original category response tensors. For a given spatial location, its response values ​​for the six off-road terrain categories are [1.24, 0.83, 0.12, 0.09, 0.05, 0.03]. The system calculates the response difference with neighboring pixels, obtaining a pixel-level terrain transition intensity of 0.67, which, after exponential mapping, yields a transition suppression coefficient of 0.51.

[0076] After performing the Softplus nonnegative mapping, the basic nonnegative evidence values ​​are [1.53, 1.19, 0.76, 0.74, 0.72, 0.71]. The main class competition coefficient is calculated to be 1.84 using the class competition mechanism, while the average competition coefficients for the other classes are 0.63. After incorporating the transition suppression coefficient, the final evidence parameter values ​​are 1.44 for the main class and 0.28 for the other classes. Calculating the evidence difference within a local neighborhood (5×5 window) yields a local consistency coefficient of 0.62 and a class bias coefficient of 0.71. After generating adaptive prior parameters using the prior parameters, the Dirichlet distribution parameter for the main class is 2.21, while the average for the other classes is 0.93.

[0077] During the semantic and uncertainty calculation phase, the semantic expectation value was 0.46 and the uncertainty value was 0.38. After the system performed convolutional gated recurrent unit temporal filtering on 10 consecutive frames of data, the standard deviation of the semantic expectation fluctuation decreased from 0.083 to 0.021, and the standard deviation of the uncertainty fluctuation decreased from 0.067 to 0.018, indicating a significant enhancement in temporal consistency.

[0078] During the bird's-eye view projection stage, the system maps the image space to a 200×200 bird's-eye view grid. At a given bird's-eye view location, projection values ​​from three image pixels are received, with semantic values ​​of 0.52, 0.47, and 0.49, respectively. After maximum aggregation, the result is 0.52. The corresponding uncertainty values ​​are 0.61, 0.58, and 0.63, respectively, with maximum aggregation resulting in 0.63. Since this value exceeds a preset threshold of 0.6, it is marked as an uncertainty suppression location and rewritten as an impassable location.

[0079] During the path constraint calculation phase, the system identified a total drivable area of ​​27,450 bird's-eye view grids, of which 19,820 areas (72.2%) were connected to vehicles. Within the forward constraint area (30 meters longitudinally and 10 meters laterally), there were 3,120 connected drivable locations, with a total of 4,200 candidate locations, resulting in a forward connectivity coverage rate of 0.743. The maximum longitudinal projection distance was 26.8 meters, and the minimum boundary clearance distance was 1.42 meters.

[0080] During the control decision-making phase, the vehicle's current speed is 6.2 m / s, and the system calculates that the maximum permissible speed is 4.9 m / s. Since the current speed exceeds the maximum permissible speed, a deceleration control command is triggered. At the same time, since the minimum boundary clearance distance is lower than the preset threshold of 1.5 meters, a path replanning command is activated. The safety takeover threshold is not triggered.

[0081] In the comparative experiments, the same dataset (a total of 12,000 frames, including 8,400 training frames and 1,800 test frames) was used for validation. The traditional method had an average IoU of 0.781 in the water puddle boundary region, while the method of this invention had an IoU of 0.869. In unknown obstacle recognition, the traditional method had a false positive rate of 21.7%, while the method of this invention had a false positive rate of 5.9%. In path decision safety testing, the traditional method made 31 dangerous decisions in 200 tests, while the method of this invention made only 6. The overall system average response time was 48ms, a reduction of approximately 21.3% compared to the traditional method's 61ms.

[0082] As can be seen from the above simulation process, the present invention can achieve accurate modeling of irregular boundaries in the identification of complex off-road conditions and effectively suppress the risk of erroneous decisions caused by unknown scenarios. It has good stability and security in practical applications.

[0083] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An off-road complex road recognition method based on image recognition, characterized by, include: The RGB image of the off-road scene is acquired by the imaging device installed at the front of the off-road vehicle and preprocessed to obtain a high dynamic range input image. The high dynamic range input image is input into the InternImage feature extraction network, which includes uncertainty-guided deformable convolutional layers. Based on the splicing tensor of the uncertainty mapping of the previous frame and the shallow features of the current frame, the sampling offset and weight magnitude of the deformable convolution are adaptively adjusted, and a multi-scale geometric feature tensor is output. Cross-scale residual fusion is performed on the multi-scale geometric feature tensor, and a global fused feature tensor is generated by an evidence pyramid fusion module with multi-scale uncertainty estimates as dynamic weights. The global fusion feature tensor is input into the evidence inference head, and the pixel-level evidence parameter tensor is obtained by using non-negative mapping. The Dirichlet distribution parameter tensor is then generated by combining the prior parameters. Based on the Dirichlet distribution parameter tensor, the semantic expectation map and the pixel-level uncertainty map are calculated. The semantic expectation map and the pixel-level uncertainty map are then subjected to convolutional gated recurrent unit temporal filtering to obtain the temporally consistent semantic map and the temporally consistent uncertainty map. The temporally consistent semantic mapping and the temporally consistent uncertainty mapping are projected onto the bird's-eye view coordinate system to construct an uncertainty suppression mask. Projected pixels with uncertainty exceeding a preset threshold are rewritten as impassable labels to generate a safe mask for drivable areas. The drivable path constraint information is calculated based on the safety mask of the drivable area, and the drivable path constraint information is output to the vehicle chassis control system to execute deceleration control, path replanning, or safety takeover commands.

2. The method for identifying complex off-road conditions based on image recognition according to claim 1, characterized in that, The preprocessing includes gamma correction, histogram equalization, and multi-exposure stitching based on Poisson fusion.

3. The method for identifying complex off-road conditions based on image recognition according to claim 1, characterized in that, The process of inputting a high dynamic range input image into the InternImage feature extraction network, which includes uncertainty-guided deformable convolutional layers, includes: The high dynamic range input image of frame t is input into the shallow convolutional mapping unit of the InternImage feature extraction network to obtain the shallow features of the current frame. For the initial time, the uncertainty mapping of frame t-1 is initialized to a preset no-priority tensor; for non-initial time, spatial alignment of the uncertainty mapping of frame t-1 is performed based on the self-motion information of the off-road vehicle between two adjacent frames; the spatially aligned uncertainty mapping of frame t-1 adjacent to frame t is adjusted to the same spatial resolution as the shallow features of the current frame through a scaling operator to obtain a resampled uncertainty tensor; the resampled uncertainty tensor is concatenated with the shallow features of the current frame along the channel dimension to obtain a concatenated tensor. The spliced ​​tensor is input into the corresponding hierarchical adaptation unit in the InternImage feature extraction network, and spatial and channel scale transformations are performed on the spliced ​​tensor to obtain the hierarchical guiding tensor. The hierarchical guiding tensor is input into the offset generation branch and the magnitude generation branch in the InternImage feature extraction network to obtain the sampling offset tensor and the weight magnitude tensor. The input feature tensor of the corresponding scale level is used as the convolution input of the uncertainty-guided deformable convolutional layer. The sampling offset tensor is used to adaptively adjust the position of each sampling point in the regular convolution sampling grid, and the weight amplitude tensor is used to adaptively adjust the convolution response intensity corresponding to each sampling point to obtain the output feature tensor. The output feature tensors at each scale level are aggregated to obtain a multi-scale geometric feature tensor.

4. The method for identifying complex off-road conditions based on image recognition according to claim 1, characterized in that, The cross-scale residual fusion of multi-scale geometric feature tensors includes: Perform channel-unified mapping on the output feature tensor of each scale level in the multi-scale geometric feature tensor to obtain a scale-level aligned feature tensor. The alignment feature tensors at each scale level are mapped to non-negative evidence tensors, and the corresponding multi-scale uncertainty estimate is calculated based on each non-negative evidence tensor. Based on the multi-scale uncertainty estimates corresponding to each scale level, calculate the dynamic weights corresponding to each scale level. Based on the dynamic weights corresponding to each scale level, cross-scale residual fusion is performed on the aligned feature tensors of each scale level to obtain the fused feature tensors of each scale level. The fused feature tensors at each scale level are transformed to a unified output spatial resolution and then weighted and aggregated according to the dynamic weights corresponding to each scale level to generate a global fused feature tensor.

5. The method for identifying complex off-road conditions based on image recognition according to claim 1, characterized in that, The step of inputting the global fusion feature tensor into the evidence inference head and obtaining pixel-level evidence parameter tensors using non-negative mapping includes: Input the global fusion feature tensor into the evidence inference head, perform pixel-by-pixel category mapping on the global fusion feature tensor, and obtain the pixel-level original category response tensor; The pixel-level terrain transition intensity is calculated based on the pixel-level original class response tensor, and the pixel-level transition suppression coefficient is calculated based on the pixel-level terrain transition intensity. Perform a nonnegative mapping on the pixel-level original class response tensor, and combine the class competition relationship with the pixel-level transition suppression coefficient to obtain the pixel-level evidence parameter tensor; Pixel-level local consistency coefficients and pixel-level class tendency coefficients are calculated based on pixel-level evidence parameter tensors, and pixel-level adaptive prior parameter tensors are generated by combining prior parameters. The pixel-level evidence parameter tensor and the pixel-level adaptive prior parameter tensor are superimposed position by position and category by category to generate the Dirichlet distribution parameter tensor.

6. The method for identifying complex off-road conditions based on image recognition according to claim 1, characterized in that, The calculation of semantic expectation mapping and pixel-level uncertainty mapping based on the Dirichlet distribution parameter tensor includes: Calculate the semantic expectation mapping based on the Dirichlet distribution parameter tensor; Calculate pixel-level uncertainty mapping based on the Dirichlet distribution parameter tensor; Temporally filtered convolutional gated recurrent units are applied to the semantic expectation mapping to obtain a temporally consistent semantic mapping. A temporally consistent uncertainty map is obtained by performing convolutional gated recurrent unit temporal filtering on the pixel-level uncertainty map.

7. The method for identifying complex off-road conditions based on image recognition according to claim 1, characterized in that, The step of projecting the temporally consistent semantic map and the temporally consistent uncertainty map onto the bird's-eye view coordinate system to construct an uncertainty suppression mask includes: Projecting the temporally consistent semantic map onto the bird's-eye view coordinate system yields the bird's-eye view semantic map; Projecting the time-consistent uncertainty map onto the bird's-eye view coordinate system yields the bird's-eye view uncertainty map; Construct an uncertainty suppression mask based on a bird's-eye view uncertainty mapping; Bird's-eye view semantic labels are determined based on the bird's-eye view semantic mapping, and an uncertainty suppression mask is invoked. Bird's-eye view semantic labels with uncertainty exceeding a preset threshold are rewritten as impassable labels to obtain a safe semantic label mapping and generate a safe mask for drivable areas.

8. The method for identifying complex off-road conditions based on image recognition according to claim 1, characterized in that, The step of calculating drivable path constraint information based on the drivable area safety mask and outputting the drivable path constraint information to the vehicle chassis control system includes: Based on the drivable area safety mask, determine the drivable connectivity area safety mask that connects to the current position of the off-road vehicle; Based on the vehicle projection reference position, vehicle orientation, and safety mask of the drivable connected area, construct the forward drivable constraint area; Calculate drivable path constraint information based on the drivable connected area safety mask and the forward drivable constraint area; Based on the drivable path constraint information, generate the corresponding judgment results for deceleration control command, path replanning command, and safe takeover command; The drivable path constraint information is output to the vehicle chassis control system to execute deceleration control, path replanning, or safety takeover commands.

9. An image recognition-based off-road complex road condition recognition system, used to execute the image recognition-based off-road complex road condition recognition method according to any one of claims 1-8, characterized in that, include: The image acquisition and preprocessing module is used to acquire and preprocess RGB images of off-road scenes captured by an imaging device installed at the front of the off-road vehicle to obtain high dynamic range input images. The feature extraction module is used to input high dynamic range input images into the InternImage feature extraction network, which includes uncertainty-guided deformable convolutional layers. Based on the splicing tensor of the uncertainty mapping of the previous frame and the shallow features of the current frame, the module adaptively adjusts the sampling offset and weight magnitude of the deformable convolution and outputs a multi-scale geometric feature tensor. The cross-scale fusion module is used to perform cross-scale residual fusion on multi-scale geometric feature tensors. The global fusion feature tensor is generated by the evidence pyramid fusion module with multi-scale uncertainty estimates as dynamic weights. The evidence reasoning module is used to input the global fusion feature tensor into the evidence reasoning head, obtain the pixel-level evidence parameter tensor using non-negative mapping, and generate the Dirichlet distribution parameter tensor by combining the prior parameters. The uncertainty quantization and temporal filtering module is used to calculate the semantic expectation map and pixel-level uncertainty map based on the Dirichlet distribution parameter tensor, and to perform convolutional gated recurrent unit temporal filtering on the semantic expectation map and pixel-level uncertainty map to obtain temporally consistent semantic map and temporally consistent uncertainty map. The spatial projection and safety constraint generation module is used to project the temporally consistent semantic mapping and the temporally consistent uncertainty mapping onto the bird's-eye view coordinate system, construct an uncertainty suppression mask, rewrite the projected pixels with uncertainty exceeding a preset threshold as impassable labels, and generate a drivable area safety mask. The path constraint and chassis control module is used to calculate drivable path constraint information based on the safety mask of the drivable area, and output the drivable path constraint information to the vehicle chassis control system to execute deceleration control, path replanning or safety takeover commands.

Images