A Local Path Planning Method for Undercarriage Inspection Robots Based on Cross-Modal Cooperative Learning

Abstract

This invention relates to the field of path planning technology, and in particular to a local path planning method for a vehicle undercarriage inspection robot based on cross-modal collaborative learning. The method includes preprocessing acquired multimodal perception data; constructing a multimodal fusion perception feature map based on the preprocessed data and a cross-modal collaborative modeling mechanism; mapping the multimodal fusion perception feature map to a local path navigation map based on path planning; selecting the optimal path in the local path navigation map based on a path generation mechanism; and optimizing the optimal path based on a state error estimation mechanism. It achieves functions such as deep fusion of multimodal data, dynamic mapping of spatial geometry and semantic categories, modal confidence self-adjustment, and execution feedback correction. Experimental verification shows that the method of this invention can improve the success rate of path planning tasks from 78.6% to 95.3% in typical complex vehicle undercarriage scenarios.

Description

Technical Field

[0001] This invention relates to the field of path planning technology, and in particular to a local path planning method for a vehicle undercarriage inspection robot based on cross-modal collaborative learning. Background Technology

[0002] With the widespread application of intelligent inspection equipment in security inspections, emergency patrols, and hazardous area surveys, under-vehicle inspection robots, as an automated, non-contact, and high-precision inspection method, are gradually replacing traditional manual and fixed equipment methods, becoming key equipment for improving the efficiency and safety of vehicle chassis inspections. However, the under-vehicle environment has complex characteristics such as typical spatial constraints, uneven lighting, strong surface reflections, and frequent structural obstructions, posing serious challenges to existing robot systems in terms of path planning and perception stability.

[0003] Current mainstream methods for under-vehicle path planning are mostly based on visual SLAM, LiDAR navigation, or obstacle avoidance algorithms based on geometric models. These methods often rely on a single modal input, making it difficult to maintain stable operation under multi-source interference, incomplete sensor data, or degraded perception quality. For example, LiDAR may experience distance jumps in the face of severe structural occlusion or metal reflections from under the vehicle, while visual images are prone to losing edge information in low-light or backlight environments. This deficiency in the single-modal perception architecture limits the robustness and continuity of path planning.

[0004] On the other hand, most existing path planning systems directly input perception results into traditional local planning algorithms such as DWA and Teb, ignoring the importance of semantic information in path decision-making. This leads to path selection relying too heavily on geometric distance and lacking the ability to understand the context of environmental structure and the detected target. For example, when a robot navigates in a narrow space under a vehicle, obstacle avoidance based solely on distance may prevent it from recognizing the functional differences between "vehicle edges" and "ordinary obstacles," resulting in misjudgments, path oscillations, or entering blind spots.

[0005] In summary, constructing a local path planning system that supports multimodal collaborative perception, possesses modal failure adaptation capabilities, and integrates semantic structure mapping and path optimization strategies is crucial for enabling undercarriage inspection robots to achieve stable navigation, efficient perception, and intelligent obstacle avoidance. To this end, this invention proposes a local path planning method for undercarriage inspection robots based on cross-modal collaborative learning, integrating multiple technologies such as computer vision, embedded perception, graph neural network mapping, and adaptive motion control to construct a complete, intelligent, and deployable robot path planning system for real-world scenarios. Summary of the Invention

[0006] To address the aforementioned problems, this invention provides a local path planning method for a vehicle undercarriage inspection robot based on cross-modal collaborative learning.

[0007] In a first aspect, the present invention provides a local path planning method for a vehicle undercarriage inspection robot based on cross-modal collaborative learning, which adopts the following technical solution:

[0008] A local path planning method for a vehicle undercarriage inspection robot based on cross-modal cooperative learning includes:

[0009] Acquire multimodal sensing data;

[0010] Data preprocessing is performed on the acquired multimodal sensing data;

[0011] A multimodal fusion sensing feature map is constructed based on the preprocessed data and a cross-modal collaborative modeling mechanism;

[0012] Path planning is used to map multimodal fusion perception feature maps into local path navigation maps;

[0013] Select the optimal path in the local path navigation graph based on the path generation mechanism;

[0014] Optimal path optimization is performed based on a state error estimation mechanism.

[0015] Furthermore, the acquired multimodal sensing data undergoes data preprocessing, including lidar point cloud projection and depth completion fusion. Firstly, extrinsic parameter transformation and perspective projection are performed on each 3D point, mapping all point clouds into a sparse depth map M. LiDAR , where pixel position (u k ,v k Store the corresponding location at ) For each empty pixel position (i,j) in the image, this invention searches for the nearest non-empty point (u) within a window of a certain radius r. k ,v k The interpolation is then used to complete the image, assigning a depth value to the image. With depth camera image I D Weighted fusion is represented as:

[0016]

[0017] Where, p k =(x k ,y k ,z k ) T R and t are the positions of the lidar points in the lidar coordinate system, where T is the transpose; R and t are the rigid body transformation parameters (rotation and translation) of the lidar to the camera. It represents the three-dimensional coordinates of the point in the camera coordinate system; K is the camera intrinsic parameter matrix (including focal length and principal point offset); (u k ,v k() represents the pixel coordinates of the point projected into the image space; It is the depth value of that point from the camera's perspective.

[0018] Furthermore, the data preprocessing of the acquired multimodal sensing data also includes modality normalization and numerical scaling, wherein for each modal image... Perform standardization to ensure that the mean is 0 and the standard deviation is 1, as shown below:

[0019]

[0020] Among them, I RGB ,I IR Images from an RGB camera and an infrared camera; I i (h,w) represents the pixel value of the i-th modal image at position (h,w); The mean of the entire multimodal image; H is the standard deviation of the image; W is the height of the image; ε is the width of the image; ε is the numerical stability factor to prevent the denominator from being zero. The normalized image has a standard normal distribution, ensuring that the expression weight of each modality is balanced in the subsequent network and avoiding the phenomenon of single-modality dominance.

[0021] Furthermore, the data preprocessing of the acquired multimodal sensing data also includes introducing an edge saliency enhancement operation on the normalized image to enhance the local structural features of the vehicle undercarriage structure. First, the Sobel edge gradient map is calculated, represented as:

[0022]

[0023] in, yes The partial derivative along the x-direction represents the rate of change of image intensity in the horizontal direction; x is the horizontal coordinate of the image, representing the position of the pixel in the width direction of the image. yes The partial derivative along the y-axis represents the rate of change of image intensity in the vertical direction; y is the ordinate of the image, representing the pixel's position in the image width direction, and is then weighted and fused with the original image to obtain the final image. Where λ is the marginal saliency fusion factor; E i Enhance the structural information of the edge map; This is the modal image after edge enhancement.

[0024] Furthermore, the step of constructing a multimodal fusion sensing feature map based on the preprocessed data and a cross-modal collaborative modeling mechanism includes inputting the tensor F... in ∈R H×W×5The model is constructed as an image modality branch and a structural modality branch, respectively. A two-stream neural network is used to extract deep semantic embeddings for each modality, providing a structurally distinct initial representation for subsequent modality collaboration. Specifically, the RGB image and infrared image are merged into the image modality input: F img =F in [:,:,0:4]∈R H×W×4 Where [:,:,0:4] represents the extraction of the first 4 channels, namely R, G, B, and IR; H and W represent the image height and width dimensions; the LiDAR fusion depth map is the structural modality input, and finally, two lightweight convolutional neural networks are input to extract modal features, represented as:

[0025] F′ img =CNN img (F img ),F′ geo =CNN geo (F geo )

[0026] Among them, CNN img A three-layer convolutional-normalized-ReLU network; CNN geo It is a two-layer light convolutional network; F′ img ∈R H ′×W′×C For image modal feature maps; F′ geo ∈R H′×W′×C This is a structural modal feature map.

[0027] Furthermore, the construction of a multimodal fusion perceptual feature map based on preprocessed data and a cross-modal collaborative modeling mechanism also includes introducing a cross-modal attention-guided enhancement mechanism. This mechanism leverages the information advantages of the image modality to guide the structural modality in strengthening its spatial representation, thus constructing a more semantically consistent fusion representation. Next, an attention matrix is ​​constructed, and the attention weight matrix is ​​applied to the image modality feature values. Finally, a channel fusion residual strategy is used to fuse the information from the two structural modalities. The cross-modal guidance mechanism is expressed as follows:

[0028] Q = φ q (F′ img ),K=φ k (F′ geo V=φ v (F′ img )

[0029] Where, φ q () represents the query vector; φ k () represents the key vector; φ v () represents the value vector, and the three elements are the features obtained by linearly mapping query, key, and value, respectively.

[0030] Furthermore, the construction of a multimodal fusion-aware feature map based on preprocessed data and a cross-modal collaborative modeling mechanism also includes introducing a modality confidence-aware mechanism to dynamically evaluate the confidence of the two modalities and adjust the final output features proportionally to ensure that the dominant features originate from reliable modal paths. Specifically, firstly, global average pooling (GAP) is performed on the output features of the two paths respectively: g img =GAP(F′) img ),g geo =GAP(F′) geo Then, the confidence level is obtained through a small fully connected layer and Sigmoid activation: w img =σ(W1·g img ),w geo =σ(W2·g geo Where W1 and W2 are the corresponding weight matrices, and the weighting coefficients are obtained after normalization: The final fused output features are:

[0031] Furthermore, the path planning-based mapping of the multimodal fusion sensing feature map into a local path navigation map includes mapping the multimodal feature map F... multi Design a lightweight semantic segmentation head that outputs a category probability map P. sem ∈R H′×W′×K Next, the category with the highest probability for each pixel is selected as the semantic label to form a semantic mask. in, This indicates that the position with the highest probability in class k is selected as the predicted class, where k∈{1,...,K}; M sem ∈R H′·W′ The final pixel-level semantic annotation map is based on the obtained semantic label map M. sem ∈R H′×W′ and semantic probability graph P sem ∈R H′×W′×K By integrating structural category and classification confidence information, a highly expressive local access cost graph C is constructed. nav ∈R H′×W′ Furthermore, a cost construction function based on category probability weighting is introduced, which integrates semantic category cost and confidence information to calculate the passage cost for each pixel.

[0032] Furthermore, the selection of the optimal path in the local path navigation graph based on the path generation mechanism includes first performing trajectory simulation guided by multimodal perception. Let the robot's current position be s0 = (x0, y0, θ0), where x0 and y0 represent the robot's two-dimensional coordinates in the cost graph plane; θ0 represents the robot's current orientation angle in radians; and multiple combinations of linear and angular velocities (v) are sampled. i ,ω iFor each pair of combinations, a trajectory simulation is performed for a fixed time T, with a simulation step size of Δt, to obtain a trajectory sequence; each trajectory is mapped onto the cost map plane and combined with the passage cost map C. nav With semantic mask graph M sem Rate the path:

[0033]

[0034] Among them, C nav (τ i ) represents the transit value at the current location; M sem (τ i ) represents the semantic value of the current location; I obs G(v) is the obstacle penalty function; λ1 and λ2 are the weight coefficients; G(v) i ,ω i () represents the trajectory score.

[0035] Furthermore, the optimal path optimization based on the state error estimation mechanism includes introducing a state error estimation mechanism to detect whether the trajectory deviation has exceeded the control tolerance range and to define the current actual state of the robot. in Indicate the actual execution position and orientation at step t; construct the position offset error Δd. t With orientation offset error Δθ t When Δd appears t >ε d or Δθ t >ε θ If this continues for more than K steps, the current path is considered invalid, and a refactoring process begins. Furthermore, control instruction deviation constraints are introduced to determine the state stability of the underlying execution system, define the actual instructions fed back by the current actuator, and calculate the control deviation metric. If Δu t >ε u If the mismatch continues for M steps, it is considered a control mismatch, where M is the maximum allowed number of consecutive mismatch steps. Once any of the following conditions is triggered: ①Δd t >ε d or Δθ t >ε θ Continue for K steps; ②Δu t >ε u If the process continues for M steps, then the cost graph C is invoked. nav With semantic mask graph M sem This triggers a replanning of the trajectory.

[0036] Secondly, a local path planning system for a vehicle undercarriage inspection robot based on cross-modal collaborative learning includes:

[0037] The data acquisition module is configured to acquire multimodal sensing data;

[0038] The preprocessing module is configured to perform data preprocessing on the acquired multimodal sensing data;

[0039] The fusion perception module is configured to construct a multimodal fusion perception feature map based on preprocessed data and a cross-modal collaborative modeling mechanism.

[0040] The local path planning module is configured to map the multimodal fusion perception feature map into a local path navigation map based on path planning;

[0041] The optimal path module is configured to select the optimal path in the local path navigation graph based on the path generation mechanism.

[0042] The optimization module is configured to perform optimal path optimization based on the state error estimation mechanism.

[0043] Thirdly, the present invention provides a computer-readable storage medium storing a plurality of instructions adapted for loading and execution by a processor of a terminal device of the aforementioned method for local path planning of a vehicle undercarriage inspection robot based on cross-modal collaborative learning.

[0044] Fourthly, the present invention provides a terminal device, including a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, the instructions being adapted to be loaded and executed by the processor to provide a local path planning method for a vehicle undercarriage inspection robot based on cross-modal collaborative learning.

[0045] In summary, the present invention has the following beneficial technical effects:

[0046] Compared to existing path planning systems that rely on single-modal perception and ignore semantic factors in their navigation methods, this invention achieves significant breakthroughs in structural modeling, modal collaboration, interpretability, and practical control response.

[0047] Through modular design, this system achieves deep fusion of multimodal data, dynamic mapping of spatial geometry and semantic categories, modal confidence self-adjustment, and execution feedback correction. Experimental verification shows that the method of this invention can improve the success rate of path planning tasks from 78.6% to 95.3% in typical complex under-vehicle scenarios, and control the average inference time of the diagnosis and feedback mechanism to 0.48 seconds, meeting the edge deployment requirements of under-vehicle robots.

[0048] Furthermore, through semantic structure mapping and interpretability output mechanism, a causal explanation consistency score of up to 0.91 was achieved, which significantly enhanced the system's ability to explain obstacle recognition, path deviation sources and regional planning errors. It is applicable to intelligent navigation tasks in various vehicle under-vehicle scenarios and has high practical promotion value and system deployment capability. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of a local path planning method for a vehicle under-vehicle inspection robot based on cross-modal collaborative learning, according to Embodiment 1 of the present invention.

[0050] Figure 2 This is a comparison chart of task execution success rate and explanatory factor score in Embodiment 1 of the present invention.

[0051] Figure 3 This is a comparison chart of the false alarm rate and false positive rate in Embodiment 1 of the present invention.

[0052] Figure 4 This is a comparison chart of path fitting error and inference time in Embodiment 1 of the present invention.

[0053] Figure 5 This is a normalized radar chart of multidimensional performance indicators in Embodiment 1 of the present invention. Detailed Implementation

[0054] The present invention will be further described in detail below with reference to the accompanying drawings.

[0055] Example 1

[0056] Reference Figure 1 This embodiment presents a local path planning method for a vehicle undercarriage inspection robot based on cross-modal collaborative learning. The aim is to address the following key issues in existing path planning technologies for vehicle undercarriage inspection robots in confined, unstructured, and highly reflective environments: First, relying solely on visual or radar information makes it difficult to reliably extract environmental features in the face of occlusion, thermal interference, and low light, leading to path failure or replanning. Second, the lack of a collaborative mechanism between the perception stage and path generation means that compensation or replanning cannot be performed at the path layer after local perception errors occur. Third, existing multimodal systems lack a modal confidence feedback mechanism, cannot dynamically adjust input weights based on perception quality, and cannot cope with sudden scenarios such as infrared failure and depth map jumps. Fourth, the path is generated only based on the obstacle position or grid cost, failing to understand semantic concepts such as "vehicle body", "structural components" and "road surface material", thus failing to achieve refined navigation; fifth, although some deep reinforcement learning and 3D mapping methods have accuracy, their structures are lengthy and their real-time performance is poor, making them difficult to implement on mobile robot platforms. Therefore, this invention solves the problem of path planning failure in complex environments under vehicles by constructing a cross-modal feature collaborative learning network + semantic enhancement path mapping mechanism + modal confidence feedback control module + lightweight path control algorithm that can be deployed at the edge. It achieves stable, robust and interpretable path planning output in dynamic, complex and variable scenarios, and has high engineering application prospects and cross-domain expansion value.

[0057] Specifically, it includes the following:

[0058] (1) Multimodal sensing data acquisition and standardization preprocessing module

[0059] In real-world applications of under-vehicle inspection robots, the detection systems are typically deployed in extremely harsh environments, such as enclosed and confined under-vehicle cavities, highly reflective metal structures, surfaces covered in oil, or low-light and occluded areas. To achieve stable and high-precision local path planning, the robot system needs comprehensive perception capabilities of this complex environment. Therefore, it is essential to rely on the collaborative work of multimodal sensors: RGB cameras capture visible texture information, infrared cameras supplement the identification of heat source areas, depth cameras provide spatial contours, and LiDAR is used for obstacle geometry localization. While these modalities are complementary, they exhibit high heterogeneity in spatial alignment, data density, brightness response scale, and edge structure information due to different sampling mechanisms.

[0060] If these modal data are directly input into the downstream network without unified preprocessing, several problems will arise: spatial misalignment between modalities will cause confusion in target recognition; excessive differences in the dynamic range of different modalities will lead to information imbalance; sparse LiDAR information will lack details in key areas; and blurred obstacle boundaries will weaken the structural recognition ability of subsequent models. Therefore, before path planning, a complete perception preprocessing mechanism must be introduced to perform spatial transformation, depth densification, brightness normalization, and edge enhancement on multi-source information. This will construct a perception input tensor with unified scale, aligned space, salient structure, and modal coordination, providing a solid data foundation for subsequent cross-modal fusion and dynamic path reasoning.

[0061] 1) Fusion of LiDAR point cloud projection and depth completion

[0062] To map sparse lidar point cloud data to the pixel space of an RGB image for alignment with other modal images, this invention first performs extrinsic transformation and perspective projection on each 3D point:

[0063]

[0064] Where, p k =(x k ,y k ,z k ) T R and t are the positions of the lidar points in the lidar coordinate system, where T is the transpose; R and t are the rigid body transformation parameters (rotation and translation) of the lidar to the camera. It represents the three-dimensional coordinates of the point in the camera coordinate system; K is the camera intrinsic parameter matrix (including focal length and principal point offset); (u k ,v k () represents the pixel coordinates of the point projected into the image space; It is the depth value of that point from the camera's perspective.

[0065] Through the above transformation, all point clouds can be mapped to a sparse depth map M. LiDAR , where pixel position (u k ,v k Store the corresponding location at ) Because the point cloud is sparse, the missing pixels will be empty. To obtain a dense depth map, this invention designs a dense completion algorithm based on nearest neighbor interpolation, with the completion logic as follows:

[0066] For each empty pixel position (i,j) in the image, this invention searches for the nearest non-empty point (u) within a window of a certain radius r. k ,v k ), and assign its depth:

[0067]

[0068] Where r is the interpolation radius, controlling the interpolation smoothness; ||(i,j)-(u k ,v k )‖2 represents the Euclidean distance between the two points; if‖(i,j)-(u k ,v k )‖2≤r means if point (u k ,v k If a point is less than or equal to the radius r of a point (i,j), it is considered a usable nearest neighbor and its depth value can be used to fill in the missing points (i,j); if there are no points in the neighborhood, the value is kept as 0.

[0069] Subsequently, this invention completes the interpolation image. With depth camera image I D Perform weighted fusion:

[0070]

[0071] Where α∈[0,1] is the fusion coefficient; This is a depth map with structural integrity after fusion. This result not only improves the robustness of the depth map in reflective areas, but also enhances the spatial resolution of scene edge structures.

[0072] 2) Modal normalization and numerical scaling

[0073] The numerical dynamic range of multimodal images varies greatly, and directly feeding them into the network will cause learning bias. Therefore, for each modal image... Perform standardization to make the mean 0 and the standard deviation 1, using the following formula:

[0074]

[0075] Among them, I RGB,I IR Images from an RGB camera and an infrared camera; I i (h,w) represents the pixel value of the i-th modal image at position (h,w); The mean of the entire multimodal image; H is the standard deviation of the image; W is the height of the image; ε is the width of the image; ε is the numerical stability factor to prevent the denominator from being zero. The normalized image has a standard normal distribution, ensuring that the expression weight of each modality is balanced in the subsequent network and avoiding the phenomenon of single-modality dominance.

[0076] 3) Structural edge reinforcement

[0077] Because the undercarriage structure contains numerous details such as slits, grooves, contour seams, and protruding bolts, its visual response is weak and easily erased by intermodal differences. To enhance these local structures, this invention introduces an edge saliency enhancement operation on the normalized image.

[0078] First, calculate the Sobel edge gradient map:

[0079]

[0080] in, yes The partial derivative along the x-direction represents the rate of change of image intensity in the horizontal direction (x-axis); x is the horizontal coordinate of the image, representing the position of the pixel in the width direction of the image. yes The partial derivative along the y-direction represents the rate of change of image intensity in the vertical direction (y-axis); y is the vertical coordinate of the image, representing the position of the pixel in the width direction of the image.

[0081] Then perform a weighted fusion with the original image:

[0082]

[0083] Where λ is the marginal saliency fusion factor; E i Enhance the structural information of the edge map; This is the modal image after edge enhancement; this step can significantly improve the subsequent network's ability to recognize obstacle boundaries and connecting components.

[0084] 4) Multimodal input tensor splicing and output construction

[0085] All processed modal images Concatenated into a unified input tensor:

[0086]

[0087] The tensor contains: three channels of RGB; one channel of infrared; and one channel of depth-laser fusion image.

[0088] Final output F in ∈R H×W×5 It has the following characteristics: all modalities have completed geometric mapping and pixel-level registration; all modalities are of standard normal distribution, which helps the model to train stably; edge information is significant, which facilitates structure discrimination; modality fusion is friendly: it is convenient for subsequent modules to build modality attention mechanisms and semantic alignment representations.

[0089] (2) Cross-modal collaborative sensing feature modeling module

[0090] When performing local path planning tasks, under-vehicle inspection robots face complex perception scenarios, including drastic changes in lighting, highly reflective materials, narrow structural gaps, and low field-of-view occlusion. Although the previous module provides an input tensor F with uniform size and aligned modes... in ∈R H×W×5 However, to achieve a stable and robust path planning perception foundation, it is necessary to further break down modal barriers at the feature level and construct a fusion representation that is sensitive to fine-grained structural information and has modal redundancy tolerance. Therefore, this module designs a cross-modal collaborative modeling mechanism based on dual-stream feature extraction + modal guidance enhancement + confidence dynamic fusion to construct a perception feature map that stably represents the semantics and depth contours of the complex structure under the vehicle, providing high-quality input for semantic map generation and path cost mapping.

[0091] 1) Design of a two-stream feature extraction network

[0092] In undercarriage detection, the system needs to process two types of perception data simultaneously: image modalities (RGB images, infrared images) and structural modalities (depth maps, LiDAR projection maps). These two modalities possess different information densities and physical properties. For example, RGB provides high-frequency texture details, infrared images can perceive heat source outlines, while depth maps provide geometric location information, and LiDAR point clouds reflect spatial structure. If a unified backbone network is used to directly process all channels, semantic conflicts and dynamic weight changes between modalities will make it difficult for the model to effectively focus on useful features, and it may even be misled by interfering modalities. Therefore, to avoid feature contamination caused by direct fusion of different modalities, this invention uses the input tensor F... in ∈R H×W×5 The model is constructed into image modality branches and structural modality branches, respectively. Deep semantic embeddings for each modality are extracted using a two-stream neural network, providing a structurally distinct initial representation for subsequent modality collaboration.

[0093] RGB image (3 channels) + infrared image (1 channel) are merged into an image modality input:

[0094] F img =F in[:,:,0:4]∈R H×W×4

[0095] Where [:,:,0:4] indicates that the first 4 channels (i.e., R, G, B, IR) are extracted; H and W represent the height and width dimensions of the image.

[0096] The lidar fusion depth map (1 channel) is the structural mode input:

[0097] F geo =F in [:,:,4:5]∈R H×W×1

[0098] Here, [:,:,4:5] indicates that the 4th channel (Depth) is extracted.

[0099] Two lightweight convolutional neural networks are input separately to extract modal features:

[0100] F′ img =CNN img (F img ),F′ geo =CNN geo (F geo )

[0101] Among them, CNN img It consists of a three-layer convolution-normalization-ReLU module, used to extract edge texture and color structure features from RGB+IR; CNN geo It is a two-layer lightweight convolutional network used to extract structural representations such as depth contours and distance boundaries.

[0102] ;F′ img ∈R H′×W′×C For image modal feature maps, focus on texture, heat sources, and edge cues; F′ geo ∈R H′×W′×C This is a structural modal feature map, focusing on obstacle contours and spatial convexity variations; H′=H / 4, W′=W / 4 are the dimensions after downsampling; C is the number of channels. This step ensures that stable high-dimensional representations of different modal features are extracted separately without coupling interference, and provides the basic structure for subsequent attention fusion mechanisms.

[0103] 2) Image-guided structural modal cross-modal enhancement

[0104] While the previous dual-stream features preserved modal independence, semantic shifts existed between the two feature spaces. Structural modalities often struggled to represent complete boundaries and context, especially in areas with severe under-vehicle illumination occlusion or sparse radar point clouds. To enhance intermodal collaborative representation, this invention introduces a "cross-modal attention-guided enhancement mechanism." Leveraging the information advantages of image modalities, this mechanism guides structural modalities to strengthen their spatial representation, constructing a more semantically consistent fusion representation.

[0105] The cross-modal guidance mechanism is designed as follows:

[0106] Q = φ q (F′ img ),K=φ k (F′ geo V=φ v (F′ img )

[0107] Where, φ q () represents the query vector; φ k () represents the key vector; φ v () represents the value vector

[0108] These three elements represent the features obtained through linear mapping of query, key, and value. Next, an attention matrix is ​​constructed.

[0109]

[0110] Where · represents the dot product operation; d represents the embedding dimension; The scaling factor prevents numerical explosion; softmax normalizes along the second dimension.

[0111] Then, attention will be paid to the effect of the weight matrix on the image modal eigenvalues:

[0112] F att =A·V

[0113] Then, a channel fusion residual strategy is used to fuse the information from the two structural modes:

[0114] F fused =Conv 1×1 (F′ geo +F att )

[0115] This step can effectively improve the expressive power of structural modalities in weakly textured regions and enhance their structural consistency with image modalities.

[0116] 3) Modal confidence-aware fusion

[0117] In a real-world vehicle undercarriage environment, some modes may fail due to occlusion, water stains, high-temperature reflections, or other reasons. To enhance the system's fault tolerance to abnormal modes, this module introduces a "modal confidence perception mechanism" to dynamically evaluate the confidence of two types of modes and adjust the final output features proportionally to ensure that the dominant features originate from reliable modal paths.

[0118] First, global average pooling (GAP) is performed on the output features of the two paths respectively:

[0119] g img =GAP(F′) img ),g geo =GAP(F′) geo )

[0120] The confidence level is then obtained by applying a small fully connected layer and sigmoid activation:

[0121] w img =σ(W1·g img ),w geo =σ(W2·g geo )

[0122] Where W1 and W2 are the corresponding weight matrices.

[0123] The weighting coefficients are obtained after normalization:

[0124]

[0125] The final fused output features are:

[0126]

[0127] Final output feature F multi ∈R H′×W′×C It possesses image texture guidance capabilities, deep geometric robustness, and the ability to dynamically adapt to modal degradation, providing multimodal joint feature support with consistent spatial representation, semantic accuracy, and strong fault tolerance for the next module, "semantic map construction and path cost encoding".

[0128] (3) Semantic map construction and path cost encoding module

[0129] The undercarriage area is not only structurally complex and has a narrow path, but also contains a large number of heterogeneous surface materials (such as metal plates, asphalt, and brick surfaces), and various obstacles (such as oil pipes, supports, and protruding bolts) are densely distributed within it. Therefore, after completing multimodal fusion, it is difficult to construct a navigation structure adapted to path planning based solely on the original features. This module aims to integrate the multimodal fused features F multiThe mapping is transformed into a local path map with high semantic resolution, clearly depicting the passable areas, obstacles and vehicle structure locations. A semantic value encoding strategy oriented towards path planning is introduced to dynamically construct a local navigation map, providing an environmental modeling foundation for downstream trajectory generation.

[0130] 1) Region discrimination segmentation based on semantic features

[0131] To achieve precise discrimination of "ground type" and "obstacle type" in the scene, this invention is based on the multimodal feature map F output by module (2). multi Design a lightweight semantic segmentation head that outputs a category probability map P. sem ∈R H′×W′×K ;

[0132] P sem =Softmax(Conv 1×1 (F multi ))

[0133] Where K represents the total number of semantic categories (e.g., cement road surface, metal base plate, brick surface, obstacle, car, truck, etc.); Conv 1×1 () represents a channel compression convolutional layer used to compress the feature dimension C into the number of semantic categories K; after Softmax normalization, each pixel corresponds to a semantic category probability distribution; P sem (i,j,k) represents the probability that pixel (i,j) belongs to class k.

[0134] Next, the category with the highest probability for each pixel is selected as the semantic label, forming a semantic mask image:

[0135]

[0136] in, This indicates that the position with the highest probability in class k is selected as the predicted class, where k∈{1,...,K}; M sem ∈R H′·W′ The final pixel-level semantic annotation map is generated, with each pixel being an integer class label used to locate environmental components and eliminate impassable areas. This segmentation map realizes spatial region semantic parsing, providing a semantic basis for subsequent path cost estimation.

[0137] 2) Construction of local navigation cost map

[0138] In practical vehicle undercarriage inspection tasks, even with pixel-level semantic annotations, the system still needs to further convert these into "traffic cost" expressions before they can be used for path optimization calculations. Traditional path planning methods rely on simple obstacle detection (such as obstacle grid maps from LiDAR) to generate traffic cost maps, but they ignore the impact of semantic information in the scene on path costs. For example, "wheel covers" and "metal plates" are similar in geometry, but the former is impassable while the latter can be passed slowly; "asphalt" has higher anti-skid properties and higher priority than "brick seams." Therefore, using only geometric features to build maps is insufficient to fully express complex ground materials, structural states, and traffic preferences, limiting the intelligence and adaptability of path generation strategies.

[0139] This module is based on the obtained semantic label graph M sem ∈R H′×W′ and semantic probability graph P sem ∈R H′×W′×K By integrating structural category and classification confidence information, a highly expressive local access cost graph C is constructed. nav ∈R H′×W′ .

[0140] A cost construction function based on class probability weighting is introduced, which integrates semantic class cost and confidence information to calculate the passage cost for each pixel. The cost graph is defined as follows:

[0141]

[0142] Where (i,j) are the spatial pixel coordinates on the image; ω k The cost of the k-th class (e.g., impassable areas will be assigned higher weights); P sem The probability that pixel (i,j,k) belongs to category k; C nav (i,j) represents the passage cost of the pixel, with a value range of [0,1]. The larger the value, the higher the passage cost.

[0143] (4) Local path generation and control execution module

[0144] After completing the semantic map construction and access cost map generation in module (3), the system obtains the accessibility assessment of each location in the current undercarriage scene (through cost map C). nav ) and regional semantic attributes (semantic mask map M) semAfter extraction, however, map information alone cannot directly guide robot movement. These static environmental features must be further transformed into continuous, executable path trajectories, and speed control signals must be output for the robot's actuators to invoke. Due to the limited space under the vehicle, dense structure, and irregular obstacles, relying solely on traditional DWA trajectory sampling strategies can easily lead to non-smooth paths or local traps. Therefore, this invention designs a path generation mechanism based on kinematic trajectory simulation and multimodal perception cost joint scoring. This mechanism starts from the robot's current position, simulates several candidate trajectories, scores and selects the optimal path in the semantic map and cost map, and outputs the first frame motion command to the underlying controller.

[0145] 1) Multimodal perception-guided trajectory simulation and scoring mechanism

[0146] Without motion trajectory simulation, directly predicting paths using gradient methods or heuristics makes it difficult for the system to assess the interaction between the trajectory and complex obstacle structures, easily leading to paths that "seem reasonable but actually collide." Trajectory simulation methods, on the other hand, use forward prediction to assess path feasibility in advance under a realistic motion model, a crucial step in generating robust paths.

[0147] Let the robot's current position be s0 = (x0, y0, θ0), where x0 and y0 represent the two-dimensional coordinates of the robot in the cost graph plane; θ0 represents the robot's current orientation angle in radians, defined as the angle between the robot's forward direction and the horizontal axis.

[0148] This invention samples multiple combinations of linear velocity and angular velocity (v) i ,ω i For each pair of combinations, a trajectory simulation is performed for a fixed time T, with a simulation step size of Δt, resulting in the following trajectory sequence:

[0149]

[0150] The trajectory points are calculated as follows:

[0151]

[0152] Among them, v i ω is the sampled value of the linear velocity. i θ0+ω represents the angular velocity sample value; Δt represents the simulation time per step; i tΔt represents the orientation angle of the trajectory at time t; τ i This indicates that control pair (v) will be executed from the current state. i ,ω i The trajectory path obtained after simulation.

[0153] Each trajectory will be mapped onto the cost map plane, combined with the passage cost map C output in module (3). navWith semantic mask graph M sem Rate the path:

[0154]

[0155] Among them, C nav (τ i ) represents the transit value at the current location; M sem (τ i ) represents the semantic value of the current location; I obs G(v) is the obstacle penalty function, which outputs 1 when the current position is an obstacle and 0 otherwise; λ1 and λ2 are weight coefficients that adjust the degree of influence between passage cost and semantic penalty; i ,ω i () represents the trajectory score; the smaller the score, the better.

[0156] 2) Trajectory optimization and low-level control command output

[0157] Among all sampled trajectories, select the one with the score G(v) i ,ω i The path with the smallest speed is taken as the current execution path, and the optimal speed pair is denoted as . Its corresponding trajectory is

[0158]

[0159] in, This indicates that the smallest execution path is selected.

[0160] Use its first frame control value as the output command:

[0161]

[0162] Among them, u cmd The linear and angular velocity control pair of the robot output at the current time step; τ * Optimal trajectory path sequence. Control commands u output by module (4) cmd It is a direct operating instruction used by the underlying motor controller, and also outputs the path sequence τ. * The input module (5) performs offset monitoring and trajectory replanning judgment. If the system detects a high trajectory deviation or a surge in real-time value, it triggers module (5) to perform emergency path replanning to build a complete closed-loop control system.

[0163] (5) Dynamic feedback mechanism and path reconstruction module

[0164] In the undercarriage inspection task, the robot needs to accurately execute the path trajectory τ generated by module (4) in various complex scenarios. * With control sequence u cmdThe robot is designed to perform obstacle avoidance and chassis area sampling tasks. However, due to the extremely complex environment under the vehicle, such as visual drift caused by reflections from metal surfaces, wheel speed changes caused by ramps, and slippage caused by non-rigid terrain (such as soft asphalt), the robot frequently deviates from its planned trajectory in its actual driving state. Without a feedback mechanism to continuously monitor the trajectory execution status, the original path will quickly become invalid, potentially leading to collisions or detection failures.

[0165] This module is based on the deviation perception mechanism between the planned path and the execution state, combined with the control command stability analysis mechanism, to dynamically determine whether path reconstruction needs to be triggered, and calls modules (3) and (4) to generate a new path, so as to realize the improvement of system-level closed-loop control and robust adaptive capability.

[0166] 1) Trajectory state consistency monitoring and deviation judgment mechanism

[0167] Trajectory output by module (4) This describes the ideal motion state of the robot within a local planning period T (i.e., (x...)). t ,y t ,θ t (represents the desired position and attitude angle at step t). However, due to the accumulation of execution errors or environmental disturbances, the actual driving trajectory often deviates from the target point. Therefore, a state error estimation mechanism must be introduced to detect whether the trajectory deviation has exceeded the control tolerance range.

[0168] Define the current actual state of the robot obtained through odometry as follows:

[0169]

[0170] in, This indicates the actual execution position and orientation at step t;

[0171] Construction position offset error Δd t With orientation offset error Δθ t for

[0172]

[0173] When Δd appears t >ε d or Δθ t >ε θ If more than K steps continue to occur, the current path is considered invalid and a reconstruction process is required.

[0174] Where, Δd t Spatial offset error; Δθ t For angular offset error; ε d ε is the maximum allowable position offset threshold; θK represents the maximum permissible orientation offset threshold; K is the step size of the continuous permissible deviation.

[0175] 2) Control stability monitoring and modal fault judgment mechanism

[0176] Even if the positional deviation is not significant, if the actual execution effect of the control command deviates greatly from the planned value (such as slippage, speed loss, or wheel step loss), path planning may still fail. Therefore, it is necessary to further introduce control command deviation constraints to determine the state stability of the underlying execution system.

[0177] The planning and control sequence output by module (4) is as follows:

[0178]

[0179] Among them, v t ,ω t Let be the linear velocity and angular velocity at step t.

[0180] Define the actual instruction fed back by the current executor as:

[0181]

[0182] in, The feedback value for step t;

[0183] The control deviation metric is:

[0184]

[0185] Where, α v ,α ω To control the bias weighting parameter, Δu t To control deviation indicators.

[0186] If Δu t >ε u If the mismatch continues for M steps, it is considered a control mismatch, where M is the maximum allowed number of consecutive mismatch steps.

[0187] 3) Path reconstruction and module-level closed-loop feedback process

[0188] Once any of the following conditions are triggered: ①Δd t >ε d or Δθ t >ε θ Continue for K steps; ②Δu t >ε u If the process continues for M steps, then the cost graph C output by module (3) will be called. nav With semantic mask graph M sem The trigger module (4) replans the trajectory.

[0189] Clear the current control command cache:

[0190]

[0191] Update the cost graph at the current location:

[0192]

[0193] Path is being generated:

[0194]

[0195] Where P() is the trajectory calculation function of the path generation module (4); τ *new For the newly generated trajectory sequence; The updated control commands; This represents the current robot state; all newly generated trajectories will be re-entered into the lower-level control module for tracking.

[0196] Finally, module (5) outputs the updated trajectory sequence τ. *new With control command flow This forms an execution feedback closed loop for module (4). Through the position state perception + control deviation monitoring + path regeneration mechanism, adaptive closed-loop control of various execution disturbances and perception anomalies in complex scenarios under the vehicle is realized, which greatly enhances the system's execution stability, fault tolerance and intelligent autonomy.

[0197] Experimental verification

[0198] To comprehensively verify the accuracy, perception robustness, and execution stability of the present invention in real-world complex vehicle undercarriage scenarios, a multimodal perception and navigation dataset of typical vehicles (sedans, SUVs, light trucks, and buses) in a closed undercarriage environment was constructed. This dataset contains 540 scene samples, distributed across 10 different lighting conditions, surface material types, and obstacle distribution configurations, covering various operating conditions: RGB and infrared images from low-light, reflective, and dirt-occluded environments; depth maps and sparse LiDAR point clouds reflecting the undercarriage structure and spatial unevenness; each data set is labeled with "structural category tags" (e.g., bolts, oil pipes, chassis edges), "surface material type" (asphalt, brick, composite board), and "obstacle occlusion level"; the robot's execution trajectory is recorded, and position drift, target deviation, and trajectory termination reasons are monitored. Based on the navigation task execution results, four execution status labels are set: ① Path task completed (valid trajectory); ② Modal misperception leading to deviation; ③ Obstacle collision aborts; ④ Perception degradation failure.

[0199] To evaluate the performance of the method, the following five mainstream path perception and planning methods are compared: ① ResNet-LSTM: based on image texture feature extraction + LSTM controller, lacking structure perception capability; ② Mono-LiDAR-DWA: only based on LiDAR grid map and classic dynamic window algorithm for path planning; ③ RGB-Only YOLO-DWA: uses YOLO to extract obstacle regions and combines cost graph for path planning; ④ GraphFusion-DWA: uses graph neural network to model semantic structure relationships and embeds them into DWA path calculation; ⑤ The method of this invention: multimodal collaborative perception + two-stream semantic modeling + dynamic semantic navigation cost + adaptive trajectory feedback replanning.

[0200] All methods were evaluated on the same test set, and the comparison metrics included: task execution success rate (ACC), i.e., the proportion of complete planning and navigation execution; false alarm rate (MR), i.e., the proportion of path failures caused by perception errors; false alarm rate (FAR), i.e., the proportion of path termination caused by misjudging obstacles / ground; mean square error of path deviation (Path-Dev(cm), reflecting the accuracy of the trajectory fitting to the semantic structure of the ground; average time (Time(s)) for a single path generation and control execution; and Interp-Score (0-1) for consistency between the explanatory factor and obstacle / region labeling.

[0201] Table 1 Comparison of data from different methods under five major indicators

[0202]

[0203] Traditional models such as ResNet-LSTM and monomodal LiDAR-DWA fail to provide structured semantic factor interpretation and modal fault tolerance mechanisms, thus being rated as inapplicable (N / A) in the "Interp-Score" dimension. Specifically, ResNet-LSTM constructs path control sequences solely from image information, and its output path relies only on texture regions, failing to incorporate structural semantic categories for causal interpretation. Mono-LiDAR-DWA, on the other hand, relies entirely on sparse point cloud information to generate raster images, resulting in a high probability of obstacle recognition failure in occluded or reflective environments. It lacks perceptual confidence modulation and modal redundancy mechanisms, and cannot provide any semantic judgment or explanation for navigation failures.

[0204] Because the six metrics differ significantly in their measurement methods and cannot be directly displayed in the radar chart, this invention performs the following normalization processing in the visualization metric evaluation: For positive metrics (ACC, Interp-Score), the original values ​​are retained; for negative metrics (MR, FAR, Path-Dev, Time), a "1-original value normalization" method is used to convert them into a "higher is better" form, unifying the numerical semantics of the radar chart. Considering that many metrics of ResNet-LSTM are at their lowest levels, this invention sets a "biased compression coefficient ε = 0.05" to compress all metric values ​​to the range [ε, 1-ε], ensuring that they retain visible structure in the radar chart and avoiding single-point or linear degradation effects.

[0205] The experimental results are shown in Table 1. Figure 2 , Figure 3 , Figure 4 , Figure 5 As shown, traditional methods have significant limitations in complex real-world vehicle environments. ResNet-LSTM, lacking depth modeling capabilities, achieves only a 78.6% navigation success rate in areas with numerous obstacles, with a path drift mean square error as high as 10.4 cm, making it the method with the highest false positive rate and deviation. Mono-LiDAR-DWA, while possessing some navigation capability in clean spaces, performs poorly in areas with occlusion and blurred edges, with false positive and false negative rates as high as 15.7% and 14.1%, respectively. RGB-YOLO-DWA, aided by an object detection network for path calculation, improves the ability to identify passable areas, increasing accuracy to 86.1%. However, it suffers from high false positive rates (11.2%) under thermal interference and reflective conditions, with an explanation score of only 0.54, failing to meet the requirements of task-level causal understanding.

[0206] The GraphFusion-DWA method introduces graph structure learning and semantic relationship modeling, making path planning more consistent with the environment structure, improving the accuracy to 90.4%, with an explanation score of 0.77 and a path offset of 6.1cm. However, due to its high complexity of using a two-stage graph construction + path optimization process, the processing time is relatively long, reaching 0.72 seconds, which is not conducive to real-time response in narrow scenes under vehicles.

[0207] In comparison, the method of this invention achieves systemic advantages in several core dimensions: by leveraging multimodal collaborative perception, modal confidence feedback, semantic navigation cost graph, and adaptive trajectory control mechanisms, the navigation success rate reaches 95.3%, the false alarm rate and missed detection rate are reduced to 4.2% and 5.0% respectively, the path deviation is controlled at 4.4cm, and the processing latency is only 0.48s, which is the best among all methods. The interpretation consistency score is as high as 0.91, significantly outperforming the splicing model (0.54) and graph structure method (0.77), fully demonstrating the comprehensive capabilities of this method in semantic understanding and execution feedback in complex scenarios.

[0208] In summary, this method not only achieves superior results in path accuracy and perception robustness, but also maintains optimal performance in the three key dimensions of real-time performance, interpretability, and execution stability, fully demonstrating its broad adaptability and engineering deployment potential in multimodal path planning tasks facing complex under-vehicle scenarios.

[0209] Example 2

[0210] This embodiment provides a local path planning system for a vehicle undercarriage inspection robot based on cross-modal collaborative learning, including:

[0211] The data acquisition module is configured to acquire multimodal sensing data;

[0212] The preprocessing module is configured to perform data preprocessing on the acquired multimodal sensing data;

[0213] The fusion perception module is configured to construct a multimodal fusion perception feature map based on preprocessed data and a cross-modal collaborative modeling mechanism.

[0214] The local path planning module is configured to map the multimodal fusion perception feature map into a local path navigation map based on path planning;

[0215] The optimal path module is configured to select the optimal path in the local path navigation graph based on the path generation mechanism.

[0216] The optimization module is configured to perform optimal path optimization based on the state error estimation mechanism.

[0217] A computer-readable storage medium storing a plurality of instructions adapted for loading and execution by a processor of an end device of the aforementioned method for local path planning of a vehicle undercarriage inspection robot based on cross-modal collaborative learning.

[0218] A terminal device includes a processor and a computer-readable storage medium, the processor being used to implement various instructions; the computer-readable storage medium being used to store multiple instructions, the instructions being adapted to be loaded and executed by the processor to provide a local path planning method for a vehicle undercarriage inspection robot based on cross-modal collaborative learning.

[0219] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A local path planning method for a vehicle undercarriage inspection robot based on cross-modal collaborative learning, characterized in that, include: Acquire multimodal sensing data; Data preprocessing is performed on the acquired multimodal sensing data; A multimodal fusion sensing feature map is constructed based on the preprocessed data and a cross-modal collaborative modeling mechanism; Path planning is used to map multimodal fusion perception feature maps into local path navigation maps; Select the optimal path in the local path navigation graph based on the path generation mechanism; Optimal path optimization is performed based on a state error estimation mechanism. The process of constructing a multimodal fusion sensing feature map based on preprocessed data and a cross-modal collaborative modeling mechanism includes input tensors. The model is constructed into image modality branches and structural modality branches, respectively. A two-stream neural network is used to extract deep semantic embeddings for each modality, providing a structurally distinct initial representation for subsequent modality collaboration. Specifically, RGB images and infrared images are merged into the image modality input. ,in, This indicates the extraction of the first four channels, namely R, G, B, and IR; H and W represent the image height and width dimensions; the LiDAR fusion depth map is used as the structural modality input, and finally, two lightweight convolutional neural networks are input to extract modal features, as shown below: in, It is a three-layer convolutional-normalized-ReLU network; It is a two-layer light convolutional network; Image modal feature map; This is a structural modal feature map.

2. The local path planning method for a vehicle undercarriage inspection robot based on cross-modal collaborative learning according to claim 1, characterized in that, The acquired multimodal sensing data undergoes data preprocessing, including LiDAR point cloud projection and depth completion fusion. Firstly, extrinsic parameter transformation and perspective projection are performed on each 3D point to map all point clouds into a sparse depth map. where pixel position Store the corresponding For each empty pixel position in the image Within a certain radius r Search for the nearest non-empty point within the window. Then assign its depth value, and then interpolate to complete the image. With depth camera images Weighted fusion is represented as: ， in, It represents the position of the lidar point in the radar coordinate system, where T is the transpose; These are the rigid body transformation parameters of the lidar to the camera; These are the three-dimensional coordinates of the point in the camera coordinate system; This is a camera intrinsic parameter matrix that includes focal length and principal point offset; These are the pixel coordinates of the point projected into the image space; It is the depth value of that point from the camera's perspective.

3. The local path planning method for a vehicle undercarriage inspection robot based on cross-modal collaborative learning according to claim 2, characterized in that, The data preprocessing of the acquired multimodal sensing data also includes modality normalization and numerical scaling, wherein for each modal image... Perform standardization to ensure that the mean is 0 and the standard deviation is 1, as shown below: in, Images from an RGB camera or an infrared camera; For the first i A modal image at position ( h,W The pixel value of ); The mean of the entire multimodal image; The standard deviation of the image; H Image height; W The width of the image; This is a numerical stability factor to prevent the denominator from being zero; The normalized image has a standard normal distribution, ensuring that the expression weight of each modality is balanced in the subsequent network and avoiding the phenomenon of single-modality dominance.

4. The local path planning method for a vehicle undercarriage inspection robot based on cross-modal collaborative learning according to claim 3, characterized in that, The data preprocessing of the acquired multimodal sensing data also includes introducing an edge saliency enhancement operation on the normalized image to enhance the local structural features of the vehicle undercarriage structure. First, the Sobel edge gradient map is calculated, represented as: in, yes The partial derivative along the x-direction represents the rate of change of image intensity in the horizontal direction; x is the horizontal coordinate of the image, representing the position of the pixel in the width direction of the image. yes The partial derivative along the y-axis represents the rate of change of image intensity in the vertical direction; y is the ordinate of the image, representing the pixel's position in the image width direction, and is then weighted and fused with the original image to obtain the final image. ,in, Marginal significance fusion factor; Enhance the structural information of the edge map; This is the modal image after edge enhancement.

5. The local path planning method for a vehicle undercarriage inspection robot based on cross-modal collaborative learning according to claim 4, characterized in that, The process of constructing a multimodal fusion perceptual feature map based on preprocessed data and a cross-modal collaborative modeling mechanism also includes introducing a cross-modal attention-guided enhancement mechanism. This mechanism leverages the information advantages of the image modality to guide the structural modality in strengthening its spatial representation, thus constructing a more semantically consistent fusion representation. Next, an attention matrix is ​​constructed, and attention weights are applied to the image modality feature values. Finally, a channel fusion residual strategy is used to fuse the information from the two structural modalities. The cross-modal guidance mechanism is represented as follows: in, For query vector; The key vector; The vector consists of three elements: query, key, and value, which are the features obtained by linear mapping of these elements.

6. The local path planning method for a vehicle undercarriage inspection robot based on cross-modal collaborative learning according to claim 5, characterized in that, The construction of a multimodal fusion-aware feature map based on preprocessed data and a cross-modal collaborative modeling mechanism also includes introducing a modality confidence-aware mechanism to dynamically evaluate the confidence of the two modalities and adjust the final output features proportionally to ensure that the dominant features originate from reliable modal paths. Specifically, a global average pooling (GAP) is first applied to the output features of the two paths. The confidence level is then obtained by applying a small fully connected layer and sigmoid activation. in, The corresponding weight matrix is ​​normalized to obtain the weighting coefficients: The final fused output features are: .

7. The local path planning method for a vehicle undercarriage inspection robot based on cross-modal collaborative learning according to claim 6, characterized in that, The path planning-based mapping of multimodal fusion sensing feature maps into local path navigation maps includes multimodal feature map-based mapping. Design a lightweight semantic segmentation head that outputs a category probability map. Next, the category with the highest probability for each pixel is selected as the semantic label to form a semantic mask. in, This indicates that the position with the highest probability in the Kth class is selected as the predicted class. ; The final pixel-level semantic annotation map is based on the obtained semantic label map. and semantic probability graphs By integrating structural category and classification confidence information, a highly expressive local access cost graph is constructed. Furthermore, a cost construction function based on category probability weighting is introduced, which integrates semantic category cost and confidence information to calculate the passage cost for each pixel.

8. The local path planning method for a vehicle undercarriage inspection robot based on cross-modal collaborative learning according to claim 7, characterized in that, The path generation mechanism for selecting the optimal path in the local path navigation map includes first performing trajectory simulation guided by multimodal perception, assuming the robot's current position is... ,in This represents the two-dimensional coordinates of the robot in the cost graph plane; This indicates the robot's current orientation angle, in radians, and is derived from multiple combinations of linear and angular velocities. For each pair of combinations, perform trajectory simulation for a fixed time T, with a simulation step size of . This yields a trajectory sequence; each trajectory is mapped onto the cost graph plane and combined with the passage cost graph. With semantic mask graph Rate the path: in, Indicates the transit value of the current location; The semantic value representing the current position; The obstacle penalty function; These are the weighting coefficients; This is the trajectory score.

9. A local path planning method for a vehicle undercarriage inspection robot based on cross-modal collaborative learning according to claim 8, characterized in that, The optimal path optimization based on the state error estimation mechanism includes introducing a state error estimation mechanism to detect whether the trajectory deviation has exceeded the control tolerance range and defining the current actual state of the robot. ,in Indicates the actual execution position and orientation at step t; constructs the position offset error. With orientation offset error When it appears or If more than K steps continue to occur, the current path is considered invalid, and the reconstruction process begins. Furthermore, control command deviation constraints are introduced to determine the state stability of the underlying execution system. The actual command currently fed back by the actuator is defined, and a control deviation metric is calculated. If... And continue M If the step is not completed, it is determined to be a control mismatch. M To allow the maximum number of sustained mismatch steps, once any of the following conditions are triggered: ① or Continuous K steps; ② continued M If the step is to call the cost graph, then... With semantic mask graph This triggers a re-planning of the trajectory. This is the maximum allowed position offset threshold; This is the maximum allowed orientation offset threshold.

Images