Bank hidden danger intelligent identification and risk assessment method based on multi-modal perception of walking robot

Abstract

This invention relates to the field of intelligent technology for identifying hidden dangers on reservoir and river / lake embankments, specifically to a method for intelligent identification and risk assessment of embankment hidden dangers based on multimodal perception using a walking robot. The method involves a quadruped robot equipped with multiple sensor modules autonomously inspecting reservoir and river / lake embankment areas to collect multimodal data from the embankment surface. This multimodal data is then transmitted to a data processing center. Based on this data, the types and locations of hidden dangers on the embankment surface are identified. These hidden danger types include cracks, seepage or piping, settlement, and structural deformation. The types and locations of these hidden dangers are quantitatively analyzed to obtain quantitative results. Based on these results, a risk assessment report, a visual map, and an early warning are generated. This invention improves the spatial coverage accuracy and temporal synchronization of data acquisition, and possesses higher sensitivity and complementarity in identifying hidden danger types such as embankment cracks, seepage, and deformation, providing a perceptual foundation for subsequent analysis.

Description

Technical Field

[0001] This invention relates to the field of reservoir and river / lake embankment slope monitoring and intelligent inspection technology, specifically to a method for intelligent identification and risk assessment of embankment hazards based on multimodal perception of a walking robot. Background Technology

[0002] Reservoirs and river and lake embankments are core infrastructure for flood control and disaster reduction. Their structural safety is directly related to the safety of people's lives and property and the stability of the social economy along the river basin. Affected by factors such as water erosion, soil freeze-thaw cycles, rainfall infiltration, geological settlement, and construction quality, embankment slopes are prone to hidden dangers such as cracks, seepage, piping, and settlement deformation. If these are not detected and dealt with in a timely manner, they can easily lead to major disasters such as embankment breaches and collapses.

[0003] Traditional reservoir and river / lake embankment inspections rely primarily on manual foot patrols and fixed-point instrument measurements. These methods suffer from several drawbacks: low efficiency, high labor intensity, difficulty in fully covering long-distance embankments, reliance on human experience leading to subjective judgment and potential oversight of minor cracks and hidden seepage; significant susceptibility to terrain, weather, and lighting conditions, making it unsafe to reach complex slopes and soft soil areas; fragmented and limited data collection with a single modality, lacking spatiotemporal synchronization and three-dimensional positioning capabilities; and risk assessments that are mostly qualitative, lacking quantitative indicators and visualization support, resulting in insufficient decision-making basis.

[0004] While existing technologies such as UAV aerial surveying, ground radar, and fixed monitoring have been applied, they still have significant limitations: UAVs cannot achieve precise near-ground perception and lack accuracy in identifying low-lying cracks and shallow seepage; ground radar equipment is bulky and cumbersome to deploy, making it difficult to conduct large-scale continuous inspections; fixed monitoring points have low density and cannot achieve full coverage; and single sensing modes are easily affected by environmental interference, resulting in insufficient accuracy and robustness in identification.

[0005] Therefore, developing an intelligent method that can autonomously navigate complex, unstructured terrains along reservoirs and riverbanks, perform multimodal fusion perception, accurately identify hidden dangers, and quantitatively assess risks has become an urgent need in the field of safe operation and maintenance of reservoirs and riverbanks. Summary of the Invention

[0006] This invention provides a method for intelligent identification and risk assessment of embankment hazards based on multimodal perception of walking robots. It is a novel intelligent solution that can autonomously move in complex environments, integrate multimodal perception data, and achieve accurate identification and risk assessment of hazards, thereby improving the automation, scientificity, and precision of embankment monitoring and inspection.

[0007] A method for intelligent identification and risk assessment of embankment hazards based on multimodal perception of walking robots includes the following steps:

[0008] S1: Control the quadruped robot to move in the target area and collect multimodal data of the embankment surface through the onboard multi-sensor module; the multimodal data includes visible light images, thermal infrared images and lidar point cloud data;

[0009] S2: Transmit the multimodal data to the data processing center;

[0010] S3: Based on the multimodal data, identify the types and locations of potential hazards on the embankment surface; the types of hazards include cracks, seepage points, or piping openings;

[0011] S4: Quantitative analysis is performed on the aforementioned hazard types and locations, and the quantitative results include:

[0012] Calculate the length, width, and depth of the crack based on visible light images;

[0013] Identify seepage anomaly zones, their area, and flow rate based on thermal infrared images;

[0014] Calculate the distance between potential hazard points and safe areas based on lidar point cloud data;

[0015] S5: Based on the quantitative results, generate a risk assessment report and a visualization map.

[0016] Optionally, S1 specifically includes:

[0017] S11: Generate the globally optimal inspection path based on historical terrain data of the target area or a preset embankment route;

[0018] S12: Control the quadruped robot to move along the globally optimal inspection path, and use the depth camera or lidar on the quadruped robot to perceive the terrain and obstacles in front in real time.

[0019] S13: Based on real-time perception results, the robot's gait and travel route are dynamically adjusted through a local path planning algorithm to achieve obstacle avoidance and stable walking;

[0020] S14: When the quadruped robot is moving stably or stopping at the detection point as planned, the multi-sensor module is triggered synchronously to collect spatiotemporally consistent visible light images, thermal infrared images and lidar point cloud data.

[0021] Optionally, the historical topographic data includes digital elevation models, digital surface models, lidar point clouds, topographic vector boundaries, and historical monitoring information reflecting the evolution of cracks, seepage or piping, settlement and deformation in the embankment area, used to describe the embankment structure and change characteristics; the preset embankment route is derived from engineering design drawings or manual annotation results, including the embankment top line, embankment toe line and key inspection section line.

[0022] Optionally, the local path planning algorithm includes a local environment model based on a depth camera or lidar, which evaluates the traversability of the path ahead, the degree of terrain undulation, and potential obstacle areas. Combining path smoothness, obstacle avoidance requirements, and gait stability requirements, it generates the local optimal reference path at the current moment and adjusts the robot's stride, step frequency, support phase, and swing phase motion parameters in real time to achieve autonomous obstacle avoidance and stable walking in complex environments.

[0023] Optionally, S2 specifically includes:

[0024] S21: The collected multimodal data is wirelessly transmitted to the data processing center in real time via a 5G or 5G-RedCap industrial module mounted on the quadruped robot.

[0025] S22: During transmission, the bandwidth, latency, and packet loss rate of data transmission are monitored in real time.

[0026] S23: If the monitored network status does not meet the real-time transmission requirements, control the robot to temporarily store the current data and move to a location with better network signal before continuing transmission.

[0027] Optionally, S3 specifically includes:

[0028] S31: Based on visible light images, the crack area and pixel-level location are initially identified using a trained deep learning semantic segmentation model;

[0029] Based on thermal infrared images, temperature threshold segmentation and region growing algorithms are used to initially identify temperature anomaly areas and pixel-level locations as candidate areas for seepage points or piping openings.

[0030] A three-dimensional elevation model of the embankment surface is generated based on lidar point cloud data.

[0031] Optionally, step S3 further includes spatially registering and fusing the recognition results of the visible light image and the thermal infrared image with the three-dimensional elevation model to make a comprehensive judgment:

[0032] For cracks, the crack pixel coordinates output by the deep learning semantic segmentation model are mapped onto the three-dimensional elevation model to obtain the three-dimensional position of the crack in the real-world coordinate system.

[0033] For seepage points or piping openings, the temperature anomaly area is overlaid with the three-dimensional elevation model for analysis. If a temperature anomaly and an irregular depression or hole in the surface structure exist simultaneously at a certain three-dimensional location, it is identified as a seepage point or piping opening, and the three-dimensional center location is determined.

[0034] Optionally, S4 specifically includes:

[0035] The size of the crack is calculated by fusing visible light images and lidar point cloud data. Specifically, the length and width of the crack are calculated by analyzing the pixel size of the crack in the visible light image and combining it with camera intrinsic parameters. The depth of the crack is estimated by mapping the crack trajectory to a 3D model generated from the lidar point cloud and calculating the elevation difference on both sides of the crack trajectory line.

[0036] Based on the thermal infrared image, seepage anomaly zones are identified and quantified; wherein, the contours of the temperature anomaly zones are extracted using image segmentation technology, and the actual area of ​​the seepage anomaly zones is calculated based on the spatial resolution provided by the lidar point cloud data; based on the temperature distribution gradient in the thermal infrared image, combined with a preset seepage-heat conduction model, the flow rate at the seepage point is estimated.

[0037] Optionally, S4 also includes a safety distance analysis, which uses the three-dimensional environment model of the embankment constructed based on the lidar point cloud data to spatially calculate the three-dimensional coordinates of the identified hazard points and the preset safety area, including the embankment shoulder line, embankment toe line and nearby buildings, to obtain the distance between the hazard points and the safety area.

[0038] Optionally, a multi-dimensional risk scoring model is constructed, which includes the type of hidden danger, the location of hidden danger, and the corresponding quantitative indicators. The embankment area is classified and labeled according to the risk level. Based on the spatial coordinate system of the embankment, various hidden danger indicators are superimposed and mapped onto a two-dimensional or three-dimensional map to generate a visual map including a risk heat map, a hidden danger point label map, and a high-risk section prompt. At the same time, a textual risk assessment report including the overall risk assessment, statistics of major hidden dangers, and the coverage of inspection routes is output.

[0039] The beneficial effects of this invention are:

[0040] 1. This invention employs a quadruped robot equipped with a multi-sensor module to automatically inspect the surface of a levee, simultaneously acquiring visible light images, thermal infrared images, and lidar point cloud data to form a spatiotemporally aligned multimodal perception dataset. Compared to traditional manual inspection methods, this improves the spatial coverage accuracy and temporal synchronization of data acquisition, and exhibits higher sensitivity and complementarity in identifying potential hazards such as cracks, seepage, settlement, and deformation, providing a perceptual foundation for subsequent analysis.

[0041] 2. This invention generates a globally optimal inspection path by combining historical terrain data with a preset embankment route, and uses a depth camera or LiDAR to perceive real-time terrain, dynamically adjusting the quadruped robot's gait parameters and local trajectory to achieve a combination of obstacle avoidance and stable navigation. Especially in unstructured environments such as soft soil, slopes, and collapsed embankments, it improves walking stability and the continuity of perception tasks, effectively expanding the adaptability and reliability of the intelligent monitoring and inspection system in actual operation and maintenance environments.

[0042] 3. Based on the identified hazard types and quantitative parameters, this invention constructs a multi-factor weighted risk scoring model, integrating indicators such as crack volume, seepage intensity, and spatial proximity. These indicators are then mapped to the embankment spatial coordinate system according to risk level, generating visual results including a risk heat map, hazard labeling map, and high-risk section warning map. Simultaneously, a structured risk assessment report is output, covering overall risk assessment, hazard statistics, and inspection coverage. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram of the logical framework of an embodiment of the present invention. Detailed Implementation

[0046] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. For some well-known technologies, those skilled in the art may also use other alternative methods to implement the invention. Moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0047] like Figures 1 to 2 As shown, the intelligent identification and risk assessment method for embankment hazards based on multimodal perception of walking robots includes the following steps:

[0048] S1: Control the quadruped robot to move in the target embankment area and collect multimodal data of the embankment surface through the onboard multi-sensor module; the multimodal data includes visible light images, thermal infrared images and lidar point cloud data;

[0049] S1 specifically includes:

[0050] S11, based on historical topographic data of the target embankment area Or pre-plan the embankment route A grid model of the embankment area was constructed, and the optimal inspection path was solved using a global path planning algorithm. ;

[0051] S111, Constructing a raster model is suitable for scenarios with regular structures and known terrain. The method is as follows:

[0052] The embankment area is divided into two-dimensional grid units of uniform size. Each cell has a node at its center, and each grid cell is assigned attribute values, including accessibility (0 / 1), elevation difference (slope), surface material (mud / grass / rock), and a historical stability index. The historical stability index reflects the long-term structural safety and deformation activity of a local area of ​​the embankment, and is mainly obtained by analyzing past monitoring data and environmental evolution of the area. The lower the value, the more likely the area has historically been prone to structural instability or geological anomalies.

[0053] Construct grid adjacency relationships (4 / 8 adjacency) to generate a grid graph G=(V,E).

[0054] Where V represents the node set, corresponding to the center point of the grid; E represents the edge set, connecting adjacent grid cells.

[0055] Each edge is assigned a movement cost: ;

[0056] This expression represents the robot's movement from path nodes. Move to adjacent node The path cost function comprehensively considers distance cost, terrain elevation difference cost, and risk factor cost to evaluate the overall cost of the movement operation. The path cost function is a calculation method used in robot path planning to evaluate the accessibility of each path segment. It weights and sums multiple influencing factors on the path to form a comprehensive cost value. It mainly includes three types of factors: first, the spatial distance between nodes, reflecting the path length; second, the terrain elevation difference, i.e., elevation change, used to assess the difficulty of passage due to slope; and third, the risk factor, representing the stability and safety risk level of the current area. Each factor is multiplied by its corresponding weight coefficient and then summed to form the total path cost. By minimizing this cost function during the path search process, an optimal travel path that balances short distance, gentle slope, and low risk can be generated. Represents Euclidean distance. This represents the absolute value of the elevation difference between nodes, reflecting the terrain slope or the cost of vertical ascent; the steeper the slope, the more difficult it is to traverse. This indicates the risk factor at that point. For example, if the location is near a landslide area, seepage point, or obstacle, a higher value indicates a higher risk of passage. This represents the weighting coefficients used to adjust the importance of distance, terrain, and risk in the cost function. Risk coefficient. The values ​​are normalized and set to a range of 0-1, where 0 represents a completely safe area and 1 represents a high-risk area. The evaluation results include a comprehensive assessment of factors such as historical stability indices, distribution of nearby hazard points, slope changes, and abnormal surface humidity. The weighting coefficients in the path cost function... The default configuration can be adjusted according to the actual terrain complexity. This indicates that the route planning prioritizes avoiding steep and high-risk areas. To enhance adaptability to complex terrain, the route could be appropriately increased. and The weight values. Among them... Represents a node and The Euclidean distance between them. Indicates that the robot starts from the node and The straight-line distance between nodes is calculated based on the physical spacing of the constructed raster model. This depends on the map resolution and the connection method between nodes: in a raster map, if equally spaced points are used (i.e., each cell is 0.5 or 1), then the distance between adjacent nodes... The value is 0.5–1.5, with 4 adjacencies being a unit grid length and 8 adjacencies being... Therefore, the value ranges from 0.5 to 20, mainly reflecting the physical length of the robot's movement in a single step within a local or global path.

[0057] S112, Solve for the optimal inspection path using a global path planning algorithm. Represented as:

[0058] ;

[0059] in, The variable representing the minimum value is the set of all possible paths. Find the path that minimizes the total cost. , This represents the globally optimal inspection path. On the constructed region graph model G=(V,E), a graph search algorithm is used to find a path with the minimum total cost between the start and end points. The algorithm builds upon this by introducing a heuristic function, namely Euclidean distance, to predict the estimated cost to the destination, thereby improving search efficiency. Throughout the process, the cost of each path edge is determined by the path cost function. The optimal path is provided, taking into account factors such as distance, elevation difference, and risk. It is a node sequence with the minimum cumulative total cost, used to guide the quadruped robot to perform efficient and reliable inspection movements; Represents the set of all feasible paths. This represents the total number of nodes in the path. Indicates that the robot starts from the node Move to node The cost function, - Indicates the number of path nodes. Indicates the pre-set embankment route. This represents historical topographic data. This includes digital elevation models, digital surface models, lidar point clouds, topographic vector boundaries, and historical evolution records for the embankment area, including traces of seepage and settlement, used to comprehensively characterize the embankment's geomorphological and structural evolution; and a pre-defined embankment route. These data originate from engineering design drawings or manual annotations, including the top line, toe line, and key inspection sections, and are used to guide the robot's path. When constructing a path planning model for a region, if the terrain is relatively regular, the region can be divided into equally spaced grid cells, assigning each cell accessibility, elevation difference, and risk factor to form a grid map model. Alternatively, a digital elevation model can be integrated with embankment route data, using the top line as the framework and local elevation and risk estimates as adjustment criteria to construct a graph-structured map for global path generation of the quadruped robot.

[0060] S12, when the quadruped robot follows the optimal path While moving, it uses its onboard depth camera or LiDAR to acquire real-time 3D point cloud or depth images of the area ahead, constructing a timeline. Local environment model :

[0061] ;

[0062] in, This represents a depth image captured by a depth camera. This represents lidar point cloud data. This represents the sensor data fusion function. It unifies the coordinates, synchronizes the time, and complements the features of data from different types of sensors, including depth images from depth cameras and point cloud data from LiDAR, to generate a consistent local environment model. First, it aligns the timestamps and calibrates the spatial coordinates of the various sensor data to ensure they are comparable under the same reference coordinate system. Second, it generates a dense point cloud from the depth images through projection transformation, which is then fused with the LiDAR point cloud to improve the spatial perception accuracy of obstacles. Finally, it uses methods such as voxel filtering, surface reconstruction, or semantic annotation to denoise, sparsify, or semantically enhance the fused point cloud data, outputting a unified 3D environment representation for path planning and obstacle avoidance decisions.

[0063] S13, based on the local environment model A local path planning algorithm is used to generate a locally optimal reference trajectory. It dynamically adjusts gait parameters and direction of travel based on obstacle avoidance constraints.

[0064] S131, Local path planning optimization can be expressed as:

[0065] ;

[0066] in, Indicates time The local optimal trajectory. Indicates the cost of trajectory smoothing. This indicates the cost of obstacle avoidance. This represents the cost of walking stability. This represents the weighting coefficients in the expression. and The value range is set between 0 and 10, used to adjust the influence of obstacle avoidance cost and walking stability cost in the overall path cost. The obstacle avoidance weight is set to 3–8 to ensure that the robot prioritizes avoiding obstacles. (Stability weight) is generally set to 1-5 to improve gait stability in complex terrain. It is adjusted based on terrain complexity and task requirements through empirical settings or parameter tuning methods. The local path planning algorithm dynamically adjusts the robot's trajectory and gait parameters during quadrupedal robot movement based on real-time perceived terrain information and obstacle distribution. Based on a local environment model acquired by a depth camera or LiDAR, it assesses the traversability of the path ahead, the degree of terrain undulation, and potential obstacle areas. Combining path smoothness, obstacle avoidance requirements, and gait stability requirements, it generates the local optimal reference path for the current moment and adjusts the robot's stride length, stride frequency, support phase, and swing phase in real time to achieve autonomous obstacle avoidance and stable walking in complex environments.

[0067] S132, the robot's gait period, swing phase, and support phase are dynamically adjusted based on the planning results, expressed as: ;

[0068] This expression represents the local optimal path. and the environment model at the current moment Dynamically generate robot gait parameters that adapt to the terrain. ;in, Indicates time The robot's gait parameters, including stride length, stride frequency, swing time, and stance time. Express the gait adjustment function based on the local optimum path Information such as curvature, terrain slope, and obstacle density is obtained from the current environmental model. Key features are extracted, including terrain flatness, passable width, and surface material, and the robot's stride length, stride frequency, swing time, and support time are adjusted accordingly. For example, when the path is straight and the ground is flat, stride length and stride frequency can be set to larger values ​​to improve walking efficiency; when the path is winding or there are obstacles ahead, stride length is reduced and support time is increased to enhance stability; on steep slopes or slippery areas, the swing period can be shortened and the center of gravity lowered to reduce the risk of slipping. This function is usually implemented through a rule base, parameter lookup table, or machine learning model, enabling the robot to have dynamic walking capabilities that adapt to different terrain conditions.

[0069] S14, when the quadruped robot is moving stably or stopping at a key detection point as planned, simultaneously triggers the multi-sensor module to collect spatiotemporally consistent visible light images, thermal infrared images, and lidar point cloud data, including:

[0070] S141, when the stable walking condition is met:

[0071] ;

[0072] This formula represents the current gait parameters of the robot. and environmental model After stability evaluation function After calculation, its stability index Greater than or equal to the preset threshold When the robot is considered to be in a stable moving state, multimodal data acquisition can be triggered.

[0073] S142, or when the robot reaches the preset key detection point At the same time, the system synchronously triggers the multi-sensor modules to collect a spatiotemporally consistent dataset:

[0074] ;

[0075] in, Indicates the robot's stability index. This represents the stability trigger threshold. This is a key parameter used to determine whether the robot is in a stable state in order to trigger multimodal sensor data acquisition. The value ranges from 0.6 to 0.9, with higher values ​​indicating stricter stability requirements. Specific values ​​can be adjusted based on experimental results or the sensor's sensitivity to jitter. In scenarios requiring high image clarity or point cloud accuracy, such as high-resolution infrared analysis, Set the value to 0.8 or higher. In areas with drastic terrain changes and where maintaining high stability for extended periods is difficult, set it to 0.6-0.7 to ensure continuous data collection. Represents a multimodal dataset. Represents a visible light image. Represents thermal infrared images, Represents lidar point clouds, This represents the stability determination function, which is combined with the robot's current gait parameters. and environmental model The function scores the dynamic balance of the gaiter during its movement. It typically calculates a normalized stability score by analyzing factors such as center of gravity fluctuations within the gait cycle, stability of the foot-to-spot support phase, consistency of stride length and width, and terrain slope and obstacle distribution. The value ranges from 0 to 1, with the value closer to 1 indicating more stable walking.

[0076] S2: Transmit multimodal data to the data processing center;

[0077] S2 specifically includes:

[0078] S21 controls the quadruped robot to wirelessly transmit collected multimodal data to a remote data processing center in real time using its onboard 5G or 5G-RedCap industrial communication module. Data is uploaded in real time via the 5G network, and data transmission begins immediately after the transmission path is established. The multimodal dataset includes:

[0079] ;

[0080] in, Indicates time A multimodal dataset, Represents a visible light image. Represents thermal infrared images, This represents point cloud data from a lidar system.

[0081] S22, During data transmission, the following network state parameters are monitored in real time to form a network state vector: ;

[0082] Among them, bandwidth Indicates bandwidth. Indicates a delay; This indicates the packet loss rate.

[0083] And a transmission quality judgment function is introduced: ;

[0084] in, This indicates the current transmission quality score, with a value ranging from 0 to 1. This represents a comprehensive scoring function based on bandwidth, latency, and packet loss rate, calculated using the bandwidth at the current moment. ,Delay and packet loss rate After normalization and weighted calculation, a network transmission quality score ranging from 0 to 1 is obtained. A linear weighted model is generally used, treating bandwidth as a positive indicator and latency and packet loss rate as negative indicators, assigning empirical weights to each, such as bandwidth 0.4, latency 0.3, and packet loss rate 0.3. This calculates the overall quality score for the current network state. This function can flexibly adjust the weight ratios to adapt to different application scenarios' emphasis on transmission speed, stability, or real-time performance, dynamically determining whether the triggering conditions for real-time data transmission are met.

[0085] S23, if it is determined that the current transmission quality does not meet the real-time requirements, that is: ;

[0086] This triggers a buffering mechanism, causing the robot to temporarily store the current multimodal data in a local cache.

[0087] Buffer = ;

[0088] Simultaneously, movement commands are executed to guide the robot to an area with better network coverage, where data transmission can be attempted again. This strategy ensures that multimodal data is not lost in unstable network environments and that the upload process can resume once conditions improve.

[0089] in, This represents the minimum quality threshold for real-time transmission and the minimum quality threshold for stable transmission, ranging from 0.7 to 0.85, depending on the real-time requirements of the data and the fluctuations in the network environment. If the system has high timeliness requirements for multimodal data such as images and point clouds, it will... Set it to 0.8 or higher to ensure that data is delivered in a timely manner and participates in subsequent processing; if the system tolerates a certain degree of delay and the network quality of the deployment environment fluctuates greatly, it can be set to around 0.7.

[0090] S24, during the multimodal data transmission and caching process, all data to be uploaded undergoes transmission encryption processing. This encryption includes real-time transmitted data, data retransmitted after local caching, BeiDou short message data, and satellite communication data. An encryption algorithm is used for end-to-end encryption of the data to prevent unauthorized interception or forgery during wireless transmission, thus avoiding the risks of man-in-the-middle attacks and data theft. Specifically, the multimodal dataset to be sent is encrypted on the quadruped robot side. Encrypt and encapsulate to generate encrypted data packets. The data is then transmitted to a remote data processing center via 5G, 5G-RedCap, BeiDou short message service, or satellite communication link. Upon receiving the data, the remote data processing center uses the corresponding key to decrypt and recover it, ensuring the integrity and security of the data throughout the entire transmission process.

[0091] S25 establishes a dual backup mechanism of local robot caching and cloud storage. When the network is normal, the quadruped robot synchronously uploads the collected multimodal data to the remote data processing center and cloud storage platform to achieve real-time data archiving. In scenarios with unstable networks or offline conditions, the current multimodal data is temporarily stored in the robot's local cache and automatically resumes transmission and synchronizes with the cloud after the network is restored. The local cache is used for temporary data storage in offline environments, while cloud storage is used for long-term persistent archiving and disaster recovery management. When any storage copy is damaged, lost, or inaccessible, the corresponding data is automatically restored from the other backup copy, improving the reliability and disaster recovery capability of the dike inspection data.

[0092] S3: Based on multimodal data, identify the types and locations of potential hazards on the embankment surface; hazard types include cracks, seepage or piping, settlement, and deformation;

[0093] S3 specifically includes:

[0094] S31, initial identification is performed based on data from different modalities:

[0095] S311, Input visible light image Using a trained deep learning semantic segmentation model Extract pixel-level masks for crack regions: ;

[0096] in, Indicates time Visible light images, This invention describes a semantic segmentation model for cracks. The structure of a semantic segmentation model refers to a deep learning network architecture used to classify each pixel in an image into a specific semantic category, such as cracks or background. The U-Net network is employed. In this invention, U-Net is a typical encoder-decoder semantic segmentation network with the ability to identify fine-grained targets (cracks), suitable for image scenes with small samples or local structures. The U-Net model structure mainly consists of two parts: the encoder part (downsampling path), which consists of a series of convolutional and pooling layers, progressively extracting high-level semantic features of the image and compressing spatial resolution to capture the edge texture, shape features, and contextual information of the cracks; and the decoder part (upsampling path), which progressively restores the image spatial size through deconvolution or upsampling operations, while fusing features from corresponding layers in the encoder through skip connections to enhance detail restoration capabilities, enabling the model to accurately locate and segment crack regions. In use, the input visible light image is first fed into the trained U-Net model. The model performs classification prediction for each pixel in the image and outputs a mask map of the same size as the original image. Each pixel value in this mask map represents the category label at the corresponding location. For binary classification problems (crack / non-crack), the output mask is a binary image, where a pixel value of 1 indicates a crack region and 0 indicates background. Ultimately, this mask image can be used to extract the pixel-level coordinates of crack regions, providing a foundation for subsequent 3D mapping and risk quantification.

[0097] The output is a set of pixel-level locations of the cracks: ;

[0098] This formula represents the set of two-dimensional coordinates of all crack pixels extracted from the crack semantic segmentation result. For the crack mask map generated by the model... All positions where the pixel value is 1 As pixels in the crack region, these coordinate points correspond to the parts of the image that the model identifies as cracks, forming the distribution area of ​​cracks on the image plane.

[0099] S312, Recognizes input thermal infrared image Temperature anomaly regions are identified through temperature threshold segmentation and region growing algorithms. ;

[0100] in, Indicates time Thermal infrared images, Temperature threshold is a key parameter used to segment temperature anomaly regions from thermal infrared images. Its value is determined based on the type of embankment material, ambient background temperature, and the normal surface heat distribution range. Temperature threshold segmentation is a simple image segmentation method based on grayscale intensity (which is temperature in infrared images), by setting a temperature threshold. The invention identifies all pixels in the image with temperatures exceeding a threshold as anomalous regions (candidate seepage points), and the remaining areas as background. The temperature threshold is obtained using histogram analysis, which generates a temperature histogram by statistically analyzing the temperature distribution of all pixels in the thermal infrared image and identifying multi-peak structures. Since normal surface areas and anomalous seepage areas typically exhibit different temperature distribution characteristics in thermal imaging, the histogram often displays two or more peaks. By analyzing the valley positions between adjacent peaks, the temperature corresponding to that valley is selected as the segmentation threshold. This represents a temperature anomaly mask. This represents a temperature-based region growing function. Pixels in a thermal infrared image with temperatures exceeding a set threshold are used as seed points. The function expands pixel by pixel towards its surrounding neighborhood, determining if neighboring pixels meet a similarity condition (temperature difference within an acceptable range). If so, the pixel is included in the current region, and expansion continues until no more similar pixels can be added. This process generates a set of connected temperature anomaly regions, completely marking potential seepage points or piping outlet candidate areas. The region growing algorithm is an image segmentation method based on similarity expansion, with the following basic steps:

[0101] Seed point selection: Select one or more points with significantly higher temperatures from the high-temperature pixels obtained by temperature threshold segmentation as seed points;

[0102] Similarity judgment: Taking the seed point as the center, check the temperature value of the surrounding neighboring pixels. If the temperature difference between the neighboring pixels and the seed point is within the set range (±2℃), it is determined to be the same abnormal area.

[0103] Region expansion: Add pixels that meet the conditions to the current region and use them as new seed points to continue searching outwards;

[0104] Termination condition: Growth ends when no more neighboring pixels that meet the conditions can be added, or when the maximum region size limit is reached;

[0105] Multi-region processing: Repeat the above process until all high-temperature regions have been processed, resulting in a complete temperature anomaly connectivity map.

[0106] This algorithm can eliminate isolated noise points, connect and extract anomalous blocks with spatial continuity and thermal consistency, thereby more accurately identifying potential seepage points or candidate areas for piping.

[0107] The output is a set of candidate pixels for suspected seepage points or piping openings:

[0108] ;

[0109] This formula represents the temperature anomaly mask image. Extract the coordinates of all pixels with a median value of 1 to form a set. , used to identify the pixel locations that are identified as seepage points or piping outlet candidate areas in thermal infrared images.

[0110] S313, Input LiDAR point cloud data A three-dimensional elevation model of the embankment area was constructed using a surface reconstruction algorithm: ;

[0111] in, Indicates time The lidar point cloud, Represents a three-dimensional surface model, recording the spatial coordinates of each point. , This refers to the point cloud surface modeling function, which preprocesses, filters, extracts ground points, and reconstructs surfaces from 3D point cloud data acquired by LiDAR, thereby generating a continuous 3D terrain surface model. This includes the following steps:

[0112] 1. Point cloud preprocessing: preprocessing the raw point cloud. Denoising is performed to remove outliers, overlapping points, or invalid height values, thereby improving data quality.

[0113] 2. Ground point extraction: Using methods such as progressive morphological filtering (PMF), CSF, or RANSAC plane fitting, ground points in the embankment area are distinguished from non-ground points (such as vegetation, water bodies, or equipment), retaining only valid points that represent the terrain contour.

[0114] 3. Grid Interpolation or Surface Reconstruction: Ground points are mapped onto a regular two-dimensional grid. Elevation estimation is performed using methods such as bilinear interpolation, inverse distance weighted (IDW), Kriging, or TIN (triangular mesh) to generate a continuous surface elevation model. Each grid cell contains its corresponding spatial coordinates. .

[0115] 4. Output a 3D elevation model: the constructed model The model can be represented as a regular raster (such as a DEM) or an irregular point set to support subsequent spatial registration, hazard location, and 3D feature analysis.

[0116] S32, Multimodal spatial registration and comprehensive assessment of potential hazards:

[0117] S321, set pixel positions Mapping to elevation model The corresponding location is used to obtain the set of three-dimensional spatial coordinates of the cracks. This set represents the spatial distribution of the cracks on the actual embankment surface. Three-dimensional spatial coordinate set: ;

[0118] This formula identifies the crack pixel location in the image through semantic segmentation. This is mapped to the corresponding spatial location in the 3D elevation model, and the location is found in the elevation model. height value in Obtain the set of coordinate positions of the crack in real three-dimensional space. This ultimately forms a three-dimensional crack location set. ,

[0119] in, Represents the set of three-dimensional spatial locations of cracks. This represents the position coordinates of the crack point in the three-dimensional world coordinate system. This represents the set of pixel locations in the image representing the crack region. Indicates the first in the crack region The two-dimensional coordinates of each pixel in the image. Represents the midpoint of the three-dimensional elevation model The corresponding elevation value (i.e.) ), This expression represents the corresponding three-dimensional height value extracted from the elevation model, and indicates the vertical position of the crack point. This expression is used to map the crack identification results on the image to the three-dimensional terrain model to obtain its true spatial location.

[0120] S322, temperature anomaly area When overlaid with a 3D elevation model, if a certain point Simultaneously satisfy:

[0121] It belongs to a temperature anomaly area;

[0122] The corresponding elevation model has surface depressions, voids, or local subsidence;

[0123] This confirms the location as a potential hazard point (seepage point or piping opening), and its three-dimensional center location is extracted:

[0124] ;

[0125] in, This is the set of three-dimensional locations of all confirmed potential hazards.

[0126] S4: Quantitative analysis of hazard types and locations is conducted, and the quantitative results include:

[0127] Calculate the length, width, and depth of the crack based on visible light images;

[0128] Identify seepage anomaly zones, their area, and flow rate based on thermal infrared images;

[0129] Calculate the distance between potential hazard points and safe areas based on lidar point cloud data;

[0130] S4 specifically includes:

[0131] S41, Crack Quantitative Analysis:

[0132] 1. Pixel mask for crack regions in visible light images Calculate the length of the pixel trajectory of the crack in the image plane. and opening width Combined with the camera intrinsic parameter matrix and extrinsic parameter matrix By projecting the pixel coordinates onto the real-world coordinate system, the actual length and width of the crack can be obtained:

[0133] ;

[0134] This formula is based on the pixel length of the crack in a visible light image. and pixel width Combining the camera's internal parameters, mainly the focal length and the average distance between the camera and the target Calculate the actual length of the crack in the real world. and width The camera imaging principle based on the perspective projection model states that there is a linear proportional relationship between the pixel size in an image and its actual physical size, a ratio influenced by both the camera's focal length and the observation distance. This expression allows for the precise mapping of crack dimensions in an image from a two-dimensional pixel space to a three-dimensional world coordinate system, enabling quantitative analysis of the crack's geometric features. Indicates the crack length. Indicates the width of the crack opening. This represents the pixel length and width of the crack in the image. Indicates the camera is in Focal length of direction, This represents the average distance from the camera to the target surface, which can be estimated from a depth map or point cloud.

[0135] 2. Set the crack pixel coordinates Mapping the data onto the 3D model constructed from the lidar point cloud, the elevation difference on both sides of the crack trajectory line is sampled and analyzed:

[0136] ;

[0137] This formula is selected along the crack trajectory line. For each sampling point, calculate the elevation values ​​of its left and right sides in the 3D elevation model. and Calculate the absolute value of their elevation differences, and average all the differences to obtain the estimated average depth of the crack. Structural cracks on embankment surfaces often manifest as vertical depressions of a certain degree. By mapping the crack trajectory to a 3D terrain model generated from lidar point clouds and comparing the elevations on both sides of the crack, the depth characteristics of the crack can be effectively estimated, thereby improving the ability to quantitatively identify its degree of damage. This represents the estimated crack depth. This represents the elevation values ​​of both sides of the crack trajectory line in the 3D model. This indicates the number of sampling points selected along the crack trajectory. This method reflects the degree of vertical indentation of the crack, enhancing the ability to identify and classify structural cracks.

[0138] S42, Permeability Quantification Analysis:

[0139] 1. Masking of temperature anomaly regions in thermal infrared images The region contour is obtained through image segmentation, combined with the spatial resolution of the LiDAR point cloud. Perform area conversion:

[0140] ;

[0141] This formula represents the actual area of ​​the temperature anomaly region (i.e., the seepage candidate region) identified in the thermal infrared image. .in, This is a binary mask image of the seepage region. Pixels with a value of 1 in the mask are identified as pixels within the seepage region. These are the locations of the pixels; each pixel is counted once, and the total number of pixels in the anomaly region is obtained by summing them. This total is then multiplied by the spatial resolution provided by the LiDAR point cloud. This refers to the actual area corresponding to each pixel, which can be converted into the real-world area of ​​the temperature anomaly region. This represents the actual area of ​​the abnormal seepage zone. This indicates the spatial resolution provided by the lidar point cloud. A mask image representing a region of temperature anomalies. This represents the total number of pixels in the statistically abnormal temperature region, used for area conversion.

[0142] 2. Calculate the thermal diffusion intensity based on the temperature gradient within the temperature anomaly region, combined with the empirical seepage-heat conduction mapping function. Estimate the relative flow intensity at the seepage point:

[0143] ;

[0144] in, This represents the relative flow rate intensity (normalized value) at the seepage point. This represents the temperature gradient field, reflecting the rate of heat diffusion within a local region. The mapping function, based on the seepage-heat conduction relationship, needs to be obtained through on-site calibration or simulation modeling. By establishing an empirical relationship between temperature gradient and underground seepage intensity, and utilizing the spatial gradient information of temperature distribution in thermal infrared images, the relative flow rate intensity at the seepage point is estimated. The specific scheme is as follows:

[0145] For temperature anomaly regions identified in thermal infrared images, the temperature gradient field is first calculated within them. This gradient represents the rate of temperature change per unit distance, reflecting the rate of heat diffusion within the region, and is calculated as follows: This formula represents the modulus of the temperature gradient (or the intensity of the spatial gradient), used to measure the drasticness of temperature changes in two-dimensional space. Among them, Represents pixels in an infrared image Temperature value; This represents the rate of temperature change in the horizontal and vertical directions. Since it not only calculates the gradient but also needs to have a certain noise reduction and edge enhancement effect, it can more clearly identify the thermal diffusion boundary of the seepage region and help improve the stability of subsequent gradient field analysis. Therefore, it is implemented by the Sobel operator. This indicates the intensity of temperature diffusion at that point, used to characterize the tendency and severity of heat conduction outwards. Local temperature anomalies caused by seepage typically form a temperature gradient field radiating outwards from the seepage point on the Earth's surface. Therefore, the intensity of the temperature gradient is correlated with the actual seepage velocity.

[0146] To achieve the estimation of seepage flow rate intensity from thermal diffusivity, this invention establishes an empirical mapping function. This includes the following steps:

[0147] Construction Phase (Offline Calibration): In a sample area of ​​the embankment with known seepage flow, infrared sensors are deployed to simultaneously acquire temperature distribution images; for each known seepage point, the corresponding actual flow rate is measured. Simultaneously calculate the corresponding mean or maximum local temperature gradient. Based on collection Data points are fitted with a nonlinear regression model as a mapping function. Due to temperature gradient The seepage flow rate may change rapidly at the edge of the seepage zone, but the corresponding seepage flow rate intensity will not increase exponentially in physics. The logarithmic function has the characteristic of gradually stabilizing after rapid growth, which is closer to the actual response law between water flow and heat diffusion. Therefore, the logarithmic function form can be adopted.

[0148] The formal representation is as follows: ;

[0149] This function performs nonlinear compression or amplification of the local temperature gradient and maps it to a normalized flow intensity range. This reflects the activity level of potential seepage hazards.

[0150] Application phase (actual reasoning):

[0151] For all temperature anomaly regions in the current thermal infrared image, calculate pixel by pixel. ;

[0152] For each region, the mean or peak value of its temperature gradient is taken as input and substituted into the pre-trained mapping function. ;

[0153] Output This is the relative flow intensity index of the current seepage point, which is used for subsequent risk assessment and classification.

[0154] S43, Safety Distance Analysis:

[0155] A 3D model of the embankment constructed based on LiDAR point clouds was used to define the boundary set of the safe zone. This includes the three-dimensional coordinates of the top line, toe line, or nearby key structures of the embankment. For each identified hazard point... Calculate the Euclidean distance between the hazard point and all safety boundary points; the minimum value is the safe distance for that hazard point.

[0156] ;

[0157] in, Indicates the first The three-dimensional spatial distance from each potential hazard point to the nearest safe area Represents the set of three-dimensional boundary points of the safe zone. The three-dimensional coordinates of the potential hazard point are used to assess the degree of threat posed by the hazard to the surrounding structure or the embankment itself, and can support the triggering of risk classification and early warning linkage mechanisms. This represents the Euclidean distance calculation function, which calculates the straight-line distance between any two points in three-dimensional space. The method involves summing the squared differences of the three-dimensional coordinates along each axis and then taking the square root. For applications in hazard identification, the three-dimensional coordinates of the hazard points are typically used. With any reference point in the safe area As input, its Euclidean distance value is calculated as a spatial interval metric. The specific process includes: calculating the distance difference in the x, y, and z directions respectively. , , Square them, add them together, and then take the square root to get the spatial distance between the two points.

[0158] S5: Based on the quantitative results, generate risk assessment reports and visualization maps.

[0159] S5 specifically includes:

[0160] S51, addressing each potential hazard point Based on its identification type and quantitative parameters, its comprehensive risk score is calculated. , represented as:

[0161] ;

[0162] in, Indicate potential hazards Comprehensive risk score, Indicates the first For different types of potential hazards, the following risk factor functions are designed:

[0163] 1. Crack-related hazards:

[0164] Crack size factor The actual length of the crack and width Joint composition, defined as:

[0165] This function reflects the geometric scale of the crack; the larger the crack, the higher the risk value. This is achieved through a normalization function. Mapped to .

[0166] Crack depth factor The average depth of the crack region in the 3D terrain model The result is expressed as: It is used to quantify the degree of structural damage to the embankment; the greater the depth, the higher the risk of structural penetration.

[0167] 2. Seepage-related hazards:

[0168] Area factor of seepage zone This represents the actual area of ​​the temperature anomaly region. : The wider the seepage range, the larger the affected area and the higher the risk level.

[0169] seepage intensity factor The relative flow intensity estimated based on the temperature gradient mapping function ,Right now: ; indicates the level of activity of seepage, and is in a normalized form, which can be directly used as a risk factor input.

[0170] Universal space risk factor (applicable to all types):

[0171] Safety distance factor : Indicates the closest spatial distance between the potential hazard point and the top of the dike or critical area. The closer the distance, the higher the risk; therefore, an inverse proportional function is used. ;in The small constant is set to 0.1 to prevent division by zero.

[0172] The final comprehensive risk score for each potential hazard point is calculated as follows: ;in, The risk factor function defined above; The weights of each factor satisfy the following conditions: The settings can be configured based on actual engineering experience or model training results:

[0173] Crack size weight;

[0174] Crack depth weight;

[0175] Seepage area weight;

[0176] Seepage intensity weight;

[0177] : Safety distance weight.

[0178] S52, the hazard rating value Mapped to risk level All potential hazards All are projected onto the embankment spatial coordinate system according to their risk levels. This creates a spatial distribution map and risk level. Represented as:

[0179] ;

[0180] in, , The high-risk and medium-risk scores are represented by preset values, which can be determined through historical experience or statistical regression, and are generally taken as... , .

[0181] S53, based on a 3D model or 2D plan view of the embankment, overlays all potential hazard points and their risk levels to generate multiple types of maps:

[0182] Risk heat map: assigning hazard scores Interpolate to the embankment area to generate a continuous heat map;

[0183] Hazard marking map: categorized by hazard type and location Draw icons and label quantization parameters;

[0184] High-risk segment alert map: Clustering consecutive high-risk points and marking them as key detection areas. ;

[0185] All visualizations are rendered in a unified embankment coordinate system and support 2D or 3D display.

[0186] S54, Risk Assessment Report Generation: Automatically generates a structured report including the following:

[0187] Overall Risk Assessment: Statistics on the risk distribution across the entire dike, including the percentage of high-risk points;

[0188] Major Hazard Statistics Table: Summarizes the number, average, and maximum values ​​of different types of hazards;

[0189] Inspection route coverage: Does the inspection route cover all key potential hazard points?

[0190] Key warning paragraphs: Identify consecutive high-risk paragraphs Please attach a diagram to indicate the location.

[0191] The report content is output in text, table and figure formats, and supports PDF or platform-based display.

[0192] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0193] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for intelligent identification and risk assessment of embankment hazards based on multimodal perception of a walking robot, characterized in that, Includes the following steps: S1: Train the quadruped robot to move in the target reservoir and river / lake embankment area, and collect multimodal data of the embankment surface through the onboard multi-sensor module; the multimodal data includes visible light images, thermal infrared images and lidar point cloud data; S2: Transmit the multimodal data to the data processing center; S3: Based on the multimodal data, identify the types and locations of potential hazards on the embankment surface; the types of hazards include cracks, seepage points or piping openings, settlement, and structural deformation; S4: Quantitative analysis is performed on the aforementioned hazard types and locations, and the quantitative results include: Calculate the length, width, and depth of the crack based on visible light images; Identify seepage anomaly zones, their area, and flow rate based on thermal infrared images; Calculate the distance between potential hazard points and safe areas based on lidar point cloud data; S5: Based on the quantitative results, generate a risk assessment report and a visualization map, and issue an early warning.

2. The method for intelligent identification and risk assessment of embankment hazards based on multimodal perception of a walking robot according to claim 1, characterized in that, S1 specifically includes: S11: Generate the globally optimal inspection path based on historical topographic data of the target reservoir and river / lake embankment area or preset embankment routes; S12: Control the quadruped robot to move along the globally optimal inspection path, and use the depth camera or lidar on the quadruped robot to perceive the terrain and obstacles in front in real time. S13: Based on real-time perception results, the robot's gait and travel route are dynamically adjusted through a local path planning algorithm to achieve obstacle avoidance and stable walking; S14: When the quadruped robot is moving stably or stopping at the detection point as planned, the multi-sensor module is triggered synchronously to collect spatiotemporally consistent visible light images, thermal infrared images and lidar point cloud data.

3. The method for intelligent identification and risk assessment of embankment hazards based on multimodal perception of a walking robot according to claim 2, characterized in that, The historical topographic data includes digital elevation models, digital surface models, lidar point clouds, topographic vector boundaries, and historical monitoring information reflecting the evolution of cracks, seepage or piping, settlement and deformation in reservoir and river and lake embankment areas, which are used to describe the structural stability and change characteristics of reservoir and river and lake embankments. The pre-defined embankment route is derived from engineering design drawings or manual annotations, including the embankment top line, embankment toe line, and key inspection section lines.

4. The method for intelligent identification and risk assessment of embankment hazards based on multimodal perception of a walking robot according to claim 2, characterized in that, The local path planning algorithm includes a local environment model acquired by a depth camera or LiDAR, which evaluates the traversability of the path ahead, the degree of terrain undulation, and potential obstacle areas. Combining path smoothness, obstacle avoidance requirements, and gait stability requirements, it generates the local optimal reference path at the current moment and adjusts the robot's stride, step frequency, support phase, and swing phase motion parameters in real time to achieve autonomous obstacle avoidance and stable walking in complex embankment environments.

5. The method for intelligent identification and risk assessment of embankment hazards based on multimodal perception of a walking robot according to claim 1, characterized in that, S2 specifically includes: S21: The collected multimodal data is wirelessly transmitted to the data processing center in real time via a 5G or 5G-RedCap industrial module mounted on the quadruped robot. S22: During transmission, the bandwidth, latency, and packet loss rate of data transmission are monitored in real time. S23: If the monitored network status does not meet the real-time transmission requirements, control the robot to temporarily store the current data and move to a location with better network signal before continuing transmission.

6. The method for intelligent identification and risk assessment of embankment hazards based on multimodal perception of a walking robot according to claim 1, characterized in that, S3 specifically includes: S31: Based on visible light images, the crack area and pixel-level location are initially identified using a trained deep learning semantic segmentation model; Based on thermal infrared images, temperature threshold segmentation and region growing algorithms are used to initially identify temperature anomaly areas and pixel-level locations as candidate areas for seepage points or piping openings. A three-dimensional elevation model of the embankment surface is generated based on lidar point cloud data.

7. The method for intelligent identification and risk assessment of embankment hazards based on multimodal perception of a walking robot according to claim 6, characterized in that, S3 further includes spatially registering and fusing the recognition results of the visible light image and the thermal infrared image with the three-dimensional elevation model to make a comprehensive judgment: For cracks, the crack pixel coordinates output by the deep learning semantic segmentation model are mapped onto the three-dimensional elevation model to obtain the three-dimensional position of the crack in the real-world coordinate system. For seepage points or piping openings, the temperature anomaly area is overlaid with the three-dimensional elevation model for analysis. If a temperature anomaly and an irregular depression or hole in the surface structure exist simultaneously at a certain three-dimensional location, it is identified as a seepage point or piping opening, and the three-dimensional center location is determined.

8. The method for intelligent identification and risk assessment of embankment hazards based on multimodal perception of a walking robot according to claim 1, characterized in that, S4 specifically includes: The size of the crack is calculated by fusing visible light images and lidar point cloud data. Specifically, the length and width of the crack are calculated by analyzing the pixel size of the crack in the visible light image and combining it with camera intrinsic parameters. The depth of the crack is estimated by mapping the crack trajectory to a 3D model generated from the lidar point cloud and calculating the elevation difference on both sides of the crack trajectory line. Based on the thermal infrared image, seepage anomaly zones are identified and quantified; wherein, the contours of the temperature anomaly zones are extracted using image segmentation technology, and the actual area of ​​the seepage anomaly zones is calculated based on the spatial resolution provided by the lidar point cloud data; based on the temperature distribution gradient in the thermal infrared image, combined with a preset seepage-heat conduction model, the flow rate at the seepage point is estimated.

9. The method for intelligent identification and risk assessment of embankment hazards based on multimodal perception of a walking robot according to claim 1, characterized in that, S4 also includes safety distance analysis. Based on the three-dimensional environment model of the embankment constructed from the lidar point cloud data, the three-dimensional coordinates of the identified hazard points are spatially calculated with the preset safety area, including the embankment shoulder line, the embankment toe line and nearby buildings, to obtain the distance between the hazard points and the safety area.

10. The method for intelligent identification and risk assessment of embankment hazards based on multimodal perception of a walking robot according to claim 1, characterized in that, A multi-dimensional risk scoring model is constructed, which includes the type of hidden danger, the location of hidden danger, and the corresponding quantitative indicators. The embankment area is classified and labeled according to the risk level. Based on the spatial coordinate system of the embankment, various hidden danger indicators are superimposed and mapped onto a two-dimensional or three-dimensional map to generate a visual map including a risk heat map, a hidden danger point label map, and a high-risk section prompt. At the same time, a textual risk assessment report including the overall risk assessment, statistics of major hidden dangers, and the coverage of inspection routes is output.

Images