Slope crack inspection method, device, equipment and medium for use in high mountain canyon areas

Abstract

The present disclosure provides a kind of for high mountain and valley area's side slope crack inspection method and device, equipment, medium, it is related to dam side slope safety inspection technical field.The method comprises: control inspection unmanned plane to collect the fusion three-dimensional terrain data corresponding to dam side slope with first resolution, and filter local topographic change area, to obtain local crack fine inspection route in combination with crack inspection route planning model;Based on the route control inspection unmanned plane to carry out adaptive route dynamic programming, to complete fine inspection to crack terrain with second resolution, obtain millimeter level crack inspection data.This scheme realizes one-time inspection to dam side slope, the process of patrol has coarse and fine, inspection coverage is whole and has pertinence millimeter level fine crack inspection to key area, improve identification accuracy, and one-time inspection can complete the inspection of all key attention areas of dam side slope area, effectively increase the endurance time of inspection unmanned plane, reduce redundant inspection data amount.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of dam slope safety inspection, in particular, to a slope crack inspection method and device, equipment and medium for high mountain and canyon area. BACKGROUND

[0002] In high mountain and canyon area, the accuracy and comprehensiveness of dam slope crack inspection are of great significance to the long-term safe operation of hydraulic structures. Unmanned aerial vehicle (UAV) inspection technology has become an important means of dam crack monitoring due to its flexibility and coverage.

[0003] As one of the commonly used sensors for inspection UAV, laser radar is widely used in slope inspection tasks due to its adaptability to complex terrain and strong global modeling capability. However, although laser radar can quickly obtain large-scale terrain elevation data, the discrete nature of the data limits its ability to capture micro features. In detail complex crack areas, it is difficult to provide data that meets the high-precision detection requirements. At the same time, the high power consumption characteristic of laser radar has a great impact on the endurance of UAV in the inspection task, especially when the best resolution is used to scan the entire area. Even in flat areas or areas with sparse crack distribution, the collection of redundant data will still cause waste of UAV energy, shorten the execution time of UAV inspection task, and the inspection cycle is longer and the inspection efficiency is low.

[0004] In addition, the current UAV inspection technology relies on static route design in task planning, lacks intelligent adaptive and autonomous route planning capability, and is difficult to dynamically adjust the route according to real-time collected terrain data or detection requirements. Such a static way is easy to cause the problem of insufficient coverage or low efficiency of the inspection path in complex terrain scenarios, especially in crack concentration areas, the fixed path planning method cannot achieve fine capture of crack features.

[0005] In summary, the related technology has the problems of insufficient micro feature capture capability of laser radar, endurance performance decline caused by high power consumption, and lack of dynamic adjustment capability of inspection path. These defects are particularly obvious in the complex scene of dam slope in high mountain and canyon area, and further improvement is needed to meet the current demand for lack of intelligent adaptive and autonomous route planning capability of inspection UAV.

[0006] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0007] The purpose of the embodiments of the present disclosure is to provide a slope crack inspection method for high mountain canyon areas, a slope crack inspection device, an electronic device and a computer-readable storage medium for high mountain canyon areas, so as to realize a one-time inspection of the dam slope. The inspection process is coarse and fine, the inspection coverage is full and the key areas are targeted with millimeter-level fine crack inspection, thereby improving the recognition accuracy, and a one-time inspection can complete the inspection of all key areas of the dam slope area, effectively increasing the endurance time of the inspection drone, reducing the amount of redundant inspection data, effectively improving the precision, accuracy and completeness of the collected crack data, and improving the efficiency of dam slope inspection.

[0008] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by practice of the present disclosure.

[0009] According to a first aspect of an embodiment of the present disclosure, a slope crack inspection method for use in a high mountain canyon area is provided, comprising:

[0010] Controlling the inspection drone to follow a preset initial inspection route and collect, in real time, fused three-dimensional terrain data corresponding to the dam slope in the alpine canyon area at a first scanning resolution, the fused three-dimensional terrain data being collected by a laser radar and a depth camera carried by the inspection drone;

[0011] Locally matching the fused three-dimensional terrain data with pre-constructed reference three-dimensional terrain data to determine the local terrain change area in real time;

[0012] Inputting the fused three-dimensional terrain data corresponding to the local terrain change area into the pre-trained crack inspection route planning model to obtain a local crack fine inspection route;

[0013] Based on the local crack fine inspection route, the inspection drone is controlled to perform adaptive route dynamic planning, and the crack terrain in the local terrain change area is finely inspected at a second scanning resolution to obtain millimeter-level crack inspection data.

[0014] In some example embodiments of the present disclosure, based on the aforementioned scheme, the real-time acquisition of fused three-dimensional terrain data corresponding to the dam slope in the high mountain canyon area at the first scanning resolution includes: obtaining the terrain elevation data and terrain structure point cloud data corresponding to the dam slope in the high mountain canyon area acquired by the lidar and the depth camera at the first scanning resolution in real time respectively; aligning the terrain elevation data and the terrain structure point cloud data to obtain preliminary three-dimensional terrain data; and performing deep fusion on the preliminary three-dimensional terrain data to obtain fused three-dimensional terrain data corresponding to the dam slope in the high mountain canyon area.

[0015] In some example embodiments of the present disclosure, based on the foregoing scheme, the local matching of the fused three-dimensional terrain data and the pre-constructed reference three-dimensional terrain data to determine a local terrain change region includes: extracting first key feature points in the fused three-dimensional terrain data and second key feature points in the reference three-dimensional terrain data; coarsely registering the first key feature points and the second key feature points to determine a preliminary key feature point pair; iteratively optimizing the preliminary key feature point pair to determine a target key feature point pair, and determining at least one local sub-region according to the target key feature point pair, the local sub-region including a steep region, a gentle slope region, and a crack distribution region; determining a local minimum mean square error corresponding to each local sub-region according to the target key feature point pair; and regarding a local sub-region with a local minimum mean square error greater than or equal to a preset terrain error threshold as the local terrain change region.

[0016] In some example embodiments of the present disclosure, based on the foregoing scheme, the iteratively optimizing the preliminary key feature point pair to determine a target key feature point pair includes: determining a rotation matrix and a translation vector according to the preliminary key feature point pair, the rotation matrix and the translation vector making the mean square error between the matching point pairs minimum; updating and transforming a first key feature point belonging to the preliminary key feature point pair through the rotation matrix and the translation vector, and determining the mean square error between the updated and transformed first key feature point and the second key feature point; iteratively updating the first key feature point in the preliminary key feature point pair until the mean square error is less than a preset matching error threshold, stopping the iteration, and constructing a target key feature point pair according to the latest updated and transformed first key feature point and the second key feature point.

[0017] In some example embodiments of the present disclosure, based on the foregoing scheme, the inputting the fused three-dimensional terrain data corresponding to the local terrain change region into the pre-trained crack inspection route planning model to obtain a local crack fine inspection flight path includes: extracting terrain feature parameters in the fused three-dimensional terrain data corresponding to the local terrain change region; constructing a state space vector according to the terrain feature parameters and the initial inspection flight path; inputting the state space vector into the crack inspection route planning model to determine a crack collection inspection point of the local terrain change region and a crack collection flight attitude of an inspection unmanned aerial vehicle corresponding to each crack collection inspection point; and constructing a local crack fine inspection flight path based on the crack collection inspection point and the crack collection flight attitude.

[0018] In some example embodiments of the present disclosure, based on the foregoing scheme, the inputting the state space vector into the crack inspection route planning model to determine the crack collection inspection points of the local terrain change region and the crack collection flight attitude of the inspection UAV corresponding to each crack collection inspection point comprises: inputting the state space vector into the crack inspection route planning model to determine an action vector that maximizes a cumulative reward value of a reward function of the crack inspection route planning model; updating the inspection position and flight attitude of the inspection UAV based on the action vector, and recording the UAV inspection path and inspection flight attitude inferred by the crack inspection route planning model; in response to the local terrain change region being covered or the flight time of the inspection UAV reaching a preset time, confirming that the inference of the inspection task is completed, and determining the crack collection inspection points of the local terrain change region and the crack collection flight attitude of the inspection UAV corresponding to each crack collection inspection point according to the recorded UAV inspection path and inspection flight attitude.

[0019] In some example embodiments of the present disclosure, based on the foregoing scheme, the crack inspection route planning model is obtained through a reinforcement learning training process, and the reinforcement learning training process comprises: determining a state space of the crack inspection route planning model, wherein the state space comprises inspection point coordinates and UAV flight attitude in an initial inspection route, global terrain data and historical flight path information of a dam slope in a high mountain and canyon region; determining an action space of the crack inspection route planning model, wherein the action space comprises adjusting flight height, adjusting flight direction, adjusting flight speed, adjusting the shooting angle of a laser radar and a depth camera, and updating an inspection point; sampling inspection interaction data through the state space and the action space, and storing the inspection interaction data in an interaction experience pool, wherein the inspection interaction data is used to simulate the inspection UAV performing an inspection task in the dam slope region in the high mountain and canyon region; randomly sampling inspection interaction data in the interaction experience pool, and performing model training on the crack inspection route planning model, so as to maximize the cumulative reward value of the reward function of the crack inspection route planning model, and optimize the model parameters of the crack inspection route planning model through gradient descent until the cumulative reward value is greater than or equal to a preset reward value threshold, thereby obtaining a trained crack inspection route planning model.

[0020] According to a second aspect of the embodiments of the present disclosure, a slope crack inspection device for a high mountain and canyon region is provided, comprising:

[0021] a three-dimensional terrain collection module configured to control the inspection UAV to collect fusion three-dimensional terrain data corresponding to the dam slope in the high mountain and canyon region in real time at a first scanning resolution along a preset initial inspection route, wherein the fusion three-dimensional terrain data is collected by a laser radar and a depth camera carried by the inspection UAV;

[0022] a topographic change area determination module configured to locally match the fused three-dimensional terrain data with pre-constructed reference three-dimensional terrain data to determine a local topographic change area;

[0023] an adaptive route planning module configured to input the fused three-dimensional terrain data corresponding to the local topographic change area into a pre-trained crack inspection route planning model to obtain a local crack fine inspection route;

[0024] a crack fine inspection module configured to control the inspection unmanned aerial vehicle to perform adaptive route dynamic planning based on the local crack fine inspection route, to perform fine inspection on the crack terrain of the local topographic change area at a second scanning resolution, and to obtain millimeter-level crack inspection data.

[0025] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, comprising: a processor; and a memory having computer readable instructions stored thereon, the computer readable instructions being executed by the processor to implement the method for crack inspection of a side slope in a high mountain and canyon area according to any one of the preceding aspects.

[0026] According to a fourth aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, having a computer program stored thereon, the computer program being executed by a processor to implement the method for crack inspection of a side slope in a high mountain and canyon area according to any one of the preceding aspects.

[0027] The technical solutions provided by the embodiments of the present disclosure can include the following beneficial effects:

[0028] The method for crack inspection of a side slope in a high mountain and canyon area in the example embodiments of the present disclosure uses a laser radar and a depth camera in cooperation, the laser radar can obtain three-dimensional terrain elevation data in a wide range, providing a basis for accurate description of overall terrain features, while the depth camera can provide three-dimensional image data with ultra-high resolution in a local crack area, making up for the lack of the laser radar in capturing micro-continuous features, and the data fusion of the two can significantly improve the detail information of the crack area.

[0029] During data acquisition, the depth camera can significantly improve the detection accuracy of key features such as crack width and depth through fine scanning of the crack area. Combined with the global terrain data collected by the laser radar, the depth camera provides more detailed crack information for the enhancement of local crack features, laying a reliable data foundation for subsequent crack analysis and expansion trend prediction. In addition, by dynamically adjusting the scanning resolution of the laser radar and the depth camera, the limited power energy of the unmanned aerial vehicle can be flexibly allocated in different areas. In particular, by increasing the scanning resolution in key areas while reducing the redundant collection in non-key areas, the inspection efficiency is optimized and the energy consumption of the unmanned aerial vehicle is reduced, thereby ensuring the endurance time of the unmanned aerial vehicle and improving the inspection efficiency of the dam slope.

[0030] Through comprehensive processing of the fusion data of the laser radar and the depth camera, local terrain change areas can be marked in real time, and the flight path and flight attitude of the inspection unmanned aerial vehicle can be dynamically adjusted. The global terrain information provided by the laser radar ensures the coverage and safety of path planning, while the detail capturing capability of the depth camera in key areas enables the dynamic path planning to respond more accurately to changes in crack distribution characteristics. Compared with the static planning method of related technologies, this path dynamic adjustment capability significantly improves the adaptability of the inspection task, realizes one-time inspection, and the inspection process has coarse and fine, full coverage and targeted millimeter-level fine crack inspection in key areas, avoiding the inefficient problems of insufficient coverage or repeated path, while ensuring the inspection efficiency. Through the cooperative work of the laser radar and the depth camera, the global and local data are organically combined, significantly improving the accuracy of crack detection and the integrity of terrain data. Through data fusion to support dynamic path planning, the inefficient problems of insufficient path coverage and repeated inspection in related technologies are effectively solved. At the same time, through dynamic adjustment of scanning resolution and smooth resource allocation, the limitation of high power consumption on the endurance capability of the unmanned aerial vehicle is overcome, so that the inspection task can balance the inspection efficiency and crack monitoring accuracy in complex scenes.

[0031] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0032] The drawings herein are incorporated into the specification and form part of the specification, show embodiments consistent with the present disclosure, and together with the specification serve to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor.

[0033] Figure 1A schematic diagram shows a system architecture of an exemplary application environment in which a slope crack inspection method and apparatus for use in a high mountain canyon area according to an embodiment of the present disclosure can be applied.

[0034] Figure 2 A schematic flow chart of a slope crack inspection method for a high mountain canyon area according to some embodiments of the present disclosure is shown schematically.

[0035] Figure 3 The following schematically illustrates a flow chart of determining fused three-dimensional terrain data according to some embodiments of the present disclosure.

[0036] Figure 4 The following schematically illustrates a flow chart of generating a local terrain change area according to some embodiments of the present disclosure.

[0037] Figure 5 The following schematically illustrates a flow chart of constructing target key feature point pairs according to some embodiments of the present disclosure.

[0038] Figure 6 The following schematically illustrates a flow chart of generating a fine inspection route for local cracks according to some embodiments of the present disclosure.

[0039] Figure 7 The following schematically illustrates a flow chart of obtaining a crack acquisition flight attitude by reasoning according to some embodiments of the present disclosure.

[0040] Figure 8 The flowchart of the reinforcement learning training process of the crack inspection route planning model according to some embodiments of the present disclosure is schematically shown.

[0041] Figure 9 A schematic diagram of a route for adaptive dynamic route inspection of a dam structure according to some embodiments of the present disclosure is schematically shown.

[0042] Figure 10 A schematic diagram of a route for slope-adaptive dynamic route inspection according to some embodiments of the present disclosure is schematically shown.

[0043] Figure 11 A schematic diagram of a slope crack inspection device for use in a high mountain canyon area according to some embodiments of the present disclosure is schematically shown.

[0044] Figure 12 A schematic structural diagram of a computer system of an electronic device according to some embodiments of the present disclosure is schematically shown.

[0045] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts. DETAILED DESCRIPTION

[0046] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with this specification. Rather, they are merely examples of apparatus and methods consistent with certain aspects of this specification, as detailed in the appended claims.

[0047] Furthermore, the drawings are schematic illustrations only and are not necessarily drawn to scale. The block diagrams shown in the drawings are merely functional entities and do not necessarily correspond to physically separate entities. In other words, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0048] Figure 1 A schematic diagram shows a system architecture of an exemplary application environment in which a slope crack inspection method and apparatus for use in a high mountain canyon area according to an embodiment of the present disclosure can be applied.

[0049] like Figure 1 As shown, the system architecture 100 may include a drone device 101, a network 102, and a server 103. The network 102 is used to provide a medium for a communication link between the drone device 101 and the server 103. The network 102 may include various connection types, such as wired or wireless communication links or fiber optic cables. The drone device 101 may be a remote-controlled flying device equipped with an image acquisition unit, including but not limited to various types of drones. The embodiment of the present disclosure does not specifically limit the type of the drone device 101. It should be understood that Figure 1 The number of drone devices, networks, and servers shown in the figure is merely illustrative. Any number of drone devices, networks, and servers may be used as needed. For example, server 103 may be a server cluster consisting of multiple servers.

[0050] The slope crack inspection method for high mountain canyon areas provided in the embodiment of the present disclosure can be executed by the drone device 101, and accordingly, the slope crack inspection device for high mountain canyon areas is generally installed in the drone device 101. However, it is easy for those skilled in the art to understand that the slope crack inspection method for high mountain canyon areas provided in the embodiment of the present disclosure can also be executed by the server 103, and accordingly, the slope crack inspection device for high mountain canyon areas can also be installed in the server 103, and this is not particularly limited in this exemplary embodiment.

[0051] In the example embodiment, first provided is a slope crack inspection method for high mountain and canyon areas, which can be applied to a UAV device or a server. The following takes the server as an example to illustrate the method. Figure 2 A flowchart of a slope crack inspection method for high mountain and canyon areas according to some embodiments of the present disclosure is schematically shown. Referring to Figure 2 As shown, the slope crack inspection method for high mountain and canyon areas can include the following steps:

[0052] In step S210, the inspection UAV is controlled to fly along a preset initial inspection route to collect fusion three-dimensional terrain data corresponding to the dam slope of the high mountain and canyon area in real time at a first scanning resolution, the fusion three-dimensional terrain data being collected by a laser radar and a depth camera carried by the inspection UAV;

[0053] In step S220, the fusion three-dimensional terrain data is locally matched with pre-constructed reference three-dimensional terrain data to determine a local terrain change area;

[0054] In step S230, the fusion three-dimensional terrain data corresponding to the local terrain change area is input into a pre-trained crack inspection route planning model to obtain a local crack fine inspection route;

[0055] In step S240, the inspection UAV is controlled to perform adaptive route dynamic planning based on the local crack fine inspection route to perform fine inspection of the crack terrain of the local terrain change area at a second scanning resolution to obtain millimeter-level crack inspection data.

[0056] According to the slope crack inspection method for high mountain and canyon areas in the example embodiment, through the cooperative use of the laser radar and the depth camera, the laser radar can obtain three-dimensional terrain elevation data in a wide range, providing a basis for accurate description of overall terrain features, while the depth camera can provide high-resolution three-dimensional point cloud data in the local crack area, making up for the deficiency of the laser radar in micro-feature capture capability. The fusion of the data of the two can significantly improve the detail information of the crack area, while ensuring the integrity and consistency of the global terrain data, overcoming the one-sidedness of the data caused by excessive reliance on the laser radar sensor in the related art. During the data collection process, the depth camera can significantly improve the detection accuracy of key features such as crack width and depth through fine scanning of the crack area, and in combination with the global terrain data collected by the laser radar, the depth camera provides more detailed crack information for crack analysis and expansion trend prediction, laying a reliable data foundation. In addition, by dynamically adjusting the scanning resolution of the laser radar and the depth camera, the limited power energy of the unmanned aerial vehicle can be flexibly allocated in different areas. In particular, by increasing the scanning resolution in key areas while reducing the redundant collection in non-key areas, the inspection efficiency is optimized and the energy consumption of the unmanned aerial vehicle is reduced, thereby ensuring the endurance time of the unmanned aerial vehicle while improving the inspection efficiency of the unmanned aerial vehicle for the dam slope. Through comprehensive processing of the fusion data of the laser radar and the depth camera, local terrain change areas can be marked in real time, and the flight path and flight attitude of the inspection unmanned aerial vehicle are dynamically adjusted. The global terrain information provided by the laser radar ensures the coverage and safety of the path planning, while the detail capture capability of the depth camera in the key area enables the dynamic path planning to more accurately respond to changes in crack distribution features. Compared with the static planning mode of the related art, this dynamic path adjustment capability significantly improves the adaptability of the inspection task, avoiding the inefficient problems of insufficient coverage or path repetition, while ensuring the inspection efficiency, effectively improving the accuracy and integrity of the crack data collected; through the cooperative work of the laser radar and the depth camera, the global and local data are organically combined, significantly improving the accuracy of crack detection and the integrity of terrain data; through data fusion to support dynamic path planning, the inefficient problems of insufficient path coverage and repeated inspection in the related art are effectively solved; at the same time, through dynamic adjustment of the scanning resolution and smooth resource allocation, the limitation of high power consumption on the endurance capability of the unmanned aerial vehicle is overcome, so that the inspection task can balance the inspection efficiency and crack monitoring accuracy in complex scenes.

[0057] In the following, the slope crack inspection method for high mountain and canyon areas in the example embodiment will be further described.

[0058] In step S210, the inspection drone is controlled to follow a preset initial inspection route to collect fused three-dimensional terrain data corresponding to the dam slope in the high mountain canyon area in real time at a first scanning resolution. The fused three-dimensional terrain data is collected by the laser radar and depth camera carried by the inspection drone.

[0059] In an example embodiment of the present disclosure, the initial inspection route refers to a drone cruise route for inspecting dam slopes in high mountain canyon areas that is set before the start of this inspection mission. For example, the initial inspection route can be an ideal drone cruise route that is manually set based on the topographical and environmental characteristics of the dam slopes in high mountain canyon areas and experience; the initial inspection route can also be a historical drone cruise route that was ultimately adopted and used by the drone in the last inspection mission; of course, the initial inspection route can also be a drone cruise route constructed in other ways before the start of this inspection mission, and this example embodiment does not specifically limit this. The construction process of the initial inspection route can include steps such as path smoothing, obstacle avoidance, and waypoint optimization to ensure the safety, rationality, and coverage of the route.

[0060] The first scan resolution refers to the scanning parameters used by the LiDAR and depth camera during the initial inspection route. It is typically set to a relatively balanced resolution to balance energy consumption and terrain data coverage quality. For example, the LiDAR can be set to scan a certain number of points per second, and the depth camera can be set to capture point clouds at a certain frame rate. The first scan resolution can be set based on the terrain complexity of the target area and the expected distribution of cracks, choosing a lower resolution to cover a larger area and avoid unnecessary data redundancy.

[0061] Real-time data collection involves the lidar and depth camera performing data collection tasks simultaneously as the inspection drone flies along its initial inspection route. The lidar generates three-dimensional elevation data of the dam slope by emitting laser beams and measuring the return time. The depth camera actively or passively captures high-precision three-dimensional point cloud data of localized areas, complementing the lidar's shortcomings in capturing crack details. During the data collection process, the drone must maintain a stable flight attitude to reduce errors caused by sensor data jitter. Specifically, the drone's built-in inertial navigation system (INS) and global navigation satellite system (GNSS) can be used to adjust its flight attitude and correct its positioning in real time.

[0062] Fused three-dimensional terrain data refers to complete terrain data generated by registering and fusing two different types of terrain data collected by the LiDAR and the depth camera. Registration refers to aligning the three-dimensional elevation data of the LiDAR with the point cloud data of the depth camera so that the two can express the spatial characteristics of the target area in the same coordinate system. The fusion process can adopt an alignment method based on feature points. For example, key feature points (such as corner points and edge points) in the LiDAR and depth camera data can be extracted for registration. Alternatively, a method based on a probability model can be used to complete the fusion by maximizing the spatial similarity of the two types of data. This example embodiment does not limit the registration and fusion methods. The fused three-dimensional terrain data has both the global information provided by the LiDAR and the local details provided by the depth camera, providing high-precision basic terrain information for subsequent inspection tasks.

[0063] Optionally, the LiDAR's scanning resolution and the depth camera's point cloud acquisition frequency can be dynamically adjusted based on actual inspection needs. For example, the scanning resolution can be reduced when flying over flat areas to save energy, while the scanning resolution can be increased in complex areas or areas with dense cracks to obtain higher-precision data. Furthermore, the registration process for fused 3D terrain data can be performed using an improved iterative closest point (ICP) algorithm or a deep learning-based registration network to improve the robustness and efficiency of data fusion.

[0064] In step S220, the fused three-dimensional terrain data is locally matched with pre-constructed reference three-dimensional terrain data to determine a local terrain change area.

[0065] In one exemplary embodiment of the present disclosure, the pre-built reference 3D terrain data refers to a 3D terrain model of the dam slope in a high mountain valley region, generated before the inspection mission begins based on historical inspection data or terrain mapping data. This data can be generated from the accumulated results of long-term monitoring or a one-time large-scale mapping exercise, and is used for comparison with the real-time fused 3D terrain data.

[0066] Local matching involves aligning the features of fused 3D terrain data collected in real time with reference 3D terrain data in a local area to identify areas of terrain change. During the matching process, key feature points of the two sets of data must first be extracted. For example, terrain mutation points can be extracted using the Harris corner detection algorithm, or crack edge points can be extracted using curvature calculation methods. Then, the matching can be completed in two steps: coarse registration and fine registration. Coarse registration is used to quickly establish a rough correspondence between the two sets of data, for example, through nearest neighbor search or global feature-based matching methods. Fine registration can use the iterative closest point (ICP) algorithm or other optimization methods to further reduce matching errors by adjusting the rotation matrix and translation vector.

[0067] The determination of local terrain change areas can be based on the local minimum mean square error calculated during the matching process. If the error in certain local areas is greater than a preset terrain error threshold, it can be determined that terrain changes may have occurred in that area. These areas may include areas of crack expansion, areas showing signs of landslides, or areas with significant terrain changes due to environmental changes. The error threshold can be adjusted based on historical change trends and terrain complexity to improve the sensitivity and accuracy of the determination. The specific setting can be customized based on actual usage, and this embodiment is not limited to this.

[0068] In terms of implementation, reference 3D terrain data can be generated using a global terrain scan combined with differential analysis of multiple data periods. The matching algorithm can select either a rigid transformation model or a non-rigid deformation model based on the terrain characteristics of the target area. Furthermore, the localized change areas can be further refined into categories such as cracks, landslides, and water erosion to support targeted inspection strategies.

[0069] In step S230, the fused three-dimensional terrain data corresponding to the local terrain change area is input into a pre-trained crack inspection route planning model to obtain a local crack fine inspection route.

[0070] In one exemplary embodiment of the present disclosure, a local terrain change region refers to a target area identified as potentially containing cracks or other terrain anomalies, determined by locally matching fused 3D terrain data with reference 3D terrain data, combined with a local minimum mean square error and a change threshold. This region encompasses complex areas with concentrated cracks, sudden steep terrain drops, or signs of landslides. The fused 3D terrain data corresponding to this local terrain change region refers to the 3D terrain data confined to these areas. By fusing the global terrain information from the LiDAR with the detailed point cloud data from the depth camera, this provides high-quality input data for planning detailed local crack inspection routes.

[0071] The crack inspection route planning model is a path optimization model trained through reinforcement learning. Its core principle is to dynamically generate the optimal inspection route using input terrain data features. The state space of the crack inspection route planning model can include the three-dimensional terrain features of the local terrain change area, the current flight parameters of the inspection drone (including position, speed, and altitude), and historical flight path information, describing the execution status of the drone's current mission. The action space defines the inspection adjustment actions that the model can take, including altitude adjustment, heading adjustment, and scanning resolution selection, to dynamically optimize the route. The model's reward function is designed based on indicators such as crack area coverage, inspection path smoothness, and energy optimization to maximize crack capture completeness and inspection efficiency.

[0072] When the fused three-dimensional terrain data of the local terrain change area is input into the crack inspection route planning model, the crack inspection route planning model can first extract terrain feature parameters, including terrain elevation, gradient, crack direction, and crack width distribution, and construct a state space vector according to these parameters. Subsequently, through the inference process of the crack inspection route planning model, the action with the maximum cumulative reward value is gradually selected, the flight position and attitude of the inspection unmanned aerial vehicle are updated, and the corresponding inspection path point is recorded. When the inference process of the crack inspection route planning model ends, the generated inspection route covers the crack features of the local terrain change area and provides optimal path support for subsequent high-precision detection.

[0073] The path planning method based on deep reinforcement learning can predict the best action of the unmanned aerial vehicle by using a deep Q network (DQN) model, for example, or can directly optimize the cumulative reward of the inspection task by using a policy gradient-based method. In addition, the training data of the crack inspection route planning model can be derived from the actual flight path and terrain change annotation data of historical inspection tasks, or can be generated by simulation in a simulation environment. The present example embodiment does not specially limit the source of the training data of the crack inspection route planning model.

[0074] In step S240, the inspection unmanned aerial vehicle is controlled based on the local crack fine inspection route to perform fine inspection on the crack terrain of the local terrain change area at a second scanning resolution, to obtain millimeter-level crack inspection data.

[0075] In an example embodiment of the present disclosure, the local crack fine inspection route refers to an unmanned aerial vehicle inspection path optimized and generated in combination with the characteristics of the local terrain change area and the crack distribution features, and focuses on the high change area where the cracks are located. The second scanning resolution refers to a high-resolution scanning parameter designed for the crack area, which is usually higher than the first scanning resolution, and is used to obtain fine feature data of the cracks. For example, under the second scanning resolution, the laser radar can be adjusted to a higher scanning point density, and the depth camera can improve the frame rate and resolution to ensure clear capture of the crack edges in the point cloud data.

[0076] Fine inspection refers to high-precision crack detection for the local change area based on the initial inspection. When the inspection unmanned aerial vehicle flies along the local crack fine inspection route, the flight attitude and scanning parameters are adjusted in real time to adapt to the terrain undulations and crack distribution features. Millimeter-level data collection of the cracks includes detailed measurement of the width, depth, direction, and spatial position, and the data is derived from high-precision point cloud of the depth camera and fine elevation information provided by the laser radar. During data collection, the unmanned aerial vehicle uses its inertial navigation system and real-time positioning module to ensure flight stability, and dynamically avoids possible obstacles by using an obstacle avoidance algorithm.

[0077] Millimeter-level crack inspection data ultimately generates a high-resolution three-dimensional crack model, providing a detailed description of the crack morphology, location, and characteristics. These data can be further used to predict crack expansion trends or assess the health of slope structures.

[0078] Optionally, scanning parameters for detailed inspections can be adjusted based on actual needs. For example, in areas with clear crack features, resolution can be appropriately reduced to save energy. In areas with complex terrain or blurred crack boundaries, scanning density can be increased to improve detection accuracy. Furthermore, inspection drones can be equipped with additional auxiliary sensors (such as thermal imagers or ultrasonic sensors) for multimodal detection of crack features, further enhancing the reliability and applicability of inspection results.

[0079] Next, steps S210 to S240 are described in detail.

[0080] In an exemplary embodiment of the present disclosure, Figure 3 The steps in step S210 are used to collect the fused 3D terrain data corresponding to the dam slope in the high mountain canyon area in real time at the first scanning resolution, referring to Figure 3 Specifically, it may include:

[0081] Step S310, obtaining terrain elevation data and terrain structure point cloud data corresponding to the dam slope in the high mountain canyon area collected in real time by the laser radar and the depth camera at a first scanning resolution respectively;

[0082] Step S320, registering and aligning the terrain elevation data and the terrain structure point cloud data to obtain preliminary three-dimensional terrain data;

[0083] Step S330 , performing deep fusion on the preliminary three-dimensional terrain data to obtain fused three-dimensional terrain data corresponding to the dam slope in the high mountain canyon area.

[0084] LiDAR collects terrain elevation data, which is three-dimensional point data calculated from the time between laser beam emission and return. This data reflects the elevation differences and overall contours of the dam slopes in mountainous and canyon areas. This data is based on the laser ranging formula, which measures distance based on the speed of light and half the time difference between laser emission and return. By analyzing the laser beam scanning angle, 3D point cloud data is generated.

[0085] Terrain structure point cloud data collected by depth cameras refers to a three-dimensional point cloud generated using active lighting or passive stereo vision methods, used to capture geometric details of local areas. Depth cameras operate using various principles, including structured light, time-of-flight (ToF), and stereo matching. For example, structured light projects a specific grating pattern and calculates its deformation to generate a depth value for each pixel. Compared to lidar data, the point cloud data generated by depth cameras has higher local accuracy and is suitable for capturing microscopic features such as cracks.

[0086] First scan resolution refers to the pre-set data acquisition parameters for the LiDAR and depth camera, which are used to balance the requirements of coverage and detail capture. For example, the LiDAR can be set to a medium point density (e.g., 50,000 to 100,000 points per second), while the depth camera frame rate can be set to 30 to 60 frames per second to meet the real-time and resolution requirements in complex terrain. In practice, the LiDAR and depth camera operate in a synchronized acquisition mode, combining the drone's inertial navigation system (INS) and global navigation satellite system (GPS) to record the geographic coordinates of the acquisition points in real time.

[0087] Optionally, the LiDAR can include a multi-line LiDAR to increase point cloud coverage density; or a high-frame-rate depth camera can be used to enhance local crack detection capabilities. The first scan resolution can also be dynamically adjusted to meet different mission requirements. For example, the first scan resolution of the LiDAR or depth camera in complex terrain areas can be higher than that set in flat areas.

[0088] Registration involves aligning the terrain elevation data from the LiDAR and the point cloud data from the depth camera into a unified 3D coordinate system through geometric transformation. This process aims to minimize the error between the two sets of point cloud data. This is achieved by using a rigid body transformation model, a rotation matrix, and a translation vector to express the alignment relationship. This error can then be gradually optimized through the Iterative Closest Point (ICP) algorithm, ultimately completing the data alignment.

[0089] Specifically, the registration process can extract terrain edges and steep points from the LiDAR data, crack boundaries and high-curvature feature points from the depth camera data, and then use the global characteristics of the point cloud to preliminarily estimate the relationship between point pairs. For example, through nearest neighbor search or normal-direction matching, the error between point clouds is gradually optimized based on the ICP algorithm, and the aligned point cloud data is output. Alternatively, a deep learning-based point cloud registration network (such as PointNetLK) can be used for fully automatic registration, or a multi-stage registration process can be used to perform global alignment followed by local fine-grained alignment.

[0090] Deep fusion refers to further processing of the data of the laser radar and the depth camera on the basis of registration alignment to generate a three-dimensional terrain model with consistency and enhanced details. The core lies in data weighting fusion and spatial reconstruction. Deep fusion can be to assign different weights to the point cloud data according to the characteristics of the sensors and the regional requirements. For example, laser radar data is preferentially used in wide-area terrain, and depth camera data is preferentially used in crack details; the abnormal points in the fused point cloud are filtered out, for example, using a statistical filtering method to remove outliers or using a Gaussian filtering method to smooth noise; the fused point cloud is subjected to multi-resolution processing, for example, using an octree segmentation method to increase the resolution in local detail areas and reduce the resolution in global areas to reduce the data volume; and the boundaries and overlapping areas of the fused point cloud are subjected to smoothing processing to ensure the overall consistency of the model.

[0091] The finally obtained fused three-dimensional terrain data contains the advantages of global modeling of the laser radar and the strong points of detail capture of the depth camera, and provides a comprehensive and high-precision terrain model for subsequent inspection tasks; alternative implementation manners include implementing data fusion by using a probabilistic graph model (such as Bayesian Fusion) or performing point cloud reconstruction by using a three-dimensional voxel-based deep fusion framework (such as a TSDF algorithm), and the present example embodiment does not specially limit this.

[0092] By using the laser radar and the depth camera to collect terrain data at a first scanning resolution and combining registration and deep fusion technology, fused three-dimensional terrain data with global modeling capability and detail expression capability can be generated in complex terrain. On the one hand, the wide-area coverage characteristic of the laser radar ensures the integrity of terrain modeling; on the other hand, the high-resolution acquisition of the depth camera in local areas makes up for the deficiency of the laser radar in capturing micro features. At the same time, through registration alignment and deep fusion, the problem of insufficient consistency of multi-source data is solved, high-precision terrain basic information is provided for subsequent inspection tasks, and the inspection efficiency and the reliability of crack detection are effectively improved.

[0093] In an example embodiment of the present disclosure, the step of locally matching the fused three-dimensional terrain data with the pre-constructed reference three-dimensional terrain data to determine the content of the local terrain change area in step S220 can be implemented by the steps in Figure 4 Reference is made to FIG. 4, and specifically can include the following steps. Figure 4 Step S410, extracting a first key feature point in the fused three-dimensional terrain data and extracting a second key feature point in the reference three-dimensional terrain data;

[0094] Step S420, coarsely registering the first key feature point and the second key feature point to determine a preliminary key feature point pair;

[0095] Step S430, performing fine registration on the preliminary key feature point pair to determine a final key feature point pair.

[0096] Step S430, iteratively optimizing the preliminary key feature point pairs to determine target key feature point pairs, and determining at least one local sub-region based on the target key feature point pairs, wherein the local sub-region includes a steep region, a gentle slope region, and a crack distribution region;

[0097] Step S440, determining the local minimum mean square error corresponding to each of the local sub-regions according to the target key feature point pairs;

[0098] Step S450 : taking the local sub-region where the local minimum mean square error is greater than or equal to a preset terrain error threshold as the local terrain change region.

[0099] The first key feature points refer to a set of points extracted from the fused 3D terrain data that can represent the characteristics of target terrain changes and are used for comparison with the feature points in the reference 3D terrain data. The extraction of the first key feature points is usually based on geometric characteristic analysis, mainly including corner points, edge points, and points with high curvature. The extraction method can use the Harris corner detection algorithm, which calculates the change in grayscale gradient in the pixel window. The response function that defines the corner point can be expressed by the following relationship:

[0100] ;

[0101] ;

[0102] Among them, R can represent the corner point response value. When the corner point response value is greater than or equal to the threshold, it can be judged as a corner point. can represent the gradient matrix, and It can represent the gradient of image grayscale in the x and y directions. It can represent an empirical constant, usually in the range of 0.04≤ ≤0.06, The gradient matrix determinant can be expressed as can represent the gradient matrix trace.

[0103] The second key feature points are a representative set of points extracted from the reference 3D terrain data and used to match the first key feature points. The extraction method is similar to that for the first key feature points, and their distribution can reflect the primary structural characteristics of the terrain. In practice, either a corner point extraction algorithm or a curvature-based feature point extraction algorithm can be applied to the two sets of data, respectively. The extracted feature points are stored as a point set for subsequent registration.

[0104] Coarse registration refers to establishing a preliminary key feature point pair by roughly aligning two sets of point clouds, so that the first key feature points and the second key feature points roughly correspond. For example, for each point in the first key feature point set, the point in the second key feature point set closest in Euclidean distance can be found as the matching point. The matching process can be represented by the following relationship:

[0105]

[0106] may represent the coordinates of point i in the first key feature point set, may represent the point in the second key feature point set closest to may represent the second key feature point set, may represent solving the Euclidean distance. Further, a preliminary rotation matrix and translation vector can be calculated based on the matching point pair to complete the rough alignment of the two sets of point clouds, with the goal of minimizing the mean square error (MSE) of the preliminary key feature point pair. When the error change is less than a set threshold, the iteration is stopped; otherwise, return to the first step to update the point pair. After optimization is completed, the boundaries of local sub-regions are determined according to the distribution of the target key feature point pair. These sub-regions can be classified into steep regions, gentle slope regions, and crack distribution regions according to point cloud density, curvature, and other characteristics.

[0107] Iterative optimization uses a fine registration algorithm (such as the ICP algorithm) to minimize the error by gradually adjusting the rotation matrix and translation vector. Specifically, on the basis of the preliminary alignment, the point pair relationship is updated. For each point in the first key feature point set, the closest point in the second key feature point set is found, and based on the new point pair, the rotation matrix and translation vector are optimized. The process of optimizing the rotation matrix and translation vector can be represented by the following relationship:

[0108]

[0109] may represent a 3x3 rotation matrix, may represent a 3x1 translation vector, may represent a point in the initial point cloud, may represent a point after rotation and translation transformation; the goal is to minimize the mean square error of the preliminary key feature points, which can be represented by the following relationship:

[0110]

[0111] may represent the mean square error of the preliminary key feature points, may represent a point after rotation and translation transformation, ​​​​​​​can represent matching points in the reference point cloud, It can represent the number of matching point pairs. After optimization, the target key feature point pairs are obtained. Then, the boundaries of the local sub-region can be determined based on the distribution of the target key feature point pairs. The local sub-region can be classified into steep areas, gentle slope areas, and crack distribution areas based on characteristics such as point cloud density and curvature.

[0112] The local minimum mean square error (LMSE) is the matching error between pairs of target key feature points within a local subregion. When the LMSSE exceeds a preset terrain error threshold, the subregion is determined to be a local terrain change region. The preset terrain error threshold can be set based on historical terrain change statistics or task requirements, and this embodiment does not impose any specific restrictions on this threshold.

[0113] Through key feature point extraction, point pair matching, and iterative optimization techniques, the system efficiently and accurately completes local matching between the fused 3D terrain data and the reference 3D terrain data. The extracted key feature points ensure matching accuracy; the calculation of the local minimum mean square error and the determination of change areas enable precise identification of local terrain changes. The system demonstrates robustness in determining steep areas, gentle slopes, and crack distribution areas, providing a basis for precise area marking and path planning for subsequent inspection tasks.

[0114] Optionally, you can pass Figure 5 The steps in the above are used to iteratively optimize the preliminary key feature point pairs and determine the target key feature point pairs. Figure 5 Specifically, it may include:

[0115] Step S510, determining a rotation matrix and a translation vector based on the preliminary key feature point pair, wherein the rotation matrix and the translation vector minimize the mean square error between the matching point pairs;

[0116] Step S520, performing an update transformation on the first key feature point in the preliminary key feature point pair using the rotation matrix and the translation vector, and determining a mean square error between the first key feature point and the second key feature point after the update transformation;

[0117] Step S530, iteratively update the first key feature point in the preliminary key feature point pair until the mean square error is less than a preset matching error threshold, stop iteration, and construct a target key feature point pair based on the latest transformed first key feature point and the second key feature point.

[0118] The rotation matrix is ​​a three-dimensional spatial transformation tool used to adjust the orientation of the first key feature point set to align with the orientation of the second key feature point set, while the translation vector is used to correct the displacement deviation between the two point clouds. By analyzing the distribution characteristics of the initial key feature point pairs, the coordinates of the center points of the two point clouds can be calculated and the data can be decentralized based on the center points. The purpose of decentralization is to eliminate the interference of global offsets on rotation and translation calculations, thereby focusing on the relationship between the local features of the point clouds.

[0119] By comparing the directional differences between the point clouds, a rotation matrix can be preliminarily calculated to adjust the directional consistency of the two point clouds. The calculation of the rotation matrix is ​​based on the geometric distribution of the key feature points of the two point clouds. For example, by analyzing the spatial vectors between the feature points, the rotation parameters that minimize the directional differences of the point clouds can be found. Simultaneously, the calculation of the translation vector is based on the repositioning of the decentralized data. By adjusting the positional deviation of the point clouds, the center points of the two point clouds are ensured to coincide.

[0120] In specific implementations, the calculation of the rotation matrix and translation vector can be performed through an iterative optimization method. The goal of iterative optimization is to gradually adjust the rotation and displacement parameters so that the matching error between the two sets of point clouds gradually decreases and eventually reaches a preset error threshold. This process can significantly improve the accuracy of point cloud data alignment and provide high-quality initial results for subsequent steps.

[0121] The first key feature point in the preliminary key feature point pair is updated using a rotation matrix and translation vector, and the error between the updated first key feature point and the second key feature point is determined. The essence of the update transformation is to apply the rotation matrix and translation vector to the first key feature point set, adjusting the spatial position of each feature point to bring it closer to the second key feature point. After the feature point update is completed, the spatial error between each pair of matching points is calculated to evaluate the optimization effect of the current parameters. If the error is still higher than the preset threshold, it is necessary to further adjust the rotation matrix and translation vector, and repeat the feature point update and error calculation process.

[0122] The update transformation of feature points can be performed through batch processing, that is, the rotation and translation operations are applied to the entire feature point set at the same time, thereby improving processing efficiency; for error calculation, the partition evaluation method can better reflect the alignment of local areas of the point cloud. For example, the feature points can be divided into crack areas and non-crack areas, and the errors of the two types of areas can be calculated separately to provide a reference for the optimization of specific areas.

[0123] The first key feature point in the preliminary key feature point pair can be iteratively updated until the error is less than the preset matching error threshold. The iteration is stopped and the target key feature point pair is constructed based on the latest transformed first and second key feature points to ensure that the final accuracy of the point cloud alignment meets the task requirements. The iterative update process can include re-matching feature point pairs, calculating new rotation matrices and translation vectors, updating feature point sets, and evaluating the current error. After each iteration, the accuracy of the point cloud alignment is further optimized by dynamically adjusting the transformation parameters. The iterative process stops when the error meets the preset threshold to avoid wasting computing resources due to over-optimization.

[0124] The preset matching error threshold can be flexibly adjusted based on task requirements. For example, for crack detection tasks requiring millimeter-level inspections, the error threshold is typically set to the millimeter level to ensure the reliability of the detection results. After the iteration is completed, each pair of corresponding points between the first and second key feature points is used as a target key feature point pair based on the final matching results. This pair is then used for subsequent path planning or crack detail analysis.

[0125] By determining the rotation matrix and translation vector, spatial alignment of point cloud data is achieved, so that the two sets of point clouds can be accurately matched in the same coordinate system, thus providing high-quality initial conditions for subsequent feature point optimization; the application of feature point update transformation further improves the accuracy of point cloud alignment, and significantly enhances the description ability of local features by gradually optimizing error parameters; the iterative update mechanism ensures the stability and convergence of the point cloud alignment process, and can achieve the optimal alignment state after multiple adjustments; in addition, through the dynamic evaluation of feature point matching errors, it is possible to accurately identify terrain areas with significant feature changes in high mountain canyon areas, providing reliable spatial data support for crack detection and inspection route planning.

[0126] In an exemplary embodiment of the present disclosure, Figure 6 The steps in the above are to input the fused 3D terrain data corresponding to the local terrain change area into the pre-trained crack inspection route planning model to obtain the local crack fine inspection route. Figure 6 Specifically, it may include:

[0127] Step S610, extracting terrain characteristic parameters from the fused three-dimensional terrain data corresponding to the local terrain change area;

[0128] Step S620, constructing a state space vector according to the terrain characteristic parameters and the initial inspection route;

[0129] Step S630, inputting the state space vector into the crack inspection route planning model to determine the crack collection inspection points of the local terrain change area and the crack collection flight attitude of the inspection unmanned aerial vehicle corresponding to each crack collection inspection point;

[0130] Step S640, constructing a local crack fine inspection route based on the crack collection inspection points and the crack collection flight attitude.

[0131] The terrain feature parameters refer to a set of parameters that have key guiding significance for path planning by analyzing the geometric features and spatial distribution information of the local terrain data. For example, the terrain feature parameters can include but are not limited to height variation, slope, surface curvature, crack width, crack direction, and area complexity. The terrain feature parameters are an abstract description of the geometric characteristics and environmental conditions of the local terrain, which can provide necessary input for inspection path optimization.

[0132] The state space vector can be constructed according to the terrain feature parameters and the initial inspection route, specifically referring to integrating the local terrain features and the unmanned aerial vehicle flight parameters into a high-dimensional vector for describing the current inspection state and serving as the input of the path planning. For example, the state space vector can contain key parameters describing the local terrain change, such as height variation, slope, crack direction, etc., and also include the current flight state information of the unmanned aerial vehicle, such as flight height, speed and direction. The purpose of constructing the state space vector is to uniformly express multi-dimensional information and provide sufficient data support for dynamic adjustment of the inspection path.

[0133] In specific implementation, the state space vector can be formed by splicing multi-dimensional data, for example, the height variation can be taken as the first-dimensional data, the slope as the second-dimensional data, the crack width and direction as the third and fourth-dimensional data, and the flight height, speed and direction of the unmanned aerial vehicle as subsequent dimensional data. In order to enhance the perception ability of the model to the state, the dimensional data in the state space vector can be normalized to ensure consistency in the numerical range of each dimension.

[0134] The state space vector can be input into the crack inspection route planning model, aiming to optimize the inspection path of the unmanned aerial vehicle through the crack inspection route planning model, and ensure that the crack detection task can be efficiently and accurately completed in complex terrain. The crack inspection route planning model is a path optimization model based on reinforcement learning, whose input is the state space vector and the output is an action vector for guiding the flight operation of the unmanned aerial vehicle, including adjusting the flight height, direction, speed and scanning resolution of the sensor, etc.

[0135] In the implementation process, the crack inspection route planning model can perform feature analysis on the input state space vector, extract high-dimensional features that have a key impact on path planning, such as the relationship between height change and flight height, the adaptability of slope and flight direction, etc. The crack inspection route planning model can calculate the optimal action vector in the current state according to these features through the strategy network, such as adjusting the flight height in steep areas to avoid obstacles, and increasing the sensor resolution in areas with dense crack distribution to obtain higher-precision data. The selection of the action vector is based on the experience reward value accumulated during the model training process, that is, through repeated trial and error learning, the model can find the optimal strategy to complete the inspection task in complex terrain.

[0136] Optionally, the traditional Dijkstra algorithm can also be used for global planning of the path in complex areas, and the reinforcement learning model can be used for local path optimization.

[0137] By extracting terrain feature parameters and constructing a state space vector, accurate expression of complex terrain features can be achieved, providing complete environmental perception input for the path planning model. The crack inspection route planning model dynamically adjusts the inspection path using the state space vector, which not only optimizes the flight trajectory in complex terrain, but also adjusts the scanning resolution of the sensor and the flight parameters of the unmanned aerial vehicle in real time according to the crack distribution characteristics and terrain changes. The above technical means enables the unmanned aerial vehicle to maintain high-efficiency inspection capability in complex terrain, while significantly improving the accuracy and comprehensiveness of crack detection, overcoming the defects of rigid path planning and insufficient crack capture capability in existing inspection techniques.

[0138] Optionally, the state space vector can be input into the crack inspection route planning model by the steps in Figure 7 , to determine the crack collection inspection points in the local terrain change area and the crack collection flight attitude of the inspection unmanned aerial vehicle corresponding to each crack collection inspection point. As shown in Figure 7 , the specific steps can include:

[0139] Step S710, inputting the state space vector into the crack inspection route planning model to determine the action vector with the maximum cumulative reward value under the state space vector;

[0140] Step S720, updating the inspection position and flight attitude of the inspection unmanned aerial vehicle based on the action vector, and recording the unmanned aerial vehicle inspection path and inspection flight attitude inferred by the crack inspection route planning model;

[0141] Step S730, in response to the local terrain change area being covered or the flight time of the inspection UAV reaching the preset time, confirming that the inspection task reasoning is completed, and determining the crack collection inspection points of the local terrain change area and the crack collection flight attitude of the inspection UAV corresponding to each crack collection inspection point according to the recorded inspection path and flight attitude of the UAV.

[0142] Wherein, the action vector with the maximum cumulative reward value of the state space vector refers to using the pre-trained crack inspection route planning model to analyze and decide in real time for the current inspection state to select the optimal inspection action. The state space vector can include the terrain feature parameters of the local terrain change area and the flight state information of the UAV, such as flight height, speed and direction, etc. The crack inspection route planning model is usually constructed based on reinforcement learning algorithm, which can predict the most favorable action selection strategy in different states by learning a large amount of experience data in inspection tasks.

[0143] In specific implementation, the crack inspection route planning model can receive the input state space vector, extract and process features through multi-layer neural network or other machine learning models, and output an action vector corresponding to the action with the maximum cumulative reward value in the current state, such as adjusting flight height, changing flight direction or modifying scanning resolution, etc. Optionally, different types of reinforcement learning algorithms can be used, such as proximal policy optimization (PPO), deep deterministic policy gradient (DDPG), etc., to adapt to inspection tasks with different complexity and requirements.

[0144] The current position and flight state of the UAV can be adjusted in real time according to the action vector output by the model, so as to optimize the inspection path. Specifically, the instructions contained in the action vector can be analyzed, such as adjusting flight height, changing heading angle or modifying scanning parameters; these instructions can be converted into specific flight actions by using the flight control system of the UAV, so as to realize the required attitude change by adjusting the motor, rudder or other control components of the UAV; at the same time, the system will record the current position, flight direction and attitude change process of the UAV in real time, and generate complete inspection path and flight attitude record. These records can be used for subsequent data analysis, path optimization and task evaluation.

[0145] The terrain change area covered by the inspection path can be analyzed in real time. If all target areas have been covered by the inspection path, the task is determined to be completed. At the same time, the flight time of the drone can be monitored. When the flight time reaches the preset upper limit, the inspection task can be determined to be completed regardless of whether the task is completely covered. After confirming that the task is completed, the location of the crack collection inspection points and the flight attitude information of the drone at these points can be extracted based on the recorded inspection path and flight attitude data. The flight attitude information can be used to generate the final crack detection report or guide subsequent crack analysis work. Optionally, other different task end conditions can be set, such as judging the end of the task based on the signal-to-noise ratio or data quality indicators of the crack detection. In addition, an event-triggered end mechanism can be used. For example, when a specific number or type of cracks are detected, the task is automatically terminated. This example embodiment does not specifically limit this.

[0146] By inputting the state space vector into the crack inspection route planning model and selecting the action vector with the largest cumulative reward value, the intelligent optimization and dynamic adjustment of the UAV inspection path can be achieved; the selection of action vectors is based on real-time terrain characteristics and flight status information, allowing the UAV to flexibly respond to various changes in complex terrain, ensuring the efficiency and comprehensive coverage of the inspection path; the inspection position and flight attitude update based on the motion vector further improves the response speed and adaptability of the UAV during the inspection mission. At the same time, by recording the inspection path and flight attitude, the traceability of the mission data and the accuracy of subsequent analysis are enhanced; by setting the task end condition, the optimization of resource utilization and time management of the inspection task is ensured, and it can cover as many crack areas as possible within the limited flight time, or automatically terminate the task when all coverage is completed; the comprehensive application of these technical means enables UAV inspection to achieve efficient, accurate and reliable crack detection in the complex terrain environment of high mountains and canyons, overcoming the defects of rigid path planning and low inspection efficiency in related technologies.

[0147] In an exemplary embodiment of the present disclosure, the crack inspection route planning model is obtained through a reinforcement learning training process. Figure 8 As shown, the reinforcement learning training process can specifically include:

[0148] Step S810, determining the state space of the crack inspection route planning model, wherein the state space includes the coordinates of the inspection points in the initial inspection route and the flight posture of the UAV, the global terrain data of the dam slope in the alpine canyon area, and the historical flight path information;

[0149] Step S820, determining the action space of the crack inspection route planning model, wherein the action space includes adjusting the flight altitude, adjusting the flight direction, adjusting the flight speed, adjusting the shooting angle of the laser radar and the depth camera, and updating the inspection points;

[0150] Step S830: obtaining inspection interaction data through sampling the state space and the action space, and storing the inspection interaction data in an interaction experience pool, wherein the inspection interaction data is used to simulate the inspection drone performing an inspection task in a dam slope area in a high mountain canyon region;

[0151] Step S840: randomly sample inspection interaction data from the interaction experience pool, and perform model training on the crack inspection route planning model, with the goal of maximizing the cumulative reward value of the reward function of the crack inspection route planning model. The model parameters of the crack inspection route planning model are optimized by gradient descent until the cumulative reward value is greater than or equal to the preset reward value threshold, thereby obtaining a trained crack inspection route planning model.

[0152] The state space is a collection of information used by the reinforcement learning model to describe the current environment and task state. In this case, the state space can include the coordinates of inspection points and the flight attitude of the drone in the initial inspection route, as well as global terrain data for the dam slope in the alpine canyon area and historical flight path information.

[0153] The action space refers to the set of operations that the model can choose in each state. The action space covers various adjustments and operations that the drone may perform during the inspection process. For example, the action space can include adjusting the flight altitude, adjusting the flight direction, adjusting the flight speed, adjusting the shooting angle of the lidar and depth camera, and updating the inspection points.

[0154] Inspection interaction data refers to the record of states and actions generated by the drone's interaction with the environment during the execution of an inspection mission. Inspection interaction data can be used to train reinforcement learning models, enabling them to make better decisions in similar inspection missions. Specifically, by performing various actions in different states, the interaction data between the inspection drone and the environment can be collected. For example, when the drone adjusts its flight altitude, direction, and speed under different terrain conditions, its flight attitude changes, sensor data collection results, and the execution effect of the inspection mission can be recorded. The collected interaction data can then be stored in the interaction experience pool, a database used to store past interaction data. The interaction experience pool can include information such as state, action, reward, and next state. This data will be used for subsequent model training and strategy optimization.

[0155] The sampling of interactive data can be achieved in a variety of ways, such as random sampling, Prioritized Experience Replay, etc., to ensure that the model can fully utilize the diverse inspection task experience during the training process. The design of the experience pool should take into account the storage efficiency and retrieval speed of the data to support the rapid access and processing of large-scale data. Of course, different types of data sampling strategies can also be adopted, such as time series-based sampling or task importance-based sampling, to improve the efficiency and effectiveness of model training. In addition, the storage structure of the experience pool can also be optimized according to specific application requirements, such as using a hash table or tree structure to improve data retrieval speed, but this embodiment is not limited to this.

[0156] Model training is a crucial step in the reinforcement learning process. Its goal is to continuously optimize model parameters so that the model can make optimal decisions during inspection tasks and maximize cumulative rewards. Specifically, a batch of inspection interaction data can be randomly sampled from the interaction experience pool. This data includes feedback information after the drone performs actions in different states, such as reward values ​​and new states. Random sampling helps break the temporal correlation of the data, improving the stability and generalization of training. The extracted inspection interaction data can then be input into the crack inspection route planning model for model training. During training, the model adjusts its parameters to improve prediction accuracy by comparing the error between the actual reward value and the predicted reward value. Common optimization methods include stochastic gradient descent (SGD) and the Adam optimizer to accelerate the convergence of model parameters.

[0157] The training goal is to maximize the model's cumulative reward. This means selecting optimal actions to enable the drone to cover more crack areas, optimize its flight path, and conserve energy during inspection missions. The definition of the cumulative reward can be based on the specific requirements of the inspection mission, such as crack detection coverage, inspection time, and energy consumption. During training, the model iteratively adjusts its parameters, gradually improving the quality and efficiency of its decisions. Training terminates when the cumulative reward reaches or exceeds a preset reward threshold, ensuring that the model has learned an effective inspection path planning strategy.

[0158] Optionally, different types of reinforcement learning algorithms, such as policy gradient methods or value-based algorithms, can be used to adapt to inspection tasks of varying complexity and requirements. Furthermore, parameter optimization methods during training can be adjusted based on specific circumstances, such as using adaptive learning rates or batch normalization techniques, to improve training effectiveness and model convergence speed.

[0159] By defining the state space and action space of the crack inspection route planning model and using sampled inspection interaction data for reinforcement learning training, the model can achieve intelligent optimization and adaptive learning. The comprehensive definition of the state space ensures a deep understanding of the inspection mission environment and the UAV's flight status, while the diversity of the action space enables the model to flexibly adjust the inspection path.

[0160] Through data sampling and model training in the interactive experience pool, the crack inspection route planning model can continuously accumulate and optimize inspection strategies to maximize the cumulative reward value, enabling the model to have a high degree of decision-making ability and adaptability in complex terrain environments, and can respond to terrain changes and crack distribution characteristics in real time, and dynamically adjust the flight parameters and inspection paths of the drone; the trained crack inspection route planning model can not only improve the efficiency and coverage of inspection tasks, but also significantly improve the accuracy and reliability of crack detection, overcoming the defects of rigid inspection path planning and low inspection efficiency in related technologies.

[0161] Figure 9 A schematic diagram of a route for adaptive dynamic route inspection of a dam structure according to some embodiments of the present disclosure is schematically shown.

[0162] refer to Figure 9 As shown, when conducting drone inspections on dam structures, the traditional method is to preset a fixed route 810, and the drone directly inspects the dam structure according to the preset fixed route 810. However, this inspection method is difficult to dynamically adjust the route according to real-time collected terrain data or detection needs, and this static method is prone to insufficient inspection path coverage or low efficiency in complex terrain scenarios, especially in areas where cracks are concentrated, and the fixed path planning method cannot achieve precise capture of crack characteristics.

[0163] In combination with the slope crack inspection method for high mountain canyon areas provided in the embodiments of the present disclosure, the inspection drone equipped with a laser radar and a depth camera can be controlled to follow a preset initial inspection route 820 to collect fused three-dimensional terrain data corresponding to the dam structure in real time at a first scanning resolution, wherein the initial inspection route 820 can be an inspection route determined by the inspection drone during the historical inspection of the dam structure, or it can be an adaptive dynamic route determined in real time by the inspection drone based on data collected by the laser radar and the depth camera; the inspection drone can locally match the collected fused three-dimensional terrain data with the reference three-dimensional terrain data, and determine in real time the local terrain change area 830, such as the local terrain change area 830. 0 can be a collapsed deformation area or a crack area; then the inspection drone can input the fused three-dimensional terrain data corresponding to the local terrain change area 830 into the crack inspection route planning model to obtain a local crack fine inspection route 840; then the inspection drone can combine the local crack fine inspection route 840 to perform adaptive route dynamic planning on the initial inspection route 820, and realize fine inspection of the crack terrain of the local terrain change area 830 with the second scanning resolution to obtain millimeter-level crack inspection data; after completing the inspection of the local terrain change area 830, the inspection drone can return to the remaining initial inspection route 820 to continue the inspection until the inspection of the entire dam structure is completed.

[0164] Figure 10 A schematic diagram of a route for slope-adaptive dynamic route inspection according to some embodiments of the present disclosure is schematically shown.

[0165] refer to Figure 10 As shown, when conducting drone inspections on slopes, the traditional method uses a preset first fixed route 911 and a second fixed route 912, and the drone directly inspects the slopes according to the first fixed route 911 and the second fixed route 912. However, this inspection method makes it difficult to dynamically adjust the route according to the real-time collected slope terrain data or detection needs, resulting in the inability to timely discover the areas that require key attention on the slope terrain, or multiple inspections are required to complete the data collection of all key areas of the slope terrain.

[0166] Combined with the slope crack inspection method for high mountain canyon areas provided in the embodiment of the present disclosure, Figure 9During the inspection of the dam structure, the inspection drone equipped with a lidar and a depth camera can be controlled to collect the fused three-dimensional terrain data corresponding to the slope in real time at a first scanning resolution along the preset first initial inspection route 921 or the second initial inspection route 924; the inspection drone can locally match the collected fused three-dimensional terrain data with the reference three-dimensional terrain data, and determine the first local terrain change area 922 or the second local terrain change area 925 in real time; then the inspection drone can input the fused three-dimensional terrain data corresponding to the first local terrain change area 922 or the second local terrain change area 925 into the crack inspection route planning model, and obtain the first local crack fine inspection route 923 or the second local crack fine inspection route 924. Fine inspection route 926; then the inspection drone can combine the first local crack fine inspection route 923 or the second local crack fine inspection route 926 to perform adaptive route dynamic planning on the first initial inspection route 921 or the second initial inspection route 924, and realize fine inspection of the crack terrain of the first local terrain change area 922 or the second local terrain change area 925 with the second scanning resolution to obtain millimeter-level crack inspection data; after completing the inspection of the first local terrain change area 922 or the second local terrain change area 925, the inspection drone can return to the remaining first initial inspection route 921 or the second initial inspection route 924 to continue inspection until the inspection of the entire slope is completed.

[0167] It should be noted that although the steps of the method disclosed herein are depicted in a particular order in the accompanying drawings, this does not require or imply that the steps must be performed in that particular order, or that all steps must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one, and / or one step may be decomposed into multiple steps.

[0168] In addition, in this exemplary embodiment, a slope crack inspection device for use in mountain valley areas is also provided. Figure 11 As shown, the slope crack inspection device 1100 for high mountain valley areas includes: a three-dimensional terrain acquisition module 1110, a terrain change area determination module 1120, an adaptive route planning module 1130 and a crack fine inspection module 1140.

[0169] The 3D terrain acquisition module 1110 is configured to control the inspection drone to follow a preset initial inspection route and to acquire, in real time, fused 3D terrain data corresponding to the dam slope in the alpine canyon area at a first scanning resolution. The fused 3D terrain data is acquired by the laser radar and depth camera carried by the inspection drone.

[0170] A terrain change region determination module 1120 is configured to locally match the fused 3D terrain data with pre-constructed reference 3D terrain data to determine a local terrain change region;

[0171] The adaptive route planning module 1130 is used to input the fused three-dimensional terrain data corresponding to the local terrain change area into the pre-trained crack inspection route planning model to obtain a local crack fine inspection route;

[0172] The crack fine inspection module 1140 is used to control the inspection drone based on the local crack fine inspection route, perform fine inspection on the crack terrain in the local terrain change area at a second scanning resolution, and obtain millimeter-level crack inspection data.

[0173] In some example embodiments of the present disclosure, based on the aforementioned scheme, the three-dimensional terrain acquisition module 1110 is configured to: obtain the terrain elevation data and terrain structure point cloud data corresponding to the dam slope in the high mountain canyon area, which are respectively collected in real time by the laser radar and the depth camera at the first scanning resolution; align the terrain elevation data and the terrain structure point cloud data to obtain preliminary three-dimensional terrain data; and perform deep fusion on the preliminary three-dimensional terrain data to obtain fused three-dimensional terrain data corresponding to the dam slope in the high mountain canyon area.

[0174] In some example embodiments of the present disclosure, based on the aforementioned scheme, the terrain change area determination module 1120 is configured to: extract a first key feature point from the fused three-dimensional terrain data, and extract a second key feature point from the reference three-dimensional terrain data; perform coarse alignment on the first key feature point and the second key feature point to determine a preliminary key feature point pair; iteratively optimize the preliminary key feature point pair to determine a target key feature point pair, and determine at least one local sub-area based on the target key feature point pair, the local sub-area including a steep area, a gentle slope area, and a crack distribution area; determine the local minimum mean square error corresponding to each of the local sub-areas based on the target key feature point pair; and use the local sub-area whose local minimum mean square error is greater than or equal to a preset terrain error threshold as the local terrain change area.

[0175] In some example embodiments of the present disclosure, based on the aforementioned scheme, the terrain change area determination module 1120 is configured to: determine a rotation matrix and a translation vector based on the preliminary key feature point pair, the rotation matrix and the translation vector minimizing the mean square error between the matching point pairs; update the first key feature point belonging to the preliminary key feature point pair through the rotation matrix and the translation vector, and determine the mean square error between the first key feature point after the updated transformation and the second key feature point; iteratively update the first key feature point in the preliminary key feature point pair until the mean square error is less than a preset matching error threshold, stop the iteration, and construct a target key feature point pair based on the first key feature point and the second key feature point after the latest transformation.

[0176] In some example embodiments of the present disclosure, based on the aforementioned scheme, the adaptive route planning module 1130 is configured to: extract terrain feature parameters from the fused three-dimensional terrain data corresponding to the local terrain change area; construct a state space vector based on the terrain feature parameters and the initial inspection route; input the state space vector into the crack inspection route planning model to determine the crack collection inspection points in the local terrain change area and the crack collection flight posture of the inspection drone corresponding to each of the crack collection inspection points; and construct a local crack fine inspection route based on the crack collection inspection points and the crack collection flight posture.

[0177] In some example embodiments of the present disclosure, based on the aforementioned scheme, the adaptive route planning module 1130 is configured to: input the state space vector into the crack inspection route planning model, and determine the action vector with the largest cumulative reward value under the state space vector; update the inspection position and flight attitude of the inspection drone based on the action vector, and record the drone inspection path and inspection flight attitude inferred by the crack inspection route planning model; in response to the local terrain change area being covered, or the flight time of the inspection drone reaching a preset time, confirm that the inspection task reasoning is completed, and determine the crack collection inspection points in the local terrain change area and the crack collection flight attitude of the inspection drone corresponding to each crack collection inspection point based on the recorded drone inspection path and the inspection flight attitude.

[0178] In some example embodiments of the present disclosure, based on the foregoing scheme, the slope crack inspection device 1100 for high mountain and canyon areas comprises a reinforcement learning training module configured to: determine a state space of the crack inspection route planning model, the state space comprising coordinates of inspection points in an initial inspection flight path and a flight attitude of the unmanned aerial vehicle, global terrain data and historical flight path information of the dam slope in the high mountain and canyon area; determine an action space of the crack inspection route planning model, the action space comprising adjusting the flight height, adjusting the flight direction, adjusting the flight speed, adjusting the shooting angle of the laser radar and the depth camera, and updating the inspection point; sample the inspection interaction data through the state space and the action space, and store the inspection interaction data in an interaction experience pool, the inspection interaction data being used to simulate the inspection unmanned aerial vehicle performing an inspection task in the dam slope area in the high mountain and canyon area; randomly sample the inspection interaction data in the interaction experience pool, and train the crack inspection route planning model to maximize the cumulative reward value of a reward function of the crack inspection route planning model, optimize the model parameters of the crack inspection route planning model through gradient descent until the cumulative reward value is greater than or equal to a preset reward value threshold, and obtain the trained crack inspection route planning model.

[0179] The specific details of the modules of the above-mentioned slope crack inspection device for high mountain and canyon areas have been described in detail in the corresponding slope crack inspection method for high mountain and canyon areas, and therefore will not be described here.

[0180] It should be noted that although several modules or units of the slope crack inspection device for high mountain and canyon areas are mentioned in the foregoing detailed description, such division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into a plurality of modules or units.

[0181] In addition, in the example embodiments of the present disclosure, an electronic device capable of implementing the above-mentioned slope crack inspection method for high mountain and canyon areas is also provided.

[0182] Those skilled in the art can understand that each aspect of the present disclosure can be implemented as a system, a method or a program product. Therefore, each aspect of the present disclosure can be embodied as a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" here.

[0183] The following refers to the accompanying drawings, which are incorporated in the present disclosure and used for purposes of explanation. Figure 121000 according to this embodiment of the present disclosure will be described. Figure 12 The electronic device 1000 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0184] like Figure 12 As shown, electronic device 1000 is implemented as a general-purpose computing device. Components of electronic device 1000 may include, but are not limited to, the aforementioned at least one processing unit 1010, the aforementioned at least one storage unit 1020, a bus 1030 connecting various system components (including storage unit 1020 and processing unit 1010), and a display unit 1040.

[0185] The storage unit stores program codes, which can be executed by the processing unit 1010, so that the processing unit 1010 performs the steps described in the "Exemplary Method" section of the present disclosure according to various exemplary embodiments. For example, the processing unit 1010 can perform the following steps: Figure 2 In step S210 shown in the figure, the inspection drone is controlled to collect fused three-dimensional terrain data corresponding to the dam slope in the high mountain canyon area in real time at a first scanning resolution along a preset initial inspection route, wherein the fused three-dimensional terrain data is collected by the lidar and depth camera carried by the inspection drone; in step S220, the fused three-dimensional terrain data is locally matched with the pre-constructed reference three-dimensional terrain data to determine the local terrain change area; in step S230, the fused three-dimensional terrain data corresponding to the local terrain change area is input into the pre-trained crack inspection route planning model to obtain a local crack fine inspection route; in step S240, the inspection drone is controlled to perform adaptive route dynamic planning based on the local crack fine inspection route, and the crack terrain of the local terrain change area is finely inspected at a second scanning resolution to obtain millimeter-level crack inspection data.

[0186] The storage unit 1020 may include a readable medium in the form of a volatile memory unit, such as a random access memory unit (RAM) 1021 and / or a cache memory unit 1022 , and may further include a read-only memory unit (ROM) 1023 .

[0187] The storage unit 1020 may also include a program / utility 1024 having a set (at least one) of program modules 1025, such program modules 1025 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0188] Bus 1030 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0189] Electronic device 1000 can also communicate with one or more external devices 1070 (e.g., a keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 1000, and / or any device that enables electronic device 1000 to communicate with one or more other computing devices (e.g., a router, modem, etc.). This communication can occur via input / output (I / O) interface 1050. Furthermore, electronic device 1000 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via network adapter 1060. As shown, network adapter 1060 communicates with other modules of electronic device 1000 via bus 1030. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with electronic device 1000, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0190] Through the description of the above embodiments, it will be readily understood by those skilled in the art that the example embodiments described herein can be implemented via software or via a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or mobile hard drive) or on a network and includes several instructions for causing a computing device (such as a personal computer, server, terminal device, or network device) to execute the methods according to the embodiments of the present disclosure.

[0191] Furthermore, the figures above are merely illustrative of the processes included in the methods according to exemplary embodiments of the present disclosure and are not intended to be limiting. It is readily understood that the processes illustrated in the figures above do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0192] Those skilled in the art can easily understand, through the above description of the embodiments, that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.) or a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to perform the methods according to the embodiments of the present disclosure.

[0193] Other embodiments of the present disclosure will be apparent to those skilled in the art with the accomplishment of the present disclosure as set forth in the specification and practice of the invention disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known or customary practice in the art of the present disclosure not specifically disclosed. The specification and examples are to be regarded as illustrative only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0194] It should be understood that the present disclosure is not limited to the precise structures described above and illustrated in the drawings and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the claims appended hereto.

Claims

1. A slope crack inspection method for high mountain canyon areas, characterized in that: include: Controlling the inspection drone to follow a preset initial inspection route and collect, in real time, fused three-dimensional terrain data corresponding to the dam slope in the alpine canyon area at a first scanning resolution, the fused three-dimensional terrain data being collected by a laser radar and a depth camera carried by the inspection drone; Extracting first key feature points from the fused three-dimensional terrain data and extracting second key feature points from the reference three-dimensional terrain data; performing coarse registration on the first key feature points and the second key feature points to determine preliminary key feature point pairs; Iteratively optimize the preliminary key feature point pairs to determine target key feature point pairs, and determine at least one local subregion based on the target key feature point pairs, wherein the local subregions include steep areas, gentle slope areas, and crack distribution areas; determine the local minimum mean square error corresponding to each of the local subregions based on the target key feature point pairs; and define the local subregions for which the local minimum mean square error is greater than or equal to a preset terrain error threshold as local terrain change areas; Inputting the fused three-dimensional terrain data corresponding to the local terrain change area into the pre-trained crack inspection route planning model to obtain a local crack fine inspection route; Based on the local crack fine inspection route, the inspection drone is controlled to perform adaptive route dynamic planning, and the crack terrain in the local terrain change area is finely inspected at a second scanning resolution to obtain millimeter-level crack inspection data.

2. The slope crack inspection method for high mountain canyon areas according to claim 1 is characterized in that: The real-time acquisition of fused three-dimensional terrain data corresponding to the dam slope in the high mountain canyon area at the first scanning resolution includes: Acquire terrain elevation data and terrain structure point cloud data corresponding to the dam slope in the high mountain canyon area collected in real time by the laser radar and the depth camera at a first scanning resolution respectively; Aligning the terrain elevation data with the terrain structure point cloud data to obtain preliminary three-dimensional terrain data; The preliminary three-dimensional terrain data is deeply fused to obtain fused three-dimensional terrain data corresponding to the dam slope in the high mountain canyon area.

3. The slope crack inspection method for high mountain canyon areas according to claim 1 is characterized in that: The iterative optimization of the preliminary key feature point pairs to determine target key feature point pairs includes: Determining a rotation matrix and a translation vector based on the preliminary key feature point pairs, wherein the rotation matrix and the translation vector minimize a mean square error between the matching point pairs; Performing an update transformation on a first key feature point in the preliminary key feature point pair using the rotation matrix and the translation vector, and determining a mean square error between the first key feature point and the second key feature point after the update transformation; Iteratively update the first key feature point in the preliminary key feature point pair until the mean square error is less than a preset matching error threshold, stop the iteration, and construct a target key feature point pair based on the latest transformed first key feature point and the second key feature point.

4. The slope crack inspection method for high mountain canyon areas according to claim 1 is characterized in that: The method of inputting the fused three-dimensional terrain data corresponding to the local terrain change area into the pre-trained crack inspection route planning model to obtain a local crack fine inspection route includes: Extracting terrain feature parameters from the fused three-dimensional terrain data corresponding to the local terrain change area; Constructing a state space vector according to the terrain characteristic parameters and the initial inspection route; Inputting the state space vector into the crack inspection route planning model to determine the crack acquisition inspection points in the local terrain change area and the crack acquisition flight posture of the inspection drone corresponding to each crack acquisition inspection point; A local crack fine inspection route is constructed based on the crack collection inspection points and the crack collection flight posture.

5. The slope crack inspection method for high mountain canyon areas according to claim 4 is characterized in that: Inputting the state space vector into the crack inspection route planning model to determine the crack acquisition inspection points in the local terrain change area and the crack acquisition flight posture of the inspection drone corresponding to each crack acquisition inspection point includes: Inputting the state space vector into the crack inspection route planning model, and determining the action vector with the maximum cumulative reward value under the state space vector; Update the inspection position and flight attitude of the inspection drone based on the motion vector, and record the inspection path and inspection flight attitude of the drone inferred by the crack inspection route planning model; In response to the local terrain change area being covered, or the flight time of the inspection drone reaching the preset time, the inspection task reasoning is confirmed to be completed, and based on the recorded drone inspection path and the inspection flight posture, the crack collection inspection points in the local terrain change area and the crack collection flight posture of the inspection drone corresponding to each crack collection inspection point are determined.

6. The slope crack inspection method for high mountain canyon areas according to claim 1 or 4, characterized in that: The crack inspection route planning model is obtained through a reinforcement learning training process, which includes: Determining the state space of the crack inspection route planning model, the state space including the coordinates of the inspection points in the initial inspection route and the flight attitude of the UAV, the global terrain data of the dam slope in the alpine canyon area, and historical flight path information; Determine the action space of the crack inspection route planning model, wherein the action space includes adjusting the flight altitude, adjusting the flight direction, adjusting the flight speed, adjusting the shooting angle of the laser radar and the depth camera, and updating the inspection points; Obtaining inspection interaction data through sampling of the state space and the action space, and storing the inspection interaction data in an interaction experience pool, wherein the inspection interaction data is used to simulate the inspection drone performing an inspection task in a dam slope area in a high mountain canyon area; Inspection interaction data are randomly sampled from the interaction experience pool, and model training is performed on the crack inspection route planning model, with the goal of maximizing the cumulative reward value of the reward function of the crack inspection route planning model. The model parameters of the crack inspection route planning model are optimized by gradient descent until the cumulative reward value is greater than or equal to the preset reward value threshold, thereby obtaining a trained crack inspection route planning model.

7. A slope crack inspection device for use in mountain and canyon areas, characterized in that: include: a three-dimensional terrain acquisition module, configured to control the inspection drone to follow a preset initial inspection route and collect, in real time, fused three-dimensional terrain data corresponding to the dam slope in the alpine canyon area at a first scanning resolution, the fused three-dimensional terrain data being acquired by a laser radar and a depth camera carried by the inspection drone; a terrain change region determination module, configured to extract first key feature points from the fused three-dimensional terrain data and second key feature points from the reference three-dimensional terrain data; and to perform coarse registration of the first key feature points and the second key feature points to determine preliminary key feature point pairs; Iteratively optimize the preliminary key feature point pairs to determine target key feature point pairs, and determine at least one local subregion based on the target key feature point pairs, wherein the local subregions include steep areas, gentle slope areas, and crack distribution areas; determine the local minimum mean square error corresponding to each of the local subregions based on the target key feature point pairs; and define the local subregions for which the local minimum mean square error is greater than or equal to a preset terrain error threshold as local terrain change areas; An adaptive route planning module is used to input the fused three-dimensional terrain data corresponding to the local terrain change area into a pre-trained crack inspection route planning model to obtain a local crack fine inspection route; The crack fine inspection module is used to control the inspection drone to perform adaptive route dynamic planning based on the local crack fine inspection route, perform fine inspection of the crack terrain in the local terrain change area at a second scanning resolution, and obtain millimeter-level crack inspection data.

8. An electronic device, characterized in that: include: processor; as well as A memory having computer-readable instructions stored thereon, wherein the computer-readable instructions, when executed by the processor, implement the slope crack inspection method for a high mountain canyon area according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the slope crack inspection method for a high mountain canyon area according to any one of claims 1 to 6 is implemented.

Images