Slope crack inspection method and device for alpine and valley areas, equipment and medium

Abstract

The invention provides a slope crack inspection method and device for alpine and valley areas, equipment and a medium, and relates to the technical field of dam slope safety inspection. The method comprises the following steps: controlling an inspection unmanned aerial vehicle to collect fused three-dimensional topographic data corresponding to a dam slope at a first resolution, and screening a local topographic change region, thereby obtaining a local crack fine inspection route in combination with a crack inspection route planning model; and based on the route, controlling the inspection unmanned aerial vehicle to carry out self-adaptive route dynamic planning so as to finish fine inspection of the crack terrain at a second resolution and obtain millimeter-level crack inspection data. According to the scheme, one-time inspection of the dam slope is achieved, the inspection process is coarse and fine, the inspection coverage is complete, targeted millimeter-level fine crack inspection is conducted on key areas, the recognition accuracy is improved, inspection of all key areas of interest of the dam slope area can be completed through one-time inspection, the endurance time of the inspection unmanned aerial vehicle is effectively prolonged, and the inspection efficiency is improved. And the redundant inspection data volume is reduced.

Description

Background Art

[0002] In alpine canyon areas, the accuracy and comprehensiveness of crack inspection on the dam slope are of great significance for the long-term safe operation of hydraulic structures. Due to its flexibility and coverage ability, the unmanned aerial vehicle (UAV) inspection technology has become an important means for dam crack monitoring.

[0003] As one of the commonly used sensors for inspection UAVs, lidar is widely used in slope inspection tasks due to its adaptability to complex terrains and strong global modeling ability. However, although lidar can quickly obtain terrain elevation data over a large range, the discreteness of the data determines its limited ability to capture microscopic features. In areas with complex crack details, it is difficult to provide data that meets the requirements of high-precision detection. At the same time, the high power consumption characteristic of lidar has a great impact on the endurance of UAVs during inspection tasks. Especially when scanning the entire area at the best resolution, even in flat areas or areas with sparse crack distributions, the acquisition of redundant data will still cause waste of UAV energy, shorten the execution time of UAV inspection tasks, result in a long inspection cycle, and low inspection efficiency.

[0004] In addition, current UAV inspection technologies mostly rely on static flight path design in task planning and lack the ability of intelligent adaptive autonomous flight path planning. It is difficult to dynamically adjust the flight path according to real-time acquired terrain data or detection requirements. This static method is prone to problems such as insufficient coverage or low efficiency of the inspection path in complex terrain scenarios. Especially in areas with concentrated cracks, the fixed path planning method cannot achieve fine capture of crack features.

[0005] In summary, the related technologies have problems such as the insufficient ability of lidar to capture microscopic features, the decline in endurance performance caused by high power consumption, and the lack of dynamic adjustment ability of the inspection path. These defects are particularly obvious in the complex scenarios of dam slopes in alpine canyon areas and urgently need to be further improved to meet the current demand for the lack of intelligent adaptive autonomous flight path planning ability of inspection UAVs.

[0006] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present disclosure. Therefore, it may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0007] The purpose of the embodiments of the present disclosure is to provide a method for inspecting slope cracks in alpine canyon areas, a device for inspecting slope cracks in alpine canyon areas, an electronic device, and a computer-readable storage medium, so as to be able to achieve a one-time inspection of the dam slope. The inspection process is detailed in some parts and general in others. The inspection coverage is complete, and there is targeted millimeter-level fine crack inspection in key areas, improving the recognition accuracy. Moreover, a one-time inspection can complete the inspection of all key attention areas in the dam slope area, effectively increasing the flight endurance time of the inspection drone, reducing the amount of redundant inspection data, effectively improving the accuracy, accuracy, and integrity of the collected crack data, and improving the inspection efficiency of the dam slope.

[0008] Other features and advantages of the present disclosure will become apparent from the following detailed description, or be learned in part through the practice of the present disclosure.

[0009] According to a first aspect of the embodiments of the present disclosure, there is provided a method for inspecting slope cracks in alpine canyon areas, including: Controlling an inspection drone to collect in real time the fused three-dimensional terrain data corresponding to the dam slope in the alpine canyon area along a preset initial inspection route, where the fused three-dimensional terrain data is collected by a lidar and a depth camera carried by the inspection drone; Locally matching the fused three-dimensional terrain data with pre-constructed reference three-dimensional terrain data to determine a local terrain change area in real time; Inputting the fused three-dimensional terrain data corresponding to the local terrain change area into a pre-trained crack inspection route planning model to obtain a local crack fine inspection route; Based on the local crack fine inspection route, controlling the inspection drone to perform adaptive route dynamic planning to finely inspect the crack terrain in the local terrain change area at a second scanning resolution to obtain millimeter-level crack inspection data.

[0010] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the step of collecting in real time the fused three-dimensional terrain data corresponding to the dam slope in the alpine canyon area at the first scanning resolution includes: obtaining the terrain elevation data and terrain structure point cloud data corresponding to the dam slope in the alpine canyon area collected by the lidar and the depth camera respectively at the first scanning resolution; registering and aligning the terrain elevation data and the terrain structure point cloud data to obtain preliminary three-dimensional terrain data; and performing depth fusion on the preliminary three-dimensional terrain data to obtain the fused three-dimensional terrain data corresponding to the dam slope in the alpine canyon area.

[0011] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the local matching of the fused three-dimensional terrain data with the pre-constructed reference three-dimensional terrain data to determine the local terrain change region includes: extracting first key feature points from the fused three-dimensional terrain data, and extracting second key feature points from the reference three-dimensional terrain data; performing rough registration on the first key feature points and the second key feature points to determine preliminary key feature point pairs; performing iterative optimization on the preliminary key feature point pairs to determine target key feature point pairs, and determining at least one local sub-region according to the target key feature point pairs, the local sub-region including a steep region, a gentle slope region, and a crack distribution region; determining the local minimum mean square error corresponding to each local sub-region according to the target key feature point pairs; and using the local sub-regions with the local minimum mean square error greater than or equal to a preset terrain error threshold as the local terrain change region.

[0012] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the performing iterative optimization on the preliminary key feature point pairs to determine target key feature point pairs includes: determining a rotation matrix and a translation vector according to the preliminary key feature point pairs, the rotation matrix and the translation vector minimizing the mean square error between the matching point pairs; updating and transforming the first key feature points belonging to the preliminary key feature point pairs through the rotation matrix and the translation vector, and determining the mean square error between the updated and transformed first key feature points and the second key feature points; iteratively updating the first key feature points in the preliminary key feature point pairs until the mean square error is less than a preset matching error threshold, stopping the iteration, and constructing target key feature point pairs according to the first key feature points and the second key feature points after the latest transformation.

[0013] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the inputting the fused three-dimensional terrain data corresponding to the local terrain change region into a pre-trained crack inspection route planning model to obtain a local crack fine inspection route includes: extracting terrain feature parameters from the fused three-dimensional terrain data corresponding to the local terrain change region; constructing a state space vector according to the terrain feature parameters and the initial inspection route; inputting the state space vector into the crack inspection route planning model to determine the crack collection inspection points of the local terrain change region and the crack collection flight postures of the inspection drones corresponding to each crack collection inspection point; and constructing a local crack fine inspection route based on the crack collection inspection points and the crack collection flight postures.

[0014] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the step of inputting the state space vector into the crack inspection route planning model to determine the crack collection inspection points in the local terrain change area and the crack collection flight postures of the inspection UAVs corresponding to the respective crack collection inspection points includes: inputting the state space vector into the crack inspection route planning model to determine an action vector with the maximum cumulative reward value under the state space vector; updating the inspection position and flight posture of the inspection UAV based on the action vector, and recording the UAV inspection path and inspection flight posture inferred by the crack inspection route planning model; in response to the fact that the local terrain change area is completely covered, or the flight time of the inspection UAV reaches a preset time, confirming that the inspection task inference is completed, and determining the crack collection inspection points in the local terrain change area and the crack collection flight postures of the inspection UAVs corresponding to the respective crack collection inspection points according to the recorded UAV inspection path and inspection flight posture.

[0015] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the crack inspection route planning model is obtained through a reinforcement learning training process, and the reinforcement learning training process includes: determining the state space of the crack inspection route planning model, where the state space includes the inspection point coordinates and UAV flight postures in the initial inspection route, the global terrain data of the dam slope in the alpine canyon area, and the historical flight path information; determining the action space of the crack inspection route planning model, where the action space includes adjusting the flight height, adjusting the flight direction, adjusting the flight speed, adjusting the shooting angles of the lidar and depth camera, and updating the inspection points; sampling inspection interaction data through the state space and the action space, and storing the inspection interaction data in an interaction experience pool, where the inspection interaction data is used to simulate the inspection UAV performing an inspection task in the dam slope area of the alpine canyon area; randomly sampling inspection interaction data in the interaction experience pool, training the crack inspection route planning model, aiming to maximize the cumulative reward value of the reward function of the crack inspection route planning model, and optimizing the model parameters of the crack inspection route planning model through gradient descent until the cumulative reward value is greater than or equal to a preset reward value threshold, thereby obtaining a trained crack inspection route planning model.

[0016] According to a second aspect of the embodiments of the present disclosure, there is provided a slope crack inspection device for alpine canyon areas, including: A three-dimensional terrain acquisition module, configured to control an inspection UAV to collect, in real time at a first scanning resolution, the fused three-dimensional terrain data corresponding to the dam slope in the alpine canyon area along a preset initial inspection route, where the fused three-dimensional terrain data is collected by a lidar and a depth camera carried by the inspection UAV. A terrain change area determination module, configured to locally match the fused three-dimensional terrain data with pre-constructed reference three-dimensional terrain data to determine a local terrain change area; An adaptive flight path planning module, configured to input the fused three-dimensional terrain data corresponding to the local terrain change area into a pre-trained crack inspection route planning model to obtain a local crack fine inspection flight path; A crack fine inspection module, configured to control the inspection drone to perform adaptive flight path dynamic planning based on the local crack fine inspection flight path, and finely inspect the crack terrain of the local terrain change area at a second scanning resolution to obtain millimeter-level crack inspection data.

[0017] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including: a processor; and a memory, where computer-readable instructions are stored on the memory, and when the computer-readable instructions are executed by the processor, the method for slope crack inspection in alpine canyon areas described in any one of the above is implemented.

[0018] According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method for slope crack inspection in alpine canyon areas described in any one of the above is implemented.

[0019] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects: In the method for slope crack inspection in alpine canyon areas in the exemplary embodiments of the present disclosure, through the collaborative use of a lidar and a depth camera, the lidar can obtain three-dimensional terrain elevation data in a wide area, providing a basis for the accurate description of the overall terrain features, while the depth camera can provide ultra-high-resolution three-dimensional image data in the local crack area, making up for the deficiency of the lidar in capturing microscopic continuity features. The data fusion of the two significantly improves the detail information of the crack area.

[0020] During the data collection process, the depth camera can significantly improve the detection accuracy of key features such as crack width and depth through refined scanning of the crack area. Combining with the global terrain data collected by the lidar, the depth camera provides more detailed crack information for enhancing the crack features in the local area, laying a reliable data foundation for subsequent crack analysis and prediction of the expansion trend. In addition, by dynamically adjusting the scanning resolution of the lidar and the depth camera, the limited power energy of the drone can be flexibly allocated in different areas. Especially while increasing the scanning resolution in key areas, redundant collection in non-key areas is reduced, thereby optimizing the inspection efficiency and reducing the energy consumption of the drone. While ensuring the flight time of the drone, the inspection efficiency of the drone for the dam slope is improved.

[0021] Through comprehensive processing of the fused data from lidar and depth cameras, it is possible to mark the areas of local terrain changes in real time, dynamically adjust the flight path and flight attitude of the inspection UAV. The global terrain information provided by lidar ensures the coverage and safety of path planning, while the detail capture ability of the depth camera in key areas enables dynamic path planning to more accurately respond to changes in the crack distribution characteristics. Compared with the static planning methods of related technologies, this ability to dynamically adjust the path significantly improves the adaptability of the inspection task, enabling one-time inspection. The inspection process is both rough and detailed, with full coverage and targeted millimeter-level fine crack inspection in key areas, avoiding inefficient problems such as insufficient coverage or path duplication. While ensuring the inspection efficiency, through the collaborative work of lidar and depth cameras, the organic combination of global and local data is achieved, significantly improving the accuracy of crack detection and the integrity of terrain data. Through data fusion to support dynamic path planning, it effectively solves the inefficient problems of insufficient path coverage and repeated inspection in related technologies. At the same time, by dynamically adjusting the scanning resolution and smoothing resource allocation, the limitation of high power consumption on the UAV's endurance ability is overcome, enabling the inspection task to balance inspection efficiency and crack monitoring accuracy in complex scenarios.

[0022] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure and, together with the specification, used to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0024] Figure 1 A schematic diagram of the system architecture showing an exemplary application environment of a slope crack inspection method and device for alpine canyon areas to which the embodiments of the present disclosure can be applied.

[0025] Figure 2 A schematic flow diagram showing a method for slope crack inspection in alpine canyon areas according to some embodiments of the present disclosure.

[0026] Figure 3 A schematic flow diagram showing the process of determining fused three-dimensional terrain data according to some embodiments of the present disclosure.

[0027] Figure 4 A schematic flow diagram showing the process of generating local terrain change areas according to some embodiments of the present disclosure.

[0028] Figure 5Schematically shows a flowchart of constructing target key feature points according to some embodiments of the present disclosure.

[0029] Figure 6 Schematically shows a flowchart of generating a fine inspection route for local cracks according to some embodiments of the present disclosure.

[0030] Figure 7 Schematically shows a flowchart of inferring the flight attitude for crack collection according to some embodiments of the present disclosure.

[0031] Figure 8 Schematically shows a flowchart of the reinforcement learning training process of a crack inspection route planning model according to some embodiments of the present disclosure.

[0032] Figure 9 Schematically shows a route diagram of the adaptive dynamic route inspection of a dam structure according to some embodiments of the present disclosure.

[0033] Figure 10 Schematically shows a route diagram of the adaptive dynamic route inspection of a slope according to some embodiments of the present disclosure.

[0034] Figure 11 Schematically shows a schematic diagram of a slope crack inspection device for alpine canyon areas according to some embodiments of the present disclosure.

[0035] Figure 12 Schematically shows a schematic diagram of the computer system of an electronic device according to some embodiments of the present disclosure.

[0036] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. Detailed Description of the Embodiments

[0037] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0038] In addition, the drawings are only schematic diagrams and are not necessarily drawn to scale. The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0039] Figure 1 A schematic diagram of a system architecture showing an exemplary application environment of a slope crack inspection method and device for alpine canyon areas to which embodiments of the present disclosure can be applied.

[0040] As Figure 1 shown, the system architecture 100 may include a drone device 101, a network 102, and a server 103. The network 102 is used to provide a medium for a communication link between the drone device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc. The drone device 101 may be a remotely controlled flying device equipped with an image acquisition unit, including but not limited to various types of drones. Embodiments of the present disclosure do not make special limitations on the type of the drone device 101. It should be understood that Figure 1 the numbers of the drone device, the network, and the server in

[0041] are merely illustrative. According to the implementation requirements, there may be any number of drone devices, networks, and servers. For example, the server 103 may be a server cluster composed of multiple servers, etc.

[0042] In this exemplary embodiment, first, a slope crack inspection method for alpine canyon areas is provided. The slope crack inspection method for alpine canyon areas can be applied to the drone device or the server. Hereinafter, an example of the server executing this method will be used for illustration. Figure 2 Schematically shows a flowchart of a slope crack inspection method for alpine canyon areas according to some embodiments of the present disclosure. Refer to Figure 2 shown, the slope crack inspection method for alpine canyon areas may include the following steps: Step S210, controlling the inspection drone to collect in real time the fused three-dimensional terrain data corresponding to the dam slope of the alpine canyon area along a preset initial inspection route, where the fused three-dimensional terrain data is collected by a lidar and a depth camera carried by the inspection drone; Step S220, locally matching the fused three-dimensional terrain data with pre-constructed reference three-dimensional terrain data to determine a local terrain change area; Step S230: Input the fused three-dimensional terrain data corresponding to the local terrain change area into a pre-trained crack inspection route planning model to obtain a local crack fine inspection flight path; Step S240: Based on the local crack fine inspection flight path, control the inspection UAV to perform adaptive flight path dynamic planning, and finely inspect the crack terrain of the local terrain change area at a second scanning resolution to obtain millimeter-level crack inspection data.

[0043] According to the slope crack inspection method for alpine canyon areas in this exemplary embodiment, through the collaborative use of lidar and depth camera, the lidar can obtain three-dimensional terrain elevation data within a wide area, providing a basis for the accurate description of the overall terrain features. The depth camera can provide high-resolution three-dimensional point cloud data in the local crack area, making up for the deficiency of the lidar in capturing microscopic features. The data fusion of the two significantly improves the detailed information of the crack area, while ensuring the integrity and consistency of the global terrain data, overcoming the problem of data one-sidedness caused by over-reliance on lidar sensors in related technologies; during the data acquisition process, the depth camera can significantly improve the detection accuracy of key features such as crack width and depth through refined scanning of the crack area. Combining with the global terrain data collected by the lidar, the depth camera provides more detailed crack information for the enhancement of crack features in the local area, laying a reliable data foundation for subsequent crack analysis and prediction of the expansion trend. In addition, by dynamically adjusting the scanning resolution of the lidar and depth camera, the limited power energy of the drone can be flexibly allocated in different areas. Especially when improving the scanning resolution in key areas, redundant acquisition in non-key areas can be reduced, thereby optimizing the inspection efficiency and reducing the energy consumption of the drone. While ensuring the flight time of the drone, the inspection efficiency of the drone for the dam slope is improved; through the comprehensive processing of the fused data of the lidar and depth camera, the local terrain change area can be marked in real time, and the flight path and flight attitude of the inspection drone can be dynamically adjusted. The global terrain information provided by the lidar ensures the coverage and safety of path planning, while the detail capture ability of the depth camera in key areas enables the dynamic path planning to more accurately respond to the changes in the crack distribution characteristics. Compared with the static planning method in related technologies, this path dynamic adjustment ability significantly improves the adaptability of the inspection task, avoiding the inefficient problems of insufficient coverage or path repetition, and effectively improving the accuracy and integrity of the collected crack data while ensuring the inspection efficiency; through the collaborative work of the lidar and depth camera, the organic combination of global and local data is realized, significantly improving the accuracy of crack detection and the integrity of terrain data; through data fusion to support dynamic path planning, the inefficient problems of insufficient path coverage and repeated inspections in related technologies are effectively solved; at the same time, by dynamically adjusting the scanning resolution and smoothly allocating resources, the limitation of high power consumption on the drone's endurance ability is overcome, enabling the inspection task to balance the inspection efficiency and crack monitoring accuracy in complex scenarios.

[0044] Next, the slope crack inspection method for alpine canyon areas in this exemplary embodiment will be further described.

[0045] In step S210, the inspection UAV is controlled to collect in real time the fused three-dimensional terrain data corresponding to the dam slope in the alpine canyon area along a preset initial inspection route, where the fused three-dimensional terrain data is collected by the lidar and depth camera carried by the inspection UAV.

[0046] In an exemplary embodiment of the present disclosure, the initial inspection route refers to the UAV cruise route set before the start of the current inspection task for inspecting the dam slope in the alpine canyon area. For example, the initial inspection route can be an ideal UAV cruise route set manually based on the terrain environment characteristics and experience of the dam slope in the alpine canyon area; the initial inspection route can also be the historical UAV cruise route finally adopted and used by the UAV in the previous inspection task; of course, the initial inspection route can also be a UAV cruise route constructed in other ways before the start of the current inspection task, and this exemplary embodiment does not make special limitations on this. The construction process of the initial inspection route can include steps such as path smoothing, obstacle avoidance, and waypoint optimization to ensure the safety, rationality, and coverage rate of the route.

[0047] The first scanning resolution refers to the scanning parameters of the lidar and depth camera on the initial inspection route, and is usually set to a relatively balanced resolution to balance energy consumption and the coverage quality of terrain data. For example, the lidar can be set to a certain number of scan points per second, and the depth camera can be set to a certain frame rate of point cloud acquisition frequency. The setting of the first scanning resolution can be based on the terrain complexity of the target area and the expected crack distribution. A lower resolution can be selected to cover a larger area and avoid unnecessary data redundancy.

[0048] Real-time collection means that while the inspection UAV is flying along the initial inspection route, the lidar and depth camera respectively perform data collection tasks. The lidar can generate three-dimensional elevation data of the dam slope by emitting laser beams and measuring the echo time; the depth camera can capture high-precision three-dimensional point cloud data of a local area through active or passive methods to supplement the deficiencies of the lidar in capturing crack details. During the collection process, the UAV needs to maintain the stability of its flight attitude to reduce the errors generated by sensor data due to jitter; specifically, the flight attitude can be adjusted in real time and the positioning can be corrected through the inertial navigation system (INS) and global navigation satellite system (GNSS) carried by the UAV.

[0049] The fused three-dimensional terrain data refers to the complete terrain data generated after registering and fusing two different types of terrain data collected by lidar and depth cameras. Registration means aligning the three-dimensional elevation data of the lidar with the point cloud data of the depth camera so that both can express the spatial characteristics of the target area in the same coordinate system. The fusion process can adopt an alignment method based on feature points. For example, key feature points (such as corner points and edge points) in the lidar and depth camera data can be extracted for registration, or a method based on a probability model can be used to complete the fusion by maximizing the spatial similarity of the two types of data. The present exemplary embodiment does not limit the registration and fusion methods. The fused three-dimensional terrain data has both the global information provided by the lidar and the local details provided by the depth camera, providing high-precision terrain basic information for subsequent inspection tasks.

[0050] Optionally, the scanning resolution of the lidar and the point cloud acquisition frequency of the depth camera can be dynamically adjusted according to the actual inspection requirements. For example, the scanning resolution can be reduced when flying over a flat area to save energy consumption, and the scanning resolution can be increased in complex areas or areas with dense cracks to obtain higher-precision data. In addition, the registration process of the fused three-dimensional terrain data can be completed by an improved Iterative Closest Point (ICP) algorithm or a registration network based on deep learning to improve the robustness and efficiency of data fusion.

[0051] In step S220, the fused three-dimensional terrain data is locally matched with the pre-constructed reference three-dimensional terrain data to determine the local terrain change area.

[0052] In an exemplary embodiment of the present disclosure, the pre-constructed reference three-dimensional terrain data refers to a three-dimensional terrain model of the dam slope in the alpine canyon area generated according to historical inspection data or topographic survey data before the start of the inspection task. This data can be generated through the cumulative results of long-term monitoring or formed through a one-time large-scale surveying and mapping, and is used to compare with the fused three-dimensional terrain data collected in real time.

[0053] Local matching means aligning the features of the fused three-dimensional terrain data collected in real time with the reference three-dimensional terrain data in a local area to determine the terrain change area. During the matching process, first, the key feature points of the two sets of data need to be extracted. For example, terrain mutation points can be extracted through the Harris corner detection algorithm or crack edge points can be extracted through the curvature calculation method. Then, the matching can be completed in two steps: rough registration and fine registration. Rough registration is used to quickly establish a rough correspondence between the two sets of data. For example, it can be achieved through nearest neighbor search or a global feature-based matching method; fine registration can adopt the Iterative Closest Point (ICP) algorithm or other optimization methods to further reduce the matching error by adjusting the rotation matrix and translation vector.

[0054] The determination of the local terrain change area can be based on the locally calculated minimum mean square error during the matching process. If the error in certain local areas is greater than the preset terrain error threshold, it can be determined that terrain changes may have occurred in these areas. These areas may include crack propagation areas, landslide indication areas, or areas with significant terrain changes due to environmental changes. The setting of the error threshold can be adjusted according to historical change trends and terrain complexity to improve the sensitivity and accuracy of the determination. Specifically, it can be customized according to the actual usage, and this embodiment is not limited thereto.

[0055] In terms of implementation, the generation of the reference three-dimensional terrain data can adopt a global terrain scanning method and be constructed by combining the differential analysis of multi-period data. The matching algorithm can select a rigid body transformation model or a non-rigid deformation model according to the terrain characteristics of the target area. In addition, the division of the local change area can be further refined into categories such as crack areas, landslide areas, and water erosion areas to support targeted inspection strategies.

[0056] In step S230, the fused three-dimensional terrain data corresponding to the local terrain change area is input into a pre-trained crack inspection route planning model to obtain a local crack fine inspection flight path.

[0057] In an exemplary embodiment of the present disclosure, the local terrain change area refers to a target area where cracks or other terrain anomalies may exist, determined by the local matching of the fused three-dimensional terrain data and the reference three-dimensional terrain data, combined with the local minimum mean square error and the change threshold. The local terrain change area includes complex areas such as concentrated crack distribution, steep terrain mutation, or landslide indication. The fused three-dimensional terrain data corresponding to the local terrain change area refers to the three-dimensional terrain data limited within the scope of these areas, which integrates the global terrain information of the lidar and the detailed point cloud data of the depth camera to provide high-quality input data for planning the local crack fine inspection flight path.

[0058] The crack inspection route planning model refers to a path optimization model obtained through reinforcement learning training. Its core lies in dynamically generating an optimal inspection path using the input terrain data features. The state space of the crack inspection route planning model can include the three-dimensional terrain features of the local terrain change area, the current flight parameters of the inspection unmanned aerial vehicle (including position, speed, and altitude), and the historical flight path information, which are used to describe the execution state of the current task of the unmanned aerial vehicle. The action space can define the inspection adjustment actions that the model can take, including flight altitude adjustment, heading adjustment, scan resolution selection, etc., which are used to dynamically optimize the path. The reward function of the model is designed based on indicators such as crack area coverage, inspection path smoothness, and energy consumption optimization to maximize the integrity of crack capture and inspection efficiency.

[0059] When the fused three-dimensional terrain data of the local terrain change area is input into the crack inspection route planning model, the crack inspection route planning model can first extract terrain feature parameters, including terrain height difference, gradient, crack direction and width distribution, etc., and construct a state space vector based on these parameters. Subsequently, through the inference process of the crack inspection route planning model, actions with the maximum cumulative reward value can be gradually selected, the flight position and attitude of the inspection UAV can be updated, and the corresponding inspection path points can be recorded. When the inference process of the crack inspection route planning model ends, the generated inspection flight path covers the crack features in the local terrain change area and provides optimal path support for subsequent high-precision detection.

[0060] A path planning method based on deep reinforcement learning can be adopted. For example, a deep Q-network (DQN) model can be used to predict the best actions of the UAV; a method based on policy gradient can also be adopted to directly optimize the cumulative reward of the inspection task. In addition, the training data of the crack inspection route planning model can be sourced from the actual flight paths and terrain change annotation data of historical inspection tasks, or can be generated through simulation in a simulation environment. The source of the training data of the crack inspection route planning model in this exemplary embodiment is not specifically limited.

[0061] In step S240, control the inspection UAV based on the local crack fine inspection flight path, and perform a fine inspection on the crack terrain in the local terrain change area at the second scanning resolution to obtain millimeter-level crack inspection data.

[0062] In an exemplary embodiment of the present disclosure, the local crack fine inspection flight path refers to a UAV inspection path optimized by combining the characteristics of the local terrain change area and the crack distribution characteristics, focusing on the high-change area where the cracks are located. The second scanning resolution refers to the high-resolution scanning parameters designed for the crack area, which is usually higher than the first scanning resolution and is used to obtain fine feature data of the cracks. For example, at the second scanning resolution, the lidar can be adjusted to a higher scanning point density, and the depth camera can increase the frame rate and resolution to ensure clear capture of the crack edges in the point cloud data.

[0063] Fine inspection refers to high-precision crack detection for the local change area on the basis of the initial inspection. When the inspection UAV flies along the local crack fine inspection flight path, it adjusts the flight attitude and scanning parameters in real time to adapt to the terrain undulation and crack distribution characteristics. The millimeter-level data collection of the cracks includes detailed measurements of width, depth, direction and spatial position. The data is sourced from the high-precision point cloud of the depth camera and the fine elevation information provided by the lidar. During the data collection process, the UAV uses its own inertial navigation system and real-time positioning module to ensure flight stability and dynamically avoids possible obstacles in combination with the obstacle avoidance algorithm.

[0064] The millimeter-scale crack inspection data is finally used to generate a high-resolution three-dimensional crack model, which provides a detailed description of the crack morphology, location, and characteristics. These data can be further used for predicting the crack propagation trend or assessing the health of the slope structure.

[0065] Optionally, the scanning parameters for the fine inspection can be adjusted according to actual requirements. For example, the resolution can be appropriately reduced in areas with clear crack characteristics to save energy consumption; while in complex terrains or areas with blurred crack boundaries, the detection accuracy can be improved by increasing the scanning density. In addition, the inspection drone can be equipped with additional auxiliary sensors (such as thermal imagers or ultrasonic sensors) for multimodal detection of crack characteristics to further enhance the reliability and applicability of the inspection results.

[0066] Next, steps S210 to S240 will be described in detail.

[0067] In an exemplary embodiment of the present disclosure, step S210 of acquiring the fused three-dimensional terrain data corresponding to the dam slope in the alpine canyon area in real time at the first scanning resolution can be implemented through the steps in Figure 3 , as shown in Figure 3 , and specifically may include: Step S310, acquiring the terrain elevation data and the terrain structure point cloud data corresponding to the dam slope in the alpine canyon area, which are respectively acquired by the lidar and the depth camera in real time at the first scanning resolution; Step S320, registering and aligning the terrain elevation data and the terrain structure point cloud data to obtain preliminary three-dimensional terrain data; Step S330, performing depth fusion on the preliminary three-dimensional terrain data to obtain the fused three-dimensional terrain data corresponding to the dam slope in the alpine canyon area.

[0068] Among them, the terrain elevation data collected by the lidar refers to the three-dimensional spatial point data calculated through the emission of laser beams and the echo time, which is used to reflect the terrain height difference and the overall contour characteristics of the dam slope in the alpine canyon area. Its principle is based on the laser ranging formula, that is, the distance is measured according to the speed of light and half of the time difference between the laser emission and the echo, and through the analysis of the scanning angle of the laser beam, three-dimensional point cloud data is further generated.

[0069] The terrain structure point cloud data collected by a depth camera refers to the three-dimensional point cloud generated by an active light source or passive stereo vision method, which is used to obtain the geometric details of a local area. The working principles of depth cameras include structured light method, Time-of-Flight (ToF) method, or stereo matching method, etc. For example, in the structured light method, a specific pattern of grating is projected and its deformation is calculated to generate the depth value of each pixel point. The point cloud data generated by the depth camera has higher local accuracy compared with lidar data, and is suitable for capturing microscopic features such as cracks.

[0070] The first scan resolution refers to the data acquisition parameters of the lidar and depth camera set in advance, which are used to balance the requirements of coverage and detail capture. For example, the lidar can be set to a medium point density (such as 50,000 to 100,000 points per second), and the frame rate of the depth camera can be set to 30 to 60 frames per second to meet the real-time and resolution requirements under complex terrains. In specific implementation, the lidar and depth camera operate in a synchronous acquisition mode, and the geographic coordinates of the acquisition points are recorded in real time in combination with the inertial navigation system (INS) and global navigation satellite system (GPS) of the unmanned aerial vehicle.

[0071] Optionally, the lidar can include a multi-line lidar to increase the coverage density of the point cloud; or a high-frame-rate depth camera can be adopted to enhance the local crack capture ability. For different task requirements, the first scan resolution can also be dynamically adjusted. For example, in complex terrain areas, the first scan resolution of the lidar or depth camera can be greater than that set in flat areas.

[0072] Registration alignment refers to aligning the terrain elevation data of the lidar and the point cloud data of the depth camera into a unified three-dimensional coordinate system through geometric transformation. The implementation principle lies in minimizing the error between the two sets of point cloud data. Through the rigid body transformation model, the alignment relationship is expressed by a rotation matrix and a translation vector, and then the error can be gradually optimized through the Iterative Closest Point (ICP) algorithm, and finally the data alignment is completed.

[0073] Specifically, the registration process can be to extract terrain edge points and steep points from the lidar data, extract crack boundary points and feature points with higher curvature from the depth camera data, and use the global features of the point cloud to initially estimate the point pair relationship. For example, through nearest neighbor search or normal direction-based matching methods, based on the ICP algorithm, the error between the point clouds is gradually optimized, and the aligned point cloud data is output. Optionally, a point cloud registration network based on deep learning (such as PointNetLK) can also be used for fully automatic registration, or a multi-stage registration process can be used to first perform global alignment and then local fine alignment.

[0074] Deep fusion refers to further processing the data of lidar and depth camera on the basis of registration and alignment to generate a three-dimensional terrain model with consistency and enhanced details. Its core lies in data weighted fusion and spatial reconstruction. Deep fusion can assign different weights to point cloud data according to the characteristics of sensors and regional requirements. For example, lidar data is preferentially used in wide-area terrain, and depth camera data is preferentially used in crack details; abnormal points in the fused point cloud are filtered out, such as using statistical filtering to remove outliers, or using Gaussian filtering to smooth the noise effect; multi-resolution processing is performed on the fused point cloud, such as using octree segmentation method to increase the resolution in local detail areas and reduce the resolution in global areas to reduce the data volume; the boundaries and overlapping areas of the fused point cloud are smoothed to ensure the overall consistency of the model.

[0075] The finally obtained fused three-dimensional terrain data not only contains the advantages of lidar global modeling but also integrates the strengths of depth camera in detail capture, providing a comprehensive and high-precision terrain model for subsequent inspection tasks; alternative implementation methods include using a probabilistic graph model (such as Bayesian Fusion) to achieve data fusion, or using a depth fusion framework based on three-dimensional voxels (such as TSDF algorithm) for point cloud reconstruction, and this example embodiment does not make special limitations on this.

[0076] By using lidar and depth camera to collect terrain data at the first scanning resolution and combining registration and depth fusion technologies, it is possible to generate fused three-dimensional terrain data with both global modeling ability and detail expression ability in complex terrain. On the one hand, the wide-area coverage characteristic of lidar ensures the integrity of terrain modeling; on the other hand, the high-resolution acquisition of depth camera in local areas makes up for the deficiency of lidar in capturing microscopic features. At the same time, through registration and alignment and depth fusion, the problem of insufficient consistency of multi-source data is solved, providing high-precision terrain basic information for subsequent inspection tasks and effectively improving the inspection efficiency and the reliability of crack detection.

[0077] In an example embodiment of the present disclosure, the local matching of the fused three-dimensional terrain data and the pre-constructed reference three-dimensional terrain data in step S220 to determine the content of the local terrain change area can be implemented through the steps in Figure 4 , and with reference to Figure 4 shown, it can specifically include: Step S410, extracting the first key feature points from the fused three-dimensional terrain data and extracting the second key feature points from the reference three-dimensional terrain data; Step S420, performing rough registration on the first key feature points and the second key feature points to determine the preliminary key feature point pairs; Step S430: Iteratively optimize the preliminary key feature point pairs, determine the target key feature point pairs, and determine at least one local sub-region according to the target key feature point pairs. The local sub-region includes a steep region, a gentle slope region, and a crack distribution region; Step S440: Determine the local minimum mean square error corresponding to each local sub-region according to the target key feature point pairs; Step S450: Take the local sub-regions with the local minimum mean square error greater than or equal to the preset terrain error threshold as the local terrain change regions.

[0078] Among them, the first key feature point refers to a set of points extracted from the fused three-dimensional terrain data that can represent the target terrain change characteristics and is used to compare with the feature points in the reference three-dimensional terrain data. The extraction of the first key feature points is usually based on geometric feature analysis, mainly including corner points, edge points, and points with higher curvature. The extraction method can use the Harris corner detection algorithm. Its principle is to calculate the change of the gray gradient in the pixel window. The response function defining the corner points can be expressed by the following relational expressions: ; ; Among them, R can represent the corner response value. When the corner response value is greater than or equal to the threshold, it can be judged as a corner point. can represent the gradient matrix. and can represent the gradients of the image gray level in the x and y directions. can represent the empirical constant, and its general value range is 0.04 ≤ ≤ 0.06. can represent the determinant of the gradient matrix. can represent the trace of the gradient matrix.

[0079] The second key feature point is a set of representative points extracted from the reference three-dimensional terrain data and is used to match with the first key feature points. Its extraction method is similar to that of the first key feature point. The distribution of the key feature points can reflect the main structural characteristics of the terrain. In specific implementation, the corner point extraction algorithm or the feature point extraction algorithm based on curvature can be respectively used for the two groups of data, and the extracted feature points are stored in the form of a point set for subsequent registration use.

[0080] Coarse registration means that by initially aligning two groups of point clouds, the first key feature points and the second key feature points are roughly corresponding, and preliminary key feature point pairs are established. For example, for each point in the first key feature point set, the point with the closest Euclidean distance in the second key feature point set can be found as the matching point. The matching process can be expressed by the following relational expressions: ; wherein, can represent the coordinates of point i in the first set of key feature points, can represent the point in the second set of key feature points that is closest to, can represent the second set of key feature points, can represent solving the Euclidean distance. Furthermore, a preliminary rotation matrix and translation vector can be calculated based on the matching point pairs to complete a rough alignment of the two sets of point clouds. The goal is to minimize the mean square error (MSE) of the preliminary key feature point pairs. When the error change is less than the set threshold, the iteration stops; otherwise, return to the first step to update the point pairs. After the optimization is completed, the boundaries of the local sub-regions are determined according to the distribution of the target key feature point pairs. These sub-regions can be classified into steep regions, gentle slope regions, and crack distribution regions according to characteristics such as point cloud density and curvature.

[0081] The iterative optimization uses a fine registration algorithm (such as the ICP algorithm), which gradually adjusts the rotation matrix and translation vector to minimize the error. Specifically, based on the preliminary alignment, the point pair relationship is updated. For each point in the first set of key feature points, find the closest point in the second set of key feature points. Based on the new point pairs, optimize the rotation matrix and translation vector. The process of optimizing the rotation matrix and translation vector can be represented by the following relational expressions: ; wherein, can represent a 3×3 rotation matrix, can represent a 3×1 translation vector, can represent a point in the initial point cloud, can represent the point after rotation and translation transformation; the goal is to minimize the mean square error of the preliminary key feature points, and the mean square error can be represented by the following relational expressions: ; wherein, can represent the mean square error of the preliminary key feature points, can represent the point after rotation and translation transformation, can represent the matching point in the reference point cloud, can represent the number of matching point pairs. After the optimization is completed, the target key feature point pairs are obtained. Furthermore, the boundaries of the local sub-regions can be determined according to the distribution of the target key feature point pairs. The local sub-regions can be classified into steep regions, gentle slope regions, and crack distribution regions according to characteristics such as point cloud density and curvature.

[0082] The local minimum mean square error is the matching error between target key feature points within a local sub-region. When the local minimum mean square error exceeds the preset terrain error threshold, it is determined that this local sub-region is a local terrain change region. The preset terrain error threshold can be set according to historical statistical data of terrain changes or task requirements, and this embodiment does not make special limitations on this.

[0083] Through key feature point extraction, point pair matching, and iterative optimization techniques, the local matching of fused three-dimensional terrain data and reference three-dimensional terrain data can be efficiently and accurately completed. On the one hand, the extracted key feature points ensure the matching accuracy; on the other hand, the calculation of the local minimum mean square error and the determination of the change region achieve the precise identification of local terrain changes, showing good robustness in the determination of steep regions, gentle slope regions, and crack distribution regions, providing an accurate regional marker and path planning basis for subsequent inspection tasks.

[0084] Optionally, the iterative optimization of the preliminary key feature point pairs can be achieved through the steps in Figure 5 to determine the target key feature point pairs. As shown in Figure 5 , it can specifically include: Step S510: Determine the rotation matrix and translation vector according to the preliminary key feature point pairs, where the rotation matrix and the translation vector minimize the mean square error between the matching point pairs; Step S520: Update and transform the first key feature point belonging to the preliminary key feature point pairs through the rotation matrix and the translation vector, and determine the mean square error between the updated and transformed first key feature point and the second key feature point; Step S530: Iteratively update the first key feature point in the preliminary key feature point pairs until the mean square error is less than the preset matching error threshold, stop the iteration, and construct the target key feature point pairs based on the latest transformed first key feature point and the second key feature point.

[0085] Among them, the rotation matrix is a three-dimensional space transformation tool used to adjust the direction of the first key feature point set to be consistent with the direction of the second key feature point set, while the translation vector is used to correct the displacement deviation between the two point clouds. By analyzing the distribution characteristics of the preliminary key feature point pairs, the center point coordinates of the two point clouds can be calculated, and the data can be de-centered based on the center points. The purpose of de-centralization is to eliminate the interference of global offsets on the rotation and translation calculations, so as to focus on the relationship between the local features of the point clouds.

[0086] The direction difference between point clouds can be compared to initially calculate the rotation matrix to adjust the direction consistency of the two sets of point clouds. The calculation of the rotation matrix is based on the geometric distribution of the key feature points of the two sets of point clouds. For example, by analyzing the spatial vectors between the feature points, the rotation parameters that minimize the direction difference of the point clouds can be found. At the same time, the translation vector is calculated based on the repositioning of the data after decentralization. By adjusting the position deviation of the point cloud, it is ensured that the central points of the two sets of point clouds coincide.

[0087] In the specific implementation, the calculation of the rotation matrix and the translation vector can be carried out by the method of iterative optimization. The goal of iterative optimization is to gradually adjust the rotation and displacement parameters to gradually reduce the matching error between the two sets of point clouds and finally reach the preset error threshold. This process can significantly improve the accuracy of point cloud data alignment and provide high-quality initial results for the subsequent steps.

[0088] The first key feature point in the initially key feature point pair is updated and transformed through the rotation matrix and the translation vector, and the error between the updated and transformed first key feature point and the second key feature point is determined. The essence of the update transformation is to apply the rotation matrix and the translation vector to the first key feature point set, adjust the spatial position of each feature point, and make it closer to the second key feature point. After the feature point update is completed, the optimization effect of the current parameters is evaluated by calculating the spatial error between each pair of matching points. If the error is still higher than the preset threshold, the rotation matrix and the translation vector need to be further adjusted, and the process of feature point update and error calculation is repeated.

[0089] The update transformation of the feature points can be carried out in a batch processing manner, that is, the rotation and translation operations are applied to the entire feature point set at the same time, so as to improve the processing efficiency; for the error calculation, the method of partition evaluation can better reflect the alignment of the local area of the point cloud. For example, the feature points can be divided into a crack area and a non-crack area, and the errors of the two types of areas are calculated respectively to provide a reference for the optimization of specific areas.

[0090] The first key feature point in the initially key feature point pair can be iteratively updated until the error is less than the preset matching error threshold, and the iteration is stopped. The target key feature point pair is constructed according to the latest transformed first key feature point and the second key feature point to ensure that the final accuracy of the point cloud alignment meets the task requirements. The iterative update process can include rematching the feature point pairs, calculating the new rotation matrix and translation vector, updating the feature point set, and evaluating the current error. After each iteration, the alignment accuracy of the point cloud is further optimized by dynamically adjusting the transformation parameters. The iteration process stops when the error meets the preset threshold to avoid wasting computing resources caused by over-optimization.

[0091] The setting of the preset matching error threshold can be flexibly adjusted according to task requirements. For example, for millimeter-level inspection in crack detection tasks, the error threshold is usually set at the millimeter level to ensure the reliability of the detection results. After the iteration is completed, according to the final matching results, each pair of corresponding points in the first key feature points and the second key feature points is used as the target key feature point pair, which is further used for subsequent path planning or crack detail analysis.

[0092] By determining the rotation matrix and translation vector, the spatial alignment of the point cloud data is achieved, enabling the accurate matching of the two groups of point clouds in the same coordinate system, thus providing high-quality initial conditions for subsequent feature point optimization; the application of the feature point update transformation further improves the accuracy of point cloud alignment, and significantly enhances the description ability of local features by gradually optimizing the error parameters; the iterative update mechanism ensures the stability and convergence of the point cloud alignment process, and can reach the optimal alignment state through multiple adjustments; in addition, through the dynamic evaluation of the feature point matching error, it is possible to accurately identify the terrain areas with significant feature changes in alpine canyon areas, providing reliable spatial data support for crack detection and inspection route planning.

[0093] In an exemplary embodiment of the present disclosure, it can be achieved through Figure 6 the steps of inputting the fused three-dimensional terrain data corresponding to the local terrain change area into a pre-trained crack inspection route planning model to obtain a local crack fine inspection route. Referring to Figure 6 as shown, it may specifically include: Step S610, extracting the terrain feature parameters in the fused three-dimensional terrain data corresponding to the local terrain change area; Step S620, constructing a state space vector according to the terrain feature parameters and the initial inspection route; Step S630, inputting the state space vector into the crack inspection route planning model to determine the crack collection inspection points in the local terrain change area and the crack collection flight postures of the inspection drones corresponding to each of the crack collection inspection points; Step S640, constructing a local crack fine inspection route based on the crack collection inspection points and the crack collection flight postures.

[0094] Among them, the terrain feature parameters refer to a set of parameters that are refined by analyzing the geometric features and spatial distribution information of the local terrain data and have key guiding significance for path planning. For example, the terrain feature parameters may include but are not limited to height change, slope, surface curvature, crack width, crack direction, and regional complexity. The terrain feature parameters are an abstract description of the local terrain geometric characteristics and environmental conditions, and can provide necessary inputs for inspection path optimization.

[0095] A state space vector can be constructed based on terrain feature parameters and an initial inspection route. Specifically, it refers to integrating local terrain features and UAV flight parameters into a high-dimensional vector, which is used to describe the current inspection state and serve as the input for path planning. For example, the state space vector can include key parameters describing local terrain changes, such as height changes, slopes, crack directions, etc., and at the same time, it also includes the current flight state information of the UAV, such as flight height, speed, and direction. The purpose of constructing the state space vector is to uniformly express multi-dimensional information and provide sufficient data support for the dynamic adjustment of the inspection path.

[0096] In specific implementation, the state space vector can be formed by splicing multi-dimensional data. For example, the height change can be used as the first-dimensional data, the slope as the second-dimensional data, the crack width and direction as the third and fourth-dimensional data respectively, and the flight height, speed, and direction of the UAV as subsequent-dimensional data. To enhance the model's perception ability of the state, normalization processing can be performed on each dimension data in the state space vector to ensure the consistency of each dimension data within the numerical range.

[0097] The state space vector can be input into the crack inspection route planning model, aiming to optimize the inspection path of the UAV through the crack inspection route planning model to ensure the efficient and accurate completion of the crack detection task in complex terrains. The crack inspection route planning model is a path optimization model based on reinforcement learning. Its input is the state space vector, and the output is an action vector, which is used to guide the flight operations of the UAV, including adjusting the flight height, direction, speed, and the scanning resolution of the sensor, etc.

[0098] In the implementation process, the crack inspection route planning model can perform feature analysis on the input state space vector, extract high-dimensional features that have a key impact on path planning, such as the relationship between height change and flight height, the adaptability between slope and flight direction, etc. The crack inspection route planning model can calculate the optimal action vector in the current state based on these features through the policy network. For example, adjust the flight height in steep areas to avoid obstacles, and increase the sensor resolution in areas with dense crack distributions to obtain higher-precision data. The selection of the action vector is based on the experience reward values accumulated during the model training process, that is, through repeated trial and learning, the model can find the optimal strategy to complete the inspection task in complex terrains.

[0099] Optionally, the traditional Dijkstra algorithm can also be combined to globally plan the path in complex areas, while using the reinforcement learning model for local path optimization.

[0100] By extracting terrain feature parameters and constructing a state space vector, the precise expression of complex terrain features can be achieved, providing complete environmental perception input for the path planning model. The crack inspection route planning model dynamically adjusts the inspection path using the state space vector, which can not only optimize the flight trajectory in complex terrain but also adjust the scanning resolution of the sensor and the flight parameters of the UAV in real time according to the crack distribution characteristics and terrain changes. The above technical means enable the UAV to maintain high-efficiency inspection capabilities in complex terrain, while significantly improving the accuracy and comprehensiveness of crack detection, overcoming the defects of rigid path planning and insufficient crack capture ability in existing inspection technologies.

[0101] Optionally, the state space vector can be input into the crack inspection route planning model by Figure 7 the steps in, determining the crack collection inspection points in the locally terrain-changing area and the crack collection flight postures of the inspection UAVs corresponding to each crack collection inspection point, referring to Figure 7 shown as follows, which may specifically include: Step S710: Input the state space vector into the crack inspection route planning model to determine the action vector with the maximum cumulative reward value under the state space vector; Step S720: Update the inspection position and flight posture of the inspection UAV based on the action vector, and record the UAV inspection path and inspection flight posture inferred by the crack inspection route planning model; Step S730: Respond that all the locally terrain-changing areas are covered, or the flight time of the inspection UAV reaches the preset time, confirm that the inspection task inference ends, and determine the crack collection inspection points in the locally terrain-changing area and the crack collection flight postures of the inspection UAVs corresponding to each crack collection inspection point according to the recorded UAV inspection path and inspection flight posture.

[0102] Among them, the action vector with the maximum cumulative reward value under the state space vector refers to using the pre-trained crack inspection route planning model to perform real-time analysis and decision-making on the current inspection state to select the optimal inspection action. The state space vector can include terrain feature parameters of the locally terrain-changing area and the flight state information of the UAV, such as flight altitude, speed, and direction, etc. The crack inspection route planning model is usually constructed based on the reinforcement learning algorithm. By learning the experience data in a large number of inspection tasks, it can predict the most favorable action selection strategy in different states.

[0103] In specific implementation, the crack inspection route planning model can receive the input state space vector, perform feature extraction and processing through a multi-layer neural network or other machine learning models, and output an action vector. This action vector corresponds to the action with the maximum cumulative reward value in the current state, such as adjusting the flight altitude, changing the flight direction, or modifying the scanning resolution, etc. Optionally, different types of reinforcement learning algorithms can also be used, such as Proximal Policy Optimization (PPO), Deep Deterministic Policy Gradient (DDPG), etc., to adapt to inspection tasks with different complexities and requirements.

[0104] The current position and flight state of the UAV can be adjusted in real time according to the action vector output by the model, so as to optimize the inspection route. Specifically, the instructions contained in the action vector can be parsed, such as adjusting the flight altitude, changing the heading angle, or modifying the scanning parameters; the flight control system of the UAV can be used to convert these instructions into specific flight actions, and the required attitude changes can be achieved by adjusting the motors, control surfaces, or other control components of the UAV; at the same time, the system will record the current position, flight direction, and attitude change process of the UAV in real time to generate a complete inspection route and flight attitude record. These records can be used for subsequent data analysis, route optimization, and task evaluation.

[0105] The terrain change area covered by the inspection route can be analyzed in real time. If all target areas have been covered by the inspection route, the task is determined to be completed; at the same time, the flight time of the UAV can be monitored. When the flight time reaches the preset upper limit value, the inspection task can be determined to end regardless of whether the task is fully covered. After confirming the end of the task, based on the recorded inspection route and flight attitude data, the positions of the crack collection inspection points and the flight attitude information of the UAV at these points can be extracted. The flight attitude information can be used to generate the final crack detection report or guide subsequent crack analysis work. Optionally, other different task end conditions can also be set, such as judging the end of the task according to the signal-to-noise ratio or data quality index of crack detection. In addition, an event-triggered end mechanism can also be adopted, for example, when a specific number or type of cracks are detected, the task is automatically terminated. This example embodiment does not make special limitations on this.

[0106] By inputting the state space vector into the crack inspection route planning model and selecting the action vector with the maximum cumulative reward value, the intelligent optimization and dynamic adjustment of the UAV inspection path can be realized; the selection of the action vector is based on real-time terrain features and flight state information, enabling the UAV to flexibly respond to various changes in complex terrains and ensuring the efficiency and comprehensive coverage of the inspection path; the update of the inspection position and flight attitude based on the action vector further improves the response speed and adaptability of the UAV during the execution of the inspection task. At the same time, by recording the inspection path and flight attitude, the traceability of the task data and the accuracy of subsequent analysis are enhanced; by setting the task end condition, the optimization of resource utilization and time management in the inspection task is ensured, and more crack areas can be covered as much as possible within the limited flight time, or the task can be automatically terminated when all areas are covered; the comprehensive application of these technical means enables the UAV inspection to achieve efficient, accurate and reliable crack detection in the complex terrain environment of high mountains and valleys, overcoming the defects of rigid path planning and low inspection efficiency in related technologies.

[0107] In an exemplary embodiment of the present disclosure, the crack inspection route planning model is obtained through a reinforcement learning training process. Referring to Figure 8 as shown, the reinforcement learning training process may specifically include: Step S810, determining the state space of the crack inspection route planning model, where the state space includes the inspection point coordinates and UAV flight attitude in the initial inspection route, the global terrain data of the dam slope in the high mountain and valley area, and the historical flight path information; Step S820, determining the action space of the crack inspection route planning model, where the action space includes adjusting the flight altitude, adjusting the flight direction, adjusting the flight speed, adjusting the shooting angles of the lidar and depth camera, and updating the inspection points; Step S830, sampling inspection interaction data through the state space and the action space, and storing the inspection interaction data in the interaction experience pool. The inspection interaction data is used to simulate the inspection UAV performing the inspection task in the dam slope area of the high mountain and valley area; Step S840, randomly sampling inspection interaction data from the interaction experience pool, training the crack inspection route planning model, aiming to maximize the cumulative reward value of the reward function of the crack inspection route planning model, and optimizing the model parameters of the crack inspection route planning model through gradient descent until the cumulative reward value is greater than or equal to a preset reward value threshold, obtaining the trained crack inspection route planning model.

[0108] Among them, the state space is a set used by the reinforcement learning model to describe the current environment and task state. In the present invention, the state space may include the inspection point coordinates and the UAV flight attitude in the initial inspection route, the global terrain data of the dam slope in the alpine and canyon area, and the historical flight path information.

[0109] The action space refers to the set of operations that the model can select in each state. The action space covers various adjustments and operations that the UAV may perform during the inspection process. For example, the action space may include adjusting the flight altitude, adjusting the flight direction, adjusting the flight speed, adjusting the shooting angles of the lidar and the depth camera, and updating the inspection points.

[0110] The inspection interaction data refers to the records of the states and actions generated by the UAV during the execution of the inspection task when interacting with the environment. The inspection interaction data can be used to train the reinforcement learning model so that it can make better decisions in similar inspection tasks. Specifically, by performing various actions in different states, the interaction data between the inspection UAV and the environment can be collected. For example, when the UAV adjusts the flight altitude, direction, and speed under different terrain conditions, record its flight attitude changes, sensor data acquisition results, and the execution effects of the inspection tasks. Then, the collected interaction data can be stored in the interaction experience pool. The interaction experience pool is a database for storing past interaction data, which may include information such as states, actions, rewards, and the next state. These data will be used for subsequent model training and policy optimization.

[0111] The sampling of the interaction data can be achieved through various methods, such as random sampling, Prioritized Experience Replay, etc., to ensure that the model can fully utilize diverse inspection task experiences during the training process. The design of the experience pool should consider the data storage efficiency and retrieval speed to support the fast access and processing of large-scale data. Of course, different types of data sampling strategies can also be adopted, such as time-series-based sampling or task-importance-based sampling, to improve the efficiency and effect of model training. In addition, the storage structure of the experience pool can also be optimized according to specific application requirements, such as using a hash table or a tree structure to improve the data retrieval speed. This embodiment is not limited thereto.

[0112] Model training is a crucial step in the reinforcement learning process. The aim is to continuously optimize the model parameters so that the model can make optimal decisions in the inspection task and maximize the cumulative reward. Specifically, a batch of inspection interaction data can be randomly sampled from the interaction experience pool. This data includes the feedback information after the drone executes actions in different states, such as the reward value and the new state. Random sampling helps to break the temporal correlation of the data and improve the stability and generalization ability of training. Then, the sampled inspection interaction data can be input into the crack inspection route planning model for model training. During the training process, the model adjusts its parameters to improve the prediction accuracy by comparing the error between the actual reward value and the predicted reward value. Common optimization methods include Stochastic Gradient Descent (SGD), Adam optimizer, etc., to accelerate the convergence process of the model parameters.

[0113] The goal of training is to maximize the cumulative reward value of the model, that is, by selecting the optimal actions, the drone can cover more crack areas, optimize the flight path and save energy in the inspection task. The definition of the cumulative reward value can be based on the specific requirements of the inspection task, such as indicators like crack detection coverage rate, inspection time, and energy consumption. During the training process, the model will continuously iterate and adjust its parameters to gradually improve the quality and efficiency of decision-making. The training process will terminate when the cumulative reward value reaches or exceeds the preset reward value threshold, ensuring that the model has learned an effective inspection route planning strategy.

[0114] Optionally, different types of reinforcement learning algorithms can be used, such as policy gradient methods or value-based algorithms, to adapt to inspection tasks with different complexities and requirements. In addition, the parameter optimization methods during the training process can also be adjusted according to the specific situation, such as adopting adaptive learning rates or batch normalization techniques, to improve the training effect and the convergence speed of the model.

[0115] By determining the state space and action space of the crack inspection route planning model and using the sampled inspection interaction data for reinforcement learning training, the intelligent optimization and adaptive learning of the crack inspection route planning model can be achieved. The comprehensive definition of the state space ensures the model's in-depth understanding of the inspection task environment and the drone's flight state, while the diversity of the action space endows the model with the ability to flexibly adjust the inspection path.

[0116] Through data sampling and model training in the interactive experience pool, the crack inspection route planning model can continuously accumulate and optimize inspection strategies to maximize the cumulative reward value, enabling the model to have a high level of decision-making ability and adaptability in complex terrain environments. It can respond in real time to terrain changes and crack distribution characteristics, dynamically adjusting the flight parameters and inspection paths of the unmanned aerial vehicle (UAV). The trained crack inspection route planning model can not only improve the efficiency and coverage of inspection tasks but also significantly enhance the accuracy and reliability of crack detection, overcoming the defects of rigid inspection path planning and low inspection efficiency in related technologies.

[0117] Figure 9 Schematically shows a route schematic diagram of the adaptive dynamic route inspection of a dam structure according to some embodiments of the present disclosure.

[0118] Reference Figure 9 As shown, when using a UAV to inspect a dam structure, in the traditional method, a preset fixed route 810 is set, and the UAV directly inspects the dam structure according to the preset fixed route 810. However, this inspection method is difficult to dynamically adjust the route according to the terrain data collected in real time or the detection requirements. Such a static method is prone to problems such as insufficient coverage or low efficiency of the inspection path in complex terrain scenarios. Especially in areas with concentrated cracks, the fixed path planning method cannot achieve fine capture of crack characteristics.

[0119] Combined with the slope crack inspection method for alpine canyon areas provided in the embodiments of the present disclosure, it is possible to control the inspection UAV equipped with a lidar and a depth camera to collect the fused three-dimensional terrain data corresponding to the dam structure in real time along the preset initial inspection route 820 at a first scanning resolution. The initial inspection route 820 can be the inspection route determined during the historical inspection of the dam structure by the inspection UAV, or it can be an adaptive dynamic route determined in real time by the inspection UAV based on the data collected by the lidar and the depth camera. The inspection UAV can perform local matching of the collected fused three-dimensional terrain data with the reference three-dimensional terrain data to determine the local terrain change area 830 in real time. For example, the local terrain change area 830 can be a collapsed deformation area or a crack area. Then, the inspection UAV can input the fused three-dimensional terrain data corresponding to the local terrain change area 830 into the crack inspection route planning model to obtain the local crack fine inspection route 840. Furthermore, the inspection UAV can perform adaptive route dynamic planning on the initial inspection route 820 in combination with the local crack fine inspection route 840 to perform fine inspection of the crack terrain in the local terrain change area 830 at a second scanning resolution, obtaining millimeter-level crack inspection data. After the inspection UAV completes the inspection of the local terrain change area 830, it can return to the remaining initial inspection route 820 to continue the inspection until the inspection of the entire dam structure is completed.

[0120] Figure 10 Schematically shows a route schematic diagram of slope adaptive dynamic route inspection according to some embodiments of the present disclosure.

[0121] Reference Figure 10 As shown, when using a drone to inspect a slope, in the traditional method, there are a preset first fixed route 911 and a second fixed route 912. The drone directly inspects the slope according to the first fixed route 911 and the second fixed route 912. However, this inspection method is difficult to dynamically adjust the route according to the slope terrain data collected in real time or the detection requirements, resulting in the inability to timely discover the areas that need to be focused on in the slope terrain, or it is necessary to conduct multiple inspections to complete the data collection of all the key terrain areas of the slope terrain.

[0122] Combined with the slope crack inspection method for alpine canyon areas provided in the embodiments of the present disclosure, similar to Figure 9 the inspection of the dam structure, it is possible to control an inspection drone equipped with a lidar and a depth camera to collect the fused three-dimensional terrain data corresponding to the slope in real time along a preset first initial inspection route 921 or a second initial inspection route 924 at a first scanning resolution; the inspection drone can perform local matching of the collected fused three-dimensional terrain data with the reference three-dimensional terrain data to determine a first local terrain change area 922 or a second local terrain change area 925 in real time; then the inspection drone can input the fused three-dimensional terrain data corresponding to the first local terrain change area 922 or the second local terrain change area 925 into a crack inspection route planning model to obtain a first local crack fine inspection route 923 or a second local crack fine inspection route 926; furthermore, the inspection drone can adaptively plan the dynamic route of the first initial inspection route 921 or the second initial inspection route 924 in combination with the first local crack fine inspection route 923 or the second local crack fine inspection route 926, and perform fine inspection of the crack terrain in the first local terrain change area 922 or the second local terrain change area 925 at a second scanning resolution to obtain millimeter-level crack inspection data; after the inspection drone completes the inspection of the first local terrain change area 922 or the second local terrain change area 925, it can return to the remaining first initial inspection route 921 or the second initial inspection route 924 to continue the inspection until the inspection of the entire slope is completed.

[0123] It should be noted that although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that these steps must be executed in this specific order, or that all the shown steps must be executed to achieve the desired result. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc.

[0124] In addition, in the present exemplary embodiment, a slope crack inspection device for alpine canyon areas is also provided. Referring to Figure 11 As shown, the slope crack inspection device 1100 for alpine canyon areas includes: a three-dimensional terrain acquisition module 1110, a terrain change area determination module 1120, an adaptive flight path planning module 1130, and a crack fine inspection module 1140. Among them: The three-dimensional terrain acquisition module 1110 is configured to control the inspection drone to collect, in real time at a first scanning resolution, the fused three-dimensional terrain data corresponding to the dam slope in the alpine canyon area along a preset initial inspection flight path, and the fused three-dimensional terrain data is collected by a lidar and a depth camera carried by the inspection drone; The terrain change area determination module 1120 is configured to perform local matching on the fused three-dimensional terrain data and pre-constructed reference three-dimensional terrain data to determine a local terrain change area; The adaptive flight path planning module 1130 is configured to input the fused three-dimensional terrain data corresponding to the local terrain change area into a pre-trained crack inspection route planning model to obtain a local crack fine inspection flight path; The crack fine inspection module 1140 is configured to control the inspection drone based on the local crack fine inspection flight path to finely inspect the crack terrain in the local terrain change area at a second scanning resolution to obtain millimeter-level crack inspection data.

[0125] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the three-dimensional terrain acquisition module 1110 is configured to: obtain the terrain elevation data and terrain structure point cloud data corresponding to the dam slope in the alpine canyon area respectively collected in real time by the lidar and the depth camera at a first scanning resolution; register and align the terrain elevation data and the terrain structure point cloud data to obtain preliminary three-dimensional terrain data; and perform depth fusion on the preliminary three-dimensional terrain data to obtain the fused three-dimensional terrain data corresponding to the dam slope in the alpine canyon area.

[0126] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the terrain change area determination module 1120 is configured to: extract the first key feature points in the fused three-dimensional terrain data, and extract the second key feature points in the reference three-dimensional terrain data; perform rough registration on the first key feature points and the second key feature points to determine preliminary key feature point pairs; perform iterative optimization on the preliminary key feature point pairs to determine target key feature point pairs, and determine at least one local sub-region according to the target key feature point pairs, the local sub-region including a steep region, a gentle slope region, and a crack distribution region; determine the local minimum mean square error corresponding to each local sub-region according to the target key feature point pairs; use the local sub-region with the local minimum mean square error greater than or equal to the preset terrain error threshold as the local terrain change area.

[0127] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the adaptive flight path planning module 1130 is configured to: determine a rotation matrix and a translation vector according to the preliminary key feature point pairs, the rotation matrix and the translation vector minimizing the mean square error between the matching point pairs; perform an update transformation on the first key feature points belonging to the preliminary key feature point pairs through the rotation matrix and the translation vector, and determine the mean square error between the updated first key feature points and the second key feature points; iteratively update the first key feature points in the preliminary key feature point pairs until the mean square error is less than the preset matching error threshold, stop the iteration, and construct a target key feature point pair according to the first key feature points and the second key feature points after the latest transformation.

[0128] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the adaptive flight path planning module 1130 is configured to: extract the terrain feature parameters in the fused three-dimensional terrain data corresponding to the local terrain change area; construct a state space vector according to the terrain feature parameters and the initial inspection flight path; input the state space vector into the crack inspection route planning model to determine the crack collection inspection points in the local terrain change area and the crack collection flight postures of the inspection unmanned aerial vehicles corresponding to each crack collection inspection point; construct a local crack fine inspection flight path based on the crack collection inspection points and the crack collection flight postures.

[0129] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the adaptive route planning module 1130 is configured to: input the state space vector into the crack inspection route planning model to determine an action vector with the maximum cumulative reward value under the state space vector; update the inspection position and flight attitude of the inspection UAV based on the action vector, and record the UAV inspection path and inspection flight attitude inferred by the crack inspection route planning model; in response to all the locally terrain-changed areas being covered, or the flight time of the inspection UAV reaching a preset time, confirm that the inspection task inference ends, and determine the crack collection inspection points in the locally terrain-changed area and the crack collection flight attitudes of the inspection UAV corresponding to each of the crack collection inspection points according to the recorded UAV inspection path and inspection flight attitude.

[0130] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the slope crack inspection device 1100 for alpine canyon areas includes a reinforcement learning training module, and the reinforcement learning training module is configured to: determine the state space of the crack inspection route planning model, where the state space includes the inspection point coordinates and UAV flight attitude in the initial inspection route, the global terrain data of the dam slope in the alpine canyon area, and the historical flight path information; determine the action space of the crack inspection route planning model, where the action space includes adjusting the flight altitude, adjusting the flight direction, adjusting the flight speed, adjusting the shooting angles of the lidar and depth camera, and updating the inspection points; sample inspection interaction data through the state space and the action space, and store the inspection interaction data in an interaction experience pool, where the inspection interaction data is used to simulate the inspection UAV performing inspection tasks in the dam slope area of the alpine canyon area; randomly sample inspection interaction data from the interaction experience pool, perform model training on the crack inspection route planning model, with the goal of maximizing the cumulative reward value of the reward function of the crack inspection route planning model, and optimize the model parameters of the crack inspection route planning model through gradient descent until the cumulative reward value is greater than or equal to a preset reward value threshold to obtain a trained crack inspection route planning model.

[0131] The specific details of each module of the above-mentioned slope crack inspection device for alpine canyon areas have been described in detail in the corresponding slope crack inspection method for alpine canyon areas, so they will not be repeated here.

[0132] It should be noted that although several modules or units of the slope crack inspection device for alpine canyon areas are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0133] In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above-described slope crack inspection method for alpine canyon areas is also provided.

[0134] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, a method, or a program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module", or "system" here.

[0135] The following refers to Figure 12 to describe the electronic device 1000 according to such an embodiment of the present disclosure. Figure 12 The illustrated electronic device 1000 is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0136] As Figure 12 shown, the electronic device 1000 is presented in the form of a general-purpose computing device. The components of the electronic device 1000 may include, but are not limited to: the at least one processing unit 1010 described above, the at least one storage unit 1020 described above, a bus 1030 connecting different system components (including the storage unit 1020 and the processing unit 1010), and a display unit 1040.

[0137] Among them, the storage unit stores program code, and the program code can be executed by the processing unit 1010, so that the processing unit 1010 executes the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of this specification. For example, the processing unit 1010 can execute as Figure 2In step S210 shown in the figure, the inspection UAV is controlled to collect in real time the fused three-dimensional terrain data corresponding to the dam slope in the alpine canyon area along a preset initial inspection route. The fused three-dimensional terrain data is obtained by the lidar and depth camera carried by the inspection UAV; in step S220, the fused three-dimensional terrain data is locally matched with the pre-constructed reference three-dimensional terrain data to determine the local terrain change area; in step S230, the fused three-dimensional terrain data corresponding to the local terrain change area is input into a pre-trained crack inspection route planning model to obtain a local crack fine inspection route; in step S240, based on the local crack fine inspection route, the inspection UAV is controlled to perform adaptive route dynamic planning, and the crack terrain in the local terrain change area is finely inspected at a second scan resolution to obtain millimeter-level crack inspection data.

[0138] The storage unit 1020 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 1021 and / or a cache storage unit 1022, and may further include a read-only storage unit (ROM) 1023.

[0139] The storage unit 1020 may further include a program / utilities 1024 having a set (at least one) of program modules 1025. Such program modules 1025 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment.

[0140] The bus 1030 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0141] The electronic device 1000 can also communicate with one or more external devices 1070 (such as a keyboard, a pointing device, a Bluetooth device, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 1000, and / or communicate with any device that enables the electronic device 1000 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 1050. Moreover, the electronic device 1000 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 1060. As shown in the figure, the network adapter 1060 communicates with other modules of the electronic device 1000 through the bus 1030. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 1000, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0142] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or can be implemented by the way of software combined with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, and the software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0143] In addition, the above drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, rather than for limiting purposes. It can be easily understood that the processes shown in the above drawings do not indicate or limit the time sequence of these processes. Additionally, it can be easily understood that these processes can be executed synchronously or asynchronously, for example, in multiple modules.

[0144] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or can be implemented by the way of software combined with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, and the software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0145] Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and examples are only illustrative, and the true scope and spirit of the present disclosure are pointed out by the claims.

[0146] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A method for inspecting slope cracks in alpine valley areas, characterized in that, Including: Controlling the inspection UAV to collect in real time the fused three-dimensional terrain data corresponding to the dam slope in the alpine canyon area along a preset initial inspection route at a first scanning resolution, where the fused three-dimensional terrain data is collected by a lidar and a depth camera carried by the inspection UAV; Performing local matching between the fused three-dimensional terrain data and pre-constructed reference three-dimensional terrain data to determine the local terrain change area in real time; Inputting the fused three-dimensional terrain data corresponding to the local terrain change area into a pre-trained crack inspection route planning model to obtain a local crack fine inspection route; Controlling the inspection UAV to perform adaptive route dynamic planning based on the local crack fine inspection route, and performing fine inspection on the crack terrain in the local terrain change area at a second scanning resolution to obtain millimeter-level crack inspection data.

2. The slope crack inspection method for alpine canyon areas according to claim 1, wherein, The step of collecting in real time the fused three-dimensional terrain data corresponding to the dam slope in the alpine canyon area at a first scanning resolution includes: Obtaining the terrain elevation data and terrain structure point cloud data corresponding to the dam slope in the alpine canyon area collected in real time by the lidar and the depth camera respectively at a first scanning resolution; Registering and aligning the terrain elevation data and the terrain structure point cloud data to obtain preliminary three-dimensional terrain data; Performing depth fusion on the preliminary three-dimensional terrain data to obtain the fused three-dimensional terrain data corresponding to the dam slope in the alpine canyon area.

3. The slope crack inspection method for alpine canyon areas according to claim 1, wherein The step of performing local matching between the fused three-dimensional terrain data and pre-constructed reference three-dimensional terrain data to determine the local terrain change area includes: Extracting first key feature points from the fused three-dimensional terrain data and second key feature points from the reference three-dimensional terrain data; Performing rough registration on the first key feature points and the second key feature points to determine preliminary key feature point pairs; Performing iterative optimization on the preliminary key feature point pairs to determine target key feature point pairs, and determining at least one local sub-region according to the target key feature point pairs, where the local sub-region includes a steep area, a gentle slope area, and a crack distribution area; Determining the local minimum mean square error corresponding to each local sub-region according to the target key feature point pairs; Regarding the local sub-regions with the local minimum mean square error greater than or equal to a preset terrain error threshold as the local terrain change area.

4. The slope crack inspection method for alpine canyon areas according to claim 3, characterized in that The step of performing iterative optimization on the preliminary key feature point pairs to determine target key feature point pairs includes: Determining a rotation matrix and a translation vector according to the preliminary key feature point pairs, where the rotation matrix and the translation vector minimize the mean square error between the matching point pairs; Updating and transforming the first key feature points belonging to the preliminary key feature point pairs through the rotation matrix and the translation vector, and determining the mean square error between the updated and transformed first key feature points and the second key feature points; Iteratively updating the first key feature points in the preliminary key feature point pairs until the mean square error is less than a preset matching error threshold, stopping the iteration, and constructing target key feature point pairs according to the first key feature points and the second key feature points after the latest transformation.

5. The slope crack inspection method for alpine canyon areas according to claim 1, characterized in that, Inputting the fused three-dimensional terrain data corresponding to the local terrain change area into a pre-trained crack inspection route planning model to obtain a fine local crack inspection route, including: Extracting terrain feature parameters from the fused three-dimensional terrain data corresponding to the local terrain change area; Constructing a state space vector based on the terrain feature parameters and the initial inspection route; Inputting the state space vector into the crack inspection route planning model to determine the crack collection inspection points in the local terrain change area and the crack collection flight postures of the inspection UAVs corresponding to the respective crack collection inspection points; Constructing a fine local crack inspection route based on the crack collection inspection points and the crack collection flight postures.

6. The slope crack inspection method for alpine canyon areas according to claim 5, characterized in that, The step of inputting the state space vector into the crack inspection route planning model to determine the crack collection inspection points in the local terrain change area and the crack collection flight postures of the inspection UAVs corresponding to the respective crack collection inspection points includes: Inputting the state space vector into the crack inspection route planning model to determine an action vector with the maximum cumulative reward value under the state space vector; Updating the inspection position and flight posture of the inspection UAV based on the action vector, and recording the UAV inspection path and inspection flight posture inferred by the crack inspection route planning model; Responding to the situation that the local terrain change area is all covered or the flight time of the inspection UAV reaches a preset time, confirming the end of the inspection task inference, and determining the crack collection inspection points in the local terrain change area and the crack collection flight postures of the inspection UAVs corresponding to the respective crack collection inspection points according to the recorded UAV inspection path and inspection flight posture.

7. The slope crack inspection method for alpine valley areas according to claim 1 or 5, characterized in that The crack inspection route planning model is obtained through a reinforcement learning training process, and the reinforcement learning training process includes: Determining the state space of the crack inspection route planning model, where the state space includes the inspection point coordinates and UAV flight postures in the initial inspection route, the global terrain data of the dam slope in the alpine and canyon area, and the historical flight path information; Determining the action space of the crack inspection route planning model, where the action space includes adjusting the flight height, adjusting the flight direction, adjusting the flight speed, adjusting the shooting perspectives of the lidar and depth camera, and updating the inspection points; Sampling inspection interaction data through the state space and the action space, and storing the inspection interaction data in an interaction experience pool, where the inspection interaction data is used to simulate the inspection UAV performing inspection tasks in the dam slope area of the alpine and canyon area; Randomly sampling inspection interaction data from the interaction experience pool, training the crack inspection route planning model, aiming to maximize the cumulative reward value of the reward function of the crack inspection route planning model, and optimizing the model parameters of the crack inspection route planning model through gradient descent until the cumulative reward value is greater than or equal to a preset reward value threshold to obtain a trained crack inspection route planning model.

8. A slope crack inspection device for alpine canyon areas, characterized in that, Including: A three-dimensional terrain acquisition module, which is used to control the inspection UAV to collect the fused three-dimensional terrain data corresponding to the dam slope in the alpine canyon area in real time along a preset initial inspection route at a first scanning resolution. The fused three-dimensional terrain data is collected by a lidar and a depth camera carried by the inspection UAV; A terrain change area determination module, which is used to perform local matching on the fused three-dimensional terrain data and a pre-constructed reference three-dimensional terrain data to determine the local terrain change area in real time; An adaptive route planning module, which is used to input the fused three-dimensional terrain data corresponding to the local terrain change area into a pre-trained crack inspection route planning model to obtain a local crack fine inspection route; A crack fine inspection module, which is used to control the inspection UAV to perform adaptive route dynamic planning based on the local crack fine inspection route, and to finely inspect the crack terrain in the local terrain change area at a second scanning resolution to obtain millimeter-level crack inspection data.

9. An electronic device, characterized in that, It includes: A processor; And A memory, on which computer-readable instructions are stored. When the computer-readable instructions are executed by the processor, the method for slope crack inspection in alpine canyon areas described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that, On which a computer program is stored. When the computer program is executed by the processor, the method for slope crack inspection in alpine canyon areas described in any one of claims 1 to 7 is implemented.

Images