Slope crack inspection method and device based on unmanned aerial vehicle route dynamic planning

Abstract

The invention provides a slope crack inspection method and device based on unmanned aerial vehicle route dynamic planning, and relates to the technical field of dam slope safety monitoring. The slope crack inspection method based on unmanned aerial vehicle route dynamic planning comprises the following steps: controlling an unmanned aerial vehicle to inspect along an initial inspection route, and dynamically adjusting the initial inspection route according to a real flight attitude of the unmanned aerial vehicle to obtain a full-coverage inspection route; routing inspection is carried out along the full-coverage routing inspection route, and an area-of-interest with cracks on the dam slope area is determined; locally updating the full-coverage inspection route according to the crack extension direction determined by the region of interest to obtain a crack key inspection route; the unmanned aerial vehicle is controlled to complete inspection of the dam slope area through the crack key inspection route, and a global crack inspection image is obtained. According to the technical scheme, the unmanned aerial vehicle can adapt to a complex environment, the image quality of the crack inspection image is improved, and the unmanned aerial vehicle crack monitoring precision and inspection efficiency are improved.

Description

Background Art

[0002] With the increasing importance of the safety of dam slopes in alpine and canyon regions, unmanned aerial vehicles (UAVs), as a flexible and efficient inspection tool, have gradually been widely used in the monitoring of cracks or rolling stone risks on dam slopes. In related technologies, UAVs usually conduct inspections on dam slopes along preset fixed routes. However, the terrain in alpine and canyon regions is complex, the distribution forms of cracks are diverse and their extension directions are variable, and the shapes and positions of rolling stones also have a high degree of uncertainty. The mode of fixed inspection routes often fails to comprehensively cover the target area in such environments, which may lead to the existence of monitoring blind spots, thus affecting the integrity and reliability of inspection data.

[0003] The detection of cracks is one of the key technologies to ensure the stability of dam slopes. Related methods usually identify cracks through aerial images taken by UAVs. However, the shapes of cracks are diverse and their scales vary significantly. Especially in alpine and canyon regions, the width and extension direction of cracks may change significantly with the terrain. Traditional crack detection technologies often rely on simple image segmentation or manual annotation methods, and these methods have low accuracy in identifying slope cracks and tracking the extension direction, making it difficult to meet the refined requirements of actual engineering for crack detection.

[0004] In addition, the complex canyon terrain environment also poses higher requirements for the flight control of UAVs. Affected by external factors such as wind changes and canyon terrain occlusion, the actual flight attitude of UAVs is prone to deviate from the preset route. And related technologies lack a dynamic adjustment mechanism based on real-time flight attitude, resulting in coverage blind spots or data redundancy in inspection results. The limitations of this UAV inspection mode further reduce the efficiency of crack monitoring and increase the difficulty of engineering maintenance.

[0005] Therefore, the technologies for monitoring cracks or rolling stones on dam slopes in alpine and canyon regions at least face problems such as coverage blind spots, low crack identification accuracy, insufficient dynamic monitoring capabilities for cracks or rolling stones, and unstable flight attitude control. These problems limit the application ability of UAVs in complex terrain environments. There is an urgent need for a UAV inspection method that can dynamically adapt to complex environments and improve crack detection accuracy to provide reliable data support for the safety of dam slopes.

[0006] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present disclosure, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0007] The purpose of the embodiments of the present disclosure is to provide a slope crack inspection method based on dynamic planning of UAV flight routes, a slope crack inspection device based on dynamic planning of UAV flight routes, an electronic device, and a computer-readable storage medium, so as to enable the UAV to adapt to complex environments, improve the image quality of crack inspection images, and improve the accuracy and inspection efficiency of UAV crack monitoring.

[0008] Other features and advantages of the present disclosure will become apparent from the following detailed description, or will be partially learned through the practice of the present disclosure.

[0009] According to the first aspect of the embodiments of the present disclosure, a slope crack inspection method based on dynamic planning of UAV flight routes is provided, including: Obtain a pre-planned initial inspection flight route, and control the UAV to perform inspections along the initial inspection flight route; Determine the actual flight attitude of the UAV, and dynamically adjust the initial inspection flight route according to the actual flight attitude to obtain a full-coverage inspection flight route; Based on the current inspection images collected when the UAV performs inspections along the full-coverage inspection flight route, determine the regions of interest where cracks exist on the dam slope area; Determine the crack extension direction according to the regions of interest, and locally update the full-coverage inspection flight route according to the crack extension direction to obtain a key crack inspection route; Control the UAV to complete the inspection of the dam slope area through the key crack inspection route to obtain a global crack inspection image, so as to determine the slope crack parameters through the global crack inspection image.

[0010] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the determining the actual flight attitude of the UAV includes: obtaining the inspection image frames collected by the UAV during inspections, and the inertial sensor data when the inspection image frames are collected; estimating the initial flight attitude of the UAV through the inertial sensor data; determining the key image feature points in the inspection image frames, and determining the reprojection error through the key image feature points; combining a preset sliding time window, and optimizing the initial flight attitude by minimizing the reprojection error to obtain the actual flight attitude of the UAV.

[0011] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the method of dynamically adjusting the initial inspection route according to the true flight attitude to obtain a full-coverage inspection route includes: determining an ideal inspection coverage area through the part of the initial inspection route that has been inspected by the unmanned aerial vehicle (UAV); determining the actual inspection coverage area of the UAV according to the true flight attitude; determining the inspection coverage rate and the uncovered area based on the ideal inspection coverage area and the actual inspection coverage area; determining supplementary inspection points through the inspection coverage rate and the uncovered area, and dynamically updating the initial inspection route according to the supplementary inspection points to obtain a full-coverage inspection route.

[0012] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the method of determining the region of interest where cracks exist on the dam slope area based on the current inspection image collected when the UAV inspects along the full-coverage inspection route includes: performing feature extraction processing on the current inspection image to extract the crack boundary feature points in the current inspection image; determining the coordinate position of the crack feature points in the dam slope area according to the coordinates of the crack boundary feature points in the current inspection image, the full-coverage inspection route, and the true flight attitude of the UAV when collecting the current inspection image; combining the coordinate position and each segmented sub-region in the dam slope area to determine the set of segmented sub-regions where cracks exist, and obtaining the region of interest through the set of segmented sub-regions.

[0013] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the method of determining the crack extension direction according to the region of interest and locally updating the full-coverage inspection route according to the crack extension direction to obtain a key inspection route for cracks includes: obtaining the region numbers of each segmented sub-region in the region of interest, and determining the crack extension direction according to the relative position relationship between the region numbers; determining a plurality of inspection points for crack collection according to the crack extension direction, and locally updating the full-coverage inspection route according to the inspection points for crack collection to obtain a key inspection route for cracks.

[0014] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the determining of a plurality of crack acquisition and inspection points according to the crack extension direction further includes: determining a plurality of basic crack acquisition points according to the crack extension direction and the crack acquisition field of view angle of the unmanned aerial vehicle (UAV) at a preset crack acquisition height; combining a crack distribution characteristic function and crack boundary feature points to determine local gradient change data of the cracks in the region of interest; estimating potential extended crack points in the crack extension direction through the local gradient change data, and determining a crack extension region according to the potential extended crack points; determining a plurality of extended crack acquisition points based on the crack extension region and the crack acquisition field of view angle; and using the basic crack acquisition points and the extended crack acquisition points as the crack acquisition and inspection points.

[0015] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the global crack inspection image includes a millimeter-level crack image; controlling the UAV to complete the inspection of the dam slope region through the crack key inspection route to obtain the global crack inspection image, including: obtaining preset crack acquisition data corresponding to the UAV, where the preset crack acquisition data includes a crack acquisition height, a crack acquisition roll angle, and a crack acquisition camera pitch angle; in response to the UAV arriving at a crack acquisition and inspection point on the crack key inspection route, controlling the crack acquisition flight attitude of the UAV through the preset crack acquisition data, so that the camera view angle of the UAV is perpendicular to the region of interest, and acquiring the millimeter-level crack image.

[0016] According to a second aspect of the embodiments of the present disclosure, there is provided a slope crack inspection device based on dynamic planning of a UAV flight route, including: An initial inspection module, configured to obtain a pre-planned initial inspection flight route and control the UAV to perform inspections along the initial inspection flight route; A coverage inspection route planning module, configured to determine the actual flight attitude of the UAV and dynamically adjust the initial inspection flight route according to the actual flight attitude to obtain a full-coverage inspection flight route; A crack detection module, configured to determine a region of interest with cracks on the dam slope region based on a current inspection image acquired when the UAV performs inspections along the full-coverage inspection flight route; A crack inspection route planning module, configured to determine a crack extension direction according to the region of interest and locally update the full-coverage inspection flight route according to the crack extension direction to obtain a crack key inspection route; A crack image acquisition module, configured to control the UAV to complete the inspection of the dam slope region through the crack key inspection route to obtain a global crack inspection image, so as to determine slope crack parameters through the global crack inspection image.

[0017] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, including: a processor; and a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, the method for inspecting slope cracks based on dynamic planning of UAV flight routes described in any one of the above is implemented.

[0018] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the method for inspecting slope cracks based on dynamic planning of UAV flight routes described in any one of the above is implemented.

[0019] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects: In the method for inspecting slope cracks based on dynamic planning of UAV flight routes in the exemplary embodiments of the present disclosure, in a complex environment in alpine and canyon areas, by dynamically adjusting the UAV flight route, it can adapt to the complex terrain characteristics of the dam slope. The initially planned inspection flight route combined with the real-time obtained UAV flight attitude enables the UAV to dynamically adjust the flight route according to environmental changes, thereby optimizing the coverage range of the flight route and reducing inspection blind spots caused by complex terrain or environmental interference.

[0020] Combined with the processing of inspection images, it can accurately identify cracks in the target area, mark the areas of interest during the inspection process, and further optimize the flight route using the crack extension direction in the areas of interest. It can generate key inspection routes according to the actual distribution characteristics of the cracks, thereby improving the recognition ability of crack distribution and solving the problem of inaccurate recognition of crack morphology and extension direction in traditional crack detection technologies.

[0021] Through the dynamic adjustment of the key inspection route for cracks, it can more efficiently conduct refined inspections on crack areas in complex slopes, enabling the UAV to effectively cover key areas within a limited flight time, avoiding the phenomena of redundant inspection data and omission of important areas in traditional methods. Especially in the case where there is an interactive influence between cracks and rolling stones, the dynamic inspection route can increase key coverage in areas prone to rolling stones, enabling more comprehensive collection of relevant data on the distribution of rolling stones and cracks.

[0022] Utilizing the dual optimization of the full-coverage inspection flight route and the key inspection route for cracks can reduce the dependence on manual marking, realize automatic identification and dynamic tracking of crack areas, and significantly improve the monitoring efficiency. At the same time, by adjusting the flight attitude control in real time, the UAV can maintain stable flight when affected by the external environment. Through the means of dynamically adjusting the flight route, the continuity and coverage rate of the inspection results are significantly improved, thus meeting the high-precision requirements for monitoring dam slope cracks and rolling stones in complex alpine and canyon terrains.

[0023] In a complex area with cracks and rolling stones distributed on the dam slope, by using the method of dynamically adjusting the flight path, the inspection path can be quickly updated after initially identifying the crack area, key inspections can be carried out on the crack extension direction and the dynamic changes of rolling stones, and the generated global inspection image can completely reflect the distribution characteristics of cracks and rolling stones, providing detailed and reliable data support for subsequent safety assessment and repair. Through the above method, problems such as insufficient inspection coverage, insufficient crack detection accuracy, and unstable flight control in related technologies can be effectively solved, and the comprehensiveness and accuracy of dam slope inspection can be improved.

[0024] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0026] Figure 1 The schematic diagram of the system architecture showing an exemplary application environment of a slope crack inspection method and device based on dynamic planning of UAV flight paths to which the embodiments of the present disclosure can be applied.

[0027] Figure 2 The schematic flow chart showing the slope crack inspection method based on dynamic planning of UAV flight paths according to some embodiments of the present disclosure.

[0028] Figure 3 The schematic flow chart showing the process of determining the true flight attitude of the UAV according to some embodiments of the present disclosure.

[0029] Figure 4 The schematic flow chart showing the process of updating to obtain a full-coverage inspection flight path according to some embodiments of the present disclosure.

[0030] Figure 5 The schematic flow chart showing the process of generating crack collection inspection points according to some embodiments of the present disclosure.

[0031] Figure 6 The schematic diagram showing the slope crack inspection device based on dynamic planning of UAV flight paths according to some embodiments of the present disclosure.

[0032] Figure 7 The schematic diagram of the structure of the computer system of an electronic device showing according to some embodiments of the present disclosure.

[0033] Figure 8 FIG. schematically shows a schematic diagram of a computer-readable storage medium according to some embodiments of the present disclosure.

[0034] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts. DETAILED DESCRIPTION

[0035] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0036] In addition, the drawings are only schematic diagrams and are not necessarily drawn to scale. The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0037] Figure 1 FIG. shows a schematic diagram of a system architecture of an exemplary application environment of a slope crack inspection method and apparatus based on dynamic planning of UAV flight routes to which embodiments of the present disclosure can be applied.

[0038] As Figure 1 shown, the system architecture 100 may include a UAV device 101, a network 102, and a server 103. The network 102 is used to provide a medium for a communication link between the UAV device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc. The UAV device 101 may be a remotely controlled flight device equipped with an image acquisition unit, including but not limited to various types of UAVs. The embodiments of the present disclosure do not make special limitations on the type of the UAV device 101. It should be understood that Figure 1 the number of UAV devices, networks, and servers in

[0039] The slope crack inspection method based on dynamic planning of UAV flight routes provided by the embodiments of the present disclosure can be executed by the UAV device 101. Correspondingly, the slope crack inspection device based on dynamic planning of UAV flight routes is generally arranged in the UAV device 101. However, those skilled in the art can easily understand that the slope crack inspection method based on dynamic planning of UAV flight routes provided by the embodiments of the present disclosure can also be executed by the server 103. Correspondingly, the slope crack inspection device based on dynamic planning of UAV flight routes can also be arranged in the server 103. No special limitation is made in this exemplary embodiment.

[0040] In the present exemplary embodiment, first, a slope crack inspection method based on dynamic planning of UAV flight routes is provided. The slope crack inspection method based on dynamic planning of UAV flight routes can be applied to the UAV device or the server. Hereinafter, the case where the server executes this method will be taken as an example for description. Figure 2 A flowchart of the slope crack inspection method based on dynamic planning of UAV flight routes according to some embodiments of the present disclosure is schematically shown. Refer to Figure 2 As shown, the slope crack inspection method based on dynamic planning of UAV flight routes may include the following steps: Step S210: Obtain a pre-planned initial inspection flight route, and control the UAV to perform inspections along the initial inspection flight route; Step S220: Determine the actual flight attitude of the UAV, and dynamically adjust the initial inspection flight route according to the actual flight attitude to obtain a full-coverage inspection flight route; Step S230: Based on the current inspection images collected when the UAV performs inspections along the full-coverage inspection flight route, determine the regions of interest where cracks exist on the dam slope area; Step S240: Determine the crack extension direction according to the regions of interest, and locally update the full-coverage inspection flight route according to the crack extension direction to obtain a crack key inspection route; Step S250: Control the UAV to complete the inspection of the dam slope area through the crack key inspection route to obtain a global crack inspection image, so as to determine the slope crack parameters through the global crack inspection image.

[0041] According to the slope crack inspection method based on dynamic planning of UAV flight routes in this exemplary embodiment, by combining the pre-planned initial inspection route with the real-time obtained UAV flight attitude, the UAV can dynamically adjust the route according to environmental changes, so as to optimize the route coverage range and reduce inspection blind spots caused by complex terrain or environmental interference; combined with the processing of inspection images, cracks in the target area can be accurately identified, and areas of interest can be marked during the inspection process. Using the crack extension direction in the areas of interest to further optimize the route, key inspection routes can be generated according to the actual distribution characteristics of the cracks, thereby improving the ability to identify the crack distribution; through the dynamic adjustment of the key inspection routes for cracks, the crack areas in complex slopes can be more efficiently and finely inspected, enabling the UAV to effectively cover key areas within a limited flight time, avoiding the phenomena of redundant inspection data and omission of important areas in traditional methods; by using the dual optimization of the full-coverage inspection route and the key inspection route for cracks, the dependence on manual marking can be reduced, automatic identification and dynamic tracking of crack areas can be achieved, and the monitoring efficiency can be significantly improved. At the same time, the flight attitude control is adjusted in real time to keep the UAV flying stably when affected by the external environment. By means of dynamically adjusting the route, the continuity and coverage rate of the inspection results are significantly improved, so as to meet the high-precision requirements for dam slope crack and rolling stone monitoring in complex terrains of high mountains and valleys; in areas with complex distributions of dam slope cracks and rolling stones, by using the method of dynamically adjusting the route, the inspection path can be quickly updated after initially identifying the crack area, key inspections can be carried out on the crack extension direction and the dynamic changes of rolling stones, and the generated global inspection images can completely reflect the distribution characteristics of cracks and rolling stones, providing detailed and reliable data support for subsequent safety assessment and repair. Through the above method, problems such as insufficient inspection coverage, insufficient crack detection accuracy, and unstable flight control in related technologies can be effectively solved, and the comprehensiveness and accuracy of dam slope inspection can be improved.

[0042] Next, the slope crack inspection method based on dynamic planning of UAV flight routes in this exemplary embodiment will be further described.

[0043] In step S210, obtain the pre-planned initial inspection route, and control the UAV to perform inspections along the initial inspection route.

[0044] In an exemplary embodiment of the present disclosure, the initial inspection flight path refers to a set of flight paths pre-planned based on the topographic characteristics of the dam slope area, historical data of crack distribution, and performance parameters of the inspection equipment, including multiple waypoints defined by spatial coordinates, and the connection between waypoints forms a flight path. The generation of the flight path can be combined with the geometric model and segmentation method of the dam slope area. For example, the target area can be divided into several sub-areas to ensure that each sub-area is at least covered once on the initial inspection flight path; in a specific implementation, a grid algorithm can be used to generate basic coverage waypoints, and the distribution and order of waypoints can be adjusted through an optimization algorithm to make the flight path more in line with the flight efficiency and coverage requirements of the unmanned aerial vehicle (UAV). When the UAV flies along the initial inspection flight path, the flight path data is loaded through the flight control system, and the thrust and direction are adjusted in real time according to the target coordinates of the current waypoint to ensure that the UAV flies stably along the set path.

[0045] After taking off, the UAV loads the initial inspection flight path and visits each waypoint one by one through the flight control system to complete flight path control. The flight control system calculates the flight path based on the loaded waypoint coordinates and the real-time acquired position information, and ensures that the UAV flies stably along the initial flight path by adjusting the thrust, attitude, and direction of the UAV. During the inspection process, high-resolution images are captured by triggering at a set time interval or waypoint position, and the captured images are associated and stored with the waypoint numbers. It can be understood that the flight path control instructions can also be sent in real time through the ground station to control the UAV instead of the UAV autonomously loading the flight path.

[0046] In step S220, determine the true flight attitude of the UAV, and dynamically adjust the initial inspection flight path according to the true flight attitude to obtain a full-coverage inspection flight path.

[0047] In an exemplary embodiment of the present disclosure, the true flight attitude is a real-time three-dimensional attitude calculated through a visual inertial algorithm based on multi-source data obtained by an inertial measurement unit (IMU) and a camera carried by the UAV. The inertial measurement unit is responsible for collecting the acceleration and angular velocity data of the UAV, and the camera provides visual data through feature point extraction and tracking. After inputting the two types of data into the visual inertial algorithm, the drift error of the inertial measurement unit is corrected by minimizing the reprojection error of the feature points, and an accurate real-time pose is generated.

[0048] The process of dynamically adjusting the initial inspection flight path may include operations such as calculating the flight path coverage rate, identifying uncovered areas, and generating supplementary waypoints. The flight path coverage rate can be calculated based on the coverage of the target area grid by the current flight path, and the grids with insufficient coverage rate are identified as uncovered areas. New waypoints can be generated according to the positions of the uncovered areas, and the supplementary waypoints are added to the initial flight path.

[0049] It is understandable that to ensure the smoothness of the flight path and the stability of execution, the generated waypoint sequence can be optimized in terms of path. For example, the shortest path algorithm or Bezier curve smoothing can be used to optimize and supplement the path form of the flight path. After the adjustment is completed, the new full-coverage inspection flight path is stored in the form of a file and loaded into the UAV flight control system for execution.

[0050] By dynamically adjusting the UAV flight path, it is possible to adapt to the complex terrain characteristics of the dam slope. The pre-planned initial inspection flight path, combined with the real-time obtained UAV flight attitude, enables the UAV to dynamically adjust the flight path according to environmental changes, thereby optimizing the coverage range of the flight path and reducing inspection blind spots caused by complex terrain or environmental interference.

[0051] In step S230, based on the current inspection image collected when the UAV patrols along the full-coverage inspection flight path, an area of interest where cracks exist on the dam slope area is determined.

[0052] In an exemplary embodiment of the present disclosure, the current inspection image collected in real time by a high-resolution camera can be input into a crack recognition algorithm for processing. The crack recognition algorithm can complete the detection of the crack area through three steps: feature extraction, classification, and segmentation. First, a convolutional neural network (CNN) can be used to extract crack features in the image, including color differences, texture changes, and geometric characteristics; second, the image area can be divided into a crack area and a non-crack area through a classification model; finally, a segmentation algorithm is used to generate a set of pixel coordinates of the crack boundary points.

[0053] The area of interest can be further divided according to the distribution range of the crack boundary points by using a clustering algorithm or a region growing algorithm; the clustering algorithm generates the boundaries of multiple areas of interest by calculating the density and distance threshold of the crack boundary points; while the region growing algorithm completes the spatial calibration of the area of interest by setting seed points to expand to adjacent pixels; finally, the spatial position of the area of interest is associated with the inspection image data for subsequent calculation of the crack extension direction and flight path adjustment. Optionally, traditional image processing methods, such as the combination of edge detection and threshold segmentation, can also be used to extract the area of interest; the area of interest can also be determined by screening a set of segmented sub-regions. This embodiment does not make special limitations on the method of confirming the area of interest.

[0054] In step S240, the crack extension direction is determined according to the area of interest, and the full-coverage inspection flight path is locally updated according to the crack extension direction to obtain a key crack inspection route.

[0055] In an exemplary embodiment of the present disclosure, the determination of the crack extension direction can be based on the distribution of boundary points in the region of interest. A variety of algorithms can be used to perform geometric analysis on the boundary points. For example, the crack main axis can be generated by fitting the crack boundary points based on the least squares method, and the direction vector of the main axis is the crack extension direction. In addition, by calculating the local gradient change of the crack boundary points, a refined description of the crack extension trend can be obtained. To ensure accuracy, the crack extension direction can be corrected by combining time-series images of the dynamic changes in the crack morphology.

[0056] The process of locally updating the full-coverage inspection flight path can include adding waypoints in the crack extension direction and adjusting the waypoint spacing to improve the monitoring accuracy of the crack area. The added waypoints are generated by vector operations in the extension direction, and the optimal waypoint distribution is calculated in combination with the flight performance parameters of the UAV. To ensure the continuity of the path, path optimization can be performed on the added waypoints so that the generated key inspection route for cracks meets the requirements of inspection efficiency and flight path feasibility. The key inspection route for cracks after local update is loaded into the UAV control system to replace the original full-coverage inspection flight path.

[0057] Combined with the processing of inspection images, cracks in the target area can be accurately identified, and the region of interest can be marked during the inspection process. By further optimizing the flight path using the crack extension direction of the region of interest, a key inspection route for cracks can be generated according to the actual distribution characteristics of the cracks, thereby improving the ability to identify the crack distribution and solving the problem of inaccurate identification of crack morphology and extension direction in traditional crack detection technologies.

[0058] It should be noted that although the descriptions of "initial inspection flight path", "full-coverage inspection flight path", and "key inspection route for cracks" are used in this embodiment, those skilled in the art can easily understand that these three descriptions actually represent the same UAV inspection flight path. That is, "initial inspection flight path", "full-coverage inspection flight path", and "key inspection route for cracks" are only used to distinguish the content concerned during the dynamic planning process of the same UAV inspection flight path. For example, the "full-coverage inspection flight path" is to ensure that the deviation of the actual flight attitude of the UAV can cover all the dam slope areas corresponding to the "initial inspection flight path", and the "key inspection route for cracks" is locally updated on the basis of the "full-coverage inspection flight path" to facilitate the acquisition of crack images with better image quality, rather than three completely different flight paths.

[0059] In step S250, the UAV is controlled to complete the inspection of the dam slope area through the key inspection route for cracks, and a global crack inspection image is obtained to determine the slope crack parameters through the global crack inspection image.

[0060] In an exemplary embodiment of the present disclosure, when the drone completes the inspection along the key inspection route, the image data collected by the high-resolution camera can be stored in real time in the on-board storage module and associated with the waypoint coordinates. The global crack inspection image can include the distribution, shape, and extension information of the cracks in the dam slope area. The determination of the crack parameters can be based on the global inspection image, including the calculation of indicators such as width, length, and depth. Using the segmented crack boundary points and combining the relationship between the spatial position and the image resolution, the crack width and length can be obtained through geometric calculations, and the depth can be obtained by reconstructing the depth information of the three-dimensional information of the crack from the images taken at multiple angles.

[0061] The generation process of the global crack inspection image supports the comprehensive evaluation of slope cracks and provides basic data for the safety analysis and repair measures of the dam slope. At the same time, the image data can be transmitted to the ground station through the wireless communication module for real-time monitoring, and can also be used for subsequent crack propagation prediction and rolling stone risk analysis, providing more comprehensive data support for slope safety management.

[0062] By dynamically adjusting the key crack inspection route, it is possible to more efficiently conduct refined inspections on the crack areas in complex slopes, enabling the drone to effectively cover key areas within a limited flight time, avoiding the phenomena of redundant inspection data and omission of important areas in traditional methods. Especially in the case where there is an interactive effect between cracks and rolling stones, the dynamic inspection route can increase key coverage in areas prone to rolling stones, enabling more comprehensive collection of the correlation data between the distribution of rolling stones and cracks.

[0063] The content in steps S210 to S250 will be described in detail below.

[0064] In an exemplary embodiment of the present disclosure, the determination of the true flight attitude of the drone in step S220 can be achieved through the steps in Figure 3 As shown in Figure 3 , it can specifically include: Step S310, obtaining the inspection image frames collected by the drone during the inspection, and the inertial sensor data when collecting the inspection image frames; Step S320, estimating the initial flight attitude of the drone through the inertial sensor data; Step S330, determining the key image feature points in the inspection image frames, and determining the reprojection error through the key image feature points; Step S340, combining a preset sliding time window, and optimizing the initial flight attitude by minimizing the reprojection error to obtain the true flight attitude of the drone.

[0065] Among them, the inspection image frames of the drone are captured by a high-resolution camera. Through the sensor synchronization mechanism, the acceleration and angular velocity data provided by the inertial measurement unit (IMU) are recorded simultaneously during each image acquisition. The timestamps of the inspection image frames are precisely aligned with the timestamps of the inertial sensor data to ensure data synchronization. The acquired image frames are mainly used for subsequent feature point extraction and flight attitude correction, while the inertial data is used for the preliminary estimation of the flight attitude.

[0066] The initial flight attitude of the drone can be estimated from the inertial sensor data. The inertial sensor data calculates the initial position and attitude of the drone through integration. For example, the initial flight attitude of the drone can be estimated through the following relational expressions: ; ; Among them, can represent the spatial position coordinates of the drone at the current moment ; can represent the spatial position coordinates of the drone at the previous moment can represent the flight speed of the drone at the current moment ; can represent the flight speed of the drone at the previous moment can represent the acceleration of the drone at the current moment ; can represent the time interval between the current moment [[ID=3G]] and the previous moment.

[0067] The key feature points of the inspection image frames can be obtained by feature matching algorithms. For example, the key feature points of the inspection image frames can be obtained by the Oriented FAST and Rotated BRIEF (ORB) algorithm. The ORB algorithm can detect corner points through FAST and generate feature points in combination with the BRIEF descriptor. Of course, the Histogram of Oriented Gradient (HOG) algorithm can also be used to obtain the key feature points of the inspection image frames. In this embodiment, no special limitation is imposed on the type of algorithm for extracting the key feature points of the inspection image frames. Specifically, the matching results of the key feature points in consecutive image frames can be used to calculate the reprojection error. The calculation process of the reprojection error can be expressed by the following relational expressions: ; Among them, can represent the reprojection error, can represent the pixel coordinates of the observation point, can represent the camera intrinsic matrix, and can respectively represent a rotation matrix and a translation vector, and can represent the three-dimensional spatial coordinates of the key image feature points.

[0068] It can be combined with a preset sliding time window to optimize the initial flight attitude by minimizing the reprojection error, and obtain the true flight attitude of the drone. Multiple frames of image data and inertial data can be processed simultaneously within the sliding time window, and a non-linear optimization model can be constructed to determine the true flight attitude of the drone. For example, visual error and inertial error can be constructed based on the reprojection error, and then a non-linear optimization model can be constructed through the visual error and inertial error. The non-linear optimization model can be expressed by the following relational expression: ; wherein, can represent the visual error of the th iterative optimization, can represent the inertial error of the th iterative optimization. Furthermore, the optimization problem can be iteratively solved by the Gauss-Newton method or the Levenberg-Marquardt algorithm to obtain an accurate true flight attitude.

[0069] By acquiring the inspection image frames and inertial sensor data of the drone and performing fusion processing on the visual data and inertial data, the problem of reduced attitude estimation accuracy caused by the cumulative drift of inertial data can be effectively solved; the inertial data provides high-frequency attitude change information, and the key feature points in the inspection image frames correct the attitude solution result through the visual-inertial algorithm, thereby ensuring the accuracy of the solution result; optimizing the flight attitude solution by minimizing the reprojection error through the sliding time window can reduce the impact of instantaneous noise on attitude estimation, and at the same time further improve the stability of pose solution under the joint processing of multiple frames of data, effectively avoiding the problem of insufficient accuracy relying on traditional single inertial sensors, and being more robust and consistent compared to single-frame image correction, thus laying a foundation for subsequent dynamic adjustment of the flight path.

[0070] In an exemplary embodiment of the present disclosure, step S220 of dynamically adjusting the initial inspection flight path according to the true flight attitude to obtain a full-coverage inspection flight path can be implemented through the steps in Figure 4 , as shown in reference Figure 4 , and specifically may include: Step S410, determining an ideal inspection coverage area through the part of the initial inspection flight path that the drone has already inspected; Step S420, determining the true inspection coverage area of the drone according to the true flight attitude; Step S430: Determine the inspection coverage rate and the uncovered area based on the ideal inspection coverage area and the actual inspection coverage area; Step S440: Determine supplementary inspection points through the inspection coverage rate and the uncovered area, and dynamically update the initial inspection route based on the supplementary inspection points to obtain a full-coverage inspection route.

[0071] Among them, the ideal inspection coverage area refers to the target area that the unmanned aerial vehicle (UAV) can theoretically cover according to the preset route plan, which is calculated by combining the waypoint positions, the camera field of view angle, and the flight altitude. For example, assuming that the field of view coverage area of the on-board camera of the UAV is rectangular, its width and height can be calculated by the following relational expressions: ; ; Among them, can represent the width of the field of view coverage area, can represent the height of the field of view coverage area, can represent the flight altitude of the UAV, can represent the horizontal field of view angle of the on-board camera, can represent the vertical field of view angle of the on-board camera. The image coverage area actually collected by the UAV can be calculated by combining the actual flight attitude data and the camera internal parameter matrix. The actual coverage area may deviate due to flight offset, angle error, or environmental factors. Specifically, it can be determined by calculating the actual position of the camera shooting point and its coverage range. This embodiment does not make special limitations on this.

[0072] The inspection coverage rate and the uncovered area can be determined based on the ideal inspection coverage area and the actual inspection coverage area. The inspection coverage rate is calculated by the ratio of the overlapping area to the ideal coverage area. For example, the inspection coverage rate can be calculated by the following relational expression: ; Among them, can represent the overlapping area, can represent the ideal coverage area. The uncovered area can be calculated by the difference set between the overlapping area and the ideal coverage area. The position of the uncovered area can be used to generate supplementary waypoints.

[0073] Supplementary inspection points can be determined through the inspection coverage rate and the uncovered area, and the initial inspection route can be dynamically updated based on the supplementary inspection points to obtain a full-coverage inspection route. The generation of supplementary inspection points can be based on the geometric center of the uncovered area, and the distribution between points can be optimized in combination with the smoothness requirement of the route. After the route is updated, the supplementary waypoints can be integrated into the initial inspection route, and then a full-coverage inspection route can be generated.

[0074] By comparing the ideal coverage area and the actual coverage area of the initial inspection route, and calculating the inspection coverage rate and the uncovered area, the precise identification of the blind area of the route coverage can be achieved; based on the method of dynamically generating supplementary inspection points for the uncovered area, the full coverage of the target area can be ensured, and the repeated coverage and redundancy of the inspection path can be effectively reduced through the optimization and adjustment of the route; compared with the traditional method of presetting fixed routes, the adaptability of the inspection route can be significantly improved, and under complex terrain or external environmental interference, the integrity and rationality of the inspection path can still be dynamically guaranteed, effectively solving the inevitable problem of the blind area of route coverage in the related technology, and enabling the unmanned aerial vehicle to more efficiently complete the inspection task in complex areas.

[0075] In an exemplary embodiment of the present disclosure, the following steps can be used to determine the region of interest with cracks on the dam slope area based on the current inspection image collected when the unmanned aerial vehicle inspects along the full-coverage inspection route, which can specifically include: The current inspection image can be subjected to feature extraction processing to extract the crack boundary feature points in the current inspection image; according to the coordinates of the crack boundary feature points in the current inspection image, the full-coverage inspection route, and the actual flight attitude of the unmanned aerial vehicle when collecting the current inspection image, the coordinate position of the crack feature points in the dam slope area can be determined; by combining the coordinate position and each segmented sub-region in the dam slope area, the set of segmented sub-regions with cracks can be determined, and the region of interest can be obtained through the set of segmented sub-regions.

[0076] Among them, the current inspection image can be analyzed through an image processing algorithm. For example, the Canny edge detection algorithm can be used to extract the crack boundary in the current inspection image. Of course, a deep learning network can also be used to extract the crack boundary in the current inspection image. In this embodiment, there is no special limitation on the extraction method of the crack boundary of the current inspection image. The features of the crack boundary feature points can be screened by calculating the gradient intensity and direction of the crack boundary, and the points that meet the key features of the cracks are marked as crack boundary feature points.

[0077] Optionally, a feature extraction method based on texture and shape can be used. For example, the HOG feature can be used to classify the key feature points of the crack boundary to exclude the noise points in the non-crack area.

[0078] According to the coordinates of the crack boundary feature points in the current inspection image, the full-coverage inspection route, and the actual flight attitude of the unmanned aerial vehicle when collecting the current inspection image, the coordinate position of the crack feature points in the dam slope area can be determined. For example, the mapping from the pixel coordinates in the image to the actual three-dimensional space coordinates can be calculated through the actual flight attitude information to determine the coordinate position of the crack feature points in the dam slope area, which can be specifically determined by the following relational expression: ; Among them, it can represent the actual spatial coordinate position of the crack feature point in the dam slope area, it can represent the rotation matrix of the camera, it can represent the image coordinates of the crack boundary feature point in the current inspection image, it can represent the camera internal parameter matrix, it can represent the translation vector of the camera.

[0079] By combining the coordinate position and each segmented sub-region in the dam slope area, the set of segmented sub-regions with cracks can be determined, and the region of interest can be obtained through the set of segmented sub-regions. Specifically, the segmented sub-regions of the dam slope area can be defined by pre-divided grids, and the coordinate position of the crack feature point can be used to mark whether each sub-region contains cracks. Based on the distribution of the segmented sub-region numbers, the boundary of the region of interest can be generated, and this boundary can be further optimized through a clustering algorithm, such as the Density-Based Spatial Clustering of Applications with Noise (DBSCAN), to ensure that the region of interest accurately covers the crack distribution area.

[0080] By extracting the crack boundary feature points in the current inspection image and performing precise mapping of the three-dimensional coordinates in combination with the actual flight attitude of the UAV, the spatial position positioning of the crack feature points can be realized; the method of using segmented sub-regions is used to calibrate the region of interest where the cracks are located, so that the results of crack detection can reflect the actual situation of crack distribution with higher spatial resolution; compared with the existing technology, this technical means integrates inspection images, flight routes, and flight attitude information, avoiding the problem of insufficient positioning accuracy caused by the limitation of a single data source in traditional methods, and at the same time, the demarcation of the crack area is more refined, providing accurate input data for the further analysis of the crack extension direction.

[0081] In an exemplary embodiment of the present disclosure, the crack extension direction can be determined according to the region of interest, and the full-coverage inspection flight route can be locally updated according to the crack extension direction to obtain the key inspection route for cracks. Specifically, it can include: The region numbers of each segmented sub-region in the region of interest can be obtained, and the crack extension direction can be determined according to the relative position relationship between the region numbers; multiple crack collection inspection points can be determined according to the crack extension direction, and the full-coverage inspection flight route can be locally updated based on the crack collection inspection points to obtain the key inspection route for cracks.

[0082] Among them, the region number of the divided sub-region can be defined by its grid coordinates. For example, if the region number of the divided sub-region is (i, j), i and j can respectively represent the indices of the divided sub-region in the horizontal and vertical directions. The determination of the crack extension direction can be achieved by analyzing the continuity of the region numbers of the sub-regions containing crack feature points. Specifically, the crack extension direction can be determined through the following relational expression: ; Among them, can represent the direction vector of the crack extension direction, can represent the central point coordinates of the th divided sub-region, can represent the central point coordinates of the +1 th divided sub-region, can represent the total number of divided sub-regions containing crack feature points.

[0083] Multiple crack collection and inspection points can be determined according to the crack extension direction, and the full-coverage inspection route can be locally updated based on the crack collection and inspection points to obtain the key crack inspection route. The generation of the crack collection and inspection points can be based on the step size calculation of the extension direction. The step size can be determined by the flight speed of the UAV and the camera field of view coverage. The distribution of the inspection points can be further adjusted through a path optimization algorithm (such as the dynamic programming algorithm) to ensure the shortest path and cover all extension directions of the crack area.

[0084] By analyzing the number relationship of the divided sub-regions in the region of interest and calculating the crack extension direction according to the relative positions of the numbers, the dynamic trend of the crack can be captured in the spatial distribution of the crack area; combining the crack extension direction to generate crack collection and inspection points and update the inspection route enables the UAV to dynamically adjust the inspection path to adapt to the actual distribution changes of the crack; effectively solving the problem of fuzzy identification of the crack extension direction in the existing technology, being able to achieve refined coverage of key areas, and reducing the ineffective flight range through the optimized distribution of the inspection points, thereby significantly improving the efficiency and coverage accuracy of crack inspection.

[0085] Optionally, multiple crack collection and inspection points can be determined according to the crack extension direction through the steps in Figure 5 , which specifically may include: Step S510, determining multiple basic crack collection points according to the crack extension direction and the crack collection field of view angle of the UAV at the preset crack collection height; Step S520, combining the crack distribution characteristic function and the crack boundary feature points to determine the local gradient change data of the crack in the region of interest; Step S530: Estimate potential extended crack points in the crack extension direction based on the local gradient change data, and determine the crack extension area according to the potential extended crack points; Step S540: Determine multiple extended crack collection points based on the crack extension area and the crack collection field of view angle; Step S550: Use the basic crack collection points and the extended crack collection points as the crack collection inspection points.

[0086] Among them, the crack collection field of view angle can be determined by the field of view angle of the camera and the flight altitude of the UAV. The basic crack collection points are distributed at fixed intervals in the crack extension direction, and the interval can be determined by the following relational expression: ; Among them, can represent the interval between adjacent basic crack collection points, can represent the preset crack collection altitude of the UAV, can represent the crack collection field of view angle of the UAV.

[0087] The local gradient change data of the cracks in the region of interest can be determined by combining the crack distribution characteristic function and the crack boundary feature points. The distribution characteristic function can be generated by statistically analyzing the gradient direction and density change of the crack boundary points, and is used to describe the complexity of the crack region. The gradient calculation process of the crack boundary feature points can be expressed by the following relational expression: ; ; Among them, can represent the gradient vector of the crack boundary feature point, can represent the pixel intensity function, can represent the intensity change rate of the image in the region of interest in the horizontal direction, can represent the intensity change rate of the image in the region of interest in the vertical direction.

[0088] The potential extended crack points in the crack extension direction can be estimated through the local gradient change data, and the crack extension area can be determined according to the potential extended crack points. The extension area can be generated by expanding the coordinate set of the known crack boundary points, and the expansion rule can be estimated based on the direction and amplitude of the gradient change to ensure that the extension area reasonably covers the potential distribution of the cracks.

[0089] Multiple extended crack acquisition points can be determined based on the crack extension area and the crack acquisition viewing angle. The extended crack acquisition points can be generated by uniformly sampling the crack extension area, and the sampling density can be optimized in combination with the acquisition viewing angle to make the distribution of the acquisition points fully cover the area while minimizing the quantity. Finally, the basic crack acquisition points and the extended crack acquisition points can be integrated into crack acquisition inspection points as the update basis for the key inspection route.

[0090] By combining the crack extension direction, the crack distribution characteristic function, and the local gradient change data, it is possible to accurately predict the range of the potential extension area of the crack, and through the joint generation of the basic crack acquisition points and the extended crack acquisition points, achieve full coverage of complex crack morphologies; compared with the traditional inspection method that only collects known crack areas, it can actively explore the extension range of the crack, thereby realizing the early monitoring and coverage of potential crack areas; at the same time, by optimizing the distribution of the crack acquisition points, redundant data collection and unnecessary flight paths are reduced, improving the efficiency and data quality of the inspection task.

[0091] In an exemplary embodiment of the present disclosure, the global crack inspection image may include a crack millimeter-level image; the following steps can be used to control the drone to complete the inspection of the dam slope area through the key crack inspection route to obtain the global crack inspection image, which may specifically include: The preset crack acquisition data corresponding to the drone can be obtained. The preset crack acquisition data may include the crack acquisition height, the crack acquisition roll angle, and the crack acquisition camera pitch angle; in response to the drone arriving at the crack acquisition inspection point on the key crack inspection route, the crack acquisition flight attitude of the drone can be controlled through the preset crack acquisition data so that the camera view of the drone is perpendicular to the area of interest, and a crack millimeter-level image can be acquired.

[0092] Among them, the preset acquisition data can be determined according to the complexity of the crack area and the performance parameters of the drone during the mission planning stage. The crack acquisition height determines the coverage range of a single acquisition, and the roll angle and pitch angle can be used to adjust the camera view so that it is aligned with the crack area to be acquired.

[0093] When it is detected that the drone arrives at the crack acquisition inspection point on the key crack inspection route, the crack acquisition flight attitude of the drone can be controlled through the preset crack acquisition data. Specifically, the adjustment of the flight attitude of the drone can be executed by the flight control system, which may specifically include thrust adjustment to maintain the acquisition height, and yaw angle and pitch angle adjustment to ensure that the camera view can be perpendicular to the crack area. Finally, the acquired crack millimeter-level image can be associated with the inspection point position information and stored in the storage module of the drone for subsequent analysis.

[0094] Through precise settings of altitude, roll angle, and camera pitch angle in data collection via preset cracks, it is possible to ensure that the drone maintains the optimal flight attitude at the crack collection inspection points, making the camera's perspective perpendicular to the crack area. This can significantly improve the resolution and accuracy of crack image collection, especially in areas with complex crack details, enabling efficient collection of millimeter-scale crack images. Compared with the single fixed shooting angle in the prior art, by dynamically adjusting the flight attitude, it is ensured that the best shooting effect can be obtained in the collection of different crack areas, providing a reliable data basis for subsequent refined analysis and evaluation of cracks.

[0095] It should be noted that although the steps of the methods in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that these steps must be executed in that specific order, or that all the steps shown must be executed to achieve the desired result. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc.

[0096] In addition, in the present exemplary embodiment, a slope crack inspection device based on dynamic planning of the drone flight path is also provided. Referring to Figure 6 As shown, the slope crack inspection device 600 based on dynamic planning of the drone flight path includes: an initial inspection module 610, a full-coverage inspection route planning module 620, a crack detection module 630, a crack inspection route planning module 640, and a crack image acquisition module 650. Among them: The initial inspection module 610 is configured to obtain a pre-planned initial inspection flight path and control the drone to perform inspections along the initial inspection flight path; The full-coverage inspection route planning module 620 is configured to determine the actual flight attitude of the drone and dynamically adjust the initial inspection flight path according to the actual flight attitude to obtain a full-coverage inspection flight path; The crack detection module 630 is configured to determine the region of interest where cracks exist on the dam slope area based on the current inspection images collected when the drone performs inspections along the full-coverage inspection flight path; The crack inspection route planning module 640 is configured to determine the crack extension direction according to the region of interest and locally update the full-coverage inspection flight path according to the crack extension direction to obtain a crack key inspection flight path; The crack image acquisition module 650 is configured to control the drone to complete the inspection of the dam slope area through the crack key inspection flight path to obtain global crack inspection images, so as to determine the slope crack parameters through the global crack inspection images.

[0097] In an exemplary embodiment of the present disclosure, based on the foregoing solution, the coverage inspection route planning module 620 is configured to: obtain the inspection image frames collected by the drone during inspection, and the inertial sensor data when the inspection image frames are collected; estimate the initial flight attitude of the drone through the inertial sensor data; determine the key image feature points in the inspection image frames, and determine the reprojection error through the key image feature points; combine a preset sliding time window, and optimize the initial flight attitude by minimizing the reprojection error to obtain the true flight attitude of the drone.

[0098] In an exemplary embodiment of the present disclosure, based on the foregoing solution, the coverage inspection route planning module 620 is configured to: determine an ideal inspection coverage area through the part of the initial inspection route that the drone has inspected; determine the true inspection coverage area of the drone according to the true flight attitude; determine the inspection coverage rate and the uncovered area based on the ideal inspection coverage area and the true inspection coverage area; determine supplementary inspection points through the inspection coverage rate and the uncovered area, and dynamically update the initial inspection route according to the supplementary inspection points to obtain a full-coverage inspection route.

[0099] In an exemplary embodiment of the present disclosure, based on the foregoing solution, the crack detection module 630 is configured to: perform feature extraction processing on the current inspection image, and extract the crack boundary feature points in the current inspection image; determine the coordinate position of the crack feature points in the dam slope area according to the coordinates of the crack boundary feature points in the current inspection image, the full-coverage inspection route, and the true flight attitude of the drone when the current inspection image is collected; combine the coordinate position and each segmentation sub-region in the dam slope area to determine the set of segmentation sub-regions where cracks exist, and obtain the region of interest through the set of segmentation sub-regions.

[0100] In an exemplary embodiment of the present disclosure, based on the foregoing solution, the crack inspection route planning module 640 is configured to: obtain the region numbers of each segmentation sub-region in the region of interest, and determine the crack extension direction according to the relative position relationship between the region numbers; determine a plurality of crack collection inspection points according to the crack extension direction, and locally update the full-coverage inspection route according to the crack collection inspection points to obtain a crack key inspection route.

[0101] In an exemplary embodiment of the present disclosure, based on the foregoing solution, the crack inspection route planning module 640 is configured to: determine a plurality of basic crack collection points according to the crack extension direction and the crack collection field of view angle of the unmanned aerial vehicle at a preset crack collection height; combine the crack distribution characteristic function and the crack boundary feature points to determine the local gradient change data of the cracks in the region of interest; estimate potential extended crack points in the crack extension direction through the local gradient change data, and determine the crack extension region according to the potential extended crack points; determine a plurality of extended crack collection points based on the crack extension region and the crack collection field of view angle; and use the basic crack collection points and the extended crack collection points as the crack collection inspection points.

[0102] In an exemplary embodiment of the present disclosure, based on the foregoing solution, the global crack inspection image includes a millimeter-level crack image; the crack image acquisition module 650 is configured to: obtain the preset crack collection data corresponding to the unmanned aerial vehicle, where the preset crack collection data includes the crack collection height, the crack collection roll angle, and the crack collection camera pitch angle; in response to the unmanned aerial vehicle arriving at the crack collection inspection point on the key crack inspection route, control the crack collection flight attitude of the unmanned aerial vehicle through the preset crack collection data, so that the camera view angle of the unmanned aerial vehicle is perpendicular to the region of interest, and acquire the millimeter-level crack image.

[0103] The specific details of each module of the slope crack inspection device based on the dynamic planning of the unmanned aerial vehicle route have been described in detail in the corresponding slope crack inspection method based on the dynamic planning of the unmanned aerial vehicle route, and thus will not be elaborated here.

[0104] It should be noted that although several modules or units of the slope crack inspection device based on the dynamic planning of the unmanned aerial vehicle route are mentioned in the foregoing detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0105] In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the foregoing slope crack inspection method based on the dynamic planning of the unmanned aerial vehicle route is also provided.

[0106] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, a method, or a program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module", or "system" here.

[0107] Reference will be made hereinafter Figure 7 to describe the electronic device 700 according to such an embodiment of the present disclosure. Figure 7 The illustrated electronic device 700 is merely an example and should not impose any limitation on the functions and the scope of use of the embodiments of the present disclosure.

[0108] As Figure 7 shown, the electronic device 700 is presented in the form of a general-purpose computing device. The components of the electronic device 700 may include, but are not limited to: at least one of the above-mentioned processing units 710, at least one of the above-mentioned storage units 720, a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710), and a display unit 740.

[0109] Among them, the storage unit stores program code, and the program code can be executed by the processing unit 710, so that the processing unit 710 executes the steps according to various exemplary embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification. For example, the processing unit 710 may execute steps S210 as Figure 2 shown, obtain a pre-planned initial inspection route, and control the drone to perform inspections along the initial inspection route; step S220, determine the actual flight attitude of the drone, and dynamically adjust the initial inspection route according to the actual flight attitude to obtain a full-coverage inspection route; step S230, based on the current inspection images collected when the drone performs inspections along the full-coverage inspection route, determine the region of interest where cracks exist on the dam slope area; step S240, determine the crack extension direction according to the region of interest, and locally update the full-coverage inspection route according to the crack extension direction to obtain a crack key inspection route; step S250, control the drone to complete the inspection of the dam slope area through the crack key inspection route to obtain a global crack inspection image, so as to determine the slope crack parameters through the global crack inspection image.

[0110] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 721 and / or a cache storage unit 722, and may further include a read-only storage unit (ROM) 723.

[0111] The storage unit 720 may also include a program / utility 724 having a set (at least one) of program modules 725. Such program modules 725 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment.

[0112] The bus 730 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus structures.

[0113] The electronic device 700 may also communicate with one or more external devices 770 (such as a keyboard, a pointing device, a Bluetooth device, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or may communicate with any device that enables the electronic device 700 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication may be through an input / output (I / O) interface 750. Also, the electronic device 700 may communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 760. As shown, the network adapter 760 communicates with other modules of the electronic device 700 through the bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0114] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software or by a combination of software and necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which may be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0115] In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium, on which a program product capable of implementing the above-described method of the present specification is stored. In some possible embodiments, various aspects of the present disclosure may also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of the present specification.

[0116] Reference Figure 8 As shown, a program product 800 for implementing the above-described slope crack inspection method based on dynamic planning of UAV flight routes according to an embodiment of the present disclosure is described. It may be a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0117] The program product may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may, for example, but not be limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0118] The computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the readable program code. Such a propagated data signal may take various forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable signal medium may also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0119] The program code contained on the readable medium may be transmitted by any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination of the above.

[0120] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0121] In addition, the above-mentioned drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not for limiting purposes. It is easy to understand that the processes shown in the above-mentioned drawings do not indicate or limit the chronological order of these processes. Additionally, it is also easy to understand that these processes may be executed synchronously or asynchronously in, for example, multiple modules.

[0122] Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be implemented by software, or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0123] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. This application is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not disclosed herein. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the claims.

[0124] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A method for slope crack inspection based on dynamic planning of UAV flight routes, characterized in that, Including: Obtain a pre-planned initial inspection route, and control the drone to perform inspections along the initial inspection route; Determine the true flight attitude of the drone, and dynamically adjust the initial inspection route according to the true flight attitude to obtain a full-coverage inspection route; Based on the current inspection images collected when the drone inspects along the full-coverage inspection route, determine the regions of interest where cracks exist on the dam slope area; Determine the crack extension direction according to the regions of interest, and locally update the full-coverage inspection route according to the crack extension direction to obtain a key crack inspection route; Control the drone to complete the inspection of the dam slope area through the key crack inspection route to obtain a global crack inspection image, so as to determine the slope crack parameters through the global crack inspection image.

2. The slope crack inspection method based on dynamic planning of UAV flight routes according to claim 1, wherein, The determining the true flight attitude of the drone includes: Obtain the inspection image frames collected by the drone during inspections, and the inertial sensor data when collecting the inspection image frames; Estimate the initial flight attitude of the drone through the inertial sensor data; Determine the key image feature points in the inspection image frames, and determine the reprojection error through the key image feature points; Combined with a preset sliding time window, optimize the initial flight attitude by minimizing the reprojection error to obtain the true flight attitude of the drone.

3. The slope crack inspection method based on dynamic planning of UAV flight routes according to claim 1, characterized in that, The dynamically adjusting the initial inspection route according to the true flight attitude to obtain a full-coverage inspection route includes: Determine the ideal inspection coverage area through the part of the initial inspection route that the drone has inspected along; Determine the true inspection coverage area of the drone according to the true flight attitude; Based on the ideal inspection coverage area and the true inspection coverage area, determine the inspection coverage rate and the uncovered area; Determine supplementary inspection points through the inspection coverage rate and the uncovered area, and dynamically update the initial inspection route according to the supplementary inspection points to obtain a full-coverage inspection route.

4. The slope crack inspection method based on dynamic planning of UAV flight routes according to claim 1, wherein, The determining the regions of interest where cracks exist on the dam slope area based on the current inspection images collected when the drone inspects along the full-coverage inspection route includes: Perform feature extraction processing on the current inspection image to extract the crack boundary feature points in the current inspection image; According to the coordinates of the crack boundary feature points in the current inspection image, the full-coverage inspection route, and the true flight attitude of the drone when collecting the current inspection image, determine the coordinate position of the crack feature points in the dam slope area; Combined with the coordinate position and each segmented sub-region in the dam slope area, determine the set of segmented sub-regions where cracks exist, and obtain the regions of interest through the set of segmented sub-regions.

5. The slope crack inspection method based on dynamic planning of UAV flight routes according to claim 4, characterized in that Determine the crack extension direction according to the regions of interest, and locally update the full-coverage inspection route according to the crack extension direction to obtain a key crack inspection route, including: Obtain the region numbers of each segmented sub-region in the regions of interest, and determine the crack extension direction according to the relative position relationship between the region numbers; Determine a plurality of crack collection and inspection points according to the crack extension direction, and locally update the full-coverage inspection route according to the crack collection and inspection points to obtain a key crack inspection route.

6. The method for inspecting slope cracks based on dynamic planning of UAV flight routes according to claim 5, characterized in that The step of determining a plurality of crack collection and inspection points according to the crack extension direction further includes: Determine a plurality of basic crack collection points according to the crack extension direction and the crack collection field of view angle of the unmanned aerial vehicle (UAV) at a preset crack collection height; Combine the crack distribution characteristic function and the crack boundary feature points to determine the local gradient change data of the cracks in the region of interest; Estimate potential extended crack points in the crack extension direction through the local gradient change data, and determine the crack extension region according to the potential extended crack points; Determine a plurality of extended crack collection points based on the crack extension region and the crack collection field of view angle; Use the basic crack collection points and the extended crack collection points as the crack collection and inspection points.

7. The method for inspecting slope cracks based on dynamic planning of UAV flight routes according to claim 1, characterized in that, The global crack inspection image includes a millimeter-level crack image; Control the UAV to complete the inspection of the dam slope area through the key crack inspection route to obtain a global crack inspection image, including: Obtain the preset crack collection data corresponding to the UAV, where the preset crack collection data includes the crack collection height, the crack collection roll angle, and the crack collection camera pitch angle; In response to the UAV arriving at the crack collection and inspection points on the key crack inspection route, control the crack collection flight attitude of the UAV through the preset crack collection data, so that the camera view angle of the UAV is perpendicular to the region of interest, and collect the millimeter-level crack image.

8. An inspection device for slope cracks based on dynamic planning of UAV flight routes, characterized in that, It includes: An initial inspection module, configured to obtain a pre-planned initial inspection route and control the UAV to perform inspections along the initial inspection route; A full-coverage inspection route planning module, configured to determine the actual flight attitude of the UAV and dynamically adjust the initial inspection route according to the actual flight attitude to obtain a full-coverage inspection route; A crack detection module, configured to determine the region of interest with cracks on the dam slope area based on the current inspection image collected when the UAV performs inspections along the full-coverage inspection route; A crack inspection route planning module, configured to determine the crack extension direction according to the region of interest, and locally update the full-coverage inspection route according to the crack extension direction to obtain a key crack inspection route; A crack image collection module, configured to control the UAV to complete the inspection of the dam slope area through the key crack inspection route to obtain a global crack inspection image, so as to determine the slope crack parameters through the global crack inspection image.

9. An electronic device, characterized in that, It includes: A processor; And A memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, the slope crack inspection method based on dynamic planning of the UAV route according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and when the computer program is executed by a processor, the slope crack inspection method based on dynamic planning of the UAV route according to any one of claims 1 to 7 is implemented.

Images