Ground surface crack positioning and grading system based on unmanned aerial vehicle cruise and image recognition

By combining drone patrols with spatiotemporal fusion and deep learning, a crack localization and grading system has been developed, which solves the problems of accuracy and grading dependence in drone crack detection and achieves automation of crack detection and high efficiency of crack repair.

CN121259656APending Publication Date: 2026-01-02XIAN UNIV OF SCI & TECH
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202511422454.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-09-30
Publication Date
2026-01-02

AI Technical Summary

Technical Problem

Existing UAV-based crack detection methods lack deep learning and multi-level models, resulting in insufficient accuracy and precision in crack identification. Furthermore, traditional positioning techniques are greatly affected by environmental factors, and crack classification relies on manual judgment, which cannot be automated or made intelligent.

Method used

The system uses drones to collect image data, combines spatiotemporal fusion algorithms to accurately locate cracks, uses deep learning technology to classify cracks, and transmits the data to a ground platform in real time via wireless communication to support crack repair decisions.

Benefits of technology

It has achieved automation and high precision in crack detection, improved the accuracy of crack location and the scientific nature of crack classification, and ensured the timeliness and efficiency of crack repair.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121259656A_ABST
    Figure CN121259656A_ABST
Patent Text Reader

Abstract

The invention discloses a surface crack positioning and grading system based on unmanned aerial vehicle cruise and image recognition, and the system comprises an unmanned aerial vehicle flight control module which is used for controlling an unmanned aerial vehicle to carry out autonomous cruise and collecting original image data; the data preprocessing module is used for preprocessing the real-time data to generate standard image data; the crack automatic identification module is used for automatically identifying the crack and generating a preliminary detection result of the crack; the wireless communication module is used for transmitting the crack image data to a ground control platform; the ground control platform is used for receiving the real-time data and storing, processing and visually displaying the real-time data; the space-time fusion positioning module is used for calculating crack space coordinates through a space-time fusion positioning algorithm and correcting potential errors; the grading module is used for intelligently grading the cracks and generating a primary grading result of the cracks; and the repair decision support module is used for evaluating the priority of crack repair and making a repair decision. The invention constructs an earth surface crack positioning and grading system.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a surface crack location and classification system based on UAV cruise and image recognition. Background Technology

[0002] With the rapid development of society and industry, the detection and repair of surface cracks has become particularly important. Traditional methods for detecting surface cracks mostly rely on manual inspections or monitoring with fixed equipment. These methods usually require a lot of manpower and resources. Especially in complex or dangerous environments, manual inspections are difficult to cover a wide area and are inefficient. Furthermore, traditional crack detection methods rely on manual recording and analysis of image data, which is cumbersome and easily affected by environmental conditions, resulting in significant uncertainty in the priority judgment of crack detection and repair.

[0003] In some cases, even a single positioning technique cannot accurately pinpoint the location of cracks, especially in complex terrain and adverse weather conditions, where crack positioning accuracy drops significantly, increasing the difficulty of subsequent repair processes.

[0004] In recent years, UAV-based crack detection has emerged as a new technological direction due to the high efficiency, flexibility, and speed of UAVs, which can cover a wide area in a short time, avoiding the limitations of manual inspection. However, existing UAV-based crack detection methods still have some problems: existing crack identification technologies typically rely on a single image processing algorithm, lacking effective deep learning and multi-level model applications, resulting in insufficient accuracy and precision in crack identification; crack spatial coordinate calculation mainly relies on navigation and positioning, but UAV flight errors and environmental factors significantly affect positioning accuracy, leading to low precision in crack location; existing crack grading methods mostly rely on manual judgment, failing to automatically and intelligently grade cracks based on their characteristics and environment, resulting in unscientific and unreasonable repair decisions.

[0005] Therefore, how to provide a surface crack location and classification system based on UAV patrol and image recognition is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a surface crack location and classification system based on UAV patrol and image recognition. This invention collects raw image data through UAV patrol, accurately locates crack positions using a spatiotemporal fusion algorithm, classifies cracks using deep learning technology, and transmits the data to a ground platform in real time via wireless communication, ultimately providing scientific decision support for crack repair.

[0007] A surface crack location and classification system based on UAV patrol and image recognition according to an embodiment of the present invention includes:

[0008] The drone flight control module is used to control the drone to perform autonomous navigation and collect raw image data;

[0009] The data preprocessing module is used to preprocess real-time data from the UAV to generate standard image data;

[0010] The automatic crack identification module is used to automatically identify cracks in standard image data and generate preliminary crack detection results.

[0011] The wireless communication module is used to transmit crack image data, grading reports, and repair decisions to the ground control platform in real time.

[0012] The ground control platform is used to receive real-time data from UAVs and to store, process, and visualize the data.

[0013] The spatiotemporal fusion positioning module is used to accurately calculate the spatial coordinates of the crack on the ground surface through the spatiotemporal fusion positioning algorithm, and to correct potential errors in the crack location;

[0014] The grading module is used to intelligently grade cracks based on their spatial coordinates and feature information using a deep learning grading model, generating a preliminary grading result for the cracks.

[0015] The Repair Decision Support module is used to assess the priority of crack repair and make repair decisions, guiding the repair team to carry out repair operations.

[0016] Optionally, modules can be integrated using the following methods:

[0017] The drone autonomously cruises along a preset flight path to collect raw image data covering the ground surface area.

[0018] The original image data is transmitted to the ground control platform and preprocessed to obtain standard image data. The processing includes noise reduction, contrast enhancement, and automatic white balance adjustment to ensure image clarity.

[0019] Automatic crack identification is performed on standard image data, and crack contours and feature information in the images are extracted to generate preliminary crack detection results.

[0020] Based on the preliminary crack detection results, combined with the real-time navigation and positioning data of the UAV, the spatial coordinates of the crack on the ground surface are accurately calculated through spatiotemporal fusion positioning to correct potential errors in the crack location.

[0021] Based on the spatial coordinates and characteristic information of the cracks, a multi-level deep learning grading model with topology optimization is used to intelligently grade the cracks. The cracks are automatically classified as mild, moderate or severe based on their width, depth and extent of expansion, and the initial crack grading results are obtained.

[0022] The initial crack classification results are combined with the crack spatial coordinates to generate a detailed crack classification report, which is then transmitted to the ground control platform in real time via wireless communication.

[0023] As the drone continues to fly, it continuously collects new image data based on the crack classification report and updates crack identification and location information in real time, feeding the updated crack information back to the ground platform.

[0024] Based on the updated crack information, ground operators assess the priority of crack repair and make repair decisions, guiding the repair team to carry out repair operations.

[0025] Optionally, the acquisition of the raw image data covering the land surface area specifically includes:

[0026] Set the flight path parameters for the UAV and transmit the flight path information to the UAV's flight control system through the ground control platform;

[0027] The drone automatically starts up based on the flight path parameters and begins to cruise along the set path, flying within the designated area according to the planned path;

[0028] During the cruise, the drone uses its onboard high-resolution camera to collect image data of the ground area in real time, and uses built-in sensors to simultaneously record auxiliary information including flight position coordinates, flight altitude, camera parameters and shooting angle, to obtain raw image data covering the ground area.

[0029] Optionally, the process of generating the preliminary detection results of the cracks specifically includes:

[0030] The raw image data is transmitted from the UAV to the ground control platform in real time via a wireless communication module. The ground control platform performs noise reduction processing on the image, uses a Gaussian filtering algorithm to smooth the image, removes noise from the image, and obtains clear and interference-free image data.

[0031] Automatic white balance processing is performed on clear and interference-free image data, and grayscale equalization is used to enhance the contrast and brightness of the image to obtain a white balance processed image, ensuring that image details are more obvious and colors are balanced;

[0032] Edge detection is performed on the white-balanced image. The Canny operator is used to identify and extract crack edge information in the image. The crack contour is enhanced by image dilation to ensure that the crack features are clearer and more complete, thus obtaining standard image data.

[0033] Optionally, the generation of the preliminary detection results of the cracks specifically includes:

[0034] Standard image data is input into the crack automatic recognition module, which automatically identifies and extracts low-level features in the image, including edges, corners, and textures, ensuring that the model can effectively extract subtle crack information in the image.

[0035] Based on the extracted low-level features, multi-layer convolution and pooling operations are used to further extract the geometric features of the cracks, identify high-level features of the cracks including shape, width, length, depth, propagation direction and texture, generate depth information and feature maps of the cracks, and provide a preliminary description and feature distribution of the cracks.

[0036] Based on the depth information and feature map of the crack, region extraction is used to accurately locate the crack region. By detecting the crack region in the image through a sliding window, the precise coordinate position of each crack region is obtained. The contour boundary of the crack is corrected by combining context information, and clear and accurate preliminary crack detection results are generated.

[0037] Optionally, the calculation of the crack spatial coordinates specifically includes:

[0038] The preliminary crack detection results are combined with the real-time navigation and positioning data of the UAV to obtain the latitude and longitude information of the current position of the UAV. The time information of crack detection is recorded by timestamp to obtain the flight trajectory data of the UAV. By combining the real-time flight trajectory of the UAV with the crack detection results, it is ensured that the crack location is highly consistent with the flight path.

[0039] Based on the flight trajectory data of the UAV, multiple location points collected by the UAV during flight are fused with the location data of the preliminary crack detection results. The spatial coordinates of the crack on the ground surface are calculated using a spatiotemporal fusion positioning method. This spatiotemporal fusion positioning method uses spatiotemporal weighted interpolation to calculate the preliminary correction value of the crack position and applies an adaptive correction particle swarm optimization method to dynamically adjust the spatiotemporal weighting coefficients. In the adaptive correction particle swarm optimization method, each particle represents a correction scheme for the crack position in the particle swarm. The particle adjusts its position according to the velocity and heading angle of different correction schemes. The particle position is updated in each time interval to obtain different spatiotemporal reference points. The dynamic adjustment of the spatiotemporal weighting coefficients is achieved by determining the spatiotemporal reference points, including the preliminary crack position and the historical position data of the UAV, and calculating the spatiotemporal distance difference between each reference point based on the preliminary crack detection position and the flight trajectory of the UAV. A two-dimensional Gaussian function is used to calculate the weighting coefficient of each reference point based on the spatiotemporal distance difference. By correcting the crack position through spatiotemporal weighted interpolation, multiple reference points in time and space can be considered to ensure the accuracy of the crack position.

[0040] Combining the initial detection results of the crack, the initial correction value of the crack location, and the flight dynamic data of the UAV, the spatial coordinates of the crack are further corrected by incremental correction using data on flight speed and heading angle. This corrects potential errors in the UAV's position and eliminates errors caused by environmental factors and changes in flight trajectory.

[0041] Optionally, the process of obtaining the primary crack classification result specifically includes:

[0042] Based on the spatial coordinates and feature information of the cracks, the geometric feature data of the cracks are extracted, including the length, width, depth, extent of expansion, distance between cracks, and distribution density. The geometric feature data is then processed by graph structuring to construct a spatial topology graph of the cracks. The adjacency relationship of each crack feature in the graph is calculated and input into a deep learning hierarchical model. The graph structuring process can fully consider the spatial dependence between cracks and effectively extract feature data with high discriminative ability for crack classification, ensuring the representativeness and discriminativeness of the model input data.

[0043] In the first layer of the deep learning hierarchical model, a multi-scale convolutional neural network is used to perform preliminary processing of crack feature data, extracting local and global features of cracks from different scales. Smaller-scale convolutional layers focus on the detailed features of cracks, while larger-scale convolutional layers focus on the overall shape of cracks. Multi-level convolution and pooling operations can better capture the complex geometric structure and spatial information of cracks.

[0044] In the second layer of the deep learning grading model, topology optimization is used to perform deep processing on the extracted local and global features of the cracks. By calculating the relationship between the local and global characteristics of the cracks, the influence of each feature of the crack is evaluated, the weight of the crack features is dynamically adjusted, and the optimal grading path for each crack is calculated through a global path optimization strategy. The topology optimization adaptively adjusts the importance of crack features according to the local complexity and global distribution pattern of the cracks, thereby optimizing the structure of the crack grading model so that the contribution of different features can dynamically match the actual state and risk level of the cracks.

[0045] The optimal grading path for each crack is fed into the grading module of the deep learning grading model. Based on the crack's feature data, the Softmax activation function is used to calculate the probability distribution of the crack in different grading categories, including mild cracks, moderate cracks, and severe cracks. The hierarchical Softmax activation function can accurately assign cracks to the corresponding grading categories based on the crack's geometric features and spatial relationships, and obtain the initial crack grading results.

[0046] Optionally, the process of generating the detailed crack classification report specifically includes:

[0047] Based on the spatial coordinates of the crack and environmental information, the risk level of the crack is calculated using a weighted average. The risk level of the crack is obtained by weighting the primary risk level and the environmental weight coefficient. The environmental weight coefficient is adjusted based on the crack location and environmental conditions to ensure that the classification result reflects the true risk.

[0048] The initial crack classification results are combined with the crack spatial coordinates to generate a detailed crack classification report, including the crack spatial coordinates, crack type and corresponding risk level, ensuring that the classification result and location of each crack correspond one-to-one.

[0049] Detailed crack classification reports are transmitted to the ground control platform in real time via a wireless communication module. The real-time transmission process encapsulates the report data according to a time sequence and transmits the report through a standard communication protocol to ensure stable data transmission and synchronization to the ground platform.

[0050] Optionally, the real-time updating of the crack identification and location information specifically includes:

[0051] During the continued flight of the UAV, the crack location data in the crack classification report is monitored in real time. Based on the current flight path and location, the UAV is controlled to continue to collect new image data along the predetermined route. The spatial coordinates corresponding to the current image data are determined through the UAV's navigation system to obtain newly collected crack image data, ensuring that each image is associated with the current flight position.

[0052] The newly acquired crack image data is compared with the existing crack classification report. When the data difference in crack location and type in the comparison results does not exceed a specified threshold, the crack identification and location information is updated, and the crack type, risk level and location coordinates are adjusted in real time. The specified threshold is one percent of the total crack data.

[0053] The updated crack information is transmitted to the ground control platform in real time via a wireless communication module, ensuring that the ground platform receives the latest crack information in a timely manner.

[0054] Optionally, the assessment of the priority of crack repair specifically includes:

[0055] Based on the updated crack information, including crack spatial coordinates, type and classification level, ground operators assess the priority of crack repair, taking into account the crack's risk level, extent of propagation, location, environmental factors and potential impact on the structure.

[0056] Ground operators make repair decisions based on the repair priorities and detailed information about the cracks obtained from the assessment. The repair decisions include determining the specific repair methods, required repair materials, tools, and timing of the repairs. Based on the type of crack, the scope of impact, and the environmental conditions, and in conjunction with existing repair resources, operators select appropriate repair plans and operational steps to obtain repair decision information.

[0057] Ground operators transmit repair decision information to the repair team in real time via wireless communication modules. The repair team then carries out on-site repair operations based on the repair decision information to ensure that the cracks are repaired in a timely and effective manner.

[0058] The beneficial effects of this invention are:

[0059] 1. This invention utilizes drones for automatic patrol, which can quickly cover large areas of the ground, avoiding the inefficiency and limitations of traditional manual inspections and ensuring the comprehensiveness of crack detection. The high-resolution image acquisition and real-time transmission of the drones enable crack image data to be transmitted to the ground control platform in an instant, avoiding the tedious process of manual recording and analysis and reducing human error.

[0060] 2. This invention achieves automated and high-precision crack detection by combining data preprocessing and an automatic crack identification module. Through deep learning algorithms and image recognition technology, it can accurately identify the contour and feature information of cracks and generate preliminary crack detection results, significantly improving the accuracy and speed of crack detection. The application of the spatiotemporal fusion positioning module makes the spatial coordinate calculation of cracks more accurate, overcomes the error problem of traditional navigation and positioning, and further improves the accuracy of crack location.

[0061] 3. The intelligent grading module of the present invention can classify cracks according to their spatial coordinates and characteristic information, automatically generate primary crack grading results, and, in conjunction with the repair decision support module, help ground operators scientifically assess the priority of crack repair and make timely and effective repair decisions.

[0062] 4. This invention enables real-time data transmission and feedback through a wireless communication module, ensuring rapid response and implementation of crack repair decisions. The automation level of crack detection, location, classification, and repair decisions is greatly improved, significantly enhancing the efficiency and accuracy of crack repair. Attached Figure Description

[0063] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0064] Figure 1This is a flowchart of the surface crack location and classification system based on UAV patrol and image recognition proposed in this invention;

[0065] Figure 2 This is a schematic diagram of the algorithm structure of the surface crack location and classification system based on UAV cruise and image recognition proposed in this invention. Detailed Implementation

[0066] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0067] refer to Figure 1-2 A surface crack location and classification system based on UAV patrol and image recognition includes:

[0068] The drone flight control module is used to control the drone to perform autonomous navigation and collect raw image data;

[0069] The data preprocessing module is used to preprocess real-time data from the UAV to generate standard image data;

[0070] The automatic crack identification module is used to automatically identify cracks in standard image data and generate preliminary crack detection results.

[0071] The wireless communication module is used to transmit crack image data, grading reports, and repair decisions to the ground control platform in real time.

[0072] The ground control platform is used to receive real-time data from UAVs and to store, process, and visualize the data.

[0073] The spatiotemporal fusion positioning module is used to accurately calculate the spatial coordinates of the crack on the ground surface through the spatiotemporal fusion positioning algorithm, and to correct potential errors in the crack location;

[0074] The grading module is used to intelligently grade cracks based on their spatial coordinates and feature information using a deep learning grading model, generating a preliminary grading result for the cracks.

[0075] The Repair Decision Support module is used to assess the priority of crack repair and make repair decisions, guiding the repair team to carry out repair operations.

[0076] In this embodiment, the modules are interconnected using the following method:

[0077] The drone autonomously cruises along a preset flight path to collect raw image data covering the ground surface area.

[0078] The original image data is transmitted to the ground control platform and preprocessed to obtain standard image data. The processing includes noise reduction, contrast enhancement, and automatic white balance adjustment to ensure image clarity.

[0079] Automatic crack identification is performed on standard image data, and crack contours and feature information in the images are extracted to generate preliminary crack detection results.

[0080] Based on the preliminary crack detection results, combined with the real-time navigation and positioning data of the UAV, the spatial coordinates of the crack on the ground surface are accurately calculated through spatiotemporal fusion positioning to correct potential errors in the crack location.

[0081] Based on the spatial coordinates and characteristic information of the cracks, a multi-level deep learning grading model with topology optimization is used to intelligently grade the cracks. The cracks are automatically classified as mild, moderate or severe based on factors such as width, depth and extent of expansion, and the initial crack grading results are obtained.

[0082] The initial crack classification results are combined with the crack spatial coordinates to generate a detailed crack classification report, which is then transmitted to the ground control platform in real time via wireless communication.

[0083] As the drone continues to fly, it continuously collects new image data based on the crack classification report and updates crack identification and location information in real time, feeding the updated crack information back to the ground platform.

[0084] Based on the updated crack information, ground operators assess the priority of crack repair and make repair decisions, guiding the repair team to carry out repair operations.

[0085] This invention improves the efficiency and accuracy of crack detection through UAV flight control, data preprocessing, and automatic crack identification modules. The spatiotemporal fusion positioning module can accurately calculate crack locations and correct potential errors to ensure positioning accuracy. The deep learning grading model realizes intelligent crack grading, automatically generates primary grading results, and transmits the data to the ground platform in real time through the wireless communication module. The ground control platform receives and processes the data in real time, supporting the repair decision support module to evaluate and decide on crack repair priorities, thereby optimizing the repair process and significantly improving the accuracy, timeliness, and automation level of crack repair.

[0086] In this embodiment, the acquisition of the original image data covering the land surface area specifically includes:

[0087] Set the flight path parameters for the UAV and transmit the flight path information to the UAV's flight control system through the ground control platform;

[0088] The drone automatically starts up based on the flight path parameters and begins to cruise along the set path, flying within the designated area according to the planned path;

[0089] During the cruise, the drone uses its onboard high-resolution camera to collect image data of the ground area in real time, and uses built-in sensors to simultaneously record auxiliary information including flight position coordinates, flight altitude, camera parameters and shooting angle, to obtain raw image data covering the ground area.

[0090] This invention ensures clear, interference-free images by transmitting and preprocessing image data in real time, providing accurate data for automatic crack identification and feature extraction. Combined with real-time positioning information from UAVs, the system can accurately calculate crack spatial coordinates and correct potential errors, significantly improving crack location accuracy. Utilizing a multi-level deep learning grading model, the system can intelligently classify cracks as mild, moderate, or severe, generating detailed crack grading reports and feeding them back to the ground control platform in real time. By continuously acquiring new image data and updating crack identification and location information, the system ensures the accuracy and timeliness of repair decisions, improving the efficiency and reliability of crack repair.

[0091] In this embodiment, the process of generating the preliminary detection results of the crack specifically includes:

[0092] The raw image data is transmitted from the UAV to the ground control platform in real time via a wireless communication module. The ground control platform performs noise reduction processing on the image, uses a Gaussian filtering algorithm to smooth the image, removes noise from the image, and obtains clear and interference-free image data.

[0093] Automatic white balance processing is performed on clear and interference-free image data. Gray-scale equalization is used to enhance the contrast and brightness of the image, resulting in a white balance-processed image that ensures more obvious image details and color balance, facilitating crack feature extraction.

[0094] Edge detection is performed on the white-balanced image. The Canny operator is used to identify and extract crack edge information in the image. The crack contour is enhanced by image dilation to ensure that the crack features are clearer and more complete, thus obtaining standard image data.

[0095] This invention transmits raw image data to a ground control platform in real time via a wireless communication module, performs noise reduction and Gaussian filtering to ensure clear and interference-free images, and employs automatic white balance and grayscale equalization techniques to enhance image contrast and brightness, making crack details more apparent and facilitating subsequent feature extraction. Edge detection is performed using the Canny operator to extract crack edge information, and image dilation is used to enhance crack contours, ensuring clearer and more complete crack features. Finally, standard image data is generated, providing an accurate image basis for crack detection.

[0096] In this embodiment, the generation of the preliminary detection results of the crack specifically includes:

[0097] Standard image data is input into the crack automatic recognition module, which automatically identifies and extracts low-level features in the image, including edges, corners, and textures, ensuring that the model can effectively extract subtle crack information in the image.

[0098] Based on the extracted low-level features, multi-layer convolution and pooling operations are used to further extract the geometric features of the cracks, identify high-level features of the cracks including shape, width, length, depth, propagation direction and texture, generate depth information and feature maps of the cracks, and provide a preliminary description and feature distribution of the cracks.

[0099] Based on the depth information and feature map of the crack, region extraction is used to accurately locate the crack region. By detecting the crack region in the image through a sliding window, the precise coordinate position of each crack region is obtained. The contour boundary of the crack is corrected by combining context information, and clear and accurate preliminary crack detection results are generated.

[0100] This invention effectively extracts low-level features, such as edges, corners, and textures, from standard image data through an automatic crack identification module, ensuring accurate identification of subtle crack information. Utilizing multi-layer convolution and pooling operations, it further extracts the geometric features of the cracks, including shape, width, length, and depth, generating detailed crack depth information and feature maps to provide a precise crack description. Region extraction and sliding window techniques are used to accurately locate crack regions, and contextual information is combined to correct crack boundaries, ensuring clear and accurate crack detection results. This method significantly improves the accuracy and efficiency of crack detection, supporting subsequent grading and repair decisions.

[0101] In this embodiment, the calculation of the crack spatial coordinates specifically includes:

[0102] The preliminary crack detection results are combined with the real-time navigation and positioning data of the UAV to obtain the latitude and longitude information of the current position of the UAV. The time information of crack detection is recorded by timestamp to obtain the flight trajectory data of the UAV. By combining the real-time flight trajectory of the UAV with the crack detection results, it is ensured that the crack location is highly consistent with the flight path.

[0103] Based on the flight trajectory data of the UAV, multiple location points collected by the UAV during flight are fused with the location data of the preliminary crack detection results. The spatial coordinates of the crack on the ground surface are calculated using a spatiotemporal fusion positioning method. This spatiotemporal fusion positioning method uses spatiotemporal weighted interpolation to calculate the preliminary correction value of the crack location.

[0104]

[0105] Where t is the time for crack location detection and UAV positioning data, and P corrected (t) represents the initial correction value for the crack location, P detected P represents the location data from the initial crack detection results. i (t) represents the UAV location information at time i, ω i (t) represents the spatiotemporal weighting coefficient, indicating the weight under different time and spatial locations, and n is the number of sample points used for spatiotemporal fusion. Simultaneously, an adaptive correction particle swarm optimization method is applied to dynamically adjust the spatiotemporal weighting coefficient. Each particle in the particle swarm represents a correction scheme for the crack position. The particle adjusts its position according to the velocity and heading angle of different correction schemes, updating the particle's position in each time interval to obtain different spatiotemporal reference points. The dynamic adjustment of the spatiotemporal weighting coefficient is achieved by determining the spatiotemporal reference points, including the initial crack position and the UAV's historical position data. Based on the initial crack detection position and the UAV's flight trajectory, the spatiotemporal distance difference between each reference point is calculated. A two-dimensional Gaussian function is used to calculate the weighting coefficient for each reference point based on the spatiotemporal distance difference. By correcting the crack position through spatiotemporal weighted interpolation, multiple reference points in time and space can be considered, ensuring the accuracy of the crack position.

[0106] Combining the initial detection results of the crack, the initial correction value of the crack location, and the flight dynamic data of the UAV, the spatial coordinates of the crack are further corrected by incremental correction using data on flight speed and heading angle. This corrects potential errors in the UAV's position and eliminates errors caused by environmental factors and changes in flight trajectory.

[0107] This invention combines preliminary crack detection results with real-time navigation and positioning data from a UAV to accurately calculate the spatial coordinates of the crack on the ground. The spatiotemporal fusion positioning method utilizes spatiotemporal weighted interpolation and an adaptive correction particle swarm optimization algorithm to dynamically adjust the correction value of the crack position, ensuring that the crack position is highly consistent with the flight trajectory. By fusing multiple spatiotemporal reference points, the error in the crack position can be effectively corrected, and the influence of environmental factors and changes in the flight trajectory can be considered, thus improving the accuracy of crack positioning. At the same time, the incremental correction method further optimizes the crack position, ensuring that the final positioning data is accurate and reliable, and significantly improving the effectiveness of crack detection and repair decisions.

[0108] In this embodiment, the process of obtaining the primary crack classification result specifically includes:

[0109] Based on the spatial coordinates and feature information of the cracks, the geometric feature data of the cracks are extracted, including the length, width, depth, extent of expansion, distance between cracks, and distribution density. The geometric feature data is then processed by graph structuring to construct a spatial topology graph of the cracks. The adjacency relationship of each crack feature in the graph is calculated. The graph structuring process can fully consider the spatial dependence between cracks and effectively extract feature data with high discriminative ability for crack classification. This data is then input into the deep learning classification model to ensure the representativeness and discriminativeness of the model input data.

[0110] In the first layer of the deep learning hierarchical model, a multi-scale convolutional neural network is used to perform preliminary processing of crack feature data, extracting local and global features of cracks from different scales. Smaller-scale convolutional layers focus on the detailed features of cracks, while larger-scale convolutional layers focus on the overall shape of cracks. Multi-level convolution and pooling operations can better capture the complex geometric structure and spatial information of cracks.

[0111] In the second layer of the deep learning grading model, topology optimization is used to perform deep processing on the extracted local and global features of the cracks. By calculating the relationship between the local and global characteristics of the cracks, the influence of each feature of the crack is evaluated, the weight of the crack features is dynamically adjusted, and the optimal grading path for each crack is calculated through a global path optimization strategy. The topology optimization adaptively adjusts the importance of crack features according to the local complexity and global distribution pattern of the cracks, thereby optimizing the structure of the crack grading model so that the contribution of different features can dynamically match the actual state and risk level of the cracks.

[0112] The optimal grading path for each crack is fed into the grading module of the deep learning grading model. Based on the crack's feature data, the Softmax activation function is used to calculate the probability distribution of the crack in different grading categories, including mild cracks, moderate cracks, and severe cracks. The hierarchical Softmax activation function can accurately assign cracks to the corresponding grading categories based on the crack's geometric features and spatial relationships, and obtain the initial crack grading results.

[0113] This invention constructs a spatial topology map of cracks by extracting their geometric features and performing graph structuring, effectively considering the spatial dependencies between cracks and ensuring high discriminative power of crack feature data in deep learning models. A multi-scale convolutional neural network extracts local and global features of cracks in the initial processing, enhancing the understanding of crack geometry. A topology optimization algorithm dynamically adjusts the weights of crack features and optimizes the grading path, making the grading results more consistent with the actual crack state and risk level. Through the Softmax activation function, cracks are accurately assigned to appropriate grading categories, thereby improving the accuracy and reliability of crack detection and grading.

[0114] In this embodiment, the process of generating the detailed crack classification report specifically includes:

[0115] Based on the spatial coordinates of the crack and environmental information, the risk level of the crack is calculated using a weighted average. The risk level of the crack is obtained by weighting the primary risk level and the environmental weight coefficient. The environmental weight coefficient is adjusted based on the crack location and environmental conditions to ensure that the classification result reflects the true risk.

[0116] The initial crack classification results are combined with the crack spatial coordinates to generate a detailed crack classification report, including the crack spatial coordinates, crack type and corresponding risk level, ensuring that the classification result and location of each crack correspond one-to-one.

[0117] Detailed crack classification reports are transmitted to the ground control platform in real time via a wireless communication module. The real-time transmission process encapsulates the report data according to a time sequence and transmits the report through a standard communication protocol to ensure stable data transmission and synchronization to the ground platform.

[0118] This invention calculates the risk level of cracks using a weighted average method, combines the initial classification results with the crack's spatial coordinates to generate a detailed crack classification report, and dynamically adjusts the crack risk assessment using an environmental weight coefficient to ensure that the classification results accurately reflect the actual risk of the cracks. The classification result of each crack corresponds precisely to its location, ensuring the integrity and accuracy of the data. Through a wireless communication module, the detailed crack classification report is transmitted to the ground control platform in real time, ensuring timely and stable data transmission and synchronous updates, further improving the timeliness and scientific nature of crack repair decisions.

[0119] In this embodiment, the real-time updating of the crack identification and location information specifically includes:

[0120] During the continued flight of the UAV, the crack location data in the crack classification report is monitored in real time. Based on the current flight path and location, the UAV is controlled to continue to collect new image data along the predetermined route. The spatial coordinates corresponding to the current image data are determined through the UAV's navigation system to obtain newly collected crack image data, ensuring that each image is associated with the current flight position.

[0121] The newly acquired crack image data is compared with the existing crack classification report. When the data difference in crack location and type in the comparison results does not exceed a specified threshold, the crack identification and location information is updated, and the crack type, risk level and location coordinates are adjusted in real time. The specified threshold is one percent of the total crack data.

[0122] The updated crack information is transmitted to the ground control platform in real time via a wireless communication module, ensuring that the ground platform receives the latest crack information in a timely manner.

[0123] This invention ensures that the UAV continuously collects new image data during flight by monitoring the crack location data in the crack classification report in real time, and compares it with the existing report to update the crack identification, type, risk level and location coordinates in a timely manner. This dynamic adjustment mechanism can improve the accuracy of crack location, reduce errors and ensure the consistency and accuracy of data. Through the wireless communication module, the updated crack information can be transmitted to the ground control platform in real time to ensure that the platform can obtain the latest data in a timely manner.

[0124] In this embodiment, the evaluation of the priority of crack repair specifically includes:

[0125] Based on the updated crack information, including crack spatial coordinates, type, and classification level, ground operators assess the priority of crack repair. The assessment process considers the crack's risk level, extent of propagation, location, environmental factors, and potential impact on the structure.

[0126] Ground operators make repair decisions based on the repair priorities and detailed information about the cracks obtained from the assessment. The repair decisions include determining the specific repair methods, required repair materials, tools, and timing of the repairs. Based on the type of crack, the scope of impact, and the environmental conditions, and in conjunction with existing repair resources, operators select appropriate repair plans and operational steps to obtain repair decision information.

[0127] Ground operators transmit repair decision information to the repair team in real time via wireless communication modules. The repair team then carries out on-site repair operations based on the repair decision information to ensure that the cracks are repaired in a timely and effective manner.

[0128] This invention provides ground operators with a scientific assessment of repair priorities by updating crack information in real time and combining the crack's spatial coordinates, type, and classification level. Based on the assessment results, operators can make accurate repair decisions according to the crack's risk level, extent of expansion, and environmental factors, and select appropriate repair methods, materials, and tools. Through a wireless communication module, the repair decision information can be transmitted to the repair team in real time, ensuring timely repair operations.

[0129] Example 1:

[0130] To verify the feasibility of this invention in practice, it was applied to the crack detection and repair process in an industrial park. This park comprises multiple buildings, pipelines, and road systems, covering an area of ​​300,000 square meters. The facilities have been subjected to long-term natural factors and transportation loads, resulting in cracks of varying degrees on their surfaces. Most cracks in the park are mild to moderate, with some areas also exhibiting severe structural cracks. Traditional crack detection methods mainly rely on manual inspections and ground sensors for location and assessment. This method is not only time-consuming but also easily affected by human factors. Especially in complex environments, the inefficiency of traditional methods leads to slow decision-making regarding crack repair, making it difficult to guarantee the repair effect.

[0131] Against this backdrop, the surface crack location and classification system based on UAV navigation and image recognition has been introduced into the crack detection and repair work in industrial parks. The system first plans and controls the UAV to autonomously navigate along a preset path using a UAV flight control module. The UAV uses a high-resolution camera to collect images of cracks on the surfaces of buildings, pipelines, and roads within the park, simultaneously recording information such as flight position, flight altitude, and camera parameters. Each flight covers an area of ​​approximately 20,000 square meters and lasts for 40 minutes, ensuring comprehensive crack detection within the park.

[0132] The acquired raw image data is transmitted to the ground control platform in real time via a wireless communication module. The data preprocessing module performs noise reduction, contrast enhancement, and automatic white balance on the images to generate standard image data, ensuring that the images are interference-free and have clear details. The crack automatic identification module identifies cracks in the standard images, extracts the contours and feature information of the cracks, and generates preliminary crack detection results. This process is analyzed using a deep learning hierarchical model to accurately calculate the spatial coordinates of the cracks. Combined with the real-time navigation and positioning data of the UAV, a spatiotemporal fusion positioning method is used to correct potential errors in the crack location, ensuring accurate crack positioning.

[0133] As the drone continues its flight, the system continuously collects new image data and updates crack identification and location information in real time. When new crack information is collected and updated, ground operators can immediately assess the crack repair priority and make repair decisions. The repair decisions consider the crack's risk level, extent of expansion, environmental factors, and potential impact on the structure. Based on the assessment results, ground operators determine the crack repair method, required materials, tools, and timing. The repair decision information is transmitted to the repair team in real time via a wireless communication module, and the repair team performs on-site repair operations according to the instructions.

[0134] In practical applications, this invention has demonstrated significant advantages and effects. Using drones for patrols not only improves the efficiency of crack detection but also enables coverage of large areas. Traditional manual inspection methods can cover a maximum area of ​​approximately 5,000 square meters per patrol, while this system can cover approximately 20,000 square meters per patrol, increasing efficiency by about four times. The automation of image processing and crack recognition also greatly shortens crack detection time; each image is processed in just 4 seconds, whereas manual inspection requires manual judgment, which is much more time-consuming.

[0135] In terms of repair decision-making, the system of this invention is more efficient than traditional manual inspection methods. Manual inspection requires manual judgment of crack repair priorities, and the assessment and decision-making time usually exceeds 2 hours. However, with this system, the assessment time for repair priorities is shortened to 12 minutes, and the transmission time for repair decisions is reduced from traditional manual recording and transmission to 3 seconds. The data collected by the drone not only improves the crack positioning accuracy, controlling the crack location error within ±5cm, but also enables the repair decisions to be transmitted in real time through the wireless communication system, greatly improving the timeliness and accuracy of the repair process.

[0136] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A surface crack location and classification system based on UAV patrol and image recognition, characterized in that, include: The drone flight control module is used to control the drone to perform autonomous navigation and collect raw image data; The data preprocessing module is used to preprocess real-time data from the UAV to generate standard image data; The automatic crack identification module is used to automatically identify cracks in standard image data and generate preliminary crack detection results. The wireless communication module is used to transmit crack image data, grading reports, and repair decisions to the ground control platform in real time. The ground control platform is used to receive real-time data from UAVs and to store, process, and visualize the data. The spatiotemporal fusion positioning module is used to accurately calculate the spatial coordinates of the crack on the ground surface through the spatiotemporal fusion positioning algorithm, and to correct potential errors in the crack location; The grading module is used to intelligently grade cracks based on their spatial coordinates and feature information using a deep learning grading model, generating a preliminary grading result for the cracks. The Repair Decision Support module is used to assess the priority of crack repair and make repair decisions, guiding the repair team to carry out repair operations.

2. The surface crack location and classification system based on UAV patrol and image recognition according to claim 1, characterized in that, The modules are connected in the following way: The drone autonomously cruises along a preset flight path to collect raw image data covering the ground surface area. The high-definition image data is transmitted to the ground control platform, and the images are preprocessed to obtain standard image data; Automatic crack identification is performed on standard image data, and crack contours and feature information in the images are extracted to generate preliminary crack detection results. Based on the preliminary crack detection results, combined with the real-time navigation and positioning data of the UAV, the spatial coordinates of the crack on the ground surface are accurately calculated through spatiotemporal fusion positioning to correct potential errors in the crack location. Based on the spatial coordinates and feature information of the cracks, a multi-level deep learning hierarchical model with topology optimization is used to intelligently classify the cracks and obtain the primary crack classification results. The initial crack classification results are combined with the crack spatial coordinates to generate a detailed crack classification report, which is then transmitted to the ground control platform in real time via wireless communication. As the drone continues to fly, it continuously collects new image data based on the crack classification report and updates crack identification and location information in real time, feeding the updated crack information back to the ground platform. Based on the updated crack information, ground operators assess the priority of crack repair and make repair decisions, guiding the repair team to carry out repair operations.

3. The surface crack location and classification system based on UAV patrol and image recognition according to claim 2, characterized in that, The acquisition of the original image data covering the land surface area specifically includes: Set the flight path parameters for the UAV and transmit the flight path information to the UAV's flight control system through the ground control platform; The drone automatically starts up based on the flight path parameters and begins to cruise along the set path, flying within the designated area according to the planned path; During the cruise, the drone uses its onboard high-resolution camera to collect real-time image data of the ground area and records auxiliary information synchronously through its built-in sensors to obtain raw image data of the covered ground area.

4. The surface crack location and classification system based on UAV patrol and image recognition according to claim 2, characterized in that, The process of generating the preliminary detection results of the cracks specifically includes: The raw image data is transmitted from the UAV to the ground control platform in real time via a wireless communication module. The ground control platform performs noise reduction processing on the image, uses a Gaussian filtering algorithm to smooth the image, removes noise from the image, and obtains clear and interference-free image data. Automatic white balance processing is performed on clear and interference-free image data, and grayscale equalization is used to enhance the contrast and brightness of the image to obtain the white balance processed image. Edge detection is performed on the white-balanced image. The Canny operator is used to identify and extract crack edge information in the image. The crack contour is enhanced by image dilation to obtain standard image data.

5. The surface crack location and classification system based on UAV patrol and image recognition according to claim 2, characterized in that, The generation of the preliminary detection results of the cracks specifically includes: Standard image data is input into the crack automatic identification module, which automatically identifies and extracts low-level features from the image; Based on the extracted low-level features, multi-layer convolution and pooling operations are used to further extract the geometric features of the cracks, identify the high-level features of the cracks, generate the depth information and feature map of the cracks, and provide a preliminary description and feature distribution of the cracks. Based on the depth information and feature map of the crack, region extraction is used to accurately locate the crack region. By detecting the crack region in the image through a sliding window, the precise coordinate position of each crack region is obtained. The contour boundary of the crack is corrected by combining context information, and clear and accurate preliminary crack detection results are generated.

6. The surface crack location and classification system based on UAV patrol and image recognition according to claim 2, characterized in that, The calculation of the crack spatial coordinates specifically includes: The preliminary crack detection results are combined with the real-time navigation and positioning data of the UAV to obtain the latitude and longitude information of the current position of the UAV, and the time information of crack detection is recorded by timestamp to obtain the flight trajectory data of the UAV. Based on the flight trajectory data of the UAV, multiple location points collected by the UAV during flight are fused with the location data of the preliminary crack detection results. The spatial coordinates of the crack on the ground surface are calculated using a spatiotemporal fusion positioning method. The spatiotemporal fusion positioning method uses spatiotemporal weighted interpolation to calculate the preliminary correction value of the crack position and applies an adaptive correction particle swarm optimization method to dynamically adjust the spatiotemporal weighting coefficients. Each particle in the adaptive correction particle swarm optimization method represents a correction scheme for the crack position. The particle adjusts its position according to the velocity and heading angle of different correction schemes. The particle position is updated in each time interval to obtain different spatiotemporal reference points. The dynamic adjustment of the spatiotemporal weighting coefficients is achieved by determining the spatiotemporal reference points and calculating the spatiotemporal distance difference between each reference point. A two-dimensional Gaussian function is used to calculate the weighting coefficient of each reference point based on the spatiotemporal distance difference. Combining the initial detection results of the crack, the initial correction value of the crack location, and the flight dynamic data of the UAV, incremental correction is used to further correct the spatial coordinates of the crack, correct the potential error of the UAV's position, and eliminate the error caused by environmental factors and flight trajectory changes.

7. The surface crack location and classification system based on UAV patrol and image recognition according to claim 2, characterized in that, The process of obtaining the primary crack classification results specifically includes: Based on the spatial coordinates and feature information of the cracks, the geometric feature data of the cracks are extracted, and the geometric feature data is processed into a graph structure to construct a spatial topology graph of the cracks. The adjacency relationship of each crack feature in the graph is calculated and input into a deep learning hierarchical model. In the first layer of the deep learning hierarchical model, a multi-scale convolutional neural network is used to perform preliminary processing of crack feature data, extracting local and global features of cracks from different scales; In the second layer of the deep learning hierarchical model, topology optimization is used to perform deep processing on the local and global features of the extracted cracks. By calculating the relationship between the local and global characteristics of the cracks, the influence of each feature of the crack is evaluated, the weight of the crack features is dynamically adjusted, and the optimal hierarchical path for each crack is calculated through a global path optimization strategy. The optimal grading path for each crack is fed into the grading module of the deep learning grading model. Based on the crack's feature data, the Softmax activation function is used to calculate the probability distribution of the crack in different grading categories, and the initial crack grading results are obtained.

8. The surface crack location and classification system based on UAV patrol and image recognition according to claim 2, characterized in that, The process of generating the detailed crack classification report specifically includes: Based on the spatial coordinates of the crack and environmental information, the risk level of the crack is calculated using a weighted average. The risk level of the crack is obtained by weighting the primary risk level and the environmental weight coefficient, which is adjusted based on the crack location and environmental conditions. The initial crack classification results are combined with the crack spatial coordinates to generate a detailed crack classification report; Detailed crack classification reports are transmitted to the ground control platform in real time via a wireless communication module. The real-time transmission process encapsulates the report data according to a time sequence and transmits the report through a standard communication protocol.

9. The surface crack location and classification system based on UAV cruise and image recognition according to claim 2, characterized in that, The real-time updating of the crack identification and location information specifically includes: During the continued flight of the UAV, the location data of the crack in the crack classification report is monitored in real time. Based on the current flight path and location, the UAV is controlled to continue to collect new image data along the predetermined route. The spatial coordinates corresponding to the current image data are determined through the UAV's navigation system to obtain the newly collected crack image data. The newly acquired crack image data is compared with the existing crack classification report. When the data difference in crack location and type in the comparison results does not exceed the specified threshold, the crack identification and location information is updated, and the crack type, risk level and location coordinates are adjusted in real time. The updated crack information is transmitted to the ground control platform in real time via a wireless communication module.

10. The surface crack location and classification system based on UAV cruise and image recognition according to claim 2, characterized in that, The assessment of the priority of crack repair specifically includes: Based on the updated crack information, ground operators assessed the priority of crack repair. Ground operators make repair decisions based on the repair priorities and detailed information about the cracks obtained from the assessment. Based on the type of crack, the scope of impact, and the environmental conditions, and in combination with the existing repair resources, the operators select appropriate repair plans and operation steps to obtain repair decision information. Ground operators transmit repair decision information to the repair team in real time via wireless communication modules, and the repair team carries out on-site repair operations based on the repair decision information.

Citation Information

Cited By

  • Regional geological disaster prevention and control management method, device and equipment and storage medium

    CN121545295A