Crack information acquisition method based on unmanned aerial vehicle photography

Abstract

The invention provides a crack information acquisition method based on unmanned aerial vehicle photography. The crack information acquisition method comprises the following steps: calculating an object distance during shooting and an actual size represented by a single pixel at the object distance according to factory parameters of a camera; shooting the target crack structure through a camera to obtain a crack image; obtaining a vertical distance from the camera to the crack according to the distance measuring sensor, and obtaining an object distance during shooting; the crack image is preprocessed through an OpenCV library, the pixel width and the pixel distance of the crack are obtained from the crack image, and crack information is obtained according to the pixel width and the pixel distance of the crack; marking the crack information into the crack image through an OpenCV library; and repeating the above steps, marking crack information of all cracks in the target crack structure into corresponding crack images, and performing classified storage to form a crack database. According to the method, structured management of massive crack images and crack information can be realized, and crack database support is provided for subsequent repair and maintenance and operation and maintenance.

Description

Technical Field

[0001] The present invention relates to the field of engineering structure cracks and digital technology, and in particular to a method for obtaining crack information based on drone photography. Background Art

[0002] In concrete structures, cracks not only damage the appearance of the building, but also provide channels for moisture and harmful substances to penetrate into the concrete structure, which in turn causes steel corrosion and affects the bearing capacity and durability of the structure. Structural operation and maintenance engineers need to repair the cracks in the concrete structure to ensure the long-term safety and stability of the building. Traditional crack detection technologies such as visual inspection, ultrasonic testing, radiographic testing, and core sampling have limitations such as high reliance on manual operation, low detection quality and efficiency, and greater risks to the safety of operators.

[0003] With the continuous development of science and technology, drone technology has been widely used in various fields, one of which is building crack detection. Crack measurement technology based on drone photography has gradually become an important tool in the field of civil engineering with its high efficiency, safety and economy. The core principle of drone photography crack measurement technology is computer vision technology. The surface of the building is photographed by high-resolution photography equipment carried by the drone to obtain crack images. Then, image processing and analysis software is used to automatically identify cracks and measure their geometric parameters. Drone photography crack measurement technology provides a new solution for building crack detection.

[0004] At present, drones in the UAV photogrammetry crack technology mainly obtain images and videos of crack structures. The images obtained or converted from video frames are used to identify cracks and obtain parameters through intelligent algorithms on the background server, and then the damage state of the crack structure is evaluated. Because image acquisition and image recognition belong to different work teams or different working environments, only images are used as deliverables for data handover. Key information such as the latitude and longitude coordinates of the crack image, the position relative to the structure, and the distance between the drone shooting position and the structure surface are not obtained or are not transmitted to the downstream working environment due to insufficient understanding after being obtained, resulting in unsatisfactory application of structural damage technology based on crack parameters. There are a lot of related image data but no effective data has been formed, which is not convenient for later crack identification and archiving management. Summary of the invention

[0005] The purpose of the present invention is to provide a method for obtaining crack information based on drone photography, which is used to solve the problems in the prior art that there is a lot of image data but no effective data is formed, which is not convenient for subsequent crack identification and archiving management. It can realize structured management of massive crack images and crack information, and provide crack database support for subsequent maintenance and operation.

[0006] In order to achieve the above object, the present invention provides a method for obtaining crack information based on drone photography. The drone is provided with a camera, a ranging sensor and a server. The server is provided with a crack recognition algorithm and an OpenCV library. The method comprises: Step 1, calculating the object distance during shooting and the actual size represented by a single pixel at the object distance according to the factory parameters of the camera; Step 2: After the crack recognition algorithm determines the position of the crack in the target crack structure, the target crack structure is photographed by a camera to obtain a crack image; Step 3, obtaining the vertical distance from the camera to the crack according to the distance measuring sensor, and obtaining the object distance during shooting; Step 4: Preprocess the crack image through the OpenCV library, and then obtain the pixel width and pixel distance of the crack, and obtain the crack information according to the pixel width and pixel distance of the crack; Step 5: Annotate the crack information into the crack image through the OpenCV library; Step 6: Repeat steps 2 to 5 for multiple times, annotate the crack information of all cracks in the target crack structure into the corresponding crack image, and save them by classification to form a crack database.

[0007] According to a method for obtaining crack information based on drone photography provided by the present invention, the crack image includes multiple independent cracks, each crack includes a node and a crack body, the node records the position coordinates of the two ends of the crack, and the crack body records the crack information. All cracks are connected through common nodes to form a crack network; the crack information includes crack position parameters, basic crack geometric parameters and crack shape.

[0008] According to a method for obtaining crack information based on drone photography provided by the present invention, crack position parameters include crack axis intersection endpoint numbers and crack actual endpoint numbers.

[0009] According to a method for obtaining crack information based on drone photography provided by the present invention, the crack shape refers to the curve shapes of the upper side, the lower side and the central axis of the crack described respectively by three mathematical expressions.

[0010] According to a method for obtaining crack information based on drone photography provided by the present invention, the three mathematical expressions specifically include: constructing a global coordinate system based on the crack grid image, and using polynomial fitting to generate mathematical expressions of the upper side, lower side and central axis of the crack.

[0011] According to a method for obtaining crack information based on drone photography provided by the present invention, basic crack geometric parameters include average crack width, maximum crack width, minimum crack width, and crack length.

[0012] According to a method for obtaining crack information based on drone photography provided by the present invention, the average crack width, the maximum crack width, and the minimum crack width are obtained by calculating the shortest distance through mathematical expressions of the upper side and the lower side of the crack, and the crack length is calculated through a mathematical expression of the center axis of the crack.

[0013] According to a method for obtaining crack information based on drone photography provided by the present invention, the distance measuring sensor includes a laser rangefinder, a radar, a sonar, an infrared rangefinder, an ultrasonic rangefinder, an optical rangefinder and a GPS rangefinder.

[0014] According to a crack information acquisition method based on drone photography provided by the present invention, crack recognition algorithms include Faster R-CNN, SSD, YOLO series, Inception series, MobileNet, Unet series, transform series, and deeplab series.

[0015] The present invention provides a method for obtaining crack information based on drone photography, including: step 1, calculating the object distance during shooting and the actual size represented by a single pixel at the object distance according to the factory parameters of the camera; step 2, after the crack recognition algorithm determines the position of the crack in the target crack structure, the target crack structure is photographed by the camera to obtain a crack image; step 3, the vertical distance from the camera to the crack is obtained according to the ranging sensor, and the object distance during shooting is obtained; step 4, the crack image is preprocessed by the OpenCV library, and then the pixel width and pixel distance of the crack are obtained therefrom, and the crack information is obtained according to the pixel width and pixel distance of the crack; step 5, the crack information is annotated to the crack image by the OpenCV library; step 6, repeating steps 2 to 5 for many times, annotating the crack information of all cracks in the target crack structure to the corresponding crack image, and classifying and saving to form a crack database. The present invention can realize the structured management of massive crack images and crack information, and provide crack database support for subsequent maintenance and operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] In the attached picture: Figure 1 The original crack map of the target crack structure of the present invention; Figure 2 A schematic diagram of a fracture network of a target fracture structure of the present invention; Figure 3 is a schematic diagram of the fracture network map of the present invention; Figure 4 This is a schematic diagram of the fracture geometry fitting of the present invention; Figure 5 A schematic diagram of a crack image and corresponding crack information in the crack database of the present invention; Figure 6 The present invention is a flow chart of a method for obtaining crack information based on drone photography. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Some embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0020] The crack information acquisition method based on drone photography provided by the present invention has the following general principles: a reasonable shooting object distance and the actual size represented by a single pixel at different shooting object distances are calculated according to the factory parameters of the airborne camera; a real-time image of the target crack structure is acquired by a camera, the position of the crack in the target crack structure is determined by the airborne crack recognition algorithm, the camera shutter is triggered to capture the image, and the vertical distance from the camera to the crack is acquired according to the range-finding sensor carried. After the crack image is pre-processed by the OpenCV library in the airborne server, the pixel width and pixel distance of the crack are acquired, and the actual shape, width, length and other crack information of the target crack are calculated according to the conversion relationship between the size of the crack and the pixel distance and pixel width, the crack information is annotated to the crack image through the OpenCV library in the airborne server, and the crack image with the crack information is saved to form a crack database for downstream maintenance and maintenance decision-making.

[0021] See also Figure 6 The embodiment of the present invention provides a method for obtaining crack information based on drone photography, wherein the drone is provided with a camera, a ranging sensor and a server, and the server is provided with a crack recognition algorithm and an OpenCV library, and the method comprises: Step 1: Calibrate the camera parameters. Calculate the object distance during shooting and the actual size of a single pixel at the object distance according to the factory parameters of the camera. Step 2: Photograph the target crack structure. After the crack recognition algorithm determines the location of the crack in the target crack structure, the target crack structure is photographed by a camera to obtain a crack image; Step 3: Obtain the object distance. Obtain the vertical distance from the camera to the crack using the distance measuring sensor to obtain the object distance during shooting; this provides object distance parameters for the subsequent acquisition of the proportional relationship between crack information and pixel width and pixel distance.

[0022] Step 4, obtain crack information. Use the OpenCV library to perform preprocessing such as correction and binarization on the crack image, and then obtain the pixel width and pixel distance of the crack, and obtain the crack information based on the pixel width and pixel distance of the crack; Step 5: Crack information annotation: The crack information is annotated into the crack image using the OpenCV library.

[0023] Step 6: Crack database storage. Repeat steps 2 to 5 multiple times, annotate the crack information of all cracks in the target crack structure into the corresponding crack image, and save them according to the target crack structure type, crack structure location, etc. to form a crack database. This achieves structured management of a massive crack database consisting of crack images and crack information, providing crack database data support for subsequent maintenance and operation.

[0024] Specifically, the object distance during shooting and the actual size represented by a single pixel at the object distance are calculated according to the factory parameters of the camera. This is based on the basic principle of lens imaging, and the corresponding relationship formula is: Imaging formula

[0025] Magnification formula

[0026] The actual size represented by the unit pixel

[0027] Final formula

[0028] In the formula, W is the actual width of the object, p is the imaging width of the object on the imaging plane (in pixels), d 0 is the object distance, f is the focal length. The formula here assumes that the distance between the imaging plane and the lens is h i Equal to the distance d i , and the imaging plane is linear and has no distortion.

[0029] In order to obtain a clearer crack image, the focal length of the camera needs to be adjusted according to the distance between the camera and the crack. The reasonable focal length corresponding to different distances and the actual size represented by the corresponding unit pixel need to be calibrated according to the test. The width of the general crack ranges from 0.1mm to 2mm, and the empirical shooting distance ranges from 1m to 15m, which is related to the camera model.

[0030] Distance measuring sensors mainly include laser rangefinders (LIDAR), radars, sonars, infrared rangefinders, ultrasonic rangefinders, optical rangefinders, and GPS rangefinders. They each use different technical principles to measure distances and are suitable for a variety of application scenarios such as terrain mapping, obstacle avoidance, object detection, and visual positioning. When choosing, you need to decide which sensor to use based on specific needs and environmental conditions. Laser rangefinders and optical rangefinders are generally used for concrete crack measurement.

[0031]

[0032] The main crack recognition algorithms include Faster R-CNN, SSD, YOLO series, Inception series, MobileNet, Unet series, transform series, deeplab series, or combinations of these architectures and other types of architectures.

[0033] Table 2. Commonly used crack identification algorithms

[0034] Of course, it is understandable that the crack recognition algorithm needs to be trained with a crack detection database and can only be deployed and applied to actual crack detection after meeting the target requirements. The relevant crack detection database is shown in Table 3.

[0035] Table 3. Related crack detection databases

[0036] In some embodiments, the crack image includes multiple independent cracks, each crack includes a node and a crack body, the node records the position coordinates of the two ends of the crack, and the crack body records the crack information. All cracks are connected through common nodes to form a crack network, and the crack network can be represented as a crack network map composed of "node-crack body-node"; the crack information includes crack position parameters, basic crack geometric parameters and crack shape. The crack information can be stored through data structures such as JSON and arrays, and parsed through corresponding parsing algorithms.

[0037] Among them, the crack position parameters include the intersection endpoint number of the crack axis and the actual endpoint number of the crack. The basic geometric parameters of the crack include information such as the average width of the crack, the maximum width of the crack, the minimum width of the crack, and the length of the crack, which can be obtained by calculating the mathematical expression of the crack shape. The width of the crack (average width, maximum width, minimum width) is obtained by the shortest distance between the upper side and the lower side of the crack. The length of the crack refers to the length calculated by the mathematical expression of the crack center axis. The crack shape refers to the shape of the crack described by a mathematical expression, including three mathematical expressions for the upper side, lower side and center axis of the crack, specifically including: constructing a global coordinate system based on the image pixel grid map of the crack (crack grid image), using polynomial fitting to generate the mathematical expression of the crack, and storing the coefficients of the mathematical expression in order of the power size of the variable, generally an array or linked list data structure can be selected.

[0038] In some embodiments, step 4 specifically includes: The crack network in the crack image is located and cropped through the OpenCV library to form an image unit with a single crack. The background noise of the single crack is removed through filtering algorithms such as candy and Gauss to form a single crack skeleton. For the image unit with a single crack, polynomial fitting is performed on the geometric shapes of the upper side, lower side and central axis of the crack to form a mathematical expression. According to the proportional relationship between the crack size and the pixel width and pixel distance (that is, the basic principle of lens imaging mentioned above), mathematical calculations are performed based on the above three mathematical formulas to obtain the length of the crack, the average width of the crack, the maximum width of the crack and the position coordinates of the two ends.

[0039] It should be noted that Figure 1 is the original crack map of the target crack structure. Figure 2 is a schematic diagram of the crack network of the target crack structure, Figure 2 There are three cracks in the figure, marked as crack 1, crack 2, and crack 3. The intersection of the axes of crack 1 and crack 2 is point A. Crack 1 and crack 3 do not intersect, but the intersection of their axes is point B, that is, one of the nodes of crack 3 is at a distance from the starting point of crack 1. Accordingly, Figure 3 In the figure, the two nodes of crack 1 are numbered as node[1] and node[5]; the two nodes of crack 2 are numbered as node[2] and node[4]; the two nodes of crack 3 are actually numbered as node[3,1] and node[6,1], and the intersection points of the axes are numbered as node[3] and node[6]; there are five crack bodies formed between the three cracks: edge[1,2], edge[2,3], edge[3,5], edge[2,4], and edge[3,6]. Figure 4 In the figure, it is shown that in the crack grid image, the three mathematical expressions of the upper side, lower side and middle axis of the crack are Y u=f(x1),Y c =a0+a1x+a2x 2 +a3x 3 +……、Y b = f(x2), and the maximum crack width, average crack width and length are shown. Figure 5 The crack image of a bridge pier on a certain shooting date in the crack database and the corresponding crack information are annotated, including crack number, structure name, coordinate position, maximum width, crack length, crack width, crack shape, etc.

[0040] In summary, the present invention provides a method for obtaining crack information based on drone photography, which obtains crack information by obtaining the vertical distance between the camera and the crack and the crack image, and marks it in the crack image, thereby realizing structured management of massive crack images and crack information, and providing crack database support for subsequent maintenance and operation. The relevant technology is mature, the research and development difficulty is low, the application range is wide, and the market potential is huge, and at the same time, it promotes the development of intelligent operation and maintenance technology based on crack information.

[0041] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the embodiments disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed by the present invention. It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for obtaining crack information based on drone photography, characterized in that: The drone is provided with a camera, a distance measuring sensor and a server, the server is provided with a crack recognition algorithm and an OpenCV library, and the method comprises: Step 1, calculating the object distance during shooting and the actual size represented by a single pixel at the object distance according to the factory parameters of the camera head; Step 2: after the crack recognition algorithm determines the position of the crack in the target crack structure, the target crack structure is photographed by the camera to obtain a crack image; Step 3: Obtain the vertical distance from the camera to the crack according to the distance measuring sensor to obtain the object distance during shooting; Step 4: preprocess the crack image using the OpenCV library, and then obtain the pixel width and pixel distance of the crack therefrom, and obtain the crack information according to the pixel width and pixel distance of the crack; Step 5: annotating the crack information into the crack image through the OpenCV library; Step 6: Repeat steps 2 to 5 for multiple times, annotate the crack information of all cracks in the target crack structure into the corresponding crack image, and save them by classification to form a crack database.

2. The method for obtaining crack information based on drone photography according to claim 1 is characterized in that: The crack image includes multiple independent cracks, each crack includes a node and a crack body, the node records the position coordinates of the two ends of the crack, the crack body records the crack information, all cracks are connected through common nodes to form a crack network; the crack information includes crack position parameters, basic crack geometric parameters and crack shape.

3. The method for obtaining crack information based on drone photography according to claim 2 is characterized in that: The crack position parameters include the crack axis intersection endpoint number and the crack actual endpoint number.

4. The method for obtaining crack information based on drone photography according to claim 2 is characterized in that: The crack shape refers to the curve shapes of the upper side, lower side and middle axis of the crack described by three mathematical expressions respectively.

5. The method for obtaining crack information based on drone photography according to claim 4 is characterized in that: The three mathematical expressions specifically include: constructing a global coordinate system based on the crack grid image, and generating mathematical expressions of the upper side, lower side and middle axis of the crack by using a polynomial fitting method.

6. The method for obtaining crack information based on drone photography according to claim 5 is characterized in that: The basic geometric parameters of the crack include the average crack width, the maximum crack width, the minimum crack width, and the crack length.

7. The method for obtaining crack information based on drone photography according to claim 6 is characterized in that: The average width of the crack, the maximum width of the crack, and the minimum width of the crack are obtained by calculating the shortest distance between the upper side and the lower side of the crack through a mathematical expression, and the length of the crack is calculated through a mathematical expression of the center axis of the crack.

8. The method for obtaining crack information based on drone photography according to claim 1, characterized in that: The distance measuring sensors include laser rangefinders, radars, sonars, infrared rangefinders, ultrasonic rangefinders, optical rangefinders and GPS rangefinders.

9. The method for obtaining crack information based on drone photography according to claim 1, characterized in that: The crack recognition algorithms include Faster R-CNN, SSD, YOLO series, Inception series, MobileNet, Unet series, transform series, and deeplab series.

Images