Aqueduct crack inspection method and device
Through the combination of drone automatic cruise technology and aqueduct fracture segmentation model, efficient and accurate detection of aqueduct fractures is achieved, and the problems of low automation and insufficient detection accuracy in the existing technology are solved, and inspection efficiency and data accuracy are improved. .
Patent Information
- Application Number
- CN202411073685.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-11-12
AI Technical Summary
The existing drones have low degree of automation during aqueduct inspection, rely on manual control, have low patrol efficiency, high failure rate, and are difficult to accurately detect the location and width of the cracks, which cannot meet engineering needs.
UAV automatic cruise technology is used to obtain the aqueduct video to be tested through on-board imaging equipment, input the pre-trained aqueduct fracture segmentation model to determine the crack position, perform fixed-point shooting, and analyze the binary map using the skeletonized central axis algorithm and object distance method. Extract crack direction and width.
It improves the automation and efficiency of aqueduct crack patrol, acquires high-quality crack images, realizes accurate extraction of crack features, provides an accurate data basis, and provides detailed information for aqueduct evaluation and repair.
Smart Images

Figure CN118941993A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image recognition, and more particularly to a method and device for inspecting cracks in aqueducts. Background Art
[0002] At present, the autonomous cruise technology of drones has been widely used in the inspection of power transmission lines, but in the field of hydraulic engineering, especially aqueducts, the use of drones for inspections is still at the stage of manual control and shooting. The reasons are: First, after extreme weather or emergencies, the aqueduct and its surrounding environment are destroyed and the environment is complex, so temporary inspections of the aqueduct are required. However, it is difficult to ensure the flight safety of drones in complex environments, and highly professional flight controllers are required to control drones for manual inspections; second, drones face environmental restrictions when inspecting aqueducts. Due to the complex structure of aqueducts, the trough body is usually U-shaped, V-shaped, rectangular or circular. When drones fly inside aqueducts with complex structures, satellite positioning signals are damaged, positioning accuracy is low, and obstacle avoidance systems cannot be effectively used. The drones will face problems such as remote control failure, flight loss of control, and wall collisions, and manual control is required to ensure the safe flight of the drones.
[0003] In addition, the pixels and shooting clarity of the airborne imaging equipment carried by drones are limited. The quality of the image data obtained in one test is difficult to guarantee, and multiple tests are required. The inspection efficiency is low. If high-precision imaging equipment is replaced, the cost will increase. Moreover, in the existing crack image recognition technology, edge recognition technology, threshold segmentation method, feature transformation algorithm and morphological algorithm are generally used to extract crack features, which makes it difficult to further obtain the actual width of the crack and meet the actual needs of aqueduct detection.
[0004] Therefore, the current drone aqueduct inspection has a low degree of automation, is overly dependent on manual labor, has low inspection efficiency, a high inspection failure rate, poor crack detection effect, low safety, and is unable to record the exact location of cracks remotely. There is an urgent need for an aqueduct crack inspection method that can meet the actual engineering needs. Summary of the invention
[0005] The present application provides an aqueduct crack inspection method and device, which adopts drone automatic cruising technology to realize the inspection of aqueduct cracks, has a high degree of automation, improves the efficiency of aqueduct crack inspection, and can realize non-contact extraction of real crack information based on the extraction of crack characteristics, so that the crack information obtained is more accurate, providing an accurate data basis for aqueduct crack assessment and repair.
[0006] On the one hand, an embodiment of the present application provides a method for inspecting cracks in an aqueduct, comprising:
[0007] Acquire a video of the aqueduct to be tested, wherein the video of the aqueduct to be tested is taken by an airborne imaging device when the drone performs automatic inspection along an automatic inspection route;
[0008] Inputting the aqueduct video to be tested into a pre-trained aqueduct crack segmentation model to determine the crack development position;
[0009] Determine the target track point D according to the crack development position t , send a second inspection command to the UAV to obtain the position of the airborne imaging device at the target track point D t A regional crack image captured at a fixed point; wherein, for each crack, the pixel size of the crack in the regional crack image is larger than the pixel size of the crack in the video of the aqueduct to be tested;
[0010] The regional crack image is input into the aqueduct crack segmentation model to obtain a regional crack binary map, and the regional crack binary map is analyzed using a skeletonized medial axis algorithm and an object distance method to obtain crack directions and crack widths.
[0011] On the other hand, an embodiment of the present application further provides an aqueduct crack inspection device, including a processor and a memory:
[0012] The memory is used to store the aqueduct crack inspection program;
[0013] The processor is used to read the aqueduct crack inspection program and perform the aqueduct crack inspection method as described in the above embodiment.
[0014] Compared with the related art, the aqueduct crack inspection method and device of the embodiment of the present application adopts the automatic cruising technology of unmanned aerial vehicle to realize the inspection of aqueduct cracks, with a high degree of automation, thus improving the efficiency of aqueduct crack inspection. On the basis of determining the crack development position through the aqueduct video, the cracks are photographed at a fixed point to obtain high-quality crack images, which is helpful for the accurate observation of cracks on the aqueduct surface. On the basis of accurately extracting the crack characteristics, the non-contact extraction of the real information of the cracks can be realized, and the crack information obtained is more accurate, which increases the understanding of the detailed information of the cracks and provides an accurate data basis for the evaluation and repair of aqueduct cracks.
[0015] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by implementing the present application. Other advantages of the present application can be realized and obtained by the schemes described in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The accompanying drawings are used to provide an understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.
[0017] Figure 1is a flow chart of the aqueduct crack inspection method according to an embodiment of the present application;
[0018] Figure 2 The regional crack binary map and skeleton map of the embodiment of the present application are shown, wherein the left side is the regional crack binary map and the right side is the skeleton map;
[0019] Figure 3 It is a schematic diagram of the aqueduct crack inspection device according to an embodiment of the present application. DETAILED DESCRIPTION
[0020] The present application describes multiple embodiments, but the description is exemplary rather than restrictive, and it is obvious to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present application. Although many possible feature combinations are shown in the drawings and discussed in the specific embodiments, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with any other feature or element in any other embodiment, or may replace any other feature or element in any other embodiment.
[0021] The present application includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The embodiments, features and elements disclosed in the present application may also be combined with any conventional features or elements to form a unique invention scheme defined by the claims. Any features or elements of any embodiment may also be combined with features or elements from other invention schemes to form another unique invention scheme defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in the present application may be implemented individually or in any appropriate combination. Therefore, except for the limitations made according to the attached claims and their equivalents, the embodiments are not subject to other restrictions. In addition, various modifications and changes may be made within the scope of protection of the attached claims.
[0022] In addition, when describing representative embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps of the specific order described. As will be understood by those of ordinary skill in the art, other sequences of steps are also possible. Therefore, the specific sequence of the steps set forth in the specification should not be interpreted as a limitation to the claims. In addition, the claims for the method and / or process should not be limited to the steps of performing them in the order written, and those skilled in the art can easily understand that these sequences can be changed and still remain within the spirit and scope of the embodiments of the present application.
[0023] The present application embodiment provides a method for inspecting cracks in an aqueduct, including steps S1000-S4000, such as Figure 1 As shown,
[0024] S1000: Acquire a video of the aqueduct to be tested, wherein the video of the aqueduct to be tested is shot by an airborne imaging device when the drone performs automatic inspection along an automatic inspection route;
[0025] S2000: inputting the aqueduct video to be tested into a pre-trained aqueduct crack segmentation model to determine the crack development position;
[0026] S3000: Determine the target track point D according to the crack development position t , send a second inspection command to the UAV to obtain the position of the airborne imaging device at the target track point D t A regional crack image captured at a fixed point; wherein, for each crack, the pixel size of the crack in the regional crack image is larger than the pixel size of the crack in the video of the aqueduct to be tested;
[0027] S4000: Inputting the regional crack image into the aqueduct crack segmentation model to obtain a regional crack binary map, and using a skeletonized medial axis algorithm and an object distance method to analyze the regional crack binary map to obtain crack directions and crack widths.
[0028] In this embodiment, the UAV performs automatic inspections according to the automatic inspection route, and not only obtains the video of the aqueduct to be tested through the airborne imaging equipment, but also obtains the time synchronization information of the automatic inspection route and the video of the aqueduct to be tested through the integrated control system of the airborne imaging equipment and the flight control system, or through the satellite navigation system and timestamp of the UAV, or through the UAV flight data recorder.
[0029] In this embodiment, the aqueduct crack segmentation model is used to perform feature analysis on the input image to obtain a binary image, and the cracks can be predicted based on the binary image output by the aqueduct crack segmentation model; the aqueduct crack segmentation model can be a crack segmentation model based on a convolutional neural network, which improves the accuracy and efficiency of crack identification through deep learning technology. When constructing the aqueduct crack segmentation model, a special aqueduct crack image data set can be collected, and the model can be trained and optimized.
[0030] In this embodiment, the aqueduct crack segmentation model includes an encoding module, a decoding module and an output module connected in sequence; the encoding module includes a plurality of short-circuit connected residual blocks, each residual block includes a plurality of convolutional layers, which are used to perform deep convolutional neural network downsampling and feature extraction on the aqueduct crack image to obtain a multi-channel downsampling feature map; wherein the first convolutional layer and the last convolutional layer are convolutional layers that introduce an attention mechanism, which are used to automatically increase the weight of the crack information in the aqueduct crack image; the decoding module includes a plurality of transposed convolutional layers, which are used to perform deep convolutional neural network upsampling and resolution enhancement on the multi-channel downsampling feature map to generate a multi-channel upsampling feature map; the output module includes a 1*1 convolutional layer, which is used to generate a single-channel crack prediction segmentation map with the same size as the aqueduct crack image according to the multi-channel upsampling feature map, and the single-channel crack prediction segmentation map is a binary map.
[0031] In this embodiment, the drone can be controlled manually to reach the target track point D. t , or the UAV can be controlled to automatically reach the target track point D through automatic control commands t .
[0032] In one implementation of this embodiment, the drone reaches the target track point D t After that, under the condition of ensuring the safety of the UAV, the target track point D t Manually adjust the distance between the drone and the crack to make the drone closer to the crack, so that the onboard imaging device can capture a clearer regional crack image, so that the pixel size of the crack in the regional crack image is larger than the pixel size of the crack in the aqueduct video to be tested. For example, the target track point D t Directly above the crack, D can be lowered while ensuring the safety of the drone. t and then taking a crack image of the area.
[0033] In another implementation of this embodiment, the drone reaches the target track point D t Afterwards, the airborne imaging device may be optically zoomed in so that the pixel size of the crack in the regional crack image is larger than the pixel size of the crack in the video of the aqueduct to be tested.
[0034] In this embodiment, the crack image of the area with a larger crack pixel size is input into the aqueduct crack segmentation model, so that the crack features can be accurately extracted.
[0035] In this embodiment, a propeller protective cover can be designed and 3D printed according to the propeller size of the drone. When the drone performs automatic inspection and fixed-point shooting tasks, the propeller can be used to ensure the safe flight of the drone.
[0036] In this embodiment, the automatic inspection route of the drone can be saved in the built-in memory of the drone. During automatic inspection, the drone can fly according to the automatic inspection route without interacting with other external devices and without facing interference from interactive signals. The external devices can be remote controls, remote drone management platforms, etc., and no manual operation by the flight controller is required, thus avoiding dependence on external equipment and personnel. The drone performs inspection tasks according to the automatic inspection route, and can fly flexibly in the space inside or outside the aqueduct while ensuring the safety of the drone itself and the quality of shooting, thereby improving the inspection efficiency.
[0037] The aqueduct crack inspection method of the present embodiment adopts the automatic cruising technology of unmanned aerial vehicles to realize the inspection of aqueduct cracks, has a high degree of automation, and improves the efficiency of aqueduct crack inspection. On the basis of determining the crack development position through the aqueduct video, the cracks are photographed at fixed points to obtain high-quality crack images, which is helpful for the accurate observation of cracks on the aqueduct surface. On the basis of accurately extracting the crack characteristics, the non-contact extraction of the real crack information can be realized, and the crack information obtained is more accurate, which increases the understanding of the crack detailed information and provides an accurate data basis for the evaluation and repair of aqueduct cracks.
[0038] In an exemplary embodiment, before step S1000, the automatic inspection route is further determined, which may include steps S0100-S0200:
[0039] S0100: Determine a track point to be optimized in a test route according to a test video of the aqueduct to be tested, wherein the test video is acquired by the airborne imaging device when the drone flies along the test route;
[0040] S0200: Optimize the track points to be optimized to obtain the automatic inspection route.
[0041] In this embodiment, before executing step S0100, the number of inner inspections P and the number of outer inspections Q of the aqueduct to be tested can be determined according to the scale of the aqueduct to be tested, the field of view of the airborne imaging equipment and the flight time of the drone; wherein, after P+Q inspections, the acquired video of the aqueduct to be tested can cover the inner and outer surfaces of the aqueduct to be tested; test routes are pre-designed for each inner inspection and outer inspection, and inspection tests can be performed according to the test routes and test videos of the aqueduct to be tested can be acquired.
[0042] In one implementation of this embodiment, while the drone is flying along a test route, the test video can be viewed remotely in real time, the track points to be optimized where safety hazards exist during the flight of the drone can be determined in real time, and the test route can be adjusted in real time to ensure the safe flight of the drone.
[0043] In another implementation of this embodiment, after the UAV obtains the test video along the test route, the test video can be received and analyzed to obtain the track points to be optimized.
[0044] In this embodiment, the track points to be optimized include position points in the test route that are too close to the trough wall or rack, position points where the video is not clear, position points where the video shooting angle and range cannot cover the predetermined observation range, and position points close to other obstacles that affect the safety of the drone or the image shooting effect. Among them, other obstacles may include tree branches and fallen rocks near the aqueduct or close to the aqueduct wall after extreme weather or emergencies. The above-mentioned track points to be optimized are all exemplary descriptions and are not intended to be limitations of this application, and will not be described one by one.
[0045] When executing step S0200, the track points to be optimized on the test route can be fine-tuned and optimized, and track points can be added to the test route to obtain the best automatic inspection route; and the shooting parameters of the onboard imaging equipment and parameters such as the gimbal pitch angle can be optimized or reset according to the shooting effect of the test video.
[0046] In this embodiment, the obtained automatic inspection route realizes the precise planning of automatic cruising of the UAV. During the automatic inspection process of the UAV, it can completely rely on the automatic inspection route without relying on external navigation and positioning signals, thereby ensuring the flight safety of the UAV during the inspection process. It is more suitable for complex environments and ensures the efficient inspection of aqueduct cracks.
[0047] In an exemplary embodiment, step S2000 may include steps S2100-S2300:
[0048] S2100: extracting frames of the aqueduct video to be tested according to a predetermined frame extraction frequency to obtain a plurality of first images of the aqueduct to be tested, and saving the frame number and time point t of each first image in the aqueduct video to be tested;
[0049] S2200: Perform the following processing on each of the first images: input the first image into the aqueduct crack segmentation model to generate a corresponding first image crack prediction binary map; analyze the first image crack prediction binary map to determine whether it contains cracks, and if not, discard the first image; if contained, determine the crack development position based on the time point t of the first image and the automatic inspection route.
[0050] In this embodiment, when executing step S2100, an appropriate frame sampling frequency can be selected according to the flight speed of the drone and the video frame rate when shooting the video of the aqueduct to be tested. For example, if the video frame rate is 30fps and the flight speed is 5m / s, the frame sampling frequency is set to 30fps, which is equivalent to extracting a first image every 5 meters of flight distance. There may be overlapping parts between the first images of two adjacent frames to ensure that all first images can cover the surface of the aqueduct and no aqueduct surface information is missed. However, the overlapping part should not be too large. Too large an overlapping part will reduce the analysis efficiency of the aqueduct crack inspection.
[0051] In this embodiment, when executing step S2200, the first image is input into the aqueduct crack segmentation model to generate a corresponding first image crack prediction binary image. In the first image crack prediction binary image, the grayscale value of each pixel is 0 or 1, the grayscale value at the crack is 1, and the grayscale value at the non-crack is 0; when judging whether cracks are included, the grayscale values corresponding to all pixels in the first image crack prediction binary image are added to obtain the sum of the grayscale values. If the sum of the grayscale values is greater than the grayscale threshold, the first image is judged to contain cracks, otherwise the first image is judged to contain no cracks and is removed; when judging whether cracks are included, the crack pixel width can also be calculated according to the first image crack prediction binary image. When the crack pixel width is greater than the pixel width threshold, the first image is judged to contain cracks, otherwise the first image is judged to contain no cracks and is removed; when judging whether cracks are included, the sum of the grayscale values and the crack pixel width can also be used to jointly judge whether the first image contains cracks, and the weighted value of the sum of the grayscale values and the crack pixel width can be used as a parameter for judging whether cracks are included.
[0052] In this embodiment, by analyzing the changes in the grayscale values of the pixels in the first image crack prediction binary image obtained in step S2200, the distribution, shape and length of the cracks can also be obtained.
[0053] In this embodiment, when executing step S2200, the first image containing cracks is screened out, and the crack development position is determined based on the first image containing cracks. After this step, when executing subsequent steps S3000 and S4000, the acquisition and computing resources can be focused only on areas where cracks exist, excluding areas that do not contain cracks, thereby reducing unnecessary detection work and improving overall detection efficiency.
[0054] In an exemplary embodiment, the inspection route consists of n track points D i Connected, i = 1...n;
[0055] Among them, the UAV flight speed remains unchanged during the automatic inspection, the inspection route between any two adjacent track points in the automatic inspection route is a straight line, and the total flight time of the UAV corresponding to the inspection route is T, D1 is the starting point of the inspection route, D n is the end point of the inspection route, D i The coordinates of (x i ,y i ,h i ), x i Represents the longitude of the track point, y i Represents the latitude of the track point, h i Represents the altitude of the track point; each track point D i With the starting point D 1 Inspection distance S i Calculated according to the first formula, the first formula is:
[0056]
[0057] In this embodiment, the distance between any two adjacent track points is less than the preset track point distance threshold; and the altitude h of any two track points i If the difference is less than the preset threshold, the UAV can ensure the stability of the flight altitude and speed during automatic cruising, ensure the consistency of the viewing angle and coverage of the onboard imaging equipment, prevent video jitter caused by altitude changes, and obtain clearer and more stable images.
[0058] In this embodiment, the inspection route between any two adjacent track points in the automatic inspection route is a straight line, which can not only ensure the stable flight of the drone, but also ensure that the inspection distance S calculated according to the first formula is i The accuracy of the target track point D is t Laid the theoretical foundation.
[0059] In an exemplary embodiment, the step S2200 of “determining the crack development position according to the time point t and the inspection route of the first image” may include steps S2210-S2230:
[0060] S2210: Calculate the flight distance of the drone corresponding to the time t
[0061] S2220: The UAV is flown a distance S t and n inspection distances S i For comparison, we will compare with S t The two closest S i Recorded as S k-1 and S k , where 2≤k≤n;
[0062] S2230: At n track points Di Get S k-1 The corresponding track point D k-1 , and S k The corresponding track point D k , the track point D k-1 and D k The corresponding position of the aqueduct to be tested is taken as the crack development position, wherein the track point D k-1 The coordinates of (x k-1 ,y k-1 ,h k-1 ), the track point D k The coordinates of (x k ,y k ,h k ).
[0063] In this embodiment, the crack development position is the position of the crack in the automatic inspection route, and the track point D k-1 and D k The coordinates are the head and tail coordinates of the inspection section where the crack is located.
[0064] In an exemplary embodiment, in step S3000, "determine the target track point D according to the crack development position" t ” may include step S3100:
[0065] S3100: Calculate the target track point D based on the second formula t The coordinates (x t ,y t ,h t ), x t Representative D t Longitude, y t Representative D t Latitude, h t Representative D t Altitude above sea level;
[0066] The second formula is
[0067] In this embodiment, the second formula is a distance interpolation formula, and the target track point D is obtained by the second formula. t The coordinates of the target track point D are obtained t After that, the flight attitude of the gimbal and the drone can be adjusted automatically or manually to control the drone to fly to the target track point D. t , all-round fixed-point shooting is carried out on the cracks to obtain a regional crack image. The crack pixel size of the regional crack image is larger than the crack pixel size of the first image. Fixed-point shooting can obtain more stable and clear images, and can capture more subtle crack features, reducing the interference of drone movement on shooting.
[0068] In this embodiment, since the cracks on the surface of the aqueduct are usually several millimeters wide and the crack width is usually less than one centimeter, a high-precision regional crack image is required. When acquiring the video of the aqueduct to be tested and acquiring the regional crack image, the light for fixed-point shooting can also be controlled to capture the regional crack image under the best lighting effect. It is also possible to capture the regional crack image at each target track point D t Multiple regional crack images are taken at fixed points, and the regional crack image with the best quality is selected after comparison.
[0069] In an exemplary embodiment, the step S4000 of "analyzing the regional crack binary image using a skeletonized medial axis algorithm and an object distance method to obtain crack directions and crack widths" may include steps S4100-S4300:
[0070] S4100: Processing the regional crack binary image using the skeletonized medial axis algorithm to obtain a skeleton image, wherein the skeleton image includes a plurality of crack skeleton pixel points with a single pixel width, and the extension direction of the crack skeleton pixel points represents the direction of the crack;
[0071] S4200: Overlap the skeleton image with the regional crack binary image, and obtain the maximum pixel width W of each crack indicated in the skeleton image in the regional crack binary image. Pixel ;
[0072] S4300: According to the maximum pixel width W Pixel The maximum actual width W of each crack is obtained by the object distance method. Actual .
[0073] In this embodiment, before executing step S4100, the regional crack image is input into the aqueduct crack segmentation model to obtain a regional crack binary map. Compared with the first image crack prediction binary map in step S2200, the regional crack binary map can obtain more detailed crack details to meet the needs of further detailed analysis of the cracks.
[0074] In this embodiment, the skeletonization medial axis algorithm of step S4100 can adopt any one of the methods based on iterative shrinkage, Hilditch algorithm, Zhang-Suen algorithm, morphology-based skeletonization method, distance transformation-based method, Voronoi diagram-based method, and discrete curve evolution-based method. The above-mentioned skeletonization medial axis algorithms are all exemplary descriptions and are not intended to be limitations of this application. This application will not describe them one by one.
[0075] In this embodiment, the skeleton diagram can clearly show the crack direction, such as Figure 2 The regional crack binary image and skeleton image are shown. Figure 2The skeleton image on the right is obtained by processing the regional crack binary image on the left using the skeletonized medial axis algorithm. The skeleton image can show the shape characteristics, development morphology and direction of the cracks, retain the shape information of the cracks, locate the position of the crack centerline, and is the basis for obtaining the crack width information.
[0076] In an exemplary embodiment, in step S4200, “obtaining the maximum pixel width W of each crack indicated in the skeleton image in the regional crack binary image” Pixel ", each of the cracks may be processed respectively including steps S4210-S4220:
[0077] S4210: For each crack skeleton pixel point of the crack, in a direction perpendicular to the crack direction, the grayscale values in the regional crack binary image are accumulated to obtain a grayscale accumulation value ∑Gdl corresponding to each crack skeleton pixel point; wherein the pixel points in the regional crack binary image include crack pixel points and non-crack pixel points, G represents the grayscale value, G=1 at the crack pixel point, G=0 at the non-crack pixel point, and dl represents the pixel width in the direction perpendicular to the crack direction in the regional crack binary image;
[0078] S4220: Obtain the maximum value max(∑Gdl) in the grayscale accumulated value ∑Gdl as the maximum pixel width W of the crack Pixel .
[0079] In this embodiment, before executing step S4210, the skeleton image needs to be overlapped with the regional crack binary image. When executing step S4210, the crack pixel width is obtained in a direction perpendicular to the crack direction, avoiding errors caused by angle deviation. The obtained crack pixel width is more accurate, and the detection direction of the crack width is unified, which facilitates comparative analysis between different cracks.
[0080] In an exemplary embodiment, step S4300 may perform processing including steps S4310-S4320 on each of the cracks:
[0081] S4310: Obtaining the zoom ratio of the onboard imaging device Wherein, u represents the distance between the lens of the airborne imaging device and the crack, f represents the focal length of the lens, p represents the ratio of the lateral resolution to the longitudinal resolution of the airborne imaging device, and d represents the ratio of the lateral size to the longitudinal size of the sensor of the airborne imaging device;
[0082] S4320: According to formula W Actual = k*W Pixel Calculate the maximum actual width W of the crack Actual .
[0083] In this embodiment, the scaling ratio k is a parameter used by the airborne imaging device to obtain regional crack images. By combining the skeletonized medial axis algorithm and the object distance method, the maximum pixel width of the crack is converted into the maximum actual width, thereby realizing non-contact extraction of the actual width of the crack.
[0084] In order to illustrate the aqueduct crack inspection method of the embodiment of the present application, a specific example is used to describe it in detail below.
[0085] S1: Determine the test route according to the scale of the aqueduct to be tested, the field of view of the onboard influencing equipment, and the flight time of the drone. Manually control the drone to conduct inspection tests on the aqueduct to be tested according to the test route, obtain the test video, adjust and optimize the test route according to the test video, and obtain the best automatic inspection route;
[0086] S2: Control the drone to perform automatic inspection of the aqueduct to be tested according to the automatic inspection route obtained in step S1, and obtain a video of the aqueduct to be tested;
[0087] S3: extract frames from the video of the aqueduct to be tested to obtain a first image, and save the frame number and time point t corresponding to the first image;
[0088] S4: inputting each first image into the aqueduct crack segmentation model to generate a corresponding first image crack prediction binary image;
[0089] S5: analyzing the crack prediction binary image of the first image to determine whether cracks are included, and if cracks are not included, discarding the first image;
[0090] S6: Determine the crack development position based on the retained first image and set the track point D k-1 and D k The position of the corresponding aqueduct to be tested is used as the crack development position;
[0091] S7: According to track point D k-1 and D k Determine the target track point D t ;
[0092] S8: Control the drone to fly to the target track point D t , enlarge the optical zoom of the onboard imaging equipment, take fixed-point photos, and obtain regional crack images;
[0093] S9: inputting the regional crack image into the aqueduct crack segmentation model to obtain a regional crack binary image;
[0094] S10: converting the regional crack binary image into a skeleton image through a skeletonized medial axis algorithm;
[0095] S11: Overlap the skeleton image with its corresponding regional crack binary image, and obtain the maximum pixel width W of the crack in the direction perpendicular to the crack direction. Pixel ;
[0096] S12: Obtain the scaling ratio k of the airborne imaging device, according to the formula W Actual = k*W Pixel Calculate the maximum actual width W of the crack Actual .
[0097] The present application also provides a device for inspecting cracks in an aqueduct. Figure 3 As shown, including a processor and a memory,
[0098] The memory is used to store the aqueduct crack inspection program;
[0099] The processor is used to read the aqueduct crack inspection program and perform the aqueduct crack inspection method as described in the above embodiment.
[0100] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
Claims
1. A method for inspecting cracks in an aqueduct, characterized in that: include: Acquire a video of the aqueduct to be tested, wherein the video of the aqueduct to be tested is taken by an airborne imaging device when the drone performs automatic inspection along an automatic inspection route; Inputting the aqueduct video to be tested into a pre-trained aqueduct crack segmentation model to determine the crack development position; Determine the target track point D according to the crack development position t , send a second inspection command to the UAV to obtain the position of the airborne imaging device at the target track point D t A regional crack image captured at a fixed point; wherein, for each crack, the pixel size of the crack in the regional crack image is larger than the pixel size of the crack in the video of the aqueduct to be tested; The regional crack image is input into the aqueduct crack segmentation model to obtain a regional crack binary map, and the regional crack binary map is analyzed using a skeletonized medial axis algorithm and an object distance method to obtain crack directions and crack widths.
2. The aqueduct crack inspection method according to claim 1, characterized in that: Before obtaining the video of the aqueduct to be tested, the automatic inspection route is determined, including: Determining the track points to be optimized in the test route according to the test video of the aqueduct to be tested, wherein the test video is acquired by the airborne imaging device when the UAV flies along the test route; The track points to be optimized are optimized to obtain the automatic inspection route.
3. The aqueduct crack inspection method according to claim 1, characterized in that: The step of inputting the video of the aqueduct to be tested into a pre-trained aqueduct crack segmentation model to determine the crack development position comprises: Extract frames of the aqueduct video to be tested according to a predetermined frame extraction frequency to obtain a plurality of first images of the aqueduct to be tested, and save the frame number and time point t of each first image in the aqueduct video to be tested; The following processing is performed on each of the first images: the first image is input into the aqueduct crack segmentation model to generate a corresponding first image crack prediction binary map; based on the first image crack prediction binary map, analysis is performed to determine whether cracks are contained; if not, the first image is discarded; if contained, the crack development position is determined according to the time point t of the first image and the automatic inspection route.
4. The aqueduct crack inspection method according to claim 3, characterized in that: The automatic inspection route consists of n track points D i Connected, i = 1...n; The UAV flight speed remains unchanged during the automatic inspection, the inspection route between any two adjacent track points in the automatic inspection route is a straight line, the total flight time of the UAV corresponding to the automatic inspection route is T, D1 is the starting point of the automatic inspection route, D n is the end point of the automatic inspection route, D i The coordinates of (x i ,y i ,h i ), x i Represents the longitude of the track point, y i Represents the latitude of the track point, h i Represents the altitude of the track point; each track point D i The inspection distance S between the starting point D1 i Calculated according to the first formula, the first formula is:
5. The aqueduct crack inspection method according to claim 4, characterized in that: The step of determining the crack development position according to the time point t of the first image and the automatic inspection route includes: Calculate the flight distance of the drone corresponding to the time t The UAV is flown a distance S t and n inspection distances S i For comparison, we will compare with S t The two closest S i Recorded as S k-1 and S k , where 2≤k≤n; At n track points D i Get S k-1 The corresponding track point D k-1 , and S k The corresponding track point D k , the track point D k-1 and D k The corresponding position of the aqueduct to be tested is taken as the crack development position, wherein the track point D k-1 The coordinates of (x k-1 ,y k-1 ,h k-1 ), the track point D k The coordinates of (x k ,y k ,h k ).
6. The aqueduct crack inspection method according to claim 5, characterized in that: Determining the target track point D according to the crack development position t ,include: Calculate the target track point D based on the second formula t The coordinates (x t ,y t ,h t ), x t Representative D t Longitude, y t Representative D t Latitude, h t Representative D t Altitude above sea level; The second formula is 7. The aqueduct crack inspection method according to claim 1, characterized in that: The method of using the skeletonized median axis algorithm and the object distance method to analyze the regional crack binary map to obtain the crack direction and crack width includes: The regional crack binary image is processed by the skeletonized medial axis algorithm to obtain a skeleton image, wherein the skeleton image includes a plurality of crack skeleton pixel points with a single pixel width, and the extension direction of the crack skeleton pixel points represents the direction of the crack; The skeleton image is overlapped with the regional crack binary image, and the maximum pixel width W of each crack indicated in the skeleton image in the regional crack binary image is obtained. Pixel ; According to the maximum pixel width W Pixel The maximum actual width W of each crack is obtained by the object distance method. Actual .
8. The aqueduct crack inspection method according to claim 7, characterized in that: The step of obtaining the maximum pixel width W of each crack indicated in the skeleton image in the regional crack binary image Pixel ,include: The following treatments are performed on each of the cracks: For each crack skeleton pixel point of the crack, the grayscale values in the regional crack binary image are accumulated in a direction perpendicular to the crack direction to obtain a grayscale accumulation value ∑Gdl corresponding to each crack skeleton pixel point; wherein the pixel points in the regional crack binary image include crack pixel points and non-crack pixel points, G represents the grayscale value, G=1 at the crack pixel point, G=0 at the non-crack pixel point, and dl represents the pixel width in the regional crack binary image in a direction perpendicular to the crack direction; The maximum value max(∑Gdl) in the grayscale accumulated value ∑Gdl is obtained as the maximum pixel width W of the crack. Pixel .
9. The aqueduct crack inspection method according to claim 7, characterized in that:According to the maximum pixel width W Pixel The maximum actual width W of each crack is obtained by the object distance method. Actual ,include: The following treatments are performed on each of the cracks: Get the zoom ratio of the onboard imaging device Wherein, u represents the distance between the lens of the airborne imaging device and the crack, f represents the focal length of the lens, p represents the ratio of the lateral resolution to the longitudinal resolution of the airborne imaging device, and d represents the ratio of the lateral size to the longitudinal size of the sensor of the airborne imaging device; According to the formula W Actual =k*W Pixel Calculate the maximum actual width W of the crack Actual .
10. An aqueduct crack inspection device, comprising a processor and a memory, characterized in that: The memory is used to store the aqueduct crack inspection program; The processor is used to read the aqueduct crack inspection program and perform the aqueduct crack inspection method as described in any one of claims 1-9.
