Three-dimensional measurement method, device and equipment for multiple diseases on surface of structure and medium

Abstract

The invention provides a structure surface multi-disease three-dimensional measurement method and device, equipment and a medium. The method comprises the steps that an image sequence carrying GPS coordinate information is collected; performing depth calculation on the image sequence by using a three-dimensional reconstruction technology, and performing three-dimensional reconstruction on a scene to generate an initial three-dimensional model; carrying out disease determination and point cloud processing to obtain a three-dimensional point cloud model of a specified color and each disease point cloud instance; using a skeleton extraction method and a boundary point extraction algorithm to extract a boundary point set and a skeleton point set of the skeleton point set corresponding to the crack point cloud, and according to each obtained skeleton point set and the corresponding boundary point set, performing quantitative calculation on the length and the width of each crack; generating a spalling surface grid model by using a Poisson surface reconstruction algorithm and a hole filling algorithm, and calculating and determining the volume of the three-dimensional spalling instance according to the spalling surface grid model; and determining the normal direction of each point of the corrosion cloud instance, generating a grid model by using a Poisson surface reconstruction algorithm, and calculating the corrosion area.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method, device, equipment and medium for three-dimensional measurement of multiple defects on a structure surface. Background Art

[0002] During the service life of an engineering structure, cracks will appear under the coupling of multiple factors such as long-term loads and natural disasters. For example, after a concrete bridge is used for a long time, the steel bars will be exposed to typical surface diseases such as rust and peeling, which will make the structure enter a sub-healthy state. If these diseases are not detected and repaired in a timely and reasonable manner, they are likely to accumulate continuously, seriously threatening the safe use of the engineering structure. Therefore, in order to ensure the safe operation of the engineering structure, it is necessary to effectively detect and evaluate the surface diseases.

[0003] In recent years, the popularity of consumer drones and the development of computer vision technology have promoted the process of intelligent disease detection and improved detection efficiency. Drones are used to take photos of areas that are difficult to reach by manual inspection, and digital image processing technologies such as machine learning are used to extract disease information. The following problems exist in related technologies:

[0004] 1) Surface defect detection of structures is mainly based on manual observation, and the detection results are highly subjective, the work surface is large, and the efficiency is low; the type, location, and size of the defect need to be manually written, supplemented by drawing and taking pictures to record in the inspection report, which makes it difficult to fully record the information of all surface defects of the structure, and lacks effective means of visualizing the location and size of the defect; the physical size of the defect is measured by traditional tools such as rulers and electronic micrometers, and the size information that is difficult to measure is simply estimated. Therefore, the traditional manual method of measuring the apparent defects of structures can no longer meet the rapidly rising detection needs today.

[0005] 2) UAV images combined with 3D reconstruction technology can generate 3D point clouds of engineering structures and provide a basic 3D model for 3D detection. However, the resolution of UAV images, aerial photography paths and parameters that meet the accuracy of local disease detection and positioning have not been fully considered.

[0006] 3) Due to the limitation of viewing angle and the influence of image distortion, the defect recognition and measurement methods of two-dimensional images cannot directly provide the location information of local defects on the overall structure. At the same time, the conditions for accurate measurement are very harsh, and it is difficult to ensure accuracy in practical applications. The three-dimensional detection method that directly uses local point cloud features for defect recognition is sensitive to noise and requires a lot of manual intervention; at the same time, the calculation cost of the deep learning model based on point cloud data is too high.

[0007] 4) The existing surface disease measurement methods only focus on simple morphological diseases in laboratory environments, and lack consideration of typical complex surface diseases in actual engineering environments, resulting in limited practical value of many similar measurement technologies.

[0008] Therefore, there is an urgent need to provide a method that can realize effective automated measurement of complex surface defects in real engineering projects. Summary of the invention

[0009] In order to overcome the problems existing in the related art, the present disclosure provides a three-dimensional measurement method, device, equipment and medium for multiple defects on the surface of a structure to solve the related technical problems in the related art.

[0010] One or more embodiments of this specification provide a method for three-dimensionally measuring multiple defects on a structure surface, comprising the following steps:

[0011] Image sequence acquisition: Use drones to collect image sequences with GPS coordinate information;

[0012] Initial 3D model construction: Use 3D reconstruction technology to calculate the depth of the image sequence and perform 3D reconstruction of the scene to generate an initial 3D model;

[0013] Defect determination and point cloud processing: The semantic segmentation network is used to identify the image sequence to obtain the semantic map of each defect, and the pixel value of the defect in each semantic map is converted into the color of the specified RGB attribute value; the depth map is fused with the processed semantic map of each defect through the depth map fusion algorithm to obtain a three-dimensional point cloud model of the specified color; then based on the RGB color range, the point cloud data of the three-dimensional point cloud model is extracted through the point cloud extraction method, and the points are divided into several independent point cloud clusters through the clustering algorithm to obtain each defect point cloud instance; the structural surface points around each defect point cloud are used as background points, and the clustering algorithm is used to fit the background plane equation of each defect, where the defects include cracks, rust and / or peeling;

[0014] Crack width calculation: The skeleton point set of each plane crack point cloud is extracted using the skeleton extraction method, and the boundary point set of the crack point cloud corresponding to the skeleton point set is extracted using the boundary point extraction algorithm. The width of each crack is quantitatively calculated based on the obtained skeleton point sets and the corresponding boundary point sets.

[0015] Calculation of the volume of the peeling point: flatten the peeling point cloud instance corresponding to the background plane equation of each peeling point to obtain the peeling plane point, determine that the normal directions of the peeling point cloud instance and each point of the peeling plane point are kept facing each other, then merge the peeling point cloud instance and the peeling plane point to obtain a three-dimensional peeling instance, adjust the normal direction of the concave side of the peeling point cloud instance to be facing the flat peeling normal direction, then use the Poisson surface reconstruction algorithm and the hole filling algorithm to generate a peeling surface mesh model, and calculate and determine the volume of the three-dimensional peeling instance based on the peeling surface mesh model; and

[0016] Calculation of rust area: Calculate the normal direction of each point of the rust cloud instance, and use the Poisson surface reconstruction algorithm to generate a mesh model and calculate the rust area.

[0017] One or more embodiments of this specification provide a three-dimensional measurement device for multiple defects on a structure surface, including:

[0018] An image sequence acquisition module is used to use a drone to acquire image sequences carrying GPS coordinate information;

[0019] An initial model generation module is used to calculate the depth of the image sequence using a 3D reconstruction technique and to perform 3D reconstruction of the scene to generate an initial 3D model;

[0020] The defect determination and point cloud processing module is used to obtain semantic maps of each defect by recognizing the image sequence through the semantic segmentation network, and convert the defect pixel value in each defect semantic map into the color of the specified RGB attribute value; the depth map is fused with the processed semantic map of each defect by the depth map fusion algorithm to obtain a three-dimensional point cloud model of the specified color; then based on the RGB color range, the point cloud data of the three-dimensional point cloud model is extracted by the point cloud extraction method, and the points are divided into several independent point cloud clusters by the clustering algorithm to obtain each defect point cloud instance; the structural surface points around each defect point cloud are used as background points, and the clustering algorithm is used to fit the background plane equation of each defect, wherein the defects include cracks, rust and / or peeling;

[0021] The crack width calculation module uses the skeleton extraction method to extract the skeleton point sets of each plane crack point cloud, and uses the boundary point extraction algorithm to extract the boundary point sets of the crack point cloud corresponding to the skeleton point set. According to the obtained skeleton point sets and the corresponding boundary point sets, the width of each crack is quantitatively calculated;

[0022] The peeling point volume calculation module is used to obtain peeling plane points by flattening the peeling point cloud instances corresponding to the background plane equations of each peeling point, determine that the normal directions of the peeling point cloud instances and each point of the peeling plane points are kept facing each other, then merge the peeling point cloud instances and the peeling plane points to obtain a three-dimensional peeling instance, adjust the normal direction of the concave side of the peeling point cloud instance to be facing the flat peeling normal direction, then use the Poisson surface reconstruction algorithm and the hole filling algorithm to generate a peeling surface mesh model, and calculate and determine the volume of the three-dimensional peeling instance based on the peeling surface mesh model;

[0023] The corrosion area calculation module is used to calculate the normal direction of each point of the corrosion cloud instance, and use the Poisson surface reconstruction algorithm to generate a mesh model and calculate the corrosion area.

[0024] One or more embodiments of the present specification provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the three-dimensional measurement method for multiple defects on a structure surface as described above when executing the computer program.

[0025] One or more embodiments of the present specification provide a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the three-dimensional measurement method for multiple defects on a structure surface as described above.

[0026] The present disclosure provides a method, device, equipment and medium for three-dimensional measurement of multiple defects on a structure surface. The advantages are that, based on the image sequence taken by a drone, a realistic surface three-dimensional model is reconstructed through three-dimensional reconstruction technology, the defect area of ​​interest is automatically and accurately marked, and mapped to the three-dimensional model, and then the identified local image defects are located at the corresponding position of the spatial three-dimensional point cloud model through deep learning semantic segmentation, so as to achieve global and accurate detection and positioning of the defects on the structure. A point cloud clustering algorithm is used to identify and extract individual defect instances from a point cloud containing multiple defects of the same type; and a more three-dimensional defect point cloud is quickly generated through the idea of ​​point cloud merging, which helps the surface reconstruction algorithm to better generate dense triangular facets. For peeling defects, a Poisson surface reconstruction algorithm is used to generate triangular facet surfaces for rust and peeling defects, respectively, which is convenient for measuring area and volume. The triangular surface model of peeling defects is processed using a hole filling algorithm to generate a completely closed mesh model, thereby improving the accuracy of volume measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 A flow chart of a method for three-dimensional measurement of multiple defects on a structure surface provided for one or more embodiments of this specification;

[0029] Figure 2 A schematic diagram of an image collected by a drone provided in one or more embodiments of this specification;

[0030] Figure 3 A schematic diagram of three-dimensional disease detection and positioning provided in one or more embodiments of this specification;

[0031] Figure 4 A schematic diagram of the three-dimensional global positioning of multiple diseases of bridge piers provided in one or more embodiments of this specification;

[0032] Figure 5 A schematic diagram of a multi-disease semantic point cloud extracted for one or more embodiments of this specification;

[0033] Figure 6 A schematic diagram of an example of identifying spalling disease using a clustering algorithm provided in one or more embodiments of this specification;

[0034] Figure 7 A schematic diagram of complex mesh crack skeleton points and boundary points generated for one or more embodiments of this specification;

[0035] Figure 8 A schematic diagram of an automatic calculation algorithm for three-dimensional crack width based on skeleton point guidance provided in one or more embodiments of this specification;

[0036] Fig. 9 A flow chart of the spalling damage mesh reconstruction process provided for one or more embodiments of this specification;

[0037] Fig.10 A block diagram of a three-dimensional measurement device for multiple defects on a structure surface provided for one or more embodiments of this specification;

[0038] as well as

[0039] Fig.11 A schematic diagram of the structure of a computer device provided for one or more embodiments of this specification. DETAILED DESCRIPTION

[0040] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0041] The present invention is described in detail below in conjunction with specific implementation methods and the accompanying drawings.

[0042] Method Embodiment

[0043] According to an embodiment of the present invention, a method for three-dimensional measurement of multiple defects on a structure surface is provided. Figure 1 As shown, it is a flow chart of the method for three-dimensional measurement of multiple defects on the surface of a structure provided in this embodiment. According to the method for three-dimensional measurement of multiple defects on the surface of a structure provided in this embodiment, the steps include:

[0044] Step S1, image sequence acquisition;

[0045] Use drones to collect image sequences with GPS coordinate information.

[0046] Step S2, initial three-dimensional model construction;

[0047] The 3D reconstruction technology is used to calculate the depth of the image sequence and perform 3D reconstruction of the scene to generate an initial 3D model.

[0048] Step S3, disease determination and point cloud processing;

[0049] The semantic segmentation network is used to identify the image sequence to obtain the semantic map of each defect, and the pixel value of the defect in each semantic map is converted into the color of the specified RGB attribute value; the depth map is fused with the processed semantic map of each defect through the depth map fusion algorithm to obtain a three-dimensional point cloud model of the specified color; based on the RGB color range, the point cloud data of the three-dimensional point cloud model is extracted through the point cloud extraction method, and the points are divided into several independent point cloud clusters through the clustering algorithm to obtain each defect point cloud instance; wherein the defects include cracks, rust and / or peeling; the structural surface points around each defect point cloud are used as background points, and the clustering algorithm is used to fit the background plane equation of each defect;

[0050] Step S4, crack width calculation;

[0051] The skeleton point sets of the crack point cloud are extracted using the skeleton extraction method, and the boundary point sets of the crack point cloud corresponding to the skeleton point set are extracted using the boundary point extraction algorithm. The width of each crack is quantitatively calculated based on the obtained skeleton point sets and the corresponding boundary point sets.

[0052] Step S5, calculation of the peeling point volume;

[0053] According to the flattening of the background plane equation of each peeling point, the corresponding peeling point cloud instance is obtained to obtain the peeling plane point, and the normal directions of the peeling point cloud instance and each point of the peeling plane point are determined to be opposite. Then the peeling point cloud instance and the peeling plane point are merged to obtain a three-dimensional peeling instance, and the normal direction of the concave side of the peeling point cloud instance is adjusted to be opposite to the flat peeling normal direction to achieve the distinction between the inside and outside of the peeling surface; then the Poisson surface reconstruction algorithm and the hole filling algorithm are used to generate the peeling surface mesh model, and the volume of the three-dimensional peeling instance is calculated and determined according to the peeling surface mesh model.

[0054] Step S6, calculation of corrosion area;

[0055] The normal direction of each point of the rust cloud instance is calculated, and the Poisson surface reconstruction algorithm is used to generate a mesh model and calculate the rust area.

[0056] The three-dimensional measurement method for multiple defects on the surface of a structure provided in this embodiment discloses an image sequence taken by a drone, reconstructs a realistic three-dimensional surface model through three-dimensional reconstruction technology, automatically and accurately marks the defect area of ​​interest, and maps it to the three-dimensional model, and then locates the identified local image defects to the corresponding position of the spatial three-dimensional point cloud model through deep learning semantic segmentation, so as to achieve global and accurate detection and positioning of the defects on the structure. A point cloud clustering algorithm is used to identify and extract individual defect instances from a point cloud containing multiple defects of the same type; and a more three-dimensional defect point cloud is quickly generated through the idea of ​​point cloud merging, which helps the surface reconstruction algorithm to better generate dense triangular facets. For peeling defects, a Poisson surface reconstruction algorithm is used to generate triangular surfaces for rust and peeling defects, respectively, which is convenient for measuring area and volume, and a hole filling algorithm is used to process the peeling defect triangular surface model to generate a completely closed mesh model, thereby improving the accuracy of volume measurement.

[0057] In this embodiment, in order to realize the establishment of a three-dimensional model of an engineering structure without scale marking, a drone is used to collect an image sequence carrying GPS coordinate information. The specific collection process is as follows:

[0058] Image data acquisition is to use the camera to capture a set of image sequences of the target structure from top to bottom or from bottom to top, ensuring that the image overlap rate between two adjacent shots is not less than 50%. Figure 2As shown, for example, a DJI Phantom 4RTK drone is used to fly around the structure in an aerial photography manner, taking pictures from top to bottom. The drone is within 5m of the structure surface, and a total of 412 high-definition images with a resolution of 4864×3864 are obtained. The GPS coordinates of the camera center when the drone camera sensor takes each image are stored in the gps_data.txt text file in the form of {image name, latitude, longitude, altitude}.

[0059] In step S2 of this embodiment, the depth of the image sequence is calculated using the 3D reconstruction technology, and the 3D reconstruction of the scene is performed to generate an initial 3D model, which specifically may include the following steps:

[0060] Each image is scaled, and sparse reconstruction and initial camera parameter calculation are performed on the scaled image using 3D reconstruction technology, and depth calculation is performed on the scaled image to generate absolute camera parameters, and the absolute camera parameters are used to replace the initial camera parameters and generate a 3D model of absolute scale. This embodiment uses 3D reconstruction technology (such as COLMAP motion recovery structure) to calculate relative camera parameters and sparse models to provide initial parameters for GPS correction scale. The focus is on the alignment of the model scale with the real physical space scale, and the establishment of a realistic surface 3D model that can reconstruct the real physical size without scale marking is achieved.

[0061] This embodiment has multiple views Figure 3 The dimensional reconstruction algorithm can be COLMAP, OpenMVS, OpenMVG, VisualSFM, MVE, PhotoScan, PMVS or DJI Zhitu software.

[0062] In a specific embodiment, using a 3D reconstruction technique to calculate the depth of an image sequence and to perform 3D reconstruction of a scene to generate an initial 3D model specifically includes the following steps:

[0063] S21, sparse point cloud generation and initial camera parameter calculation;

[0064] The image is scaled and sparsely reconstructed using the Structure from Motion (SfM) algorithm to calculate the relative camera pose as the initial camera parameters and generate a sparse scene model.

[0065] S22, depth information calculation;

[0066] Based on the sparse scene model generated in step S21, a multi-view stereo (MVS) algorithm is used to calculate a depth map for each image, and an initial dense surface model is generated at the same time.

[0067] S23, scale recovery;

[0068] According to the GPS coordinate information (λ, φ, h) of the image, the coordinates are converted into absolute camera parameters, the absolute camera parameters replace the initial camera parameters, and the 3D similarity matrix is ​​solved using the rigid transformation principle. The coordinate conversion process is shown in formula (1):

[0069]

[0070] Among them, λ represents longitude, φ represents latitude, h represents altitude, α represents the major axis of the earth, e represents the first eccentricity of the earth, and T abs is the translation parameter, R abs is the rotation parameter. Since the rotation can be ignored in the scale recovery process, T is set abs =R rel .

[0071] S24, generating a sparse model of real scale;

[0072] The 3D similarity matrix of step S23 is dot-multiplied with the sparse scene model of step S21, that is, P real-size =M similarity ·P init , to realize the initial point cloud P init The point cloud coordinates in are corrected to P real-size , thus generating a true-scale sparse model.

[0073] S25, generating a real-scale dense surface three-dimensional model;

[0074] Based on the sparse model of step S24, a fusion calculation is performed in combination with the depth map calculated in step S22 and the image after scale scaling in step S21 to generate a realistic three-dimensional model of real-scale structure.

[0075] This embodiment uses a drone to collect images with GPS longitude and latitude coordinates in a circular flight mode, extracts the GPS coordinate information corresponding to the center of each image, and restores the scale. The depth of the scaled drone image is calculated using three-dimensional reconstruction technology, and the image pixel information is integrated to generate an initial three-dimensional model. The GPS coordinate information is used to provide a scale reference, and the camera parameters are recalculated using a three-dimensional reconstruction method to replace the original camera parameters and generate a three-dimensional model with an absolute scale, so as to realize the scale correction of the three-dimensional structure model using the longitude and latitude coordinates of the drone image. This method uses spatiotemporal continuity and point cloud representation to simultaneously estimate the camera displacement and the three-dimensional structure of the scene without pre-calculating the camera position. This method is more robust in terms of posture estimation and perspective generation, and the generated image quality is higher.

[0076] In this embodiment, step S3 trains a neural network through an image sample set including a single-channel grayscale label image of various disease semantics to obtain a disease semantic segmentation network, and uses the disease semantic segmentation network to perform disease recognition on each image in the image sequence to obtain each disease semantic map, and converts the disease pixel value in the corresponding image into a color of a specified RGB attribute value; then, each disease semantic map is fused with the initial three-dimensional model to obtain a three-dimensional point cloud model with disease semantics.

[0077] In a specific embodiment, the following steps are included:

[0078] S310, multi-disease dataset production;

[0079] Use software such as LabelMe to annotate the single-channel grayscale label images of three typical surface diseases, namely cracks, spalling and rust, in the scaled image in step S21. For example, grayscale values ​​1, 2, and 3 in the label image represent cracks, spalling and rust, respectively, thereby obtaining a data set with disease labels.

[0080] S311, semantic segmentation of 2D image defects;

[0081] The semantic segmentation network is trained using the data set until convergence to obtain a disease semantic segmentation network. The trained disease semantic segmentation network is then used to identify the disease of each image in the image sequence and mark them with corresponding colors, thereby obtaining a crack semantic map, a rust semantic map and / or a spalling semantic map.

[0082] In this embodiment, the semantic segmentation network can be the existing SegFormer, Crackdeep, U-Net, DenseNet, FasterR-CNN, CrackFormer, IBR-Former, CrackSeg9k network. The training and optimization of the network can be achieved through existing technologies and will not be elaborated here. In this embodiment, the colors of the three diseases of cracks, spalling and rust are respectively marked as (128, 0, 0), (255, 195, 0), (192, 0, 192), for example, to facilitate distinction.

[0083] In this embodiment, step S3 point cloud processing specifically includes the following steps:

[0084] S320, generating semantic models and locating multiple types of diseases;

[0085] Each disease semantic image in step S311 replaces the corresponding scaled original image in step S21, and performs fusion calculation again to generate a three-dimensional point cloud model with disease semantics, and the corresponding disease color mark in the image is mapped to the three-dimensional space, realizing the positioning of the disease on the three-dimensional model, which can be shown as follows: Figure 3The information of each point in the three-dimensional model is represented by six-dimensional data (x, y, z, r, g, b), where (x, y, z) represents the three-dimensional spatial coordinates of the point, and r, g, b represents the RGB color attribute value of the point.

[0086] In a specific embodiment, each disease semantic image in step S311 replaces the corresponding scaled original image in step S21, performs fusion calculation, and directly generates a semantic model. The disease color mark in the image is mapped to the three-dimensional space, and the three-dimensional positioning of the disease is realized. Figure 4 As shown in FIG. 1 , a schematic diagram of the three-dimensional global positioning of multiple diseases of bridge piers provided in this embodiment is shown. The cracks, spalling and rust diseases in the point cloud are displayed in the colors of the semantic segmentation image, such as Figure 5 , which is a schematic diagram of the extracted multi-disease semantic point cloud provided in this embodiment.

[0087] S321, extraction of disease point cloud;

[0088] According to the color attribute values ​​corresponding to each disease point cloud, the crack point cloud model, the peeling point cloud model and the rust point cloud model in the three-dimensional point cloud model are extracted.

[0089] Specifically, whether a point in the point cloud is a crack point can be determined by judging whether the color of each point falls within the specified RGB color interval. The extracted subset of crack points is recorded as P′, which can be expressed as: P′={p∈P|R1≤R(p)≤R2,G1≤G(p)≤G2,B1≤B(p)≤B2}, where P is a three-dimensional crack semantic point cloud, R(p), G(p) and B(p) represent functions that return the red, green and blue values ​​of point p, respectively; (R1, R2), (G1, G2) and (B1, B2) are specific interval thresholds for the red, green and blue channels, respectively.

[0090] S322, noise point filtering;

[0091] In order to remove noise points in the extracted crack point cloud data and improve the quality of each point cloud, the point cloud data is preprocessed to remove noise points in the crack point cloud data; specifically, a statistical filtering algorithm is used to filter out outliers / abnormal points in the different disease point clouds output in step S321, and filtered crack, spalling and rust point cloud models are obtained respectively, wherein the statistical filtering algorithm can be radius filtering, pass filtering, bilateral filtering and voxel filtering methods.

[0092] In a specific embodiment, noise point filtering uses a statistical filtering algorithm to filter out outliers / abnormal points in each defect point cloud outputted in step S321, wherein the specific technical parameters of the crack defect point cloud filtering algorithm are: 20 adjacent points, and a standard deviation ratio of 0.2; the specific technical parameters of the statistical filtering for filtering outliers / abnormal points of block surface defects such as rust and peeling are: 20 adjacent points, and a standard deviation ratio of 0.2.

[0093] S323, background plane detection;

[0094] The structural surface points around the defects are manually cropped from each defect point cloud after denoising in step S322 as background points, and the RANSAC algorithm is used to fit the background plane P0=ax+by+cz+d=0.

[0095] In a specific embodiment, the parameters of plane P0 are: a=0.837, b=0.492, c=0.241, d=0.832.

[0096] S324, extract disease instances;

[0097] The point cloud clustering algorithm is used to process the defect point clouds filtered in step S322 respectively. For the same type of defects, multiple clusters are obtained after clustering, and each cluster represents an instance of the corresponding type of defect. This not only can count the number of defects, but also provides convenience for subsequent measurement methods to calculate the three-dimensional size of the defect instance. The point cloud clustering algorithm can be the OPTICS point cloud clustering algorithm, the K-means algorithm, the DBSCAN algorithm, etc.

[0098] In a specific embodiment, the OPTICS point cloud clustering algorithm is used to process the rust and spalling point clouds filtered in step S11 to obtain rust and spalling instances. In this embodiment, the spalling point cloud identifies two different instance clusters, such as Figure 6 As shown. In this embodiment, the OPTICS algorithm technical parameters for extracting corrosion examples are: neighborhood radius is 0.005, maximum reachable distance is 0.01, minimum number of samples is 10, and minimum cluster size is 30; the technical parameters for extracting spalling examples are: neighborhood radius is 0.005, maximum reachable distance is 0.01, minimum number of samples is 30, and minimum cluster size is 100. The crack measurement method of this technology can measure network cracks, so in this embodiment, the crack point cloud crack instance is not extracted, and the crack point cloud filtered by S11 is directly used as the crack instance.

[0099] In step S4 of this embodiment, the skeleton point sets of the crack point cloud are extracted by using the skeleton extraction method, and the boundary point set corresponding to the skeleton point set of the crack point cloud is extracted by using the boundary point extraction algorithm, which specifically includes the following steps:

[0100] S410, boundary point extraction;

[0101] The three-dimensional crack instance obtained in step S14 is processed using the boundary point detection function in the OPEN3D open source point cloud processing library to accurately extract and store fine crack boundary points.

[0102] S411, crack skeleton point generation;

[0103] The crack boundary points in step S410 are processed using the Laplace shrinkage algorithm or the L1-median skeleton extraction method, and a center point is generated at the center of the crack width along the length direction of the crack as the crack skeleton point, and the calculation of the crack width is arranged in a sequence so that each width value can be calculated. In a specific embodiment, the crack boundary points in step S410 are processed using the Laplace shrinkage algorithm, such as Figure 7 As shown, a series of center points are fitted in the middle of the crack width along the crack length direction as crack skeleton points; the technical parameters of the Laplace shrinkage algorithm in this embodiment are: the number of filtered neighboring points is 20, the filtering standard deviation is 2, and the downsampling rate is 0.00001.

[0104] S412, determining and calculating skeleton point pairs;

[0105] The crack skeleton points generated in step S411 are processed using the Kruskal minimum spanning tree algorithm to generate a skeleton point pair set, which is used to determine the position order for crack width calculation. The Kruskal minimum spanning tree method processes skeleton points by taking skeleton points as vertices, calculating the Euclidean distance between any two vertices as the edge weight between the two points, and forming a weighted connected graph (connected network); without forming a loop in advance, find a path P that can connect all vertices and has the smallest sum of edge weights, output the two points of each edge of the path, and form a skeleton point pair set S = {(p i ,p i+1 )|edge(p i ,p i+1 ) belongs to path P}.

[0106] In this embodiment, quantitatively calculating the length and width of each crack based on each acquired skeleton point set and the corresponding boundary point set specifically includes the following steps:

[0107] S420, determining the radius of the adaptive spherical region;

[0108] Based on the skeleton point pair set in step S412 and the preset initial radius r0, the first point p of each skeleton point pair iAs the center, determine the radius of the sphere that meets the corresponding preset number of crack boundary points. In this embodiment, considering that this type of crack has many branches and a long calculation distance, the proposed method for calculating the crack width based on the adaptive sphere enables the width of the position where the crack geometry changes drastically to be accurately calculated. In this embodiment, the initial sphere radius is set to 2mm.

[0109] Specifically, for each skeleton point pair in the skeleton point pair set, i As the center, a sphere with an initial radius of r0 is generated, and the boundary points within the sphere area are selected into the point set P b (r), the number of points is len(P b (r)) indicates that if the number of boundary points is within the preset range, that is, 2 <len(P b (r))<15, the radius of the sphere at the current position is r=r0; otherwise, adjust the radius, that is, if len(P b (r))<2, then let r=r+r0 / 10, if len(P b (r))>15, then r=r-r0 / 10; then determine whether the number of boundary points in the sphere falls within the preset range, and finally obtain the set of candidate points in the sphere corresponding to each skeleton point.

[0110] S420, crack width calculation;

[0111] The plane P0 fitted in step S323, the boundary points extracted in step S410, the skeleton point pair set S obtained in step S412, and the spherical candidate point set P calculated in step S420 are used. c As input, the crack width at each skeleton point is calculated.

[0112] In this embodiment, based on a preset number of boundary points in a spherical domain and taking the point centers of skeleton point pairs, an adaptive spherical domain crack width estimation method is used to quantitatively calculate the width of each crack according to each skeleton point set and the corresponding boundary point set obtained.

[0113] In this embodiment, reference Figure 8 FIG. 1 is a schematic diagram of an automatic calculation algorithm for three-dimensional crack width based on skeleton point guidance provided in this embodiment. The calculation process of the crack width at each skeleton point specifically includes the following steps:

[0114] A1, respectively from the skeleton point set S and the spherical candidate point set P c Extract the current skeleton point p i and skeleton point p i The corresponding candidate point set p in the sphere c,i, the candidate points in the candidate point set are used to calculate the distance between them and the reference plane, and the candidate points are screened according to the distance, so as to determine the point pair that is finally used to calculate the crack width at the location;

[0115] A2, determine the skeleton point pair and the connecting line corresponding to the current skeleton point, and determine the first plane P1 perpendicular to the connecting line and passing through the current skeleton point using the connecting line direction as the normal vector;

[0116] A3. Based on the current skeleton point, determine a second plane P2 that is perpendicular to both the first plane and the crack point cloud plane, divide the boundary candidate points into positive and negative sides by using the second plane P2, and calculate and determine the boundary candidate point pairs with the shortest distances from each boundary candidate point on the positive and negative sides to the first plane respectively;

[0117] The specific process of determining the boundary candidate point pair is as follows: based on the candidate points p located on different sides of the second plane P2 L,i,j and p R,i,k , calculate the vertical distance d to the first plane P1 respectively L,i,j and distance d R,i,k (where i represents the i-th calculation point, j represents the j-th L-side candidate point, j∈{1,2,3,...,n}, n is the number of L-side candidate points in the current sphere, and k represents the k-th R-side candidate point, k∈{1,2,3,...,m}, m is the number of R-side candidate points in the current sphere).

[0118] A4, the crack width is the distance between the two opposite points closest to the first plane P1 L ,p R =Minimize(|d Lij +d Rij |) between the distance w = ||p L (x L ,y L )-p R (x R ,y R )||2.

[0119] A5. Repeat steps A1-A4 to determine the crack width of the next skeleton point.

[0120] In this embodiment, the above process can be implemented by written program code, and this embodiment uses RGB color to easily extract distinct crack disease semantics in the three-dimensional model; uses point cloud noise points to filter out noise points in the crack point cloud; uses a boundary point detection algorithm to extract crack point cloud boundary points; uses a Laplace shrinkage algorithm to fit crack skeleton points; uses a minimum spanning tree algorithm to generate skeleton point pairs, so that the crack skeleton points that originally have no order can reasonably guide the calculation of crack width, and at the same time uses a sphere to preliminarily select candidate points that meet the conditions from nearby boundary points, generates a sphere and a dual plane to select candidate points for width calculation and determine the final width calculation boundary point pairs, and completes the calculation of crack width.

[0121] In a specific embodiment, 12 crack measurement points are selected, and the crack width measurement values ​​at the manual measurement points are measured using tools such as a crack width comparison card and an electronic vernier caliper. The measured values ​​are compared with the calculated values ​​obtained by the crack width calculation method provided in this embodiment. The calculated average absolute error is 0.183 mm and the average relative error is 9.6%, as shown in the following Table 1.

[0122] Table 1. Comparison of crack width calculation

[0123]

[0124] In this embodiment, the process of calculating the volume of the peeling point in step S5 specifically includes the following steps:

[0125] S510, peeling plane point generation;

[0126] The peeling point cloud instance obtained in step S4 is flattened according to the background plane equation of each peeling point to obtain the peeling plane point. Specifically, the peeling point cloud instance output in step S4 can be projected onto the fitted background plane P0 using the RANSAC algorithm to obtain the peeling plane point.

[0127] S511, normal direction alignment of peeling point cloud;

[0128] The normal direction of all points of the peeling point cloud instance and all points corresponding to the peeling plane points output in step S510 is estimated, and then the normal direction of the concave side of the peeling point cloud instance is adjusted to be opposite to the normal direction of the plane point, so as to distinguish the inside and outside of the peeling surface, thereby improving the accuracy of the peeling three-dimensional volume measurement. Specifically, the estimate_normals() function in the OPEN3D open source library can be used to calculate and align the normals for each point of the three-dimensional peeling instance in step S13 and the peeling plane point output in step S510, and then the three-dimensional peeling instance and the plane point are merged to obtain the three-dimensional peeling instance, and the normal direction of the concave side of the peeling point cloud instance is adjusted to be opposite to the normal direction of the peeling plane point.

[0129] In this embodiment, before the peeled mesh is reconstructed, it is necessary to specify the inside and outside of the three-dimensional closed area to be reconstructed to guide the reconstruction. The method used here is to adjust the normal directions of points from different parts to uniformly face inward (which can be understood as the center position of the entire merged point cloud) to distinguish the front and back of the model.

[0130] S512, generating a closed surface mesh peeling model;

[0131] The adjusted three-dimensional peeling instance and plane points outputted in step S511 are merged to obtain a more complete three-dimensional peeling instance, and then a closed triangular patch-based surface mesh peeling model is generated based on the three-dimensional peeling instance using the Poisson surface reconstruction algorithm and the hole filling algorithm. In a specific embodiment, the technical parameters of the peeling surface of the Poisson surface reconstruction algorithm are as follows: the density threshold is 0.0001; the fill_small_boundaries() function in the pymeshfix open source library is used to fill the holes on the surface of the reconstructed mesh model to generate a closed surface mesh peeling model, which can be as follows Fig. 9 shown.

[0132] S513, spalling volume measurement;

[0133] The get_geometric_measures() function in the PyMeshLab open source library can be used to measure the three-dimensional volume of the closed surface mesh peeling model in step S512.

[0134] In this embodiment, a plane fitting algorithm is used to fit the projection plane for the spalling point cloud instance to generate a flattened spalling point cloud; a point normal estimation algorithm is used to calculate the normal vectors of the rust point cloud and the spalling point cloud; a surface reconstruction algorithm is used to generate triangular surfaces for the rust disease and the spalling disease, respectively, to facilitate the measurement of area and volume, and a hole filling algorithm is used to process the spalling disease triangular surface model to generate a completely closed mesh model, thereby further improving the quality of the generated model and the measurement accuracy.

[0135] The calculation of the corrosion area in step S6 of this embodiment may be implemented by the following steps:

[0136] S610, generating a surface mesh corrosion model;

[0137] The estimate_normals() function in the OPEN3D open source library estimates the normals for the corrosion point cloud instance obtained in step S4, and generates a closed surface mesh corrosion model using the Poisson surface reconstruction algorithm; the technical parameters of the Poisson surface reconstruction algorithm in this embodiment for reconstructing the corrosion surface mesh are: the density threshold is 0.3.

[0138] S23: Rust surface area measurement;

[0139] The surface area of ​​the rusted surface mesh in step S610 was measured using the get_geometric_measures() function in the PyMeshLab open source library, which is 9.63 cm 2 .

[0140] The method of this embodiment makes full use of the positioning information of the drone to automatically restore the absolute size of the three-dimensional model without any additional physical scale reference; the point cloud clustering algorithm is used to identify and extract individual disease instances from point clouds containing multiple diseases of the same type. Only images taken by ordinary consumer drones are needed to achieve surface disease measurement. It is simple and flexible, with a high degree of automation, and can greatly improve the detection cost; this method overcomes the problems of limited detection field of view and distortion of two-dimensional images, and can quickly achieve global three-dimensional positioning of diseases; then the idea of ​​point cloud merging is used to quickly generate a more three-dimensional disease point cloud, which helps the surface reconstruction algorithm to better generate dense triangular facets, and uses a hole filling algorithm to further improve the quality of the generated model. The measurement of the width of complex mesh fine cracks and the area and volume measurement technology of block-shaped rust and peeling diseases with drastic surface curvature changes are aimed at accurately quantifying the three-dimensional size of the disease under severe environmental interference.

[0141] Device Embodiment

[0142] According to an embodiment of the present invention, a three-dimensional measurement device for multiple defects on a structure surface is provided. Fig.10 As shown, it is a block diagram of a three-dimensional measurement device for multiple defects on a structure surface provided in this embodiment. The three-dimensional measurement device for multiple defects on a structure surface according to an embodiment of the present invention comprises:

[0143] The image sequence acquisition module 10 is used to use a drone to acquire an image sequence carrying GPS coordinate information.

[0144] The initial model generation module 20 is used to calculate the depth of the image sequence using a 3D reconstruction technique and to perform 3D reconstruction of the scene to generate an initial 3D model.

[0145] The defect determination and point cloud processing module 30 is used to obtain semantic maps of each defect by recognizing the image sequence through the semantic segmentation network, and convert the defect pixel value in each defect semantic map into the color of the specified RGB attribute value; fuse the depth map with the processed semantic map of each defect through the depth map fusion algorithm to obtain a three-dimensional point cloud model of the specified color; extract the point cloud data of the three-dimensional point cloud model through the point cloud extraction method based on the RGB color range, and divide the points into a number of independent point cloud clusters through the clustering algorithm to obtain each defect point cloud instance; wherein the defects include cracks, rust and / or peeling; use the structural surface points around each defect point cloud as background points, and use the clustering algorithm to fit the background plane equation of each defect;

[0146] The crack width calculation module 40 is used to use the skeleton extraction method to extract the skeleton point sets of the crack point cloud respectively, and use the boundary point extraction algorithm to extract the boundary point sets of the skeleton point sets corresponding to the crack point cloud, and quantitatively calculate the length and width of each crack based on the obtained skeleton point sets and the corresponding boundary point sets.

[0147] The peeling point volume calculation module 50 is used to obtain the peeling plane point by flattening the corresponding peeling point cloud instance according to the background plane equation of each peeling point, determine that the normal directions of the peeling point cloud instance and each point of the peeling plane point remain opposite, then merge the peeling point cloud instance and the peeling plane point to obtain a three-dimensional peeling instance, adjust the normal direction of the concave side of the peeling point cloud instance to remain opposite to the flat peeling normal direction, so as to distinguish the inside and outside of the peeling surface; then use the Poisson surface reconstruction algorithm and the hole filling algorithm to generate a peeling surface mesh model, and calculate and determine the volume of the three-dimensional peeling instance based on the peeling surface mesh model.

[0148] The corrosion area calculation module 60 is used to calculate the normal direction of each point of the corrosion cloud instance, and use the Poisson surface reconstruction algorithm to generate a mesh model and calculate the corrosion area.

[0149] The three-dimensional measurement device for multiple defects on the surface of a structure provided in this embodiment discloses an image sequence based on drone photography, reconstructs a realistic three-dimensional surface model through three-dimensional reconstruction technology, automatically and accurately marks the defect area of ​​interest, and maps it to the three-dimensional model, and then locates the identified local image defects to the corresponding position of the spatial three-dimensional point cloud model through deep learning semantic segmentation, so as to achieve global and accurate detection and positioning of the defects on the structure. A point cloud clustering algorithm is used to identify and extract individual defect instances from a point cloud containing multiple defects of the same type; and a more three-dimensional defect point cloud is quickly generated through the idea of ​​point cloud merging, which helps the surface reconstruction algorithm to better generate dense triangular facets. For peeling defects, a Poisson surface reconstruction algorithm is used to generate triangular surfaces for rust and peeling defects, respectively, which is convenient for measuring area and volume, and a hole-filling algorithm is used to process the triangular surface model of peeling defects to generate a completely closed mesh model, thereby improving the accuracy of volume measurement.

[0150] The embodiment of the present invention is an apparatus embodiment corresponding to the above-mentioned method embodiment. The specific operations of the processing steps of each module can be understood by referring to the description of the method embodiment, which will not be repeated here.

[0151] like Fig.11 As shown, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the three-dimensional measurement method for multiple defects on the surface of the structure in the above-mentioned embodiment is implemented.

[0152] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the three-dimensional measurement method for multiple defects on the surface of the structure in the above-mentioned embodiment is implemented; or when the computer program is executed by a processor, the three-dimensional measurement method for multiple defects on the surface of the structure in the above-mentioned embodiment is implemented.

[0153] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0154] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and the contents not described in detail in the specification of the present invention belong to the common knowledge of those skilled in the art.

Claims

1. A three-dimensional measurement method for multiple defects on a structure surface, characterized in that The following steps are involved: Image sequence acquisition: Use drones to collect image sequences with GPS coordinate information; Initial 3D model construction: Use 3D reconstruction technology to calculate the depth of the image sequence and perform 3D reconstruction of the scene to generate an initial 3D model; Defect determination and point cloud processing: The semantic segmentation network is used to identify the image sequence to obtain the semantic map of each defect, and the pixel value of the defect in each semantic map is converted into the color of the specified RGB attribute value; the depth map is fused with the processed semantic map of each defect through the depth map fusion algorithm to obtain a three-dimensional point cloud model of the specified color; then based on the RGB color range, the point cloud data of the three-dimensional point cloud model is extracted through the point cloud extraction method, and the points are divided into several independent point cloud clusters through the clustering algorithm to obtain each defect point cloud instance; the structural surface points around each defect point cloud are used as background points, and the clustering algorithm is used to fit the background plane equation of each defect, where the defects include cracks, rust and / or peeling; Calculation of crack width: Use the skeleton extraction method to extract the skeleton point sets of each plane crack point cloud, and use the boundary point extraction algorithm to extract the boundary point sets of the crack point cloud corresponding to the skeleton point set. According to the obtained skeleton point sets and the corresponding boundary point sets, the width of each crack is quantitatively calculated; Calculation of the volume of the peeling point: flatten the peeling point cloud instance corresponding to the background plane equation of each peeling point to obtain the peeling plane point, determine that the normal directions of the peeling point cloud instance and each point of the peeling plane point are kept facing each other, then merge the peeling point cloud instance and the peeling plane point to obtain a three-dimensional peeling instance, adjust the normal direction of the concave side of the peeling point cloud instance to be facing the flat peeling normal direction, then use the Poisson surface reconstruction algorithm and the hole filling algorithm to generate a peeling surface mesh model, and calculate and determine the volume of the three-dimensional peeling instance based on the peeling surface mesh model; and Calculation of rust area: Calculate the normal direction of each point of the rust cloud instance, and use the Poisson surface reconstruction algorithm to generate a mesh model and calculate the rust area.

2. The three-dimensional measurement method for multiple defects on a structure surface according to claim 1, characterized in that: Image data acquisition uses a camera to capture a set of image sequences of the target structure in a manner of surrounding shooting from top to bottom or from bottom to top, ensuring that the image overlap rate between two adjacent shots is not less than 50%.

3. The three-dimensional measurement method for multiple defects on a structure surface according to claim 1, characterized in that: The method of using the 3D reconstruction technology to calculate the depth of the image sequence and to perform 3D reconstruction of the scene to generate an initial 3D model comprises the following steps: Each image is scaled, and the scaled images are sparsely reconstructed and the initial camera parameters are calculated using 3D reconstruction technology. Depth calculation is performed on the scaled images, and absolute camera parameters are generated based on the GPS coordinate information of the drone. The absolute camera parameters replace the initial camera parameters and generate an initial 3D model of absolute scale.

4. The three-dimensional measurement method for multiple defects on a structure surface according to claim 1, characterized in that: The disease determination implementation steps are as follows: Preparation of multi-disease dataset: Use annotation software to obtain a dataset with disease labels for the single-channel grayscale labels of three typical surface diseases, namely cracks, spalling and rust, in the scaled images; Semantic segmentation of defects in two-dimensional images: Use the dataset to train the semantic segmentation network until convergence to obtain the defect semantic segmentation network. Then use the trained defect semantic segmentation network to identify defects in each image in the image sequence and mark them with corresponding colors to obtain crack semantic maps, rust semantic maps and / or spalling semantic maps.

5. The three-dimensional measurement method for multiple defects on a structure surface as claimed in claim 4, characterized in that: The point cloud processing specifically includes the following steps: Generate semantic models and locate multiple types of diseases: The semantic images of each disease replace the original image after scaling, and re-calculate the fusion to generate a 3D point cloud model with disease semantics. The corresponding disease color mark in the image is mapped to the 3D space to locate the disease on the 3D model. Extraction of defect point cloud: Extract the crack point cloud model, peeling point cloud model and rust point cloud model in the 3D point cloud model according to the color attribute value corresponding to each defect point cloud; Background plane detection: manually cut out the structural surface points around the defect from each defect point cloud as background points, and fit the background plane; Extract disease instances: Use point cloud clustering algorithm to process the disease point cloud and obtain multiple clusters of various diseases. Each cluster represents an instance of the corresponding type of disease.

6. The three-dimensional measurement method for multiple defects on a structure surface according to claim 1, characterized in that: The method of extracting the skeleton point sets of the crack point cloud by using the skeleton extraction method and extracting the boundary point sets of the crack point cloud corresponding to the skeleton point sets by using the boundary point extraction algorithm specifically includes the following steps: Boundary point extraction: Use the boundary point detection function to process the three-dimensional crack instance and extract the fine crack boundary points; Crack skeleton point generation: Use the Laplace shrinkage algorithm or L1-median skeleton extraction method to process the crack boundary points, generate a center point at the center of the crack width along the crack length direction and use it as the crack skeleton point; Determine and calculate skeleton point pairs: Use Kruskal minimum spanning tree algorithm to process crack skeleton points and generate a set of skeleton point pairs; Adaptive spherical domain radius determination: Based on the obtained skeleton point pair set and the preset initial radius, the first point of each skeleton point pair is taken as the center to determine the spherical domain radius that satisfies the corresponding preset number of crack boundary points, and determine the candidate point set in the spherical domain corresponding to each skeleton point; Crack width calculation: Based on the fitted background plane, the extracted boundary points, the skeleton point pair set and the spherical candidate point set, the crack width at each skeleton point is calculated.

7. The three-dimensional measurement method for multiple defects on a structure surface according to claim 6, characterized in that: The adaptive spherical radius determination specifically comprises the following steps: For the first point p of each skeleton point pair in the skeleton point pair set i As the center, a sphere with an initial radius of r0 is generated, and the boundary points within the sphere area are selected into the point set P b (r), the number of points is len(P b (r)) indicates; Determine if the number of boundary points is within the preset range, that is, 2 <len(P b (r))<15, the radius of the sphere at the current position is r=r0; otherwise, adjust the radius, that is, if len(P b (r))<2, then let r=r+r0 / 10, if len(P b (r))>15, then r=r-r0 / 10; then determine whether the number of boundary points in the sphere falls within the preset range, and finally obtain the set of candidate points in the sphere corresponding to each skeleton point.

8. The three-dimensional measurement method for multiple defects on a structure surface according to claim 6, characterized in that: The spalling point volume calculation process specifically includes the following steps: Generation of peeling plane points: The peeling plane points are obtained by flattening the peeling point cloud instance obtained by the background plane equation of each peeling point; Align the normal direction of the peeled point cloud: estimate the normal direction of all points of the peeled point cloud instance and all points corresponding to the peeled plane points, and then adjust the normal direction of the concave side of the peeled point cloud instance to keep it facing the normal direction of the plane points; Generate a closed surface mesh peeling model: merge the peeling plane points and the three-dimensional peeling instance after the normal direction is adjusted to obtain a more complete three-dimensional peeling instance, and then use the Poisson surface reconstruction algorithm and the hole filling algorithm to generate a closed surface mesh peeling model based on the three-dimensional peeling instance; Spalling volume measurement: The get_geometric_measures() function in the PyMeshLab open source library can be used to measure the three-dimensional volume of a closed surface mesh spalling model.

9. A three-dimensional measurement device for multiple defects on a structure surface, characterized in that include: An image sequence acquisition module is used to use a drone to acquire image sequences carrying GPS coordinate information; An initial model generation module is used to calculate the depth of the image sequence using a 3D reconstruction technique and to perform 3D reconstruction of the scene to generate an initial 3D model; The defect determination and point cloud processing module is used to obtain semantic maps of each defect by recognizing the image sequence through the semantic segmentation network, and convert the defect pixel value in each defect semantic map into the color of the specified RGB attribute value; the depth map is fused with the processed semantic map of each defect by the depth map fusion algorithm to obtain a three-dimensional point cloud model of the specified color; then based on the RGB color range, the point cloud data of the three-dimensional point cloud model is extracted by the point cloud extraction method, and the points are divided into several independent point cloud clusters by the clustering algorithm to obtain each defect point cloud instance; the structural surface points around each defect point cloud are used as background points, and the clustering algorithm is used to fit the background plane equation of each defect, wherein the defects include cracks, rust and / or peeling; The crack width calculation module uses the skeleton extraction method to extract the skeleton point sets of the crack point cloud respectively, and uses the boundary point extraction algorithm to extract the boundary point sets of the crack point cloud corresponding to the skeleton point sets. According to the obtained skeleton point sets and the corresponding boundary point sets, the width of each crack is quantitatively calculated; The peeling point volume calculation module is used to obtain peeling plane points by flattening the peeling point cloud instances corresponding to the background plane equations of each peeling point, determine that the normal directions of the peeling point cloud instances and each point of the peeling plane points are kept facing each other, then merge the peeling point cloud instances and the peeling plane points to obtain a three-dimensional peeling instance, adjust the normal direction of the concave side of the peeling point cloud instance to be facing the flat peeling normal direction, then use the Poisson surface reconstruction algorithm and the hole filling algorithm to generate a peeling surface mesh model, and calculate and determine the volume of the three-dimensional peeling instance based on the peeling surface mesh model; The corrosion area calculation module is used to calculate the normal direction of each point of the corrosion cloud instance, and use the Poisson surface reconstruction algorithm to generate a mesh model and calculate the corrosion area.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the three-dimensional measurement method for multiple defects on a structure surface as described in any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for three-dimensional measurement of multiple defects on a structure surface as claimed in any one of claims 1 to 8 is implemented.

Images