A three-dimensional measurement method, device, equipment and medium for multiple defects on a structural surface

Abstract

The present application provides a three-dimensional measurement method, device, equipment and medium for multiple defects on a structural surface. The method includes collecting an image sequence carrying GPS coordinate information; using three-dimensional reconstruction technology to calculate the depth of the image sequence, and performing three-dimensional reconstruction of the scene to generate an initial three-dimensional model; determining the defects and processing the point cloud to obtain a three-dimensional point cloud model of a specified color and each defect point cloud instance; using a skeleton extraction method and a boundary point extraction algorithm to extract the boundary point set and the skeleton point set corresponding to the skeleton point set of the crack point cloud, and quantitatively calculating the length and width of each crack based on the obtained skeleton point set and the corresponding boundary point set; then using a Poisson surface reconstruction algorithm and a hole filling algorithm to generate a spalling surface mesh model, and calculating the volume of the three-dimensional spalling instance based on the spalling surface mesh model; determining the normal direction of each point of the rust cloud instance, and using a Poisson surface reconstruction algorithm to generate a mesh model and calculate the rust area.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method, device, equipment and medium for three-dimensional measurement of multiple defects on a structure surface. Background Art

[0002] During their service life, engineering structures can develop cracks due to the combined effects of long-term loads and natural disasters. For example, after prolonged use, concrete bridges can develop typical surface defects such as rust and spalling of steel reinforcement, rendering the structure sub-healthy. If these defects are not promptly and properly detected and repaired, they can accumulate and pose a serious threat to the safe operation of engineering structures. Therefore, effective detection and assessment of surface defects is essential for the safe operation of engineering structures.

[0003] In recent years, the popularity of consumer drones and the development of computer vision technology have advanced the intelligentization of disease detection and improved detection efficiency. Drones are used to capture images of areas difficult to reach by manual inspection, and digital image processing techniques such as machine learning are used to extract disease information. However, related technologies have the following problems:

[0004] 1) Surface defect inspections are primarily based on manual observation, resulting in highly subjective results, extensive workload, and low efficiency. The type, location, and size of defects must be manually recorded in inspection reports, supplemented by drawings and photographs. This makes it difficult to fully document all surface defect information and lacks effective visualization of defect location and size. Physical dimensions are measured using traditional tools like rulers and electronic micrometers, and difficult-to-measure dimensions are simply estimated. Therefore, traditional manual methods for measuring surface defects are no longer adequate for today's rapidly increasing inspection needs.

[0005] 2) UAV imagery combined with 3D reconstruction technology can generate 3D point clouds of engineering structures, providing a foundational 3D model for 3D inspection. However, the resolution, path, and parameters of drone images required for accurate localized defect detection and location have not been fully considered.

[0006] 3) Due to viewing angle limitations and image distortion, 2D image-based defect recognition and measurement methods cannot intuitively provide information about the location of local defects within the overall structure. Furthermore, accurate measurement conditions are extremely demanding, making it difficult to guarantee accuracy in practical applications. 3D inspection methods that directly utilize local point cloud features for defect recognition are sensitive to noise and require extensive manual intervention. Furthermore, deep learning models based on point cloud data are computationally expensive.

[0007] 4) Existing surface defect measurement methods only focus on simple morphological defects in laboratory environments, and lack consideration of typical complex surface defects in actual engineering environments, resulting in limited practical value of many similar measurement technologies.

[0008] Therefore, there is an urgent need to provide a method that can realize effective automated measurement of complex surface defects in real engineering projects. Summary of the Invention

[0009] In order to overcome the problems existing in the related art, the present disclosure provides a three-dimensional measurement method, device, equipment and medium for multiple defects on the surface of a structure to solve the relevant technical problems in the related art.

[0010] One or more embodiments of this specification provide a method for three-dimensionally measuring multiple defects on a structure surface, comprising the following steps:

[0011] Image sequence acquisition: Use drones to collect image sequences with GPS coordinate information;

[0012] Initial 3D model construction: Use 3D reconstruction technology to calculate the depth of the image sequence and perform 3D reconstruction of the scene to generate an initial 3D model;

[0013] Defect identification and point cloud processing: Semantic maps of each defect are obtained by recognizing image sequences through a semantic segmentation network, and the pixel values ​​of each defect in the semantic map are converted into colors with specified RGB attribute values. A depth map fusion algorithm is used to fuse the depth map with the processed semantic maps of each defect to obtain a 3D point cloud model with the specified color. Based on the RGB color range, a point cloud extraction method is used to extract the point cloud data of the 3D point cloud model. A clustering algorithm is used to divide the points into several independent point cloud clusters to obtain point cloud instances for each defect. The structural surface points surrounding each defect point cloud are used as background points, and a clustering algorithm is used to fit the background plane equation for each defect. Defects include cracks, rust, and / or spalling.

[0014] Crack width calculation: The skeleton point set of each plane crack point cloud is extracted using the skeleton extraction method. The boundary point extraction algorithm is then used to extract the boundary point set of the corresponding skeleton point set of the crack point cloud. Based on the obtained skeleton point sets and the corresponding boundary point sets, the width of each crack is quantitatively calculated.

[0015] Calculation of the volume of the peeling points: Flattening the corresponding peeling point cloud instance according to the background plane equation of each peeling point to obtain the peeling plane point, ensuring that the normal directions of the peeling point cloud instance and each point of the peeling plane point are oriented oppositely, then merging the peeling point cloud instance and the peeling plane point to obtain a three-dimensional peeling instance, adjusting the normal direction of the concave side of the peeling point cloud instance to be oriented oppositely to the flat peeling normal direction, then using the Poisson surface reconstruction algorithm and the hole filling algorithm to generate a peeling surface mesh model, and calculating the volume of the three-dimensional peeling instance based on the peeling surface mesh model; and

[0016] Corrosion area calculation: Calculate the normal direction of each point of the rust cloud instance, and use the Poisson surface reconstruction algorithm to generate a mesh model and calculate the rust area.

[0017] One or more embodiments of this specification provide a three-dimensional measurement device for multiple defects on a structure surface, comprising:

[0018] Image sequence acquisition module, used to use drones to collect image sequences carrying GPS coordinate information;

[0019] An initial model generation module is used to calculate the depth of the image sequence using 3D reconstruction technology and perform 3D reconstruction of the scene to generate an initial 3D model;

[0020] The defect determination and point cloud processing module is used to identify image sequences through a semantic segmentation network to obtain semantic maps of each defect, and convert the pixel values ​​of the defects in each semantic map into colors with specified RGB attribute values. The depth map fusion algorithm is used to fuse the depth map with the processed semantic maps of each defect to obtain a three-dimensional point cloud model of the specified color. The point cloud data of the three-dimensional point cloud model is then extracted using a point cloud extraction method based on the RGB color range. The points are divided into several independent point cloud clusters using a clustering algorithm to obtain point cloud instances of each defect. The structural surface points surrounding each defect point cloud are used as background points, and the clustering algorithm is used to fit the background plane equation of each defect. The defects include cracks, rust, and / or spalling.

[0021] The crack width calculation module uses the skeleton extraction method to extract the skeleton point set of each plane crack point cloud, and uses the boundary point extraction algorithm to extract the boundary point set of the crack point cloud corresponding to the skeleton point set. Based on the obtained skeleton point sets and the corresponding boundary point sets, the width of each crack is quantitatively calculated;

[0022] The peeling point volume calculation module is used to flatten the peeling point cloud instance corresponding to the background plane equation of each peeling point to obtain the peeling plane point, determine the normal direction of each point of the peeling point cloud instance and the peeling plane point to be opposite, then merge the peeling point cloud instance and the peeling plane point to obtain a three-dimensional peeling instance, adjust the normal direction of the concave side of the peeling point cloud instance to be opposite to the flat peeling normal direction, and then use the Poisson surface reconstruction algorithm and the hole filling algorithm to generate a peeling surface mesh model. The volume of the three-dimensional peeling instance is calculated based on the peeling surface mesh model;

[0023] The corrosion area calculation module is used to calculate the normal direction of each point of the corrosion cloud instance, and use the Poisson surface reconstruction algorithm to generate a mesh model and calculate the corrosion area.

[0024] One or more embodiments of this specification provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-described three-dimensional measurement method for multiple defects on a structural surface when executing the computer program.

[0025] One or more embodiments of the present specification provide a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the three-dimensional measurement method for multiple defects on a structural surface as described above.

[0026] The present disclosure provides a method, device, equipment, and medium for three-dimensional measurement of multiple defects on a structural surface. The advantages of the method are that, based on a sequence of images taken by a drone, a realistic three-dimensional surface model is reconstructed through three-dimensional reconstruction technology, the defect area of ​​interest is automatically and accurately marked, and mapped to the three-dimensional model. Then, through deep learning semantic segmentation, the identified local image defects are located at the corresponding position of the spatial three-dimensional point cloud model, thereby achieving global and accurate detection and positioning of defects on the structure. A point cloud clustering algorithm is used to identify and extract individual defect instances from a point cloud containing multiple defects of the same type; and a more three-dimensional defect point cloud is quickly generated through the idea of ​​point cloud merging, helping the surface reconstruction algorithm to better generate dense triangular facets. For spalling defects, a Poisson surface reconstruction algorithm is used to generate triangular facet surfaces for rust and spalling defects, respectively, to facilitate the measurement of area and volume. The spalling defect triangular surface model is processed using a hole-filling algorithm to generate a completely closed mesh model, thereby improving the accuracy of volume measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0028] Figure 1 A flow chart of a method for three-dimensional measurement of multiple defects on a structure surface provided in one or more embodiments of this specification;

[0029] Figure 2 A schematic diagram of image acquisition by a drone provided in one or more embodiments of this specification;

[0030] Figure 3 A schematic diagram of three-dimensional disease detection and location provided in one or more embodiments of this specification;

[0031] Figure 4 A schematic diagram of the three-dimensional global positioning of multiple diseases of bridge piers provided in one or more embodiments of this specification;

[0032] Figure 5 A schematic diagram of a multi-disease semantic point cloud extracted according to one or more embodiments of this specification;

[0033] Figure 6 A schematic diagram of an example of identifying spalling disease using a clustering algorithm provided in one or more embodiments of this specification;

[0034] Figure 7 A schematic diagram of complex network crack skeleton points and boundary points generated for one or more embodiments of this specification;

[0035] Figure 8 A schematic diagram of an automatic calculation algorithm for three-dimensional crack width based on skeleton point guidance provided in one or more embodiments of this specification;

[0036] Figure 9 A flowchart of the spalling damage mesh reconstruction process provided in one or more embodiments of this specification;

[0037] Figure 10 A block diagram of a three-dimensional measurement device for multiple defects on a structural surface provided for one or more embodiments of this specification;

[0038] as well as

[0039] Figure 11 A schematic diagram of the structure of a computer device provided in one or more embodiments of this specification. DETAILED DESCRIPTION

[0040] In order to help those skilled in the art better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0041] The present invention will be described in detail below with reference to specific implementation methods and the accompanying drawings.

[0042] Method Example

[0043] According to an embodiment of the present invention, a method for three-dimensional measurement of multiple defects on a structure surface is provided. Figure 1 FIG. 1 is a flow chart of a method for three-dimensionally measuring multiple defects on a structure surface provided in this embodiment. According to the method for three-dimensionally measuring multiple defects on a structure surface provided in this embodiment, the method comprises the following steps:

[0044] Step S1, image sequence acquisition;

[0045] Use drones to collect image sequences with GPS coordinate information.

[0046] Step S2, initial three-dimensional model construction;

[0047] The 3D reconstruction technology is used to calculate the depth of the image sequence and perform 3D reconstruction of the scene to generate an initial 3D model.

[0048] Step S3, disease determination and point cloud processing;

[0049] The semantic segmentation network is used to identify image sequences to obtain semantic maps of each defect, and the pixel values ​​of the defect in each semantic map are converted into colors with specified RGB attribute values. The depth map is fused with the processed semantic maps of each defect using a depth map fusion algorithm to obtain a 3D point cloud model of the specified color. Based on the RGB color range, the point cloud data of the 3D point cloud model is extracted using a point cloud extraction method, and the points are divided into several independent point cloud clusters using a clustering algorithm to obtain point cloud instances of each defect. Defects include cracks, rust, and / or spalling. The structural surface points surrounding each defect point cloud are used as background points, and the clustering algorithm is used to fit the background plane equation of each defect.

[0050] Step S4, crack width calculation;

[0051] The skeleton point sets of the crack point cloud are extracted using the skeleton extraction method, and the boundary point extraction algorithm is used to extract the boundary point sets of the crack point cloud corresponding to the skeleton point set. The width of each crack is quantitatively calculated based on the obtained skeleton point sets and the corresponding boundary point sets.

[0052] Step S5, calculating the volume of the peeling point;

[0053] According to the background plane equation of each peeling point, the corresponding peeling point cloud instance is flattened to obtain the peeling plane point. The normal directions of the peeling point cloud instance and each point of the peeling plane point are determined to be opposite. The peeling point cloud instance and the peeling plane point are then merged to obtain a three-dimensional peeling instance. The normal direction of the concave side of the peeling point cloud instance is adjusted to be opposite to the flat peeling normal direction to distinguish the inside and outside of the peeling surface. The Poisson surface reconstruction algorithm and the hole filling algorithm are then used to generate a peeling surface mesh model. The volume of the three-dimensional peeling instance is calculated based on the peeling surface mesh model.

[0054] Step S6, calculating the corrosion area;

[0055] The normal direction of each point of the rust cloud instance is calculated, and the Poisson surface reconstruction algorithm is used to generate a mesh model and calculate the rust area.

[0056] The three-dimensional measurement method for multiple defects on a structural surface provided in this embodiment discloses an image sequence captured by a drone, reconstructs a realistic three-dimensional surface model through three-dimensional reconstruction technology, automatically and accurately marks the defect area of ​​interest, and maps it to the three-dimensional model. Then, through deep learning semantic segmentation, the identified local image defects are located at the corresponding position of the spatial three-dimensional point cloud model, thereby achieving global and accurate detection and positioning of defects on the structure. A point cloud clustering algorithm is used to identify and extract individual defect instances from a point cloud containing multiple defects of the same type; and a more three-dimensional defect point cloud is quickly generated through the idea of ​​point cloud merging, helping the surface reconstruction algorithm to better generate dense triangular facets. For spalling defects, a Poisson surface reconstruction algorithm is used to generate triangular surfaces for rust and spalling defects respectively, facilitating the measurement of area and volume. A hole-filling algorithm is used to process the spalling defect triangular surface model to generate a completely closed mesh model, thereby improving the accuracy of volume measurement.

[0057] In this embodiment, in order to achieve the construction of a three-dimensional model of an engineering structure without scale marking, a drone is used to collect an image sequence carrying GPS coordinate information. The specific collection process is as follows:

[0058] Image data acquisition is to use the camera to capture a set of image sequences of the target structure from top to bottom or from bottom to top, ensuring that the image overlap rate between two adjacent shots is not less than 50%. Figure 2As shown in the figure, for example, a DJI Phantom 4RTK drone is used to fly around the structure in an aerial photography mode, taking pictures from top to bottom. The drone is within 5 meters of the structure surface, and a total of 412 high-definition images with a resolution of 4864×3864 are obtained. The GPS coordinates of the camera center when the drone camera sensor takes each image are stored in the gps_data.txt text file in the format of {image name, latitude, longitude, altitude}.

[0059] In step S2 of this embodiment, the depth of the image sequence is calculated using a 3D reconstruction technology, and the 3D reconstruction of the scene is performed to generate an initial 3D model, which specifically may include the following steps:

[0060] Each image is scaled, and 3D reconstruction techniques are used to perform sparse reconstruction and initial camera parameter calculation on the scaled images. Depth calculation is then performed on the scaled images to generate absolute camera parameters, which are then used to replace the initial camera parameters and generate a 3D model at an absolute scale. This embodiment utilizes 3D reconstruction techniques (such as COLMAP Structure from Motion) to calculate relative camera parameters and a sparse model, providing initial parameters for GPS scale correction. This approach focuses on aligning the model scale with the real physical space scale, enabling the reconstruction of realistic surface 3D models that can restore the real physical dimensions without requiring scale markers.

[0061] This embodiment has multiple views Figure 3 The dimensional reconstruction algorithm can be COLMAP, OpenMVS, OpenMVG, VisualSFM, MVE, PhotoScan, PMVS or DJI Terra software.

[0062] In a specific embodiment, using 3D reconstruction technology to calculate the depth of the image sequence and perform 3D reconstruction of the scene to generate an initial 3D model specifically includes the following steps:

[0063] S21, sparse point cloud generation and initial camera parameter calculation;

[0064] The image is scaled and sparsely reconstructed using the Structure from Motion (SfM) algorithm to calculate the relative camera pose as the initial camera parameters and generate a sparse scene model.

[0065] S22, depth information calculation;

[0066] Based on the sparse scene model generated in step S21, a multi-view stereo (MVS) algorithm is used to calculate a depth map for each image and generate an initial dense surface model.

[0067] S23, scale recovery;

[0068] According to the GPS coordinate information (λ, φ, h) of the image, the coordinates are converted into absolute camera parameters, the absolute camera parameters replace the initial camera parameters, and the 3D similarity matrix is ​​solved using the rigid transformation principle. The coordinate conversion process is shown in formula (1):

[0069]

[0070] Among them, λ represents longitude, φ represents latitude, h represents altitude, α represents the long axis of the earth, e represents the first eccentricity of the earth, T abs is the translation parameter, R abs is the rotation parameter. Since the rotation can be ignored in the scale recovery process, we set T abs =R rel .

[0071] S24, generates a sparse model of real scale;

[0072] Perform point multiplication on the 3D similarity matrix of step S23 and the sparse scene model of step S21, that is, P real-size =M similarity ·P init , to realize the initial point cloud P init The point cloud coordinates in are corrected to P real-size , thus generating a true-scale sparse model.

[0073] S25, generating a real-scale dense surface 3D model;

[0074] Based on the sparse model of step S24, a fusion calculation is performed in combination with the depth map calculated in step S22 and the scaled image in step S21 to generate a realistic three-dimensional model of the real-scale structure.

[0075] This embodiment uses a drone to capture images carrying GPS longitude and latitude coordinates in a circular flight, extracts the GPS coordinate information corresponding to the center of each image, and restores the scale. Three-dimensional reconstruction technology is used to calculate the depth of the scaled drone images, fuses the image pixel information to generate an initial three-dimensional model, uses GPS coordinate information to provide a scale reference, and uses a three-dimensional reconstruction method to recalculate the camera parameters, replacing the original camera parameters to generate a three-dimensional model with absolute scale. This method uses the longitude and latitude coordinates of the drone images to correct the scale of the three-dimensional structure model. This method utilizes spatiotemporal continuity and point cloud representation to simultaneously estimate the camera displacement and the three-dimensional structure of the scene without precalculating the camera position. This method is more robust in terms of pose estimation and perspective generation, and the generated image quality is higher.

[0076] In this embodiment, step S3 trains a neural network using an image sample set including a single-channel grayscale label image of various disease semantics to obtain a disease semantic segmentation network. The disease semantic segmentation network is used to perform disease identification on each image in the image sequence to obtain each disease semantic map, and the disease pixel value in the corresponding image is converted into a color with a specified RGB attribute value; then, each disease semantic map is fused with the initial three-dimensional model to obtain a three-dimensional point cloud model with disease semantics.

[0077] In a specific embodiment, the following steps are included:

[0078] S310, multi-disease dataset production;

[0079] Use software such as LabelMe to annotate the single-channel grayscale label images of three typical surface defects, namely cracks, spalling, and rust, in the scaled image in step S21. For example, grayscale values ​​1, 2, and 3 in the label image represent cracks, spalling, and rust, respectively, thereby obtaining a dataset with disease labels.

[0080] S311, semantic segmentation of 2D image defects;

[0081] The semantic segmentation network is trained using the dataset until convergence to obtain a defect semantic segmentation network. The trained defect semantic segmentation network is then used to identify defects in each image in the image sequence and mark them with corresponding colors, thereby obtaining a crack semantic map, a rust semantic map, and / or a spalling semantic map.

[0082] In this embodiment, the semantic segmentation network can be the existing SegFormer, Crackdeep, U-Net, DenseNet, FasterR-CNN, CrackFormer, IBR-Former, CrackSeg9k network. The training and optimization of the network can be achieved through existing technologies and will not be elaborated here. In this embodiment, the colors of the three diseases of cracks, spalling and rust are marked as (128, 0, 0), (255, 195, 0), and (192, 0, 192) respectively, for example, to facilitate distinction.

[0083] In this embodiment, step S3 point cloud processing specifically includes the following steps:

[0084] S320, generating semantic models and locating multiple types of diseases;

[0085] The semantic images of each disease in step S311 are used to replace the corresponding scaled original images in step S21, and the fusion calculation is performed again to generate a three-dimensional point cloud model with disease semantics. The corresponding disease color mark in the image is mapped to the three-dimensional space, and the positioning of the disease on the three-dimensional model is realized. Figure 3The information of each point in the three-dimensional model is represented by six-dimensional data (x, y, z, r, g, b), where (x, y, z) represents the three-dimensional spatial coordinates of the point and r, g, b represents the RGB color attribute value of the point.

[0086] In a specific embodiment, the semantic images of each disease in step S311 are substituted for the corresponding scaled original images in step S21, and a fusion calculation is performed to directly generate a semantic model. The color marks of the disease in the image are mapped into the three-dimensional space, thereby realizing the three-dimensional positioning of the disease. Figure 4 As shown in FIG, a schematic diagram of the three-dimensional global positioning of multiple bridge pier defects provided by this embodiment is provided. The cracks, spalling and rust defects in the point cloud are displayed in the colors of their semantic segmentation images, as shown in FIG. Figure 5 , which is a schematic diagram of the extracted multi-disease semantic point cloud provided by this embodiment.

[0087] S321, extraction of disease point cloud;

[0088] According to the color attribute values ​​corresponding to each disease point cloud, the crack point cloud model, peeling point cloud model and rust point cloud model are extracted from the three-dimensional point cloud model.

[0089] Specifically, whether a point in the point cloud is a crack point can be determined by judging whether the color of each point falls within the specified RGB color interval. The extracted subset of crack points is recorded as P′, which can be expressed as: P′={p∈P|R1≤R(p)≤R2,G1≤G(p)≤G2,B1≤B(p)≤B2}, where P is a three-dimensional crack semantic point cloud, R(p), G(p) and B(p) represent functions that return the red, green and blue values ​​of point p, respectively; (R1, R2), (G1, G2) and (B1, B2) are specific interval thresholds of the red, green and blue channels, respectively.

[0090] S322, noise point filtering;

[0091] In order to remove noise points in the extracted crack point cloud data and improve the quality of each point cloud, the point cloud data is preprocessed to remove noise points in the crack point cloud data; specifically, a statistical filtering algorithm is used to filter out outliers / abnormal points in the different disease point clouds output in step S321, and filtered crack, spalling and rust point cloud models are obtained respectively, wherein the statistical filtering algorithm can be radius filtering, straight-through filtering, bilateral filtering and voxel filtering methods.

[0092] In a specific embodiment, noise point filtering uses a statistical filtering algorithm to filter out outliers / abnormal points in each defect point cloud output in step S321, wherein the specific technical parameters of the crack defect point cloud filtering algorithm are: 20 adjacent points, a standard deviation ratio of 0.2, and the specific technical parameters of the statistical filtering for filtering outliers / abnormal points of block surface defects such as rust and spalling are: 20 adjacent points, a standard deviation ratio of 0.2.

[0093] S323, background plane detection;

[0094] The structural surface points around the defects are manually cropped from each defect point cloud after denoising in step S322 as background points, and the RANSAC algorithm is used to fit the background plane P0=ax+by+cz+d=0.

[0095] In a specific embodiment, the parameters of plane P0 are: a=0.837, b=0.492, c=0.241, d=0.832.

[0096] S324, extract disease instances;

[0097] Use a point cloud clustering algorithm to process the defect point clouds filtered in step S322. For the same type of defects, multiple clusters are obtained through clustering, and each cluster represents an instance of the corresponding type of defect. This not only can count the number of defects, but also facilitates the subsequent measurement method to calculate the three-dimensional size of the defect instance. Among them, the point cloud clustering algorithm can be the OPTICS point cloud clustering algorithm, the K-means algorithm, the DBSCAN algorithm, etc.

[0098] In a specific embodiment, the OPTICS point cloud clustering algorithm is used to process the rust and spalling point clouds filtered in step S11 to obtain rust and spalling instances. In this embodiment, the spalling point cloud identifies two different instance clusters, such as Figure 6 As shown in the figure. In this embodiment, the OPTICS algorithm technical parameters for extracting rust instances are: neighborhood radius of 0.005, maximum reachable distance of 0.01, minimum number of samples of 10, and minimum cluster size of 30; the technical parameters for extracting spalling instances are: neighborhood radius of 0.005, maximum reachable distance of 0.01, minimum number of samples of 30, and minimum cluster size of 100. This crack measurement method can measure network cracks. Therefore, in this embodiment, the crack point cloud is not extracted for crack instances, and the crack point cloud filtered by S11 is directly used as the crack instance.

[0099] In step S4 of this embodiment, the skeleton point sets of the crack point clouds are extracted using the skeleton extraction method, and the boundary point extraction algorithm is used to extract the boundary point sets corresponding to the skeleton point sets of the crack point clouds. Specifically, the following steps are included:

[0100] S410, boundary point extraction;

[0101] The boundary point detection function in the OPEN3D open source point cloud processing library is used to process the three-dimensional crack instance obtained in step S14, and fine crack boundary points are accurately extracted and stored.

[0102] S411, crack skeleton point generation;

[0103] The crack boundary points in step S410 are processed using the Laplace shrinkage algorithm or the L1-median skeleton extraction method, and a center point is generated at the center of the crack width along the length direction of the crack as the crack skeleton point. The crack width calculation is arranged in a sequential order so that each width value can be calculated. In a specific embodiment, the crack boundary points in step S410 are processed using the Laplace shrinkage algorithm, such as Figure 7 As shown, a series of center points are fitted in the middle of the crack width along the crack length direction as the crack skeleton points; the technical parameters of the Laplace shrinkage algorithm in this embodiment are: the number of filtered neighboring points is 20, the filtering standard deviation is 2, and the downsampling rate is 0.00001.

[0104] S412, determining and calculating skeleton point pairs;

[0105] The crack skeleton points generated in step S411 are processed using the Kruskal minimum spanning tree algorithm to generate a skeleton point pair set, which is used to determine the position order for crack width calculation. The Kruskal minimum spanning tree method processes skeleton points by taking the skeleton points as vertices, calculating the Euclidean distance between any two vertices as the edge weight between the two points, and forming a weighted connected graph (connected network); without forming a loop, find a path P that can connect all vertices and has the minimum sum of edge weights, output the two points of each edge of the path, and form a skeleton point pair set S = {(p i ,p i+1 )|edge(p i ,p i+1 ) belongs to path P}.

[0106] In this embodiment, quantitatively calculating the length and width of each crack based on each acquired skeleton point set and the corresponding boundary point set specifically includes the following steps:

[0107] S420, determining the radius of the adaptive spherical domain;

[0108] Based on the skeleton point pair set in step S412 and the preset initial radius r0, the first point p of each skeleton point pair iAs the center, the radius of a sphere that includes a predetermined number of fracture boundary points is determined. In this embodiment, considering the large number of branching fractures of this type and the long calculation distance, the proposed crack width calculation method based on an adaptive sphere allows accurate calculation of the width even at locations where the fracture geometry changes dramatically. In this embodiment, the initial sphere radius is set to 2 mm.

[0109] Specifically, for each skeleton point pair in the skeleton point pair set, the first point p i As the center, a sphere with an initial radius of r0 is generated, and the boundary points within the sphere area are selected into the point set P b (r), the number of points is len(P b (r)) indicates that if the number of boundary points is within the preset range, that is, 2 <len(P b (r))<15, then the radius of the sphere at the current position is r=r0; otherwise adjust the radius, that is, judge if len(P b (r))<2, then let r=r+r0 / 10, if len(P b If (r))>15, then set r=r-r0 / 10; then determine whether the number of boundary points in the sphere falls within the preset range, and finally obtain the set of candidate points in the sphere corresponding to each skeleton point.

[0110] S420, crack width calculation;

[0111] The plane P0 fitted in step S323, the boundary points extracted in step S410, the skeleton point pair set S obtained in step S412, and the spherical candidate point set P calculated in step S420 are used. c As input, the crack width at each skeleton point is calculated.

[0112] In this embodiment, based on a preset number of boundary points within a spherical domain and the point centers of skeleton point pairs, an adaptive spherical domain crack width estimation method is used to quantitatively calculate the width of each crack based on each acquired skeleton point set and the corresponding boundary point set.

[0113] In this embodiment, reference Figure 8 FIG. 1 is a schematic diagram of an automatic calculation algorithm for three-dimensional crack width based on skeleton point guidance provided in this embodiment. The crack width calculation process at each skeleton point specifically includes the following steps:

[0114] A1, respectively from the skeleton point set S and the spherical candidate point set P c Extract the current skeleton point p i and skeleton point p i The candidate point set p in the corresponding sphere c,i, the candidate points in the candidate point set are used to calculate their distance from the reference plane, and the candidate points are screened according to the distance size to determine the point pair that is finally used to calculate the crack width at that location;

[0115] A2: Determine the skeleton point pair and the connecting line corresponding to the current skeleton point, and use the connecting line direction as the normal vector to determine the first plane P1 that is perpendicular to the connecting line and passes through the current skeleton point.

[0116] A3. Based on the current skeleton point, determine a second plane P2 that is perpendicular to both the first plane and the crack point cloud plane. Use the second plane P2 to divide the boundary candidate points into positive and negative sides. Calculate and determine the boundary candidate point pairs with the shortest distances from each boundary candidate point on the positive and negative sides to the first plane.

[0117] The specific process of determining the boundary candidate point pair is as follows: based on the candidate points p located on different sides of the second plane P2 L,i,j and p R,i,k , calculate the vertical distance d to the first plane P1 respectively L,i,j and distance d R,i,k (where i represents the i-th calculation point, j represents the j-th L-side candidate point, j∈{1,2,3,...,n}, n is the number of L-side candidate points in the current sphere, and k represents the k-th R-side candidate point, k∈{1,2,3,...,m}, m is the number of R-side candidate points in the current sphere).

[0118] A4, the crack width is the width of the two opposite points p closest to the first plane P1 L ,p R =Minimize(|d Lij +d Rij |) between the distance w=||p L (x L ,y L )-p R (x R ,y R )||2.

[0119] A5. Repeat steps A1-A4 to determine the crack width of the next skeleton point.

[0120] In this embodiment, the above process can be implemented by written program code, and this embodiment uses RGB color to easily extract distinct crack disease semantics in the three-dimensional model; uses point cloud noise points to filter out noise points in the crack point cloud; uses a boundary point detection algorithm to extract the boundary points of the crack point cloud; uses a Laplace shrinkage algorithm to fit the crack skeleton points; uses a minimum spanning tree algorithm to generate skeleton point pairs, so that the originally unordered crack skeleton points can reasonably guide the calculation of the crack width, and at the same time uses a sphere to preliminarily select candidate points that meet the conditions from nearby boundary points, generates a sphere and a dual plane to select candidate points for width calculation and determine the final width calculation boundary point pairs, thereby completing the calculation of the crack width.

[0121] In a specific embodiment, 12 crack measurement points were selected, and the crack width measurement values ​​at the measurement points were manually measured using tools such as a crack width comparison card and an electronic vernier caliper. The measured values ​​were compared with the calculated values ​​obtained by the crack width calculation method provided in this embodiment. The calculated average absolute error was 0.183 mm and the average relative error was 9.6%, as shown in Table 1 below.

[0122] Table 1. Comparison of crack width calculations

[0123]

[0124] In this embodiment, the process of calculating the volume of the peeling point in step S5 specifically includes the following steps:

[0125] S510, peeling plane point generation;

[0126] The peeling point cloud instances obtained in step S4 are flattened according to the background plane equation of each peeling point to obtain peeling plane points. Specifically, the peeling point cloud instances output in step S4 can be projected onto the fitted background plane P0 using the RANSAC algorithm to obtain peeling plane points.

[0127] S511, alignment of the normal direction of the peeling point cloud;

[0128] The normal directions of all points in the peeling point cloud instance and all points corresponding to the peeling plane points output in step S510 are estimated, and the normal directions of the concave side of the peeling point cloud instance are adjusted to be opposite to the normal directions of the plane points, thereby distinguishing the inside and outside of the peeling surface, thereby improving the accuracy of the peeling 3D volume measurement. Specifically, the estimate_normals() function in the OPEN3D open source library can be used to calculate and align the normals for each point in the three-dimensional peeling instance in step S13 and the peeling plane points output in step S510. The three-dimensional peeling instance and the plane points are then merged to obtain the three-dimensional peeling instance, and the normal directions of the concave side of the peeling point cloud instance are adjusted to be opposite to the normal directions of the peeling plane points.

[0129] In this embodiment, before reconstructing the peeled mesh, it is necessary to specify the interior and exterior of the 3D closed region to guide the reconstruction. The method used here is to adjust the normal direction of points from different parts to face inward (which can be understood as the center position of the entire merged point cloud) to distinguish the front and back of the model.

[0130] S512, generating a closed surface mesh peeling model;

[0131] The adjusted three-dimensional peeling instance and the plane points output in step S511 are merged to obtain a more complete three-dimensional peeling instance. Then, a Poisson surface reconstruction algorithm and a hole filling algorithm are used to generate a closed triangular patch-based surface mesh peeling model based on the three-dimensional peeling instance. In a specific embodiment, the technical parameters of the Poisson surface reconstruction algorithm for peeling the surface are: the density threshold is 0.0001; the fill_small_boundaries() function in the pymeshfix open source library is used to fill the holes on the surface of the reconstructed mesh model to generate a closed surface mesh peeling model. Figure 9 shown.

[0132] S513, spall volume measurement;

[0133] The three-dimensional volume of the closed surface mesh peeling model in step S512 can be measured using the get_geometric_measures() function in the PyMeshLab open source library.

[0134] In this embodiment, a plane fitting algorithm is used to fit a projection plane to the spalling point cloud instance to generate a flattened spalling point cloud; a point normal estimation algorithm is used to calculate the normal vectors of the rust point cloud and the spalling point cloud; a surface reconstruction algorithm is used to generate triangulated surfaces for the rust disease and the spalling disease, respectively, to facilitate the measurement of area and volume, and a hole filling algorithm is used to process the spalling disease triangulated surface model to generate a completely closed mesh model, further improving the quality of the generated model and the measurement accuracy.

[0135] The calculation of the corrosion area in step S6 of this embodiment may be implemented by the following steps:

[0136] S610, generating a surface mesh corrosion model;

[0137] The estimate_normals() function in the OPEN3D open source library estimates the normals for the corrosion point cloud instance obtained in step S4 and uses the Poisson surface reconstruction algorithm to generate a closed surface mesh corrosion model. The technical parameters of the Poisson surface reconstruction algorithm in this embodiment for reconstructing the corrosion surface mesh are: a density threshold of 0.3.

[0138] S23: Rust surface area measurement;

[0139] The surface area of ​​the corroded surface mesh in step S610 was measured using the get_geometric_measures() function in the PyMeshLab open source library, which was 9.63 cm 2 .

[0140] The method of this embodiment fully utilizes the positioning information of the drone to automatically restore the absolute size of the three-dimensional model without any additional physical scale reference. It uses a point cloud clustering algorithm to identify and extract individual disease instances from a point cloud containing multiple diseases of the same type. Surface disease measurement can be achieved using images captured by ordinary consumer-grade drones. This method is simple, flexible, and highly automated, significantly reducing detection costs. It overcomes the problems of the limited detection field of view and distortion of two-dimensional images and can quickly achieve global three-dimensional positioning of the disease. It then uses the idea of ​​point cloud merging to quickly generate a more three-dimensional disease point cloud, helping the surface reconstruction algorithm to better generate dense triangular facets, and uses a hole-filling algorithm to further improve the quality of the generated model. This method is designed to accurately quantify the three-dimensional size of diseases in the presence of severe environmental interference, focusing on measuring the width of complex mesh-like fine cracks and measuring the area and volume of block-like rust and spalling diseases with drastic changes in surface curvature.

[0141] Device embodiment

[0142] According to an embodiment of the present invention, a three-dimensional measurement device for multiple defects on a structure surface is provided. Figure 10 FIG. 1 is a block diagram of a device for three-dimensionally measuring multiple defects on a structure surface according to an embodiment of the present invention. The device for three-dimensionally measuring multiple defects on a structure surface according to an embodiment of the present invention includes:

[0143] The image sequence acquisition module 10 is used to use a drone to acquire an image sequence carrying GPS coordinate information.

[0144] The initial model generation module 20 is used to calculate the depth of the image sequence using a 3D reconstruction technology and perform 3D reconstruction of the scene to generate an initial 3D model.

[0145] The defect determination and point cloud processing module 30 is configured to obtain semantic maps of each defect using a semantic segmentation network to identify image sequences, convert the defect pixel values ​​in each defect semantic map into colors with specified RGB attribute values, fuse the depth map with the processed semantic maps of each defect using a depth map fusion algorithm, and obtain a three-dimensional point cloud model of the specified color. Based on the RGB color range, the point cloud data of the three-dimensional point cloud model is extracted using a point cloud extraction method, and the points are divided into a number of independent point cloud clusters using a clustering algorithm to obtain point cloud instances of each defect. Defects include cracks, rust, and / or spalling. The structural surface points surrounding each defect point cloud are used as background points, and the clustering algorithm is used to fit the background plane equation of each defect.

[0146] The crack width calculation module 40 is used to use the skeleton extraction method to extract the skeleton point sets of the crack point cloud respectively, and use the boundary point extraction algorithm to extract the boundary point sets of the skeleton point sets corresponding to the crack point cloud. Based on the obtained skeleton point sets and the corresponding boundary point sets, the length and width of each crack are quantitatively calculated.

[0147] The peeling point volume calculation module 50 is used to obtain peeling plane points by flattening the peeling point cloud instances corresponding to the background plane equations of each peeling point, determine that the normal directions of the peeling point cloud instances and each point of the peeling plane points remain opposite to each other, then merge the peeling point cloud instances and the peeling plane points to obtain a three-dimensional peeling instance, adjust the normal direction of the concave side of the peeling point cloud instance to remain opposite to the flat peeling normal direction, so as to distinguish the inside and outside of the peeling surface; then use the Poisson surface reconstruction algorithm and the hole filling algorithm to generate a peeling surface mesh model, and calculate and determine the volume of the three-dimensional peeling instance based on the peeling surface mesh model.

[0148] The corrosion area calculation module 60 is used to calculate the normal direction of each point of the corrosion cloud instance, and use the Poisson surface reconstruction algorithm to generate a mesh model and calculate the corrosion area.

[0149] The present embodiment provides a three-dimensional measurement device for multiple defects on a structural surface. It discloses an image sequence captured by a drone, reconstructs a realistic three-dimensional surface model through three-dimensional reconstruction technology, automatically and accurately marks the defect area of ​​interest, and maps it to the three-dimensional model. Deep learning semantic segmentation is then used to locate the identified local image defects to the corresponding position on the spatial three-dimensional point cloud model, thereby achieving global and accurate detection and location of defects on the structure. A point cloud clustering algorithm is used to identify and extract individual defect instances from a point cloud containing multiple defects of the same type. A more three-dimensional defect point cloud is quickly generated through point cloud merging, helping the surface reconstruction algorithm to better generate dense triangular facets. For peeling defects, a Poisson surface reconstruction algorithm is used to generate triangular surfaces for rust and peeling defects, respectively, facilitating area and volume measurement. A hole-filling algorithm is used to process the peeling defect triangular surface model to generate a completely closed mesh model, improving volume measurement accuracy.

[0150] The embodiment of the present invention is an apparatus embodiment corresponding to the above-mentioned method embodiment. The specific operations of the processing steps of each module can be understood by referring to the description of the method embodiment, and will not be repeated here.

[0151] like Figure 11 As shown, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the three-dimensional measurement method for multiple defects on the surface of the structure in the above embodiment is implemented.

[0152] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the three-dimensional measurement method for multiple defects on the surface of the structure in the above-mentioned embodiment is implemented, or when the computer program is executed by a processor, the three-dimensional measurement method for multiple defects on the surface of the structure in the above-mentioned embodiment is implemented.

[0153] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0154] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For the relevant parts, refer to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. A person of ordinary skill in the art can understand and implement it without making any creative efforts.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and the contents not described in detail in the specification of the present invention are common knowledge to those skilled in the art.

Claims

1. A three-dimensional measurement method for multiple defects on a structure surface, characterized in that The following steps are involved: Image sequence acquisition: Use drones to collect image sequences with GPS coordinate information; Initial 3D model construction: Use 3D reconstruction technology to calculate the depth of the image sequence and perform 3D reconstruction of the scene to generate an initial 3D model; Defect identification and point cloud processing: Semantic maps of each defect are obtained by recognizing image sequences through a semantic segmentation network, and the pixel values ​​of each defect in the semantic map are converted into colors with specified RGB attribute values. A depth map fusion algorithm is used to fuse the depth map with the processed semantic maps of each defect to obtain a 3D point cloud model with the specified color. Based on the RGB color range, a point cloud extraction method is used to extract the point cloud data of the 3D point cloud model. A clustering algorithm is used to divide the points into several independent point cloud clusters to obtain point cloud instances for each defect. The structural surface points surrounding each defect point cloud are used as background points, and a clustering algorithm is used to fit the background plane equation for each defect. Defects include cracks, rust, and spalling. Crack width calculation: The skeleton point set of each plane crack point cloud is extracted using the skeleton extraction method, and the boundary point set of the crack point cloud corresponding to the skeleton point set is extracted using the boundary point extraction algorithm. Based on the obtained skeleton point sets and the corresponding boundary point sets, the width of each crack is quantitatively calculated; Calculation of the volume of the peeling points: Flattening the corresponding peeling point cloud instance according to the background plane equation of each peeling point to obtain the peeling plane point, ensuring that the normal directions of the peeling point cloud instance and each point of the peeling plane point are oriented oppositely, then merging the peeling point cloud instance and the peeling plane point to obtain a three-dimensional peeling instance, adjusting the normal direction of the concave side of the peeling point cloud instance to be oriented oppositely to the flat peeling normal direction, then using the Poisson surface reconstruction algorithm and the hole filling algorithm to generate a peeling surface mesh model, and calculating the volume of the three-dimensional peeling instance based on the peeling surface mesh model; and Corrosion area calculation: Calculate the normal direction of each point of the rust cloud instance, and use the Poisson surface reconstruction algorithm to generate a mesh model and calculate the rust area.

2. The three-dimensional measurement method for multiple defects on a structure surface according to claim 1, characterized in that: Image data acquisition is performed by using a camera to capture a set of image sequences of the target structure in a top-down or bottom-up surround shooting manner, ensuring that the image overlap rate between two adjacent shots is not less than 50%.

3. The three-dimensional measurement method for multiple defects on a structure surface according to claim 1, characterized in that: The method of using the 3D reconstruction technology to calculate the depth of the image sequence and perform 3D reconstruction of the scene to generate an initial 3D model includes the following steps: Each image is scaled, and 3D reconstruction technology is used to perform sparse reconstruction and initial camera parameter calculation on the scaled image. Depth calculation is also performed on the scaled image. Absolute camera parameters are generated based on the GPS coordinate information of the drone. The absolute camera parameters are used to replace the initial camera parameters and generate an initial 3D model of absolute scale.

4. The three-dimensional measurement method for multiple defects on a structure surface according to claim 1, characterized in that: The disease determination is implemented in the following steps: Preparation of a multi-disease dataset: Using annotation software, we generated single-channel grayscale labels for three typical surface diseases, namely cracks, spalling, and rust, in the scaled images to obtain a dataset with disease labels. Semantic segmentation of defects in two-dimensional images: Use the dataset to train the semantic segmentation network until convergence to obtain the defect semantic segmentation network. Then use the trained defect semantic segmentation network to identify defects in each image in the image sequence and mark them with corresponding colors, thereby obtaining crack semantic maps, rust semantic maps, and / or spalling semantic maps.

5. The three-dimensional measurement method for multiple defects on a structure surface according to claim 4, characterized in that: The point cloud processing specifically includes the following steps: Generate semantic models and locate multiple types of diseases: The semantic images of each disease are used to replace the scaled original image and re-fused to generate a 3D point cloud model with disease semantics. The corresponding disease color marks in the image are mapped into 3D space to locate the disease on the 3D model. Extraction of defect point cloud: Extract crack point cloud model, peeling point cloud model and rust point cloud model from the 3D point cloud model according to the color attribute value corresponding to each defect point cloud; Background plane detection: Manually crop the structural surface points around each defect point cloud as background points, and fit the background plane; Extract disease instances: Use the point cloud clustering algorithm to process the disease point cloud to obtain multiple clusters of various types of diseases, each cluster representing an instance of the corresponding type of disease.

6. The three-dimensional measurement method for multiple defects on a structure surface according to claim 1, characterized in that: The method of extracting the skeleton point sets of the crack point cloud by using the skeleton extraction method and extracting the boundary point sets corresponding to the skeleton point sets of the crack point cloud by using the boundary point extraction algorithm specifically includes the following steps: Boundary point extraction: Use the boundary point detection function to process three-dimensional crack instances and extract fine crack boundary points; Crack skeleton point generation: Use the Laplace shrinkage algorithm or L1-median skeleton extraction method to process the crack boundary points, and generate a center point at the center of the crack width along the crack length direction and use it as the crack skeleton point; Determine and calculate skeleton point pairs: Use Kruskal minimum spanning tree algorithm to process crack skeleton points and generate a set of skeleton point pairs; Adaptive spherical radius determination: Based on the obtained skeleton point pair set and the preset initial radius, the first point of each skeleton point pair is used as the center to determine the spherical radius that satisfies the preset number of crack boundary points, and the candidate point set within the spherical region corresponding to each skeleton point is determined; Crack width calculation: Based on the fitted background plane, the extracted boundary points, the skeleton point pair set, and the spherical candidate point set, the crack width at each skeleton point is calculated.

7. The three-dimensional measurement method for multiple defects on a structure surface according to claim 6, characterized in that: The adaptive spherical radius determination specifically includes the following steps: For the first point p of each skeleton point pair in the skeleton point pair set i As the center, a sphere with an initial radius of r0 is generated, and the boundary points within the sphere area are selected into the point set P b (r), the number of points is len(P b (r)) indicates; Determine if the number of boundary points is within the preset range, i.e. 2 <len(P b (r))<15, then the radius of the sphere at the current position is r=r0; otherwise adjust the radius, that is, judge if len(P b (r))<2, then let r=r+r0 / 10, if len(P b If (r))>15, then set r=r-r0 / 10; then determine whether the number of boundary points in the sphere falls within the preset range, and finally obtain the set of candidate points in the sphere corresponding to each skeleton point.

8. The three-dimensional measurement method for multiple defects on a structure surface according to claim 6, characterized in that: The spalling point volume calculation process specifically includes the following steps: Generation of peeling plane points: The peeling plane points are obtained by flattening the peeling point cloud instance obtained by the background plane equation of each peeling point; Align the normal direction of the peeling point cloud: estimate the normal direction of all points of the peeling point cloud instance and all points corresponding to the peeling plane points, and then adjust the normal direction of the concave side of the peeling point cloud instance to keep it facing the normal direction of the plane points; Generate a closed surface mesh peeling model: Merge the peeling plane points and the three-dimensional peeling instance after normal direction adjustment to obtain a more complete three-dimensional peeling instance. Then use the Poisson surface reconstruction algorithm and hole filling algorithm to generate a closed surface mesh peeling model based on the three-dimensional peeling instance. Spalling volume measurement: The get_geometric_measures() function in the PyMeshLab open source library can be used to measure the three-dimensional volume of a closed surface mesh spalling model.

9. A three-dimensional measurement device for multiple defects on a structure surface, characterized in that include: Image sequence acquisition module, used to use drones to collect image sequences carrying GPS coordinate information; An initial model generation module is used to calculate the depth of the image sequence using 3D reconstruction technology and perform 3D reconstruction of the scene to generate an initial 3D model; The defect identification and point cloud processing module is used to identify image sequences through a semantic segmentation network to obtain semantic maps of each defect, and convert the pixel values ​​of the defect in each semantic map into colors with specified RGB attribute values. The depth map fusion algorithm is used to fuse the depth map with the processed semantic maps of each defect to obtain a three-dimensional point cloud model of the specified color. Based on the RGB color range, the point cloud extraction method is used to extract the point cloud data of the three-dimensional point cloud model, and the clustering algorithm is used to divide the points into several independent point cloud clusters to obtain point cloud instances of each defect. The structural surface points surrounding each defect point cloud are used as background points, and the clustering algorithm is used to fit the background plane equation of each defect. The defects include cracks, rust, and spalling. The crack width calculation module uses the skeleton extraction method to extract the skeleton point sets of the crack point cloud, and uses the boundary point extraction algorithm to extract the boundary point sets of the crack point cloud corresponding to the skeleton point sets. Based on the obtained skeleton point sets and the corresponding boundary point sets, the width of each crack is quantitatively calculated; The peeling point volume calculation module is used to flatten the peeling point cloud instance corresponding to the background plane equation of each peeling point to obtain the peeling plane point, determine the normal direction of each point of the peeling point cloud instance and the peeling plane point to be opposite, then merge the peeling point cloud instance and the peeling plane point to obtain a three-dimensional peeling instance, adjust the normal direction of the concave side of the peeling point cloud instance to be opposite to the flat peeling normal direction, and then use the Poisson surface reconstruction algorithm and the hole filling algorithm to generate a peeling surface mesh model. The volume of the three-dimensional peeling instance is calculated based on the peeling surface mesh model; The corrosion area calculation module is used to calculate the normal direction of each point of the corrosion cloud instance, and use the Poisson surface reconstruction algorithm to generate a mesh model and calculate the corrosion area.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the three-dimensional measurement method for multiple defects on a structure surface according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the three-dimensional measurement method for multiple defects on a structure surface according to any one of claims 1 to 8 is implemented.

Images