Cable force estimation method, device and readable storage medium based on three-dimensional point cloud data

Abstract

The present invention provides a cable force estimation method, device, and readable storage medium based on three-dimensional point cloud data. The method comprises: projecting bridge point cloud data onto the xOy plane and downsampling to obtain two-dimensional point cloud data; calculating the skeleton lines of the two-dimensional point cloud data, and then dividing the bridge point cloud data into several segments; processing each segment of the bridge point cloud data to obtain a semantic segmentation result of the entire bridge; extracting point cloud data classified as cables, then slicing the point cloud data and fitting the center of a circle, and projecting the center of the circle onto a two-dimensional plane; and obtaining the average cable force of each cable based on the Irvine equation and the center of the two-dimensional circle. The computer device includes a processor and a memory. The readable storage medium stores program instructions for implementing the above method. The present invention can effectively solve the problems of low efficiency in cable-stayed bridge cable force testing and the high dependence of cable force point cloud data identification on a large amount of labeled point cloud data, and can significantly improve the efficiency of cable-stayed bridge cable force measurement.

Description

Technical Field

[0001] The present invention relates to the fields of information technology and bridge safety management, and in particular to a cable force estimation method, device and readable storage medium based on three-dimensional point cloud data. Background Art

[0002] Long-span bridges and high-rise buildings are critical infrastructure in modern society. However, bridges can become damaged or corroded during their service life, necessitating regular safety inspections. For cable-stayed bridges, cables are critical components, necessitating accurate measurement of their forces. Traditional cable force measurement methods rely primarily on contact-based measurement devices, such as attached accelerometers or elastic magnetic sensors, from which the average force and time-varying cable tension are calculated. However, these methods are time-consuming and labor-intensive due to the need to install sensors on long-span bridges and move data acquisition systems. In recent years, non-contact measurement technologies such as 3D laser scanning and photogrammetry have been increasingly used as part of bridge inspection and management. These non-contact measurement technologies generate point cloud data containing object coordinates and radiometric properties (such as color and intensity) through scanning or reconstruction techniques. Point cloud data stores the geometric and relative coordinate information of cables, allowing for the processing and analysis of these point cloud data to estimate cable forces in cable-stayed bridges. However, raw bridge point cloud data lacks semantic meaning and contains no specific information about bridge components. Therefore, it is necessary to identify the cable point cloud data from the entire bridge point cloud data. However, previous studies have either relied on prior knowledge from specialized fields or used deep learning methods to segment point cloud data. These methods are either less applicable or require a large amount of labeled bridge point cloud data, requiring extensive manual labor. Therefore, there is an urgent need to develop a cable point cloud data identification method that is more applicable and does not rely on a large amount of labeled data.

[0003] Based on a large segmentation model in the field of computer vision, the present invention proposes a cable force estimation method based on three-dimensional point cloud data, which avoids the need for a large amount of labeled point cloud data in the neural network pre-training step and can greatly improve the efficiency of cable force measurement in cable-stayed bridges. Summary of the Invention

[0004] The present invention aims to address at least one of the above-mentioned deficiencies in the prior art. For example, one of the objectives of the present invention is to significantly improve the efficiency of cable-stayed bridge cable force measurement by eliminating the need for large amounts of labeled point cloud data in the neural network pre-training step.

[0005] In order to achieve the above objectives, the present invention provides a cable tension estimation method based on three-dimensional point cloud data.

[0006] The method may include: projecting bridge point cloud data onto the xOy plane and performing downsampling to obtain two-dimensional point cloud data; calculating the skeleton line of the two-dimensional point cloud data, and then dividing the bridge point cloud data into several segments according to the preset bridge segment length and skeleton line; processing each segment of the bridge point cloud data to obtain the semantic segmentation result of the entire bridge; extracting the point cloud data classified as cables, slicing the corresponding point cloud data for each cable and fitting the center of a circle, and projecting the center of the circle onto a two-dimensional plane; obtaining the average cable force of each cable according to the Irvine equation and the two-dimensional center of the circle; wherein processing each segment of the bridge point cloud data includes: mapping a segment of the bridge point cloud data into N RGB images, and selecting one as the primary image and the rest as secondary images; obtaining the primary image The method comprises the following steps: a segmentation mask of the primary image and the corresponding 3D point cloud data is obtained; prompt points are generated in the remaining N-1 secondary images based on the segmentation mask of the primary image and the corresponding 3D point cloud data; the segmentation results of each secondary image are obtained according to the prompt points on each secondary image; the segmentation results of the N RGB images are integrated to obtain a preliminary segmentation result of the point cloud data of the section of bridge; the preliminary segmentation result is corrected to obtain a final segmentation result of the point cloud data of the section of bridge; according to the type k of bridge components, a point cloud data support set of n groups of k components is prepared; the features of the point cloud data are obtained according to the final segmentation result and the point cloud data support set; the similarity between the features is calculated, and the data points in the point cloud data of the section of bridge are classified into the component category with the highest similarity.

[0007] Optionally, the step of calculating the skeleton line of the two-dimensional point cloud data is implemented based on an optimal transmission algorithm.

[0008] Furthermore, the step of calculating the skeleton line of the two-dimensional point cloud data includes: extracting the topological relationship, checking the topological structure of the skeleton curve, performing geometric optimization, and obtaining a smooth skeleton curve; converting the smooth skeleton curve into a piecewise linear curve, and calculating the distance between every two adjacent skeleton points from the starting point to the end point; and dividing the skeleton line and the bridge point cloud data into several segments according to the preset bridge segment length and the distance between adjacent skeleton points.

[0009] Optionally, the step of mapping the point cloud data of a section of bridge into N RGB images is implemented based on direct point projection technology.

[0010] Furthermore, the step of mapping the point cloud data of a section of bridge into N RGB images includes: specifying N different projection angles; processing each projection angle to obtain N RGB images; wherein the processing steps for each projection angle include: for each projection angle, mapping the point cloud data to a plane perpendicular to the projection direction, gridding the two-dimensional point cloud data according to a set grid size, and recording the grid where each data point is located; setting the virtual position of the camera; clustering the data points in the same grid to obtain multiple clusters, and calculating the distance from each cluster to the virtual position of the camera, setting the data points in the cluster with the smallest distance as direct points, and the points in the remaining clusters as occluded points; taking the RGB value of the projected image as the average of the RGB values ​​of the direct points; performing the same processing for each of the N projection angles to obtain N RGB images. Furthermore, clustering can be performed using a clustering algorithm.

[0011] Optionally, the segmentation mask based on the main image and the corresponding three-dimensional point cloud data generates prompt points in the remaining N-1 secondary images; according to each secondary image and the prompt points on the image, the segmentation results of each secondary image are obtained, which may include: based on the segmentation result of the main image, each mask corresponds to a bridge component, and the following processing is performed on each mask: the direct points and occlusion points corresponding to the mask are extracted; the pixel points corresponding to the direct points are found in the N-1 secondary images, and the prompt points are calculated using these pixel points; each secondary image and its prompt points are used as the input of the large model, and the corresponding mask, as well as the direct points and occlusion points corresponding to the mask can be segmented.

[0012] Optionally, the integration of the segmentation results of the N RGB images to obtain the preliminary segmentation results of the bridge point cloud data may include: counting the segmentation results of the N RGB images in a statistical way; for each mask, recording the corresponding direct point belonging to the component p i The score is 1.0, and the corresponding occlusion point belongs to component p i The score is 0.5; each data point is classified into the component category with the highest score.

[0013] Optionally, the correction of the preliminary segmentation result may include: dividing the data points into one-dimensional points and high-dimensional points based on the geometric features of the data points; for point cloud data composed of one-dimensional points, using a region growing algorithm based on the main direction to divide the point cloud data into several point cloud clusters; for point cloud data composed of high-dimensional points, using a graph cutting method based on normal vectors to divide the point cloud data into several point cloud clusters; for each point cloud cluster, counting the component category to which the majority of data points belong, and classifying all data points into that component category.

[0014] Another aspect of the present invention provides a computer device,

[0015] The computer device includes: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, and the program instructions include instructions for executing the method described above.

[0016] Another aspect of the present invention provides a computer-readable storage medium.

[0017] The medium stores computer program instructions, wherein the computer program instructions implement the above method when executed by a processor.

[0018] Compared with the prior art, the present invention has the following advantages:

[0019] (1) The method of the present invention is simple and can accurately estimate the cable force.

[0020] (2) It can effectively solve the problems of low efficiency in cable-stayed bridge cable force estimation and high dependence of cable force point cloud data identification on a large amount of labeled point cloud data.

[0021] (3) The present invention avoids the need for a large amount of labeled point cloud data in the neural network pre-training step, which can greatly improve the efficiency of cable-stayed bridge cable force measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and other objects and / or features of the present invention will become more apparent from the following description in conjunction with the accompanying drawings, in which:

[0023] Figure 1 A schematic flow chart of a cable force estimation method based on three-dimensional point cloud data is shown;

[0024] Figure 2 The point cloud data of the cable-stayed bridge in the application example is shown;

[0025] Figure 3 The point cloud data of the cable in the application example is shown;

[0026] Figure 4 The cable force of a single cable marked ① is shown. DETAILED DESCRIPTION

[0027] Hereinafter, the cable force estimation method, device and readable storage medium based on three-dimensional point cloud data of the present invention will be described in detail with reference to exemplary embodiments.

[0028] Exemplary embodiment 1

[0029] This paper proposes a cable tension estimation method based on 3D point cloud data. This method, based on a large segmentation model in computer vision, segments the point cloud data into images in an unsupervised manner and then uses a point cloud neural network to obtain semantic labels for the point cloud data. Figure 1 A flow chart of the cable force estimation method based on three-dimensional point cloud data of the present invention is shown.

[0030] like Figure 1 As shown, the cable force estimation method based on three-dimensional point cloud data may include the following steps:

[0031] S101 : Project the bridge point cloud data P onto the xOy plane and perform downsampling to obtain two-dimensional point cloud data Q.

[0032] S102. Based on the optimal transmission algorithm, the skeleton line of Q is calculated, and then the bridge point cloud data P is divided into several segments according to the preset bridge segment length and skeleton line.

[0033] The following steps S103 to S111 are performed for each segment of bridge point cloud data.

[0034] S103. Based on the direct point projection technology, each segment of bridge point cloud data is mapped into N RGB images, that is, each segment of bridge point cloud data corresponds to N RGB images.

[0035] S104 . Select one of the N images as the primary image, and the rest as secondary images.

[0036] S105: Input the main image obtained in step S104 into a large segmentation model in the field of computer vision to obtain the image segmentation mask, and further obtain the 3D point cloud data corresponding to each mask. The 3D point cloud data is used to suggest other perspectives in step 106.

[0037] S106. Based on the segmentation mask of the main image and the corresponding three-dimensional point cloud data, cue points are automatically generated in the remaining N-1 perspectives (i.e., the N-1 secondary images mentioned above), and the mapping images and cue points are respectively input into the segmentation model to obtain the segmentation results of the mapping images of each perspective; in other words, an image and the cue points on the image are input at the same time, and N-1 images need to be input N-1 times to obtain the segmentation results of N-1 perspectives.

[0038] S107 , integrating the segmentation results of the N perspectives to obtain a preliminary segmentation result of the bridge point cloud data.

[0039] S108 , dividing the data points into one-dimensional points and high-dimensional points, and using different over-segmentation methods to correct the preliminary segmentation results obtained in step S107 to obtain the final segmentation results.

[0040] S109. According to the type k of the bridge component, prepare n groups of point cloud data of k components to form a support set.

[0041] S110: Input the point cloud data segmented in step S108 (i.e., the final segmentation result of a segment of point cloud data) and the point cloud data support set from step S109 into a point cloud neural network to obtain features of the point cloud data. This step performs feature extraction on the final segmentation result of a segment of point cloud data. The segmentation process divides a segment of point cloud data into several point cloud clusters, and features are extracted for each point cloud cluster. Simultaneously, features are extracted for each point cloud data in the support set from S109.

[0042] S111. Calculate the similarity between features and classify the data points in the bridge point cloud data into the category with the highest similarity.

[0043] S112. Perform steps S103 to S111 on each segment of bridge point cloud data, and finally obtain the semantic segmentation result of the entire bridge.

[0044] S113. Extract point cloud data classified as cables, slice the point cloud data for each cable, fit the center of a circle, and project the center of the circle onto a two-dimensional plane using a dimensionality reduction algorithm.

[0045] S114. Based on the Irvine equation and the center of the two-dimensional circle, obtain the average cable force of each cable.

[0046] In this embodiment, step S102 may include:

[0047] First, using the 2D point cloud data Q as input, the optimal transfer algorithm extracts topological relationships, then examines the topological structure of the skeleton curve, and finally performs geometric optimization to produce a smooth skeleton curve. Next, the skeleton curve is converted into a piecewise linear curve, and the distance between each two adjacent skeleton points from the starting point to the end point is calculated. Finally, the skeleton line and bridge point cloud data P are divided into several segments based on the preset bridge segment length and the distance between skeleton points (i.e., the distance between adjacent skeleton points).

[0048] In this embodiment, step S103 may include:

[0049] First, specify N different projection angles; second, for each projection angle, map the point cloud data to a plane perpendicular to the projection direction, and grid the 2D point cloud data according to the set grid size, and record the grid where each data point is located; second, set the virtual position of the camera to (0,3y min ,0), where y minThe minimum y-coordinate value of the bridge point cloud data is used. Next, the DBSCAN algorithm is used to cluster the data points in the same grid, resulting in multiple clusters. The distance from each cluster to the camera's virtual position is calculated, with the data points in the cluster with the smallest distance being designated as direct points, and the points in the remaining clusters as occluded points. The RGB values ​​of the projected image are taken as the mean of the RGB values ​​of the direct points. Finally, the same process is performed for N projection angles to obtain N RGB images. The projection process is the process of generating a two-dimensional image from three-dimensional point cloud data. This process first requires converting the three-dimensional data into two-dimensional data, which is called dimensionality reduction. The projection angle indicates the direction of dimensionality reduction. Different projection directions result in different projected images.

[0050] In this embodiment, step S106 may include:

[0051] First, based on the segmentation results of the main image, it is assumed that each mask corresponds to a bridge component; i For example, assuming that its corresponding component c i ; Then, according to the data points obtained in the projection step and their grid positions, the direct points Pvi and occlusion points IPvi corresponding to mi are extracted; then, the pixel points corresponding to Pvi are found in the N-1 secondary images, and the prompt points are calculated using these pixels; then, the secondary images and the prompt points are used as the input of the large model together, and the corresponding masks, as well as the direct points and occlusion points corresponding to the masks can be segmented, where one image and its prompt points are input at a time; finally, the same process is performed on each mask of the main image.

[0052] In this embodiment, step S107 may include:

[0053] First, the segmentation results of N perspectives (i.e., the N RGB images mentioned above) are counted in the form of statistical scores; for each mask n i , record the corresponding direct point belongs to component p i The score is 1.0, and the corresponding occlusion point belongs to component p i The score is 0.5; each data point is then classified into the component category with the highest score.

[0054] In this embodiment, step S108 may include:

[0055] First, based on the geometric features of the data points, the data points are divided into one-dimensional points and high-dimensional points. Then, for the point cloud data composed of one-dimensional points, the region growing algorithm based on the main direction is used to divide the point cloud data into several point cloud clusters. For the point cloud data composed of high-dimensional points, the graph cutting method based on the normal vector is used to divide the point cloud data into several point cloud clusters. Finally, for each point cloud cluster, the component category to which the majority of data points belong is counted, and all data points are classified into this component category.

[0056] In order to better understand the above exemplary embodiments, they are further described below in conjunction with specific application examples.

[0057] Application Examples

[0058] Taking a cable-stayed bridge as an example, the method of the present invention is used to calculate the cable force of a single cable numbered ① to be 14459 kN. This result is close to the designed cable force of the bridge, verifying the feasibility of the method.

[0059] The registered bridge point cloud data is shown in Figure 2 The point cloud data is semantically segmented using steps S101 to S112 to extract the point cloud data of the cable. The results are as follows: Figure 3 As shown, different cables are represented by different colors. Figure 2 The point cloud data of the cable marked ① is processed in step S113 to obtain the center point located in the two-dimensional plane. Continuing to process in step S114, the cable force can be calculated to be 14459KN. Figure 4 shown.

[0060] The specific subdivision steps of step S114 are as follows:

[0061] 1) Take the first point of the circle as the origin of the new coordinate system, and the line between the first and last points of the circle as the x-axis of the new coordinate system. Rotate the x-axis 90° counterclockwise to obtain the y-axis of the new coordinate system; convert the coordinates of the circle center to the new coordinate system to obtain the new coordinates of the circle center;

[0062] 2) Irvine's formula is:

[0063]

[0064] where x and y are the x and y coordinates of the new coordinates of the circle center, respectively; l is the chordal length of the cable, which can be calculated from the point cloud data; g is the acceleration of gravity, which is 9.8ms²; m is the mass per unit length of the cable, which can be taken as the design value; θ is the angle between the chordal direction of the cable and the horizontal direction, which can be calculated from the point cloud data; and T is the cable end force.

[0065] According to Irvine's formula, a cable force value can be obtained at each center point;

[0066] 3) Remove the noise points in the cable force values ​​and take the average of the remaining cable force values ​​to obtain the average cable force of 14459 kN.

[0067] According to the rope tension estimation method based on three-dimensional point cloud data of the present invention, it can be programmed as a computer program and the corresponding program code or instructions can be stored in a computer-readable storage medium. When the program code or instructions are executed by the processor, the processor executes the above method. The following processor and memory can be included in the computer device.

[0068] Exemplary embodiment 2

[0069] This exemplary embodiment provides a computer device, including:

[0070] at least one processor;

[0071] A memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, the program instructions including instructions for executing the cable force estimation method based on three-dimensional point cloud data according to exemplary embodiment 1.

[0072] Exemplary embodiment 3

[0073] The present exemplary embodiment provides a computer-readable storage medium.

[0074] The storage medium stores a computer program, and when the computer program instructions are executed by the processor, the cable force estimation method based on three-dimensional point cloud data as described in exemplary embodiment 1 is implemented.

[0075] The computer-readable storage medium may be any data storage device that stores data that can be read by a computer system. Examples of computer-readable storage media include read-only memory, random access memory, read-only optical discs, magnetic tapes, floppy disks, optical data storage devices, and carrier waves (such as data transmission via the Internet via a wired or wireless transmission path).

[0076] Although the present invention has been described above with reference to the exemplary embodiments and the accompanying drawings, it will be apparent to those skilled in the art that various modifications may be made to the above embodiments without departing from the spirit and scope of the claims.

Claims

1. A cable force estimation method based on three-dimensional point cloud data, characterized in that: The method comprises: Project the bridge point cloud data onto the xOy plane and perform downsampling to obtain two-dimensional point cloud data; Calculate the skeleton line of the 2D point cloud data, and then divide the bridge point cloud data into several segments according to the preset bridge segment length and skeleton line; Process the point cloud data of each bridge segment to obtain the semantic segmentation results of the entire bridge; Extract the point cloud data classified as cables. For each cable, slice the corresponding point cloud data, fit the center of the circle, and project the center of the circle onto a two-dimensional plane. According to the Irvine equation and the center of the two-dimensional circle, the average cable force of each cable is obtained; The processing of point cloud data of each bridge section includes: Map the point cloud data of a section of bridge into N RGB images, and select one as the primary image and the rest as secondary images; Obtain the segmentation mask of the main image and obtain the 3D point cloud data corresponding to each segmentation mask; Based on the segmentation mask of the primary image and the corresponding 3D point cloud data, cue points are generated in the remaining N-1 secondary images; and segmentation results of each secondary image are obtained according to the cue points on each secondary image. Integrate the segmentation results of N RGB images to obtain the preliminary segmentation results of the bridge point cloud data; Correct the preliminary segmentation result to obtain the final segmentation result of the bridge point cloud data; According to the type k of bridge components, prepare n groups of point cloud data support sets of k components; According to the final segmentation results and the point cloud data support set, the features of the point cloud data are obtained; The similarity between features is calculated, and the data points in the bridge point cloud data are classified into the component category with the highest similarity.

2. The cable force estimation method based on three-dimensional point cloud data according to claim 1, characterized in that: The step of calculating the skeleton line of the two-dimensional point cloud data is implemented based on an optimal transmission algorithm.

3. The cable force estimation method based on three-dimensional point cloud data according to claim 2, characterized in that: The step of calculating the skeleton line of the two-dimensional point cloud data includes: Extract topological relationships, check the topological structure of the skeleton curve, perform geometric optimization, and obtain a smooth skeleton curve; Convert the smooth skeleton curve into a piecewise linear curve and calculate the distance between every two adjacent skeleton points from the starting point to the end point; The skeleton line and the bridge point cloud data are divided into several segments according to a preset bridge segment length and a distance between adjacent skeleton points.

4. The cable force estimation method based on three-dimensional point cloud data according to claim 1, characterized in that: The step of mapping the point cloud data of a section of bridge into N RGB images is implemented based on direct point projection technology.

5. The cable force estimation method based on three-dimensional point cloud data according to claim 1, characterized in that: The step of mapping a section of bridge point cloud data into N RGB images includes: Specify N different projection angles; Process each projection angle to obtain N RGB images; The steps of processing each projection angle include: For a projection angle, the point cloud data is mapped onto a plane perpendicular to the projection direction, and the two-dimensional point cloud data is gridded according to the set grid size, and the grid where each data point is located is recorded; Set the virtual position of the camera; For the data points in the same grid, cluster them to obtain multiple clusters, and calculate the distance from each cluster to the virtual position of the camera. The data points in the cluster with the smallest distance are set as direct points, and the points in the remaining clusters are blocked points. The RGB value of the projected image is the mean of the RGB values ​​of the direct points. The same process is performed on N projection angles to obtain N RGB images.

6. The cable force estimation method based on three-dimensional point cloud data according to claim 1, characterized in that: The segmentation mask of the primary image and the corresponding three-dimensional point cloud data are used to generate cue points in the remaining N-1 secondary images; According to each secondary image and the cue points on the image, the segmentation results of each secondary image are obtained, including: Based on the segmentation results of the main image, each mask corresponds to a bridge component, and the following processing is performed on each mask: Extract both the direct points and the blocked points corresponding to the mask; Find the pixels corresponding to the direct point in the N-1 secondary images, and use these pixels to calculate the hint point; Each secondary image and its cue points are taken as the input of the large model, and the corresponding mask, as well as the direct points and occluded points corresponding to the mask, can be obtained by segmentation.

7. The cable force estimation method based on three-dimensional point cloud data according to claim 1, characterized in that: The segmentation results of the N RGB images are integrated to obtain the preliminary segmentation results of the bridge point cloud data, including: Count the segmentation results of N RGB images in the form of statistical scores; For each mask, record the corresponding direct point belonging to component p i The score is 1.0, and the corresponding occlusion point belongs to component p i The score is 0.5; Classify each data point into the component category with the highest score.

8. The cable force estimation method based on three-dimensional point cloud data according to claim 1, characterized in that: The correction of the preliminary segmentation result includes: Based on the geometric characteristics of data points, the data points are divided into one-dimensional points and high-dimensional points; For point cloud data composed of one-dimensional points, the region growing algorithm based on the main direction is used to divide the point cloud data into several point cloud clusters; for point cloud data composed of high-dimensional points, the graph cut method based on the normal vector is used to divide the point cloud data into several point cloud clusters; For each point cloud cluster, the component category to which most data points belong is counted, and all data points are classified into this component category.

9. A computer device, characterized in that: include: at least one processor; A memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, and the program instructions include instructions for executing the method according to any one of claims 1-8.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images