Cable force estimation method and device based on three-dimensional point cloud data and readable storage medium

Abstract

The invention provides a cable force estimation method and device based on three-dimensional point cloud data and a readable storage medium. The method comprises the steps of projecting bridge point cloud data to an xOy plane and performing down-sampling to obtain two-dimensional point cloud data; calculating a skeleton line of the two-dimensional point cloud data, and then dividing the bridge point cloud data into a plurality of segments; processing the point cloud data of each section of the bridge to obtain a semantic segmentation result of the whole bridge; extracting point cloud data classified as inhaul cables, then slicing the point cloud data and fitting a circle center, and projecting the circle center to a two-dimensional plane; and obtaining the average cable force of each inhaul cable according to an Irvine equation and the two-dimensional circle center. A computer device includes a processor and a memory. Program instructions for implementing the method are stored on the readable storage medium. According to the method, the problems that the cable-stayed bridge cable force testing efficiency is low and the dependence of cable force point cloud data identification on a large amount of labeled point cloud data is high can be effectively solved, and the cable-stayed bridge cable force measuring efficiency can be greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of information technology and bridge safety management, and in particular to a cable force estimation method, device and readable storage medium based on three-dimensional point cloud data. Background Art

[0002] Long-span bridges and high-rise buildings are key infrastructures of modern society. However, bridges may be damaged or corroded during their use, so they need to be regularly inspected for safety. For cable-stayed bridges, cables are key components of the bridge, so their cable forces need to be accurately measured. Traditional cable force measurement methods mainly rely on contact measurement devices, such as additional accelerometers or elastic magnetic sensors, from which the average force of the cables and the time-varying cable tension are calculated. However, these methods are very time-consuming and require a lot of manpower due to the need to install sensors on long-span bridges and move data acquisition systems. In recent years, non-contact measurement technologies such as 3D laser scanning and photogrammetry have been increasingly used as part of bridge inspection and management. These non-contact measurement technologies can obtain point cloud data containing object coordinates and radiation properties (such as color and intensity information) through scanning or reconstruction technology. Point cloud data can store the geometric and relative coordinate information of cables. Therefore, point cloud data can be processed and analyzed to estimate the cable forces of cable-stayed bridges. However, the original bridge point cloud data does not contain any semantics or specific information about bridge components. Therefore, it is necessary to identify the point cloud data of the cable from the point cloud data of the entire bridge. However, previous studies have either used prior knowledge in professional fields or deep learning methods to segment point cloud data. These methods are either less applicable or require a large amount of labeled bridge point cloud data, which requires a lot of manual labor. Therefore, it is urgent to propose a cable point cloud data recognition method that is more applicable and does not rely on a large amount of labeled data.

[0003] Based on the large segmentation model in the field of computer vision, the present invention proposes a cable force estimation method based on three-dimensional point cloud data, which avoids the need for a large amount of labeled point cloud data in the neural network pre-training step and can greatly improve the efficiency of cable force measurement of cable-stayed bridges. Summary of the invention

[0004] The purpose of the present invention is to solve at least one of the above-mentioned deficiencies in the prior art. For example, one of the purposes of the present invention is to avoid the need for a large amount of labeled point cloud data in the neural network pre-training step, thereby greatly improving the efficiency of cable force measurement of cable-stayed bridges.

[0005] In order to achieve the above objectives, the present invention provides a cable tension estimation method based on three-dimensional point cloud data.

[0006] The method may include: projecting the bridge point cloud data onto the xOy plane and performing downsampling to obtain two-dimensional point cloud data; calculating the skeleton line of the two-dimensional point cloud data, and then dividing the bridge point cloud data into several segments according to the preset bridge segment length and skeleton line; processing each segment of the bridge point cloud data to obtain the semantic segmentation result of the entire bridge; extracting the point cloud data classified as cables, slicing the corresponding point cloud data for each cable and fitting the center of a circle, and projecting the center of the circle onto a two-dimensional plane; obtaining the average cable force of each cable according to the Irvine equation and the two-dimensional center of the circle; wherein processing each segment of the bridge point cloud data includes: mapping a segment of the bridge point cloud data into N RGB images, and selecting one as the primary image and the rest as secondary images; obtaining the primary image The invention discloses a method for obtaining a segmentation mask of the main image and the 3D point cloud data corresponding to each segmentation mask; based on the segmentation mask of the main image and the corresponding 3D point cloud data, generating prompt points in the remaining N-1 secondary images; obtaining the segmentation results of each secondary image according to the prompt points on each secondary image; integrating the segmentation results of N RGB images to obtain the preliminary segmentation result of the point cloud data of the section of bridge; correcting the preliminary segmentation result to obtain the final segmentation result of the point cloud data of the section of bridge; preparing a point cloud data support set of n groups of k components according to the type k of bridge components; obtaining the features of the point cloud data according to the final segmentation result and the point cloud data support set; calculating the similarity between the features, and classifying the data points in the point cloud data of the section of bridge into the component category with the highest similarity.

[0007] Optionally, the step of calculating the skeleton line of the two-dimensional point cloud data is implemented based on an optimal transmission algorithm.

[0008] Furthermore, the step of calculating the skeleton line of the two-dimensional point cloud data includes: extracting the topological relationship, checking the topological structure of the skeleton curve, performing geometric optimization, and obtaining a smooth skeleton curve; converting the smooth skeleton curve into a piecewise linear curve, and calculating the distance between every two adjacent skeleton points from the starting point to the end point; dividing the skeleton line and the bridge point cloud data into several segments according to the preset bridge segment length and the distance between adjacent skeleton points.

[0009] Optionally, the step of mapping a section of bridge point cloud data into N RGB images is implemented based on direct point projection technology.

[0010] Furthermore, the step of mapping a section of bridge point cloud data into N RGB images includes: specifying N different projection angles; processing each projection angle to obtain N RGB images; wherein the processing of each projection angle includes: for one projection angle, mapping the point cloud data to a plane perpendicular to the projection direction, and gridding the two-dimensional point cloud data according to the set grid size, and recording the grid where each data point is located; setting the virtual position of the camera; clustering the data points in the same grid to obtain multiple clusters, and calculating the distance from each cluster to the virtual position of the camera, making the data points in the cluster with the smallest distance the direct point, and the points in the remaining clusters the occlusion points; the RGB value of the projection image takes the average of the RGB value of the direct point; performing the same processing on N projection angles to obtain N RGB images. Further, clustering can be performed using a clustering algorithm.

[0011] Optionally, the segmentation mask based on the main image and the corresponding three-dimensional point cloud data generates prompt points in the remaining N-1 secondary images; according to each secondary image and the prompt points on the image, the segmentation results of each secondary image are obtained, which may include: based on the segmentation result of the main image, each mask corresponds to a bridge component, and the following processing is performed on each mask: the direct points and occlusion points corresponding to the mask are extracted; the pixel points corresponding to the direct points are found in the N-1 secondary images, and the prompt points are calculated using these pixel points; each secondary image and its prompt points are used together as the input of the large model, and the corresponding mask, as well as the direct points and occlusion points corresponding to the mask can be segmented.

[0012] Optionally, the integrating the segmentation results of the N RGB images to obtain the preliminary segmentation results of the bridge point cloud data may include: counting the segmentation results of the N RGB images in a statistical way; for each mask, recording the corresponding direct point belonging to the component p i The score is 1.0, and the corresponding occlusion point belongs to component p i The score is 0.5; each data point is classified into the component category with the highest score.

[0013] Optionally, the correction of the preliminary segmentation result may include: dividing the data points into one-dimensional points and high-dimensional points based on the geometric features of the data points; for point cloud data composed of one-dimensional points, dividing the point cloud data into a number of point cloud clusters using a region growing algorithm based on the main direction; for point cloud data composed of high-dimensional points, dividing the point cloud data into a number of point cloud clusters using a graph cutting method based on normal vectors; for each point cloud cluster, counting the component category to which the majority of data points belong, and classifying all data points into the component category.

[0014] Another aspect of the present invention provides a computer device,

[0015] The computer device comprises: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, and the program instructions comprise instructions for executing the method as described above.

[0016] Another aspect of the present invention provides a computer-readable storage medium.

[0017] The medium stores computer program instructions, wherein the computer program instructions implement the method described above when executed by a processor.

[0018] Compared with the prior art, the beneficial effects of the present invention include at least one of the following:

[0019] (1) The method of the present invention is simple and can accurately estimate the cable force.

[0020] (2) It can effectively solve the problems of low efficiency in cable-stayed bridge cable force estimation and high dependence of cable force point cloud data identification on a large amount of labeled point cloud data.

[0021] (3) The present invention avoids the need for a large amount of labeled point cloud data in the neural network pre-training step, which can greatly improve the efficiency of cable force measurement of cable-stayed bridges. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and other objects and / or features of the present invention will become more apparent through the following description in conjunction with the accompanying drawings, in which:

[0023] Figure 1 A schematic flow chart of a cable force estimation method based on three-dimensional point cloud data of the present invention is shown;

[0024] Figure 2 The point cloud data of the cable-stayed bridge in the application example is shown;

[0025] Figure 3 The point cloud data of the cable in the application example is shown;

[0026] Figure 4 The cable tension of a single cable marked ① is shown. DETAILED DESCRIPTION

[0027] Hereinafter, the cable force estimation method, device and readable storage medium based on three-dimensional point cloud data of the present invention will be described in detail in conjunction with exemplary embodiments.

[0028] Exemplary Embodiment 1

[0029] The present invention proposes a cable force estimation method based on three-dimensional point cloud data. The method is based on a large segmentation model in the field of computer vision, segments the point cloud data mapping image in an unsupervised manner, and then obtains the semantic label of the point cloud data based on a point cloud neural network. Figure 1 A schematic flow chart of the cable force estimation method based on three-dimensional point cloud data of the present invention is shown.

[0030] like Figure 1 As shown, the cable force estimation method based on three-dimensional point cloud data may include the following steps:

[0031] S101 , projecting the bridge point cloud data P onto the xOy plane and performing downsampling to obtain two-dimensional point cloud data Q.

[0032] S102, based on the optimal transmission algorithm, the skeleton line of Q is calculated, and then the bridge point cloud data P is divided into several segments according to the preset bridge segment length and skeleton line.

[0033] The following steps S103 to S111 are performed for each segment of bridge point cloud data.

[0034] S103. Based on the direct point projection technology, each section of the bridge point cloud data is mapped into N RGB images, that is, each section of the bridge point cloud data corresponds to N RGB images.

[0035] S104 . Select one of the N images as a primary image, and the rest as secondary images.

[0036] S105: Input the main image obtained in step S104 into a large segmentation model in the field of computer vision to obtain a segmentation mask of the image, and further obtain the three-dimensional point cloud data corresponding to each mask. The three-dimensional point cloud data is used to prompt other perspectives in step 106.

[0037] S106. Based on the segmentation mask of the main image and the corresponding three-dimensional point cloud data, cue points are automatically generated in the remaining N-1 perspectives (i.e., the N-1 secondary images mentioned above), and the mapping images and cue points are respectively input into the segmentation model to obtain the segmentation results of the mapping images of each perspective; in other words, an image and the cue points on the image are input at the same time, and N-1 images need to be input N-1 times to obtain the segmentation results of N-1 perspectives.

[0038] S107 , integrating the segmentation results of the N perspectives to obtain a preliminary segmentation result of the bridge point cloud data.

[0039] S108, dividing the data points into one-dimensional points and high-dimensional points, and using different over-segmentation methods to correct the preliminary segmentation result obtained in step S107 to obtain the final segmentation result.

[0040] S109. According to the type k of bridge components, prepare n groups of point cloud data of k components to form a support set.

[0041] S110, input the point cloud data obtained by the segmentation in step S108 (i.e., the final segmentation result of a segment of point cloud data) and the point cloud data support set of step S109 into a point cloud neural network to obtain the features of the point cloud data. Among them, this step performs feature extraction on the final segmentation result of a segment of point cloud data. The segmentation process will segment a segment of point cloud data into several point cloud clusters, and extract features for each point cloud cluster. At the same time, for each point cloud data in the support set of S109, its features are also extracted.

[0042] S111. Calculate the similarity between the features, and classify the data points in the point cloud data of this section of the bridge into the category with the highest similarity.

[0043] S112. For each segment of bridge point cloud data, the operations from steps S103 to S111 are performed to finally obtain the semantic segmentation result of the entire bridge.

[0044] S113, extracting point cloud data classified as cables, slicing the point cloud data for each cable, fitting the center of a circle, and projecting the center of the circle onto a two-dimensional plane using a dimensionality reduction algorithm.

[0045] S114. Based on the Irvine equation and the center of the two-dimensional circle, the average cable force of each cable is obtained.

[0046] In this embodiment, step S102 may include:

[0047] First, with the two-dimensional point cloud data Q as input, the optimal transmission algorithm will first extract the topological relationship, then check the topological structure of the skeleton curve, and finally perform geometric optimization to obtain a smooth skeleton curve. Secondly, the skeleton curve is converted into a piecewise linear curve, and the distance between every two adjacent skeleton points from the starting point to the end point is calculated; finally, according to the preset bridge segment length and the distance between skeleton points (that is, the distance between adjacent skeleton points), the skeleton line and bridge point cloud data P are divided into several segments.

[0048] In this embodiment, step S103 may include:

[0049] First, specify N different projection angles; second, for each projection angle, map the point cloud data to a plane perpendicular to the projection direction, and grid the 2D point cloud data according to the set grid size, and record the grid where each data point is located; second, set the virtual position of the camera to (0,3y min ,0), where y minis the minimum y coordinate value of the point cloud data of this section of the bridge; then, for the data points in the same grid, the DBSCAN algorithm is used to cluster them to obtain multiple clusters, and the distance from each cluster to the virtual position of the camera is calculated, and the data points in the cluster with the smallest distance are set as direct points, and the points in the remaining clusters are occluded points; the RGB value of the projected image takes the average of the RGB values ​​of the direct points; finally, the same processing is performed on N projection angles to obtain N RGB images. Among them, the projection process is the process of generating a two-dimensional image from three-dimensional point cloud data. This process first requires converting three-dimensional data into two-dimensional data, that is, dimensionality reduction. The projection angle indicates the direction of dimensionality reduction. Different directions of dimensionality reduction will result in different projected graphics.

[0050] In this embodiment, step S106 may include:

[0051] First, based on the segmentation results of the main image, it is assumed that each mask corresponds to a bridge component; i For example, assuming that its corresponding component c i ; Then, according to the data points obtained in the projection step and their grid positions, the direct points Pvi and occlusion points IPvi corresponding to mi are extracted; then, the pixel points corresponding to Pvi are found in the N-1 secondary images, and the prompt points are calculated using these pixel points; then, the secondary images and the prompt points are used as the input of the large model together, and the corresponding masks, as well as the direct points and occlusion points corresponding to the masks can be segmented, where an image and its prompt points are input at a time; finally, the same process is performed on each mask of the main image.

[0052] In this embodiment, step S107 may include:

[0053] First, the segmentation results of N perspectives (i.e., the N RGB images mentioned above) are counted in the form of statistical scores; for each mask n i , record the corresponding direct point belongs to component p i The score is 1.0, and the corresponding occlusion point belongs to component p i The score is 0.5; each data point is then classified into the component category with the highest score.

[0054] In this embodiment, step S108 may include:

[0055] Firstly, based on the geometric features of the data points, the data points are divided into one-dimensional points and high-dimensional points. Then, for the point cloud data composed of one-dimensional points, the region growing algorithm based on the main direction is used to divide the point cloud data into several point cloud clusters. For the point cloud data composed of high-dimensional points, the graph cutting method based on the normal vector is used to divide the point cloud data into several point cloud clusters. Finally, for each point cloud cluster, the component category to which the majority of data points belong is counted, and all data points are classified into this component category.

[0056] In order to better understand the above exemplary embodiments, they are further described below in conjunction with specific application examples.

[0057] Application Examples

[0058] Taking a cable-stayed bridge as an example, the method of the present invention is used to calculate that the cable force of a single cable marked ① is 14459 KN, which is close to the designed cable force of the bridge, verifying the feasibility of the method.

[0059] The registered bridge point cloud data is shown in Figure 2 Steps S101 to S112 are used to perform semantic segmentation on the point cloud data and extract the point cloud data of the cable. The result is as follows: Figure 3 As shown, different cables are represented by different colors. Figure 2 The point cloud data of the cable marked with ① is processed in step S113 to obtain the center point located in the two-dimensional plane. Continuing to process in step S114, the cable force can be calculated to be 14459 KN. Figure 4 shown.

[0060] The specific subdivision steps of step S114 are as follows:

[0061] 1) Take the first point of the circle center as the origin of the new coordinate system, take the line between the first and last points of the circle center as the x-axis of the new coordinate system, rotate the x-axis 90° counterclockwise to get the y-axis of the new coordinate system; convert the coordinates of the circle center to the new coordinate system to get the new coordinates of the circle center;

[0062] 2) Irvine formula is:

[0063]

[0064] Where x and y are the x and y coordinates of the new coordinates of the center of the circle respectively; l is the chord length of the cable, which can be calculated through the point cloud data; g is the acceleration of gravity, which is 9.8ms2; m is the mass per unit length of the cable, which can be taken as the design value; θ is the angle between the chord direction of the cable and the horizontal direction, which can be calculated through the point cloud data; T is the cable end force.

[0065] According to Irvine's formula, a cable force value can be obtained for each center point;

[0066] 3) Remove the noise points in the cable force values ​​and take the average of the remaining cable force values ​​to obtain the average cable force of 14459 KN.

[0067] According to the cable tension estimation method based on three-dimensional point cloud data of the present invention, it can be programmed as a computer program and the corresponding program code or instructions can be stored in a computer-readable storage medium. When the program code or instructions are executed by the processor, the processor executes the above method. The following processor and memory can be included in the computer device.

[0068] Exemplary Embodiment 2

[0069] This exemplary embodiment provides a computer device, including:

[0070] at least one processor;

[0071] A memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, and the program instructions include instructions for executing the cable force estimation method based on three-dimensional point cloud data according to exemplary embodiment 1.

[0072] Exemplary Embodiment 3

[0073] The present exemplary embodiment provides a computer-readable storage medium.

[0074] The storage medium stores a computer program, and when the computer program instructions are executed by the processor, the cable force estimation method based on three-dimensional point cloud data as described in exemplary embodiment 1 is implemented.

[0075] The computer-readable storage medium may be any data storage device that stores data that can be read by a computer system. For example, examples of computer-readable storage media may include: read-only memory, random access memory, read-only optical disk, magnetic tape, floppy disk, optical data storage device, and carrier wave (such as data transmission via the Internet via a wired or wireless transmission path).

[0076] Although the present invention has been described above in conjunction with the exemplary embodiments and the accompanying drawings, it should be apparent to those skilled in the art that various modifications may be made to the above-described embodiments without departing from the spirit and scope of the claims.

Claims

1. A cable force estimation method based on three-dimensional point cloud data, characterized in that: The method comprises: Project the bridge point cloud data onto the xOy plane and perform downsampling to obtain two-dimensional point cloud data; Calculate the skeleton line of the two-dimensional point cloud data, and then divide the bridge point cloud data into several segments according to the preset bridge segment length and skeleton line; Process the point cloud data of each bridge section to obtain the semantic segmentation result of the entire bridge; Extract the point cloud data classified as cables, slice the corresponding point cloud data for each cable, fit the center of the circle, and project the center of the circle onto a two-dimensional plane; According to the Irvine equation and the center of the two-dimensional circle, the average cable force of each cable is obtained; The processing of point cloud data of each bridge section includes: Map a section of bridge point cloud data into N RGB images, and select one as the primary image and the rest as secondary images; Obtain the segmentation mask of the main image and obtain the three-dimensional point cloud data corresponding to each segmentation mask; Based on the segmentation mask of the main image and the corresponding 3D point cloud data, cue points are generated in the remaining N-1 secondary images; and the segmentation results of each secondary image are obtained according to each secondary image and the cue points on the image; The segmentation results of N RGB images are integrated to obtain the preliminary segmentation results of the bridge point cloud data; The preliminary segmentation result is modified to obtain the final segmentation result of the bridge point cloud data; According to the type k of bridge components, prepare n groups of point cloud data support sets of k components; According to the final segmentation result and the point cloud data support set, the features of the point cloud data are obtained; The similarity between the features is calculated, and the data points in the point cloud data of this section of the bridge are classified into the component category with the highest similarity.

2. The cable force estimation method based on three-dimensional point cloud data according to claim 1, characterized in that: The step of calculating the skeleton line of the two-dimensional point cloud data is implemented based on an optimal transmission algorithm.

3. The cable force estimation method based on three-dimensional point cloud data according to claim 2, characterized in that: The step of calculating the skeleton line of the two-dimensional point cloud data comprises: Extract topological relationships, check the topological structure of the skeleton curve, perform geometric optimization, and obtain a smooth skeleton curve; Convert the smooth skeleton curve into a piecewise linear curve and calculate the distance between every two adjacent skeleton points from the starting point to the end point; The skeleton line and the bridge point cloud data are divided into several segments according to a preset bridge segment length and a distance between adjacent skeleton points.

4. The cable force estimation method based on three-dimensional point cloud data according to claim 1, characterized in that: The step of mapping a section of bridge point cloud data into N RGB images is implemented based on direct point projection technology.

5. The cable force estimation method based on three-dimensional point cloud data according to claim 1, characterized in that: The step of mapping a section of bridge point cloud data into N RGB images comprises: Specify N different projection angles; Process each projection angle to obtain N RGB images; The steps of processing each projection angle include: For a projection angle, the point cloud data is mapped onto a plane perpendicular to the projection direction, and the two-dimensional point cloud data is gridded according to the set grid size, and the grid where each data point is located is recorded; Set the virtual position of the camera; For the data points in the same grid, cluster them to get multiple clusters, and calculate the distance from each cluster to the virtual position of the camera. The data points in the cluster with the smallest distance are set as direct points, and the points in the remaining clusters are blocked points. The RGB value of the projected image is the average of the RGB values ​​of the direct points. The same process is performed on N projection angles to obtain N RGB images.

6. The cable force estimation method based on three-dimensional point cloud data according to claim 1, characterized in that: The segmentation mask based on the primary image and the corresponding three-dimensional point cloud data, generating cue points in the remaining N-1 secondary images; According to each secondary image and the cue points on the image, the segmentation results of each secondary image are obtained, including: Based on the segmentation results of the main image, each mask corresponds to a bridge component, and the following processing is performed on each mask: Extract both the direct points and the occluded points corresponding to the mask; Find the pixels corresponding to the direct point in the N-1 secondary images, and use these pixels to calculate the hint point; Each secondary image and its cue points are taken as the input of the large model, and the corresponding mask, as well as the direct points and occluded points corresponding to the mask, can be segmented.

7. The cable force estimation method based on three-dimensional point cloud data according to claim 1, characterized in that: The segmentation results of the N RGB images are integrated to obtain the preliminary segmentation results of the bridge point cloud data, including: Count the segmentation results of N RGB images in the form of statistical scores; For each mask, record the corresponding direct point belonging to component p i The score is 1.0, and the corresponding occlusion point belongs to component p i The score is 0.5; Classify each data point into the component category with the highest score.

8. The cable force estimation method based on three-dimensional point cloud data according to claim 1, characterized in that: The correction of the preliminary segmentation result comprises: Based on the geometric features of data points, the data points are divided into one-dimensional points and high-dimensional points; For point cloud data composed of one-dimensional points, the region growing algorithm based on the main direction is used to divide the point cloud data into several point cloud clusters; for point cloud data composed of high-dimensional points, the graph cutting method based on the normal vector is used to divide the point cloud data into several point cloud clusters; For each point cloud cluster, count the component categories to which most data points belong, and classify all data points into this component category.

9. A computer device, characterized in that: include: at least one processor; A memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, and the program instructions include instructions for executing the method according to any one of claims 1-8.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images