Bridge construction facility and material positioning method and device, and computer storage medium

Abstract

The application discloses a bridge construction facility and material positioning method and device and a computer storage medium, and belongs to the technical field of facility positioning. The method comprises the following steps: acquiring a semantic segmentation dataset; training a semantic segmentation model by using the semantic segmentation dataset to obtain an optimal semantic segmentation model; performing semantic segmentation on point cloud data collected from a construction site according to the optimal semantic segmentation model, and positioning facilities and materials in the construction site based on a fast positioning algorithm; and generating a visual result according to the positioning result of the fast positioning algorithm. The method can quickly position the positions of all facilities and materials and count all objects in each category on the basis of combining unmanned aerial vehicles to collect site data, avoids labor-intensive work such as manual inspection, and can be used for regular safety inspection, so that the accident rate can be reduced.

Description

Technical Field

[0001] This invention relates to the field of construction facility and material positioning technology, and more specifically to a method, apparatus and computer storage medium for positioning bridge construction facilities and materials. Background Technology

[0002] Safety management at construction sites is one of the most challenging tasks in the construction industry. According to statistics on safety accidents in housing and municipal engineering projects, nearly 40% of construction safety accidents are caused by the improper use of facilities and the haphazard storage of materials. Therefore, effective management of facilities and materials at construction sites to reduce the likelihood of accidents is extremely necessary.

[0003] Traditional safety management at road and bridge construction sites relies entirely on site patrols, monitoring, and spatial planning. Generally, appropriate spatial planning methods are used to reduce the likelihood of collisions between facilities, between facilities and people, and between facilities and materials. This involves determining the space required to safely complete different activities and attempting to minimize overlap to avoid accidents. While these methods are effective in reducing the likelihood of collisions between different facilities on a macro level, they cannot completely eliminate safety risks caused by human error and unforeseen circumstances. Furthermore, current safety inspections at road and bridge construction sites largely rely on visual inspections. These inspections require experienced inspectors with inspection equipment to be on-site, a labor-intensive, time-consuming, and sometimes risky process. At the same time, traditional methods also suffer from inefficiency and slow information feedback.

[0004] At present, with the progress of sensor technology and the Internet of Things, many studies have recognized the positive role of automation in improving the efficiency and safety of construction projects, and the construction industry in China is transforming towards intelligence. For this reason, some researchers have considered using visual-based tracking and real-time positioning systems to generate a dangerous proximity warning system. These methods consider that facilities / materials should have a safety space to avoid potential collisions, explosions and other dangers in the near future. This not only improves productivity compared to traditional methods, but also fills the gap in skill shortages by using technology and budget savings to quickly and accurately monitor construction sites, which can effectively reduce the risk of safety accidents. However, in reality, due to the dynamic and complex nature of construction sites, the difficulty of monitoring facility layout, the need for periodic updates of layout locations, and the shielding of site facilities make it difficult for video monitoring technology to fully play its role in road and bridge construction sites, and it is difficult to set up a full-coverage monitoring system on site. Image visual inspection based on unmanned aerial vehicles still has strong subjectivity. In recent years, under the leadership of visual technology, three-dimensional point cloud reconstruction technology has been continuously developed, and more deep learning-based applications have been widely used in construction environments to solve automated safety inspection and on-site safety monitoring problems, such as identifying structural defects, unsafe labor practices, and potential risk factors, which provides a more effective and intelligent method for safety management on road and bridge construction sites.

[0005] Therefore, it is urgent for those skilled in the art to provide a method and device for automatically and quickly positioning facilities and materials in road and bridge construction sites to assist intelligent safety inspection and promote the development of intelligent safety inspection in road and bridge construction sites. SUMMARY

[0006] Therefore, the present application provides a method and device for positioning bridge construction facilities and materials, and a computer storage medium, which uses a combination of deep learning and three-dimensional reconstruction technology to achieve intelligent and rapid positioning of facilities / materials in outdoor construction site scenarios, solving the problems of long time consumption and easy omission in the manual investigation process.

[0007] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0008] On the one hand, the present application provides a method for positioning bridge construction facilities and materials, comprising the following steps:

[0009] Obtaining a point cloud dataset of facilities and materials at the construction site, and labeling the point cloud dataset to obtain a labeled point cloud dataset;

[0010] Constructing a point cloud semantic segmentation model and training the point cloud semantic segmentation model using the labeled point cloud dataset to obtain an optimal point cloud semantic segmentation model;

[0011] According to the best point cloud semantic segmentation model, the point cloud data collected from the construction site is subjected to semantic segmentation, and a semantic segmentation file is generated;

[0012] The facilities and materials at the construction site are positioned based on the fast positioning algorithm;

[0013] A visualization result is generated according to the positioning result of the fast positioning algorithm.

[0014] Preferably, a point cloud data set of the construction site is acquired, and the point cloud data set is labeled, specifically including:

[0015] The construction site is divided into a plurality of regions, and images of the plurality of regions are collected by using a drone;

[0016] Information extraction is performed on the images collected by the drone, and a plurality of regional point cloud data is generated;

[0017] The plurality of regional point cloud data is labeled to generate a labeled point cloud data set.

[0018] Preferably, after the information extraction is performed on the image data collected by the drone to generate a point cloud image, the method further includes: pre-processing the plurality of regional point cloud data, the pre-processing including sequentially performing regional cropping, point cloud noise reduction, and point cloud down-sampling on the plurality of regional point cloud data, and outputting the point cloud data.

[0019] Preferably, the labeled point cloud data set includes x, y, and z coordinates, RGB information, and a class label of each point cloud.

[0020] Preferably, the indicators for evaluating the semantic segmentation result of the pre-trained semantic segmentation model include overall accuracy OA, class average accuracy mAcc, and average intersection over union mIoU, and the formulas are as follows:

[0021]

[0022]

[0023]

[0024] wherein, IoU i is the intersection over union of the model segmentation result and the data set result of the i-th class, A i is the model point cloud classification accuracy of the i-th class, C is the total number of classes in the data set, N T is the number of correctly classified points, N 总 is the total number of all input points.

[0025] Preferably, the facilities and materials at the construction site are positioned based on the fast positioning algorithm, specifically including:

[0026] reading a semantic segmentation file of one facility or material class one by one;

[0027] performing a denoising on the point cloud data in the semantic segmentation file by using a Gaussian filter;

[0028] calculating the distance between all point clouds in the facility or material class, if the distance is less than a distance threshold, it is determined that the point clouds belong to the same object, and repeatedly dividing all points under the same class into several single objects;

[0029] calculating the centroid point of each single object, the centroid point as the coordinate position of the single object;

[0030] After reading all facility and material data, finally reading the point cloud file divided into terrain class, generating a visualization result.

[0031] Preferably, in the visualization result, the same class of facilities and materials are marked with the same color, the coordinate value of each facility or material is displayed, and the number of objects in each class is counted.

[0032] In another aspect, the application also provides a positioning device for bridge construction facilities and materials, which is used to realize the above-mentioned positioning method for bridge construction facilities and materials, and the device comprises:

[0033] an image acquisition module for acquiring images of the construction site;

[0034] a data set processing module for extracting point cloud data of the construction site of the bridge project according to the acquired images, pre-processing the point cloud data, and labeling the pre-processed point cloud data set to obtain a semantic segmentation data set;

[0035] a model training module for constructing a pre-trained semantic segmentation model, and training the pre-trained semantic segmentation model by using the semantic segmentation data set to obtain an optimal pre-trained semantic segmentation model;

[0036] a processing module, the processing module comprising a pre-processing unit and a semantic segmentation unit, the pre-processing unit being configured to extract point cloud data of the construction site according to the acquired images, and pre-process the point cloud data; the semantic segmentation unit being configured to perform semantic segmentation on the pre-processed point cloud data by using the optimal semantic segmentation model to obtain a semantic segmentation file;

[0037] a positioning module for positioning the facilities and materials at the construction site based on a fast positioning algorithm;

[0038] a display module for displaying the visualization result generated by the positioning module.

[0039] In another aspect, the present application also provides a computer readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the steps of any of the above positioning methods based on deep learning.

[0040] Via the above technical solution, compared with the prior art, the present application provides a bridge construction facility and material positioning method, device and computer storage medium. The method can quickly position all facility and material positions and count all objects in each category on the basis of collecting on-site data by matching a UAV. This avoids labor-intensive work such as manual inspection, and the method can be used for regular safety inspection, can reduce the accident rate, and can standardize the use and placement of facilities and materials in the bridge construction site, and can well assist managers in safety management of the bridge construction site. The method can provide new ideas and technical support for future intelligent construction safety inspection. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.

[0042] Figure 1 A flowchart of a bridge construction facility and material positioning method in the present application.

[0043] Figure 2 A principle diagram of a rapid positioning algorithm in the present application.

[0044] Figure 3 A frame diagram of a bridge construction facility and material positioning device in the present application. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0046] Embodiment 1

[0047] A bridge construction facility and material positioning method is disclosed in various embodiments of the present application, comprising the following steps:

[0048] Obtain a point cloud dataset of facilities and materials at the construction site, and label the point cloud dataset to obtain a labeled point cloud dataset;

[0049] Construct a point cloud semantic segmentation model, and train the point cloud semantic segmentation model using the labeled point cloud dataset to obtain an optimal point cloud semantic segmentation model;

[0050] According to the optimal point cloud semantic segmentation model, the point cloud data collected at the construction site is subjected to semantic segmentation to generate a semantic segmentation file;

[0051] Based on the fast positioning algorithm, the facilities and materials at the construction site are positioned;

[0052] According to the positioning result of the fast positioning algorithm, a visual result is generated.

[0053] Preferably, the point cloud dataset of the construction site is obtained, and the point cloud dataset is labeled, specifically including:

[0054] The construction site is divided into a plurality of regions, and images of the plurality of regions are collected by a UAV;

[0055] Information extraction is performed on the images collected by the UAV to generate a plurality of regional point cloud data, and each region is saved as a ".las" file;

[0056] The plurality of regional point cloud data is labeled to generate a labeled point cloud dataset. Before labeling, a three-dimensional model is automatically generated by reading the ".las" file information using CloudCompare, and then the three-dimensional point cloud model is labeled to form a dataset.

[0057] Preferably, after the image data collected by the UAV is subjected to information extraction to generate a point cloud image, the method further includes: pre-processing the point cloud image, wherein the pre-processing includes sequentially performing regional cropping, point cloud noise reduction, and point cloud down-sampling on the plurality of regional point cloud data, and then outputting the point cloud data.

[0058] Preferably, the labeled point cloud dataset includes x, y, z coordinates, RGB information, and a category label of each point cloud.

[0059] Preferably, the indicators used to evaluate the semantic segmentation result of the pre-trained semantic segmentation model include overall accuracy OA, class average accuracy mAcc, and average intersection over union mIoU, and the formulas are as follows:

[0060]

[0061]

[0062]

[0063] wherein, IoUi The intersection over union of the model segmentation result and the dataset result of the first class is A i The model point cloud classification accuracy of the first class is C, and N T The number of points correctly classified is N, and Ntotal is the total number of all points input. i The model point cloud classification accuracy of the first class is C. Adjust the model hyperparameters multiple times, and record the training results (i.e., the above three indicators) after each training. Take mIoU as the main indicator, and select the model with Max(mIoU) as the best PointNeXt semantic segmentation model. If mIoU is the same, then take OA and mACC as the second and third indicators for screening. The semantic segmentation indicator is only used to screen the best model for deployment into the system. In the subsequent system use, the trained best model is directly used for semantic segmentation of new data, and the semantic segmentation effect of the system is only judged by directly observing the actual segmentation effect of the system.

[0064] Preferably, the facilities and materials on the construction site are positioned based on a fast positioning algorithm, specifically including:

[0065] The semantic segmentation file of one facility or material category is read one by one;

[0066] The point cloud data in the semantic segmentation file is denoised once using Gaussian filtering;

[0067] The distances between all point clouds in the facility or material category are calculated, and if the distance is less than the distance threshold, it is determined that the point clouds belong to the same object. All points under the same category are repeatedly divided into several single objects;

[0068] The centroid point of each single object is calculated, and the centroid point is taken as the coordinate position of the single object;

[0069] After reading all the facility and material data, the point cloud file classified as the terrain category is finally read, and a visualization result is generated.

[0070] Preferably, in the visualization result, the facilities and materials of the same category are marked with the same color, the coordinate values of each facility or material are displayed, and the objects of each category are counted.

[0071] Embodiment 2

[0072] Another embodiment of the present application discloses a positioning method for bridge construction facilities and materials, as shown in Figure 1 The method comprises the following steps:

[0073] According to the construction characteristics of the bridge project, the construction stages are divided into bridge substructure construction stage, beam slab construction stage, bridge deck system and road construction stage, and the facilities and materials used in each construction stage are summarized; for facilities, the categories generally involved are tower cranes, gantry cranes, bridge erection machines, etc., and for materials, generally include reinforcing steel bars, flammable and explosive materials, prefabricated components, etc.

[0074] A point cloud dataset of the construction site of the bridge project is obtained, and the point cloud dataset is labeled to obtain a semantic segmentation dataset; in order to ensure the quality of the collected data and the limitation of the battery capacity of the unmanned aerial vehicle, each bridge site is generally divided into multiple regions. At the same time, in order to improve the segmentation effect of the model, as much data as possible of several bridge site projects is collected. The site data of the bridge site project is collected by using an unmanned aerial vehicle. Since the data collected by the unmanned aerial vehicle is an image, the image data collected in each region is extracted by using PIX4DMapper to generate point cloud data, and the point cloud data is output in ".las" format. The point cloud image is preprocessed, which includes sequentially cropping, denoising and downsampling the point cloud image, and then outputting the point cloud data; the point cloud image is labeled according to the facilities and materials used in each construction stage to generate output point cloud data. Preferably, the output point cloud data includes the x, y, z coordinates, RGB information and category label of each point cloud. Specifically, the image data collected at a time is extracted by using PIX4DMapper to generate point cloud data. The ".las" point cloud data generated by PIX4DMapper needs to be preprocessed, and the point cloud data preprocessing software is CloudCompare. First, the point cloud scene is sequentially cropped, denoised and downsampled by using CloudCompare, then labeled according to the facilities and material categories existing in the collected data, and the point cloud data is output in ".txt" format, with each region saved as a point cloud file. The ".txt" file contains 7 columns of data, which are the x, y, z coordinates, RGB information and category label of each point.

[0075] A point cloud semantic segmentation model is constructed, and the point cloud semantic segmentation model is trained by using the semantic segmentation dataset. The indicators for evaluating the semantic segmentation results of the semantic segmentation model include overall accuracy OA, class average accuracy mAcc and average intersection over union mIoU, and the formulas are as follows:

[0076]

[0077]

[0078]

[0079] wherein, IoU iThe intersection over union of the model segmentation result of the i-th class and the dataset result, C is the total number of classes in the dataset, N T The number of points correctly classified, N 总 The total number of all input points.

[0080] Generally, mIoU is considered the most important indicator for evaluating semantic segmentation effect, while OA and mAcc are used as auxiliary indicators to comprehensively evaluate the performance of the PointNeXt model.

[0081] According to the best point cloud semantic segmentation model, the collected point cloud data of the construction site is subjected to semantic segmentation to generate a semantic segmentation file; the facilities and materials in the construction site are positioned based on a fast positioning algorithm; and a visualization result is generated according to the positioning result of the fast positioning algorithm. After obtaining the trained point cloud segmentation model, a facility and material positioning algorithm is developed based on the Python Open3D library. As shown in Figure 2 The semantic segmentation result of PointNeXt saves the points of different classes in a small area as “.ply” files respectively. The semantic segmentation results of different facilities, different materials and terrain classes in a small area are read one by one, and a file of one facility or material class is read each time, and Gaussian filtering is used to denoise the point cloud data after semantic segmentation of the class to reduce the points classified incorrectly in each class as much as possible. Then, the distance of each point is calculated, and if the distance is less than the distance threshold, the point is determined to belong to the same object. All points under the same class are repeatedly divided into several single objects, and then the centroid point of each object is calculated as the coordinate position of the facility / material. After reading all the facility and material data, the point cloud file classified as the “terrain” class is finally read. Finally, a visualization result is generated, in which facilities and materials of the same class are marked with the same color, and the objects of each class are counted and the coordinate values of each object are displayed. The safety supervisor can check the construction site according to the visualization result to check whether there are flammable and explosive materials, whether the materials are stacked in the designated position, whether the materials are randomly stacked, whether the facilities are recycled in time, and take remedial measures in a timely manner according to the problems found.

[0082] On the other hand, the present application also provides a bridge construction facility and material positioning device for implementing the above-mentioned deep learning-based positioning method, as shown in Figure 3 The positioning device comprises:

[0083] An image acquisition module for acquiring images of the construction site;

[0084] A dataset processing module for pre-processing the point cloud data according to the acquired point cloud data of the construction site, and labeling the pre-processed point cloud dataset to obtain a semantic segmentation dataset;

[0085] a model training module, configured to construct a pre-training semantic segmentation model, and train the pre-training semantic segmentation model by using a semantic segmentation dataset to obtain an optimal pre-training semantic segmentation model;

[0086] a processing module, the processing module comprising a preprocessing unit and a semantic segmentation unit, the preprocessing unit being configured to extract point cloud data of a construction site according to the collected images and pre-process the point cloud data, and the semantic segmentation unit being configured to perform semantic segmentation on the pre-processed point cloud data by using the optimal semantic segmentation model to obtain a semantic segmentation file;

[0087] a positioning module, configured to position facilities and materials at the construction site based on a fast positioning algorithm;

[0088] a display module, configured to display a visual result generated by the positioning module.

[0089] In another aspect, the present application further provides a computer readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps of any of the positioning methods based on deep learning.

[0090] The various embodiments are described in the present specification in progressive order, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be mutually referred to. For the device disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, it is described more simply, and the relevant parts can be referred to the method part.

[0091] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for positioning bridge construction facilities and materials, characterized in that, Includes the following steps: Obtain a point cloud dataset of facilities and materials at the construction site, and annotate the point cloud dataset to obtain an annotated point cloud dataset; A point cloud semantic segmentation model is constructed, and the semantic segmentation model is trained using the labeled point cloud dataset to obtain the optimal point cloud semantic segmentation model. The point cloud data collected from the construction site is semantically segmented according to the optimal point cloud semantic segmentation model to generate a semantic segmentation file. The facilities and materials at the construction site are located based on a fast positioning algorithm; Generate visualization results based on the positioning results from the fast positioning algorithm; The facilities and materials at the construction site are located based on a fast positioning algorithm, specifically including: Read the semantic segmentation files of a facility or material category one by one; A Gaussian filter is used to reduce noise in the point cloud data of the semantic segmentation file. Calculate the distance between all point clouds in the facility or material category. If the distance is less than the distance threshold, determine that the point cloud belongs to the same object. Repeat this process to divide all points in the same category into several individual objects. Calculate the centroid of each of the individual objects, and use the centroid as the coordinate position of the individual object; After reading all facility and material data, the point cloud file categorized by terrain type is read last to generate visualization results.

2. The method for positioning bridge construction facilities and materials according to claim 1, characterized in that, Obtain a point cloud dataset of facilities and materials at the construction site, and annotate the point cloud dataset to obtain an annotated point cloud dataset, specifically including: The construction site was divided into several areas, and images of these areas were collected using drones. Information is extracted from the images collected by the UAV to generate point cloud data for several regions; The point cloud data of the aforementioned regions are labeled to generate a labeled point cloud dataset.

3. The method for positioning bridge construction facilities and materials according to claim 2, characterized in that, After extracting information from the image data collected by the UAV and generating point cloud data for several regions, the method further includes: preprocessing the point cloud data for several regions, wherein the preprocessing includes performing region cropping, point cloud noise reduction and point cloud downsampling on the point cloud data for several regions in sequence, and then outputting the point cloud data.

4. The method for positioning bridge construction facilities and materials according to claim 2, characterized in that, The labeled point cloud dataset includes the x, y, and z coordinates, RGB information, and category labels for each point cloud.

5. The method for positioning bridge construction facilities and materials according to claim 1, characterized in that, Metrics used to evaluate the semantic segmentation results of pre-trained semantic segmentation models include global accuracy. OA Category average accuracy mAcc and average intersection ratio mIoU The formula is shown below: ； ； ； in, IoU i For the first i The intersection-union ratio (IoU) of the model segmentation results and the dataset results for each class. A i For the first i The accuracy of point cloud classification in the model. C The total number of categories in the dataset. N T The number of correctly classified points. This represents the total number of all input points.

6. The method for positioning bridge construction facilities and materials according to claim 1, characterized in that, In the visualization results, facilities and materials of the same category are labeled with the same color, the coordinate values ​​of each facility or material are displayed, and the number of objects in each category is counted.

7. A method and apparatus for positioning bridge construction facilities and materials, characterized in that, The apparatus for implementing a method for positioning bridge construction facilities and materials as described in any one of claims 1 to 6, the apparatus comprising: The image acquisition module is used to acquire images of the construction site; The dataset processing module is used to extract point cloud data of the construction site based on the acquired images, preprocess the point cloud data, and annotate the preprocessed point cloud dataset to obtain an annotated point cloud dataset. The model training module is used to construct a pre-trained semantic segmentation model and train the pre-trained semantic segmentation model using the labeled point cloud dataset to obtain the optimal semantic segmentation model. The processing module includes a preprocessing unit and a semantic segmentation unit. The preprocessing unit is used to extract point cloud data of the construction site based on the acquired images and preprocess the point cloud data. The semantic segmentation unit is used to perform semantic segmentation on the preprocessed point cloud data using the optimal semantic segmentation model to obtain a semantic segmentation file. The positioning module is used to locate the facilities and materials at the construction site based on a fast positioning algorithm; The display module is used to display the visualization results generated by the positioning module.

8. A computer storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the method for positioning bridge construction facilities and materials as described in any one of claims 1 to 6.

Images