Knowledge-driven bridge construction progress automatic tracking method and device

Abstract

The application provides a kind of knowledge-driven bridge structure construction progress automatic tracking method and device, the method comprises: using the topography of bridge construction engineering, multi-source data of remote sensing image, unmanned aerial vehicle data, constructs the three-dimensional basic scene model for representing bridge construction site and surrounding environment topographic information and more detailed multi-level tilt three-dimensional model;Periodically obtained construction monitoring video is identified and feature extraction using an improved Yolo V5S model, obtain the identification extraction result of each bridge construction structure, accurately identify the structure being constructed in each construction stage from the identification extraction result based on knowledge-driven;On the basis of multi-level tilt three-dimensional model, the three-dimensional reconstruction of the dynamically changing structure being constructed is carried out, the dynamic registration fusion and updating of model can be realized, the actual construction progress of each stage can be displayed;Based on the comparison and analysis of the updated model of each construction stage and bridge BIM design model, the automatic tracking of construction progress is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of bridge engineering fine management and control technology, and particularly relates to a knowledge-driven bridge structure construction progress automatic tracking method and device. BACKGROUND

[0002] As an important traffic hub, the bridge has important significance for the construction of basic transportation infrastructure. More and more large bridges are under construction in remote and complex terrain environments, however, the complex engineering environment, high-difficulty structure design, and large-scale engineering construction result in long construction period, complicated construction process, and upgraded construction difficulty of the bridge engineering. The construction progress management directly affects the project economic benefits and is the core of the engineering construction management. If the bridge engineering construction progress cannot be timely mastered, there will be deviation, lag and out-of-control of the construction plan, which will seriously affect the allocation plan of the engineering human, material and financial resources, and seriously affect the bridge engineering progress and quality, and damage the economic benefits, image and reputation of the enterprise. Therefore, monitoring and tracking the entire bridge engineering construction progress, and timely and accurately mastering the engineering progress situation are crucial for the entire bridge engineering construction.

[0003] In the practice of engineering project management, the traditional construction progress management method mostly fills the engineering log report by hand, checks the site image file and data record by hand, and manages the project construction progress by using the Gantt chart method. The work efficiency is low, the subjective factors have great influence, and the construction progress information updating is prone to lag, which leads to difficulty in tracking and analyzing the construction progress. With the digitalization and informatization construction, the construction progress management also widely introduces informatization and BIM technology to improve the intelligent management level. Among them, the improved SOLOv2 algorithm is used to identify the construction progress, and the actual progress and the difference progress are integrated into the BIM model to realize the monitoring of the foundation construction progress of the construction site, but only the category of the completed structure can be identified, the actual size information of the structure in construction cannot be identified, and the specific feature information of the bridge construction structure cannot be accurately identified. Based on the time sequence and the plan model, the building model is dynamically generated, the construction progress is visualized in real time by combining the geographic information and the BIM model, the dynamic control model of the construction progress is constructed, and the BIM+GIS construction progress management system is formed, which can improve the progress optimization frequency and the efficiency of dynamic control of the progress. However, the system cannot timely feedback the visual model of the construction progress, the prefabricated model cannot be dynamically changed by real-time information, and the user's dynamic model perception effect of the construction construction process is poor. The use of simultaneous localization and mapping (SLAM) technology or three-dimensional laser scanning technology can quickly and accurately obtain the construction site progress information, and the project progress evaluation is performed by combining the BIM four-dimensional attribute and the 4D simulation model comparison analysis. However, the laser scanning and positioning equipment used for data collection has high cost, and the operation process is too complex.

[0004] In summary, the existing bridge structure construction progress management method has problems such as difficulty in accurately obtaining the bridge engineering construction progress, difficulty in timely dynamic adjustment of the construction plan, lack of three-dimensional feature information, and difficulty in balancing the construction progress management and cost control. SUMMARY

[0005] In view of this, the embodiments of the present application provide a knowledge-driven bridge structure construction progress automatic tracking method and device to eliminate or improve one or more defects in the prior art.

[0006] One aspect of the present application provides a knowledge-driven bridge structure construction progress automatic tracking method, which comprises:

[0007] determine the geographic spatial range of the bridge construction site, obtain the terrain model data and high-resolution remote sensing image in the geographic spatial range, and obtain a three-dimensional basic scene model of terrain spatial feature information for representing the bridge construction site and the surrounding geographical environment based on the terrain model data and the high-resolution remote sensing image;

[0008] determine the task route of the unmanned aerial vehicle based on the three-dimensional basic scene model, obtain the unmanned aerial vehicle image and the pose positioning data by flying the unmanned aerial vehicle according to the task route, and obtain a multi-level oblique three-dimensional model of spatial feature information for representing the bridge construction details based on the unmanned aerial vehicle image and the pose positioning data;

[0009] periodically obtain the construction monitoring video, construct an image sample training set and an image sample test set of the bridge construction structure based on the construction monitoring video, train the improved Yolo V5S model using the image sample training set, identify and extract the features of the bridge construction structure in the image sample test set using the trained improved Yolo V5S model, output the type and edge information of the bridge construction structure, and the improved Yolo V5S model adds a RepVGG module for identifying the edge contour of the bridge construction structure and a detection head for enhancing the identification ability of the RepVGG module to the edge contour of the bridge construction structure based on the Yolo V5S model;

[0010] perform knowledge reasoning on the type and edge information of the bridge construction structure to obtain the type and edge information of all currently constructed structures, and extract key feature information from the edge information of all currently constructed structures, wherein the bridge construction structure includes all currently constructed structures;

[0011] perform three-dimensional reconstruction on each constructed structure based on the type and key feature information of all currently constructed structures to obtain a plurality of local structure models, respectively register and fuse the plurality of local structure models with the multi-level oblique three-dimensional model, and periodically update the plurality of local structure models to obtain a plurality of bridge local structure dynamic change models;

[0012] obtain a bridge BIM design model, obtain the current actual construction progress and the planned construction progress of the bridge construction structure based on the plurality of bridge local structure dynamic change models and the bridge BIM design model, respectively, and obtain the bridge structure construction progress result by comparing and analyzing the current actual construction progress and the planned construction progress of each bridge construction structure.

[0013] In some embodiments of the present application, the multi-level oblique three-dimensional model of spatial feature information for representing the bridge construction details based on the unmanned aerial vehicle image and the pose positioning data comprises:

[0014] perform quality inspection and image correction preprocessing on the unmanned aerial vehicle image, and perform differential solution preprocessing on the pose positioning data;

[0015] The features of the pretreated unmanned aerial vehicle image and the pose positioning data are extracted by using a motion structure recovery algorithm to obtain a ground feature point cloud;

[0016] The ground feature point cloud is subjected to aerial triangulation encryption calculation to generate a grid model, and the grid model is subjected to texture mapping based on the pretreated unmanned aerial vehicle image to obtain a high-resolution three-dimensional network model;

[0017] The three-dimensional grid model is subjected to three-dimensional space subdivision by using the principle of octree data structure to obtain a multi-level inclined three-dimensional model with different resolutions.

[0018] In some embodiments of the present application, the improved Yolo V5S model comprises:

[0019] An input end module is configured to perform image enhancement, image scaling and normalization preprocessing on an input image sample;

[0020] A backbone network module comprises a plurality of feature extractors connected in sequence, each feature extractor comprising a plurality of convolution layers and pooling layers, and the plurality of feature extractors are configured to extract feature maps of different scales from the preprocessed image sample;

[0021] A neck network module comprises a multi-scale feature fusion module and a RepVGG module, the multi-scale feature fusion module is configured to perform multi-scale feature fusion on the feature maps of different scales and capture the detail information and semantic information of the bridge construction structure in the feature maps, and the RepVGG module is configured to perform edge contour detection of the bridge construction structure on the feature map output by the last feature extractor;

[0022] A detection head module comprises four detection heads, three of which comprise convolution layers and a loss function GIoU Loss, and are configured to perform target detection on the results output by the multi-scale feature fusion module to output the type and positioning result of the bridge construction structure; the fourth detection head comprises convolution layers and a cross-entropy loss discriminant function, and is configured to enhance the edge contour detection result output by the RepVGG module to output the edge information of the bridge construction structure, the edge information being represented in two-dimensional plane point cloud coordinates.

[0023] In some embodiments of the present application, the type and edge information of the bridge construction structure are subjected to knowledge reasoning to obtain the type and edge information of all the structures currently under construction, and key feature information is extracted from the edge information of all the structures currently under construction, comprising:

[0024] The knowledge interpreter fusing the experience knowledge of experts reasons the types and edge information of the bridge construction structure, and obtains the types and edge information of all the current construction structures;

[0025] The edge information of all the current construction structures is smoothed by using a Canny edge detection operator;

[0026] The key corner points of the smoothed edge information of all the current construction structures are extracted by using a Douglas-Puke algorithm, and two-dimensional plane coordinates of the key corner points of all the current construction structures are obtained.

[0027] In some embodiments of the present application, the three-dimensional reconstruction is performed on each construction structure based on the types and key feature information of all the current construction structures, a plurality of local structure models are obtained, the plurality of local structure models are respectively fused with the multi-level inclined three-dimensional model and are regularly updated, and a plurality of bridge local structure dynamic change models are obtained, including:

[0028] The database information of the bridge construction design structure is obtained, all the current construction structures are matched with the database information of the bridge construction design structure based on the types of all the current construction structures, and the geometric features of the current construction structures after completion of the design construction are obtained;

[0029] The bridge design and construction scheme, the construction technology and the exterior orientation element information of a plurality of cameras for obtaining the construction monitoring video are obtained, the numbering information of all the current construction structures is obtained based on the bridge design and construction scheme, the construction technology and the exterior orientation element information of the plurality of cameras, and the spatial position and attitude information of all the current construction structures in the entire bridge structure is obtained based on the numbering information;

[0030] The three-dimensional space coordinates of the key corner points are obtained by performing coordinate conversion on the two-dimensional plane coordinates of the key corner points of all the current construction structures based on the matching and alignment technology of the corner points;

[0031] The design size parameters of the bridge construction structure are obtained based on the bridge design and construction scheme and the construction technology, and the spatial semantic constraint relationship between the bridge construction structures is established based on the design size parameters of the bridge construction structure;

[0032] The three-dimensional reconstruction is performed on each construction structure based on the geometric features, the spatial position and attitude information of all the current construction structures in the entire bridge structure and the three-dimensional space coordinates of the key corner points, a plurality of local structure models are obtained, and the plurality of local structure models are respectively fused with the multi-level inclined three-dimensional model and are regularly updated based on the spatial semantic constraint relationship, so as to obtain a plurality of bridge local structure dynamic change models.

[0033] In some embodiments of the present application, the types of the bridge construction structures include at least a pile cap, a pier, a tower column, a bridge deck, and a box girder.

[0034] In some embodiments of the present application, the current actual construction progress and the planned construction progress of each bridge construction structure are obtained based on the plurality of dynamic change models of the bridge local structures and the bridge BIM design model, respectively, and the bridge construction progress result is obtained by comparing and analyzing the current actual construction progress and the planned construction progress of each bridge construction structure, including:

[0035] The spatial relationship between the bridge construction structures is analyzed based on the plurality of dynamic change models of the bridge local structures, and the three-dimensional reconstruction information of all the current construction structures and / or all the completed construction structures in the plurality of dynamic change models of the bridge local structures is extracted using expert experience knowledge;

[0036] The actual construction size of all the current construction structures and / or all the completed construction structures is obtained based on the spatial relationship between the bridge construction structures and the three-dimensional reconstruction information of all the current construction structures and / or all the completed construction structures;

[0037] The planned construction size of all the current construction structures and / or all the completed construction structures is obtained based on the bridge BIM design model;

[0038] The actual construction size and the corresponding planned construction size of all the current construction structures and / or all the completed construction structures are matched and compared based on the spatial relationship between the bridge construction structures, and the bridge construction progress result is obtained.

[0039] Another aspect of the present application provides a knowledge-driven bridge construction progress automatic tracking device, which comprises a computer device including a processor and a memory, the memory storing computer instructions, and the processor being configured to execute the computer instructions stored in the memory, so that the device implements the steps of the foregoing method.

[0040] Another aspect of the present application provides a computer readable storage medium storing a computer program, which is executed by a processor to implement the steps of the foregoing method.

[0041] Another aspect of the present application provides a computer program product comprising computer instructions, which are executed by a processor to implement the steps of the foregoing method.

[0042] Additional advantages, objects, and features of the application will be set forth in part by the description that follows, and will become apparent to those skilled in the art upon examination of the following detailed description and drawings in which illustrating the principles of the application. The objects and other advantages of the present application can be realized and attained by means of the structure particularly pointed out in the description and claims hereinafter.

[0043] Those skilled in the art will appreciate that the objects and advantages of the application can be realized and attained by means of the structure particularly pointed out in the appended claims. Understanding that these and other objects and advantages of the application can be achieved by the structures described above, the application is further described by the following detailed description and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0044] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description, serve to explain the principles of the application.

[0045] Figure 1 Flowchart of a knowledge-driven bridge structure construction progress automatic tracking method according to an embodiment of the present application;

[0046] Figure 2 Flowchart of a knowledge-driven bridge structure construction progress automatic tracking method according to an embodiment of the present application;

[0047] Figure 3 Network structure diagram of an improved Yolo V5S model according to an embodiment of the present application;

[0048] Figure 4 Process diagram of three branch reparameterization operations of a RepVGG module according to an embodiment of the present application;

[0049] Figure 5 Flowchart of step S140 according to an embodiment of the present application;

[0050] Figure 6 Flowchart of step S150 according to an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the embodiments and drawings. Herein, the illustrative embodiments of the present application and their descriptions are used to explain the present application, but are not intended to limit the present application.

[0052] Herein, it also needs to be noted that, in order to avoid the present application being obscured by unnecessary details, only the structures and / or processing steps closely related to the scheme according to the present application are shown in the drawings, and other details not closely related to the present application are omitted.

[0053] It should be emphasized that the term "comprises / comprising" when used in this specification is taken to specify the presence of stated features, elements, steps or components but does not preclude the presence or addition of one or more other features, elements, steps, components, or groups thereof.

[0054] Embodiments of the present application will be described below with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0055] In order to solve the problems of low intelligent management level, difficulty in accurately obtaining the bridge structure construction progress, time-consuming and laborious construction progress comparison calculation, and difficulty in balancing progress management and cost control in the existing bridge structure construction progress management method, an embodiment of the present application proposes a knowledge-driven bridge structure construction progress automatic tracking method and device, so that the construction plan can be dynamically adjusted in time according to the accurate bridge structure construction progress result obtained.

[0056] Figure 1 is a flowchart of a knowledge-driven bridge structure construction progress automatic tracking method of an embodiment of the present application, Figure 2 is a specific technical route diagram of a knowledge-driven bridge structure construction progress automatic tracking method of an embodiment of the present application. As shown in Figure 1 and Figure 2 The knowledge-driven bridge structure construction progress automatic tracking method of the present embodiment includes the following steps:

[0057] Step S110, determine the geographical space range of the bridge construction site, obtain the terrain model data and high-resolution remote sensing image in the geographical space range, and obtain the three-dimensional basic scene model for characterizing the spatial feature information of the bridge construction site and the surrounding geographical environment based on the terrain model data and the high-resolution remote sensing image.

[0058] Specifically, first, the geographical space range of the bridge construction site is determined, which includes the bridge construction area and the bridge surrounding geographical environment area, and then the high-precision terrain model data and the high-resolution remote sensing image in the geographical space range are obtained, wherein the high-precision terrain model data is a digital elevation model (DEM) or a digital surface model (DSM) or the like, which can be obtained through Google Maps. The high-resolution remote sensing image is sliced by using a quadtree division method to obtain a high-resolution remote sensing image slice, which can be texture mapped to a high-precision terrain model such as a digital elevation model or a digital surface model to obtain a three-dimensional basic scene model with a realistic terrain level. The three-dimensional basic scene model with the terrain level is used to characterize the relatively rough terrain spatial feature information of the bridge construction site and its surrounding large-scale geographical environment.

[0059] In step S120, the task flight path of the unmanned aerial vehicle is determined based on the three-dimensional basic scene model, the unmanned aerial vehicle image and the pose positioning data are obtained by the unmanned aerial vehicle flying according to the task flight path, and the multi-level oblique three-dimensional model for representing the spatial feature information of the bridge construction details is obtained based on the unmanned aerial vehicle image and the pose positioning data.

[0060] In an embodiment of the present application, the process of obtaining the multi-level oblique three-dimensional model for representing the spatial feature information of the bridge construction details based on the unmanned aerial vehicle image and the pose positioning data in step S120 specifically includes the following steps:

[0061] In step S122, the obtained unmanned aerial vehicle image is preprocessed by quality inspection and image correction, and the pose positioning data (POS data) is preprocessed by differential solution;

[0062] In step S123, the feature extraction is performed on the preprocessed unmanned aerial vehicle image and the POS data by using the structure from motion (SFM) algorithm, and the ground feature point cloud is obtained.

[0063] In step S124, the ground feature point cloud is subjected to aerial triangulation encryption solution to generate a grid model, and the high-definition texture mapping is performed on the network model based on the preprocessed unmanned aerial vehicle image, so as to obtain a high-resolution three-dimensional grid model.

[0064] In step S125, the three-dimensional space is divided by using the principle of octree data structure to obtain a multi-level oblique three-dimensional model with different resolutions, so as to realize the efficient organization and management of the three-dimensional model data. The different resolutions form different levels of the oblique three-dimensional model, and specifically represent the details of each construction structure of the bridge construction in a more refined manner.

[0065] The three-dimensional basic scene model of the terrain level and the multi-level inclined three-dimensional model of the bridge construction site can not only intuitively express the bridge construction environment, but also support simulation and progress deduction in the construction process, and provide a digital foundation for establishing a bridge construction intelligent site information management platform. The bridge construction intelligent site construction needs to timely and comprehensively perceive the construction environment and progress, and the commonly used monitoring methods include regular inspection by a drone and all-day video monitoring. Since the bridge construction is different from other building constructions, the occupied area is relatively narrow, and only the surrounding area of the bridge pier or tower column needs to be occupied for construction, and the surrounding other scenes are basically in a constant state. Therefore, when constructing the above-mentioned three-dimensional model of the bridge construction, the entire model does not need to be updated every time, and only the changed scene needs to be modeled. Therefore, the focus is on change detection of the regularly obtained drone images and construction monitoring videos, judging whether there is a changed scene (the subsequent structure under construction) in the region and identifying the changed scene, three-dimensional modeling of the changed scene, generating a local three-dimensional scene model, and fusing the local three-dimensional scene model with the city-level multi-level inclined three-dimensional model, and dynamically updating the local three-dimensional scene model, which can guarantee the quality of the three-dimensional model while reducing the workload and data volume of the three-dimensional modeling, and provide support for intelligent tracking of the bridge structure construction progress.

[0066] In step S130, the construction monitoring video is regularly obtained, the image sample training set and the image sample test set of the bridge construction structure are constructed based on the construction monitoring video, the improved Yolo V5S model is trained by using the image sample training set, the features of the bridge construction structure in the image sample test set are recognized and extracted by using the trained improved Yolo V5S model, and the type and edge information of the bridge construction structure are output, wherein the improved Yolo V5S model adds a RepVGG module for recognizing the edge contour of the bridge construction structure and a detection head for enhancing the recognition ability of the RepVGG module to the edge contour of the bridge construction structure on the basis of the Yolo V5S model.

[0067] Specifically, part of the regularly obtained construction monitoring video comes from a plurality of cameras / cameras carried on a drone (obtained by regular inspection by the drone), and the other part comes from a plurality of cameras / cameras arranged at different positions and different angles in the construction site for monitoring the bridge construction site. The images in the construction monitoring video contain various bridge construction structures (i.e., various structures constituting the entire bridge) to be constructed for building the bridge. By data enhancement on the images in the regularly obtained construction monitoring video, the image sample training set and the image sample test set of the bridge construction structure are constructed.

[0068] In an embodiment of the present application, as shown in Figure 3As shown, the improved YOLO V5S model includes: an input module, a backbone network module, a neck network module, and a detection head module. The input module performs preprocessing on the input image samples, such as image enhancement, image scaling, and normalization. The backbone network module includes multiple feature extractors connected in sequence. Figure 3 The example shown only includes five feature extractors (not limited to this example). Each feature extractor comprises multiple convolutional and pooling layers. These multiple feature extractors extract features from preprocessed image samples, resulting in feature maps at different scales, from low-level (small scale) to high-level (large scale). The neck network module includes a multi-scale feature fusion module and a RepVGG module. The multi-scale feature fusion module performs multi-scale feature fusion on feature maps of different scales through a series of upsampling (magnification) and downsampling (reduction) operations to further enhance the diversity and robustness of the feature maps, as well as feature fusion and feature enhancement operations. It captures detailed and semantic information of the bridge construction structure in the feature maps, outputting a multi-scale fused feature map that simultaneously contains both detailed and semantic information of the bridge construction structure. The RepVGG module is used to detect the edge contours of the bridge construction structure on the feature map output by the last feature extractor. The detection head module includes four detection heads, for example... Figure 3 The information detection shown is 1-4, of which three detection heads ( Figure 3 The information detection shown in 1-3 includes convolutional layers and a loss function GIoU Loss, used to perform target detection on the output of the multi-scale feature fusion module, outputting the type and location results of the bridge construction structure. Specifically, the convolutional layers identify specific patterns in the multi-scale fusion feature map output by the multi-scale feature fusion module, and the GIoU Loss loss function outputs accurate classification and location results, including the type of the bridge construction structure; the fourth detection head ( Figure 3The information detection 4) shown includes a convolutional layer and a cross-entropy loss discriminant function, which is used to enhance the edge contour detection result output by the RepVGG module, and output the edge information of the bridge construction structure. The edge information of the bridge construction structure includes key corner points, internal points, contours and edge points of the bridge construction structure, which are expressed in two-dimensional plane point cloud coordinates, that is, two-dimensional plane point cloud coordinates with spatial semantic information of the bridge construction structure. It can be seen that the recognition and extraction result output by the improved Yolo V5S model mainly includes the type and edge information of the bridge construction structure. Compared with the classic Yolo V5S model (the existing Yolo V5S model), the improved Yolo V5S model adds the RepVGG module as the target edge contour detection layer in the neck network of the classic Yolo V5S model, and introduces the cross-entropy loss discriminant function, so as to construct a new target edge detection head module as the fourth detection head, which can enhance the recognition ability of the model to the boundary contour of the bridge construction structure, so as to accurately extract the edge information of the bridge construction structure on the image. The existing Yolo V5S model can only recognize the type of the construction structure, and cannot effectively extract the edge feature information of the construction structure.

[0069] Specifically, the specific implementation process of the RepVGG module in the training and inference stages is as follows, wherein the inference stage is the test stage:

[0070] Training stage: design a RepVGG module with depth N=2, and the last feature extractor in the backbone network module (such as Figure 3 The output result of the feature extractor 5) shown is used as the input of the RepVGG module, and each depth has a 3*3 convolutional layer branch, a 1*1 convolutional layer branch and a direct connection branch, each branch is connected with a Batch Normalization (Batch Normalization, BN) layer and an activation function ReLU Figure 3 In the training stage, the three branch structures are used to extract feature information, first, the output results of the above three branches are normalized by the respective BN layers, and then the three normalized output results are combined together by addition operation. The addition operation here is element-wise, which means that the elements at the same position output by each branch are added to form a single output tensor. The combined output result is nonlinearly transformed by the activation function ReLU, and then transmitted to the next layer.

[0071] After the training is completed, the three branches are reparameterized, and the process is as follows Figure 4As shown, the 3*3 convolution layer branch, the 1*1 convolution layer branch and the direct connection branch are respectively transmitted to the corresponding BN layer for matching, and the special weight convolution layers of BN(3), BN(1) and BN(0) are obtained. Based on the special weight convolution layer of the output, the bias update and weight adjustment are carried out through the following BN fusion formula, and three branches form an equivalent single 3*3 convolution layer weight and bias, and finally three 3*3 convolution layer branches with bias vectors are output:

[0072]

[0073] Wherein, BN() represents the BN fusion inference function, M represents the input of the RepVGG module, μ represents the average value, σ represents the standard deviation, γ represents the learning scaling factor, β represents the bias, and i represents the branch number, that is, the channel index of the feature map represented by the three branches. Using the linear superposition principle of convolution, the weights and biases of the three 3*3 convolution layer branches with bias vectors are added respectively to generate a final 3*3 convolution layer, so as to realize the combination of the three 3*3 convolution layer branches into a 3*3 convolution layer output.

[0074] Inference stage: the RepVGG module directly uses the above-mentioned final combined 3*3 convolution layer to infer and predict the input M of the RepVGG module through the structure reparameterization idea, so as to realize the automatic conversion of the multi-path structure(three 3*3 convolution layer branches) of the improved Yolo V5S model in the training stage into a single-path structure(single 3*3 convolution layer) in the inference stage. Using the simplified single-path network structure of the RepVGG module for inference calculation can realize the ability to extract the complex edge contour of the bridge construction structure, ensure that the inference result reaches the effect of accurately extracting the complex edge contour of the bridge construction structure by the three branches in the training stage, and effectively improve the inference speed.

[0075] Based on the above introduced RepVGG module, the feature image of the feature extractor 5 super parameter is recalculated, the integration method of the small scale feature extracted by the last layer of the feature extractor is improved, and the quantization deep learning perception training of each kernel convolution layer parameter is constrained, so as to construct the corresponding fourth detection head, and combine the actual label one-hot encoding y i,j and the model prediction class probability P i,j The original target classification GIoU Loss loss function of the classic Yolo V5S model is replaced by the multi-class Cross-Entropy function as the loss function, as shown in the following formula:

[0076]

[0077] Wherein, Loss ce represents the loss function, yi,j represent the actual label one-hot encoding, P i,j The fourth detection head participates in the training of the front feedback neural network node together with the other three detection heads, learns and predicts more fine-grained target information, improves the prediction probability of the model on the edge features of the target, and realizes accurate identification of the edge contour of the bridge construction structure.

[0078] In step S140, the type and edge information of the bridge construction structure are subjected to knowledge reasoning to obtain the type and edge information of all the structures currently under construction, and key feature information is extracted from the edge information of all the structures currently under construction.

[0079] Specifically, the bridge construction structure refers to various construction structures to be constructed for building a bridge, i.e., various structures constituting a bridge, and the structure under construction refers to a structure currently under construction in the bridge construction structure.

[0080] In an embodiment of the present application, as shown in Figure 5 In step S140, the type and edge information of the bridge construction structure are subjected to knowledge reasoning to obtain the type and edge information of all the structures currently under construction, and key feature information is extracted from the edge information of all the structures currently under construction.

[0081] In step S141, the type and edge information of the bridge construction structure are subjected to reasoning by a knowledge interpreter fusing expert experience knowledge to obtain the type and edge information of all the structures currently under construction.

[0082] Specifically, after the inference phase of the improved Yolo V5S model ends, an expert knowledge interpreter fusing expert experience knowledge (i.e., converting the expert experience knowledge into a form recognizable by a machine) can be added to decode the output result of the improved Yolo V5S model, i.e., the type and edge information of the bridge construction structure, and to translate the two-dimensional planar point cloud coordinates with spatial semantic information of the bridge construction structure into two-dimensional planar point cloud coordinates with spatial semantic information of all the structures currently under construction, i.e., edge feature information of all the structures currently under construction with visual features and semantic features.

[0083] In step S142, the edge information of all the structures currently under construction is subjected to smoothing processing by a Canny edge detection operator.

[0084] Specifically, in the actual construction environment of the bridge structure, there may be the problem of shielding by scaffolding or green cloth, which causes random false edge noise points to interfere with the bridge construction structure edge information extracted by the deep learning network. Therefore, the Canny operator is introduced to set two thresholds, such as a first threshold and a second threshold, the first threshold being smaller than the second threshold. The planar point cloud between the two thresholds is taken as the two-dimensional planar point cloud of the structure under construction, so as to identify and connect the two-dimensional planar point cloud coordinates of the edge feature information (edge information). By removing the planar point cloud smaller than the first threshold and larger than the second threshold, the edge of the structure under construction is smoothed by removing redundant edge noise interference information, so as to realize the edge detection of the structure under construction.

[0085] In step S143, the Douglas-Peucker algorithm is used to extract the key corner points of the current all construction structure edge information after smoothing, to obtain the key corner point two-dimensional plane coordinates of the current all construction structure.

[0086] Specifically, in order to reduce the computational complexity and improve the detection efficiency, the Douglas-Peucker (DP) algorithm is introduced to calculate the topological coupling relationship between each feature point, analyze the morphological structure features of the feature points, realize the adaptive thinning of the target structure edge feature points, and realize the extraction of the key corner feature information of the construction structure contour by combining the constructed morphological and spatial semantic knowledge.

[0087] The Douglas-Peucker algorithm is used to extract the key corner point two-dimensional plane coordinates of the current all construction structure from the current all construction structure edge feature information after smoothing according to the following formula:

[0088]

[0089] Wherein, DP(P i ,P0,P n ) represents the perpendicular distance between point P i and line segment P0P n , P i represents the point to be calculated, P0 is the starting point of line segment P0P n , P n is the end point of line segment P0P n , and |(P n -P0)×(P n -P i )| represents the modulus of the cross product between two vectors. Whether DP(P i ,P0,P n ) is greater than the set threshold is judged to determine whether point P iWhether it is a key corner point. Taking a common rectangular box or cylindrical pier structure as an example, the projection image obtained by projecting the structure on the two-dimensional photo plane shot by the monitoring camera has a regular geometric shape feature, and the vertex or inflection point is extracted from a large number of two-dimensional point clouds on the edge of the structure by the DP algorithm. The bridge pier similar to the rectangular box or cylinder can be identified as the 7 vertices (the number of vertices is determined by the characteristics of the bridge construction structure and the shooting angle of the camera shooting the bridge construction structure) and their coordinates, that is, the key corner points.

[0090] Step S150, based on the type and key feature information of all the structures currently under construction, three-dimensional reconstruction is performed on each structure under construction to obtain a plurality of local structure models, and the plurality of local structure models are respectively registered and fused with the multi-level inclined three-dimensional model and periodically updated to obtain a plurality of bridge local structure dynamic change models.

[0091] In an embodiment of the present application, as shown in Figure 6 In step S150, based on the type and key feature information of all the structures currently under construction, three-dimensional reconstruction is performed on each structure under construction to obtain a plurality of local structure models, and the plurality of local structure models are respectively registered and fused with the multi-level inclined three-dimensional model and periodically updated to obtain a plurality of bridge local structure dynamic change models. The process specifically includes the following steps:

[0092] Step S151, obtaining the database information of the bridge construction design structure, based on the type of all the structures currently under construction, matching the database information of the bridge construction design structure with the current all structures under construction to obtain the corresponding design of the geometric features after the completion of the construction of the current all structures under construction.

[0093] Specifically, the database information of the bridge construction design structure is used to store the corresponding design of the geometric features after the completion of the construction of each bridge construction structure constituting the bridge designed for the construction of the bridge. The geometric features include various regular and irregular geometric shapes such as rectangles, circles, Y-shaped, U-shaped and H-shaped. The types of structures under construction include pile caps, piers, tower columns, bridge decks and box girders.

[0094] Step S152, obtaining the bridge design and construction scheme, construction technology and exterior orientation element information of the plurality of cameras for obtaining the construction monitoring video, based on the bridge design and construction scheme, construction technology and exterior orientation element information of the plurality of cameras, obtaining the numbering information of the current all structures under construction, and based on the numbering information, obtaining the spatial position and attitude information of the current all structures under construction in the entire bridge structure.

[0095] Specifically, a plurality of cameras for acquiring construction monitoring videos are arranged on the actual construction site of the bridge structure, and construction monitoring image samples are periodically acquired in a manner of real-time monitoring of the construction of each bridge construction structure by the plurality of cameras with different shooting angles, and in a manner of monitoring of each bridge construction structure by the plurality of oblique photography cameras with different shooting angles carried on the unmanned aerial vehicle when the unmanned aerial vehicle patrols according to the task route. The bridge design and construction scheme and the construction process include information (lens focal length and distortion coefficient, etc.) of the plurality of cameras arranged on the site and carried on the unmanned aerial vehicle for monitoring each bridge construction structure and exterior orientation element information of the cameras, type and number information corresponding to each bridge construction structure, and spatial position and spatial attitude information of each bridge construction structure in the entire bridge structure (i.e., spatial position and spatial attitude information of each bridge construction structure in the real world-actual construction environment), etc. The number information may be, for example, No. 1 pier, No. 2 pier, No. 3 pier, etc., or No. 1 tower column, No. 2 tower column, etc.

[0096] In step S153, the two-dimensional plane coordinates of the key corner points of all the structures under construction are converted into three-dimensional space coordinates based on the matching and alignment technology of the corner points.

[0097] Specifically, since the recognition and extraction result of the bridge construction structure is a two-dimensional image, and the key corner point coordinates of the structure under construction are also two-dimensional image coordinates, the key corner point coordinates need to be converted and three-dimensionally reconstructed for direct comparison and analysis with the BIM design model. Specifically, the specific process of the coordinate conversion includes: obtaining the spatial position and attitude (spatial position and angle information), lens focal length and distortion coefficient, etc. of the plurality of oblique photography cameras / cameras carried on the unmanned aerial vehicle and the plurality of cameras arranged on the construction site based on the unmanned aerial vehicle images and construction monitoring video data collected in the bridge construction process, establishing the camera intrinsic parameter matrix and the camera rotation matrix, calculating the rotation angles θ x , θ y and θ z of each camera along the X, Y and Z axes, and then converting the key corner point coordinates of the structure under construction from pixel coordinates to camera coordinates P C by the following formula:

[0098]

[0099] wherein f represents the camera focal length, f x and f y represent the focal length of the image in the x and y directions respectively, (u0, v0) represents the image coordinate system origin coordinates (principal point coordinates), (p.x, p.y) represents the pixel coordinates of the key corner point coordinates, (x c , yc z c ) represents the camera coordinate system coordinate with the camera optical center as the coordinate origin.

[0100] Next, the key corner point coordinates are converted from camera coordinates P C to world coordinates (P W ) by rotation in the order of Z, Y, and X axes using the following formula: W W W

[0101]

[0102] In step S154, the design size parameters of the bridge construction structure are obtained based on the bridge design and construction scheme and the construction process, and the spatial semantic constraint relationship between the bridge construction structures is established based on the design size parameters of the bridge construction structure.

[0103] Specifically, the design size parameters herein are parameters such as the length, width, and height of the construction structure, and the spatial semantic constraint relationship includes the strong coupling relationship and the spatial correlation relationship between the bridge construction structures. The top and bottom, left and right, and containing relationships of each bridge construction structure entity can be obtained through the topological relationship reasoning of the spatial semantic constraint relationship.

[0104] In step S155, each construction structure is three-dimensionally reconstructed based on the geometric features, the spatial position and attitude information of all the construction structures in the entire bridge structure, and the three-dimensional space coordinates of the key corner points, to obtain a plurality of local structure models. The plurality of local structure models are fused with the multi-level inclined three-dimensional model based on the spatial semantic constraint relationship, and are periodically updated to obtain a plurality of bridge local structure dynamic change models.

[0105] ​​​Specifically, the periodically updated process is to generate a local structure model of the structure under construction based on the geometric features obtained each time, the spatial position and attitude information of the structure under construction, and the three-dimensional spatial coordinates of the key corner points, fuse the local structure model generated each time with the model obtained after the previous fusion corresponding to the structure under construction (if there is no fusion last time, fuse with the multi-level inclined three-dimensional model), thereby obtaining the corresponding bridge local structure dynamic change model, and realizing the periodic update of the structure under construction on the three-dimensional model. Each bridge local structure dynamic change model is used to display different structures under construction, such as bridge piers and tower columns, and the construction progress of No. 1 pier, No. 2 pier, etc. In the process of automatically matching the three-dimensional reconstruction parameter information of the structure under construction to the corresponding bridge structure on the city-level multi-level inclined three-dimensional model, the three-dimensional reconstruction parameter information is restored to a three-dimensional model with actual meaning in combination with the aforementioned established spatial semantic constraint relationship, accurate registration of the structure can be realized, and parameterized three-dimensional reconstruction of the structure under construction can be realized.

[0106] In an embodiment of the present application, step S150 can further include the following steps:

[0107] In step S156, the Primitive API technology and point-line-surface graphic object programming method are used to simulate the contour of the irregular geometric model of the bridge, render and update the plurality of bridge local structure dynamic change models, and make them more visualized.

[0108] In an embodiment of the present application, the types of the bridge construction structure include a pile cap, a pier, a tower column, a bridge deck, and a box girder.

[0109] In step S160, the bridge BIM design model is obtained, the current actual construction progress and the planned construction progress of each bridge construction structure are obtained based on the plurality of bridge local structure dynamic change models and the bridge BIM design model, and the construction progress result of the bridge structure is obtained by comparing and analyzing the current actual construction progress and the planned construction progress of each bridge construction structure.

[0110] In an embodiment of the present application, the process of obtaining the current actual construction progress and the planned construction progress of each bridge construction structure based on the plurality of bridge local structure dynamic change models and the bridge BIM design model in step S160, and obtaining the construction progress result of the bridge structure by comparing and analyzing the current actual construction progress and the planned construction progress of each bridge construction structure, specifically includes the following steps:

[0111] Step S162, analyzing the spatial relationship between the bridge construction structures based on the plurality of bridge local structure dynamic change models, and extracting the three-dimensional reconstruction information of all current structures under construction and / or all completed construction structures in the plurality of bridge local structure dynamic change models using expert experience knowledge.

[0112] Specifically, based on the plurality of bridge local structure dynamic change models corresponding to the current all structures under construction, the bounding box outside the model is derived, which is used to analyze the spatial relationship between different structures, such as the up-down and left-right adjacent relationship of the structure entity. The spatial relationship between structures is determined by the bounding box, and the spatial distance between two points is calculated according to the following formula according to the center A(x A ,y A ,z A ) of the bounding box of structure A and the center B(x B ,y B ,z B ) of the bounding box of structure B:

[0113]

[0114] If the above spatial distance is less than or equal to the sum of the radii R of the two structures A and B, there is no adjacent relationship between structures A and B, and if the condition is met, a spatial plane P parallel to the XOY plane and passing through the point B(x B ,y B ,z B ) is constructed according to the center of the bounding box of structure B (assuming that the z B value of the center of the bounding box of structure B is the smallest), and the plane equation is: Z=z B ; then connecting the point B(x B ,y B ,z B ) and the point A(x A ,y A ,z A ), a spatial straight line L is constructed, as shown in the following formula:

[0115] (X-x B ) / (x A -x B )=(Y-y B ) / (y A -y B )=(Z-z B ) / (z A -z B )

[0116] Finally, the relationship between the straight line and the spatial plane is determined according to the following formula:

[0117]

[0118] wherein, represents the normal vector of the spatial plane P represents the direction vector of the spatial straight line L, and is α represents the included angle between the spatial straight line L and the spatial plane P, if sinα = 0, represents the normal vector and the direction vector are parallel relationship, further, structure A and structure B are upper and lower adjacent relationship; if sinα = 1, represents the normal vector and the direction vector are perpendicular relationship, further, structure A and structure B are left and right adjacent relationship. The above expert experience knowledge can accurately identify, distinguish and judge the construction structure being constructed and the construction structure completed.

[0119] Step S163, obtaining the actual construction size of the current all construction structures being constructed and / or the current all construction structures completed based on the spatial relationship between the bridge construction structures, the three-dimensional reconstruction information of the current all construction structures being constructed and / or the current all construction structures completed.

[0120] By assisting the calculation of the actual construction size of each construction structure being constructed and / or each construction structure completed through the spatial relationship between the bridge construction structures, the accuracy of the calculation can be improved.

[0121] Step S164, obtaining the planned construction size of the current all construction structures being constructed and / or the current all construction structures completed based on the bridge BIM design model.

[0122] Step S165, matching and comparing the actual construction size of the current all construction structures being constructed and / or the current all construction structures completed with the corresponding planned construction size based on the spatial relationship between the bridge construction structures, to obtain the bridge structure construction progress result.

[0123] By assisting the matching of the dynamic change model of all bridge local structures in each construction stage with the BIM design model and the comparison and analysis of the construction progress through the spatial relationship between the bridge construction structures, not only the actual construction situation and the design situation of the bridge structure in each construction stage can be clearly displayed at the same time, but also the accuracy of the matching can be improved and the accurate bridge construction overall construction progress result can be obtained, to realize the automatic tracking of the bridge structure construction progress.

[0124] In addition, the three-dimensional model of the structure under construction can be obtained by removing the completed and under-construction three-dimensional structure model in the bridge BIM design model, the remaining part can be selected to display the structure under construction in the bridge construction project, and the three-dimensional model of the structure under construction can be intuitively displayed by using semi-transparency or other colors. The method of the embodiment of the application can also automatically generate a bridge construction progress report and compare it with the construction plan, and then dynamically adjust the construction plan and the construction scheme according to the comparison result, so as to automatically track the construction progress of the bridge structure. If the construction progress is consistent with the construction plan, the construction is carried out according to the original plan, otherwise, the construction plan is ahead of or behind the construction plan, and then the reason can be found according to the construction situation, and the construction plan and the construction scheme are dynamically adjusted.

[0125] The above knowledge-driven bridge structure construction progress automatic tracking method of the embodiment of the application fully utilizes the existing terrain, satellite remote sensing image, unmanned aerial vehicle image data, and video monitoring image of the bridge construction project, and multi-source data such as pose positioning data, to construct a three-dimensional basic scene model of the bridge construction terrain level and a multi-level inclined three-dimensional model with more construction details and finer details. The improved Yolo V5S model is used to identify and extract features of the construction monitoring video image data obtained periodically, to obtain identification and extraction results of each bridge construction structure containing type and edge information, to accurately identify the structure under construction, i.e., the changing part, in each construction stage based on knowledge driving. The parameterized three-dimensional reconstruction is performed on the dynamically changing structure under construction based on the multi-level inclined three-dimensional model, to realize dynamic registration fusion and updating of the model, and to display the actual construction progress in each stage. Based on the comparison and analysis of the updated model in each construction stage and the bridge BIM design model, the automatic tracking of the construction progress is realized, and the construction progress report is automatically generated, which reduces the cost of data acquisition and data calculation, simplifies the workload of the bridge construction personnel in progress management, reduces the operation requirements of the construction personnel for complex calculation of big data information, and provides technical support for the intelligent construction site construction of modern bridge engineering.

[0126] Corresponding to the above method, a knowledge-driven bridge structure construction progress automatic tracking device of an embodiment of the application comprises a computer device, the computer device comprises a processor and a memory, the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device realizes the steps of the foregoing method.

[0127] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the steps of the foregoing method. The computer readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0128] The embodiment of the present application further provides a computer program product, which comprises computer instructions. The computer instructions are executed by a processor to implement the steps of the foregoing method.

[0129] Those skilled in the art should understand that each example component, system and method described in connection with the embodiments disclosed herein can be implemented in hardware, software or a combination of both. The actual implementation depends on the specific application and design constraints imposed on the overall system. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the present application are program or code segments used to perform the required tasks. The program or code segments can be stored in a machine readable medium or transmitted through a data signal carried in a carrier wave in a transmission medium or communication link.

[0130] It should be noted that the present application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of well-known methods are omitted. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order of the steps, after understanding the spirit of the present application.

[0131] In the present application, the features described and / or illustrated for one embodiment can be used in the same way or in a similar way in one or more other embodiments, and / or in combination with or instead of features of other embodiments.

[0132] The above description is merely preferred embodiments of the present application, and is not intended to limit the present application. The embodiments of the present application can be variously changed and modified by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall fall within the scope of the present application.

Claims

1. A knowledge driven bridge construction progress automatic tracking method, characterized in that, The method comprises: determine the geographic space range of the bridge construction site, obtain the terrain model data and high-resolution remote sensing image in the geographic space range, and obtain the three-dimensional basic scene model of the terrain space feature information for characterizing the bridge construction site and the surrounding geographical environment based on the terrain model data and the high-resolution remote sensing image; determine the task route of the unmanned aerial vehicle based on the three-dimensional basic scene model, obtain the unmanned aerial vehicle image and the pose positioning data by flying the unmanned aerial vehicle according to the task route, and obtain the multi-level oblique three-dimensional model of the space feature information for characterizing the bridge construction details based on the unmanned aerial vehicle image and the pose positioning data; periodically obtain the construction monitoring video, construct the image sample training set and the image sample test set of the bridge construction structure based on the construction monitoring video, train the improved Yolo V5S model using the image sample training set, identify and extract the features of the bridge construction structure in the image sample test set using the trained improved Yolo V5S model, output the type and edge information of the bridge construction structure, and the improved Yolo V5S model adds a RepVGG module for identifying the edge contour of the bridge construction structure and a detection head for enhancing the identification ability of the RepVGG module to the edge contour of the bridge construction structure based on the Yolo V5S model; knowledge reasoning is performed on the type and edge information of the bridge construction structure to obtain the type and edge information of all currently constructed structures, and key feature information is extracted from the edge information of all currently constructed structures, wherein the bridge construction structure includes all currently constructed structures; based on the type and key feature information of all currently constructed structures, three-dimensional reconstruction is performed on each constructed structure to obtain a plurality of local structure models, the plurality of local structure models are respectively registered and fused with the multi-level oblique three-dimensional model and periodically updated to obtain a plurality of bridge local structure dynamic change models; obtain the bridge BIM design model, obtain the current actual construction progress and the planned construction progress of the bridge construction structure based on the plurality of bridge local structure dynamic change models and the bridge BIM design model respectively, and obtain the bridge structure construction progress result by comparing and analyzing the current actual construction progress and the planned construction progress of each bridge construction structure.

2. The method of claim 1, wherein, The multi-level oblique three-dimensional model of the space feature information for characterizing the bridge construction details based on the unmanned aerial vehicle image and the pose positioning data comprises: preprocessing of quality inspection and image correction is performed on the unmanned aerial vehicle image, and differential solution preprocessing is performed on the pose positioning data; features of the preprocessed unmanned aerial vehicle image and the pose positioning data are extracted using a motion structure recovery algorithm to obtain ground feature point cloud; ground feature point cloud is encrypted and solved in space, a grid model is generated, the grid model is texture mapped based on the preprocessed unmanned aerial vehicle image, and a three-dimensional network model is obtained; the three-dimensional grid model is divided in three-dimensional space using the principle of octree data structure to obtain a multi-level oblique three-dimensional model with different resolutions.

3. The method of claim 1, wherein, The improved Yolo V5S model comprises: An input module is configured to perform image enhancement, image scaling and normalization preprocessing on input image samples; A backbone network module includes a plurality of feature extractors connected in sequence, each feature extractor including a plurality of convolutional layers and pooling layers, and the plurality of feature extractors are configured to extract feature maps of different scales from the preprocessed image samples; A neck network module includes a multi-scale feature fusion module and a RepVGG module, the multi-scale feature fusion module is configured to perform multi-scale feature fusion on the feature maps of different scales and capture the detail information and semantic information of the bridge construction structure in the feature maps; the RepVGG module is configured to perform edge contour detection of the bridge construction structure on the feature map output by the last feature extractor; A detection head module includes four detection heads, three of which include convolutional layers and a loss function GIoU Loss, configured to perform target detection on the results output by the multi-scale feature fusion module, and output the type and positioning results of the bridge construction structure; the fourth detection head includes convolutional layers and a cross-entropy loss discriminant function, configured to enhance the edge contour detection results output by the RepVGG module, and output the edge information of the bridge construction structure, which is represented by two-dimensional planar point cloud coordinates.

4. The method of claim 3, wherein, The type and edge information of the bridge construction structure are subjected to knowledge reasoning to obtain the type and edge information of all current construction structures, and key feature information is extracted from the edge information of all current construction structures, including: A knowledge interpreter that integrates expert experience knowledge is used to reason the type and edge information of the bridge construction structure to obtain the type and edge information of all current construction structures; A Canny edge detection operator is used to smooth the edge information of all current construction structures; A Douglas-Puik algorithm is used to extract key corner points from the smoothed edge information of all current construction structures to obtain two-dimensional planar coordinates of key corner points of all current construction structures.

5. The method of claim 4, wherein, Based on the type and key feature information of all current construction structures, a three-dimensional reconstruction is performed on each construction structure to obtain a plurality of local structure models, which are respectively registered and fused with a multi-level inclined three-dimensional model and periodically updated to obtain a plurality of bridge local structure dynamic change models, including: Database information of bridge construction design structures is obtained, and based on the type of all current construction structures, the current construction structures are matched with the database information of the bridge construction design structures to obtain the geometric features of the current construction structures after completion of the corresponding design construction; Bridge design and construction schemes, construction processes and exterior orientation element information of a plurality of cameras used to obtain construction monitoring videos are obtained, based on the bridge design and construction schemes, the construction processes and the exterior orientation element information of the plurality of cameras, the numbering information of all current construction structures is obtained, and based on the numbering information, the spatial position and attitude information of all current construction structures in the entire bridge structure is obtained. The key corner point two-dimensional plane coordinates of all the structures under construction are converted into three-dimensional space coordinates of the key corner points based on the corner point matching and alignment technology; The design size parameters of the bridge construction structures are obtained based on the bridge design and construction scheme and the construction process, and the spatial semantic constraint relationship between the bridge construction structures is established based on the design size parameters of the bridge construction structures; Based on the geometric features, the spatial position and attitude information of all the structures under construction in the entire bridge structure, and the three-dimensional space coordinates of the key corner points, the three-dimensional reconstruction of each structure under construction is performed to obtain a plurality of local structure models, and the plurality of local structure models are fused with the multi-level inclined three-dimensional model based on the spatial semantic constraint relationship and are updated regularly to obtain a plurality of bridge local structure dynamic change models.

6. The method of claim 3, wherein, The types of the bridge construction structures include at least a pile cap, a pier, a tower column, a bridge deck, and a box girder.

7. The method of claim 1, wherein, The current actual construction progress and the planned construction progress of the bridge construction structures are obtained based on the plurality of bridge local structure dynamic change models and the bridge BIM design model, respectively, the current actual construction progress and the planned construction progress of each bridge construction structure are compared and analyzed to obtain a bridge structure construction progress result, including: The spatial relationship between the bridge construction structures is analyzed based on the plurality of bridge local structure dynamic change models, and the three-dimensional reconstruction information of all the structures under construction and / or all the completed construction structures in the plurality of bridge local structure dynamic change models is extracted based on the expert experience knowledge; The actual construction size of all the structures under construction and / or all the completed construction structures is obtained based on the spatial relationship between the bridge construction structures and the three-dimensional reconstruction information of all the structures under construction and / or all the completed construction structures; The planned construction size of all the structures under construction and / or all the completed construction structures is obtained based on the bridge BIM design model; The actual construction size and the corresponding planned construction size of all the structures under construction and / or all the completed construction structures are matched and compared based on the spatial relationship between the bridge construction structures to obtain the bridge structure construction progress result.

8. A knowledge driven bridge construction progress automatic tracking apparatus comprising a processor and a memory, characterized in that, The memory stores computer instructions, and the processor is configured to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the device implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method according to any one of claims 1 to 7.

10. A computer program product comprising computer instructions, characterized in that, The computer instructions are executed by the processor to implement the steps of the method according to any one of claims 1 to 7.

Images