Bridge construction progress intelligent recognition method based on improved yolov5s

Abstract

This invention discloses an intelligent bridge construction progress recognition method based on an improved YOLOv5S model. This method addresses the shortcomings of existing technologies, which lack the ability to identify construction components and intelligently calculate construction progress, making it difficult to quickly and accurately obtain bridge construction progress. The invention obtains current construction component information based on a constructed bridge component dataset, acquired video monitoring data from the bridge construction site, and an improved YOLOv5S model. Based on the current construction component information obtained in step 1 and spatial semantic constraint rules, the bridge construction progress is calculated to obtain the current construction progress. This invention is used for intelligent bridge construction progress recognition.

Description

Technical Field

[0001] A bridge construction progress intelligent identification method based on improved YOLOv5S is proposed, which is used for intelligent identification of bridge construction progress and belongs to the field of bridge engineering management technology. Background Technology

[0002] my country's transportation infrastructure construction has developed rapidly, with bridge construction entering a new stage of development, and the number and scale of construction projects continuously expanding. Construction progress monitoring and management is a crucial component of bridge engineering construction. Fast and accurate construction progress monitoring and management methods can help project managers understand the construction progress in a timely manner and make scientific and reasonable construction decisions. However, bridge construction is a complex and ever-changing process. Bridge engineering projects are characterized by long construction periods, complex construction technologies, and numerous participating entities. Furthermore, the real-time information generated by bridge construction monitoring is dynamic and voluminous, placing higher demands on bridge construction progress management. Traditional bridge construction progress management focuses primarily on two-dimensional displays, such as Gantt charts, S-curve comparison methods, and banana-shaped curve comparison methods, with limited visualization of construction progress information and a low level of intelligence in progress management. Existing intelligent project management mainly focuses on using images and videos as information carriers to record the on-site construction environment over long periods and on a large scale, but lacks the ability to identify construction components and intelligently calculate construction progress, making it difficult to quickly obtain bridge construction progress. Therefore, how to efficiently and accurately identify bridge construction components and calculate construction progress is a critical issue.

[0003] Therefore, existing technologies for identifying bridge construction progress from video images have the following technical problems:

[0004] 1. Lack of identification of construction components and intelligent calculation of construction progress (traditional construction progress is often recorded in the form of two-dimensional reports, which is time-consuming and labor-intensive), making it difficult to quickly and accurately obtain the bridge construction progress;

[0005] 2. It cannot meet the need for rapid and accurate detection of construction progress at different times, which is not conducive to the decision-making of managers. Summary of the Invention

[0006] The purpose of this invention is to provide an intelligent identification method for bridge construction progress based on an improved YOLOv5S, which solves the problem that the existing technology lacks the ability to identify construction components and intelligently calculate construction progress, making it difficult to quickly and accurately obtain bridge construction progress.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A method for intelligent identification of bridge construction progress based on an improved YOLOv5S includes the following steps:

[0009] Step 1. Based on the constructed bridge component dataset, the acquired video monitoring data of the bridge construction site, and the improved YOLOv5S model, obtain the current construction component information;

[0010] Step 2. Based on the current construction component information and spatial semantic constraint rules obtained in Step 1, calculate the bridge construction progress to obtain the current construction progress.

[0011] Furthermore, the improved YOLOv5S model in step 1 is based on the YOLOv5S model, by adding an attention module SimAM after each C3 module and at the very end of the backbone network of the YOLOv5S model, so that the convolutional module Conv, the C3 module and the attention module SimAM are integrated into the CCS module.

[0012] Furthermore, the specific steps of step 1 are as follows:

[0013] Step 1.1. Determine the main structure of the bridge construction and construct a bridge component dataset for target detection based on the acquired bridge images. The main structure included in the bridge component dataset includes piers, abutments, pile caps, bridge deck pavement, and ancillary facilities.

[0014] Step 1.2. Train the improved YOLOv5S model based on the bridge component dataset to obtain the trained improved YOLOv5S model;

[0015] Step 1.3. Obtain video surveillance data from the bridge construction site, extract key frame images and time information from the video surveillance data, and name the key frame images with the extracted time, so that the key frame images correspond one-to-one with the time information;

[0016] Step 1.4. Input the named keyframe images into the trained improved YOLOv5S model to identify the components that are under construction or have been completed during this period, and obtain the current construction component information, including the category, location information and number of components of each main structure.

[0017] Furthermore, the specific steps of step 1.1 are as follows:

[0018] Step 1.11. Perform Mosaic data augmentation on the acquired bridge images. This involves randomly scaling, cropping, and arranging four bridge images randomly from the acquired bridge images, and then stitching the four processed bridge images together into a new bridge image.

[0019] Step 1.12. Determine the main structure of the bridge construction, use the open-source image annotation tool LabelImg to mark each main structure in the new bridge image, obtain the label category, and save the annotation in PASCAL VOC format into an .xml file. The .xml file records the size, number of channels, label name and selection box position of the new bridge image after marking.

[0020] Step 1.13. Use Python to convert the .xml file to txt format, which gives you the bridge component dataset for object detection.

[0021] Furthermore, the specific steps of step 2 are as follows:

[0022] Step 2.1. Analyze the relationships between various components of the bridge based on the bridge construction plan and BIM design model;

[0023] Step 2.2. Based on the results of the association analysis, use knowledge graph technology to construct a knowledge graph and establish spatial semantic constraint rules;

[0024] Step 2.3. Calculate the bridge construction progress based on the current construction component information and spatial semantic constraint rules to obtain the current construction progress.

[0025] Furthermore, the specific steps of step 2.1 are as follows:

[0026] Step 2.11. Define the ontological concept objects and divide them from top to bottom according to the bridge engineering construction plan. Determine the sub-objects of each ontological concept object. The ontological concept objects include bridge structure, construction procedures and schedule. The sub-objects of bridge structure include superstructure, substructure, bearings and ancillary facilities. The sub-objects of construction procedures include overall project overview, expected completion time, main construction sequence and construction stages. The sub-objects of schedule include construction operation indicators, main construction period arrangement and construction schedule calendar arrangement. The bridge engineering construction plan includes construction organization plan and construction schedule table.

[0027] Step 2.12. After manually extracting the entities, relationships and attributes between each ontology concept object, describe the attribute characteristics of each main structure under construction based on the BIM design model and clarify the semantic relationships between each ontology concept object in the bridge construction scenario, that is, describe the construction sequence relationship and spatial relationship between each main structure during the bridge construction process.

[0028] Furthermore, the specific steps of step 2.2 are as follows:

[0029] Step 2.21. Construct a knowledge graph based on the results of the relationship analysis;

[0030] Step 2.22. Based on the knowledge graph, the construction progress of the bridge project is roughly determined by construction constraints in a bottom-up order. That is, based on the constraints, the construction scenario is divided into substructure construction, bearing construction and superstructure construction in sequence, and the construction progress of the current construction component information is roughly determined from the perspective of construction procedures.

[0031] Step 2.23. In the BIM design model that describes semantic relationships, the initially known design dimensions and measured dimensions are used as constraints to make a precise determination of the construction progress of the main structure under construction. That is, it is used to make a precise determination of the construction progress based on the design dimensions and measured dimensions of each main structure under construction.

[0032] Furthermore, the specific steps of step 2.3 are as follows:

[0033] Step 2.31. Under the constraints of coarse judgment, based on the construction sequence of each main structure in the current construction component information, and in combination with the bridge engineering construction plan, determine the construction progress of each main structure;

[0034] Step 2.32. Under the constraints of fine determination, based on the main structure under construction obtained after coarse determination, calculate the actual two-dimensional dimensions of each main structure under construction according to the photographic scale conversion coefficient between the design dimensions of each main structure under construction and the measured dimensions monitored by the surveillance camera.

[0035] Furthermore, the specific steps of step 2.31 are as follows:

[0036] When multiple main structures of the same type are detected, the number of these structures is compared with the number of similar components in the bridge construction plan. If the numbers are equal, construction of this type of component is completed; if the number is less than the number of similar components in the bridge construction plan, the progress is... If the quantity is greater than the quantity of similar components in the bridge construction plan, the inspection is incorrect and the inspection should be repeated.

[0037] The specific steps of step 2.32 are as follows:

[0038] The actual two-dimensional dimensions of each main structure under construction are calculated based on the photographic scale conversion factor between the design dimensions of each main structure under construction and the measured dimensions monitored by surveillance cameras:

[0039] The formula for calculating the actual size based on the photographic scale conversion factor is as follows:

[0040] Actual size (meters) = Bridge image pixel size ÷ Photographic scale conversion factor × 0.001.

[0041] Furthermore, it also includes step 3: comparing the construction progress based on the current construction component information with the corresponding planned construction information to determine whether the current construction status of each main component is normal or stalled, and updating each main structure in real time in the BIM design model.

[0042] Compared with the prior art, the advantages of the present invention are as follows:

[0043] I. This invention introduces a Transformer-like attention mechanism model into the backbone network of YOLOv5S for feature fusion. In the extraction of the main structure of a bridge, it can effectively extract global feature information, thereby facilitating accurate judgment of the subsequent bridge construction progress.

[0044] Second, this invention constructs a knowledge graph of bridge construction procedures and schedule plans, analyzes and mines bridge engineering construction data, and can clearly clarify the relationship between various components (i.e., various main structures) of the bridge. Introducing the knowledge graph into the field of bridge engineering construction schedule calculation can build spatial semantic constraint rules for construction schedule calculation and improve the efficiency of construction schedule calculation. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of the overall framework of the present invention;

[0047] Figure 2 This is a structural diagram of the improved YOLOv5S model in this invention. In the diagram, Focus is a special convolutional layer in the YOLOv5S network, C3 is a residual structure used to divide the input feature map into two parts. One part is directly connected to the output, and the other part is processed by multiple bottleneck layers and convolutional layers before being added to the output to improve feature fusion capability. SPPF is a spatial pyramid pooling layer module used to obtain feature maps of different scales by using max pooling operations of different sizes, and then stitching them together to increase the receptive field and feature diversity.

[0048] Figure 3 This is a schematic diagram illustrating the construction of the main structure dataset, i.e., the bridge component dataset, in bridge construction according to the present invention.

[0049] Figure 4 This is a schematic diagram illustrating the conceptual objects and semantic descriptions of bridge construction scenarios in this invention;

[0050] Figure 5 This is a schematic diagram of the spatial semantic constraint rules in this invention;

[0051] Figure 6 This is the construction progress calculation process in this invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] This invention conducts research on the accurate identification of construction components and intelligent calculation of construction progress based on real-time image data from video monitoring of bridge construction sites. It proposes an intelligent identification method for bridge construction progress based on an improved YOLOv5S, which can further realize intelligent monitoring of bridge construction progress and is of great significance for achieving refined and intelligent management of construction progress.

[0054] The overall research approach is as follows: Figure 1 As shown, this mainly includes two aspects: improving the YOLOv5S model for accurate detection of construction components and intelligent recognition of construction progress based on spatial semantic constraints. The first aspect involves obtaining current construction component information based on a constructed bridge component dataset, acquired video surveillance data from bridge construction sites, and an improved YOLOv5S model. Firstly, the YOLOv5S model is improved by incorporating an attention mechanism module into its backbone network to achieve bridge feature fusion and maximize the preservation of global features. Secondly, the main structure of the bridge construction is determined, and a bridge component dataset for target detection is constructed. Then, video surveillance data from the bridge construction site is accessed, and image processing is performed on the collected data (bridge images). Based on the collected construction image data (bridge images), the improved YOLOv5S model is used to detect the main structure of the bridge construction, obtaining information such as the category, quantity, and location of the current construction components (i.e., the completed and / or under-construction main structures). Secondly, based on the obtained current construction component information and spatial semantic constraint rules, the bridge construction progress is calculated to obtain the current construction progress: First, the relationship between various bridge components (in this case, the main structure) is analyzed based on the bridge engineering construction plan and BIM design model; then, knowledge graph technology is used to construct a knowledge graph and establish spatial semantic constraint rules to provide a knowledge foundation for intelligent calculation of bridge construction progress; the bridge construction progress is compared and calculated by combining the identification results of bridge construction components and spatial semantic constraint rules.

[0055] Improve the accuracy of construction component detection in YOLO V5S. This involves obtaining current construction component information based on a constructed bridge component dataset, acquired video monitoring data from bridge construction sites, and an improved YOLO V5S model. The details are as follows:

[0056] Improve the backbone network of YOLO V5S:

[0057] The YOLOv5S model has achieved excellent results in object detection, but the diverse types of objects and complex environments at bridge construction sites significantly increase the difficulty of detection. Furthermore, the unclear feature information of construction components, coupled with significant differences in target scale, leads to low accuracy in feature extraction from construction components in videos using the YOLOv5S model. The YOLOv5S model's feature extraction backbone network uses a single convolutional kernel, resulting in a limited receptive field and insufficient control over global feature information in the construction scene. To address these issues, this invention proposes fusing the parameter-free attention mechanism (i.e., the SimAM attention module) from the Transformer type with the YOLOv5S model's backbone network. This increases the network model's global perspective, enabling feature fusion of construction components and grasping both local and global features. The improved YOLOv5S model's network structure is as follows: Figure 2 As shown. This invention utilizes the idea of ​​adding small-module networks, adding an attention module SimAM after each C3 module and at the very end of the backbone network of the YOLOv5S model, integrating the convolutional module Conv, the C3 module, and the attention module SimAM into a CCS module. Adding the attention module SimAM after each C3 module allows the attention mechanism to see local features, and adding it at the very end allows the attention mechanism to see the feature map of the entire backbone network. Without additional parameters, it extracts more feature information of construction components at the same location in different channels, providing a global perspective. The attention module SimAM in the YOLOv5S model can infer the 3D attention weights of the feature map in each layer without introducing additional parameters.

[0058] In neuroscience, active neurons can inhibit the activity of surrounding neurons; this phenomenon is called spatial inhibition. Therefore, neurons exhibiting significant spatial inhibition effects in visual tasks should be given higher priority. The attention module SimAM measures the linear separability of a target neuron from other neurons using a defined energy function, identifying the highest-priority neuron and assigning a unique weight to each neuron in the feature map to estimate the importance of individual neurons in the feature map. The minimum energy calculation function is as follows:

[0059]

[0060] In the formula, t refers to the target neuron of the input feature X. and It represents the average and variance of all neurons in a certain channel, where M = height × width of the input feature X, and x is the number of neurons in that channel; i For each channel of the input feature X, there are other neurons besides the target neuron t. The hyperparameter λ is a balancing factor obtained through a network model search; the specific value of λ depends on the different networks and tasks. For example, on the ImageNet classification task, the λ parameter value is 0.25 for the ResNet-50 network and 0.5 for the ResNeXt-50 network. From the energy function in formula (1), we can see that the energy... The lower the value, the greater the difference between neuron t and other surrounding neurons, and the more important that neuron is. Based on this, the feature map features are refined using a scaling operation based on the gain effect of neuronal responses. The refinement function is as follows:

[0061]

[0062] Among them, E is applicable to all features and spatial dimensions. Grouping is performed using the energy function for each channel. The sigmoid function (the activation function of a neural network) is added to limit excessively large values ​​in E. Output results. The enhanced features are represented by X, which is the input feature, and ⊙ represents the dot product operation.

[0063] Video surveillance data from the bridge construction site and inspection of previous construction components:

[0064] First, before performing object detection, it is necessary to determine which bridge structures to identify and construct a dataset containing these bridge structures to provide a data foundation for subsequent object detection. This invention selects structures such as piers, abutments, and bridge deck beam segments as the main structures for monitoring bridge construction progress. Then, based on the determined main structures of the bridge construction, bridge images are collected to construct a dataset. The dataset sources include construction sites and online image sources. The construction site dataset is collected using monitoring equipment at the construction site. Based on the URL address of the monitoring equipment, OpenCV is used to access the monitoring video data, read keyframes, and save the keyframe images. Online image sources are obtained by manually searching publicly available online platforms for images containing structures such as piers, abutments, and beam segments suitable for construction progress identification.

[0065] Secondly, construction data in specific scenarios is often incomplete. Therefore, this invention performs Mosaic data augmentation on the collected image data. Simply put, this data augmentation method involves taking four random images, processing them through random scaling, cropping, and arrangement, and then stitching them together into a new bridge image. This greatly enriches the dataset and reduces GPU memory usage. After augmenting the bridge images, various construction components in each new bridge image need to be labeled, creating label categories. This invention uses the open-source image annotation tool LabelImg to annotate the augmented bridge images in the dataset. The annotations are stored in a .xml file in PASCAL VOC format. The .xml file records parameters such as the size, number of channels, label name, and selection box position of the labeled bridge image. However, the file format required for improving the YOLOv5 model training is YOLO format (txt format). This invention uses Python to convert the dataset label file format. During the format conversion, the training set, validation set, and test set are divided according to the K-fold cross-validation method to improve data utilization and obtain more accurate evaluation results. The dataset construction process is as follows: Figure 3 As shown.

[0066] After creating the dataset, the improved YOLOv5S model was trained using it to determine the optimal parameters of the functions in the model and obtain the model weights. Finally, monitoring video data from the construction site was integrated, keyframe images and time information were extracted, and the keyframe image files were named after the extracted time, ensuring a one-to-one correspondence between the keyframe images and the time information, providing a basis for subsequent comparison of construction progress. The extracted keyframe images, with their one-to-one correspondence with the time information, were then input into the improved YOLOv5S model to identify components under construction or completed during that time period, obtaining information such as the type, location, and number of components of the main structure under construction.

[0067] Intelligent calculation of construction progress with spatial semantic constraints involves calculating the bridge construction progress based on the current construction component information and spatial semantic constraint rules obtained in step 1, as detailed below:

[0068] Relationship extraction for bridge construction schemes:

[0069] Bridge construction is a complex process that requires consideration of a series of construction procedures, schedule arrangements, and model designs. Clarifying the various elements and their relationships within these construction plans is crucial to expressing the relationships between different objects in a more intuitive and clear way, thus providing a foundation for establishing spatial semantic constraint rules. Therefore, this invention classifies the element objects in a bridge construction scenario into three categories of ontological concepts: bridge structure, construction procedures, and schedule. Specifically, the bridge structure includes sub-objects such as superstructure, substructure, supports, and ancillary facilities; the construction procedures include sub-objects such as overall project overview, estimated completion time, main construction sequence, and construction stages; and the schedule includes sub-objects such as construction work indicators, main schedule arrangements, and construction calendar arrangements. The ontological concepts and semantic constraints of the bridge construction scenario are as follows: Figure 4 As shown, by extracting ontology concept objects to build a conceptual hierarchy knowledge system for the bridge construction scenario, a foundation is laid for the derivation of subclasses and attributes from ontology concept objects.

[0070] After clarifying the ontological concept objects, a top-down division is performed based on the bridge construction plan, construction schedule, and other construction schemes. Subclasses of each ontological concept object are determined, and information such as entities, relationships, and attributes between these objects is manually extracted. This describes the attribute characteristics of construction components and clarifies the semantic relationships between various concept objects in the bridge construction scenario. In other words, this invention addresses the complex component relationships in bridge construction scenarios, analyzes the associations between various engineering schemes, and describes the construction sequence and spatial relationships between construction components during bridge construction from the perspectives of bridge construction scheme constraints, design parameter constraints, and initial 3D scene constraints. Considering factors such as bridge project management requirements, this invention decomposes the bridge's engineering structure into components such as piers, pile foundations, abutments, and beam segments. The attribute information for piers includes center mileage, type, number, and geometric parameters (width, pier height, etc.); the attribute information for pile foundations includes length, number, arrangement, and number; the attribute information for abutments includes number, number, length, width, and height; and the attribute information for beam segments includes type, starting mileage, ending mileage, and number.

[0071] Constructing spatial semantic constraints based on knowledge graphs:

[0072] After describing the construction sequence and spatial relationships among various construction components during bridge construction, this paper aims to effectively guide and constrain the calculation of bridge construction progress in complex scenarios, laying the foundation for intelligent management of construction progress.

[0073] The construction plan (bridge engineering construction plan) constraint relies on the construction sequence plan, construction organization plan, or construction design plan. Given the construction sequence, type, and quantity of construction components, it makes a rough determination of the construction progress based on the current construction components. Bridge engineering follows a bottom-up construction sequence; therefore, this constraint divides the construction scenario into substructure construction, bearing construction, and superstructure construction, thus constraining the determination of construction progress from the perspective of construction sequence.

[0074] The initial 3D scene constraints use the initially known pixel dimensions (design dimensions) and measured dimensions (i.e., actual dimensions) as constraints. Combined with the pixel dimension information of the construction components output by the target detection model (an improved YOLOv5S model), and the measured dimensions obtained through the camera's scaling factor, the true 2D dimensions of the construction components can be calculated, enabling precise determination of construction progress. On the other hand, this constraint provides an initial 3D model of the construction scene. After obtaining the actual dimensions of the constructed components, 3D modeling software can update the model of the constructed portion in the 3D scene, providing a more intuitive and clear display of the current construction progress.

[0075] Design parameter constraints rely on BIM design models or bridge engineering design schemes to provide constraints on the bridge construction structure, the dimensions of each component, and the spatial relationship between them. When the actual two-dimensional dimensions of the construction components are obtained, the three-dimensional dimensions can be obtained based on the component design dimensions. The three-dimensional geometric parameters of the construction components can be completed from the perspective of geometric parameters, providing a data foundation for the completion of the three-dimensional model in the construction scenario, and also providing a reference for the progress comparison of construction components.

[0076] Intelligent calculation of construction progress based on combined knowledge-driven approaches and target detection results:

[0077] This section is mainly divided into two parts: construction progress calculation and progress comparison. For construction progress calculation, the improved YOLOv5S model detects the categories, quantities, and locations of construction components (main structure) as input data. Combined with spatial semantic constraint rules, the construction progress is calculated hierarchically from type to quantity to size. The specific progress calculation method is as follows: Figure 6 As shown.

[0078] The target detection network identifies the category name of the construction component and, combined with the construction sequence plan, determines whether the structure constructed before that component has been completed. For example, if a bridge pier structure is detected, the pile foundation and abutment structure have been completed. When multiple main structures of the same type are detected, the number of these structures is compared with the number of similar components in the bridge construction plan. If the numbers are equal, the construction of that type of component is complete; if the number is less than the number of similar components in the bridge construction plan, the progress is considered complete. If the number exceeds the number of similar components in the bridge construction plan, the detection is incorrect and the detection is repeated. When a component under construction is detected, its pixel size information is obtained based on the component's identification location parameters. Combined with the known initial parameters in the initial scene constraints, the actual two-dimensional dimensions of the construction are calculated. This is then compared with the design dimensions of the construction components. Specifically, the actual two-dimensional dimensions of each main structure under construction are calculated using the photographic scale conversion factor between the design dimensions and the measured dimensions monitored by the surveillance camera. The construction progress is then calculated as follows: Actual size (meters) = Bridge image pixel size ÷ Photographic scale conversion factor × 0.001. After completing the construction progress calculation, the construction data is stored, and the storage format is set as {type; structureIdstate; progress; time; 2DSize;}, where type is the type of construction component, structureId is the number of the construction component, state is the construction status (true: completed or false: not completed), progress is the percentage of construction progress, time is the data acquisition time, and 2DSize is the two-dimensional size of the construction component.

[0079] This invention proposes using Neo4j, a common graph database, to construct a knowledge graph containing entities, relationships, and attributes related to bridge construction scenarios. The constructed graph is exported as a .json file, and design information for current construction components is retrieved through querying, followed by unified data format processing. The data format is as follows:

[0080] {id; type; startTime; endTime; designSize; designPro}

[0081] Wherein, id represents the component number, type represents the component category, startTime represents the planned construction start time, endTime represents the planned construction end time, designSize represents the design dimension, and designPro represents the planned progress. The construction progress and data collection time of the current construction component information are compared with the planned construction start time, planned construction end time, and planned progress to determine whether the current construction status is normal or stalled, and the 3D dimensions of the construction component are restored. By adding bridge components constructed during this period to the 3D modeling software, the 3D model is completed, the 3D scene is updated, and a simple visualization of the construction progress is achieved. Managing the construction progress from a 3D perspective allows project managers to have a clearer and more intuitive understanding of the project's progress.

[0082] The YOLOv5 network model is one of the one-stage object detection algorithms in deep learning. Its main network structure includes an input layer, a backbone network, a neck network, and a head output layer. There are four YOLOv5 models: YOLOv5-S / M / L / X, with YOLOv5S being the most lightweight and well-suited for practical engineering projects. Applying the YOLOv5S network to monitoring videos of bridge construction sites reduces the generation of candidate bounding boxes, effectively identifies the probability and coordinates of the main bridge components under construction, and achieves fast detection speed, thus reducing the time spent capturing the actual progress of the construction site. Considering the complexity of bridge construction site environments, the YOLOv5S network model uses a single convolutional kernel, resulting in a limited receptive field and an inability to grasp the global features of the construction site. This invention introduces a Transformer-like attention mechanism model into the YOLOv5S backbone network for feature fusion, improving the model's feature extraction capabilities. Knowledge graphs are a major technology in the cognitive intelligence field of current artificial intelligence, possessing powerful semantic processing capabilities and providing a foundation for intelligent information applications. By constructing a knowledge graph of bridge construction procedures and schedules, and analyzing and mining bridge construction data, the relationships between various bridge components can be clearly clarified. Introducing the knowledge graph into the field of bridge construction schedule calculation allows for the creation of spatial semantic constraint rules for schedule calculation, thereby improving the efficiency of schedule calculation.

Claims

1. An improved YOLOV5S-based bridge construction progress intelligent identification method, characterized in that, Comprise the following steps: Step 1. Based on the constructed bridge component dataset, the obtained video monitoring data of the bridge construction site and the improved YOLOV5S model, the current construction component information is obtained; The improved YOLOV5S model is based on the YOLOV5S model, and an attention module SimAM is added behind each C3 module in the backbone network of the YOLOV5S model and at the end, so that the convolution module Conv, the C3 module and the attention module SimAM are integrated into the CCS module; Step 2. Based on the current construction component information obtained in step 1 and the spatial semantic constraint rule, the bridge construction progress is calculated to obtain the current construction progress; The specific steps are: Step 2.

1. Based on the bridge engineering construction scheme and the BIM design model, the correlation analysis between the components of the bridge is carried out; Step 2.

2. Based on the correlation analysis result, the knowledge graph technology is used to construct the knowledge graph to establish the spatial semantic constraint rule; The specific steps are: Step 2.

21. Based on the correlation analysis result, the knowledge graph is constructed; Step 2.

22. Based on the knowledge graph, the bridge engineering is constructed in the order from bottom to top, that is, based on the constraint, the construction scene is divided into lower structure construction, support construction and upper structure construction in turn, and the construction progress of the current construction component information is coarsely judged from the construction process; Step 2.

23. The initial design size and measured size in the BIM design model which describes the semantic relationship are used as constraint conditions to perform fine judgment on the construction progress of the main structure under construction, that is, the design size and measured size information of each main structure under construction are used to realize the fine judgment of the construction progress; Step 2.

3. Based on the current construction component information and the spatial semantic constraint rule, the bridge construction progress is calculated to obtain the current construction progress.

2. The improved YOLOV5S-based bridge construction progress intelligent identification method according to claim 1, characterized in that, The specific steps of step 1 are: Step 1.

1. Determine the main structure of the bridge construction, and construct the bridge component dataset for target detection based on the obtained bridge image, wherein the main structure included in the bridge component dataset includes pier, abutment, pile cap, bridge deck pavement and auxiliary facilities; Step 1.

2. Based on the bridge component dataset, the improved YOLOV5S model is trained to obtain the trained improved YOLOV5S model; Step 1.

3. Obtain the video monitoring data of the bridge construction site, extract the key frame image and time information in the video monitoring data, and name the key frame image as the extracted time, so that the key frame image and the time information correspond one by one; Step 1.

4. The named key frame image is input into the trained improved YOLOV5S model to identify the components under construction or completed construction in this period, and the current construction component information is obtained, including the category, position information and component quantity of each main structure.

3. The improved YOLOV5S-based bridge construction progress intelligent identification method according to claim 2, characterized in that, The specific steps of step 1.1 are: Step 1.

11. Mosaic data enhancement is performed on the collected bridge images, that is, four random bridge images are selected from the collected bridge images, and the four bridge images are randomly scaled, randomly cropped and randomly arranged, and then the four processed bridge images are spliced into a new bridge image; Step 1.

12. The main structure of the bridge construction is determined, and the open-source image labeling tool LabelImg is used to mark each main structure in the new bridge image to obtain the marking category, and the annotation is stored in the.xml file in the PASCAL VOC format. The.xml file records the size, channel number, label name and bounding box position of the new bridge image after marking; Step 1.

13. The.xml file is converted into a.txt format using Python, that is, a bridge component dataset for target detection is obtained.

4. The improved YOLOV5S-based bridge construction progress intelligent identification method according to claim 3, characterized in that, The specific steps of step 2.1 are as follows: Step 2.

11. The ontology concept object is determined, and the top-down division is performed according to the bridge engineering construction scheme to determine the subclass objects of each ontology concept object, wherein the ontology concept object includes bridge structure, construction process and progress plan, the subclass objects of the bridge structure include superstructure, substructure, support and auxiliary facilities, the subclass objects of the construction process include engineering general situation, expected completion time, main construction sequence and construction stage, the subclass objects of the progress plan include construction operation index, main construction period arrangement and construction plan calendar arrangement, and the bridge engineering construction scheme includes construction organization scheme and construction progress arrangement table; Step 2.

12. After manually extracting the entities, relationships and attributes between each ontology concept object, the attribute characteristics of each main structure in the construction are described based on the BIM design model, and the semantic relationships between each ontology concept object in the bridge construction scene are clarified, that is, the construction sequence relationship and the spatial relationship between each main structure in the bridge construction process are described.

5. The improved YOLOV5S-based bridge construction progress intelligent identification method according to claim 4, characterized in that, The specific steps of step 2.3 are as follows: Step 2.

31. Under the constraint condition of rough determination, the construction progress of each main structure is determined based on the construction process of each main structure in the current construction component information and the bridge engineering construction scheme; Step 2.

32. Under the constraint condition of fine determination, the real two-dimensional size of each main structure under construction is calculated based on the design size of each main structure under construction and the photographic proportion conversion coefficient of the measured size monitored by the monitoring camera.

6. The improved YOLOV5S-based bridge construction progress intelligent identification method according to claim 5, characterized in that, The specific steps of step 2.31 are as follows: When the multiple same type of subject structures are detected, the number of the detected subject structures of the same type is compared with the number of the same type of components in the bridge engineering construction scheme. If the numbers are equal, the construction of the same type of components is completed. If the number is less than the number of the same type of components in the bridge engineering construction scheme, the progress is , if the number is greater than the number of the same type of components in the bridge engineering construction scheme, the detection is incorrect, and the detection is re-performed. The specific steps of step 2.32 are as follows: The real construction of each main structure two-dimensional size is calculated according to the design size of each main structure under construction and the photographic proportional conversion coefficient of the measured size monitored by the monitoring camera: ; The formula for calculating the real size according to the photographic proportion conversion coefficient is as follows: 。 7. The improved YOLOV5S-based bridge construction progress intelligent identification method according to claim 6, characterized in that, Step 3: Compare the construction progress based on the current construction component information with the corresponding planned construction information to determine whether the current construction state of each main component is normal or stagnant, and simultaneously update each main structure in the BIM design model in real time.

Images