Truss floor support plate defect detection improved by fusing ShuffleNet-YOLO model

Abstract

The invention provides an improved truss floor support plate defect detection method fused with a ShuffleNet-YOLO model. A truss floor support plate defect data set with damage, voids and pitted surfaces, cracks and steel bar corrosion is constructed. The method comprises the following steps: firstly, fusing an attention mechanism CBAM into a ShuffleNetv2 model, then fusing a lightweight network ShuffleNetv2-CBAM with a target detection algorithm YOLOv5s.60, and finally, adopting a WIOU and Soft-NMS improved strategy at an output end to form a final network model. According to the method, the characteristic of high detection speed of the YOLO model and the advantage of light weight of the ShuffleNet algorithm are well utilized, so that the model has the advantages of high recognition precision, high detection speed, high generalization ability and the like, is convenient to deploy on embedded equipment, can be widely applied to field real-time monitoring scenes, improves the construction quality and efficiency, and is suitable for popularization and application. And a more reliable and efficient solution is provided for defect detection of the truss floor support plate, so that technical support is provided for intelligent transformation of the building industry.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent construction site quality inspection, and particularly relates to a method for detecting defects of truss floor slabs improved by integrating the ShuffleNet-YOLO model. Background Technique

[0002] As a core load-bearing component in the floor, roof and foundation parts of building projects, the structural integrity and durability of precast concrete reinforced truss floor slabs are directly related to building safety and service life. In practical applications, such truss floor slabs often have potential hazards such as plate surface damage, concrete honeycombing and pockmarking, through cracks and steel bar corrosion due to construction process defects, material aging or environmental erosion, resulting in a decrease in bearing capacity and an increased risk of leakage, seriously threatening the overall stability of the building and the safety of personnel. Especially the internal structure deterioration problem caused by steel bar corrosion has the characteristics of strong concealment and high repair costs. If not intervened in time, it may lead to catastrophic consequences. At present, the industry's control of the quality of truss floor slabs still generally relies on manual visual inspection and local sampling inspection, which has disadvantages such as low efficiency, limited coverage and strong subjectivity. Traditional methods are difficult to comprehensively capture the distribution law of honeycombing and pockmarking, the condition of plate surface damage, the crack propagation trend and the degree of internal steel bar corrosion, and are even less able to evaluate the long-term impact of environmental temperature and humidity and load changes on the floor slab in real time, and it is very easy to miss early defects or misjudge the risk level. In complex scenarios such as high-rise buildings and large-span spaces, the lag and limitations of manual inspection are more obvious, often resulting in quality hazards evolving into irreversible structural damage.

[0003] Therefore, it is urgent to build an intelligent full-cycle quality monitoring system, deeply integrate intelligent quality inspection technology with the building engineering management process, and realize dynamic tracking of truss floor slabs through means such as image recognition, data analysis and early warning notification, so as to achieve the transformation from "post-facto repair" to the proactive prevention and control mode of "pre-warning - in-process control", and provide technical guarantee for the safety of the whole life cycle of the building.

[0004] In actual application, floor decking has exposed many quality problems, which pose a serious threat to the safety and stability of construction projects. First of all, the phenomenon of board surface damage is common. During transportation, lifting and on-site installation, if the operation is improper or the protective measures are not in place, the board surface is very easy to be hit and scratched, resulting in cracks, holes, missing corners and other damage. These damages not only seriously affect the appearance of the floor deck, but more importantly, they will weaken the bearing capacity of the floor deck, thereby reducing the overall performance of the entire structure. Secondly, the problem of concrete honeycomb surface should not be ignored. During the concrete pouring process, if the vibration is not dense, the surface of the formwork is not smooth or the release agent is used improperly, honeycomb holes or rough surfaces will be formed on the board surface. This defect will reduce the bonding force between concrete and steel bars, thereby weakening the durability of the floor deck and accelerating the corrosion process of steel bars, thereby affecting the service life of the building structure. Furthermore, concrete cracks are a problem that needs to be solved urgently. During the hardening process of concrete, cracks are easily generated due to the influence of various factors such as cement hydration heat, shrinkage deformation, and temperature changes. These cracks will not only reduce the waterproof performance of the floor deck, but also expose the steel bars and accelerate their corrosion process. In severe cases, the cracks may even lead to structural damage of the floor deck, thus endangering people's lives. In addition, the problem of steel bar corrosion also occurs from time to time. If the steel bars in the truss floor deck are in a humid, acidic and alkaline environment for a long time, and the surface anti-corrosion treatment of the steel bars is not proper, it will cause corrosion. The corrosion of steel bars will lead to a reduction in the cross-sectional area and strength of the steel bars, and the loss of their bonding with the concrete. This will seriously affect the load-bearing and deformation performance of the truss floor deck, bringing huge safety hazards to the building structure.

[0005] Among the numerous links in ensuring the quality of floor decking, the on-site acceptance is undoubtedly a crucial one. Nowadays, we should make full use of artificial intelligence technologies such as deep learning to collect data on quality defects of truss floor decking and conduct real-time and accurate analysis of inspection records, photos or videos during on-site acceptance. These data sets detail various potential quality and safety defects, such as damaged deck surface, honeycombing and pitting of concrete, cracks, steel bar corrosion, and dimensional deviations. The deeply trained model can automatically identify the appearance features, specifications, and overall quality status of the floor decking and accurately judge whether there are quality and safety defects. Once any abnormalities or non-compliance with quality standards are found during on-site acceptance, the system will immediately trigger an alarm, reminding the acceptance personnel to quickly mark and isolate the problem batches. At the same time, the system will notify the relevant construction or management personnel to take necessary corrective measures. This can ensure that only qualified truss floor decking enters the construction site, laying a solid foundation for subsequent installation and use, and comprehensively guaranteeing the structural stability and safety of the construction project. For the already installed truss floor decking, continuous quality and safety monitoring is equally important. We can timely detect potential quality and safety hazards, such as structural deformation or crack expansion. On this basis, we can take corresponding preventive measures to ensure the quality and safety of the truss floor decking during long-term use. In addition, the quality and safety monitoring of truss floor decking also needs to pay special attention to the operation safety of construction personnel. When installing and maintaining truss floor decking, construction personnel must wear compliant safety equipment and strictly abide by safety operation procedures. To monitor the safety status of construction personnel in real time, we can also introduce intelligent wearable devices and combine the analysis of monitoring videos with deep learning algorithms to ensure that construction personnel follow safety regulations during operation, thus further guaranteeing the quality and safety of truss floor decking. Therefore, closely integrating advanced artificial intelligence technologies with the quality and safety monitoring of truss floor decking is the core of realizing full-chain quality and safety supervision and optimization in a smart construction site. By constructing a comprehensive quality and safety data set, training an efficient model, introducing intelligent wearable devices and Internet of Things technologies, we can achieve comprehensive, real-time, and intelligent quality and safety monitoring and management of the installation and use process of truss floor decking, thus ensuring the overall quality and safety of the construction project.

[0006] In recent years, the rapid development of deep learning technology, especially the continuous breakthroughs in the field of computer vision, has brought new solutions to the detection of construction quality defects in intelligent construction sites. These technologies have not only promoted the in-depth exploration of artificial intelligence theory but also demonstrated great potential and value in practical applications, especially in construction projects with extremely high requirements for construction quality. In the early stage, although some traditional image processing and machine learning algorithms achieved certain results in construction quality detection, they were often limited by detection accuracy, speed, and generalization ability. With the rise of deep learning technology, especially the emergence of convolutional neural networks (CNNs) and their various optimized architectures, such as the YOLO series of algorithms, the detection of construction quality defects has undergone revolutionary changes. The YOLO algorithm realizes the instant and dynamic analysis of construction site images or videos through a single forward propagation process, greatly improving the detection efficiency and maintaining a high recognition accuracy. However, the exploration of technology never ends. After the YOLO series of algorithms, more deep learning architectures have been introduced into the detection of construction quality defects. Among them, ShuffleNet, as an emerging lightweight network architecture, has attracted wide attention with its efficient computing performance and good recognition accuracy. ShuffleNet effectively reduces the model parameters and computational amount through innovative channel shuffle operations and grouped convolution strategies, enabling the model to achieve fast inference speed while maintaining a high recognition accuracy. This feature makes ShuffleNet particularly suitable for scenarios such as construction sites that have extremely high requirements for real-time performance and accuracy. In the detection of construction quality defects in intelligent construction sites, the application of advanced architectures such as ShuffleNet, MobileNet, and EfficientNet not only improves the real-time performance and accuracy of detection but also provides strong support for the comprehensive monitoring and management of construction quality. By introducing more optimized network structures, improving training strategies, and using more diverse datasets, the construction quality defect detection system can automatically identify and mark various quality defects in the construction process, such as cracks, damages, misalignments, etc., thus helping construction personnel to discover problems in a timely manner and take corresponding corrective measures. In addition, the construction quality defect detection system in intelligent construction sites can also be combined with technologies such as the Internet of Things and big data to achieve real-time collection, analysis, and visual display of construction data. By building a comprehensive construction quality management platform, construction personnel can grasp the construction progress and quality status in real time, discover potential problems in a timely manner, and optimize the construction plan, thereby ensuring construction quality and safety.

[0007] In the field of quality and safety inspection of smart construction sites, we can try to apply the method of combining ShuffleNet and YOLO models to the defect detection of truss floor slabs. Collect the quality defect data of truss floor slabs, and through the construction of a deep learning model based on ShuffleNet-YOLO, realize the real-time dynamic management of the whole process of truss floor slabs, so as to effectively ensure the quality and safety of truss floor slabs at the construction site. Summary of the Invention

[0008] Based on the above problems, the purpose of the present invention is to provide a method for defect detection of truss floor slabs improved by integrating the ShuffleNet-YOLO model, aiming to achieve fast and accurate identification of target defects, and at the same time ensure the ability to monitor the detection process in real time and dynamically.

[0009] A method for defect detection of truss floor slabs improved by integrating the ShuffleNet-YOLO model includes:

[0010] Step 1: Collect image data of truss floor slabs with damage, honeycombing, cracks, and steel bar corrosion.

[0011] Step 2: Preprocess the collected data to construct a defect dataset of truss floor slabs; the preprocessing includes rotation, scaling ratio, cropping, and occlusion.

[0012] Step 3: Perform data annotation for each category, and then input it into the YOLOv5s.60 model for pre-training, continuously adjust the parameters, and obtain the optimal result after the pre-training ends.

[0013] Step 4: Improve the object detection algorithm model of the YOLOv5s.60 version based on the lightweight network ShuffleNetv2, the attention mechanism CBAM, the weighted intersection over union loss function (Weighted Intersection over Union, WIOU), and the soft non-maximum suppression function (Soft Non-MaximumSuppression, Soft-NMS) to construct a new object detection network model ShuffleNetv2-YOLOv5s.60.

[0014] Step 5: Input the dataset into the ShuffleNet-YOLOv5s.60 network model for training to obtain indicators such as the trained weight parameters and accuracy.

[0015] Step 6: Deploy the model on the video monitoring end to test whether it can identify the damage, honeycombing, cracks, and steel bar corrosion of the truss floor slabs.

[0016] 2. The specific structure of the target detection network model ShuffleNet-YOLOv5s.60 in step 4 is as follows: First, replace the feature extraction backbone network Backbone in the YOLOv5s.60 model with the CBRM and {Shuffle_block}*6 modules in the lightweight network model ShuffleNetv2. Then, divide {Shuffle_block}*6 into three equal parts, and integrate the attention mechanism CBAM into every two equal parts. Finally, replace the complete loss function (Complete IoU Loss, CIoU) and non-maximum suppression function (Non-Maximum Suppression, NMS) in YOLOv5s.60 with the WIOU loss function and Soft-NMS non-maximum suppression function.

[0017] Step 3 includes:

[0018] Step 3.1: Use the preprocessed image dataset as the input of the improved YOLOv5s.60 model, and perform Mosaic data augmentation, adaptive anchor box calculation, and adaptive image scaling processing in sequence;

[0019] Step 3.2: Extract features through the CBS, C3, and SPPF modules in the feature extraction backbone network Backbone;

[0020] Step 3.3: In the Neck network module, use the combination of the feature pyramid FPN and the path aggregation network PAN to perform multi-scale fusion on different feature maps;

[0021] Step 3.4: Use CIoU to calculate the scores of the target proposal boxes;

[0022] Step 3.5: Use NMS to suppress the proposal boxes smaller than the threshold and retain the proposal boxes larger than the threshold;

[0023] Step 3.6: According to the set training parameters, when the optimal accuracy is reached, retain the training result, otherwise readjust the training parameters until the optimal is reached.

[0024] Step 4 includes:

[0025] Step 4.1: Use the preprocessed image dataset as the input of the improved ShuffleNet-YOLOv5s.60 model, and perform Mosaic data augmentation, adaptive anchor box calculation, and adaptive image scaling processing in sequence;

[0026] Step 4.2: Extract features through the CBRM, [(Shuffle_block}*2, CBAM]*3 modules in sequence in the feature extraction backbone network Backbone;

[0027] Step 4.3: In the Neck module, a combination of Feature Pyramid Network (FPN) and Path Aggregation Network (PAN) is used to perform multi-scale fusion on different feature maps;

[0028] Step 4.4: Use WIOU to calculate the scores of the target proposal boxes;

[0029] Step 4.5: Use Soft-NMS to suppress the proposal boxes smaller than the threshold and retain the proposal boxes larger than the threshold;

[0030] Step 4.6: According to the initially set training parameters, where the weight parameter with the highest pre-training accuracy is used for training. When the optimal accuracy is reached, retain the trained result; otherwise, readjust the training parameters until the optimal is achieved.

[0031] The beneficial effects of the present invention are as follows:

[0032] The present invention constructs a defect dataset for truss floor slabs, including four major categories: damage, honeycombing and pockmarking, cracks, and steel bar corrosion detection. First, replace the feature extraction backbone network Backbone in the YOLOv5s.60 model with the CBRM and {Shuffle_block}*6 modules in the lightweight network model ShuffleNetv2. Then, divide {Shuffle_block}*6 into three equal parts, and integrate the attention mechanism CBAM into every two equal parts. Finally, replace the Complete IoU Loss (CIoU) and Non-Maximum Suppression (NMS) functions in YOLOv5s.60 with the WIOU loss function and Soft-NMS non-maximum suppression function. The present invention realizes the automatic detection of truss floor slab defects. This technological breakthrough not only greatly improves the detection accuracy and efficiency, but also brings a revolutionary change to the quality control and safety supervision in the construction industry. In addition, a defect dataset for truss floor slabs is constructed. This innovation not only strengthens the quality control system of the smart construction site, but also greatly reduces the frequency and intensity of manual inspections, reduces human errors, and makes quality management more proactive and efficient. The popularization and use of the present invention will strongly promote the digital transformation of the construction industry and promote the in-depth practice of the concept of intelligent construction. By using intelligent means to improve the quality management level of the construction site, it can ensure the safety of construction workers' lives, effectively control the project quality, and improve the economic and social benefits of the overall project. Description of the Drawings

[0033] Figure 1Flow chart of construction safety detection improved by integrating ShuffleNet - YOLO model in the present invention;

[0034] Figure 2 Target detection data in the present invention; among them, (a) damage, (b) honeycombing and pockmarking, (c) cracks, (d) steel bar corrosion

[0035] Figure 3 Pre - processing of target detection data in the present invention; among them, (a) rotation, (b) scaling ratio, (c) cropping, (d) occlusion;

[0036] Figure 4 Annotation of the data set in the present invention; among them, (a) initial interface screenshot, (b) new project screenshot, (c) data set annotation screenshot, (d) save and export screenshot;

[0037] Figure 5 YOLOv5s.60 network structure diagram in the present invention;

[0038] Figure 6 Structural diagram of the lightweight network ShuffleNetv2 network unit in the present invention:

[0039] Figure 7 Schematic diagram of the attention mechanism CBAM structure in the present invention;

[0040] Figure 8 Improved network structure diagram in the present invention;

[0041] Figure 9 Final model training result diagram in the present invention; among them, (a) confusion matrix, (b) label information, (c) relationship between the horizontal and vertical coordinates of the center point and the height and width of the box, (d) visualization of training result analysis, (e) precision P value, (f) mean average precision mAP

[0042] Figure 10 Interface design, among which, (a) picture recognition function interface, (b) camera / video monitoring function interface;

[0043] Figure 11 Effect diagram of real - time construction safety detection in the present invention; among them, (a) automatic detection schematic diagram of damaged (PS) pictures, (b) automatic detection schematic diagram of honeycombing and pockmarking (FM) pictures, (c) automatic detection schematic diagram of crack pictures, (d) automatic detection schematic diagram of steel bar corrosion (corrosion) pictures, (e, f) video detection schematic diagram. Detailed implementation method

[0044] The following further illustrates the invention with reference to the attached drawings and specific implementation examples.

[0045] AsFigure 1 As shown in Figure 1 , a method for detecting defects in truss floor slabs by improving the ShuffleNet-YOLO model includes the following steps:

[0046] Step 1: Collect image data of damage, honeycombing, cracks, and rust on truss floor slabs;

[0047] In this embodiment, the data includes four categories: damage, honeycombing, cracks, and steel bar rust, as shown in Figure 2 .

[0048] Step 2: Preprocess the collected data to construct a defect dataset for truss floor slabs; the preprocessing includes rotation, scaling ratio, cropping, occlusion, and data annotation;

[0049] For damage, honeycombing, cracks, and steel bar rust, perform the following four random combination operations respectively, through rotation, scaling ratio, cropping, and occlusion, as shown in Figure 3 ; Dataset annotation: The labelimg used is an open-source image annotation tool, as shown in Figure 4 (a), click Open Dir (open the picture), as shown in Figure 4 (b), select the save label location (Change SaveDir), as shown in Figure 4 (c), select Pascal VOC (label format), click Create RectBox, perform box selection, fill in the category, and proceed to the next one until all annotations are completed.

[0050] Step 3: Select high-quality pictures for each category, and perform pre-training after data annotation. The pre-training uses the YOLOv5.60 model, which includes an input end, a feature extraction backbone network (Backbone), a neck (Neck) module, and a prediction output end (Prediction), as shown in Figure 5 , and obtain the weight parameters after pre-training;

[0051] Step 4: Improve the object detection algorithm model of the YOLOv5s.60 version based on ShuffleNetv2, CBAM, WIOU, and Soft-NMS to construct a new object detection network model ShuffleNetv2-YOLOv5s.60;

[0052] Use the CBRM and {Shuffle_block}*6 modules in the lightweight network model ShuffleNetv2, as shown in Figure 6 to replace the feature extraction backbone network Backbone in the YOLOv5s.60 model, and then divide {Shuffle_block}*6 into three equal parts, and integrate the attention mechanism CBAM into every two equal parts (as shown inFigure 7 ), finally, by replacing the Complete IoU Loss (CIoU) and Non-Maximum Suppression (NMS) in YOLOv5s.60 with the WIOU loss function and Soft-NMS non-maximum suppression function, the improved network structure diagram is shown in Figure 8 .

[0053] Step 5: Use the optimal weight parameters obtained after the pre-training to train with the ShuffleNet-YOLOv5s.60 network model, including:

[0054] Step 5.1: Use the pre-processed image dataset as the input of the improved ShuffleNet-YOLOv5s.60 model, and perform Mosaic data augmentation, adaptive anchor box calculation, and adaptive image scaling processing in sequence;

[0055] Step 5.2: Extract features through the CBRM, [(Shuffle_block)*2, CBAM]*3 modules in the feature extraction backbone network Backbone in sequence;

[0056] Step 5.3: In the Neck module, use the combination of Feature Pyramid Network (FPN) and Path Aggregation Network (PAN) to perform multi-scale fusion on different feature maps;

[0057] Step 5.4: Use WIOU to calculate the scores of the target proposal boxes;

[0058] Step 5.5: Use Soft-NMS to suppress the proposal boxes smaller than the threshold and retain the proposal boxes larger than the threshold;

[0059] Step 5.6: According to the initially set training parameters, where the weight parameters are trained with the highest pre-training accuracy, when the optimal accuracy is reached, retain the training result (see Figure 9 ), otherwise readjust the training parameters until the optimal is reached.

[0060] Step 6: Deploy the truss floor formwork detection system on the terminal device to test the target;

[0061] Including: picture recognition, video detection interface, see Figure 10

[0062] Upload pictures for detection, see Figure 11(a, b, c, d), where 11(a): PS (damage), 11(b): FM (honeycombing and pockmarking), 11(c): crack, 11(d): corrosion, video detection, see Figure 11 (e, f).

Claims

1. Defect detection of truss floor slabs improved by integrating the ShuffleNet-YOLO model, characterized in that Including: Step 1: Collect image data of precast concrete steel truss floor slabs with damage, honeycombing, cracks, and steel bar corrosion, and accumulate 1184 pictures. Step 2: Preprocess the collected various image data, and then carry out data annotation work. Step 3: Place the pictures and annotated labels into the YOLOv5s.60 model for training, and continuously adjust the parameters to obtain the optimal accuracy and weight parameters. Step 4: Improve the object detection algorithm model of YOLOv5s.60 version based on the lightweight network model ShuffleNetv2, the attention mechanism CBAM, the weighted intersection over union loss function (Weighted Intersection over Union, WIOU), and the soft non-maximum suppression function (SoftNon-MaximumSuppression, Soft-NMS) to construct a new network model ShuffleNetv2-YOLOv5s.

60. Step 5: Call the weight parameters and use ShuffleNetv2-YOLOv5s.60 for training to obtain the accuracy and indicators after training. Step 6: Deploy the model and the weight parameter file on the monitoring end to test whether the target is detected at the construction site.

2. The method for detecting defects of truss floor slabs improved by integrating the ShuffleNet-YOLO model according to claim 1, characterized in that, The preprocessing in Step 2 includes rotation, scaling ratio, cropping, occlusion, and manual data annotation.

3. The method for detecting defects of truss floor slabs improved by integrating the ShuffleNet-YOLO model according to claim 1, characterized in that, The specific structure of the object detection network model ShuffleNetv2-YOLOv5s.60 in Step 4 is as follows: First, replace the feature extraction backbone network Backbone in the YOLOv5s.60 model with the CBRM and {Shuffle_block}*6 modules in the lightweight network model ShuffleNetv2. Then, divide {Shuffle_block}*6 into three equal parts, and integrate the attention mechanism CBAM into every two equal parts. Finally, replace the complete IoU loss function (CIoU) and the non-maximum suppression function (Non-MaximumSuppression, NMS) in YOLOv5s.60 with the WIOU loss function and the Soft-NMS non-maximum suppression function.

4. The method for detecting defects of a truss floor slab improved by integrating the ShuffleNet-YOLO model according to claim 1, characterized in that The said Step 3 includes: Step 4.1: Use the preprocessed image dataset as the input of the improved YOLOv5s.60 model, and perform Mosaic data augmentation, adaptive anchor box calculation, and adaptive image scaling processing in sequence. Step 4.2: Extract features through the CBS, C3, and SPPF modules in the feature extraction backbone network Backbone. Step 4.3: In the Neck network module, use the combination of the feature pyramid FPN and the path aggregation network PAN to perform multi-scale fusion on different feature maps. Step 4.4: Use CIoU to calculate the scores of the object proposal boxes. Step 4.5: Use NMS to suppress the proposal boxes smaller than the threshold and retain the proposal boxes larger than the threshold. Step 4.6: According to the set training parameters, when the optimal accuracy is reached, retain the result after training; otherwise, readjust the training parameters until the optimal accuracy is achieved.

5. The method for detecting defects of truss floor slabs improved by integrating the ShuffleNet-YOLO model according to claim 1, characterized in that, The said Step 5 includes: Step 5.1: Use the preprocessed image dataset as the input of the improved YOLOv5s.60 model, and successively perform Mosaic data augmentation, adaptive anchor box calculation, and adaptive image scaling processing; Step 5.2: Successively extract features in the feature extraction backbone network Backbone through the CBRM, [(Shuffle_block}*2, CBAM]*3 modules; Step 5.3: In the Neck network module, use the combination of the Feature Pyramid Network (FPN) and the Path Aggregation Network (PAN) to perform multi-scale fusion on different feature maps; Step 5.4: Use WIoU to calculate the scores of the target proposal boxes; Step 5.5: Use Soft-NMS to suppress the proposal boxes smaller than the threshold and retain the proposal boxes larger than the threshold; Step 5.6: According to the initially set training parameters, where the weight parameters are trained using the one with the highest pre-training accuracy, when the optimal accuracy is reached, retain the result after training; otherwise, readjust the training parameters until the optimal accuracy is achieved.

Images