Method and system for intelligently detecting appearance quality of precast beam

Abstract

The invention relates to a precast beam appearance quality intelligent detection method and system, and the method comprises the steps: building an improved YOLOv8 network, training a target detection model through a data set, and obtaining a precast beam appearance quality detection model; the method can effectively replace a traditional detection mode that appearance quality defects of precast beams are detected through manual observation, the problems that manual detection of the appearance defects is high in subjectivity, poor in accuracy, small in detection coverage rate and the like are solved, time and manpower are saved, timely dynamic monitoring can be carried out, and wrong detection, false detection and missing detection caused by fatigue of recorders and the like are avoided; precast beam images can be automatically collected for detection and analysis, storage, query and analysis of appearance quality defect detection data and automatic generation of an evaluation report are achieved, and control over the appearance quality of the precast beam in the later period of a project is facilitated.

Description

Technical Field

[0001] The present invention belongs to the technical fields of image acquisition and processing, deep learning and target detection and recognition, and in particular relates to a method and system for intelligent detection of the appearance quality of prefabricated beams. Background Art

[0002] my country is a major country in bridge construction, and the total number of bridges ranks among the top in the world. In bridge engineering, precast beams refer to bridge components that are pre-manufactured in factories or workshops outside the construction site. Precast beam construction technology has the advantages of more efficient construction process and effective control of the quality level of bridge projects. It has been widely used in bridge engineering project construction in recent years. Precast beam quality control is a key link to ensure the safety, durability and service life of bridge structures. Traditional beam quality inspection mainly relies on manual labor, which is not only time-consuming and labor-intensive, but also prone to misdetection, false detection and missed detection. Exploring a convenient and efficient method for detecting the quality of precast beams is of great practical significance to my country's growing infrastructure needs.

[0003] In view of the many defects of the traditional manual inspection method of precast beam quality in this field, the use of artificial intelligence that has emerged in recent years is the best way to improve the current situation. Deep learning (DL) is an emerging research direction derived from machine learning (ML). The introduction of deep learning brings machine learning closer to the ultimate goal of artificial intelligence. However, the current general deep learning technology does not perform as expected in the field of construction. It performs poorly in dealing with different construction environments and has the problem of difficulty in generalization.

[0004] Therefore, it is urgent to study a method and system for intelligent detection of the appearance quality of precast beams. Through intelligent detection, the corresponding defects can be automatically identified, statistical analysis can be performed, and a detection report can be formed to improve detection efficiency and accuracy, reduce the safety risks of manual detection, ensure the safety of bridge use, and extend its service life. Summary of the invention

[0005] The purpose of the present invention is to overcome the deficiencies in the prior art and to provide a method and system for intelligently detecting the appearance quality of prefabricated beams.

[0006] This prefabricated beam appearance quality intelligent detection system includes: image acquisition equipment, image storage equipment, intelligent detection workstation, Internet and application server; several image acquisition equipment are connected to the intelligent detection workstation, the information in the intelligent detection workstation is stored in the image storage equipment, and the information in the image storage equipment is uploaded to the application server via the Internet.

[0007] The intelligent detection method for the appearance quality of precast beams by the detection system comprises the following steps:

[0008] S1. The image acquisition device acquires the appearance image of the precast beam; and produces a data set of appearance quality defects of the precast beam;

[0009] S2. Build a YOLOv8 model with single-image data enhancement and mixed-image data enhancement. Set mosaic and improve mixup algorithms for mixed-image data enhancement.

[0010] S3, using the optimized SimOTA algorithm, introducing dynamic anchor boxes to assist label allocation, and conducting model training;

[0011] S4. Build the XLion optimizer that dynamically adjusts the gradient through built-in gradient accumulation to perform parameter optimization;

[0012] S5. The appearance image of the precast beam collected by the image acquisition device in real time is input into the model to detect whether there are appearance quality defects.

[0013] As a preferred method, the mixup algorithm is improved. First, it is determined whether there is a salient target in the input. Then, random sampling from a uniform distribution is added to the label distribution to simulate the label uncertainty in the mixup process. Each sample is linked to the distribution in the label space, and only the class to be enhanced is perturbed.

[0014] As a preference, optimize the SimOTA algorithm and calculate the classification loss of each sample for each GT and regression loss Calculate the loss Loss between the sample in the candidate region and the GT. As a loss aware calculation process, the results of regression loss and classification loss are L reg and L cls , and use positive and negative samples to calculate Loss.

[0015] As a preference, in step S3, the center point B of the prediction box is pred As the center point, construct a gt , height is h gt The dynamic anchor box is used as an auxiliary box to calculate the confidence label value of the model and optimize the predicted box.

[0016] Preferably, momentum and weight decay are added to the Lion optimizer of the prior art, and gradient accumulation is added to obtain the XLion optimizer.

[0017] Preferably, in step S4, an evaluation model is established through various indicators obtained after model training in step S3, and the indicators include mAP@0.5, mAP@0.5:0.95, precision, recall, F1 score, average IoU and confusion matrix; in the evaluation model, the F1 score is first used to determine whether the appearance of the prefabricated beam is smooth and symmetrical, and then the corresponding indicator is selected to evaluate the model performance according to the type of appearance defect of the prefabricated beam to be detected.

[0018] Preferably, when the appearance defects of the precast beams to be detected are cracks or exposed reinforcement, the precision, mAP@0.5 and mAP@0.5:0.95 are selected for model performance evaluation; when the appearance defects of the precast beams to be detected are bubbles or leakage, the recall rate is selected for model performance evaluation.

[0019] The beneficial effects of the present invention are:

[0020] 1) The present invention builds an improved YOLOv8 network and uses a data set to train a target detection model to obtain a prefabricated beam appearance quality detection model; it can effectively replace the traditional prefabricated beam appearance quality defect detection method that relies on manual observation, avoids the problems of strong subjectivity, poor accuracy, and low detection coverage in manual detection of appearance defects, saves time and manpower, and can perform timely dynamic monitoring without causing false detection, misdetection, and missed detection due to recorder fatigue.

[0021] 2) The intelligent detection system for the appearance quality of precast beams proposed in the present invention can automatically collect images of precast beams, conduct detection and analysis, and realize the storage, query, analysis and automatic generation of evaluation reports of appearance quality defect detection data, which is beneficial to the control of the appearance quality of precast beams in the later stage of the project.

[0022] 3) During the implementation of the present invention, the project personnel do not need to participate too much, and have no impact on the construction site, thereby not affecting the original work progress of the project personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a flow chart of intelligent detection of appearance quality of precast beams;

[0024] Figure 2 It is a framework diagram for intelligent detection of the appearance quality of prefabricated beams;

[0025] Figure 3 This is the YOLOv8 network model diagram;

[0026] Figure 4 Schematic diagram of crack detection process.

[0027] Explanation of the accompanying drawings: image acquisition device 1, image storage device 2, intelligent detection workstation 3, Internet 4, application server 5, user PC terminal 6, mobile phone terminal 7. DETAILED DESCRIPTION

[0028] The present invention is further described below in conjunction with embodiments. The description of the following embodiments is only used to help understand the present invention. It should be noted that for ordinary persons in the art, without departing from the principle of the present invention, the present invention can also be modified in some ways, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

[0029] Embodiment 1

[0030] As an example, Figure 1 As shown, the method for intelligent detection of the appearance quality of prefabricated beams comprises the following steps:

[0031] S1. Industrial cameras are used to capture high-definition images of precast beam appearance quality defects. The images contain cracks, bubbles, exposed reinforcement, corner drop, mortar leakage, honeycomb, and roughness defects of precast box beams and precast T beams; a precast beam appearance quality defect dataset (PB-AQ-DFT-Data) is created, including training, validation, and test datasets; the data is exported as a YOLOv8 format dataset, which is divided into training, validation, and test datasets according to a ratio of 7:2:1.

[0032] S2. Set the YOLOv8 model configuration file and specify the model parameters, including input image size, anchor box, detection category, starting learning rate, optimization strategy, epochs, confidence level, etc. Figure 3 As shown, SPPF is a "spatial pyramid pooling" feature extraction network that can effectively fuse feature information from different scales; C2f is an "aligned feature pyramid" feature extraction network that is used to enhance feature expression and fusion capabilities; ConV is a "convolution" module, which is the core part of a convolutional neural network and is used for feature extraction and data processing; upsample is an "up-sampling" module, which refers to the process of converting a low-resolution feature map or image into a high-resolution one; Concat is a "fully connected" layer that is used to merge features from different sources; Detect is a "detection head" that is designed to identify objects in images or videos and determine their locations. The present invention improves the network architecture of YOLOv8, adds a layer of Detect detection head for small target detection at a high level, so that the input image enters the detection head (Detect) for target detection after two convolution (ConV) operations and one alignment feature pyramid (C2f) feature extraction; and integrates the channel attention mechanism and the spatial attention mechanism, in the detection scene of small targets such as cracks and bubbles, the detection accuracy of the model in the detection scene of small targets such as cracks and bubbles is improved.

[0033] S3, using optimized SimOTA algorithm and introducing dynamic anchor frame to assist calculation The label allocation strategy is used to train the model, reduce parameters, shorten training time, and solve the optimal allocation problem;

[0034] S4. The precast beam appearance quality defect detection model was evaluated on the validation set of PB-AQ-DFT-Data, and the model’s mAP@0.5 (mAP50), mAP@0.5:0.95 (mAP50:95), precision, recall, F1 score, mean IoU (mIoU), confusion matrix and other indicators were obtained for evaluation.

[0035] Analyze the evaluation results of the precast beam appearance quality defect detection model and adjust the model's training parameters such as learning rate, batch size, and number of iterations to find a better parameter combination;

[0036] S5. Input the real-time collected precast beam appearance image into the trained precast beam appearance quality defect detection model for reasoning to detect whether there are appearance quality defects. Figure 4 As shown, check whether there are defects such as cracks, bubbles, exposed reinforcement, corner loss, mortar leakage, honeycomb, and rough surface, and save the test results.

[0037] The application system performs statistical analysis based on the test results and presents various statistical analysis charts; generates a test evaluation report for a single beam, outputs the test results, and gives evaluation grades and improvement suggestions.

[0038] Embodiment 2

[0039] As another embodiment, this embodiment 2 proposes a more specific method for intelligent detection of the appearance quality of prefabricated beams based on the embodiment 1:

[0040] In step S1, the x-anylabeling software defines labels such as cracks, bubbles, exposed ribs, corner drop, slurry leakage, honeycomb, and rough surface, and loads the collected defect images into x-anylabeling using SAM for auxiliary labeling.

[0041] In step S2, single image data enhancement and mixed data enhancement are configured. For single image data enhancement, RandomResize, RandomCrop, HSVRandomAug, and RandomFlip are set. RandomResize randomly resizes the image to different sizes and uses different interpolation strategies to enhance the robustness of the model to targets of different sizes and positions. RandomCrop randomly crops different parts of the image to increase data diversity and enhance the generalization of the model. HSVRandomAug randomly adjusts the brightness, contrast, and saturation of the image to enhance the model's adaptability to color changes. RandomFlip randomly flips the image horizontally to increase data diversity.

[0042] For mixed class data enhancement, mosaic and mixup are set. Mosaic is turned off in the last 10 epochs to effectively prevent model overfitting and improve model accuracy. Mixup is improved by first determining whether there is a significant target in the input, then randomly sampling from a uniform distribution and adding it to the label distribution to simulate label uncertainty in the mixup process, linking each sample to the distribution in the label space, and improving the core calculation method of mixup to perturb only the class to be enhanced. The improved formula is:

[0043] λ=(1-γ1-γ2)λ0+γ1λ γ +γ2λ s

[0044] λ γ ,λ s ~Beta(α,α)

[0045] Among them, γ1 and γ2 are hyperparameters, which determine the final λ γ and λ s The ratio of γ , s It is sampled from Beta(α,α), where α is a positive parameter that controls the degree of mixing, usually between [0.1,0.4], and λ0 is the starting random ratio in the original mixup.

[0046] Embodiment 3

[0047] As another embodiment, this embodiment 3 proposes a more specific method for intelligent detection of precast beam appearance quality based on embodiments 1 and 2: In step S3, the model training adopts the SimOTA label allocation strategy to reduce parameters and shorten the training time. SimOTA solves the optimal allocation problem and achieves the lowest cost of assigning anchor points to gt. The cost cost is the regression loss, classification loss and other contents.

[0048] The expression of cost is:

[0049]

[0050] is the classification loss of the sample labeled i for the GT labeled j, is the regression loss.

[0051] The specific process of SimOTA is as follows:

[0052] 1) Determine the candidate regions of positive samples, judge them according to the center points in the spatial scale, and use the area at each center as the candidate region.

[0053] 2) Calculate the regression loss Reg loss and classification loss Cls loss of each sample for each GT, calculate the loss between the sample in the candidate region and the GT, and use it as a loss aware calculation process to obtain the results of Reg loss and Cls loss.

[0054] The expression of Reg loss is:

[0055] L reg = -log(IoU(B gt ,B pred ))

[0056] B gt , B pred are the center points of the real box and the predicted box, IoU(B gt ,B pred ) is the ratio of the intersection and union of the predicted box and the real box. However, due to the difference in width and height, IoU (B gt ,B pred ) and the confidence level are small, the prediction box may be gradually ignored in the subsequent iterative training process. pred As the center point, construct a rectangle with width and height of w gt 、h gt The dynamic anchor box, center point B predIt is continuously updated as the prediction box changes dynamically to generate an auxiliary box with more accurate width and height for calculating the confidence label value, thus promoting the prediction box to be optimized into a high-quality prediction box.

[0057] The expression of Cls loss is:

[0058]

[0059] t i is the true label of the i-th sample. If the sample belongs to the positive class, it is 1, and if it belongs to the negative class, it is 0. i is the probability that the i-th sample is predicted to be a positive class.

[0060] 3) Use the predicted samples of each GT to determine the k value in the number of positive samples (Dynamic_k) it needs to be assigned. The operation is as follows: get the top 10 samples of the current GT's iou; sum and round the iou of the top 10 samples to get the current GT's danamic_k, and the minimum danamic_k is guaranteed to be 1. The first 10 numbers of iou have little impact on the results, and the value between 5-15 has a relatively small impact on the results.

[0061] 4) Abandon the global optimization solution process and only take the first danamic_k samples with the smallest loss for each GT as positive samples.

[0062] 5) Shared positive samples are deduplicated. If the detection box is recognized as a positive sample by multiple ground truths, the weighted loss is used again to judge and assign the detection box to the ground truth with the smallest weighted loss as its unique positive sample. (6) All positive samples other than those found by the above process are negative samples. The loss is calculated using positive and negative samples, including L reg and L cls , only the target box and the filtered positive sample prediction box are calculated, L obj Calculate the anchor points of all positive and negative samples. The calculation expression of Loss is:

[0063]

[0064] Where L cls is the classification loss, L reg is the regression loss, L obj is the obj loss, λ is the weighted coefficient of positioning loss, N pos Represents the number of samples classified as positive.

[0065] It should be noted that the parts in this embodiment that are the same or similar to those in Embodiments 1 and 2 can be referenced to each other and will not be described in detail in this application.

[0066] Embodiment 4

[0067] As another embodiment, this embodiment 4 proposes a more specific method for intelligent detection of the appearance quality of precast beams on the basis of embodiments 1 to 3. In step S4, the model is evaluated by various indicators, specifically:

[0068] mAP@0.5 (mAP50) is the average precision calculated when the IoU threshold is 0.5. It measures the accuracy of the model in detecting objects.

[0069] mAP@0.5:0.95 (mAP50:95) is the average of the mean average precision calculated at different IoU thresholds. Specifically, the IoU threshold increases from 0.5 to 0.95 in steps of 0.05. It evaluates the performance of the model more rigorously and comprehensively.

[0070] Precision refers to the proportion of all instances predicted by the model as positive samples that are actually positive samples. The calculation formula is:

[0071]

[0072] Among them, TP is the number of true positive samples, and FP is the number of false positive samples.

[0073] Recall refers to the proportion of all instances that are actually positive samples that the model correctly predicts as positive samples. The expression is:

[0074]

[0075] Among them, FN is a false negative.

[0076] The F1 score is the harmonic mean of precision and recall, which takes into account the precision and recall of the model. The expression is:

[0077]

[0078] mIoU is the average intersection over union (IoU) between the predicted bounding box and the true bounding box. The calculation expression of IoU is:

[0079]

[0080] The confusion matrix shows the comparison between the predicted results and the actual results, including true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN). Through the confusion matrix, you can understand the performance of the model on different categories in detail.

[0081] In the evaluation, the F1 index is used as the main indicator to observe whether the performance of the index is smooth and symmetrical; secondly, the true positive and false positive ratios of the confusion matrix are observed to be reasonable; then, core indicators and auxiliary indicators are selected to evaluate the model performance for different goals. For defects such as cracks and exposed reinforcement that may affect the structural safety of precast beams, the precision rate, mAP50, and mAP50:95 are used as core indicators, and the recall rate is used as an auxiliary indicator to evaluate the model; for defects such as bubbles and grout leakage that affect the aesthetics of precast beams, the recall rate is used as the core indicator, and the precision rate is used as an auxiliary indicator to evaluate the model.

[0082] Embodiment 5

[0083] As another embodiment, this embodiment five proposes a more specific method for intelligent detection of prefabricated beam appearance quality on the basis of embodiments one to four. In step S4, the Lion gradient optimization algorithm is introduced and improved to improve the optimization effect of the gradient descent algorithm in the model training process.

[0084] Analyze the evaluation results of the prefabricated beam appearance quality defect detection model, adjust the model's learning rate, batch size, number of iterations and other training parameters to find a better combination of parameters. Set the model hyperparameters to use a weight decay of 0.05, use Flat Cosine to fix the learning rate in the first half, use cosine in the second half, refer to bag of freebies to perform weight decay on norm and bias, and pretrain the backbone. The Lion optimizer automatically adjusts the learning rate and step size, and adaptively updates the learning rate according to the gradient of each parameter, so that the model converges more efficiently and is not prone to falling into local optimal points. The momentum mechanism of the Lion optimizer can reduce the shock and oscillation of gradient updates, which helps to improve the stability of the model.

[0085] Based on the existing Lion optimizer, momentum and weight decay are added, and gradient accumulation is added to further improve the generalization ability and robustness of the model, and the XLion optimizer is obtained:

[0086]

[0087] where g t is the gradient of the loss function; sign() is the sign function, i.e. m t When sign(m t ) becomes 1, and negative numbers become -1. t is the original update amount, m t is the momentum value at time t, which is a combination of the momentum value at the previous time and the current gradient. The β parameter controls the contribution ratio of the momentum term. η t is the learning rate, which is used to control the step size of parameter update.t is a regularization parameter used to prevent the parameter from being too large.

[0088] To solve the problem of the optimal weight decay rate, the optimizer update strategy is first simplified as:

[0089] θ t+1 =θ t -(∝ t μ t +ρ t θ t )

[0090]

[0091] Among them, ∝0 is the relative size of the parameter change in the initial stage, which is generally at the level of 10^(-3), indicating that the change in the parameter modulus after each update is approximately one thousandth; q is a hyperparameter, which is generally set to 1; It is a hyperparameter that controls the speed of learning rate decay, which is set according to the size of training data, etc. t is the parameter value at the tth moment, which is updated by subtracting a correction value from the parameter value at the previous moment.

[0092] Because μ t is m t After the sign operation, we get K is the dimension of the parameters; Where σ is the scale of change of the parameter. For a multiplicative matrix, σ 2 is its initial variance. After a series of simplifications, we get:

[0093]

[0094] Among them, ∝ t is the learning rate at time t, ρ t is the weight decay rate at time t.

[0095] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

Claims

1. An intelligent detection system for the appearance quality of prefabricated beams, characterized in that: include: Image acquisition equipment, image storage equipment, intelligent detection workstation, Internet and application server; Several image acquisition devices are connected to the intelligent detection workstation, the information in the intelligent detection workstation is stored in the image storage device, and the information in the image storage device is uploaded to the application server via the Internet.

2. The method for intelligently detecting the appearance quality of prefabricated beams by the detection system according to claim 1 is characterized in that: The following steps are involved: S1. The image acquisition device acquires the appearance image of the precast beam; and generates a data set of appearance quality defects of the precast beam; S2. Build a YOLOv8 model with single-image data enhancement and mixed-image data enhancement. Set mosaic and improve mixup algorithms for mixed-image data enhancement. S3, using the optimized SimOTA algorithm, introducing dynamic anchor boxes to assist label allocation, and conducting model training; S4. Build the XLion optimizer that dynamically adjusts the gradient through built-in gradient accumulation to perform parameter optimization; S5. The appearance image of the precast beam collected by the image acquisition device in real time is input into the model to detect whether there are appearance quality defects.

3. The method for intelligent detection of precast beam appearance quality according to claim 2 is characterized in that: The mixup algorithm is improved by first determining whether there is a salient target in the input, then randomly sampling from a uniform distribution and adding it to the label distribution to simulate label uncertainty in the mixup process, linking each sample to the distribution in the label space, and only perturbing the class to be enhanced.

4. The method for intelligent detection of precast beam appearance quality according to claim 2 is characterized in that: In the optimization of SimOTA algorithm, the classification loss of each sample for each GT is calculated and regression loss Calculate the loss Loss between the sample in the candidate region and the GT. As a loss aware calculation process, the results of regression loss and classification loss are L reg and L cls , and use positive and negative samples to calculate Loss.

5. The method for intelligent detection of precast beam appearance quality according to claim 2, characterized in that: In step S3, the center point B of the prediction box is pred As the center point, construct a gt , height is h gt The dynamic anchor box is used as an auxiliary box to calculate the confidence label value of the model and optimize the predicted box.

6. The method for intelligent detection of precast beam appearance quality according to claim 2, characterized in that: Based on the existing Lion optimizer, momentum and weight decay are added, and gradient accumulation is added to obtain the XLion optimizer.

7. The method for intelligent detection of precast beam appearance quality according to claim 2, characterized in that: In step S4, an evaluation model is established through various indicators obtained after model training in step S3, including mAP@0.5, mAP@0.5:0.95, precision, recall, F1 score, average IoU and confusion matrix; in the evaluation model, the F1 score is first used to determine whether the appearance of the prefabricated beam is smooth and symmetrical, and then the corresponding indicator is selected for the type of appearance defect of the prefabricated beam to be detected to evaluate the model performance.

8. The method for intelligent detection of appearance quality of precast beams according to claim 7, characterized in that: When the pre- When the appearance defects of the beam are cracks or exposed reinforcement, the accuracy, mAP@0.5, and mAP@0.5:0.95 are selected to evaluate the model performance; When the appearance defects of the precast beam to be detected are bubbles or grout leakage, the recall rate is selected to evaluate the model performance.

Images