A method for detecting structural component problems based on the YOLOv3 model

Abstract

This invention discloses a method for detecting structural component problems based on the YOLOv3 model. This invention relates to the field of engineering inspection technology and includes the following steps: dividing the engineering structure to be inspected into several inspection sub-regions, characterizing the importance of each region by collecting structural geometric features and historical maintenance frequency, acquiring visual images of similar structures, determining defect feature parameters, dynamically adjusting the minimum required number of images based on importance coefficients, expanding the insufficient portion to form a dataset, manually labeling defect types and dividing the dataset into a test set and a training set, training different YOLOv3 model structures using the training set, dynamically adjusting the annealing start iteration using a preheating-platform-annealing strategy during training, and finally evaluating the model through the test set to determine the optimal network structure to identify the defect type and location of the region to be inspected, achieving both rapid convergence and good generalization ability, thus achieving higher accuracy.

Description

Technical Field

[0001] This invention relates to the field of engineering testing technology, specifically a method for detecting structural component problems based on the YOLOv3 model. Background Technology

[0002] The structural performance of engineering structures, such as steel structures, bridges, and tunnels, gradually declines with service life, often manifesting as various types and degrees of defects in their appearance, thus affecting their safe operation. Therefore, timely detection of structural component defects and assessment of their technical condition are of significant engineering importance. Traditional methods for detecting structural component defects involve visual inspection by personnel using inspection equipment. This method is time-consuming and labor-intensive, and suffers from drawbacks such as concerns about personnel safety, traffic congestion, and strong subjectivity. Furthermore, in the intelligent positioning and detection of structural components, traditional visual methods require manually designed features and classifiers for classification, which is insufficient to meet the actual inspection needs of different structural types and component forms.

[0003] In recent years, with the rapid development of deep learning and computer vision technologies, image classification and object detection technologies based on convolutional neural networks have gradually matured. These technologies offer advantages such as high detection efficiency, high automation, and high recognition accuracy, and their application in infrastructure fields can effectively overcome the shortcomings of traditional detection methods. YOLO v3, a deep learning algorithm, boasts advantages such as real-time performance and high accuracy, and has been widely used in the field of object detection.

[0004] With the continuous improvement of computer hardware performance and the development of deep learning algorithms, YOLOv3-based target detection algorithms have become one of the mainstream methods for intelligent positioning and detection of structural components. At the same time, there are also some areas for improvement, such as better adapting to different scenarios and improving detection accuracy.

[0005] In the prior art, CN111476756A discloses a method for identifying porosity defects in DR images of castings based on an improved YOLOv3 network model. The specific steps include: labeling the porosity defect dataset with rectangular boxes using an image annotation tool; establishing an improved YOLOv3 network model; training the improved YOLOv3 network model using a porosity defect data training set; testing the trained improved YOLOv3 network model using a porosity defect data test set; improving the improved YOLOv3 network model; acquiring DR images of the casting to be detected and inputting them into the improved YOLOv3 network model to determine the defect level and location coordinates of the casting. However, the dataset for this method is mainly constructed by acquiring DR images of several castings, which may lack sufficient diversity and representativeness, especially in terms of different types of castings and defect manifestations. Furthermore, the model training process mainly relies on preset parameters and hyperparameters, without considering strategies for dynamically adjusting the learning rate and other hyperparameters, which may lead to low training efficiency and unsatisfactory convergence.

[0006] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] The purpose of this invention is to provide a method for detecting structural component problems based on the YOLOv3 model, so as to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A method for detecting structural component problems based on the YOLOv3 model, the specific steps of which include:

[0010] The entire engineering structure to be inspected is divided into several inspection sub-regions. The structural geometric features of each inspection sub-region and the historical number of repairs of each inspection sub-region are collected, and the importance of each inspection sub-region is characterized by this.

[0011] Collect structural appearance images of corresponding detection sub-regions in similar engineering structures, determine the structural defect feature parameters of each structural appearance image, determine the increment ratio based on the importance coefficient of each detection sub-region, dynamically adjust the minimum required number of structural appearance images of each detection sub-region based on the increment ratio, expand the structural appearance images that are less than the minimum required number, and summarize all expanded structural appearance images to form an image dataset.

[0012] Structural defects are labeled in the structural appearance images by manual annotation to add structural defect type markers. These are recorded as labeled images and stored in the corresponding image dataset. The image dataset is divided into a test set and a training set. The structural appearance images in the training set are used as input and the corresponding labeled images are used as labels to train several YOLOv3 models with different network structures.

[0013] For each training round employing the preheat-platform-annealing strategy, based on the structural defect feature parameters of the input structural appearance image during that round of training, the initial iteration round of annealing is dynamically adjusted. The trained YOLOv3 model is tested using a test set to obtain evaluation parameters for YOLOv3 models with different network structures. Based on these evaluation parameters, the optimal network structure of the YOLOv3 model is determined. The structural appearance images of each detection sub-region in the engineering structure to be detected are then input into this model to determine the type and location of structural defects.

[0014] Furthermore, the logic behind dividing the entire engineering structure to be inspected into several inspection sub-regions is as follows: the engineering structure to be inspected is divided into regions according to the component type to obtain several inspection sub-regions;

[0015] The structural geometric features of each detection sub-region specifically include the cross-sectional area and radius of curvature parameters of the engineering structure within the detection sub-region. The importance of each detection sub-region is characterized based on its structural geometric features and the number of historical maintenance operations. The importance of each detection sub-region is then characterized by an importance coefficient.

[0016] Furthermore, specific methods for characterizing the detection importance of each detection sub-region using importance coefficients include:

[0017] For a single detection sub-region, obtain the cross-sectional area and radius of curvature of all the sub-regions, calculate the average cross-sectional area and average radius of curvature within the sub-region, and determine the maximum cross-sectional area and maximum radius of curvature within the sub-region.

[0018] The historical maintenance parameters of all detection sub-regions are normalized, and the normalized historical maintenance parameters are calibrated as the first importance coefficient of the detection sub-region.

[0019] For each detection sub-region, the ratio of the average cross-sectional area to the maximum cross-sectional area of ​​the detection sub-region is designated as the second importance coefficient of the sub-region, and the ratio of the average radius of curvature to the maximum radius of curvature of the detection sub-region is designated as the third importance coefficient of the sub-region.

[0020] The first importance coefficient, the second importance coefficient, and the third importance coefficient are weighted according to the preset importance evaluation weights to obtain the importance coefficient of the corresponding detection sub-region.

[0021] Furthermore, the structural defects in the structural appearance image are labeled manually, and a structural defect type label is added. The specific logic is as follows: The acquired structural appearance image is labeled using the MATLAB labeling tool Image Labeler. Specifically, the structural defects are labeled on the structural appearance image using the minimum bounding rectangle. The minimum bounding rectangle is recorded as the defect label box, and a type label is added to the defect label box to complete the image labeling.

[0022] The logic behind determining the increment ratio based on the importance coefficient of each detection sub-region is as follows: the percentage of the importance coefficient of each detection sub-region in the total importance coefficient is used as the increment ratio of each detection sub-region.

[0023] The steps for dynamically adjusting the minimum required number of structural appearance images for each detection sub-region based on the increment ratio include:

[0024] For each detection sub-region, the product of its increment ratio and the preset minimum required number of structural appearance images is used as the growth amount of that detection sub-region;

[0025] The minimum required number of dynamically adjusted structural appearance images is updated to the sum of the preset minimum required number of structural appearance images and the growth number.

[0026] Furthermore, the logic for determining whether expansion is needed is as follows: if the number of structural appearance images of the detected sub-region is less than the minimum required number of dynamically adjusted structural appearance images of the detected sub-region, then the simulated image data of the corresponding detected sub-region is generated through the adversarial network for expansion, expanding to the minimum required number of dynamically adjusted structural appearance images; if the number of structural appearance images of the detected sub-region is greater than or equal to the minimum required number of dynamically adjusted structural appearance images of the detected sub-region, then no expansion is performed. The adversarial network includes a generator and a discriminator.

[0027] Furthermore, several YOLOv3 models with different network structures were trained, including DarkNet-53, ResNet-18, GoogLeNet, SqueezeNet, and MobileNet-v2. The same training parameters were set for the YOLOv3 models with different network structures. The training parameters included the optimizer, initial learning rate, number of iterations in the warm-up phase, total number of iterations in a single training round, number of training rounds, and training batch size.

[0028] The structural defect feature parameters include the number of pixels and the average gray value of each pixel in each structural defect region within the structural appearance image.

[0029] Furthermore, the preheat-platform-annealing strategy specifically refers to setting a preheating stage, a platform stage, and an annealing stage during a training round. The annealing stage includes a first annealing stage and a second annealing stage, and a preset value is set for the number of iteration rounds to be performed in each stage. During the preheating stage, the learning rate increases with the number of iteration rounds until it reaches its maximum value at the platform stage. The learning rate remains unchanged during the platform stage until the first annealing stage is reached. During the first annealing stage, the learning rate remains unchanged and is lower than the learning rate of the platform stage. This continues until the second annealing stage is reached. During the second annealing stage, the learning rate remains unchanged and is lower than the learning rate of the first annealing stage. When the number of iteration rounds to be performed in the second annealing stage reaches the preset value, the training round is completed. The sum of the preset values ​​for the number of iteration rounds in each stage of the same training round is equal to the total number of iterations in a single training round.

[0030] Furthermore, the logic for dynamically adjusting the starting iteration round of the annealing phase is as follows: For each structural defect region in the structural appearance image during this training round, a set of neighboring regions is defined, and the average gray value of each neighboring region is extracted to calculate the gray value similarity of each structural defect region. The specific steps for determining the gray value similarity include:

[0031] For each structural defect region in the structural appearance image, the mean square error between the average gray value of pixels within the structural defect region and the average gray value of pixels in multiple neighboring regions is calculated, and this mean square error is used as the gray value similarity of the structural defect region.

[0032] The number of iterations corresponding to two annealing operations is adjusted using the mean grayscale similarity and the mean area of ​​structural defect regions of all input structural appearance images in one round of training. The specific adjustment steps include:

[0033] For a single training round, all structural defect regions within the entire input structural appearance image are identified, the gray value similarity and area of ​​all structural defect regions are obtained, and the mean gray value similarity and mean area of ​​all structural defect regions are calculated.

[0034] The ratio of the mean grayscale similarity value to the reference grayscale similarity value is defined as the first influence coefficient of annealing adjustment, and the ratio of the mean area of ​​the structural defect region to the reference area of ​​the structural defect region is defined as the second influence coefficient of annealing adjustment.

[0035] The difference between the first influence coefficient and the second influence coefficient is used as the adjustment ratio parameter for the starting iteration round of the first annealing. The product of the preset starting iteration round of the first annealing and the adjustment ratio parameter is calibrated as the adjustment amplitude coefficient of the starting iteration round. The sum of the adjustment amplitude coefficient of the starting iteration round and the preset starting iteration round of the first annealing is used as the starting iteration round of the adjusted first annealing.

[0036] The same method is used to adjust the starting iteration round of the second annealing.

[0037] Furthermore, evaluation parameters for YOLOv3 models with different network structures are obtained. These evaluation parameters include average prediction precision, average multi-class precision, accuracy, and recall. The evaluation parameters of YOLOv3 models with different network structures on the test set data are normalized, and the YOLOv3 model with the largest sum of evaluation parameter values ​​is selected as the optimal YOLOv3 model.

[0038] Compared with the prior art, the beneficial effects of the present invention are:

[0039] First, dividing the engineering structure to be inspected into regions allows for the accurate identification of the importance of each inspection sub-region based on different structural characteristics and historical maintenance data. This makes the inspection process more targeted, prioritizing areas of high importance and thus improving the accuracy of the inspection. Compared with traditional global inspection methods, this can effectively reduce the false detection rate and the missed detection rate, ensuring that key structures receive focused attention.

[0040] Secondly, data augmentation techniques are used to annotate structural defects in images through manual annotation, and the incremental ratio is determined according to the importance coefficient, which enhances the robustness of the model under different conditions. With the expanded dataset, the YOLOv3 model can learn richer features and improve its ability to identify structural defects.

[0041] Furthermore, the dynamic adjustment strategy of preheating-platform-annealing effectively improves the efficiency and stability of model training. During the training process, the dynamic adjustment of the learning rate enables the model to adapt more effectively to changes in data features at different stages, avoiding problems such as training instability and slow convergence caused by improper learning rate settings. This allows the model to converge quickly and maintain good generalization ability during training, thereby achieving higher accuracy in the testing phase. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the overall method flow of the present invention;

[0043] Figure 2 This is a graph showing the change in the learning rate throughout the training process of this invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0045] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0046] Example:

[0047] Please see Figures 1-2 The present invention provides a technical solution:

[0048] A method for detecting structural component problems based on the YOLOv3 model, the specific steps of which include:

[0049] Step 1: Divide the entire engineering structure to be inspected into several inspection sub-regions, collect the structural geometric features of each inspection sub-region and the historical number of repairs of each inspection sub-region, and use this to characterize the importance of each inspection sub-region;

[0050] This embodiment uses a bridge engineering structure as the target subject for explanation. The entire engineering structure to be inspected is divided into several inspection sub-regions. The specific logic is as follows: the engineering structure to be inspected is divided into regions according to different component types to obtain several inspection sub-regions. The component types of the bridge engineering structure include: pier, cap beam, abutment, wing wall, cone slope and riverbed.

[0051] The structural geometric features of each detection sub-region specifically include the cross-sectional area and radius of curvature parameters of the engineering structure within the detection sub-region. The method for obtaining the structural geometric features of each detection sub-region is as follows: within each detection sub-region, a series of cross-sections are selected, usually planes perpendicular to the principal axis of the component. The area of ​​the cross-section is calculated by analyzing the point cloud or contour on the plane. Alternatively, the cross-sectional graphic can be directly extracted from the CAD model or 3D model, and the area can be calculated using geometric calculation formulas.

[0052] The radius of curvature can be obtained by analyzing the geometric features of the boundary of the detection sub-region. In practice, curve fitting technology in a 3D model can be used to calculate the curvature of each component. During the calculation, several key points on the boundary can be selected, and the derivative of the curve can be calculated using the coordinates of these points to obtain the curvature, and the radius of curvature can be derived from this. The specific number of historical maintenance visits to the detection sub-region can be obtained from maintenance records, which are recorded in the form of logs. These logs usually list in detail the time, cause of failure, and maintenance method of each maintenance visit.

[0053] The importance of each detection sub-region is characterized based on its structural geometric features and historical maintenance frequency. The specific logic is as follows: by using the average cross-sectional area and radius of curvature parameters, as well as the maximum cross-sectional area and radius of curvature parameters of each detection sub-region, and combining them with the historical maintenance frequency of each detection sub-region, the importance of each detection sub-region is calculated. The importance of each detection sub-region is then characterized by the importance coefficient.

[0054] The structural geometric features of each detection sub-region specifically include the cross-sectional area and radius of curvature parameters of the engineering structure within the detection sub-region. The importance of each detection sub-region is characterized based on its structural geometric features and the historical maintenance frequency. The importance of each detection sub-region is further characterized by an importance coefficient, including:

[0055] For a single detection sub-region, obtain the cross-sectional area and radius of curvature of all the sub-regions, calculate the average cross-sectional area and average radius of curvature within the sub-region, and determine the maximum cross-sectional area and maximum radius of curvature within the sub-region.

[0056] The historical maintenance parameters of all detection sub-regions are normalized, and the normalized historical maintenance parameters are calibrated as the first importance coefficient of the detection sub-region.

[0057] For each detection sub-region, the ratio of the average cross-sectional area to the maximum cross-sectional area of ​​the detection sub-region is designated as the second importance coefficient of the sub-region, and the ratio of the average radius of curvature to the maximum radius of curvature of the detection sub-region is designated as the third importance coefficient of the sub-region.

[0058] The importance coefficients of the first, second, and third importance coefficients are weighted according to preset importance assessment weights to obtain the importance coefficient of the corresponding detection sub-region. Specifically, the formula used to calculate the importance coefficient of each detection sub-region is as follows:

[0059]

[0060] In the formula, Let be the importance coefficient of the i-th detection sub-region. and Let be the average cross-sectional area and the maximum cross-sectional area of ​​the i-th detection sub-region, respectively. and Let be the average radius of curvature and the maximum radius of curvature of the i-th detection sub-region, respectively. The normalized value of the historical maintenance count for the i-th detection sub-region is the first importance coefficient for the i-th detection sub-region. That is, the second importance coefficient of the i-th detection sub-region. That is, the third importance coefficient of the i-th detection sub-region, where i is the index of the detection sub-region. , and The weighting coefficients for cross-sectional area, radius of curvature, and number of historical repairs are respectively the preset importance assessment weights, where... ,and , and All are greater than 0.

[0061] It should be noted that the importance coefficient of the i-th detection sub-region Used to characterize detection priority, the magnitude of this coefficient represents the relative importance of the area in the overall structural health monitoring. The higher the importance coefficient, the greater the risk that the area may face in the structural health status, and the higher the importance of its maintenance and monitoring.

[0062] Among them, the first Average cross-sectional area of ​​each detection sub-region This parameter reflects the size of the sub-region and is typically related to its load-bearing capacity and structural stability. The maximum cross-sectional area of ​​each detection sub-region It provides information on the maximum load that the area can potentially bear, which is of significant structural importance. An increase in the value indicates a higher importance for the region's carrying capacity and stability, and therefore the value should be increased accordingly. The value indicates the high importance of the region;

[0063] The average radius of curvature and the maximum radius of curvature of the i-th detection sub-region This reflects the geometric characteristics and stress state of the region. A greater curvature indicates a more complex geometric structure and a higher probability of damage. and and Proportional;

[0064] No. Normalized value of historical repair count for each detection sub-region This parameter reflects the historical maintenance frequency of the area and is generally related to the defect incidence rate and structural health status. Areas with a high number of historical maintenance visits indicate that they may have had more defects in the past and require more attention and priority.

[0065] The frequency of historical maintenance is a key factor in assessing the importance of a structural area. The number of historical maintenance visits often indicates the frequency of potential problems in that area. Areas with frequent maintenance may have ongoing safety hazards. It is the most direct reference variable for judging the damage priority of a structural area, therefore its weight coefficient is the largest. The radius of curvature directly affects the structural stress characteristics and durability, playing an important role in importance assessment. Compared to cross-sectional area, although cross-sectional area affects the structural load-bearing capacity to some extent, its impact on safety and risk assessment is relatively small compared to historical maintenance and curvature characteristics. Therefore, it is set... ,and , and All are greater than 0.

[0066] Step 2: Collect structural appearance images of the corresponding detection sub-regions in similar engineering structures, determine the structural defect feature parameters of each structural appearance image, determine the increment ratio based on the importance coefficient of each detection sub-region, dynamically adjust the minimum required number of structural appearance images for each detection sub-region based on the increment ratio, expand the structural appearance images that are less than the minimum required number, and summarize all expanded structural appearance images to form an image dataset.

[0067] The collection of structural appearance images of similar engineering structures consists of the following two parts: manual and drone photography and network downloads, totaling 14,832 structural appearance images of various scenarios;

[0068] The logic behind manually annotating structural defects in structural appearance images and adding defect type markers is as follows: The acquired structural appearance images are annotated using the MATLAB annotation tool Image Labeler. Specifically, structural defects are annotated on the structural appearance images using the minimum bounding rectangle, which is designated as the defect marker box. At the same time, a type marker is added to the defect marker box to complete the image annotation.

[0069] The specific steps for image annotation include: using the MATLAB annotation tool "Image Labeler" for annotation (e.g., ... Figure 2First, open "Image Labeler" and import all the images that need to be labeled. Second, add labels. First, click "Label" under the "ROI Labels" tab. A "Define New ROI Label" dialog box will pop up. Enter the label name according to the image target, select "Rectangle" from the drop-down menu, and then click the color swatch below "color" to enter the "Select color" dialog box. Select any color that does not overlap with the target image color, and finally click "OK" at the bottom. If there are multiple targets, repeat the above steps in sequence, only changing the label name and the label box color. Then, label. When selecting targets, click the corresponding target name in the left label bar, hold down the left mouse button, and select the entire target. If an image has multiple different targets, repeat the above labeling steps. Finally, save the label file. The file mainly contains two types of information: one is the path of the labeled image, and the other is the label box category for each image and its position in the original image.

[0070] The logic behind determining the increment ratio based on the importance coefficient of each detection sub-region is as follows: the proportion of the importance coefficient of each detection sub-region to the total importance coefficients is used as the increment ratio of each detection sub-region. In other words, the formula for calculating the increment ratio of each detection sub-region is as follows:

[0071]

[0072] In the formula, The incremental ratio of the i-th detection sub-region This represents the total number of sub-regions to be detected.

[0073] It should be noted that the design logic of this formula is based on allocating the incremental proportion according to "relative importance." That is, the higher the importance coefficient of a region, the larger its proportion in the whole, reflecting the region's importance in maintenance or inspection. By comparing the importance coefficients of individual regions with the sum, the relative importance of each region in the overall structure can be intuitively seen, highlighting the differences in importance among regions. Divide by the sum, all incremental proportions All of these can be converted into a relative proportion to ensure that the sum of the incremental proportions of all regions is 1. By dynamically adjusting the incremental proportion of each detection sub-region based on the importance coefficient, resources or maintenance efforts can be concentrated on more important and more demanding regions, which helps to improve the overall safety and reliability of the structure and optimize resource allocation.

[0074] The steps for dynamically adjusting the minimum required number of structural appearance images for each detection sub-region based on the increment ratio include:

[0075] For each detection sub-region, the product of its increment ratio and the preset minimum required number of structural appearance images is used as the growth amount of that detection sub-region;

[0076] The minimum required number of structural appearance images after dynamic adjustment is updated to the sum of the preset minimum required number of structural appearance images and the increase number; that is, the formula based on which the minimum required number of structural appearance images for each detection sub-region is dynamically adjusted is as follows:

[0077]

[0078] In the formula, This represents the minimum number of dynamically adjusted structural appearance images required for the i-th detection sub-region. The minimum number of structural appearance images required for the i-th detection sub-region;

[0079] It should be noted that, since the importance coefficient has already been explained above... The size of the importance coefficient represents the relative importance of the region in the overall structural health monitoring. The higher the importance coefficient, the greater the risk that the region may face in terms of structural health, and the higher the importance of its maintenance and monitoring. Therefore, more appearance images are needed for training to improve the model's detection accuracy in this region. Sub-regions with higher structural complexity, more historical maintenance, or special structures may be more prone to defects, thus requiring more structural appearance images to improve the training effect of the detection model.

[0080] The logic for determining whether expansion is needed is as follows: if the number of structural appearance images of the detected sub-region is less than the minimum required number of dynamically adjusted structural appearance images of that sub-region, then the corresponding simulated image data of the detected sub-region is generated through an adversarial network to expand it to the minimum required number of dynamically adjusted structural appearance images. If the number of structural appearance images of the detected sub-region is greater than or equal to the minimum required number of dynamically adjusted structural appearance images of that sub-region, then no expansion is performed. The adversarial network includes a generator and a discriminator.

[0081] The Generative Adversarial Network (GAN) is a deep learning model widely used in tasks such as image generation and data augmentation. A GAN consists of two main parts: a generator and a discriminator. The generator's primary task is to generate simulated data that resembles real data, while the discriminator's task is to distinguish between real data and the simulated data generated by the generator. It takes both real and generated images as input and outputs the probability of whether it is a real image. In this way, the discriminator provides feedback, helping the generator continuously improve the quality of its generated images.

[0082] The minimum number of pre-set structural appearance images required for the detection sub-region can be set based on the number of images required for training historical models, combined with expert experience.

[0083] The incremental ratio calculation method based on the relative importance coefficient can adapt to different bridge structures and different inspection sub-regions. Maintenance needs under different conditions can be reflected by adjusting the coefficient, making the method highly universally applicable.

[0084] Step 3: Manually annotate structural defects in the structural appearance images to add structural defect type labels. These are recorded as labeled images and stored in the corresponding image dataset. The image dataset is divided into a test set and a training set. The structural appearance images in the training set are used as input, and the corresponding labeled images are used as labels to train several YOLOv3 models with different network structures.

[0085] Several YOLOv3 models with different network structures are trained. Specifically, the YOLOv3 models with different network structures refer to YOLOv3 models with different backbone network structures, including DarkNet-53, ResNet-18, GoogLeNet, SqueezeNet, and MobileNet-v2. The same training parameters are set for training the YOLOv3 models with different network structures. The training parameters include optimizer, initial learning rate, number of iterations in the warm-up phase, total number of iterations in a single training round, number of training rounds, and training batch size.

[0086] DarkNet-53, the backbone of YOLOv3, is a deep neural network with high accuracy and speed. Its small parameter count allows for fast training, while maintaining high accuracy. ResNet-18 is a deep residual network that effectively addresses the vanishing and exploding gradient problems and also trains quickly. However, its large parameter count results in a longer training time. GoogLeNet is a multi-branch convolutional neural network that effectively reduces the number of parameters while maintaining high accuracy. However, its training speed is slower because it requires training multiple branches. SqueezeNet is a lightweight convolutional neural network with a very small parameter count, resulting in very fast training and a small model size, allowing it to run on resource-constrained devices. However, its accuracy is relatively low. MobileNet-v2 is a lightweight convolutional neural network that enables real-time object detection on mobile devices with high accuracy. However, its training speed is slower because it requires training multiple branches. Different network structures have different advantages in terms of detection accuracy, detection speed and generalization ability. Choosing a network structure that suits your application scenario can improve detection results and efficiency.

[0087] The structural defect feature parameters include the number of pixels and the average gray value of each pixel in each structural defect region within the structural appearance image.

[0088] The specific method for obtaining the number of pixels in each structural defect region is as follows: Apply an edge detection algorithm, such as Canny edge detection, to identify the boundaries of the defect region, and then extract the defect contour. For each identified defect region, the number of pixels corresponding to it can be directly counted. Specifically, this can be done by iterating through the pixels in the region or using functions from an image processing library to quickly obtain the number of pixels.

[0089] The grayscale value of the structural defect pixel reflects the brightness information of the pixel in the defect area. The acquisition process is as follows: using the previously obtained defect marking box, the corresponding area in the input image is extracted to form an image of the defect area. For the extracted defect area, the grayscale value of its pixels is calculated. The RGB image is converted into a grayscale image. The grayscale value of all pixels in the defect area is statistically analyzed, and the average grayscale value is calculated.

[0090] Step 4: For each round of training using the preheat-platform-annealing strategy, based on the structural defect feature parameters of the input structural appearance image during that round of training, dynamically adjust the starting iteration round of annealing during that round of training. Test the trained YOLOv3 model using the test set to obtain the evaluation parameters of YOLOv3 models with different network structures. Determine the optimal YOLOv3 model based on the evaluation parameters. Input the structural appearance images of each detection sub-region in the engineering structure to be detected into this model to determine the type and location of structural defects.

[0091] The preheat-platform-annealing strategy specifically refers to setting a preheating phase, a platform phase, and an annealing phase during a training round. The annealing phase includes a first annealing phase and a second annealing phase, and a preset value is set for the number of iteration rounds to be performed in each phase. During the preheating phase, the learning rate increases with the number of iteration rounds until it reaches its maximum value at the platform phase. The learning rate remains unchanged during the platform phase until the first annealing phase is reached. During the first annealing phase, the learning rate remains unchanged and is lower than the learning rate of the platform phase. This continues until the second annealing phase is reached. During the second annealing phase, the learning rate remains unchanged and is lower than the learning rate of the first annealing phase. When the number of iteration rounds to be performed in the second annealing phase reaches the preset value, the training round is completed. The sum of the preset values ​​for the number of iteration rounds in each phase of the same training round is equal to the total number of iterations in a single training round.

[0092] The logic for dynamically adjusting the starting iteration round of the annealing phase is as follows: For each structural defect region in the structural appearance image during this training round, a set of neighboring regions is defined, and the average gray value of each neighboring region is extracted to calculate the gray value similarity of each structural defect region. The specific steps for determining the gray value similarity include:

[0093] For each structural defect region in the structural appearance image, the mean square error between the average gray value of pixels within the structural defect region and the average gray value of pixels in multiple neighboring regions is calculated, and this mean square error is used as the gray value similarity of the structural defect region.

[0094] The specific formula used to calculate the grayscale similarity is as follows:

[0095]

[0096] In the formula, Let the grayscale similarity be the grayscale values ​​of the u-th structural defect region. Let be the average grayscale value of the pixels in the u-th structural defect region. Let be the average gray value of the pixels in the h-th neighboring region of the u-th structural defect region, where u is the index of the structural defect region in an input structural appearance image, and h is the index of the neighboring region of the structural defect region. , The total number of neighboring regions;

[0097] It should be noted that, It is a commonly used metric for measuring the difference between two signals. It calculates the mean square error between the two signals. It can quantify the grayscale difference between structural defect areas and neighboring areas. The smaller the value, the higher the similarity. It selects the average grayscale value of the image area for comparison, rather than the grayscale value of a single pixel, which can better represent the overall characteristics of the area and reduce errors caused by noise or small local changes.

[0098] The logic for defining a set of adjacent neighborhood regions is as follows: determine the neighborhood range of the structural defect region, divide the neighborhood range into several neighborhood regions. The neighborhood range can be defined based on the geometric center of the structural defect region, with a fixed radius, such as the number of pixels. The area within this radius is considered the neighborhood region. Specifically, a rectangular or circular template can be used to define the neighborhood region, and an appropriate size should be selected to ensure that sufficient information is included.

[0099] The number of iterations corresponding to two annealing operations is adjusted using the mean grayscale similarity and the mean area of ​​structural defect regions of all input structural appearance images in one round of training. The specific adjustment steps include:

[0100] For a single training round, all structural defect regions within the entire input structural appearance image are identified, the gray value similarity and area of ​​all structural defect regions are obtained, and the mean gray value similarity and mean area of ​​all structural defect regions are calculated.

[0101] The ratio of the mean grayscale similarity value to the reference grayscale similarity value is defined as the first influence coefficient of annealing adjustment, and the ratio of the mean area of ​​the structural defect region to the reference area of ​​the structural defect region is defined as the second influence coefficient of annealing adjustment.

[0102] The difference between the first influence coefficient and the second influence coefficient is used as the adjustment ratio parameter for the starting iteration round of the first annealing. The product of the preset starting iteration round of the first annealing and the adjustment ratio parameter is calibrated as the adjustment amplitude coefficient for the starting iteration round. The sum of the adjustment amplitude coefficient for the starting iteration round and the preset starting iteration round of the first annealing is used as the starting iteration round of the adjusted first annealing. That is, the specific adjustment is based on the following formula:

[0103]

[0104] In the formula, This is the starting iteration round for the first annealing after adjustment. The preset starting iteration round for the first annealing. This serves as a reference value for grayscale similarity. The mean grayscale similarity of the input structural appearance images in one round of training is given. The mean area of ​​structural defect regions in the input structural appearance image for one round of training. This serves as a reference value for the average area of ​​the structural defect region; the area of ​​the structural defect region is specifically represented by the number of pixels in that region. This is the first influence coefficient of the annealing adjustment. This is the second influence coefficient of the annealing adjustment. This refers to adjusting the proportional parameters;

[0105] The same method is used to adjust the starting iteration round of the second annealing.

[0106] It should be noted that the iteration rounds refer to the index of the number of iterations. A training round includes a fixed number of iterations. The preset starting iteration rounds for the first and second annealing are set based on expert experience. When the average grayscale similarity and the average area of ​​the structural defect region are both close to the reference values, the values ​​of these two variables are small, thus making... close to The smaller the difference between the grayscale value similarity and the surrounding grayscale value, and the larger the average area of ​​the structural defect region, the easier it is to identify the defect. By increasing the initial iteration round of annealing, the number of iterations with a larger learning rate can be increased to improve the convergence speed; conversely, by decreasing the initial iteration round of annealing, the number of iterations with a larger learning rate can be reduced to achieve further detailed optimization.

[0107] Specific When Greater than When the grayscale similarity is low, it means that the defective area is very different from the surrounding area, making it easy to identify and easy to learn. Therefore, the number of iterations with a large learning rate should be increased. Conversely, the number of iterations with a large learning rate should be reduced.

[0108] when Greater than When the average structural defect area is large, the defect is easy to identify and the learning difficulty is low, so the number of iterations with a large learning rate should be increased; conversely, the number of iterations with a large learning rate should be reduced.

[0109] Evaluation parameters for YOLOv3 models with different network structures are obtained. These evaluation parameters include average prediction precision, average multi-class precision, accuracy, and recall. The evaluation parameters of YOLOv3 models with different network structures on the test set data are normalized. The YOLOv3 model with the largest sum of evaluation parameter values ​​is selected as the optimal network structure YOLOv3 model. Selecting the optimal performance network structure YOLOv3 model is a conventional technique and will not be elaborated here.

[0110] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0111] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0112] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0113] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for detecting structural component problems based on the YOLOv3 model, characterized in that, The specific steps include: The entire engineering structure to be inspected is divided into several inspection sub-regions. The structural geometric features of each inspection sub-region and the historical number of repairs of each inspection sub-region are collected, and the importance of each inspection sub-region is characterized by this. Collect structural appearance images of corresponding detection sub-regions in similar engineering structures, determine the structural defect feature parameters of each structural appearance image, determine the increment ratio based on the importance coefficient of each detection sub-region, dynamically adjust the minimum required number of structural appearance images of each detection sub-region based on the increment ratio, expand the structural appearance images that are less than the minimum required number, and summarize all expanded structural appearance images to form an image dataset. Structural defects are labeled in the structural appearance images by manual annotation to add structural defect type markers. These are recorded as labeled images and stored in the corresponding image dataset. The image dataset is divided into a test set and a training set. The structural appearance images in the training set are used as input and the corresponding labeled images are used as labels to train several YOLOv3 models with different network structures. For each training round employing the preheat-platform-annealing strategy, based on the structural defect feature parameters of the input structural appearance image during that round of training, the initial iteration round of annealing is dynamically adjusted. The trained YOLOv3 model is tested using a test set to obtain evaluation parameters for YOLOv3 models with different network structures. Based on these evaluation parameters, the optimal network structure of the YOLOv3 model is determined. The structural appearance images of each detection sub-region in the engineering structure to be detected are then input into this model to determine the type and location of structural defects.

2. The method for detecting structural component problems based on the YOLOv3 model according to claim 1, characterized in that: The logic behind dividing the entire engineering structure to be inspected into several inspection sub-regions is as follows: the engineering structure to be inspected is divided into regions according to the component type to obtain several inspection sub-regions; The structural geometric features of each detection sub-region specifically include the cross-sectional area and radius of curvature parameters of the engineering structure within the detection sub-region. The importance of each detection sub-region is characterized based on its structural geometric features and the number of historical maintenance operations. The importance of each detection sub-region is then characterized by an importance coefficient.

3. The method for detecting structural component problems based on the YOLOv3 model according to claim 2, characterized in that: Specific methods for characterizing the detection importance of each detection sub-region using importance coefficients include: For a single detection sub-region, obtain the cross-sectional area and radius of curvature of all the sub-regions, calculate the average cross-sectional area and average radius of curvature within the sub-region, and determine the maximum cross-sectional area and maximum radius of curvature within the sub-region. The historical maintenance parameters of all detection sub-regions are normalized, and the normalized historical maintenance parameters are calibrated as the first importance coefficient of the detection sub-region. For each detection sub-region, the ratio of the average cross-sectional area to the maximum cross-sectional area of ​​the detection sub-region is calibrated as the second importance coefficient of the sub-region, and the ratio of the average radius of curvature to the maximum radius of curvature of the detection sub-region is calibrated as the third importance coefficient of the sub-region; The first importance coefficient, the second importance coefficient, and the third importance coefficient are weighted according to the preset importance evaluation weights to obtain the importance coefficient of the corresponding detection sub-region.

4. The method for detecting structural component problems based on the YOLOv3 model according to claim 3, characterized in that: The logic behind manually annotating structural defects in structural appearance images and adding defect type markers is as follows: The acquired structural appearance images are annotated using the MATLAB annotation tool Image Labeler. Specifically, structural defects are annotated on the structural appearance images using the minimum bounding rectangle, which is designated as the defect marker box. At the same time, a type marker is added to the defect marker box to complete the image annotation. The logic behind determining the increment ratio based on the importance coefficient of each detection sub-region is as follows: the percentage of the importance coefficient of each detection sub-region in the total importance coefficient is used as the increment ratio of each detection sub-region. The steps for dynamically adjusting the minimum required number of structural appearance images for each detection sub-region based on the increment ratio include: For each detection sub-region, the product of its increment ratio and the preset minimum required number of structural appearance images is used as the growth amount of that detection sub-region; The minimum required number of dynamically adjusted structural appearance images is updated to the sum of the preset minimum required number of structural appearance images and the growth number.

5. The method for detecting structural component problems based on the YOLOv3 model according to claim 4, characterized in that: The logic for determining whether expansion is needed is as follows: if the number of structural appearance images of the detected sub-region is less than the minimum required number of dynamically adjusted structural appearance images of that sub-region, then the corresponding simulated image data of the detected sub-region is generated through an adversarial network to expand it to the minimum required number of dynamically adjusted structural appearance images. If the number of structural appearance images of the detected sub-region is greater than or equal to the minimum required number of dynamically adjusted structural appearance images of that sub-region, then no expansion is performed. The adversarial network includes a generator and a discriminator.

6. The method for detecting structural component problems based on the YOLOv3 model according to claim 5, characterized in that: Several YOLOv3 models with different network structures were trained, including DarkNet-53, ResNet-18, GoogLeNet, SqueezeNet, and MobileNet-v2. The same training parameters were set for the YOLOv3 models with different network structures. The training parameters included the optimizer, initial learning rate, number of iterations in the warm-up phase, total number of iterations in a single training round, number of training rounds, and training batch size. The structural defect feature parameters include the number of pixels and the average gray value of each pixel in each structural defect region within the structural appearance image.

7. The method for detecting structural component problems based on the YOLOv3 model according to claim 6, characterized in that: The preheat-platform-annealing strategy specifically refers to setting a preheating phase, a platform phase, and an annealing phase during a training round. The annealing phase includes a first annealing phase and a second annealing phase, and a preset value is set for the number of iteration rounds to be performed in each phase. During the preheating phase, the learning rate increases with the number of iteration rounds until it reaches its maximum value at the platform phase. The learning rate remains unchanged during the platform phase until the first annealing phase is reached. During the first annealing phase, the learning rate remains unchanged and is lower than the learning rate of the platform phase. This continues until the second annealing phase is reached. During the second annealing phase, the learning rate remains unchanged and is lower than the learning rate of the first annealing phase. When the number of iteration rounds to be performed in the second annealing phase reaches the preset value, the training round is completed. The sum of the preset values ​​for the number of iteration rounds in each phase of the same training round is equal to the total number of iterations in a single training round.

8. The method for detecting structural component problems based on the YOLOv3 model according to claim 7, characterized in that: The logic for dynamically adjusting the starting iteration round of the annealing phase is as follows: For each structural defect region in the structural appearance image during this training round, a set of neighboring regions is defined, and the average gray value of each neighboring region is extracted to calculate the gray value similarity of each structural defect region. The specific steps for determining the gray value similarity include: For each structural defect region in the structural appearance image, the mean square error between the average gray value of pixels within the structural defect region and the average gray value of pixels in multiple neighboring regions is calculated, and this mean square error is used as the gray value similarity of the structural defect region. The number of iterations corresponding to two annealing operations is adjusted using the mean grayscale similarity and the mean area of ​​structural defect regions of all input structural appearance images in one round of training. The specific adjustment steps include: For a single training round, all structural defect regions within the entire input structural appearance image are identified, the gray value similarity and area of ​​all structural defect regions are obtained, and the mean gray value similarity and mean area of ​​all structural defect regions are calculated. The ratio of the mean grayscale similarity value to the reference grayscale similarity value is defined as the first influence coefficient of annealing adjustment, and the ratio of the mean area of ​​the structural defect region to the reference area of ​​the structural defect region is defined as the second influence coefficient of annealing adjustment. The difference between the first influence coefficient and the second influence coefficient is used as the adjustment ratio parameter for the starting iteration round of the first annealing. The product of the preset starting iteration round of the first annealing and the adjustment ratio parameter is calibrated as the adjustment amplitude coefficient of the starting iteration round. The sum of the adjustment amplitude coefficient of the starting iteration round and the preset starting iteration round of the first annealing is used as the starting iteration round of the adjusted first annealing. The same method is used to adjust the starting iteration round of the second annealing.

9. The method for detecting structural component problems based on the YOLOv3 model according to claim 6, characterized in that: Evaluation parameters for YOLOv3 models with different network structures are obtained. These evaluation parameters include average prediction precision, average multi-class precision, accuracy, and recall. The evaluation parameters of YOLOv3 models with different network structures on the test set data are normalized, and the YOLOv3 model with the largest sum of evaluation parameter values ​​is selected as the optimal YOLOv3 model.

Images