Bridge apparent disease identification method and system

Abstract

The invention discloses a bridge apparent disease recognition method, which comprises the following steps of: 1, acquiring a two-dimensional image sample of a bridge apparent disease, marking a disease type according to a preset disease classification standard, and constructing an initial image data set; step 2, labeling the initial image data set to generate a label data set, and dividing the label data set into a training set, a verification set and a test set according to a ratio of 7: 2: 1; step 3, constructing a CSW-YOLO v9 model on the basis of a YOLOv9-m model architecture; 4, configuring hyper-parameters of the CSW-YOLO v9 model, performing iterative training by using the training set, performing performance verification through the verification set, and finally generating an optimized weight file; and step 5, inputting a to-be-detected bridge image into the CSW-YOLO v9 model loaded with the weight file, and outputting an apparent disease type and position information. The bridge apparent disease identification method provided by the invention is suitable for intelligent detection work of bridge multi-disease and small-target feature tasks.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent bridges, and particularly relates to a bridge apparent disease identification method and system. BACKGROUND

[0002] Bridge intelligent detection has become an important content of intelligent bridge operation and maintenance. At present, the bridge disease intelligent detection in bridge intelligent detection mainly relies on unmanned aerial vehicles, robots, wall climbing robots and the like.

[0003] However, the current intelligent detection still has many problems. First, the existing bridge detection work classifies diseases into 3 to 5 kinds, but the upper structure diseases of actual bridges are not limited to these kinds, and multiple types of diseases usually exist at the same time. According to JTG / T H21-2011 "Highway Bridge Technical Condition Evaluation Standard", there are 10 common bridge disease types, which are hole, crack, network crack, exposed reinforcement, honeycomb, pitted surface, spalling, corner drop, steel bar corrosion and damage.

[0004] Secondly, in the traditional intelligent detection system based on a deep learning algorithm model, a large amount of data is needed, and these large data need to be trained and reasoned. However, due to the diversity of bridge diseases and the difficulty of obtaining and collecting disease data, the training of the deep learning model is an obstacle.

[0005] Finally, the target detection and reasoning of the traditional detection method are mostly presented in the form of text and table, lack visual interface and real-time display, are not intuitive, and the digital management degree of the detection result is low.

[0006] Therefore, a new algorithm for intelligent identification of bridge diseases and a visual user interface are needed. SUMMARY

[0007] The technical problem to be solved by the present application is to provide a bridge apparent disease identification method and system, which are novel and reasonable in design and convenient to popularize and use.

[0008] To solve the above technical problems, the technical scheme adopted by the present application is as follows: a bridge apparent disease identification method, characterized by comprising the following steps: Step 1, data set construction: obtaining a two-dimensional image sample of the bridge apparent disease, labeling the disease type according to a preset disease classification standard, and constructing an initial image data set; Step 2, data set processing: labeling the initial image data set, generating a label data set, and dividing it into a training set, a validation set and a test set according to a 7:2:1 ratio; Step 3. Model construction: Build the CSW-YOLO v9 model based on the YOLOv9-m model architecture, including: Step 301: Use the ScConv convolution module in the backbone network to perform feature extraction; Step 302: Use the SARepNCSPELAN4 module in the neck network to perform multi-scale feature fusion; Step 303: Use Slideloss loss function to optimize the bounding box regression of disease detection; Step 4: Model training: Configure the hyperparameters of the CSW-YOLO v9 model, perform iterative training using the training set, verify performance using the validation set, and ultimately generate an optimized weight file. Step 5: Defect identification: Input the bridge image to be detected into the CSW-YOLO v9 model loaded with the weight file, and output the apparent defect type and location information.

[0009] The above-mentioned method for identifying apparent bridge defects is characterized in that: obtaining the two-dimensional image of the apparent bridge defects in step one includes field collection through an unmanned aerial vehicle or a mobile terminal device; importing standardized image data from an existing inspection report; and manually reviewing and confirming the defect type labeling.

[0010] The above-mentioned bridge apparent disease identification method is characterized in that: the image data set in the step 1 is divided into a training set, a verification set, and a test set in a ratio of 7:2:1.

[0011] The above-mentioned bridge surface defect identification method is characterized in that: the labeling of the initial image data set in the step 2 includes: using the labelme labeling tool to mark the defect area in the image with a rectangular frame; marking 10 types including holes, cracks, mesh cracks, exposed steel bars, honeycombs, rough surfaces, peeling, chipped corners, steel bar corrosion, and damage.

[0012] The above-mentioned bridge apparent disease identification method is characterized in that the ScConv convolution module in step 301 includes the following steps: Step 3011, spatial reconstruction unit: using a feature map grouping strategy to divide the input features into a first feature group and a second feature group along the channel dimension, performing channel-by-channel spatial attention calculation on the first feature group, and generating a spatial importance weight matrix; Step 3012: Channel reconstruction unit: performing a channel grouping shift operation on the second feature group, including: Step 30121: perform cyclic shift grouping along the channel dimension; Step 30122: Generate channel attention weights using Sigmoid activation function; Step 3013, feature fusion layer: channel splicing the weighted first feature group features and the weighted second feature group features, adjusting the channel dimension after convolution, and outputting the optimized feature map.

[0013] The bridge apparent disease identification method, characterized in that: the calculation of the Slideloss loss function in step 303 includes: calculating a global loss term based on the IoU similarity of the prediction box and the real box; calculating a local loss term by combining local feature distance measurement; and balancing the contribution degrees of the global loss term and the local loss term through an adjustable weight coefficient.

[0014] The bridge apparent disease identification method, characterized in that: the step four, model training includes the following parameterization training strategies: (a) Training parameter configuration: The random gradient descent optimizer is adopted, the momentum coefficient is set to 0.937±0.01, and the weight decay coefficient is set to 0.0005±0.0001; The progressive learning rate strategy is set: the initial learning rate is 0.01, the bias term learning rate is 0.1, and the initial momentum value is 0.8; The input image size is fixed to 640×640 pixels, and the batch size is set to an adjustable range of 8-32, which is 16; (b) Training process control: A two-stage training mechanism is implemented: 3 times of frozen training are first performed, and then 397 times of full-parameter training are performed, and the total training epoch is 400; A dynamic early stopping mechanism is set: when the average precision of the validation set does not improve for 10 consecutive epochs, the training is automatically terminated.

[0015] On the other hand, the application also proposes a bridge apparent disease identification system, characterized by comprising the following modules: Image acquisition module: real-time acquisition of bridge images through a camera or import of stored bridge images; Dataset construction module: obtain the bridge image file, label the disease type according to the preset disease classification standard, and construct an initial image dataset; Dataset processing module: label the initial image dataset to generate a label dataset, and divide it into a training set, a validation set, and a test set in a ratio of 7:2:1; Model construction module: based on the YOLOv9-m model architecture, a CSW-YOLO v9 model is constructed, including: Feature extraction module, ScConv convolution module is used for feature extraction in the backbone network; Feature fusion module, SARepNCSPELAN4 module is used for multi-scale feature fusion in the neck network; The loss optimization module optimizes the boundary box regression of disease detection by using a Slideloss loss function. The model training module configures the hyperparameters of the CSW-YOLO v9 model, iteratively trains by using the training set, performs performance verification by using the verification set, and finally generates an optimized weight file. The disease identification module inputs the bridge image to be detected into the CSW-YOLO v9 model loaded with the weight file, and outputs the apparent disease type and position information. The marking output module marks the identified disease area in the bridge image to be detected and outputs the disease type.

[0016] The bridge apparent disease identification system has the characteristics that the image acquisition module further comprises: The user interface receives the camera start instruction triggered by the user, and calls the connected camera device to perform real-time image acquisition.

[0017] The bridge apparent disease identification system has the characteristics that the image acquisition module further comprises: The file import interface receives the pre-stored image file selected by the user.

[0018] Compared with the prior art, the present application has the following advantages: 1. The method adopted by the present application first determines the bridge upper apparent disease type according to JTG / T H21-2011 "Highway Bridge Technical Condition Evaluation Standard", obtains a disease two-dimensional image, and constructs a disease image initial data set; secondly, a special data set suitable for bridge multi-disease target detection is constructed through data enhancement means, which is used for subsequent deep learning model training and reasoning; the bridge apparent method proposed by the present application is suitable for bridge multi-disease and small target feature task detection work.

[0019] 2. The system adopted by the present application integrates the improved method and realizes the accurate detection of the bridge superstructure multi-disease and high efficiency based on the user interaction design interface.

[0020] In summary, the present application is novel and reasonable, and is convenient to use.

[0021] The technical solutions of the present application will be further described in detail below with the help of the drawings and examples. DETAILED DESCRIPTION

[0022] Figure 1 The flowchart of the bridge apparent disease identification method proposed by the present application.

[0023] Figure 2 The flowchart of the data set processing of the present application.

[0024] Figure 3 A structural schematic diagram of a backbone network layer of the present application.

[0025] Figure 4 A flowchart of a bridge apparent disease identification system proposed by the present application. DETAILED DESCRIPTION

[0026] As Figure 1 shown, the bridge apparent disease identification method of the present application comprises: Step one, data set construction: obtain the two-dimensional image samples of the bridge apparent disease, label the disease type according to the preset disease classification standard, and construct an initial image data set; Exemplarily, first, 10 common disease types of the concrete bridge superstructure apparent disease are determined according to JTG / T H21-2011 “Highway Bridge Technical Condition Evaluation Standard”, then the two-dimensional images of the apparent disease are obtained through field collection, detection reports and other means, and finally the collected disease pictures are manually identified and classified through the determined disease types, so as to ensure the quality and disease accuracy of the initial data set.

[0027] Step two, data set processing: labeling the initial image data set to generate a label data set, and dividing it into a training set, a validation set and a test set according to a 7:2:1 ratio; Exemplarily, as Figure 2 shown, first, the initial data set is labeled, and this process uses the labelme labeling software to label the disease rectangular frame in the two-dimensional image according to the 10 labels that have been determined. After generating the labeled.Json file, it is batch-converted into a label.txt file through Python code, thus constructing a label data set corresponding to the disease image. Finally, the disease data set and the label data set are divided into a training set, a validation set and a test set according to a 7:2:1 ratio.

[0028] It should be noted that due to the diversity of bridge diseases, it is easy to cause the imbalance of the head and tail categories, and for part of the tail category data, data augmentation means is needed to expand the number and diversity of the data set, mainly using 8 kinds of data augmentation means including different combinations of brightness enhancement, rotation, contrast enhancement, flip, affine, skew, translation expansion and HSV data enhancement.

[0029] Step three, model construction: constructing a CSW-YOLO v9 model based on the YOLOv9-m model architecture, including: Step 301, using ScConv convolution module for feature extraction in the backbone network; ScConv is a spatial and channel reconstruction convolution used to reduce the redundant convolution module of the feature in the convolutional neural network (CNN). It aims to reduce the consumption of computing resources and improve network performance by reconstructing features in the spatial and channel dimensions. ScConv consists of two core units: spatial reconstruction unit (SRU) and channel reconstruction unit (CRU). The workflow is that the separation operation separates the redundant features through the trained parameters, and the reconstruction operation enhances the representation ability of the features through reconstruction. The spatial reconstruction unit (SRU) reduces spatial redundancy, and this unit performs separation and reconstruction operations. The channel reconstruction unit (CRU) reduces channel redundancy, and this unit performs segmentation, transformation, and fusion operations. The workflow is to first segment the feature map into multiple sub-feature maps, then perform transformation operation on each feature map, and finally fuse the transformed sub-feature maps to reduce channel redundancy. This convolution module reduces the redundancy in the spatial and channel dimensions, improving the efficiency and performance of the model.

[0030] The structure of the spatial reconstruction unit of the ScConv module is that the Split operation divides the input feature map into two groups, Group1 and Group2. Then the Group Shift operation is performed by the channel reconstruction unit to group and shift the channels. Then the channels are dynamically weighted by the Sigmoid function to suppress redundant channels. Finally, the two groups of features are merged to output the feature map without redundancy. The convolution module of YOLOv9-m uses the CSP structure with depth separable convolution, where the CSP structure reduces redundant calculations by processing part of the channels. This method may lose some channel information.

[0031] Due to the above two differences, the ScConv convolution module has lower computational complexity: by separating redundant features, GFLOPs are significantly reduced, and computational efficiency is significantly improved.

[0032] As shown in Figure 3 , the first layer, the second layer, the fourth layer, the sixth layer, and the eighth layer in the skeleton network structure are ScConv convolution layers.

[0033] Step 302, multi-scale feature fusion in the neck network using the SARepNCSPELAN4 module: The Self Attention (SA) module in the Swintransformer self-attention mechanism is integrated into the RepNCSPELAN4 module to construct the SARepNCSPELAN4 module. Swin Transformer is a Transformer-based visual model that solves the problem of huge computational complexity of traditional Transformers when processing high-resolution images through window self-attention mechanism. Swin Transformer divides the image into fixed-size windows and then performs self-attention calculation within each window.

[0034] The self-attention mechanism generates attention weights by calculating the relationship between Query, Key, and Value. The specific formula is as follows:

[0035] Where X is the input feature matrix, Wq, Wk, Wv are learnable weight matrices.

[0036] Window self-attention calculates the attention weights between all positions within the window, then performs weighted summation to get the context representation of each position within the window. The formula is as follows:

[0037] Where Q, K, V are the query, key, and value within the window, relative_position_bias is the relative position bias used to capture the relationship between positions within the window, d represents the dimension of the key used to scale the dot product.

[0038] Inter-window self-attention uses a sliding window mechanism to fuse information between adjacent windows. This method can capture long-distance dependencies while maintaining computational efficiency. The formula is as follows:

[0039] The Swin Transformer self-attention mechanism uses window segmentation and shifting strategies to effectively transfer and fuse feature maps between different scales, avoiding information loss in traditional convolutional neural networks. At the same time, it not only reduces computational complexity, but also captures local and global features.

[0040] Step 303, using the Slideloss loss function to optimize the bounding box regression of disease detection; The calculation of the Slideloss loss function includes: calculating a global loss term based on the IoU similarity of the prediction box and the real box; calculating a local loss term by combining local feature distance measurement; and balancing the contribution degrees of the two terms through an adjustable weight coefficient.

[0041] The closer the IoU value is to 1, the higher the coincidence degree of the prediction box and the real box, and the easier the sample is; on the contrary, the lower the IoU value is, the more difficult the sample is. By setting a threshold parameter, the samples are divided into three categories: easy samples: the IoU value is lower than the threshold minus a small offset; medium samples: the IoU value is between the threshold minus the offset and the threshold; and difficult samples: the IoU value is higher than the threshold.

[0042] Then, SlideLoss assigns different weights to different types of samples through a weighting function: the weight of easy samples is lower, and the weight of difficult samples is higher, so that the model pays more attention to these samples, thereby improving the processing ability of the model to difficult samples, and the formula is as follows:

[0043] Wherein, x is the IoU value, and μ is a threshold parameter for distinguishing easy samples and difficult samples.

[0044] Step four, model training: configuring the hyperparameters of the CSW-YOLO v9 model, iteratively training using the training set, and verifying the performance through the validation set to finally generate an optimized weight file; After the above improved mode is added to the configuration file, the training parameters are adjusted, and the set parameters include: the initial learning rate is 0.01; the optimizer is selected as SGD, the momentum is 0.937, the weight decay value is 0.0005; the frozen training generation number is 3; the initial momentum value is 0.8; the bias learning rate is 0.1; the input image size is 640*640; the batch picture number is 16; Epochs: 400; in order to prevent the occurrence of overfitting phenomenon, the early stopping patience value is set to 10, and the training will be stopped when the performance on the validation set does not improve in 10 consecutive training rounds.

[0045] It should be noted that Table 1 is a training parameter explanation table provided by the present application Table 1 Training parameter explanation table

[0046] Step five, disease identification: inputting the bridge image to be detected into the CSW-YOLO v9 model loaded with the weight file, and outputting the apparent disease type and position information.

[0047] In addition, the present application also provides a bridge apparent disease identification system applying the above bridge apparent disease identification method, which comprises the following modules: Image acquisition module: real-time acquisition of bridge images through the camera or import of stored bridge images; Exemplarily, the application is based on an intelligent disease real-time detection platform developed in depth based on the PyQt5 framework, and a modular design concept is adopted to construct a professional detection system integrating data acquisition, intelligent analysis and visual interaction.

[0048] Click the "start camera" button in the system interface, and the system calls the camera connected to the computer to realize real-time image acquisition. The 1600-megapixel camera can provide clear and high-resolution images, and the system uses advanced algorithms to analyze the picture in real time, quickly and accurately identifies disease characteristics, and is suitable for on-site real-time detection scenarios such as bridge routine inspection, periodic inspection, maintenance, etc., can timely detect diseases and give early warning.

[0049] It should be noted that the "stop" button can terminate the detection process at any time, and when continuous detection is not required or the detection process needs to be paused due to abnormalities, it is convenient for users to flexibly control the detection process, avoiding invalid operation and resource waste.

[0050] Dataset construction module: obtain the bridge image file, label the disease type according to the preset disease classification standard, and construct an initial image dataset; Dataset processing module: label the initial image dataset, generate a label dataset, and divide it into a training set, a validation set and a test set in a 7:2:1 ratio; Model construction module: based on the YOLOv9-m model architecture, a CSW-YOLO v9 model is constructed, including: Feature extraction module, using ScConv convolution module for feature extraction in the backbone network; Feature fusion module, using SARepNCSPELAN4 module for multi-scale feature fusion in the neck network; Loss optimization module, using Slideloss loss function to optimize the boundary box regression of disease detection; Model training module: configure the hyperparameters of the CSW-YOLO v9 model, use the training set for iterative training, verify the performance through the validation set, and finally generate an optimized weight file; Disease identification module: input the bridge image to be detected into the CSW-YOLO v9 model loaded with the weight file, and output the apparent disease type and position information.

[0051] Exemplarily, clicking the "open file" button in the system interface supports the user to import the stored image file. This is suitable for the scene of after review, comparative analysis at different stages, etc. With the high-definition file taken by the 1600 million pixel camera, the system can perform in-depth analysis on the past recorded images, and provide the basis for disease development trend judgment.

[0052] The label output module: marking the identified disease area in the image to be detected and outputting the disease type.

[0053] Exemplarily, clicking the "detect picture" button in the system interface can perform special disease detection on the imported or real-time collected single picture. The system performs comprehensive analysis on the picture, discriminates the texture, color, shape and other features in the picture by using the foregoing method of the application, marks the disease area and type, and provides detailed and accurate data support for engineering quality evaluation.

[0054] The above is only a preferred embodiment of the present application, not any limitation on the present application, any simple modification, change and equivalent structure change made according to the technical essence of the present application to the above embodiment are still within the protection scope of the technical solution of the present application.

Claims

1. A method for identifying apparent bridge defects, characterized in that: The following steps are involved: Step 1: Dataset construction: Obtain two-dimensional image samples of the apparent damage of the bridge, label the damage type according to the preset damage classification standard, and construct an initial image dataset; Step 2: Dataset processing: annotate the initial image dataset to generate a labeled dataset, and divide it into a training set, a validation set, and a test set in a ratio of 7:2:1; Step 3. Model construction: Build the CSW-YOLO v9 model based on the YOLOv9-m model architecture, including: Step 301: Use the ScConv convolution module in the backbone network to perform feature extraction; Step 302: Use the SARepNCSPELAN4 module in the neck network to perform multi-scale feature fusion; Step 303: Use Slideloss loss function to optimize the bounding box regression of disease detection; Step 4: Model training: Configure the hyperparameters of the CSW-YOLO v9 model, perform iterative training using the training set, verify performance using the validation set, and ultimately generate an optimized weight file. Step 5: Defect identification: Input the bridge image to be detected into the CSW-YOLO v9 model loaded with the weight file, and output the apparent defect type and location information.

2. A bridge apparent defect identification method according to claim 1, characterized in that: The step 1 of obtaining the two-dimensional image of the apparent defects of the bridge includes: on-site collection through a drone or a mobile terminal device; importing standardized image data from an existing inspection report; and manually reviewing and confirming the defect type labeling.

3. A bridge apparent defect identification method according to claim 2, characterized in that: The image dataset in step 1 is divided into a training set, a validation set, and a test set in a ratio of 7:2:

1.

4. A bridge apparent disease identification method according to claim 3, characterized in that: The labeling of the initial image dataset in step 2 includes: using the labelme labeling tool to mark the defective area in the initial image with a rectangular frame; marking 10 types of defects, including holes, cracks, mesh cracks, exposed steel bars, honeycombs, rough surfaces, peeling, corner loss, steel bar corrosion, and damage.

5. A bridge apparent disease identification method according to claim 4, characterized in that: The ScConv convolution module in step 301 includes the following steps: Step 3011, spatial reconstruction unit: using a feature map grouping strategy to divide the input features into a first feature group and a second feature group along the channel dimension, performing channel-by-channel spatial attention calculation on the first feature group, and generating a spatial importance weight matrix; Step 3012: Channel reconstruction unit: performing a channel grouping shift operation on the second feature group, including: Step 30121: perform cyclic shift grouping along the channel dimension; Step 30122: Generate channel attention weights using Sigmoid activation function; Step 3013, feature fusion layer: perform channel concatenation on the features of the weighted first feature group and the features of the weighted second feature group, adjust the channel dimension through convolution, and output the optimized feature map.

6. A bridge apparent disease identification method according to claim 5, characterized in that: The calculation of the Slideloss loss function in step 303 includes: calculating a global loss term based on the IoU similarity between the predicted box and the true box; calculating a local loss term in combination with a local feature distance metric; and balancing the contribution of the global loss term and the local loss term through an adjustable weight coefficient.

7. A bridge apparent disease identification method according to claim 6, characterized in that: The fourth step, model training, includes the following parameterized training strategies: (a) Training parameter configuration: The stochastic gradient descent optimizer was used, with the momentum coefficient set to 0.937 ± 0.01 and the weight decay coefficient set to 0.0005 ± 0.0001; Set the progressive learning rate strategy: the initial learning rate is 0.01, and the bias learning rate is 0.1, the initial momentum value is 0.8; The input image size is fixed at 640×640 pixels, and the number of batch images is 16; (b) Training process control: A two-stage training mechanism is implemented: 3 freeze trainings are performed first, followed by 397 full parameter trainings, for a total of 400 training rounds or epochs. Set up a dynamic early stopping mechanism: automatically terminate training when the average accuracy of the validation set does not improve for 10 consecutive epochs.

8. A bridge apparent disease identification system, characterized by: Includes the following modules: Image acquisition module: collects bridge images in real time through a camera or imports stored bridge images; Dataset construction module: obtains the bridge image file, labels the damage type according to the preset damage classification standard, and constructs the initial image dataset; Dataset processing module: annotate the initial image dataset to generate a labeled dataset, and divide it into a training set, a validation set, and a test set in a ratio of 7:2:1; Model building module: Builds the CSW-YOLO v9 model based on the YOLOv9-m model architecture, including: Feature extraction module, uses ScConv convolution module in the backbone network for feature extraction; Feature fusion module,SARepNCSPELAN4 module is used in the neck network for multi-scale feature fusion; Loss optimization module, using Slideloss loss function to optimize bounding box regression for disease detection; Model training module: configures the hyperparameters of the CSW-YOLO v9 model, performs iterative training using the training set, verifies performance using the validation set, and ultimately generates an optimized weight file; Defect recognition module: Input the bridge image to be detected into the CSW-YOLO v9 model loaded with the weight file, and output the apparent defect type and location information; Marking output module: marking the identified damaged area in the image of the bridge to be detected and outputting the type of damage.

9. According to the bridge surface defect recognition system of claim 8, the image acquisition module further comprises: The user interface receives the camera start command triggered by the user and calls the connected camera device to perform real-time image acquisition.

10. According to the bridge surface defect recognition system of claim 8, the image acquisition module further comprises: Receive the pre-stored image file selected by the user through the file import interface.

Images