Bridge plate-type rubber support multi-disease detection method based on improved YOLOR

Abstract

The invention relates to the technical field of bridge plate-type rubber support detection, and discloses a bridge plate-type rubber support multi-disease detection method based on improved YOLOR. On the basis of 6787 images, an apparent disease data set of the plate-type rubber support is established, meanwhile, a K-means clustering algorithm is adopted to optimize an anchor frame in a YOLOR model, so that the anchor frame can better accord with the characteristics of plate-type rubber support diseases, and then the optimized YOLOR deep learning model can visualize a detection result on the basis of a computer vision technology, so that the detection accuracy is improved. And the detected disease position and type can be automatically marked on the image. The method does not depend on large-scale detection equipment such as a bridge maintenance vehicle any more, detection personnel can complete detection at a safe position for a bridge in a severe detection environment, the safety of the detection personnel is improved, the subjectivity of the detection personnel is also reduced in the aspect of detection results, and objective and accurate support disease detection can be achieved.

Description

Technical Field

[0001] The invention relates to the technical field of bridge plate type rubber bearing detection, and in particular to a bridge plate type rubber bearing multi-disease detection method based on improved YOLOR. Background Art

[0002] The traditional bridge plate rubber bearing inspection method is mainly based on manual inspection. The manual inspection method not only requires professional knowledge training for the inspectors, but also requires the support of large equipment such as bridge inspection vehicles and scaffolding. In addition, due to the particularity of the bearing position, the inspectors are often in high danger. In addition, in actual inspection operations, even well-trained engineers will miss inspections and make false inspections due to fatigue and other reasons when faced with complex and changeable inspection environments, which will affect the overall efficiency of the inspection operation. Therefore, the traditional manual bridge inspection method has problems such as high inspection cost, low efficiency, strong subjectivity and poor personnel safety. In addition, the bearing health monitoring system widely used at this stage is mainly based on sensors. Due to the high price and maintenance cost of sensors, the data collection time is long and the monitoring results cannot be intuitively displayed. This method also has the disadvantages of long inspection cycle, poor stability and lack of visualization.

[0003] At present, the bearing defect detection methods based on computer vision are mainly based on image classification. This method cannot intuitively display information such as the location and type of the defect. In addition, the existing methods can detect fewer types of defects and can only detect 1 to 3 types of bearing apparent defects such as bearing cracks, shear deformation and rust, which cannot meet the requirements of the current national inspection specifications. Therefore, the current bridge bearing detection methods need to be further improved in terms of detectable disease types, detection accuracy, and display of detection results. It is necessary to develop a bridge bearing apparent disease detection method that can detect many types of diseases and has high detection accuracy. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the shortcomings of the prior art, the present invention provides a bridge plate rubber bearing multi-disease detection method based on improved YOLOR, which has the advantages of more objective and accurate detection results, rapid and convenient, intuitive and effective, and solves the above technical problems.

[0006] (II) Technical solution

[0007] To achieve the above object, the present invention provides the following technical solution: a bridge plate rubber bearing multi-disease detection method based on improved YOLOR, comprising the following steps:

[0008] S1. Collect the disease pictures of plate rubber bearings and build the plate rubber bearing apparent disease dataset;

[0009] S2, cropping, splicing, changing contrast, mirror flipping, adding noise, and rotating the image in step S1 at different angles;

[0010] S3, manually annotating the images in step S1 and the images processed in step S2 according to the annotation format of the Pascal VOC data set using VOTT software to identify the types of diseases in the images, and randomly dividing the annotated images into a training set and a validation set in a ratio of 8:2;

[0011] S4. Based on the plate rubber bearing apparent disease dataset, the anchor frame in the YOLOR model was optimized using the K-means clustering algorithm;

[0012] S5. Train the optimized YOLOR model based on the plate rubber bearing apparent disease dataset;

[0013] S6, evaluate the model YOLOR-K trained in step S5 from four aspects: accuracy P, recall R, average precision mAP and detection speed FPS;

[0014] S7. Deploy the trained model YOLOR-K on a local / cloud server, and set up a web-based detection platform on the local / cloud server to provide users with an upload identification channel. The staff will take photos of the damaged parts of the bearings at the inspection site with a camera, and upload the images to the local / cloud server for plate rubber bearing disease identification;

[0015] S8. After receiving the uploaded image, the local / cloud server performs disease detection on the uploaded image and marks the location and type of the disease in the image.

[0016] As a preferred technical solution of the present invention, the plate rubber bearing disease images in step S1 include cracking, rusting, hollowing, cracking, garbage, bulging and shear deformation.

[0017] As a preferred technical solution of the present invention, the specific steps of optimizing the anchor boxes in the YOLOR model by the K-means clustering algorithm in step S4 are as follows:

[0018] S4.1. Read the size information of all images and annotation frames in the plate rubber bearing apparent disease data set, and represent n annotation frames in the plate rubber bearing apparent disease data set as X = {X1, X2, X3, ..., X n}, where X1, X2, X3, …, X n Respectively represent the first annotation box, the second annotation box, the third annotation box, ..., the nth annotation box;

[0019] S4.2, and randomly select m labeled boxes from the n labeled boxes in step S4.1 as clusters C1, C2, ..., C m The initial cluster center in, They represent the initial cluster center of cluster C1, the initial cluster center of cluster C2, ..., C m The initial cluster center of the cluster;

[0020] S4.3, calculate the intersection over union (IoU) of the n annotation boxes and the cluster center A, and cluster the annotation boxes according to the intersection over union (IoU);

[0021] S4.4, calculate the error function of the current number of iterations, and use the median of each cluster as the new cluster center;

[0022] S4.5, repeat steps S4.3 and S4.4 until the error functions of the two iterations of the vector are equal, that is, E k =E k+1 When , the anchor box optimization ends, where E k represents the error function of the kth iteration, E k+1 represents the error function of the k+1th iteration.

[0023] As a preferred technical solution of the present invention, the intersection-and-union ratio calculation expression in step S4.3 is as follows:

[0024]

[0025] Among them, IoU(X i , A j ) represents the intersection-over-combination ratio of all annotation boxes and all cluster centers, where X i ∩A j represents the intersection of the annotation box and the cluster center, X i ∪A j represents the union of the annotation box and the cluster center, A j , X i They represent the jth cluster center and the ith annotation box in the current calculation process respectively, and X i Assigned to the largest IoU value The corresponding cluster C i .

[0026] As a preferred technical solution of the present invention, the calculation expression in step S4.4 is as follows:

[0027]

[0028] D(X i ,A j )=1-IoU(X i ,A j)

[0029] Among them, D(X i ,A j ) represents the IoU distance, m represents the number of clusters, Indicates the sum of m data, X i ∈C i Represents cluster C i Included X i , Represents cluster C i Sum all the elements in .

[0030] Compared with the prior art, the present invention provides a bridge plate rubber bearing multi-disease detection method based on improved YOLOR, which has the following beneficial effects:

[0031] 1. The present invention has established a plate rubber bearing apparent disease dataset based on 6787 images in accordance with regulatory requirements and actual engineering needs, which includes seven types of diseases, namely cracking, rusting, hollowing, cracking, garbage, bulging, and shear deformation. The YOLOR target recognition model trained on the BD dataset can accurately detect the seven types of diseases, basically meeting the requirements of disease detection types in current national regulations. Compared with traditional bearing disease detection, the present invention no longer relies on large-scale detection equipment such as bridge maintenance vehicles. At the same time, for bridges with harsh detection environments, inspectors can complete the detection in a safe location, which improves the safety of inspectors and reduces the subjectivity of inspectors in the detection results, thereby achieving objective and accurate bearing disease detection.

[0032] 2. The present invention realizes the visualization of detection results based on computer vision technology. The YOLOR deep learning model used can visualize the detection results based on computer vision technology. The location and type of the detected disease can be automatically marked in the image, and the detection results can be viewed more conveniently and intuitively, which is convenient for statistics.

[0033] 3. The web-based detection platform developed by the present invention using the trained YOLOR model in the local / cloud server can provide a channel for image upload detection for detection personnel. In addition, the web-based detection platform developed in the cloud server can also provide an API interface for third-party applications and platforms to call BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the process of the present invention;

[0035] Figure 2 This is a schematic diagram of the YOLOR network framework of the present invention;

[0036] Figure 3This is a schematic diagram of the K-means anchor frame optimization process of the present invention;

[0037] Figure 4 This is a schematic diagram of IoU calculation in the present invention;

[0038] Figure 5 Schematic diagram of optimizing the shape of the anchor frame in the present invention;

[0039] Figure 6 It is a schematic diagram for comparing the detection effects of the present invention;

[0040] Figure 7 Schematic diagram of typical detection results of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] See also Figure 1 - Figure 7 The present invention provides the following technical solution: a bridge plate rubber bearing multi-disease detection method based on improved YOLOR, comprising the following steps:

[0043] S1. Collect the disease images of plate rubber bearings and construct the plate rubber bearing apparent disease dataset (BD dataset);

[0044] First, combined with the current domestic and international specifications and actual engineering needs, the apparent defects of plate rubber bearings are divided into seven categories: cracking, rusting, hollowing, cracking, garbage, bulging, and shear deformation. At the same time, combined with the methods provided by bridge maintenance units and personal photography and collection, the defect images are collected, and all images are annotated with defects based on VOTT. Finally, a plate rubber bearing apparent defect dataset (BD dataset) containing 6787 defect images is established. There are no strict shooting requirements for all images in the BD data, so the BD dataset includes photos of different equipment, angles, environments, and resolutions, ensuring the diversity of the dataset to increase the robustness of the detection model. In order to ensure the recognition effect of each disease, a large number of difficult-to-obtain disease images (such as shear deformation) are supplemented through experiments;

[0045] S2, cropping and splicing the pictures in step S1, changing the contrast, mirror flipping, adding noise, rotating at different angles, and processing all the images obtained above by data enhancement technology, so as to further obtain more images of plate rubber bearing defects, and uniformly save them in .jpg format;

[0046] S3, manually annotate the images in step S1 and the images processed in step S2 using VOTT software according to the annotation format of the Pascal VOC dataset, and mark the bearing defects according to seven apparent defects: cracking, rusting, voiding, cracking, garbage, bulging, and shear deformation. The Pascal-VOC2007 format dataset contains three folders: Annotations, ImageSets, and JPEGImages, which are used to store the annotated .xml files, the training .txt files, and all the data-enhanced images, respectively, and then randomly divide them into a training set and a validation set at a ratio of 8:2;

[0047] S4. Based on the plate rubber bearing apparent disease dataset, the anchor frame in the YOLOR model was optimized using the K-means clustering algorithm;

[0048] YOLOR's network framework diagram is as follows Figure 2 As shown in the figure, it can be seen that the appropriate anchor frame can further improve the detection accuracy of various apparent diseases of plate rubber bearings. Figure 2 Input: input layer, Backbone: backbone network, Neck: neck structure, Head: detection layer, CSPDarknet53: 53-layer deep neural network with cross-stage partial connections, SPP (Spatial PyramidPooling): spatial pyramid pooling, PANet (Path Aggregation Network): path aggregation network;

[0049] The anchor frames in the initial YOLOR model are clustered from the public COCO dataset and cannot accurately reflect the characteristics of bearing diseases. Therefore, based on the BD dataset, the present invention uses the K-means clustering algorithm to optimize the anchor frames in the YOLOR model so that it can better fit the characteristics of plate rubber bearing diseases. The specific steps are as follows:

[0050] S4.1. Read the size information of all images and annotation frames in the plate rubber bearing apparent disease data set, and represent n annotation frames in the plate rubber bearing apparent disease data set as X = {X1, X2, X3, ..., X n}, where X1, X2, X3, …, X n Respectively represent the first annotation box, the second annotation box, the third annotation box, ..., the nth annotation box;

[0051] S4.2, and randomly select m labeled boxes from the n labeled boxes in step S4.1 as clusters C1, C2, ..., C m The initial cluster center in, They represent the initial cluster center of cluster C1, the initial cluster center of cluster C2, …, C m The initial cluster center of the cluster;

[0052] S4.3, calculate the intersection over union (IoU) of the n annotation boxes and the cluster center A, and cluster the annotation boxes according to the intersection over union (IoU);

[0053] The calculation expression of intersection ratio is as follows:

[0054]

[0055] Among them, IoU(X i , A j ) represents the intersection-over-combination ratio of all annotation boxes and all cluster centers, where X i ∩A j represents the intersection of the annotation box and the cluster center, X i ∪A j represents the union of the annotation box and the cluster center, A j , X i They represent the jth cluster center and the ith annotation box in the current calculation process respectively, and X i Assigned to the largest IoU value The corresponding cluster C i .

[0056] S4.4, calculate the error function of the current number of iterations, and use the median of each cluster as the new cluster center;

[0057] The calculation expression is as follows:

[0058]

[0059] D(X i , A j )=1-IoU(X i , A j ) (3)

[0060] Among them, D(X i , A j ) represents the IoU distance, m represents the number of clusters, Indicates the sum of m data, X i ∈C i Represents cluster C i Included X i , Represents cluster C i Sum all elements in S4.5, repeat steps S4.3 and S4.4 until the error functions of the two iterations of the vector are equal, that is, E k =E k+1When , the anchor box optimization ends, where E k represents the error function of the kth iteration, E k+1 represents the error function of the k+1th iteration;

[0061] The optimized anchor frame shape and size information are as follows Figure 5 And as shown in Table 1.

[0062] Table 1 Optimized anchor box size information

[0063]

[0064] S5. Train the optimized YOLOR model based on the plate rubber bearing apparent disease dataset;

[0065] The model training environment is based on the Pytouch1.7.0 deep learning framework, the computer operating system is Ubuntu20.04LTS, and Python 3.8.11, CUDA 11.0, cuDNN 8.4.0 and other training supporting environments are installed. The computer configuration is CPU: AMD RyzenTM95900X processor, GPU: GeForce RTXTM 3080Ti-12G, memory: 32G. Use Anaconda to install the environment required for model deployment, and use Tensorboard to view the training process in real time during the experiment. The optimized YOLOR model is trained based on the BD dataset. The specific training parameters are shown in Table 2. All images in the final BD dataset are divided into training sets and validation sets in a ratio of approximately 8:2, with 5442 training sets and 1345 validation sets in the end;

[0066] Table 2 Training parameters

[0067] parameter Numeric Batch size 4 Round 300 Learning Rate 0.01 Weight reduction 0.0005 Momentum parameter 0.937

[0068] S6, evaluate the model YOLOR-K trained in step S5 from four aspects: accuracy P, recall R, average precision mAP and detection speed FPS;

[0069] The results are shown in Table 3. The precision rate P is the ratio of the number of correctly detected diseases to the number of all detected diseases; the recall rate R is the ratio of the number of correctly detected diseases to the number of all diseases that need to be detected; mAP is a comprehensive performance evaluation index; FPS is the number of images that can be processed per frame;

[0070] Table 3 Detection performance comparison

[0071]

[0072] It can be seen that compared with the initial YOLOR, YOLOR-K has improved by 8.4%, 25.6%, and 16.2% while remaining unchanged. The detection performance has been greatly improved. The detection effect comparison between YOLOR-K and the initial YOLOR is shown in the figure below. Figure 6 As shown;

[0073] S7. Deploy the trained model YOLOR-K on a local / cloud server, and set up a web-based detection platform on the local / cloud server to provide users with an upload identification channel. The staff will take photos of the damaged parts of the bearings at the inspection site with a camera, and upload the images to the local / cloud server for plate rubber bearing disease identification. In addition, the web-based detection platform developed on the cloud server can also provide an API interface for third-party applications and platforms to call;

[0074] S8. After receiving the uploaded image, the local / cloud server performs disease detection on the uploaded image and marks the location and type of the disease in the image.

[0075] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A bridge plate rubber bearing multi-disease detection method based on improved YOLOR, characterized by: The following steps are involved: S1. Collect the disease pictures of plate rubber bearings and build the plate rubber bearing apparent disease dataset; S2, cropping, splicing, changing contrast, mirror flipping, adding noise, and rotating the image in step S1 at different angles; S3, manually annotating the images in step S1 and the images processed in step S2 according to the annotation format of the Pascal VOC data set using VOTT software to identify the types of diseases in the images, and randomly dividing the annotated images into a training set and a validation set in a ratio of 8:2; S4. Based on the plate rubber bearing apparent disease dataset, the anchor frame in the YOLOR model was optimized using the K-means clustering algorithm; S5. Train the optimized YOLOR model based on the plate rubber bearing apparent disease dataset; S6, evaluate the model YOLOR-K trained in step S5 from four aspects: accuracy P, recall R, average precision mAP and detection speed FPS; S7. Deploy the trained model YOLOR on a local / cloud server, and set up a web-based detection platform on the local / cloud server to provide users with an upload identification channel. The staff will take photos of the damaged parts of the bearings at the inspection site with a camera, and upload the images to the local / cloud server for plate rubber bearing disease identification; S8. After receiving the uploaded image, the local / cloud server performs disease detection on the uploaded image and marks the location and type of the disease in the image.

2. According to claim 1, a bridge plate rubber bearing multi-disease detection method based on improved YOLOR is characterized by: The plate rubber bearing disease images in step S1 include cracking, rusting, voiding, cracking, garbage, bulging and shear deformation.

3. According to claim 1, a bridge plate rubber bearing multi-disease detection method based on improved YOLOR is characterized by: The specific steps of optimizing the anchor boxes in the YOLOR model by the K-means clustering algorithm in step S4 are as follows: S4.

1. Read the size information of all images and annotation frames in the plate rubber bearing apparent disease data set, and represent the n annotation frames in the plate rubber bearing apparent disease data set as X = {X1, X2, X3, ..., X n }, where X1, X2, X3, …, X n Respectively represent the first annotation box, the second annotation box, the third annotation box, ..., the nth annotation box; S4.2, and randomly select m labeled boxes from the n labeled boxes in step S4.1 as clusters C1, C2, ..., C m The initial cluster center in, They represent the initial cluster center of cluster C1, the initial cluster center of cluster C2, ..., C m The initial cluster center of the cluster; S4.3, calculate the intersection over union (IoU) of the n annotation boxes and the cluster center A, and cluster the annotation boxes according to the intersection over union (IoU); S4.4, calculate the error function of the current number of iterations, and use the median of each cluster as the new cluster center; S4.5, repeat steps S4.3 and S4.4 until the error functions of the two iterations of the vector are equal, that is, E k =E k+1 When , the anchor box optimization ends, where E k represents the error function of the kth iteration, E k+1 represents the error function of the k+1th iteration.

4. The bridge plate rubber bearing multi-disease detection method based on improved YOLOR according to claim 3 is characterized by: The calculation expression of the intersection-and-union ratio in step S4.3 is as follows: Among them, IoU(X i , A j ) represents the intersection-over-combination ratio of all annotation boxes and all cluster centers, where X i ∩A j represents the intersection of the annotation box and the cluster center, X i UA j represents the union of the annotation box and the cluster center, A j , X i They represent the jth cluster center and the ith annotation box in the current calculation process respectively, and X i Assigned to the largest IoU value The corresponding cluster C i .

5. The bridge plate rubber bearing multi-disease detection method based on improved YOLOR according to claim 3 is characterized by: The calculation expression in step S4.4 is as follows: D(X i ,A i )=1-IoU(X i ,A j ) Among them, D(X i , A j ) represents the IoU distance, m represents the number of clusters, Indicates the sum of m data, X i ∈C i Represents cluster C i Included X i , Represents cluster C i Sum all the elements in .

Images