A method and system for identifying bridge bearing defects based on unmanned aerial vehicles and deep learning networks.

Abstract

This invention discloses a method and system for identifying bridge bearing defects based on unmanned aerial vehicles (UAVs) and deep learning networks. The method includes: planning a UAV inspection route; the UAV performing the inspection task and acquiring images of the bearings of the target bridge to form a bearing image set; preprocessing the bearing image set to obtain an annotated image set; constructing and training an image enhancement model based on a Real-ESRGAN network, inputting the annotated image set into the trained image enhancement model to generate a super-resolution image set; constructing a defect identification model based on a YOLOv8 network, training the defect identification model with the super-resolution image set to identify and locate bearing damage. This invention uses the Real-ESRGAN network for super-resolution image enhancement, greatly improving image quality and the accuracy of defect identification. Furthermore, the fast and accurate target detection capability of the YOLOv8 network enables rapid identification and location of bridge bearing damage.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for identifying bridge bearing defects based on unmanned aerial vehicles and deep learning networks. Background Technology

[0002] The health condition of bridge bearings is directly related to the overall safety and stability of the bridge. Under the influence of multiple factors such as the weight of the bridge beam, environmental changes, and vehicle loads, bearings are prone to damage such as corrosion, cracking, and detachment, which seriously threaten the structural safety of the bridge.

[0003] Traditional manual methods for inspecting bridge bearings suffer from drawbacks such as high risk, strong subjectivity, time-consuming and labor-intensive processes, and a high risk of missed detections. This is especially true for bearings located high and inconspicuously, making inspection even more difficult and risky. In recent years, unmanned aerial vehicle (UAV) technology, with its mobility and wide-area inspection capabilities, has been widely used in engineering inspection, becoming an important means of collecting bridge defect data. However, UAV images are susceptible to interference from flight vibrations, gusts of wind, and shadows, resulting in low image resolution and blurriness, affecting the recognition accuracy of classification detection networks. To improve the utilization rate of image information and meet the needs of bridge bearing defect detection, the initial dataset acquired by UAVs needs to be improved. With the development of deep learning technology, especially the maturity of super-resolution algorithms, technical support has been provided for image quality improvement. At the same time, the large-dataset processing capabilities of classification neural network models have further promoted the application of numerous classification detection models in bearing damage detection.

[0004] However, the types of bearing damage are diverse and difficult to observe. Existing single-classification detection network models are difficult to meet the needs of actual engineering. Furthermore, deep learning models have high requirements for the quality of the dataset, and low-quality datasets will significantly reduce the accuracy of damage identification.

[0005] In view of this, this application proposes a method and system for identifying bridge bearing defects based on unmanned aerial vehicles and deep learning network models to solve the above problems. Summary of the Invention

[0006] To address the problems in the prior art, the present invention aims to provide a method and system for identifying bridge bearing defects based on unmanned aerial vehicles (UAVs) and deep learning network models. The method involves acquiring bridge bearing images using UAVs, optimizing the clarity of the acquired images based on the Real-ESRGAN network model, and using the YOLOv8 network model to classify, identify, and locate damage at various locations on the bridge.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] In a first aspect, embodiments of the present invention provide a method for identifying bridge bearing defects based on unmanned aerial vehicles (UAVs) and deep learning networks, comprising:

[0009] S1: Plan the route for drone inspection missions;

[0010] S2: The drone performs inspection tasks and collects images of the supports of the target bridge to form a support image set;

[0011] S3: Preprocess the support image set to obtain an annotated image set;

[0012] S4: Construct and train an image augmentation model based on the Real-ESRGAN network, and input the labeled image set into the trained image augmentation model to generate a super-resolution image set;

[0013] S5: Construct a disease identification model based on the YOLOv8 network, and train the disease identification model with a super-resolution image set so that the disease identification model can identify and locate support damage.

[0014] Optionally, step S1 includes the following specific steps:

[0015] The environmental images of the target bridge are collected by drones and analyzed to obtain the geometric structure and occlusion status of the target bridge.

[0016] The system obtains the drone's flight endurance and plans the drone inspection route based on the target bridge's geometry and obstruction conditions.

[0017] Optionally, step S3 includes the following specific steps:

[0018] S301: Divide the support images in the support image set into images of equal resolution P×P;

[0019] S302: Label the damaged regions in the segmented image and classify the damaged regions according to the damage type to construct a labeled image set.

[0020] Optionally, step S4 includes the following specific steps:

[0021] S401: Build a high-order degradation model and use the high-order degradation model to perform high-order degradation processing on the labeled image set to obtain a degraded image set;

[0022] S402: Construct an image enhancement model based on the Real-ESRGAN network, and input the degraded image set into the image enhancement model;

[0023] The image enhancement model includes a generator based on an SR network and a discriminator based on a U-net convolutional neural network with skip connections; wherein, the discriminator incorporates spectral normalization to control the Lipschitz constant of the discriminator.

[0024] S403: Based on the generator, feature extraction is performed on the images in the degraded image set to generate feature maps, and upsampling and nearest interpolation operations are performed on the feature maps to obtain high-resolution images;

[0025] S404: Input the degraded image and the high-resolution image from the degraded image set into the discriminator. Based on the error between the degraded image and the high-resolution image, the discriminator returns the loss function parameter update generator.

[0026] S405: The generator and discriminator are trained alternately based on the loss function until the accuracy of the high-resolution image generated by the generator reaches the preset value. Then the training stops and the trained image enhancement model is output.

[0027] S406: Input the labeled image set into the trained image enhancement model to obtain a super-resolution image set.

[0028] Optionally, the higher-order degradation process in S401 includes a first degradation process and a second degradation process;

[0029] The first and second degradation processes each include, in sequence, the Blur algorithm, selection of default interpolation scaling, random selection of noise distribution and form, and JPEG compression;

[0030] The functional expression of the higher-order degradation model in S401 is:

[0031]

[0032] In the formula, x is the degraded image; D n (y) represents the nth degradation process; n is the number of degradations; y is the original image; k is the corresponding convolution kernel; r is the downsampling scale factor; a is noise; JPEG is the compression process.

[0033] Optionally, step S5 includes the following specific steps:

[0034] S501: Divide the super-resolution image set into a training set, a validation set, and a test set according to a preset ratio;

[0035] S502: Construct a disease identification model based on YOLOv8 network;

[0036] S503: Input the training set into the disease identification model for iterative training until the training iteration count reaches the preset number, and then output the trained disease identification model.

[0037] S504: Validate the trained disease identification model based on the validation set to obtain validation data, and use the validation data to construct a confusion matrix and loss curve to determine whether the training accuracy of the disease identification model has reached the preset accuracy.

[0038] If not, return to step S503; if yes, proceed to step S505.

[0039] S505: Test the validated disease identification model based on the test set to evaluate whether the disease identification model can identify and locate support damage; if yes, complete the training of the disease identification model; if no, return to step S503.

[0040] Optionally, the disease identification model based on the YOLOv8 neural network includes an Input module, a Backbone module, and a Head module connected in sequence.

[0041] The Backbone module includes a first CBS module, a second CBS module, a first C2f module, a third CBS module, a second C2f module, a fourth CBS module, a third C2f module, a fifth CBS module, and a fourth C2f module connected in sequence; the Head module is composed of a Classify module.

[0042] The activation function of the CBS module is as follows:

[0043]

[0044] In the formula, SiLU is the activation function, and x is the input tensor.

[0045] Secondly, embodiments of the present invention provide a bridge bearing defect identification system based on unmanned aerial vehicles and deep learning networks, comprising:

[0046] The route planning module is used to plan the route for drone inspection missions.

[0047] The image acquisition module is used by the UAV to perform inspection tasks and acquire images of the bearings of the target bridge to form a bearing image set;

[0048] An image processing module is used to preprocess the support image set to obtain an annotated image set;

[0049] The image augmentation module is used to build and train an image augmentation model based on the Real-ESRGAN network. The labeled image set is input into the trained image augmentation model to generate a super-resolution image set.

[0050] The image recognition module is used to construct a disease recognition model based on the YOLOv8 network. The disease recognition model is trained by a super-resolution image set so that the disease recognition model can identify and locate support damage.

[0051] Thirdly, embodiments of the present invention provide a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the bridge bearing defect identification method as described above.

[0052] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the bridge bearing defect identification method described above.

[0053] The beneficial effects of the embodiments provided by the present invention include:

[0054] This invention greatly enhances image quality and the accuracy of disease identification through an image enhancement model based on the Real-ESRGAN network and a disease identification model based on the YOLOv8 network. The Real-ESRGAN network performs super-resolution enhancement on images, which significantly improves the accuracy of disease identification, especially in the case of low-resolution or blurry images. Meanwhile, the fast and accurate target detection capability of the YOLOv8 network can quickly identify and locate damage to bridge bearings.

[0055] This invention utilizes unmanned aerial vehicles (UAVs) for automated inspection and image acquisition, and analyzes the environmental images acquired by the UAVs to determine the inspection route. This reduces the workload and time of manual inspection, minimizes interference from environmental factors during image acquisition, and significantly improves the efficiency and accuracy of bridge bearing defect identification. At the same time, considering the flight endurance of UAVs, the inspection route is optimized, enabling UAVs to complete more inspection tasks with limited battery power, thus improving resource utilization efficiency. Attached Figure Description

[0056] Figure 1 A flowchart illustrating a bridge bearing defect identification method based on UAV and deep learning network according to an embodiment of this specification is shown.

[0057] Figure 2 A flowchart of another bridge bearing defect identification method based on UAV and deep learning network according to an embodiment of this specification is shown.

[0058] Figure 3 A schematic diagram of the structure of the unmanned aerial vehicle (UAV) device according to an embodiment of this specification is shown;

[0059] Figure 4A schematic diagram of the drone inspection mission route is shown in an embodiment according to this specification;

[0060] Figure 5 A schematic diagram of the architecture of a high-order degradation model according to an embodiment of this specification is shown;

[0061] Figure 6 A schematic diagram of the generator architecture of the Real-ESRGAN network according to an embodiment of this specification is shown;

[0062] Figure 7 A diagram illustrating a bridge bearing training example according to an embodiment of this specification is shown;

[0063] Figure 8 The super-resolution map after image enhancement of the training instance image is shown in the embodiments according to this specification;

[0064] Figure 9 A schematic diagram of the architecture of the disease identification model according to an embodiment of this specification is shown;

[0065] Figure 10 A structural block diagram of a bridge bearing defect identification system based on UAV and deep learning network according to an embodiment of this specification is shown. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0067] However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of embodiments of this disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this disclosure.

[0068] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The term "comprising" as used herein indicates the presence of features, steps, or operations, but does not exclude the presence or addition of one or more other features. It should be noted that all terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be understood that the terms used herein should be interpreted in a manner consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0069] Example 1

[0070] like Figure 1 and Figure 2 As shown, this embodiment provides a method for identifying bridge bearing defects based on unmanned aerial vehicles (UAVs) and deep learning networks, including:

[0071] S1: Plan the route for drone inspection missions;

[0072] For example, S1 includes the following specific steps:

[0073] The environmental images of the target bridge are collected by drones and analyzed to obtain the geometric structure and occlusion status of the target bridge.

[0074] The system obtains the drone's flight endurance and plans the drone inspection route based on the target bridge's geometry and obstruction conditions.

[0075] In some embodiments, the drone device supports multiple mounts and can carry a variety of camera devices and lighting devices;

[0076] like Figure 3 As shown, in this embodiment, the drone device used is a DJI RTK350, equipped with a Zenmuse H20T camera, and paired with a GL60plus for supplementary lighting to obtain clearer images of bridge support damage.

[0077] The DJI RTK350 boasts a 55-minute flight time on a single charge and features a dual-battery system, enabling multiple uninterrupted flights. It provides sufficient endurance for shooting on most bridges, and its IP55 protection rating and 7000-meter operating altitude allow it to work in various regions and environments. The GL60 Plus can provide supplemental lighting in low-light conditions, expanding the illuminated area and addressing issues such as shadows and uneven lighting.

[0078] In this embodiment, the drone inspection mission route includes a roving route and a circular route.

[0079] For multi-span simply supported beam bridges, drones can use segmented, circling flight paths. When encountering obstructions such as trees or power lines, the shooting distance can be appropriately increased, the flight speed reduced, and the camera zoomed in stages for shooting. For bridges with fewer spans, drones can use a circular flight path to ensure complete imaging of the supports, such as... Figure 4 As shown, (a) is a loop route and (b) is a circular route.

[0080] Furthermore, different drone inspection routes can be adaptively planned based on the specific requirements of the mission, the performance of the drone, and environmental conditions.

[0081] In this embodiment, the drone conducts a wide-range environmental search around the bridge, avoiding factors that may affect its flight, such as obstructions. When taking pictures, the drone is positioned as parallel to the bridge as possible, maintaining a shooting distance of 3-5 meters, and zooming can be used as appropriate.

[0082] It should be noted that, according to the "Field Specifications for Low-Altitude Digital Aerial Photogrammetry," the forward overlap rate should generally be 60%–80%, and the lateral overlap rate should generally be 15%–60%. During shooting, the drone should be positioned as parallel to the bridge as possible, maintaining a shooting distance of 3–5 meters, with a vertical shooting angle deviation of <±15°. Zooming can be used appropriately to ensure that the overlap rate of a single image is not less than 50%.

[0083] S2: The drone performs inspection tasks and collects images of the supports of the target bridge to form a support image set;

[0084] S3: Preprocess the support image set to obtain an annotated image set;

[0085] For example, S3 includes the following specific steps:

[0086] S301: Divide the support images in the support image dataset into images of equal resolution 640×640;

[0087] Among them, uniformly segmenting the images into 640×640 images of equal resolution can not only expand the original dataset, but also avoid image distortion.

[0088] S302: Label the damaged regions in the segmented image and classify the damaged regions according to the damage type to construct a labeled dataset.

[0089] S4: Construct and train an image augmentation model based on the Real-ESRGAN network, and input the labeled image set into the trained image augmentation model to generate a super-resolution image set;

[0090] For example, S4 includes the following specific steps:

[0091] S401: Build a high-order degradation model and use the high-order degradation model to perform high-order degradation processing on the labeled image set to obtain a degraded image set;

[0092] In this embodiment, since simple downsampling is insufficient to simulate the real-world image degradation process, the classic degradation model is extended to a higher order to simulate more realistic degradation.

[0093] like Figure 5 As shown, in some embodiments, the degradation process in S401 includes a first degradation process and a second degradation process;

[0094] The first and second degradation processes each include the Blur algorithm, selection of default interpolation scaling, random selection of noise distribution and form, and JPEG compression. This embodiment builds a high-order degradation model through consecutive first and second degradation processes.

[0095] In this embodiment, the functional expression of the higher-order degradation model is:

[0096]

[0097] In the formula, x is the degraded image; D n (y) represents the nth degradation process; n is the number of degradations; y is the original image; k is the corresponding convolution kernel; r is the downsampling scale factor; a is noise; JPEG is the compression process.

[0098] S402: Construct an image enhancement model based on the Real-ESRGAN network, and input the degraded image set into the image enhancement model;

[0099] In some embodiments, the image enhancement model includes a generator based on an SR network and a discriminator based on a U-net convolutional neural network with skip connections;

[0100] like Figure 6 As shown, in this embodiment, the SR network consists of 23 RRDB modules and one Upsampling module. The use of multiple RRDB modules can better extract image features, thus having a greater advantage in restoring image details compared to previous super-resolution algorithms, and expanding the super-resolution scale factors of ×2 and ×1.

[0101] The RRDB module contains multiple convolutional and sampling layers; the super-resolution algorithm uses a super-resolution scale factor of ×4.

[0102] In this embodiment, skip connections are mainly reflected in the convolutional layer connections of the U-Net neural network. The first three convolutional layers are connected to the subsequent upsampling modules, and the convolutional output is added to the result of the upsampling module. This skip connection allows the features extracted earlier to be reused and combined with the features extracted by subsequent modules, enhancing the model's expressive power. At the same time, some detailed features may be lost after multiple convolutions, and this method can preserve some early high-resolution features and improve training efficiency.

[0103] In this embodiment, spectral normalization is introduced into the intermediate layer of the discriminator to control the Lipschitz constant of the discriminator and avoid mode collapse and unstable updates during training. Through the application of spectral normalization, the discriminator can more accurately process complex training outputs and generate finer gradient feedback for local textures in the image. This not only enhances the model's ability to capture local details but also improves the overall effect of image super-resolution enhancement.

[0104] S403: Based on the generator, feature extraction is performed on the images in the degraded image set to generate feature maps, and upsampling and nearest interpolation operations are performed on the feature maps to obtain high-resolution images;

[0105] The generator extracts features from the images in the degraded image set through convolutional layers;

[0106] S404: Input the high-resolution image and the real image into the discriminator. The discriminator returns the loss function parameter update generator based on the error between the high-resolution image and the real image.

[0107] S405: The generator and discriminator are trained alternately based on the loss function until the accuracy of the high-resolution image generated by the generator reaches the preset value. Then the training stops and the trained image enhancement model is output.

[0108] S406: Input the labeled image set into the trained image enhancement model to obtain a super-resolution image set.

[0109] like Figures 7-8 As shown, it should be noted that the Real ESRGAN network in this embodiment can perform super-resolution enhancement of images for various domains. Specifically, Real ESRGAN provides multiple weight models for different scenarios, such as realesrgan-x4plus, realesrnet-x4plus, realesrgan-x4plus-anime, and realesr-animevideov3, to adapt to different image processing needs. Among these models, using the realesrgan-x4plus weight model to build the Real ESRGAN network can more conveniently and efficiently restore image clarity while preserving more details.

[0110] S5: Construct a disease identification model based on the YOLOv8 network, and train the disease identification model with a super-resolution image set so that the disease identification model can identify and locate support damage.

[0111] For example, S5 includes the following specific steps:

[0112] S501: Divide the super-resolution image set into a training set, a validation set, and a test set in a ratio of 6:2:2;

[0113] S502: Construct a disease identification model based on YOLOv8 network;

[0114] like Figure 9 As shown, in some embodiments, the disease identification model based on the YOLOv8 network includes an Input module, a Backbone module, and a Head module connected in sequence.

[0115] Specifically, the Backbone module includes a first CBS module, a second CBS module, a first C2f module, a third CBS module, a second C2f module, a fourth CBS module, a third C2f module, a fifth CBS module, and a fourth C2f module connected in sequence; the Head module is composed of a Classify module.

[0116] Among them, the CBS module is a standard convolutional module, which uses a 3×3 small-sized convolutional kernel and the SiLU activation function. The C2F module is composed of the CBS module and multiple Bottleneck modules connected in parallel.

[0117] The activation function of the CBS module is as follows:

[0118]

[0119] In the formula, SiLU is the activation function, and x is the input tensor.

[0120] S503: Input the training set into the disease identification model for iterative training until the training iteration count reaches the preset number, and then output the trained disease identification model.

[0121] In some embodiments, the training process of the disease identification model first involves creating a disease identification model training program and calling the CUDA environment to accelerate the training process using the GPU; the number of training iterations (epochs) is set to 100, and a set of data containing 16 images is processed iteratively.

[0122] Images from the super-resolution image set are used as input data and fed into the Backbone module through the Input module; the images in the super-resolution image set can reveal clearer damage details;

[0123] Image features are extracted using convolutional modules in CBS (Cross-Boundary Segments) and C2F (Convolutional to Fully-connected). The C2F module, by introducing multiple Bottleneck structures and employing a branching design, significantly enhances the nonlinear processing and representation capabilities of the YOLOv8 network during training, thereby improving its ability to handle complex data. Merging different branches further enriches the expressive power of the features. The feature vectors extracted by the Backbone module are then input into the Head module, where they undergo one convolution, pooling, and fully connected operation to output the final classification result. This process not only improves the accuracy of damage identification but also provides crucial technical support for bridge maintenance and repair.

[0124] S504: Validate the trained disease identification model based on the validation set to obtain validation data, and use the validation data to construct a confusion matrix and loss curve to determine whether the training accuracy of the disease identification model has reached the preset accuracy.

[0125] If not, repeat steps S503 and S504; if yes, proceed to step S505.

[0126] S505: Test the validated disease identification model based on the test set to evaluate whether the disease identification model can identify and locate support damage; if yes, complete the training of the disease identification model; if no, return to step S503.

[0127] In this embodiment, the disease identification model after training can identify and classify damage, and can quickly locate damage after damage classification.

[0128] In this embodiment, the training and validation sets were divided into four categories: corrosion, cracking, voids, and deformation. Based on these four categories, the performance of different models was evaluated, and the following statistical data tables were compiled:

[0129] Table 1 Comparison of Performance Indicators of Disease Identification Models

[0130]

[0131]

[0132] In this embodiment, the YOLOv8m-cls model is used for training. It takes only about half the time of the largest model YOLOv8x-cls to process a single image on the GPU, and the model size is moderate, which can balance training speed and accuracy.

[0133] Example 2

[0134] like Figure 10 As shown, this embodiment provides a bridge bearing defect identification system 100 based on UAVs and deep learning networks, including:

[0135] Route planning module 101 is used to plan the route for drone inspection missions;

[0136] The image acquisition module 102 is used by the UAV to perform inspection tasks and acquire images of the bearings of the target bridge to form a bearing image set.

[0137] Image processing module 103 is used to preprocess the support image set to obtain an labeled image set;

[0138] Image augmentation module 104 is used to build and train an image augmentation model based on the Real-ESRGAN network, and inputs the labeled image set into the trained image augmentation model to generate a super-resolution image set;

[0139] Image recognition module 105 is used to construct a disease recognition model based on YOLOv8 network. The disease recognition model is trained by super-resolution image set so that the disease recognition model can identify and locate support damage.

[0140] Example 3

[0141] This embodiment provides a computer device, including at least one control processor and a memory for communicatively connecting to the at least one control processor;

[0142] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0143] The non-transient software program and instructions required to implement the bridge bearing defect identification method based on UAV and deep learning network in the above embodiments are stored in memory. When executed by the processor, the bridge bearing defect identification method based on UAV and deep learning network in the above embodiments is executed, for example, the method described above is executed. Figure 1 The method steps S1 to S5.

[0144] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0145] Example 4

[0146] This embodiment provides a computer-readable storage medium storing computer-executable instructions for causing a computer to execute a bridge bearing defect identification method based on UAV and deep learning network as described in Embodiment 1.

[0147] It should be noted that the computer-readable storage medium in this embodiment can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0148] In this embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this embodiment, the computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0149] The computer-readable storage medium described above can be used to write computer programs for executing this embodiment in one or more programming languages ​​or combinations thereof. These programming languages ​​include object-oriented programming languages ​​such as Python, Java, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0150] In summary, this embodiment significantly enhances image quality and the accuracy of disease identification through an image enhancement model based on the Real-ESRGAN network and a disease identification model based on the YOLOv8 network. The Real-ESRGAN network performs super-resolution enhancement on the image, which significantly improves the accuracy of disease identification, especially in the case of low-resolution or blurry images. Meanwhile, the fast and accurate target detection capability of the YOLOv8 network can quickly identify and locate damage to bridge bearings.

[0151] This embodiment utilizes automated inspection and image acquisition via drones, and analyzes the environmental images acquired by the drones to determine the inspection route. This reduces the workload and time of manual inspections, minimizes interference from environmental factors during image acquisition, and significantly improves the efficiency and accuracy of bridge bearing defect identification. Furthermore, considering the drone's flight endurance, the inspection route is optimized, enabling the drone to complete more inspection tasks with limited battery power, thus improving resource utilization efficiency.

[0152] This embodiment enhances the robustness and training stability of the model by introducing a higher-order degradation model and spectral normalization techniques. The higher-order degradation model processes images to simulate image degradation scenarios that may be encountered in real-world environments, making the model more adaptable to different environmental conditions. The application of spectral normalization techniques in the discriminator effectively controls the Lipschitz constant, avoiding mode collapse and unstable updates, and improving the stability and convergence speed of model training.

[0153] This embodiment ensures comprehensiveness and high accuracy in model training by scientifically dividing the image set and iteratively optimizing the model. The super-resolution image set is scientifically divided into training, validation, and test sets, ensuring the comprehensiveness and effectiveness of model training while also accurately evaluating model performance. Through iterative training and validation, model performance is continuously optimized until the preset accuracy is achieved, ensuring high accuracy and reliability of the model.

[0154] The above description is merely a preferred embodiment of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects. The scope of the present invention is defined by the appended claims rather than the foregoing description, and thus all variations falling within the meaning and scope of the equivalents of the claims are intended to be included within the present invention.

Claims

1. A method for identifying bridge bearing defects based on unmanned aerial vehicles (UAVs) and deep learning networks, characterized in that, include: S1: Plan the route for drone inspection missions; S2: The drone performs inspection tasks and collects images of the supports of the target bridge to form a support image set; S3: Preprocess the support image set to obtain an annotated image set; S4: Construct and train an image augmentation model based on the Real-ESRGAN network, and input the labeled image set into the trained image augmentation model to generate a super-resolution image set; S5: Construct a disease identification model based on the YOLOv8 network, and train the disease identification model with a super-resolution image set so that the disease identification model can identify and locate support damage; S4 includes the following specific steps: S401: Build a high-order degradation model and use the high-order degradation model to perform high-order degradation processing on the labeled image set to obtain a degraded image set; S402: Construct an image enhancement model based on the Real-ESRGAN network, and input the degraded image set into the image enhancement model; The image enhancement model includes a generator based on an SR network and a discriminator based on a U-net convolutional neural network with skip connections; wherein, the discriminator incorporates spectral normalization to control the Lipschitz constant of the discriminator. S403: Based on the generator, feature extraction is performed on the images in the degraded image set to generate feature maps, and upsampling and nearest interpolation operations are performed on the feature maps to obtain high-resolution images; S404: Input the degraded image and the high-resolution image from the degraded image set into the discriminator. Based on the error between the degraded image and the high-resolution image, the discriminator returns the loss function parameter update generator. S405: The generator and discriminator are trained alternately based on the loss function until the accuracy of the high-resolution image generated by the generator reaches the preset value. Then the training stops and the trained image enhancement model is output. S406: Input the labeled image set into the trained image enhancement model to obtain a super-resolution image set.

2. The method according to claim 1, characterized in that, S1 includes the following specific steps: The environmental images of the target bridge are collected by drones and analyzed to obtain the geometric structure and occlusion status of the target bridge. The system obtains the drone's flight endurance and plans the drone inspection route based on the target bridge's geometry and obstruction conditions.

3. The method according to claim 1, characterized in that, S3 includes the following specific steps: S301: Divide the support images in the support image set into images of equal resolution P×P; S302: Label the damaged regions in the segmented image and classify the damaged regions according to the damage type to construct a labeled image set.

4. The method according to claim 1, characterized in that, The higher-order degradation process in S401 includes a first degradation process and a second degradation process; The first and second degradation processes each include, in sequence, the Blur algorithm, selection of default interpolation scaling, random selection of noise distribution and form, and JPEG compression; The functional expression of the higher-order degradation model in S401 is: ； In the formula, x For degraded images; This represents n degeneration processes; where n is the number of degenerations. y Original image; k For the corresponding convolution kernel; r The scaling factor for downsampling; a For noise; JEPG This is the compression process.

5. The method according to claim 1, characterized in that, S5 includes the following specific steps: S501: Divide the super-resolution image set into a training set, a validation set, and a test set according to a preset ratio; S502: Construct a disease identification model based on YOLOv8 network; S503: Input the training set into the disease identification model for iterative training until the training iteration count reaches the preset number, and then output the trained disease identification model. S504: Validate the trained disease identification model based on the validation set to obtain validation data, and use the validation data to construct a confusion matrix and loss curve to determine whether the training accuracy of the disease identification model has reached the preset accuracy. If not, return to step S503; if yes, proceed to step S505. S505: Test the validated disease identification model based on the test set to evaluate whether the disease identification model can identify and locate support damage; if yes, complete the training of the disease identification model; if no, return to step S503.

6. The method according to claim 5, characterized in that, The disease identification model based on the YOLOv8 network includes an Input module, a Backbone module, and a Head module connected in sequence. The Backbone module includes a first CBS module, a second CBS module, a first C2f module, a third CBS module, a second C2f module, a fourth CBS module, a third C2f module, a fifth CBS module, and a fourth C2f module connected in sequence; the Head module is composed of a Classify module. The activation function of the CBS module is as follows: ； In the formula, SiLU is the activation function. x For input tensors.

7. A bridge bearing defect identification system based on unmanned aerial vehicles and deep learning networks, characterized in that, include: The route planning module is used to plan the route for drone inspection missions. The image acquisition module is used by the UAV to perform inspection tasks and acquire images of the bearings of the target bridge to form a bearing image set; An image processing module is used to preprocess the support image set to obtain an annotated image set; The image augmentation module is used to build and train an image augmentation model based on the Real-ESRGAN network. The labeled image set is input into the trained image augmentation model to generate a super-resolution image set. The image recognition module is used to construct a disease recognition model based on the YOLOv8 network. The disease recognition model is trained through a super-resolution image set so that the disease recognition model can identify and locate support damage. The image enhancement module is configured to perform the following steps: A high-order degradation model is constructed, and the labeled image set is subjected to high-order degradation processing using the high-order degradation model to obtain a degraded image set; An image enhancement model based on the Real-ESRGAN network is constructed, and the degraded image set is input into the image enhancement model; The image enhancement model includes a generator based on an SR network and a discriminator based on a U-net convolutional neural network with skip connections; wherein, the discriminator incorporates spectral normalization to control the Lipschitz constant of the discriminator. Based on the generator, feature extraction is performed on the images in the degraded image set to generate feature maps, and upsampling and nearest interpolation operations are performed on the feature maps to obtain high-resolution images; The degraded images and high-resolution images in the degraded image set are input into the discriminator. Based on the error between the degraded images and the high-resolution images, the discriminator returns a loss function parameter update generator. The generator and discriminator are trained alternately based on the loss function until the accuracy of the high-resolution image generated by the generator reaches the preset value. Then the training stops and the trained image enhancement model is output. The labeled image set is input into the trained image augmentation model to obtain a super-resolution image set.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the steps of the bridge bearing defect identification method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the bridge bearing defect identification method according to any one of claims 1-6.

Images