Bridge support disease identification method and system based on unmanned aerial vehicle and deep learning network

Abstract

The invention discloses a bridge support disease identification method and system based on an unmanned aerial vehicle and a deep learning network. The method comprises the following steps: planning an unmanned aerial vehicle inspection task route; the unmanned aerial vehicle executes an inspection task and collects a support image of the target bridge to form a support image set; preprocessing the support image set to obtain a labeled image set; constructing and training an image enhancement model based on a Real-ESRGAN network, and inputting the annotated image set into the trained image enhancement model to generate a super-resolution image set; and constructing a disease identification model based on a YOLOv8 network, and training the disease identification model through the super-division image set to identify and position support damage. According to the invention, the image is subjected to super-resolution enhancement through the Real-ESRGAN network, the image quality and the accuracy of disease identification are greatly enhanced, and the damage of the bridge support can be rapidly identified and positioned through the rapid and accurate target detection capability of the YOLOv8 network.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a bridge bearing defect recognition method and system based on an unmanned aerial vehicle and a deep learning network. Background Art

[0002] The health of bridge bearings is directly related to the overall safety and stability of the bridge. Under the influence of multiple factors such as the deadweight of the beam, environmental changes, and vehicle loads, the bearings are prone to damage such as rust, cracking, and voiding, which seriously threaten the safety of the bridge structure.

[0003] Traditional manual inspection methods for bridge bearings have shortcomings such as high risk, strong subjectivity, time-consuming and labor-intensive, and easy to miss detection. In particular, the bearings are located high and hidden, making inspection more difficult and risky. In recent years, drone technology has been widely used in the field of engineering inspection due to its mobility and large-scale inspection capabilities, and has become an important means of collecting bridge disease data. However, images taken by drones are easily disturbed by flight vibrations, gusts, shadows, etc., resulting in low image resolution and blur, which affects the recognition accuracy of the classification detection network. In order to improve the utilization rate of image information and meet the needs of bridge bearing disease detection, it is necessary to improve the initial data set obtained by drones; with the development of deep learning technology, especially the maturity of super-resolution algorithms, it provides technical support for improving image quality; at the same time, the large data set processing capability of classification neural network models has further promoted the application of a large number of classification detection models in bearing damage detection.

[0004] However, the types of bearing damage are diverse and difficult to observe. The existing single classification detection network model is difficult to meet actual engineering needs. In addition, the deep learning model has high requirements on the quality of the data set. Low-quality data sets will significantly reduce the accuracy of damage identification.

[0005] In view of this, the present application proposes a bridge bearing defect identification method and system based on drones and deep learning network models to solve the above problems. Summary of the invention

[0006] In view of the above-mentioned problems in the prior art, the purpose of the present invention is to provide a method and system for identifying bridge bearing defects based on drones and deep learning network models. The bridge bearing images are collected by drones, the clarity of the collected images is optimized based on the Real-ESRGAN network model, and the damage at various positions of the bridge is classified, identified and located using the YOLOv8 network model.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] In a first aspect, an embodiment of the present invention provides a bridge bearing disease identification method based on a drone and a deep learning network, comprising:

[0009] S1: Plan the UAV inspection mission route;

[0010] S2: The UAV performs the inspection task and collects the support images of the target bridge to form a support image set;

[0011] S3: preprocessing the support image set to obtain a labeled image set;

[0012] S4: Build and train an image enhancement model based on the Real-ESRGAN network, and input the annotated image set into the trained image enhancement model to generate a super-resolved image set;

[0013] S5: construct a disease recognition model based on the YOLOv8 network, and train the disease recognition model through a super-resolution image set so that the disease recognition model can recognize and locate bearing damage.

[0014] Optionally, S1 includes the following specific steps:

[0015] Collecting an environmental image of a target bridge based on a drone, and analyzing the environmental image to obtain a geometric structure and occlusion conditions of the target bridge;

[0016] Obtain the flight endurance of the UAV and plan the UAV inspection mission route based on the geometric structure and occlusion of the target bridge.

[0017] Optionally, S3 includes the following specific steps:

[0018] S301: uniformly dividing the support images in the support image set into images with the same resolution of P×P;

[0019] S302: annotating the damaged areas of the segmented image and classifying the damaged areas according to the damage types to construct an annotated image set.

[0020] Optionally, S4 includes the following specific steps:

[0021] S401: constructing a high-order degradation model, and performing high-order degradation processing on the annotated image set using the high-order degradation model to obtain a degraded image set;

[0022] S402: constructing an image enhancement model based on the Real-ESRGAN network, and inputting the degraded image set into the image enhancement model;

[0023] The image enhancement model includes a generator constructed based on an SR network and a discriminator constructed based on a U-net convolutional neural network with jump connections; wherein a spectral normalization technique is introduced into the discriminator to control the Lipschitz constant of the discriminator;

[0024] S403: extracting features from the images in the degraded image set based on the generator, generating a feature map, and performing upsampling and nearest interpolation operations on the feature map to obtain a high-resolution image;

[0025] S404: inputting the degraded image and the high-resolution image in the degraded image set into the discriminator, and the discriminator returns a loss function parameter update generator based on the error between the degraded image and the high-resolution image;

[0026] S405: The generator and the discriminator are alternately trained based on the loss function until the accuracy of the high-resolution image generated by the generator reaches a preset value, and the training is stopped, and the trained image enhancement model is output.

[0027] S406: Input the annotated image set into the trained image enhancement model to obtain a super-resolved image set.

[0028] Optionally, the high-order degradation processing in S401 includes a first degradation processing and a second degradation processing;

[0029] The first degradation process and the second degradation process both include Blur algorithm, selected default interpolation scaling, random selection of noise distribution and form, and JPEG compression in sequence;

[0030] The functional expression of the high-order degradation model in S401 is:

[0031]

[0032] Where x is the degraded image; D n (y) is the n-times degradation process; n is the number of degradations; y is the original image; k is the corresponding convolution kernel; r is the downsampling scale factor; a is the noise; JPEG is the compression process.

[0033] Optionally, S5 includes the following specific steps:

[0034] S501: Dividing the super-resolved image set into a training set, a validation set and a test set in sequence according to a preset ratio;

[0035] S502: Build a disease recognition model based on the YOLOv8 network;

[0036] S503: inputting the training set into the disease recognition model for iterative training until the number of training iterations reaches a preset number, and outputting the trained disease recognition model;

[0037] S504: verifying the trained disease recognition model based on the verification set to obtain verification data, and using the verification data to construct a confusion matrix and a loss curve to determine whether the training accuracy of the disease recognition model reaches a preset accuracy;

[0038] If not, return to step S503; if yes, execute step S505;

[0039] S505: Testing the verified damage recognition model based on the test set to evaluate whether the damage recognition model can identify and locate bearing damage; if so, completing the training of the damage recognition model; if not, returning to step S503.

[0040] Optionally, the disease recognition model based on the YOLOv8 neural network includes an Input module, a Backbone module and a Head module connected in sequence;

[0041] The Backbone module includes a first CBS module, a second CBS module, a first C2f module, a third CBS module, a second C2f module, a fourth CBS module, a third C2f module, a fifth CBS module and a fourth C2f module connected in sequence; the Head module is composed of a Classify module;

[0042] The activation function of the CBS module is as follows:

[0043]

[0044] Where SiLU is the activation function and x is the input tensor.

[0045] In a second aspect, an embodiment of the present invention provides a bridge bearing defect identification system based on a drone and a deep learning network, comprising:

[0046] Route planning module, used to plan the route of drone inspection missions;

[0047] An image acquisition module is used for the UAV to perform inspection tasks and to collect images of the support of the target bridge to form a support image set;

[0048] An image processing module, used for preprocessing the support image set to obtain a labeled image set;

[0049] The image enhancement module is used to build and train an image enhancement model based on the Real-ESRGAN network, and input the annotated image set into the trained image enhancement model to generate a super-resolution image set;

[0050] The image recognition module is used to build a disease recognition model based on the YOLOv8 network, and train the disease recognition model through a super-resolution image set so that the disease recognition model can recognize and locate bearing damage.

[0051] In a third aspect, an embodiment of the present invention provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the bridge bearing defect identification method as described above.

[0052] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the bridge bearing defect identification method as described above.

[0053] The beneficial effects brought by the embodiments provided by the present invention include:

[0054] The present invention greatly enhances the image quality and the accuracy of defect recognition through an image enhancement model based on the Real-ESRGAN network and a defect recognition model based on the YOLOv8 network. The Real-ESRGAN network performs super-resolution enhancement on images, which significantly improves the accuracy of defect recognition, especially in the case of low-resolution or blurred images. The fast and accurate target detection capability of the YOLOv8 network can quickly identify and locate damage to bridge bearings.

[0055] The present invention uses automated inspections and image acquisition by drones, and analyzes environmental images collected by drones to determine inspection routes, thereby reducing the workload and time of manual inspections, reducing the interference of environmental factors during image acquisition, and significantly improving the efficiency and accuracy of bridge bearing disease identification. At the same time, the flight endurance of the drone is taken into consideration, and the inspection route is optimized, so that the drone can complete more inspection tasks with limited power, thereby improving resource utilization efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic diagram of a flow chart of a bridge bearing disease identification method based on a drone and a deep learning network according to an embodiment of this specification is shown;

[0057] Figure 2 A flowchart of another bridge bearing disease identification method based on a drone and a deep learning network according to an embodiment of this specification is shown;

[0058] Figure 3 A schematic diagram of the structure of a drone device according to an embodiment of this specification is shown;

[0059] Figure 4A schematic diagram of a drone inspection task route according to an embodiment of this specification is shown;

[0060] Figure 5 A schematic diagram of the architecture of a high-order degradation model according to an embodiment of this specification is shown;

[0061] Figure 6 A schematic diagram of a generator architecture of a Real-ESRGAN network according to an embodiment of this specification is shown;

[0062] Figure 7 An example diagram of bridge bearing training according to an embodiment of this specification is shown;

[0063] Figure 8 shows a super-resolution image after image enhancement of a training example image according to an embodiment of this specification;

[0064] Fig. 9 A schematic diagram of the architecture of a disease identification model according to an embodiment of this specification is shown;

[0065] Fig.10 A structural block diagram of a bridge bearing defect identification system based on a drone and a deep learning network according to an embodiment of this specification is shown. DETAILED DESCRIPTION

[0066] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0067] However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, the description of the known technology is omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0068] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The term "comprising" used herein indicates the existence of features, steps, operations, but does not exclude the existence or addition of one or more other features. It should be noted that all terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used here should be interpreted as having meanings consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0069] Example 1

[0070] like Figure 1 and Figure 2 As shown, this embodiment provides a bridge bearing disease identification method based on a drone and a deep learning network, including:

[0071] S1: Plan the UAV inspection mission route;

[0072] Exemplarily, S1 includes the following specific steps:

[0073] Collecting an environmental image of a target bridge based on a drone, and analyzing the environmental image to obtain a geometric structure and occlusion conditions of the target bridge;

[0074] Obtain the flight endurance of the UAV and plan the UAV inspection mission route based on the geometric structure and occlusion of the target bridge.

[0075] In some embodiments, the drone device supports multiple mounts and can carry multiple camera devices and lighting devices;

[0076] like Figure 3 As shown, in this embodiment, the drone equipment uses DJI RTK350, equipped with a Zenmuse H20T camera, and is equipped with GL60plus for fill light to obtain clearer images of bridge bearing damage;

[0077] Among them, DJI RTK350 has a single flight endurance of 55 minutes and a dual-battery system, which can achieve multiple uninterrupted flights. It can have enough shooting endurance for most bridges, and the IP55 protection level and 7000 meters working altitude support it to work in different regions and environments; GL60plus can fill in the light in weak ambient light, expand the lighting area, and solve the problems of shadow occlusion, uneven lighting, etc.

[0078] In this embodiment, the drone inspection mission route includes a patrol route and a circular route.

[0079] Among them, for multi-span simply supported beam bridges, drones can use roving routes in sections. When faced with obstructions such as trees and power lines, the shooting distance can be appropriately increased, the flight speed can be reduced, and the shooting can be performed in batches with zooming. For bridges with fewer spans, drones can use circling routes to ensure the integrity of the support shots, such as Figure 4 As shown, (a) is a circuit type route, and (b) is a circular type route.

[0080] Furthermore, different drone inspection mission routes can be adaptively planned according to the specific requirements of the mission, the performance of the drone, and environmental conditions.

[0081] In this embodiment, the drone conducts a large-scale environmental search around the bridge, trying to avoid obstructions and other factors that affect the flight of the drone. When shooting, the drone is kept as parallel to the bridge as possible, maintaining a shooting distance of 3-5 meters, and zooming can be performed appropriately.

[0082] It should be noted that according to the "Field Specifications for Low-Altitude Digital Aerial Photogrammetry", the heading overlap rate should generally be 60% to 80%; the lateral overlap rate should generally be 15% to 60%. When shooting, the drone should be kept as parallel to the bridge as possible, maintaining a shooting distance of 3-5 meters, with a vertical shooting angle deviation of <±15°, and appropriate zooming can be performed to ensure that the single-frame overlap rate is not less than 50%.

[0083] S2: The UAV performs the inspection task and collects the support images of the target bridge to form a support image set;

[0084] S3: preprocessing the support image set to obtain a labeled image set;

[0085] Exemplarily, S3 includes the following specific steps:

[0086] S301: uniformly segmenting the support images in the support image data set into images with the same resolution of 640×640;

[0087] Among them, the unified segmentation into images with the same resolution of 640×640 can not only expand the original data set but also avoid image distortion;

[0088] S302: Annotate the damaged areas of the segmented image and classify the damaged areas according to the damage types to construct an annotation data set.

[0089] S4: Build and train an image enhancement model based on the Real-ESRGAN network, and input the annotated image set into the trained image enhancement model to generate a super-resolved image set;

[0090] Exemplarily, S4 includes the following specific steps:

[0091] S401: constructing a high-order degradation model, and performing high-order degradation processing on the annotated image set using the high-order degradation model to obtain a degraded image set;

[0092] In this embodiment, since simple downsampling is not enough to simulate the image degradation process in reality, the classic degradation model is extended to a higher order to simulate more realistic degradation;

[0093] like Figure 5 As shown, in some embodiments, the degradation process in S401 includes a first degradation process and a second degradation process;

[0094] The first degradation process and the second degradation process each include Blur algorithm, selected default interpolation scaling, random selection of noise distribution and form, and JPEG compression in sequence; this embodiment builds a high-order degradation model through continuous first degradation process and second degradation process;

[0095] In this embodiment, the function expression of the high-order degradation model is:

[0096]

[0097] Where x is the degraded image; D n (y) is the n-times degradation process; n is the number of degradations; y is the original image; k is the corresponding convolution kernel; r is the downsampling scale factor; a is the noise; JPEG is the compression process.

[0098] S402: constructing an image enhancement model based on the Real-ESRGAN network, and inputting the degraded image set into the image enhancement model;

[0099] In some embodiments, the image enhancement model includes a generator constructed based on an SR network and a discriminator constructed based on a U-net convolutional neural network with skip connections;

[0100] like Figure 6 As shown, in this embodiment, the SR network consists of 23 RRDB modules and one Upsampling module. The use of multiple RRDB modules can better extract image features, thereby having a greater advantage in restoring image details than the previous super-resolution algorithm, and expanding the super-resolution scaling factors of ×2 and ×1;

[0101] Among them, the RRDB module contains multiple convolution and sampling layers; the super-resolution algorithm uses a super-resolution scaling factor of ×4;

[0102] In this embodiment, the jump connection is mainly reflected in the convolution layer connection of the U-Net neural network. The first three convolution layers are connected to the subsequent upsampling modules respectively, and the convolution output is added to the result of the upsampling module. This leapfrog connection can reuse the features extracted earlier and combine them with the features extracted by the subsequent modules to enhance the expression ability of the model; at the same time, some detailed features may be lost after multiple convolutions. In this way, some early high-resolution features can be retained and the training efficiency can be improved.

[0103] In this embodiment, the spectral normalization technology is introduced into the middle layer of the discriminator to control the Lipschitz constant of the discriminator and avoid mode collapse and unstable updates during the training process; through the application of spectral normalization, the discriminator can process complex training outputs more accurately and generate more refined gradient feedback for the local texture of the image. This not only enhances the model's ability to capture local details, but also improves the overall effect of image super-resolution enhancement.

[0104] S403: extracting features from the images in the degraded image set based on the generator, generating a feature map, and performing upsampling and nearest interpolation operations on the feature map to obtain a high-resolution image;

[0105] The generator extracts features from the images in the degraded image set through a convolutional layer;

[0106] S404: Input the high-resolution image and the real image into the discriminator, and the discriminator returns the loss function parameters to update the generator based on the error between the high-resolution image and the real image;

[0107] S405: The generator and the discriminator are alternately trained based on the loss function until the accuracy of the high-resolution image generated by the generator reaches a preset value, and the training is stopped, and the trained image enhancement model is output.

[0108] S406: Input the annotated image set into the trained image enhancement model to obtain a super-resolved image set.

[0109] like Figure 7-Figure 8 As shown, it should be noted that the Real ESRGAN network in this embodiment can perform super-resolution enhancement of images for a variety of fields. Specific to different scenarios, Real ESRGAN provides a variety of weight models, such as realesrgan-x4plus, realesrnet-x4plus, realesrgan-x4plus-anime, realesr-animevideov3, etc., to meet different image processing requirements. Among these models, using the realesrgan-x4plus weight model to build a Real ESRGAN network can restore image clarity more conveniently and efficiently while retaining more details.

[0110] S5: construct a disease recognition model based on the YOLOv8 network, and train the disease recognition model through a super-resolution image set so that the disease recognition model can recognize and locate bearing damage.

[0111] Exemplarily, S5 includes the following specific steps:

[0112] S501: Divide the super-resolved image set into a training set, a validation set and a test set in a ratio of 6:2:2;

[0113] S502: Build a disease recognition model based on the YOLOv8 network;

[0114] like Fig. 9 As shown, in some embodiments, the disease recognition model based on the YOLOv8 network includes an Input module, a Backbone module, and a Head module connected in sequence;

[0115] Specifically, the Backbone module includes a first CBS module, a second CBS module, a first C2f module, a third CBS module, a second C2f module, a fourth CBS module, a third C2f module, a fifth CBS module and a fourth C2f module connected in sequence; the Head module is composed of a Classify module;

[0116] Among them, the CBS module is a standard convolution module, which uses a 3×3 small-size convolution kernel and SiLU activation function. The C2F module is composed of a CBS module and multiple Bottleneck modules in parallel.

[0117] Among them, the activation function of the CBS module is as follows:

[0118]

[0119] Where SiLU is the activation function and x is the input tensor.

[0120] S503: inputting the training set into the disease recognition model for iterative training until the number of training iterations reaches a preset number, and outputting the trained disease recognition model;

[0121] In some embodiments, the training process of the disease recognition model first creates a disease recognition model training program, calls the CUDA environment, and uses the GPU to accelerate the training process; the number of training iterations (epochs) is set to 100, and iteratively processes a set of data containing 16 images.

[0122] The images in the super-resolution image set are used as input data and input into the Backbone module through the Input module; the images in the super-resolution image set can show clearer damage details;

[0123] Image features are extracted through the convolution modules in CBS (Cross-Boundary Segments) and C2F (Convolutional to Fully-connected). Since the C2F module introduces multiple Bottleneck structures and adopts a branch structure design, the nonlinear processing and representation capabilities of the YOLOv8 network are significantly enhanced during the training process, thereby improving the YOLOv8 network's ability to handle complex data. The merging of different branches further enriches the expressive power of the features. The feature vector extracted by the Backbone module is then input into the Head module, and after a convolution, pooling, and full connection operation, the final classification result is output. This process not only improves the accuracy of damage identification, but also provides important technical support for bridge maintenance and repair work.

[0124] S504: verifying the trained disease recognition model based on the verification set to obtain verification data, and using the verification data to construct a confusion matrix and a loss curve to determine whether the training accuracy of the disease recognition model reaches a preset accuracy;

[0125] If not, repeat steps S503 and S504; if yes, execute step S505;

[0126] S505: Testing the verified damage recognition model based on the test set to evaluate whether the damage recognition model can identify and locate bearing damage; if so, completing the training of the damage recognition model; if not, returning to step S503.

[0127] In this embodiment, the disease recognition model after training is able to identify and classify damages, and can quickly locate the damages after the damages are classified.

[0128] In this embodiment, the training set and the validation set are divided into four situations: corrosion, cracking, voiding, and deformation. Based on the above four situations, the performance of different models is evaluated, and the following statistical data tables are compiled:

[0129] Table 1 Comparison of performance indicators of disease identification models

[0130]

[0131]

[0132] In this embodiment, the YOLOv8m-cls model is used for training. The time required to process a single image on the GPU is only about half of that of the largest model YOLOv8x-cls, and the model size is moderate, which can take into account both training speed and accuracy.

[0133] Example 2

[0134] like Fig.10 As shown, this embodiment provides a bridge bearing defect identification system 100 based on a drone and a deep learning network, comprising:

[0135] Route planning module 101, used to plan the route of the drone inspection mission;

[0136] An image acquisition module 102 is used for the drone to perform inspection tasks and to acquire support images of the target bridge to form a support image set;

[0137] An image processing module 103 is used to pre-process the support image set to obtain a labeled image set;

[0138] An image enhancement module 104 is used to construct and train an image enhancement model based on a Real-ESRGAN network, and input the annotated image set into the trained image enhancement model to generate a super-resolved image set;

[0139] The image recognition module 105 is used to construct a disease recognition model based on the YOLOv8 network, and train the disease recognition model through the super-resolution image set so that the disease recognition model can recognize and locate the bearing damage.

[0140] Example 3

[0141] This embodiment provides a computer device, comprising at least one control processor and a memory for communication connection with the at least one control processor;

[0142] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0143] The non-transient software program and instructions required to implement the above-mentioned method for identifying bridge bearing defects based on drones and deep learning networks are stored in the memory. When executed by the processor, the above-mentioned method for identifying bridge bearing defects based on drones and deep learning networks is executed. For example, the above-mentioned method is executed. Figure 1 The method comprises steps S1 to S5.

[0144] The system embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0145] Example 4

[0146] This embodiment provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a bridge bearing defect identification method based on a drone and a deep learning network as described in Example 1.

[0147] It should be noted that the computer-readable storage medium of the present embodiment may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0148] In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device, or device. In this embodiment, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program used by or in combination with an instruction execution system, device, or device. The computer program contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0149] The computer readable storage medium can be written in one or more programming languages ​​or a combination thereof to execute the computer program of the present embodiment, and the programming language includes an object-oriented programming language, such as Python, Java, C++, and also includes a conventional procedural programming language, such as C language or similar programming languages. The program can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0150] In summary, this embodiment greatly enhances the image quality and the accuracy of defect recognition through the image enhancement model based on the Real-ESRGAN network and the defect recognition model based on the YOLOv8 network. The Real-ESRGAN network performs super-resolution enhancement on the image, which significantly improves the accuracy of defect recognition, especially in the case of low-resolution or blurred images. The fast and accurate target detection capability of the YOLOv8 network can quickly identify and locate damage to bridge bearings.

[0151] This embodiment uses drones to automate inspections and image acquisition, and analyzes environmental images collected by drones to determine inspection routes, thereby reducing the workload and time of manual inspections, reducing the interference of environmental factors during image acquisition, and significantly improving the efficiency and accuracy of bridge bearing disease identification. At the same time, the flight endurance of the drone is taken into consideration, and the inspection route is optimized, so that the drone can complete more inspection tasks with limited power, thereby improving resource utilization efficiency.

[0152] This embodiment enhances the robustness and training stability of the model by introducing a high-order degradation model and spectral normalization technology. The high-order degradation model processes images and simulates the image degradation that may be encountered in the actual environment, making the model more adaptable to different environmental conditions. The application of spectral normalization technology in the discriminator effectively controls the Lipschitz constant, avoids mode collapse and unstable updates, and improves the stability and convergence speed of model training.

[0153] This embodiment ensures the comprehensiveness and high accuracy of model training by scientifically dividing the image set and iteratively optimizing the model. The super-resolution image set is scientifically divided into a training set, a validation set, and a test set, which ensures the comprehensiveness and effectiveness of model training and can also accurately evaluate the performance of the model. Through iterative training and verification, the model performance is continuously optimized until the preset accuracy is reached, ensuring the high accuracy and reliability of the model.

[0154] The above description is only a preferred embodiment of the present invention. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention.

Claims

1. A bridge bearing disease identification method based on drone and deep learning network, characterized in that: include: S1: Plan the UAV inspection mission route; S2: The UAV performs the inspection task and collects the support images of the target bridge to form a support image set; S3: preprocessing the support image set to obtain a labeled image set; S4: Build and train an image enhancement model based on the Real-ESRGAN network, and input the annotated image set into the trained image enhancement model to generate a super-resolved image set; S5: construct a disease recognition model based on the YOLOv8 network, and train the disease recognition model through a super-resolution image set so that the disease recognition model can recognize and locate bearing damage.

2. The method according to claim 1, characterized in that The S1 comprises the following specific steps: Collecting an environmental image of a target bridge based on a drone, and analyzing the environmental image to obtain a geometric structure and occlusion conditions of the target bridge; Obtain the flight endurance of the UAV and plan the UAV inspection mission route based on the geometric structure and occlusion of the target bridge.

3. The method according to claim 1, characterized in that The S3 includes the following specific steps: S301: uniformly dividing the support images in the support image set into images with the same resolution of P×P; S302: annotating the damaged areas of the segmented image and classifying the damaged areas according to the damage types to construct an annotated image set.

4. The method according to claim 1, characterized in that: The S4 comprises the following specific steps: S401: constructing a high-order degradation model, and performing high-order degradation processing on the annotated image set using the high-order degradation model to obtain a degraded image set; S402: constructing an image enhancement model based on the Real-ESRGAN network, and inputting the degraded image set into the image enhancement model; The image enhancement model includes a generator constructed based on an SR network and a discriminator constructed based on a U-net convolutional neural network with jump connections; wherein a spectral normalization technique is introduced into the discriminator to control the Lipschitz constant of the discriminator; S403: extracting features from the images in the degraded image set based on the generator, generating a feature map, and performing upsampling and nearest interpolation operations on the feature map to obtain a high-resolution image; S404: inputting the degraded image and the high-resolution image in the degraded image set into the discriminator, and the discriminator returns a loss function parameter update generator based on the error between the degraded image and the high-resolution image; S405: Alternately train the generator and the discriminator based on the loss function until the accuracy of the high-resolution image generated by the generator reaches a preset value, and stop the training, and output the trained image enhancement model; S406: Input the annotated image set into the trained image enhancement model to obtain a super-resolved image set.

5. The method according to claim 4, characterized in that The high-order degradation processing in S401 includes a first degradation processing and a second degradation processing; The first degradation process and the second degradation process both include Blur algorithm, selected default interpolation scaling, random selection of noise distribution and form, and JPEG compression in sequence; The functional expression of the high-order degradation model in S401 is: Where x is the degraded image; D n (y) is the n-times degradation process; n is the number of degradations; y is the original image; k is the corresponding convolution kernel; r is the downsampling scale factor; a is the noise; JPEG is the compression process.

6. The method according to claim 1, characterized in that: The S5 comprises the following specific steps: S501: Dividing the super-resolved image set into a training set, a validation set and a test set in sequence according to a preset ratio; S502: Build a disease recognition model based on the YOLOv8 network; S503: inputting the training set into the disease recognition model for iterative training until the number of training iterations reaches a preset number, and outputting the trained disease recognition model; S504: verifying the trained disease recognition model based on the verification set to obtain verification data, and using the verification data to construct a confusion matrix and a loss curve to determine whether the training accuracy of the disease recognition model reaches a preset accuracy; If not, return to step S503; if yes, execute step S505; S505: Testing the verified damage recognition model based on the test set to evaluate whether the damage recognition model can identify and locate bearing damage; if so, completing the training of the damage recognition model; if not, returning to step S503.

7. The method according to claim 6, characterized in that The disease recognition model based on the YOLOv8 neural network includes an Input module, a Backbone module and a Head module connected in sequence; The Backbone module includes a first CBS module, a second CBS module, a first C2f module, a third CBS module, a second C2f module, a fourth CBS module, a third C2f module, a fifth CBS module and a fourth C2f module connected in sequence; the Head module is composed of a Classify module; The activation function of the CBS module is as follows: Where SiLU is the activation function and x is the input tensor.

8. A bridge bearing disease identification system based on drones and deep learning networks, characterized in that: include: Route planning module, used to plan the route of drone inspection missions; An image acquisition module is used for the UAV to perform inspection tasks and to collect images of the support of the target bridge to form a support image set; An image processing module, used for preprocessing the support image set to obtain a labeled image set; The image enhancement module is used to build and train an image enhancement model based on the Real-ESRGAN network, and input the annotated image set into the trained image enhancement model to generate a super-resolution image set; The image recognition module is used to build a disease recognition model based on the YOLOv8 network, and train the disease recognition model through a super-resolution image set so that the disease recognition model can recognize and locate bearing damage.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the bridge bearing defect identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the bridge bearing defect identification method described in any one of claims 1-7 are implemented.

Images