A bridge crack disease identification method and device

Abstract

The present application discloses a method and device for identifying crack defects on bridges. The method comprises: obtaining crack images on the surface of bridges, and preprocessing the images to obtain annotated crack data sets; replacing the spatial pyramid pooling module in the YOLOv5 model with a focus modulation module, replacing the C3 module in the YOLOv5 model with a dual residual module, and introducing the SimAM attention mechanism in the YOLOv5 model to construct a recognition model; training the recognition model with the annotated crack data set until convergence to obtain a crack defect recognition model; obtaining the surface image of the bridge with a drone, and inputting the image into the crack defect recognition model to identify crack defects on the bridge. The method effectively solves the problem that the existing YOLOv5 model has low recognition accuracy when processing scenes such as small size, low contrast and complex background. Therefore, the crack defects on bridges can be identified more timely and accurately, which improves the efficiency and accuracy of bridge defect detection and avoids economic losses caused by crack defects on bridges.

Description

Technical Field

[0001] The present application relates to the technical field of crack disease detection, and in particular to a method and device for identifying bridge crack diseases. Background Art

[0002] As a critical infrastructure, bridges are often affected by natural or human factors, especially structural defects such as cracks, which seriously affect their safety and reliability. Timely and effective detection and identification of these cracks is crucial to ensure the safety of bridge structures. Traditional detection methods mainly rely on manual inspections or monitoring systems based on fixed sensors, which have problems such as low efficiency, limited coverage, and subjective detection results, making it difficult to meet actual engineering needs.

[0003] In recent years, drone technology has been widely used in bridge inspections. It can cover a wide area and obtain high-resolution image data at a very close distance. Then, deep learning technology is used to identify crack disasters from the image data collected by drones. In particular, the YOLOv5 (You Only Look Once version 5) model has fast and efficient real-time target detection capabilities and low computational costs, and has become one of the preferred models for image recognition tasks in industry and academia. The YOLOv5 model uses a single-stage detection method to divide the input image into grids through a convolutional neural network, and each grid is responsible for predicting possible targets and their bounding boxes. Compared with the traditional two-stage detection model, the YOLOv5 model has faster detection speed and better real-time performance, but it may cause the loss of target features during maximum pooling. In addition, the YOLOv5 model still faces the problems of low recognition accuracy and poor stability when dealing with scenes such as small size, low contrast and complex background. Summary of the invention

[0004] The embodiments of the present application provide a method and device for identifying bridge crack defects, thereby solving the problem of low recognition accuracy of the existing YOLOv5 model when processing scenes with small size, low contrast and complex background.

[0005] In a first aspect, an embodiment of the present application provides a method for identifying bridge crack defects, including: obtaining a crack image of a bridge surface, and preprocessing the image to obtain a labeled crack data set; replacing the spatial pyramid pooling module in the YOLOv5 model with a focus modulation module, replacing the C3 module in the YOLOv5 model with a dual residual module, and introducing a SimAM attention mechanism in the YOLOv5 model to construct a recognition model; training the recognition model with the labeled crack data set until convergence to obtain a crack defect recognition model; obtaining a bridge surface image with a drone, and inputting the image into the crack defect recognition model to identify crack defects on the bridge.

[0006] In combination with the first aspect, in a possible implementation method, the preprocessing is performed to obtain a labeled crack dataset, including: performing graphic enhancement on the crack image to obtain an enhanced crack image; determining a label type, and using the label type to draw a label area on the enhanced crack image; setting a label category, and adding a label category corresponding to the label category in the label area to obtain a labeled crack image; wherein the label category includes crack position and crack morphology; performing data enhancement processing on all the labeled crack images to obtain the labeled crack dataset.

[0007] In combination with the first aspect, in a possible implementation manner, the performing data enhancement processing on all the annotated crack images includes: horizontally flipping and / or vertically flipping and / or randomly adjusting brightness and / or saturation and / or contrast of the annotated crack images.

[0008] In combination with the first aspect, in a possible implementation method, the spatial pyramid pooling module in the YOLOv5 model is replaced by a focus modulation module, the C3 module in the YOLOv5 model is replaced by a dual residual module, and the SimAM attention mechanism is introduced into the YOLOv5 model, including: locating and removing the spatial pyramid pooling module in the YOLOv5 model; inserting the focus modulation module into the YOLOv5 model, adjusting the parameters of the YOLOv5 model according to the characteristics of the focus modulation module, and updating the configuration file; locating and removing the C3 module in the neck part of the YOLOv5 model; inserting the dual residual module into the YOLOv5 model, adjusting the parameters of the YOLOv5 model according to the characteristics of the dual residual module, and updating the configuration file; locating its detection head in the YOLOv5 model, and adding a SimAM module at the end of each detection head.

[0009] In combination with the first aspect, in a possible implementation method, the use of a drone to obtain a bridge surface image includes: the drone flies according to a planned target route to collect the bridge surface image; denoising and geometrically correcting the bridge surface image; and feature extracting the processed bridge surface image to remove the bridge surface image that does not contain cracks.

[0010] In combination with the first aspect, in a possible implementation manner, the UAV flies according to a planned target route, including:

[0011] ;

[0012] in, ,and ;

[0013] In the formula, is the position coordinate of the drone at time i, represents the control coefficient at time i, , They respectively represent the maximum and minimum distances of the UAV from the bridge in the X-axis direction during the execution of the target route. , They respectively represent the maximum and minimum distances of the UAV from the bridge in the Y-axis direction during the execution of the target route. , They respectively represent the maximum and minimum distances of the UAV from the bridge in the Z-axis direction during the execution of the target route. represents the environmental impact at time i, and N represents the total time required for the UAV to execute the target route.

[0014] In combination with the first aspect, in a possible implementation, the position coordinates at time i are the relative position of the drone to the starting position of its target route at time i.

[0015] In the second aspect, an embodiment of the present application provides a bridge crack disease identification device, including: a labeling module, used to obtain a crack image set on the bridge surface, and preprocess it to obtain a labeled crack data set; a construction module, used to replace the spatial pyramid pooling module in the YOLOv5 model with a focus modulation module, replace the C3 module in the YOLOv5 model with a dual residual module, and introduce the SimAM attention mechanism in the YOLOv5 model to construct a recognition model; a training module, used to train the recognition model with the labeled crack data set until convergence to obtain a crack disease recognition model; a recognition module, used to obtain a bridge surface image using a drone, and input it into the crack disease recognition model to identify crack diseases on the bridge.

[0016] In a third aspect, an embodiment of the present application provides a device, comprising: a processor; a memory for storing processor-executable instructions; when the processor executes the executable instructions, it implements the method described in the first aspect or any possible implementation method of the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium includes a device for storing a computer program or instructions, and when the computer program or instructions are executed, the method described in the first aspect or any possible implementation method of the first aspect is implemented.

[0018] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0019] The embodiment of the present application adopts the focus modulation module to replace the spatial pyramid pooling module, which can improve the recognition accuracy of the recognition model and adapt to inputs of different sizes. By replacing the C3 module with a dual residual module, the feature extraction capability and flexibility of the model can be improved. By introducing the SimAM attention mechanism, the accuracy and robustness of the target detection of the recognition model can be improved. It effectively solves the problem of low recognition accuracy of the existing YOLOv5 model when dealing with scenes such as small size, low contrast and complex background. Therefore, crack diseases on bridges can be identified more timely and accurately, the efficiency and accuracy of bridge disease detection are improved, and the economic losses caused by crack diseases on bridges are avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A flow chart of a bridge crack disease identification method provided in an embodiment of the present application;

[0022] Figure 2 A schematic diagram of the structure of a bridge crack disease identification device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] The following describes some of the techniques involved in the embodiments of the present application to facilitate understanding, and they should be considered as merely exemplary. Therefore, it should be appreciated by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, some descriptions of well-known functions and structures are omitted in the following description.

[0025] Figure 1 1 is a flow chart of a bridge crack disease identification method provided in an embodiment of the present application, including steps 101 to 104. Figure 1This is only an execution order shown in the embodiment of the present application, and does not represent the only execution order of a bridge crack disease identification method. In the case where the final result can be achieved, Figure 1 The steps shown may be performed in parallel or reversed.

[0026] Step 101: Obtain a crack image of the bridge surface and preprocess it to obtain a labeled crack dataset. In the embodiment of the present application, the crack image is enhanced to obtain an enhanced crack image. Specifically, the color of the crack part is usually darker than the other parts of the bridge surface, and the contrast of the crack image is enhanced to make the crack more obvious. Those skilled in the art may also grayscale the crack image here.

[0027] Determine the annotation type, and use the annotation type to draw the annotation area on the enhanced crack image. Specifically, the annotation type includes rectangle, polygon, etc., and the annotation type is determined according to the shape and complexity of the crack.

[0028] Set the label category and add the corresponding label category in the annotation area to obtain the annotated crack image. The label category includes crack location and crack morphology. Specifically, the crack morphology includes deep cracks (cracks go deeper into the bridge and the crack surface opening is larger) and shallow cracks (cracks are only on the outer surface of the bridge and the crack surface opening is smaller). Furthermore, deep cracks and shallow cracks can be classified into transverse cracks, longitudinal cracks, oblique cracks and mesh cracks.

[0029] Data enhancement processing is performed on all annotated crack images to obtain an annotated crack data set. Exemplarily, the Mosaic data enhancement method is used here to perform data enhancement processing on the annotated crack images. Specifically, the annotated crack images are horizontally flipped and / or vertically flipped and / or randomly adjusted in brightness and / or saturation and / or contrast to obtain an enhanced annotated crack image set. Here, the annotated crack images are subjected to a variety of transformations to increase the amount of data in the annotated crack data set, so that the trained model is exposed to more diverse scenes and target combinations, thereby enhancing the robustness and generalization ability of the model.

[0030] Step 102: Replace the spatial pyramid pooling module in the YOLOv5 model with the focus modulation module, replace the C3 module in the YOLOv5 model with the dual residual module, and introduce the SimAM attention mechanism in the YOLOv5 model to build a recognition model. In an embodiment of the present application, locate and remove the spatial pyramid pooling module in the YOLOv5 model. Insert the focus modulation module into the YOLOv5 model, adjust the parameters of the YOLOv5 model according to the characteristics of the focus modulation module, and update the configuration file. Locate and remove the C3 module in the neck part of the YOLOv5 model. Insert the dual residual module into the YOLOv5 model, adjust the parameters of the YOLOv5 model according to the characteristics of the dual residual module, and update the configuration file. Locate its detection head in the YOLOv5 model, and add a SimAM module at the end of each detection head.

[0031] Specifically, the YOLOv5 model includes a backbone network, PAFPN (Path Aggregation Feature Pyramid Network, a feature pyramid network structure for target detection) and a detection head. The backbone network is a 5-layer structure, including a backbone layer, a first-stage layer, a second-stage layer, a third-stage layer and a fourth-stage layer. The spatial pyramid pooling module (SPPF) in the fourth-stage layer is replaced by the focal modulation module (FocalModulation). The C3 module in the neck part (i.e., the PAFPN network) is replaced by a dual residual module (i.e., C2f_Res2Block). Finally, a SimAM module is added at the end of each detection head.

[0032] Specifically, the focus modulation module consists of a linear projection layer, a local perception layer, a global average pooling and integration layer, a normalization layer, and a linear projection post-processing layer.

[0033] The linear projection layer consists of a 1×1 two-dimensional standard convolution kernel, which performs a convolution operation on the input feature x to obtain the output feature x_linear. The shape of the output feature x_linear is . Among them, dim represents the number of channels, B represents the batch size, and H and W represent the height and width of the feature map.

[0034] The local perception layer can perform multi-level feature extraction according to the size and feature organization of different convolution kernels. There are 3 layers in this application, each layer applies depthwise separable convolution, and its groups parameter is equal to the number of channels of the input feature x. The convolution kernel sizes are 3, 5, and 7 respectively, and the independence between channels is maintained. Each layer uses GELU (an activation function based on Gaussian error function) function as the activation function. The output of the depthwise separable convolution corresponds to the processed feature ctx after being processed by the local perception layer, and its shape is . Among them, dim represents the number of channels, B represents the batch size, and H and W represent the height and width of the feature map.

[0035] Using the global average pooling and integration layer, we first calculate the average value of the input feature ctx in the height H dimension to obtain a shape of The height tensor of , and then the average value is calculated on the width W dimension, resulting in a shape of The width tensor ctx_global is obtained by using the GELU activation function to enhance the nonlinear characteristics of the width tensor ctx_global and obtain the features after global average pooling. Then, the features after global average pooling are weighted and integrated to obtain the local perception features ctx_all.

[0036] The normalization layer normalizes the local perception features ctx_all obtained in the global average pooling and integration layer by calculating ctx_all / 3, and its shape is .

[0037] The linear projection post-processing layer uses a 1×1 convolution kernel to linearly project the local perception feature ctx_all after focus modulation, and randomly inactivates the linear projection result to reduce the risk of overfitting. The output feature shape is .

[0038] The dual residual module consists of the first convolutional layer (cv1), a bottleneck structure, and the second convolutional layer (cv2).

[0039] Specifically, the first convolutional layer (cv1) receives the input feature map and expands the number of channels of the input feature map from c1 to . Where c is the number of intermediate hidden channels, and , c2 is the number of output channels, and e is an expansion factor, which is exemplarily set to 0.5 here.

[0040] The bottleneck structure consists of n Bottle2neck (a network structure that reduces the dimension of the feature map, i.e. the number of channels, through a specific combination of convolutional layers) modules, each Bottle2neck module uses 2 3×3 convolution kernels and an optional shortcut connection.

[0041] The second convolutional layer (cv2) performs a 1×1 convolution operation on the feature map after bottleneck structure processing to reduce its channel number. Compress back to c2, where n is the number of Bottle2neck in the bottleneck structure, c is the number of intermediate hidden channels, and c2 is the number of output channels.

[0042] The specific calculation steps are as follows: the input feature map first passes through the first convolutional layer for preliminary feature extraction, and then the generated feature map is split into two parts and passed to the Bottle2neck module respectively. Each Bottle2neck module receives a part of the feature map and performs further feature extraction and transformation on it. Finally, all the processed feature maps are spliced ​​together, and the spliced ​​feature maps are fused and transformed through the second convolutional layer to obtain the output feature map.

[0043] The SimAM attention mechanism is introduced into the YOLOv5 model, including:

[0044] Mean calculation: The input feature tensor is mean calculated along the height and width dimensions to obtain the mean vector of each channel. Variance calculation: The square difference between each pixel of the input feature tensor and the mean vector is calculated to obtain the square difference tensor. Weighted adjustment: The square difference tensor is weighted to correct the accuracy and stability of the feature expression. Exemplarily, the weight coefficient here is set to 0.5. Nonlinear mapping: The Sigmoid activation function is applied to the weighted square difference tensor for nonlinear mapping. Feature modulation: The input feature tensor is element-by-element multiplied with the square difference tensor activated by the Sigmoid activation function to obtain the enhanced output feature tensor.

[0045] Step 103: Use the annotated crack data to train the recognition model until convergence to obtain a crack disease recognition model. In an embodiment of the present application, if during the training process, the loss value of the recognition model tends to be stable within a period of time and no longer decreases significantly, it can be considered that the recognition model has converged. Or when the recognition model is evaluated using a validation set, it is observed that the performance of the recognition model on the validation set is stable and reaches the expected goal, and it can also be considered that the recognition model has converged. Furthermore, those skilled in the art can set reasonable training rounds (epochs) based on experience or actual experience. After reaching the set training rounds, if the performance of the recognition model has not been significantly improved, it can also be considered to stop training and consider that the recognition model at this time has converged. After the recognition model converges, the crack disease recognition model can be obtained.

[0046] Step 104: Use a drone to obtain a bridge surface image, and input the image into a crack disease recognition model to identify crack diseases on the bridge. In the embodiment of the present application, using a drone to obtain a bridge surface image includes: the drone flies according to a planned target route to collect the bridge surface image. Specifically, the specific method of the drone flying according to the planned target route is as follows:

[0047]

[0048] in, ,and .

[0049] In the formula, is the position coordinate of the drone at time i, represents the control coefficient at time i, , They represent the maximum and minimum distances from the bridge in the X-axis direction when the UAV is executing the target route. , They represent the maximum and minimum distances of the UAV from the bridge in the Y-axis direction when executing the target route. , They represent the maximum and minimum distances of the UAV from the bridge in the Z-axis direction when executing the target route. represents the environmental impact at time i, and N represents the total time required for the UAV to execute the target route.

[0050] Among them, environmental impact That is, the impact of wind force near the bridge on the flight of the UAV, and its value range is The stronger the wind, The larger the value, the greater the minimum distance between the drone and the bridge, which can prevent the drone from colliding with the bridge due to wind and causing damage.

[0051] It should be noted that the position coordinates at time i are the relative position of the drone to the starting point of its target route at time i. In other words, a spatial rectangular coordinate system is established with the starting point of the target route as the origin, the length of the bridge as the Y axis, the height of the bridge as the Z axis, and the width of the bridge as the X axis. The position coordinates at time i are the relative distances (closest distances) of the drone from the three coordinate axes at time i.

[0052] The bridge surface image is subjected to denoising and geometric correction. Specifically, a filter is used to remove noise points in the bridge surface image, and geometric correction is performed to correct camera distortion.

[0053] Feature extraction is performed on the processed bridge surface images to remove bridge surface images without cracks.

[0054] Although the present application provides method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in this embodiment is only one way of executing the order of many steps and does not represent the only execution order. When the actual device or client product is executed, it can be executed in sequence or in parallel according to the method shown in this embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment).

[0055] like Figure 2As shown, the embodiment of the present application also provides a bridge crack disease identification device 200. The device includes: a labeling module 201, a construction module 202, a training module 203 and a recognition module 204, which are specifically as follows.

[0056] The labeling module 201 is used to obtain a crack image set on the bridge surface and preprocess it to obtain a labeled crack data set.

[0057] The construction module 202 is used to replace the spatial pyramid pooling module in the YOLOv5 model with the focus modulation module, replace the C3 module in the YOLOv5 model with the dual residual module, and introduce the SimAM attention mechanism in the YOLOv5 model to construct a recognition model.

[0058] The training module 203 is used to train the recognition model using the annotated crack data set until convergence, thereby obtaining a crack disease recognition model.

[0059] The recognition module 204 is used to obtain the bridge surface image by using the drone, and input the image into the crack disease recognition model to identify the crack disease on the bridge.

[0060] Some modules in the apparatus described in the present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0061] The devices or modules described in the above application embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above devices are described in various modules according to their functions. When implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware. Of course, the module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0062] The methods, devices or modules described in this application can be implemented in the form of computer-readable program codes. The controller can be implemented in any appropriate manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program codes (such as software or firmware) that can be executed by the (micro)processor, logic gates, switches, application-specific integrated circuits (English: Application Specific Integrated Circuit; Abbreviation: ASIC), programmable logic controllers and embedded microcontrollers. Examples of controllers include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in the form of pure computer-readable program codes, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included in it for implementing various functions can also be regarded as structures within the hardware component. Or even, the means for realizing various functions may be regarded as both a software module for realizing the method and a structure within a hardware component.

[0063] An embodiment of the present application further provides a device, comprising: a processor; a memory for storing processor executable instructions; when the processor executes the executable instructions, the method described in the embodiment of the present application is implemented.

[0064] The embodiments of the present application also provide a non-volatile computer-readable storage medium on which a computer program or instruction is stored. When the computer program or instruction is executed, the method described in the embodiments of the present application is implemented.

[0065] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist independently, or two or more modules may be integrated into one module.

[0066] The above storage media include but are not limited to random access memory (RAM), read-only memory (ROM), cache, hard disk drive (HDD) or memory card. The memory can be used to store computer program instructions.

[0067] It can be seen from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary hardware. Based on this understanding, the technical solution of the present application can be essentially or partly reflected in the prior art in the form of a software product, or it can be reflected in the implementation process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0068] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. All or part of this application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, mobile communication terminals, multi-processor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0069] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the present application.

Claims

1. A bridge crack disease identification method, characterized in that: include: Obtain crack images on the bridge surface and preprocess them to obtain annotated crack datasets; The spatial pyramid pooling module in the YOLOv5 model is replaced by the focal modulation module, the C3 module in the YOLOv5 model is replaced by the dual residual module, and the SimAM module is added at the end of each detection head of the YOLOv5 model to build a recognition model; Among them, the focus modulation module includes a linear projection layer, a local perception layer, a global average pooling and integration layer, a normalization layer and a linear projection post-processing layer; the local perception layer performs multi-level feature extraction according to the size and feature organization of different convolution kernels, and applies depth-separable convolution at each level, and the output of the depth-separable convolution corresponds to the processing features after processing by the local perception layer; the global average pooling and integration layer calculates the average value of the input processing features in the height dimension to obtain a height tensor, and then calculates the average value in the width dimension to obtain a width tensor, and uses the GELU activation function to enhance the nonlinear characteristics of the width tensor to obtain the features after global average pooling, and performs weighted integration on the features after global average pooling to obtain local perception features; the normalization layer performs a normalization operation on the local perception features obtained in the global average pooling and integration layer in the calculation method of local perception features / 3; the linear projection post-processing layer uses a 1×1 convolution kernel to linearly project the local perception features after focus modulation, and randomly inactivates the linear projection results; The dual residual module includes a first convolutional layer, a bottleneck structure and a second convolutional layer; the first convolutional layer is used to receive an input feature map and expand the number of channels of the input feature map through a 1×1 convolution operation, the bottleneck structure is composed of multiple Bottle2neck modules, and the second convolutional layer performs a 1×1 convolution operation on the feature map processed by the bottleneck structure and compresses the number of channels; specifically, the input feature map is input into the first convolutional layer for preliminary feature extraction, and the generated feature map is split into multiple parts, which are respectively passed to the Bottle2neck modules; each of the Bottle2neck modules receives a part of the feature map, further extracts and transforms the feature map, and splices all the processed feature maps together; the spliced ​​feature map is subjected to feature fusion and transformation through the second convolutional layer to obtain an output feature map; Using the annotated crack data set to train the recognition model until convergence, to obtain a crack disease recognition model; The bridge surface image is obtained by using a drone, and the image is input into the crack disease recognition model to identify crack diseases on the bridge; wherein the bridge surface image is obtained by using a drone, including: The drone flies according to the planned target route to collect images of the bridge surface, as follows: ； in, ,and ; In the formula, is the position coordinate of the drone at time i, represents the control coefficient at time i, , They respectively represent the maximum and minimum distances of the UAV from the bridge in the X-axis direction during the execution of the target route. , They respectively represent the maximum and minimum distances of the UAV from the bridge in the Y-axis direction during the execution of the target route. , They respectively represent the maximum and minimum distances of the UAV from the bridge in the Z-axis direction during the execution of the target route. represents the environmental impact at time i, that is, the impact of the wind force near the bridge on the flight of the UAV at time i, N represents the total time required for the UAV to execute the target route; the position coordinate at time i is the relative position of the UAV to the starting position of its target route at time i.

2. The method according to claim 1, characterized in that The preprocessing is performed to obtain a labeled crack data set, including: Performing graphic enhancement on the crack image to obtain an enhanced crack image; Determining a marking type, and drawing a marking area on the enhanced crack image using the marking type; Setting a label category, adding a label category corresponding to the label category in the labeled area to obtain a labeled crack image; wherein the label category includes a crack position and a crack morphology; Data enhancement processing is performed on all the annotated crack images to obtain the annotated crack data set.

3. The method according to claim 2, characterized in that The performing data enhancement processing on all the annotated crack images includes: The annotated crack image is horizontally flipped and / or vertically flipped and / or randomly brightness and / or saturation adjusted and / or contrast adjusted.

4. The method according to claim 1, characterized in that: The method uses the focus modulation module to replace the spatial pyramid pooling module in the YOLOv5 model, uses the dual residual module to replace the C3 module in the YOLOv5 model, and adds the SimAM module at the end of each detection head of the YOLOv5 model, including: Locate and remove the spatial pyramid pooling module in the YOLOv5 model; Insert the focus modulation module into the YOLOv5 model, adjust the parameters of the YOLOv5 model according to the characteristics of the focus modulation module, and update the configuration file; Locate and remove the C3 module in the neck part of the YOLOv5 model; Insert the dual residual module into the YOLOv5 model, adjust the parameters of the YOLOv5 model according to the characteristics of the dual residual module, and update the configuration file; Position its detection head in the YOLOv5 model and add the SimAM module at the end of each detection head.

5. The method according to claim 1, characterized in that The method of obtaining the bridge surface image by using a drone includes: The drone flies according to the planned target route to collect images of the bridge surface; Performing denoising and geometric correction on the bridge surface image; Feature extraction is performed on the processed bridge surface image to remove the bridge surface image that does not contain cracks.

6. A bridge crack disease identification device, characterized in that: include: The labeling module is used to obtain a set of crack images on the bridge surface and preprocess them to obtain a labeled crack data set; A construction module is used to replace the spatial pyramid pooling module in the YOLOv5 model with the focal modulation module, replace the C3 module in the YOLOv5 model with the dual residual module, and add a SimAM module at the end of each detection head of the YOLOv5 model to build a recognition model; Among them, the focus modulation module includes a linear projection layer, a local perception layer, a global average pooling and integration layer, a normalization layer and a linear projection post-processing layer; the local perception layer performs multi-level feature extraction according to the size and feature organization of different convolution kernels, and applies depth-separable convolution at each level, and the output of the depth-separable convolution corresponds to the processing features after processing by the local perception layer; the global average pooling and integration layer calculates the average value of the input processing features in the height dimension to obtain a height tensor, and then calculates the average value in the width dimension to obtain a width tensor, and uses the GELU activation function to enhance the nonlinear characteristics of the width tensor to obtain the features after global average pooling, and performs weighted integration on the features after global average pooling to obtain local perception features; the normalization layer performs a normalization operation on the local perception features obtained in the global average pooling and integration layer in the calculation method of local perception features / 3; the linear projection post-processing layer uses a 1×1 convolution kernel to linearly project the local perception features after focus modulation, and randomly inactivates the linear projection results; The dual residual module includes a first convolutional layer, a bottleneck structure and a second convolutional layer; the first convolutional layer is used to receive an input feature map and expand the number of channels of the input feature map through a 1×1 convolution operation, the bottleneck structure is composed of multiple Bottle2neck modules, and the second convolutional layer performs a 1×1 convolution operation on the feature map processed by the bottleneck structure and compresses the number of channels; specifically, the input feature map is input into the first convolutional layer for preliminary feature extraction, and the generated feature map is split into multiple parts, which are respectively passed to the Bottle2neck modules; each of the Bottle2neck modules receives a part of the feature map, further extracts and transforms the feature map, and splices all the processed feature maps together; the spliced ​​feature map is subjected to feature fusion and transformation through the second convolutional layer to obtain an output feature map; A training module, used for training the recognition model using the annotated crack data set until convergence, to obtain a crack disease recognition model; The recognition module is used to obtain the bridge surface image by using the drone, and input the image into the crack disease recognition model to identify the crack disease on the bridge; wherein, obtaining the bridge surface image by using the drone includes: The drone flies according to the planned target route to collect images of the bridge surface, as follows: ； in, ,and ; In the formula, is the position coordinate of the drone at time i, represents the control coefficient at time i, , They respectively represent the maximum and minimum distances of the UAV from the bridge in the X-axis direction during the execution of the target route. , They respectively represent the maximum and minimum distances of the UAV from the bridge in the Y-axis direction during the execution of the target route. , They respectively represent the maximum and minimum distances of the UAV from the bridge in the Z-axis direction during the execution of the target route. represents the environmental impact at time i, that is, the impact of the wind near the bridge on the flight of the drone at time i, N represents the total time required for the drone to execute the target route; the position coordinate at time i is the relative position of the drone from the starting point of its target route at time i.

7. A device for performing a bridge crack disease identification method, characterized in that: include: processor; a memory for storing processor-executable instructions; When the processor executes the executable instructions, the method according to any one of claims 1 to 5 is implemented.

8. A non-volatile computer-readable storage medium, characterized in that: The device comprises a computer program or an instruction for storing the computer program or the instruction, which, when executed, enables the method according to any one of claims 1 to 5 to be implemented.

Images