A method, system, electronic device and storage medium for detecting bridge defects

Abstract

This invention provides a method, system, electronic device, and storage medium for bridge defect detection. The method includes collecting a bridge image dataset; preparing a defect dataset based on the bridge image dataset; using nested U-shaped network blocks as a network skeleton, and replacing some preset U-shaped network blocks in the network skeleton with several GHPA layers; embedding DW convolutional layers into the first layer of each group of U-shaped network blocks, and introducing SC convolutional layers in the last layer of the network skeleton to generate a detection network model; training the detection network model; identifying defect information images in the bridge image dataset based on the optimal detection network model; fusing the defect information images with the original images in the bridge image dataset; and establishing the size relationship between the defect information images and the bridge image dataset. This invention not only avoids information omissions during the detection process but also further improves detection accuracy and efficiency.

Description

Technical Field

[0001] This invention relates to the field of bridge inspection technology, and in particular to a method, system, electronic device and storage medium for bridge defect detection. Background Technology

[0002] Bridges are an essential component of roads and a key element of traffic engineering. They are generally structures built over rivers, lakes, and seas to allow vehicles and pedestrians to pass smoothly. With urban development, the construction of elevated bridges in cities can effectively alleviate traffic congestion.

[0003] As bridge construction continues to advance, the number of bridges is also increasing, making bridge inspection an increasingly important aspect. In daily life, bridges are constantly subjected to external loads and environmental factors, which can affect their structural safety. To prevent safety accidents, bridge inspections are necessary to identify potential risks in advance.

[0004] In existing technologies, machine vision or artificial intelligence is often used to detect bridge defects in order to improve detection efficiency. However, machine vision has problems with low detection accuracy and poor detection efficiency due to different shooting conditions. Artificial intelligence may miss information when processing images, which can lead to omissions in bridge detection and result in incomplete detection. Summary of the Invention

[0005] Therefore, the purpose of this invention is to provide a bridge defect detection method, system, electronic device, and storage medium to overcome the shortcomings of the prior art.

[0006] In a first aspect, the present invention provides a method for detecting bridge defects, the method comprising:

[0007] Collect a bridge image dataset, prepare a disease dataset based on the bridge image dataset, and process the disease dataset;

[0008] A network skeleton is constructed by using nested U-shaped network blocks, and a predetermined portion of the U-shaped network blocks in the network skeleton is replaced by a number of GHPA layers, so that the number of GHPA layers and the portion of the U-shaped network blocks are alternately arranged.

[0009] DW convolutional layers are embedded in the first layer of each group of the U-shaped network blocks, and SC convolutional layers are introduced in the last layer of the network skeleton to generate a detection network model.

[0010] The disease dataset is divided and processed, and the detection network model is trained based on the training dataset in the divided and processed disease dataset to obtain an optimized detection network model.

[0011] Based on the GHPA layer in the optimized detection network model, the input mapping channel is divided into several groups, and the input mapping value and several learning tensors are mapped onto the GHPA layer according to the several groups of input mapping channels to obtain the optimal detection network model.

[0012] Based on the optimal detection network model, the defect information images in the bridge image dataset are identified, the defect information images are fused with the original images in the bridge image dataset, and the size relationship between the defect information images and the bridge image dataset is established.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: by using several U-shaped network blocks as the network skeleton and replacing some of the U-shaped network blocks with GHPA layers, the GHPA layers and U-shaped network blocks are alternately set. By embedding the DW convolutional layer into the first layer of each group of U-shaped network blocks and introducing the SC convolutional layer in the last layer of the network skeleton, the generated detection network model is not only more lightweight, but also effectively improves its detection performance. In addition, by training the detection network model, not only can information omissions be avoided during the detection process, but detection accuracy and efficiency can also be further improved. By establishing the size relationship between the disease information image and the bridge image dataset, the actual size of the detected disease information can be obtained, which facilitates subsequent remedial measures for the disease.

[0014] Furthermore, the step of preparing a disease dataset based on the bridge image dataset and processing the disease dataset includes:

[0015] The LabelImg tool was used to annotate several defect images in the bridge image dataset to obtain the defect dataset;

[0016] Cropping disease images from the disease dataset based on the maximum value of the annotation box of the disease image, and annotating the size of the disease information in the disease image;

[0017] The disease dataset is then subjected to rotation, translation, mirroring, stitching, and balancing processes in sequence.

[0018] Furthermore, the step of using nested U-shaped network blocks as a network skeleton, and replacing a predetermined portion of the U-shaped network blocks in the network skeleton with several GHPA layers, so that the several GHPA layers and the portion of the U-shaped network blocks are alternately arranged, includes:

[0019] The network skeleton is constructed based on six sets of U-shaped network blocks, and four sets of the U-shaped network blocks are replaced with four sets of GHPA layers;

[0020] The last two U-shaped network blocks in the network skeleton are replaced by two of the GHPA layers, and the other two GHPA layers are interleaved with the other two unreplaced U-shaped network blocks.

[0021] Furthermore, the step of introducing an SC convolutional layer as the last layer in the network skeleton includes:

[0022] Based on the space and channels of the SC convolutional layer, the U-shaped network block of the last layer in the network skeleton is reconstructed, so that the SC convolutional layer replaces the U-shaped network block of the last layer in the network skeleton.

[0023] Furthermore, the step of dividing the input mapping channels into several groups based on the GHPA layer in the optimized detection network model, and mapping the input mapping values ​​and several learning tensors onto the GHPA layer according to the several groups of input mapping channels includes:

[0024] The input mapping execution layer in the GHPA layer is normalized, and the input mapping channel is divided into four groups based on the normalized input mapping execution layer.

[0025] Based on the four sets of input mapping channels, the input mapping values ​​are grouped and mapped to the GHPA layer to perform feature separation mapping and obtain four sets of feature mappings;

[0026] Several learning tensors are randomly initialized, and the randomly initialized learning tensors are input to the GHPA layer based on the four sets of input mapping channels.

[0027] Furthermore, the calculation expression for the input mapping value grouping mapping is:

[0028] ;

[0029] In the formula, , , , These represent the first feature, the second feature, the third feature, and the fourth feature, respectively. This represents the scaling parameter. Indicates the first One characteristic, Represents the characteristic average. Represents the variance of the input features. Represents a constant. Indicates the offset parameter;

[0030] The expressions for the four sets of feature mappings are:

[0031] ;

[0032] In the formula, , , , These represent the first feature, the second feature, the third feature, and the fourth feature, respectively. This represents depthwise separable convolution. Indicates bilinear interpolation. Represents pixel position information, Indicates the node position. This represents the matrix transpose symbol.

[0033] Furthermore, the step of fusing the disease information image with the original image in the bridge image dataset and establishing the size relationship between the disease information image and the bridge image dataset includes:

[0034] Based on the optimal detection network model, the coordinates of the disease information in the disease information image are determined, and the disease information image is fused with the bridge image dataset according to the coordinates;

[0035] The size relationship between the disease information image and the bridge image dataset is established based on the ratio of the pixel distance between the disease information image and the bridge image dataset.

[0036] Secondly, the present invention also provides a bridge defect detection system, the system comprising:

[0037] A collection module is used to collect bridge image datasets, prepare a disease dataset based on the bridge image datasets, and process the disease datasets.

[0038] The replacement module is used to use a number of nested U-shaped network blocks as a network skeleton, and to replace a predetermined portion of the U-shaped network blocks in the network skeleton with a number of GHPA layers, so that the number of GHPA layers and the portion of the U-shaped network blocks are alternately arranged.

[0039] An introduction module is used to embed DW convolutional layers into the first layer of each group of the U-shaped network blocks and to introduce SC convolutional layers into the last layer of the network skeleton to generate a detection network model.

[0040] The training module is used to divide the processed disease dataset and train the detection network model based on the training dataset in the divided disease dataset to obtain an optimized detection network model.

[0041] The mapping module is used to divide the input mapping channels into several groups based on the GHPA layer in the optimized detection network model, and to map the input mapping values ​​and several learning tensors onto the GHPA layer according to the several groups of input mapping channels to obtain the optimal detection network model.

[0042] The identification module is used to identify the defect information images in the bridge image dataset based on the optimal detection network model, fuse the defect information images with the original images in the bridge image dataset, and establish the size relationship between the defect information images and the bridge image dataset.

[0043] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described bridge defect detection method.

[0044] Fourthly, the present invention also provides a storage medium storing a computer program thereon, characterized in that the program, when executed by a processor, implements the above-described bridge defect detection method. Attached Figure Description

[0045] Figure 1 This is a flowchart of the bridge defect detection method in the first embodiment of the present invention;

[0046] Figure 2 This is a structural block diagram of the bridge defect detection system according to the second embodiment of the present invention;

[0047] Figure 3 This is a schematic diagram of the structure of the electronic device in the third embodiment of the present invention.

[0048] Explanation of key component symbols:

[0049] 10. Collection module; 20. Replacement module; 30. Introduction module; 40. Training module; 50. Mapping module; 60. Recognition module;

[0050] 70. Bus; 71. Processor; 72. Memory; 73. Communication interface.

[0051] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0052] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0053] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0055] Example 1

[0056] Please see Figure 1 The diagram shows a bridge defect detection method according to the first embodiment of the present invention, the method comprising steps S1 to S6:

[0057] S1, collect a bridge image dataset, prepare a disease dataset based on the bridge image dataset, and process the disease dataset;

[0058] Specifically, step S1 includes steps S11 to S13:

[0059] S11, Use the LabelImg tool to label several defect images in the bridge image dataset to obtain a defect dataset;

[0060] S12, based on the maximum value of the annotation box of the disease image, crop the disease image in the disease dataset, and annotate the size of the disease information in the disease image;

[0061] S13, the disease dataset is sequentially rotated, translated, mirrored, stitched together and balanced;

[0062] It is understandable that the LabelImg tool can be used to annotate images with rectangular boxes, polygonal boxes, and circular boxes. In this embodiment, different annotation boxes can be used for annotation according to different disease conditions. For example, longitudinal cracks and exposed rebar strips can be annotated with rectangular boxes, diagonal cracks, edge peeling, and surface flaking can be annotated with polygonal boxes, and diseases such as holes can be annotated with circular boxes. After annotation, the size of the disease information can be marked within the annotation box by cutting out the annotation box with the disease information. By rotating, translating, mirroring, stitching, and balancing the disease dataset in sequence, unbalanced data in the dataset can be removed.

[0063] It is worth noting that when the annotation box is a rectangle or a polygon, the coordinate expression of the annotation box in the lesion image is:

[0064] ;

[0065] When the annotation box is a circle, the coordinate expression of the annotation box in the lesion image is:

[0066] ;

[0067] In the formula, , These represent the marked disease information. The distance from the point to the left boundary of the annotation box, and the annotation of the disease information. The distance between the point and the left boundary of the original image. , These represent the marked disease information. The distance from the point to the upper boundary of the annotation box, and the annotation of the disease information. The distance between the point and the upper boundary of the original image. , These represent the distances between the left boundary of the labeled disease information and the center of the circular label box, and the distances between the center of the circular label box of the labeled disease information and the original image, respectively. , These represent the distances between the upper boundary of the labeled disease information and the center of the circular annotation box, and the distances between the upper boundary of the labeled disease information and the original image, respectively. , These represent the distance between the original bounding box and the left edge of the original image, and the distance between the original bounding box and the top edge of the original image, respectively.

[0068] S2, a number of nested U-shaped network blocks are used as the network skeleton, and a number of GHPA layers are used to replace a predetermined portion of the U-shaped network blocks in the network skeleton, so that the number of GHPA layers and the portion of the U-shaped network blocks are alternately arranged;

[0069] Specifically, step S2 includes steps S21 to S22:

[0070] S21, The network skeleton is constructed based on six sets of U-shaped network blocks, and four sets of the U-shaped network blocks are replaced with four sets of GHPA layers;

[0071] S22, based on two of the GHPA layers, replace the last two U-shaped network blocks in the network skeleton, and make the other two GHPA layers and the other two unreplaced U-shaped network blocks alternately arranged;

[0072] Understandably, in order to achieve model lightweighting, four sets of U-shaped network blocks are replaced with four sets of GHPA layers, and two of the four sets of GHPA layers replace the last two sets of U-shaped network blocks. The other two sets of GHPA layers are interleaved with the U-shaped network blocks, thereby improving the network's extraction capabilities and achieving network model lightweighting.

[0073] S3, embed the DW convolutional layer into the first layer of each group of the U-shaped network blocks, and introduce the SC convolutional layer in the last layer of the network skeleton to generate the detection network model;

[0074] Specifically, step S3 includes step S31:

[0075] S31, Reconstruct the U-shaped network block of the last layer in the network skeleton based on the space and channels of the SC convolutional layer, so that the SC convolutional layer replaces the U-shaped network block of the last layer in the network skeleton;

[0076] Understandably, in order to improve the model's ability to extract details, replacing the first layer of each U-shaped network block with a DW convolutional layer can prevent the loss of detail texture during detection. The last U-shaped network block in the network skeleton is replaced by an SC convolutional layer. This U-shaped network block was replaced by a GHPA layer, so the SC convolutional layer replaces the last GHPA layer, thereby reducing the loss of detail through the last SC convolutional layer.

[0077] S4, the disease dataset is divided into segments, and the detection network model is trained based on the training dataset in the divided disease dataset to obtain an optimized detection network model.

[0078] It is understandable that the disease dataset is divided into training, testing and validation sets in an 8:1:1 ratio. The detection network model is trained using the training set, which optimizes the detection capability of the network model and improves its ability to detect diseases.

[0079] It is worth noting that after training, the network detection model can be tested using a test set and validated using a validation set to verify its detection performance.

[0080] S5, based on the GHPA layer in the optimized detection network model, the input mapping channel is divided into several groups, and the input mapping value and several learning tensors are mapped onto the GHPA layer according to the several groups of input mapping channels to obtain the optimal detection network model;

[0081] Specifically, step S5 includes steps S51 to S53:

[0082] S51, normalize the input mapping execution layer in the GHPA layer, and divide the input mapping channel into four groups based on the normalized input mapping execution layer;

[0083] S52, based on the four sets of input mapping channels, the input mapping values ​​are grouped and mapped to the GHPA layer to perform feature separation mapping to obtain four sets of feature mappings;

[0084] The calculation expression for the input mapping value grouping mapping is as follows:

[0085] ;

[0086] In the formula, , , , These represent the first feature, the second feature, the third feature, and the fourth feature, respectively. This represents the scaling parameter. Indicates the first One characteristic, Represents the characteristic average. Represents the variance of the input features. Represents a constant. Indicates the offset parameter;

[0087] The expressions for the four sets of feature mappings are:

[0088] ;

[0089] In the formula, , , , These represent the first feature, the second feature, the third feature, and the fourth feature, respectively. This represents depthwise separable convolution. Indicates bilinear interpolation. Represents pixel position information, Indicates the node position. This represents the matrix transpose symbol.

[0090] S53, randomly initialize several learning tensors, and input several randomly initialized learning tensors into the GHPA layer based on the four sets of input mapping channels respectively;

[0091] It should be explained that, in this embodiment, the initialized learnable tensor is: ;in, This represents a learnable tensor. , These represent the height and width of the spatial dimension, respectively. Indicates the total number of channels. , These represent two hyperparameters. Represents the set of real numbers;

[0092] Understandably, dividing the input mapping into four groups can reduce the redundancy of spatial dimensions and allow for the extraction of different features from different subspaces through a multi-channel approach.

[0093] S6. Based on the optimal detection network model, identify the defect information images in the bridge image dataset, fuse the defect information images with the original images in the bridge image dataset, and establish the size relationship between the defect information images and the bridge image dataset.

[0094] Specifically, step S6 includes steps S61 to S62:

[0095] S61, Based on the optimal detection network model, determine the coordinates of the disease information in the disease information image, and fuse the disease information image with the bridge image dataset according to the coordinates;

[0096] S62, establish the size relationship between the disease information image and the bridge image dataset based on the ratio of the pixel distance between the disease information image and the bridge image dataset;

[0097] Understandably, to more accurately assess bridge defects, a dimensional relationship is established between defect information images and the bridge image dataset. In practice, an optimal detection network model identifies defect information images from the bridge image dataset. Four calibration points are set within the defect information images, and the actual distances between these points are measured. By cropping the defect information image to the calibration points, the corresponding pixel distances are obtained. By calculating the ratio of the actual distance between calibration points to the pixel distance, the ratio between the actual defect size and the pixel size can be obtained—that is, the dimensional relationship. The expression for this dimensional relationship is:

[0098] ;

[0099] In the formula, This indicates the ratio between the actual size and the pixel size. , These represent the actual length of the rectangle formed by the four calibration points and the pixel length of the rectangle formed by the four calibration points, respectively. , These represent the actual width of the rectangle formed by the four calibration points and the pixel width of the rectangle formed by the four calibration points, respectively.

[0100] In summary, the bridge defect detection method in the above embodiments of the present invention uses several U-shaped network blocks as the network skeleton and replaces some of the U-shaped network blocks with GHPA layers, so that the GHPA layers and U-shaped network blocks are alternately set. By embedding DW convolutional layers into the first layer of each group of U-shaped network blocks and introducing SC convolutional layers in the last layer of the network skeleton, the generated detection network model is not only more lightweight, but also effectively improves its detection performance. In addition, by training the detection network model, not only can information omissions be avoided during the detection process, but detection accuracy and efficiency can also be further improved. By establishing the size relationship between the defect information image and the bridge image dataset, the actual size of the detected defect information can be obtained, which facilitates subsequent remedial measures for the defects.

[0101] Example 2

[0102] The second embodiment of the present invention also proposes a bridge defect detection system, please refer to [link / reference]. Figure 2 The image shows a bridge defect detection system according to a second embodiment of the present invention. The system includes:

[0103] The collection module 10 is used to collect bridge image datasets, prepare a disease dataset based on the bridge image datasets, and process the disease datasets.

[0104] Replacement module 20 is used to use a number of nested U-shaped network blocks as a network skeleton, and to replace a preset portion of the U-shaped network blocks in the network skeleton with a number of GHPA layers, so that the number of GHPA layers and the portion of the U-shaped network blocks are alternately arranged;

[0105] Module 30 is introduced to embed the DW convolutional layer into the first layer of each group of the U-shaped network blocks and to introduce the SC convolutional layer in the last layer of the network skeleton to generate a detection network model.

[0106] Training module 40 is used to divide the processed disease dataset and train the detection network model based on the training dataset in the divided disease dataset to obtain an optimized detection network model.

[0107] The mapping module 50 is used to divide the input mapping channel into several groups based on the GHPA layer in the optimized detection network model, and to map the input mapping value and several learning tensors onto the GHPA layer according to the several groups of input mapping channels to obtain the optimal detection network model.

[0108] The recognition module 60 is used to recognize the defect information images in the bridge image dataset based on the optimal detection network model, fuse the defect information images with the original images in the bridge image dataset, and establish the size relationship between the defect information images and the bridge image dataset.

[0109] In some alternative embodiments, the collection module 10 includes:

[0110] The annotation unit is used to annotate several defect images in the bridge image dataset using the LabelImg tool to obtain the defect dataset;

[0111] The cropping unit is used to crop the disease images in the disease dataset according to the maximum value of the annotation box of the disease image, and to annotate the size of the disease information in the disease image;

[0112] The processing unit is used to perform rotation, translation, mirroring, stitching, and balancing processes on the disease dataset in sequence.

[0113] In some alternative embodiments, the replacement module 20 includes:

[0114] The first replacement unit is used to construct the network skeleton based on six sets of U-shaped network blocks, and replace four sets of the U-shaped network blocks with four sets of GHPA layers;

[0115] The second replacement unit is used to replace the last two U-shaped network blocks in the network skeleton based on two of the GHPA layers, and to interleave the other two GHPA layers with the other two unreplaced U-shaped network blocks.

[0116] In some alternative embodiments, the introducing module 30 includes:

[0117] A reconstruction unit is used to reconstruct the U-shaped network block of the last layer in the network skeleton based on the space and channels of the SC convolutional layer, so that the SC convolutional layer replaces the U-shaped network block of the last layer in the network skeleton.

[0118] In some alternative embodiments, the mapping module 50 includes:

[0119] The partitioning unit is used to perform a normalization operation on the input mapping execution layer in the GHPA layer, and divide the input mapping channel into four groups based on the input mapping execution layer after the normalization operation;

[0120] The mapping unit is used to group and map the input mapping values ​​to the GHPA layer based on the four groups of input mapping channels, so as to perform feature separation mapping to obtain four groups of feature mappings;

[0121] The calculation expression for the input mapping value grouping mapping is as follows:

[0122] ;

[0123] In the formula, , , , These represent the first feature, the second feature, the third feature, and the fourth feature, respectively. This represents the scaling parameter. Indicates the first One characteristic, Represents the characteristic average. Represents the variance of the input features. Represents a constant. Indicates the offset parameter;

[0124] The expressions for the four sets of feature mappings are:

[0125] ;

[0126] In the formula, , , , These represent the first feature, the second feature, the third feature, and the fourth feature, respectively. This represents depthwise separable convolution. Indicates bilinear interpolation. Represents pixel position information, Indicates the node position. Represents the matrix transpose symbol;

[0127] The input unit is used to randomly initialize several learning tensors and input the randomly initialized learning tensors into the GHPA layer based on the four sets of input mapping channels.

[0128] In some alternative embodiments, the identification module 60 includes:

[0129] The fusion unit is used to determine the coordinates of the disease information in the disease information image based on the optimal detection network model, and to fuse the disease information image with the bridge image dataset according to the coordinates.

[0130] The establishment unit is used to establish the size relationship between the disease information image and the bridge image dataset based on the ratio of the pixel distance between the disease information image and the bridge image dataset.

[0131] The functions or operation steps implemented by the above modules and units are largely the same as those in the above method embodiments, and will not be repeated here.

[0132] The bridge defect detection system provided in this embodiment of the invention has the same implementation principle and technical effects as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the system embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0133] Example 3

[0134] The third embodiment of the present invention also proposes an electronic device, please refer to [link / reference]. Figure 3 The image shows an electronic device according to a third embodiment of the present invention.

[0135] The electronic device may include a processor 71 and a memory 72 storing computer program instructions.

[0136] Specifically, the processor 71 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement this application.

[0137] The memory 72 may include a mass storage device for data or instructions. For example, and not limitingly, the memory 72 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 72 may include removable or non-removable (or fixed) media. Where appropriate, the memory 72 may be internal or external to a data processing device. In a particular embodiment, the memory 72 is non-volatile memory. In a particular embodiment, the memory 72 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0138] The memory 72 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 71.

[0139] The processor 71 reads and executes the computer program instructions stored in the memory 72 to implement the bridge defect detection method of the above embodiment 1.

[0140] In some embodiments, the electronic device may further include a communication interface 73 and a bus 70. For example, Figure 3 As shown, the processor 71, memory 72, and communication interface 73 are connected through bus 70 and complete communication with each other.

[0141] The communication interface 73 is used to enable communication between the various modules, devices, units, and / or equipment in this application. The communication interface 73 can also enable data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0142] Bus 70 includes hardware, software, or both, that couples the components of a device together. Bus 70 includes, but is not limited to, at least one of the following: data bus, address bus, control bus, expansion bus, and local bus. For example, and not as a limitation, bus 70 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 70 may include one or more buses. Although this application describes and illustrates a specific bus, this application considers any suitable bus or interconnection.

[0143] The electronic device can acquire the bridge defect detection system and execute the bridge defect detection method of this embodiment.

[0144] In addition, in conjunction with the bridge defect detection method in Embodiment 1 above, this application can provide a storage medium for implementation. This storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement the bridge defect detection method of Embodiment 1 above.

[0145] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0146] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for detecting bridge defects, characterized in that, The method includes: Collect a bridge image dataset, prepare a disease dataset based on the bridge image dataset, and process the disease dataset; A network skeleton is constructed by using nested U-shaped network blocks, and a predetermined portion of the U-shaped network blocks in the network skeleton is replaced by a number of GHPA layers, so that the number of GHPA layers and the portion of the U-shaped network blocks are alternately arranged. DW convolutional layers are embedded in the first layer of each group of the U-shaped network blocks, and SC convolutional layers are introduced in the last layer of the network skeleton to generate a detection network model. The disease dataset is divided and processed, and the detection network model is trained based on the training dataset in the divided and processed disease dataset to obtain an optimized detection network model. Based on the GHPA layer in the optimized detection network model, the input mapping channel is divided into several groups, and the input mapping value and several learning tensors are mapped onto the GHPA layer according to the several groups of input mapping channels to obtain the optimal detection network model. Based on the optimal detection network model, the defect information images in the bridge image dataset are identified, the defect information images are fused with the original images in the bridge image dataset, and the size relationship between the defect information images and the bridge image dataset is established.

2. The bridge defect detection method according to claim 1, characterized in that, The steps of preparing a disease dataset based on the bridge image dataset and processing the disease dataset include: The LabelImg tool was used to annotate several defect images in the bridge image dataset to obtain the defect dataset; Cropping disease images from the disease dataset based on the maximum value of the annotation box of the disease image, and annotating the size of the disease information in the disease image; The disease dataset is then subjected to rotation, translation, mirroring, stitching, and balancing processes in sequence.

3. The bridge defect detection method according to claim 1, characterized in that, The step of using nested U-shaped network blocks as a network skeleton, and replacing a predetermined portion of the U-shaped network blocks in the network skeleton with several GHPA layers, so that the several GHPA layers and some of the U-shaped network blocks are alternately arranged, includes: The network skeleton is constructed based on six sets of U-shaped network blocks, and four sets of the U-shaped network blocks are replaced with four sets of GHPA layers; The last two U-shaped network blocks in the network skeleton are replaced by two of the GHPA layers, and the other two GHPA layers are interleaved with the other two unreplaced U-shaped network blocks.

4. The bridge defect detection method according to claim 1, characterized in that, The step of introducing an SC convolutional layer in the last layer of the network skeleton includes: Based on the space and channels of the SC convolutional layer, the U-shaped network block of the last layer in the network skeleton is reconstructed, so that the SC convolutional layer replaces the U-shaped network block of the last layer in the network skeleton.

5. The bridge defect detection method according to claim 1, characterized in that, The step of dividing the input mapping channels into several groups based on the GHPA layer in the optimized detection network model, and mapping the input mapping values ​​and several learning tensors onto the GHPA layer according to the several groups of input mapping channels includes: The input mapping execution layer in the GHPA layer is normalized, and the input mapping channel is divided into four groups based on the normalized input mapping execution layer. Based on the four sets of input mapping channels, the input mapping values ​​are grouped and mapped to the GHPA layer to perform feature separation mapping and obtain four sets of feature mappings; Several learning tensors are randomly initialized, and the randomly initialized learning tensors are input to the GHPA layer based on the four sets of input mapping channels.

6. The bridge defect detection method according to claim 5, characterized in that, The calculation expression for the input mapping value grouping mapping is: ； In the formula, , , , These represent the first feature, the second feature, the third feature, and the fourth feature, respectively. This represents the scaling parameter. Indicates the first One characteristic, Represents the characteristic average. Represents the variance of the input features. Represents a constant. Indicates the offset parameter; The expressions for the four sets of feature mappings are as follows: ； In the formula, , , , These represent the first feature, the second feature, the third feature, and the fourth feature, respectively. This represents depthwise separable convolution. Indicates bilinear interpolation. Represents pixel position information, Indicates the node position. This represents the matrix transpose symbol.

7. The bridge defect detection method according to claim 1, characterized in that, The step of fusing the disease information image with the original image in the bridge image dataset and establishing the size relationship between the disease information image and the bridge image dataset includes: Based on the optimal detection network model, the coordinates of the disease information in the disease information image are determined, and the disease information image is fused with the bridge image dataset according to the coordinates; The size relationship between the disease information image and the bridge image dataset is established based on the ratio of the pixel distance between the disease information image and the bridge image dataset.

8. A bridge defect detection system, characterized in that, The system includes: A collection module is used to collect bridge image datasets, prepare a disease dataset based on the bridge image datasets, and process the disease datasets. The replacement module is used to use a number of nested U-shaped network blocks as a network skeleton, and to replace a predetermined portion of the U-shaped network blocks in the network skeleton with a number of GHPA layers, so that the number of GHPA layers and the portion of the U-shaped network blocks are alternately arranged. An introduction module is used to embed DW convolutional layers into the first layer of each group of the U-shaped network blocks and to introduce SC convolutional layers into the last layer of the network skeleton to generate a detection network model. The training module is used to divide the processed disease dataset and train the detection network model based on the training dataset in the divided disease dataset to obtain an optimized detection network model. The mapping module is used to divide the input mapping channels into several groups based on the GHPA layer in the optimized detection network model, and to map the input mapping values ​​and several learning tensors onto the GHPA layer according to the several groups of input mapping channels to obtain the optimal detection network model. The identification module is used to identify the defect information images in the bridge image dataset based on the optimal detection network model, fuse the defect information images with the original images in the bridge image dataset, and establish the size relationship between the defect information images and the bridge image dataset.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the bridge defect detection method as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the bridge defect detection method as described in any one of claims 1 to 7.

Images