A method, apparatus, equipment, medium, and product for semantic segmentation of bridge crack images.

Abstract

This application discloses a method, apparatus, device, medium, and product for semantic segmentation of bridge crack images, relating to the field of computer vision. The method includes acquiring an original bridge image; segmenting the original bridge image using a semantic segmentation model to obtain a semantic segmentation result; the semantic segmentation model is an improved DeepLabv3+ model pre-built based on a training sample set; the improved DeepLabv3+ model, based on the DeepLabv3+ model, fuses multiple low-level features of different depths extracted by the backbone network to obtain low-level fused features, and replaces the convolution in the dilated spatial pyramid pooling module with hybrid dilated convolution, while introducing cross-layer connections at the outputs of different layers of the backbone network and the output of the dilated spatial pyramid pooling module. This application improves the ability to identify small-scale and edge details in crack regions and enhances the accuracy of semantic segmentation.

Description

Technical Field

[0001] This application relates to the field of computer vision, and in particular to a method, apparatus, device, medium, and product for semantic segmentation of bridge crack images. Background Technology

[0002] Cracks in concrete bridges are a significant threat to the structural integrity, safety, and service life of bridges. Timely and effective detection and assessment of crack conditions are crucial for maintaining bridge health. Traditional detection methods typically rely on manual observation and recording, which is time-consuming, labor-intensive, and susceptible to human error, resulting in low accuracy and reliability of the results. Furthermore, close-range observation usually requires the construction of a work platform, which is not only costly and time-consuming but also poses significant safety risks, failing to meet the needs of modern intelligent crack detection for concrete bridges.

[0003] With the advancement of digital image processing technology, more and more researchers are applying image segmentation techniques to the detection of cracks in concrete bridges. Traditional image segmentation methods, such as thresholding, edge detection, and region tracking, while applicable in some simple scenarios, often fail to achieve ideal results when faced with concrete bridge crack images containing noise, complex backgrounds, or uneven lighting, making accurate segmentation of target regions a significant challenge. In recent years, deep learning technology, especially convolutional neural networks, has made breakthrough progress in image segmentation. Semantic segmentation methods based on deep learning can identify target objects in images at the pixel level and effectively separate crack regions from the background. However, current common semantic segmentation models, such as U-Net, HRNet, and PSPNet, while performing well in some tasks, still have some shortcomings in the segmentation of concrete bridge crack images. Specifically, they exhibit weak recognition capabilities for small-scale crack regions, insufficient segmentation accuracy, and inadequate handling of edge details. Therefore, existing deep learning models still face some technical challenges and room for optimization in the intelligent segmentation of concrete bridge cracks. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, device, medium and product for semantic segmentation of bridge crack images. It can use an improved DeepLabv3+ model to perform semantic segmentation on bridge images, improve the ability to identify small-scale and edge details in crack areas and improve the accuracy of semantic segmentation.

[0005] To achieve the above objectives, this application provides the following solution:

[0006] Firstly, this application provides a semantic segmentation method for bridge crack images, including:

[0007] Obtain the original bridge image;

[0008] The original bridge image is segmented using a semantic segmentation model to obtain semantic segmentation results;

[0009] The semantic segmentation model is an improved DeepLabv3+ model pre-built based on a training sample set. The training sample set includes multiple original sample images and the sample segmentation results corresponding to each original sample image. The improved DeepLabv3+ model fuses multiple low-level features of different depths extracted by the backbone network to obtain low-level fused features based on the DeepLabv3+ model. It also replaces the convolution in the dilated spatial pyramid pooling module with hybrid dilated convolution and introduces cross-layer connections at the outputs of different layers of the backbone network and the outputs of the dilated spatial pyramid pooling module.

[0010] Optionally, a semantic segmentation model is used to segment the original bridge image to obtain a semantic segmentation result, specifically including:

[0011] The backbone network extracts features of different depths from the original bridge image, resulting in multiple low-level features and multiple high-level features, wherein the low-level features are shallower than the high-level features.

[0012] Multiple low-level features are fused to obtain low-level fused features;

[0013] By improving the dilated spatial pyramid pooling module, multi-scale fusion processing is performed on the deepest high-level features to obtain multi-scale fused features; in this improved module, the convolution in the dilated spatial pyramid pooling module is replaced with hybrid dilated convolution.

[0014] By performing cross-layer connections between any low-level feature, any high-level feature, and the multi-scale fusion feature, a high-level fusion feature is obtained;

[0015] The semantic segmentation result is obtained based on the low-level fusion features and the high-level fusion features.

[0016] Optionally, multiple low-level features are fused to obtain low-level fused features, specifically including:

[0017] The resolution of each low-level feature is standardized to obtain multiple standardized low-level features;

[0018] The low-level fused features are obtained by performing feature fusion on all standardized low-level features.

[0019] Optionally, by improving the hollow space pyramid pooling module, multi-scale fusion processing is performed on the deepest high-level features to obtain multi-scale fused features, specifically including:

[0020] The deepest high-level features are subjected to 1×1 convolution to obtain the convolution result;

[0021] Global average pooling is performed on the deepest high-level features to obtain the pooling result;

[0022] The deepest high-level features are subjected to hybrid dilated convolution processing to obtain the first result;

[0023] The first result is subjected to hybrid dilated convolution to obtain the second result;

[0024] The second result is subjected to hybrid dilated convolution to obtain the third result;

[0025] The convolution result, the pooling result, the first result, the second result, and the third result are fused to obtain the multi-scale fused feature.

[0026] Optionally, any low-level feature, any high-level feature, and the multi-scale fusion feature can be connected across layers to obtain a high-level fusion feature, specifically including:

[0027] By fusing any high-level feature with the multi-scale fusion feature, a first cross-layer feature is obtained;

[0028] The first cross-layer feature is sequentially subjected to 1×1 convolution and 2x upsampling to obtain the first feature;

[0029] By fusing any low-level feature with the first feature, a second cross-layer feature is obtained;

[0030] The second cross-layer feature is sequentially subjected to 1×1 convolution and 2x upsampling to obtain the advanced fusion feature.

[0031] Optionally, the backbone network is a MobileNet network, an Xception network, a ResNet network, or a Transformer structure.

[0032] Secondly, this application provides a semantic segmentation apparatus for bridge crack images, comprising:

[0033] The image acquisition module is used to acquire raw bridge images;

[0034] The semantic segmentation module is used to segment the original bridge image using a semantic segmentation model to obtain semantic segmentation results. The semantic segmentation model is an improved DeepLabv3+ model pre-built based on a training sample set. The training sample set includes multiple original sample images and the sample segmentation results corresponding to each original sample image. The improved DeepLabv3+ model, based on the DeepLabv3+ model, fuses multiple low-level features of different depths extracted by the backbone network to obtain low-level fused features, and replaces the convolution in the dilated spatial pyramid pooling module with hybrid dilated convolution. At the same time, cross-layer connections are introduced at the outputs of different layers of the backbone network and the outputs of the dilated spatial pyramid pooling module.

[0035] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described semantic segmentation method for bridge crack images.

[0036] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described semantic segmentation method for bridge crack images.

[0037] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described semantic segmentation method for bridge crack images.

[0038] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0039] This application provides a method, apparatus, device, medium, and product for semantic segmentation of bridge crack images. It obtains low-level fused features by fusing multiple low-level features of different depths extracted from the backbone network, preserving small-scale information and reducing redundant feature information. By replacing the convolution in the dilated spatial pyramid pooling module with hybrid dilated convolution, it can extract local information more accurately while maintaining a large receptive field, reducing the loss of smaller targets or edges and solving the problem of dilated convolution becoming gridded. Furthermore, by introducing cross-layer connections at the outputs of different layers of the backbone network and the output of the dilated spatial pyramid pooling module, it effectively transmits information from each layer, enhances the model's localization ability, and thus improves the ability to recognize small-scale and edge details in crack areas and the accuracy of semantic segmentation. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is an application environment diagram of a semantic segmentation method for bridge crack images according to an embodiment of this application;

[0042] Figure 2 A flowchart illustrating a semantic segmentation method for bridge crack images provided in an embodiment of this application;

[0043] Figure 3 for Figure 1 A detailed flowchart of step 202;

[0044] Figure 4 A schematic diagram of the multi-scale feature fusion module of the backbone network of the improved DeepLabv3+ model provided in an embodiment of this application;

[0045] Figure 5 A schematic diagram of a hybrid dilated convolution module of an improved DeepLabv3+ model provided in an embodiment of this application;

[0046] Figure 6 A schematic diagram of a void space pyramid pooling module for an improved DeepLabv3+ model provided in an embodiment of this application;

[0047] Figure 7 A schematic diagram illustrating the cross-layer connection method of the improved DeepLabv3+ model provided in an embodiment of this application;

[0048] Figure 8 A schematic diagram of the overall framework of the improved DeepLabv3+ model provided in an embodiment of this application;

[0049] Figure 9 This is a schematic diagram of the functional modules of a semantic segmentation device for bridge crack images provided in an embodiment of this application;

[0050] Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0051] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0052] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0053] The semantic segmentation method for bridge crack images provided in this application can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on another server. Terminal 102 can send the original bridge image to server 104. After receiving the original bridge image, server 104 uses a semantic segmentation model to segment the original bridge image, obtaining the semantic segmentation result. Server 104 can then feed back the obtained semantic segmentation result to terminal 102. Furthermore, in some embodiments, the bridge crack image semantic segmentation method can also be implemented independently by server 104 or terminal 102. For example, terminal 102 can directly perform semantic segmentation processing on the original bridge image, or server 104 can obtain the original bridge image from the data storage system and perform semantic segmentation processing on the original bridge image.

[0054] The terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster composed of multiple servers, or it can be a cloud server.

[0055] In one exemplary embodiment, such as Figure 2 As shown, a semantic segmentation method for bridge crack images is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps 201 and 202. Wherein:

[0056] Step 201: Obtain the original bridge image.

[0057] Step 202: Use a semantic segmentation model to segment the original bridge image to obtain the semantic segmentation result.

[0058] The semantic segmentation model described above is an improved DeepLabv3+ model pre-built based on a training sample set. The training sample set includes multiple original sample images and the corresponding segmentation results for each original sample image. The improved DeepLabv3+ model, based on the DeepLabv3+ model, fuses multiple low-level features of different depths extracted by the backbone network to obtain low-level fused features, replaces the convolutions in the dilated spatial pyramid pooling module with hybrid dilated convolutions, and introduces cross-layer connections at the outputs of different layers of the backbone network and the outputs of the dilated spatial pyramid pooling module.

[0059] In one exemplary embodiment, such as Figure 3 As shown, step 202 above can be replaced by steps 301 to 305:

[0060] Step 301: Extract features of different depths from the original bridge image through the backbone network to obtain multiple low-level features and multiple high-level features. The low-level features are shallower than the high-level features.

[0061] Step 302: Fuse multiple low-level features to obtain low-level fused features.

[0062] Step 303: The deepest high-level features are fused at multiple scales using an improved dilated spatial pyramid pooling module to obtain multi-scale fused features. Specifically, the improved dilated spatial pyramid pooling module replaces the convolutions in the dilated spatial pyramid pooling module with hybrid dilated convolutions.

[0063] Step 304: Connect any low-level feature, any high-level feature, and multi-scale fusion feature across layers to obtain high-level fusion features.

[0064] Step 305: Obtain the semantic segmentation result based on the low-level fusion features and the high-level fusion features.

[0065] In an exemplary embodiment, step 302 described above can be replaced by the following steps:

[0066] Step 3021: Standardize the resolution of each low-level feature to obtain multiple standardized low-level features.

[0067] Step 3022: Perform feature fusion on all standardized low-level features to obtain low-level fused features.

[0068] In one exemplary embodiment, such as Figure 4As shown, this application designs a multi-scale feature fusion module in the backbone network based on MobileNetV2, introduces two low-level features with different hopping layers, and then fuses them with the original low-level features to obtain low-level fused features, so as to retain small-scale information and reduce redundant feature information.

[0069] In this embodiment, as Figure 4 As shown, three low-level features of different depths (0, 1, and 2) were extracted from the MobileNetV2 backbone network, with dimensions (1, 24, 128, 128), (1, 32, 64, 64), and (1, 96, 32, 32), respectively. First, the depths of the three low-level features were adjusted to 48, 64, and 96 respectively using 1×1 convolutions. Then, low-level feature 1 and low-level feature 2 were upsampled by a factor of 2 and 4 respectively to unify their size. Finally, the three features were stacked and merged to obtain the low-level fused feature, which represents the shallow semantic information of the network. The expression for the final low-level fused feature is as follows:

[0070] F shallow =Concat(Up) ×1 (f0),Up ×2 (f1),Up ×4 (f2)).

[0071] Among them, F shallow Indicates low-level fusion features; Up ×1 (f0) indicates that after performing a 1×1 convolution on the feature f0 of layer 0 to adjust the number of channels, a 1x upsampling process is applied, which does not change its spatial resolution; Up ×2 (f1) indicates that after performing a 1×1 convolution on the first-layer feature f1 to adjust the number of channels, a 2x upsampling process is applied; Up ×4 (f2) indicates that after performing a 1×1 convolution on the second layer feature f2 to adjust the number of channels, a 4x upsampling process is performed; Concat(·) indicates a concatenation operation on the channel dimension.

[0072] Upsampling operations of different factors can unify the corresponding features to the same spatial size as the feature f0 of layer 0. The final low-level fused feature F shallow Compared to the "single low-level feature" in the traditional DeepLabv3+ model, it contains more multi-scale detail information, effectively avoiding excessive loss of small-scale crack features during the layer-by-layer feature extraction process.

[0073] In an exemplary embodiment, step 303 described above can be replaced by the following steps:

[0074] Step 3031: Perform a 1×1 convolution on the deepest high-level features to obtain the convolution result.

[0075] Step 3032: Perform global average pooling on the deepest high-level features to obtain the pooling result.

[0076] Step 3033: Perform hybrid dilated convolution on the deepest high-level features to obtain the first result.

[0077] Step 3034: Perform hybrid dilated convolution on the first result to obtain the second result.

[0078] Step 3035: Perform hybrid dilated convolution on the second result to obtain the third result.

[0079] Step 3036: Perform feature fusion on the above convolution results, pooling results, first results, second results and third results to obtain multi-scale fused features.

[0080] To overcome the limitations of traditional dilated convolution in image segmentation and improve the recognition of small targets and edge details, Mixed Dilated Convolution (HDC) can effectively address this issue by cascading dilated convolutions. HDC uses a series of dilated convolutions with different dilation rates, ensuring that the final receptive field completely covers a square region without any holes or missing edges. Therefore, HDC extracts local information more accurately.

[0081] The original Deeplabv3+ Atrous Spatial Pyramid Pooling (ASPP) module employs three 3×3 dilated convolutions with dilation rates of 6%, 12%, and 18%. The ASPP module can extract information at different scales while maintaining a large receptive field. This application further introduces hybrid dilated convolutions into the ASPP module, which can extract local information more accurately while maintaining a large receptive field, reducing the loss of smaller targets or edges, and solving the problem of gridding in dilated convolutions.

[0082] In one exemplary embodiment, such as Figure 5 As shown, the HDC module of this application consists of three concatenated convolutional layers with dilation rates of 1, 2, and 3, respectively, replacing the original model's convolutions for feature extraction, thereby improving the model's feature extraction capability. Inside the HDC module, the receptive field of the dilated convolution is calculated using the following formula:

[0083] RF i+1 =RF i +(k-1)×S i .

[0084] Among them, RF i+1 Represents the receptive field size of the (i+1)th dilated convolution layer; RF iS represents the receptive field size of the i-th dilated convolution layer; κ represents the convolution kernel; S i It represents the product of all step sizes in the first i layers.

[0085] Specifically, the ASPP module of the original DeepLabv3+ model typically contains five parallel output branches: a 1×1 convolution branch with a dilation rate of 1, a 3×3 convolution branch with a dilation rate of 6, a 3×3 convolution branch with a dilation rate of 12, a 3×3 convolution branch with a dilation rate of 18, and an image pooling branch.

[0086] In the improved DeepLabv3+ model of this application, such as Figure 6 As shown, three HDC modules replace the three 3×3 convolution branches with dilation rates of 6, 12, and 18 in the original ASPP module, while keeping the other two branches unchanged. The purpose of this is to further address the "gridification" problem of traditional dilated convolutions while preserving multi-scale contextual information.

[0087] In this embodiment, as Figure 6 As shown, the deepest high-level features from the output of the MobileNetV2 backbone are first fed into three parallel branches simultaneously: a 1×1 convolution branch, the first layer of a three-layer cascaded HDC module, and an image pooling branch.

[0088] The HDC module in this application includes three convolutional branches with different dilation rates. Unlike traditional simple concatenated dilated convolutions, the three HDC branches possess both sequential concatenation and the ability to independently output feature maps. Specifically, as follows... Figure 6 As shown, the deepest high-level features output from the end of the MobileNetV2 backbone are first passed to the first layer of the three-layer cascaded HDC module. The first layer of the HDC module processes the input features sequentially using 3×3 convolutions with a dilation rate of 1, 2, and 3, yielding the first result. The second layer of the HDC module processes the first result sequentially using convolutions with dilation rates of 1, 2, and 3, yielding the second result. The third layer of the HDC module processes the second result sequentially using convolutions with dilation rates of 1, 2, and 3, yielding the third result. These three layers not only output separately, but the output of the previous layer is also passed to the next layer as input. This design achieves a balance between local details and global context. Compared to traditional methods with only single convolution outputs, the HDC module in this application fully exploits the diversity of feature representation both horizontally (parallel output of three layers) and vertically (multiple dilated convolutions cascaded within each layer).

[0089] To obtain more global contextual information, the image pooling branch performs a global average pooling on the deepest high-level features output from the end of the MobileNetV2 backbone network, and then restores the pooling result to the same spatial size as the outputs of other branches through upsampling or interpolation. This branch provides contextual information about the overall scene, which helps in the identification of large-scale targets and enhances the expressive power of overall features.

[0090] After each branch of the improved ASPP module completes its feature extraction, these feature extraction results are concatenated along the channel dimension to obtain a multi-scale, multi-hole rate fusion result. Subsequently, a 1×1 convolution is used to integrate or compress the number of channels, outputting the final multi-scale fusion feature. This multi-scale fusion feature carries richer local and global information, and is particularly advantageous in small-scale crack detection and complex edge resolution.

[0091] The improved spatial pyramid pooling module obtained through the above methods, while maintaining a large receptive field, fully captures local information near small cracks and complex edges by using cascaded convolution with multi-scale porosity, thus enabling this application to have better refined recognition and robustness in semantic segmentation of bridge crack images.

[0092] In an exemplary embodiment, step 304 above can be replaced by the following steps:

[0093] Step 3041: Perform feature fusion between any high-level feature and multi-scale fusion feature to obtain the first cross-layer feature.

[0094] Step 3042: Perform 1×1 convolution and 2x upsampling on the first cross-layer feature to obtain the first feature.

[0095] Step 3043: Perform feature fusion between any low-level feature and the first feature to obtain the second cross-layer feature.

[0096] Step 3044: Perform 1×1 convolution and 2x upsampling on the second cross-layer feature to obtain the high-level fusion feature.

[0097] In one exemplary embodiment, such as Figure 7As shown, this application designs a fine-grained upsampling strategy in conjunction with cross-layer connections. First, any high-level feature output from the MobileNetV2 backbone network is fused with a multi-scale fusion feature to obtain the first cross-layer feature, which introduces higher-level semantic information. The first cross-layer feature is then subjected to a 1×1 convolution and a 2x upsampling process to obtain the first feature, a process that recovers more details. Next, any low-level feature output from the MobileNetV2 backbone network is fused with the first feature to obtain the second cross-layer feature. The second cross-layer feature is then subjected to a 1×1 convolution and a 2x upsampling process to obtain the high-level fusion feature. This process recovers even more detailed information. The introduction of cross-layer connections effectively transmits information from each layer, enhances the model's localization ability, and avoids the information loss problem that may be caused by direct large-scale upsampling. Replacing the one-time 4x upsampling in the original model with continuous upsampling steps, this staged upsampling method not only improves pixel continuity but also makes the recovered boundary information more accurate and closer to the original labeled image.

[0098] In an exemplary embodiment, after obtaining the low-level and high-level fusion features, feature fusion processing is performed. First, the low-level and high-level fusion features are matched in terms of the number of channels using a 1×1 convolution. Then, a cascaded fusion operation is used to obtain the fused features. The fused features are obtained using the following formula:

[0099] F fused =Contact((F shallow ),(F deep )).

[0100] Among them, F fused Indicates fusion features; F shallow Indicates low-level fusion features; F deep This indicates advanced fusion features; Concat(·) indicates a concatenation operation along the channel dimension.

[0101] Finally, the fused feature F will be... fused The image is then fed into a subsequent decoding network (fine upsampling module) to gradually restore it to the same or similar spatial resolution as the input image.

[0102] Through the above steps, shallow semantic information not only provides more accurate detail support for the segmentation of edges and small-scale cracks during the decoding stage, but also, in conjunction with deep semantic information, can effectively achieve comprehensive identification of crack regions in the entire image. Compared with traditional single-scale methods, this multi-scale feature fusion strategy can significantly improve the accuracy and robustness of crack segmentation in concrete bridges.

[0103] In an exemplary embodiment, the overall framework of the improved DeepLabv3+ model provided in this application is as follows: Figure 8 As shown.

[0104] The MobileNetV2 backbone network consists of multiple layers of inverted residual blocks, extracting features from the original bridge image layer by layer. Its advantage lies in achieving high feature representation with relatively few parameters and computational cost. (Original bridge image) After certain preprocessing steps (such as normalization and scaling), the data is input into the MobileNetV2 backbone network. Three low-level features f0, f1, and f2 of different depths extracted from the backbone network are then input into the multi-scale feature fusion module. The spatial resolutions of the three low-level features are as follows: and Here, H and W represent the length and width of the original bridge image X, respectively. The high-level features extracted from the backbone network will be further fed into subsequent spatially dilated convolution and decoder modules.

[0105] The multi-scale feature fusion module includes: three 1×1 convolutional kernels, used to adjust the number of channels of low-level features without changing the spatial resolution; 2x upsampling and 4x upsampling operations, used to upsample low-level features f1 and f2 to the same resolution as low-level feature f0, respectively; and channel concatenation (Concat) operation, used to fuse the three low-level features to obtain low-level fused features.

[0106] The improved dilated spatial pyramid pooling module replaces the original three 3×3 convolutional branches with dilation rates of 6, 12, and 18 with three HDC branches. Each HDC branch consists of concatenated convolutions with dilation rates of 1, 2, and 3, and outputs a feature map independently. The deepest high-level features output from the end of the MobileNetV2 backbone are processed by the improved dilated spatial pyramid pooling module to obtain multi-scale fused features. The multi-scale fused features can be expressed by the following formula:

[0107] F HDC =Conv 1×1 (Concat(F A ,F B ,F C ,F D ,F E )).

[0108] Among them, F HDC F represents multi-scale fusion features; A F represents the convolution result obtained after 1×1 convolution; B F represents the pooling result obtained after global average pooling; C F D F EThese represent the first, second, and third results obtained after processing through three HDC branches, respectively; Concat(·) represents the concatenation operation along the channel dimension; Conv 1×1 (·) represents a 1×1 convolution operation. By compressing or mapping along the channel dimension using 1×1 convolution, more compact multi-scale fused features can be obtained.

[0109] The three HDC branches internally consist of a multi-layer dilated convolutional structure (with dilation rates of 1, 2, and 3 in sequence), and externally, they are concatenated in parallel with the other two branches at the output. Therefore, the resulting multi-scale fused features carry local details and global contextual information with different dilation rates, laying a solid foundation for subsequent fine upsampling combined with cross-layer connections.

[0110] This application employs fine-grained upsampling combined with cross-layer connections, replacing the single 4x upsampling in the original DeepLabv3+ model with two 2x upsampling operations. The first upsampling operation combines any high-level feature output from the backbone network with the multi-scale fused feature F. HDC Feature fusion is performed, followed by 1×1 convolution and 2x upsampling to obtain a more refined first feature F. up1 The second step involves fusing any low-level feature output from the backbone network with the first feature. After fusion, a 1×1 convolution and a 2x upsampling process are performed sequentially to obtain a more refined high-level fused feature F. up2 Staged upsampling effectively avoids the loss of detail caused by large-magnification interpolation at once, resulting in a more accurate restoration of the crack edge area.

[0111] Based on the high-level fusion feature F obtained after fine upsampling and cross-layer connection processing. up2 The low-level fused feature F output by the multi-scale feature fusion module shallow This process effectively addresses the shortcomings of traditional methods in areas such as loss of small-scale crack information, edge faults, and noise interference, while balancing high detection accuracy with computational efficiency. It provides a superior technical solution for the intelligent diagnosis and safety assessment of cracks in concrete bridges.

[0112] In one exemplary embodiment, the backbone network of the improved DeepLabv3+ model is a MobileNet network, an Xception network, a ResNet network, or a Transformer structure.

[0113] Based on the same inventive concept, this application also provides a bridge crack image semantic segmentation apparatus for implementing the bridge crack image semantic segmentation method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations of one or more bridge crack image semantic segmentation apparatus embodiments provided below can be found in the limitations of the bridge crack image semantic segmentation method described above, and will not be repeated here.

[0114] In one exemplary embodiment, such as Figure 9 As shown, a semantic segmentation device for bridge crack images is provided, comprising an image acquisition module 901 and a semantic segmentation module 902. The image acquisition module 901 is used to acquire original bridge images. The semantic segmentation module 902 is used to segment the original bridge images using a semantic segmentation model to obtain semantic segmentation results. The aforementioned semantic segmentation model is an improved DeepLabv3+ model pre-built based on a training sample set. The training sample set includes multiple original sample images and the corresponding sample segmentation results for each original sample image. The improved DeepLabv3+ model, based on the DeepLabv3+ model, fuses multiple low-level features of different depths extracted by the backbone network to obtain low-level fused features, replaces the convolution in the dilated spatial pyramid pooling module with hybrid dilated convolution, and introduces cross-layer connections at the outputs of different layers of the backbone network and the output of the dilated spatial pyramid pooling module.

[0115] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 10 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores raw bridge images. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a semantic segmentation method for bridge crack images.

[0116] Those skilled in the art will understand that Figure 10The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0117] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0118] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0119] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0120] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0121] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0122] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0123] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0124] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A semantic segmentation method for bridge crack images, characterized in that, The semantic segmentation method for bridge crack images includes: Obtain the original bridge image; The original bridge image is segmented using a semantic segmentation model to obtain semantic segmentation results, specifically including: The backbone network extracts features of different depths from the original bridge image, resulting in multiple low-level features and multiple high-level features, wherein the low-level features are shallower than the high-level features. Multiple low-level features are fused to obtain low-level fused features; By improving the dilated spatial pyramid pooling module, multi-scale fusion processing is performed on the deepest high-level features to obtain multi-scale fused features; in this improved module, the convolution in the dilated spatial pyramid pooling module is replaced with hybrid dilated convolution. By performing cross-layer connections between any low-level feature, any high-level feature, and the multi-scale fusion feature, a high-level fusion feature is obtained; The semantic segmentation result is obtained based on the low-level fusion features and the high-level fusion features; The semantic segmentation model is an improved DeepLabv3+ model pre-built based on a training sample set. The training sample set includes multiple original sample images and the sample segmentation results corresponding to each original sample image. The improved DeepLabv3+ model fuses multiple low-level features of different depths extracted by the backbone network to obtain low-level fused features based on the DeepLabv3+ model. It also replaces the convolution in the dilated spatial pyramid pooling module with hybrid dilated convolution and introduces cross-layer connections at the outputs of different layers of the backbone network and the outputs of the dilated spatial pyramid pooling module. By improving the hollow space pyramid pooling module, multi-scale fusion processing is performed on the deepest high-level features to obtain multi-scale fused features, specifically including: The deepest high-level features are subjected to 1×1 convolution to obtain the convolution result; Global average pooling is performed on the deepest high-level features to obtain the pooling result; The deepest high-level features are subjected to hybrid dilated convolution processing to obtain the first result; The first result is subjected to hybrid dilated convolution to obtain the second result; The second result is subjected to hybrid dilated convolution to obtain the third result; The convolution result, the pooling result, the first result, the second result, and the third result are fused to obtain the multi-scale fused feature. By performing cross-layer concatenation between any low-level feature, any high-level feature, and the multi-scale fusion feature, a high-level fusion feature is obtained, specifically including: By fusing any high-level feature with the multi-scale fusion feature, a first cross-layer feature is obtained; The first cross-layer feature is sequentially subjected to 1×1 convolution and 2x upsampling to obtain the first feature; By fusing any low-level feature with the first feature, a second cross-layer feature is obtained; The second cross-layer feature is sequentially subjected to 1×1 convolution and 2x upsampling to obtain the advanced fusion feature.

2. The semantic segmentation method for bridge crack images according to claim 1, characterized in that, Multiple low-level features are fused to obtain low-level fused features, specifically including: The resolution of each low-level feature is standardized to obtain multiple standardized low-level features; The low-level fused features are obtained by performing feature fusion on all standardized low-level features.

3. The semantic segmentation method for bridge crack images according to claim 1, characterized in that, The backbone network is a MobileNet network, Xception network, ResNet network, or Transformer structure.

4. A semantic segmentation device for bridge crack images, characterized in that, The bridge crack image semantic segmentation device includes: The image acquisition module is used to acquire raw bridge images; The semantic segmentation module is used to segment the original bridge image using a semantic segmentation model to obtain semantic segmentation results, specifically including: The backbone network extracts features of different depths from the original bridge image, resulting in multiple low-level features and multiple high-level features, wherein the low-level features are shallower than the high-level features. Multiple low-level features are fused to obtain low-level fused features; By improving the dilated spatial pyramid pooling module, multi-scale fusion processing is performed on the deepest high-level features to obtain multi-scale fused features; in this improved module, the convolution in the dilated spatial pyramid pooling module is replaced with hybrid dilated convolution. By performing cross-layer connections between any low-level feature, any high-level feature, and the multi-scale fusion feature, a high-level fusion feature is obtained; The semantic segmentation result is obtained based on the low-level fusion features and the high-level fusion features; the semantic segmentation model is an improved DeepLabv3+ model pre-built based on the training sample set; the training sample set includes multiple original sample images and the sample segmentation result corresponding to each original sample image; the improved DeepLabv3+ model fuses multiple low-level features of different depths extracted by the backbone network to obtain low-level fusion features based on the DeepLabv3+ model, and replaces the convolution in the dilated spatial pyramid pooling module with hybrid dilated convolution, while introducing cross-layer connections at the outputs of different layers of the backbone network and the outputs of the dilated spatial pyramid pooling module; By improving the hollow space pyramid pooling module, multi-scale fusion processing is performed on the deepest high-level features to obtain multi-scale fused features, specifically including: The deepest high-level features are subjected to 1×1 convolution to obtain the convolution result; Global average pooling is performed on the deepest high-level features to obtain the pooling result; The deepest high-level features are subjected to hybrid dilated convolution processing to obtain the first result; The first result is subjected to hybrid dilated convolution to obtain the second result; The second result is subjected to hybrid dilated convolution to obtain the third result; The convolution result, the pooling result, the first result, the second result, and the third result are fused to obtain the multi-scale fused feature. By performing cross-layer concatenation between any low-level feature, any high-level feature, and the multi-scale fusion feature, a high-level fusion feature is obtained, specifically including: By fusing any high-level feature with the multi-scale fusion feature, a first cross-layer feature is obtained; The first cross-layer feature is sequentially subjected to 1×1 convolution and 2x upsampling to obtain the first feature; By fusing any low-level feature with the first feature, a second cross-layer feature is obtained; The second cross-layer feature is sequentially subjected to 1×1 convolution and 2x upsampling to obtain the advanced fusion feature.

5. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the semantic segmentation method for bridge crack images according to any one of claims 1-3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the semantic segmentation method for bridge crack images as described in any one of claims 1-3.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the semantic segmentation method for bridge crack images as described in any one of claims 1-3.

Images