Method and system for semantic segmentation of elongated cracks based on hybrid architecture support

Abstract

The application discloses an elongated crack semantic segmentation method and system based on a hybrid architecture support. The method comprises the following steps: acquiring crack image data and performing preprocessing; inputting the preprocessed crack image into a hybrid encoder, wherein the hybrid encoder comprises a convolution branch and a Transformer branch, convolution features are output through the convolution branch, and conversion features are output through the Transformer branch; inputting the aligned convolution features and the conversion features into a frequency domain-spatial domain dual gating fusion module to obtain a fusion feature map; and inputting the fusion feature map into a decoder to obtain an elongated crack semantic segmentation result. The application can reduce the calculation cost and solve the problem of discontinuity of the elongated crack segmentation result.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, system, storage medium, and electronic device for semantic segmentation of narrow slits based on a hybrid architecture. Background Technology

[0002] Cracks are a crucial indicator of structural health in transportation infrastructure (such as bridges, tunnels, and road surfaces) and as buildings age. Even small cracks, if not detected and repaired in time, can develop into serious structural failures, threatening public safety. Traditional crack detection relies primarily on manual visual inspection, which is not only inefficient and costly but also heavily influenced by the subjective experience of the inspectors, posing a risk of missed cracks.

[0003] With the development of computer vision technology, deep learning-based automatic crack detection technology has gradually become mainstream. Current crack segmentation methods are mainly divided into two categories: methods based on convolutional neural networks (CNN) and methods based on Transformers.

[0004] CNN methods (such as U-Net, DeepLabV3+, etc.) excel at extracting local features (such as crack edges and textures), but are limited by the local receptive field of the convolutional kernel, making it difficult to capture long-range dependencies in the image. When dealing with long, thin cracks that extend over long distances, they are prone to segmentation breaks and poor continuity.

[0005] Transformer methods (such as SegFormer and Swin-UNet) utilize self-attention mechanisms to establish global contextual relationships, helping to maintain the overall topological structure of cracks. However, Transformer models typically have a huge number of parameters, demanding extremely high computational resources and requiring massive amounts of training data to converge. In practical engineering scenarios, high-quality pixel-level crack annotation data is often scarce, and the computing power of detection devices (such as drones and handheld devices) is limited, making it difficult to directly deploy large-scale Transformer models.

[0006] While hybrid architectures combining CNNs and Transformers have recently emerged, attempting to integrate the advantages of both, existing hybrid models typically require full parameter fine-tuning of the entire network, resulting in high training costs and a tendency to overfit on small-sample crack datasets. Furthermore, for extremely elongated topologies like cracks, conventional cross-entropy loss functions often focus on pixel-level accuracy while ignoring the continuity of the skeleton, leading to detected cracks appearing as "dashed lines." Summary of the Invention

[0007] This invention provides a semantic segmentation method, system, storage medium, and electronic device for thin and long cracks based on a hybrid architecture, which can reduce computational costs and solve the problem of discontinuous segmentation results for thin and long cracks.

[0008] This invention provides a semantic segmentation method for thin, elongated cracks based on a hybrid architecture, comprising:

[0009] Acquire crack image data and perform preprocessing;

[0010] The preprocessed crack image is input into a hybrid encoder, which includes a convolutional branch and a Transformer branch. The convolutional branch outputs convolutional features, and the Transformer branch outputs transformed features.

[0011] The aligned convolutional features and transformed features are input into the frequency domain-spatial domain dual-gated fusion module to obtain the fused feature map;

[0012] The fused feature map is input into the decoder to obtain the semantic segmentation result of the narrow crack.

[0013] Furthermore, according to the above-mentioned semantic segmentation method for thin slits supported by a hybrid architecture, the convolutional branch is an improved ResNet network, and the convolutional branch includes sequentially connected convolutional and pooling layers, two central difference convolutional layers, and two Res convolutional layers.

[0014] The Transformer branch includes four coding layers, and the output of each coding layer is connected in series with a lightweight dual-axis strip calibration module.

[0015] The number of frequency-spatial dual-gated fusion modules is the same as the number of coding layers in the Transformer branch.

[0016] Furthermore, according to the above-mentioned semantic segmentation method for elongated cracks supported by a hybrid architecture, the processing procedure of the central differential convolutional layer is represented by the following formula:

[0017]

[0018] in, The output features of the central difference convolutional layer, For learnable weights, The input feature map is the center difference convolutional layer. The center pixel coordinates, To indicate the sensory field area Neighboring pixel coordinates, For the sensory field area.

[0019] Furthermore, according to the above-mentioned semantic segmentation method for elongated cracks supported by a hybrid architecture, the lightweight biaxial strip calibration module's processing procedure includes:

[0020] Asymmetric large receptive field convolution kernels are used to extract linear features in orthogonal directions, and horizontal and vertical masks are calculated.

[0021] The input features are corrected based on the horizontal and vertical masks to obtain the corrected transformed features.

[0022] Furthermore, according to the above-mentioned semantic segmentation method for elongated cracks supported by a hybrid architecture, the horizontal and vertical masks are calculated using the following formula:

[0023]

[0024]

[0025] in, For horizontal masks, For vertical masks, It is the Sigmoid activation function. For average pooling operation, Input characteristics for a lightweight biaxial strip calibration module. and For the size of the large sensory field;

[0026] The process of correcting input features to obtain corrected transformed features is represented by the following formula:

[0027]

[0028] in, For Hadama accumulation, These are the corrected transformation features.

[0029] Furthermore, according to the above-mentioned semantic segmentation method for elongated cracks supported by a hybrid architecture, the frequency-spatial dual-gated fusion module includes parallel spatial and frequency domain paths; the processing procedure of the frequency-spatial dual-gated fusion module includes:

[0030] The spatial reliability map is calculated using channel attention and pixel absolute difference.

[0031] The frequency domain path obtains the amplitude spectrum of the convolutional features and the amplitude spectrum of the transformed features by performing a two-dimensional fast Fourier transform on the aligned convolutional features and transformed features;

[0032] Calculate the high-frequency component differences between the convolution feature amplitude spectrum and the transformation feature amplitude spectrum, and calculate a high-frequency difference map based on the high-frequency component differences;

[0033] The high-frequency difference map is subjected to Fourier reconstruction, and a frequency domain saliency mask is obtained through convolution operation and activation function;

[0034] The fusion weights are calculated based on the frequency domain saliency mask and the spatial reliability map. The aligned convolutional features and transformed features are then fused based on the fusion weights to obtain a fusion feature map.

[0035] Furthermore, according to the above-mentioned semantic segmentation method for thin, elongated cracks supported by a hybrid architecture, the method is applied to the HECS-Net model, which is trained based on a total loss function, wherein the total loss function is:

[0036]

[0037]

[0038]

[0039] in, For the total loss function, For classification loss function, For weight parameters, Let be the gradient direction consistency loss function. It is a true positive. It was a false negative. The Tversky index. All are parameters, where P is the predicted semantic segmentation result and G is the corresponding ground truth graph. and Let be the gradients of the Sobel operator in the x and y directions. This is a very small constant smoothing term used to prevent the denominator from being zero, ensuring the stability of numerical calculations. N This represents the total number of pixels involved in the calculation, used to normalize the summation result.

[0040] This invention also provides a semantic segmentation system for thin, elongated cracks based on a hybrid architecture, comprising:

[0041] The acquisition and preprocessing module is used to acquire crack image data and perform preprocessing.

[0042] The encoding module is used to input the preprocessed crack image into the hybrid encoder, which includes a convolution branch and a Transformer branch. The convolution branch outputs convolutional features, and the Transformer branch outputs transformation features.

[0043] The fusion module is used to input the aligned convolutional features and transformed features into the frequency domain-spatial domain dual-gated fusion module to obtain the fused feature map;

[0044] The decoding module is used to input the fused feature map into the decoder to obtain the semantic segmentation result of the narrow crack.

[0045] The present invention also provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute any of the above-described thin-slit semantic segmentation methods based on a hybrid architecture.

[0046] The present invention also provides an electronic device including a processor and a memory, the processor being electrically connected to the memory, the memory being used to store instructions and data, and the processor being used in the steps of the elongated slit semantic segmentation method based on hybrid architecture support described in any of the preceding claims.

[0047] This invention provides a method, system, storage medium, and electronic device for semantic segmentation of long, thin cracks based on a hybrid architecture. The invention processes crack images using a hybrid encoder, a frequency-spatial dual-gated fusion module, and a decoder to obtain predicted semantic segmentation results. This invention utilizes a frozen pre-trained Transformer network in conjunction with a lightweight dual-axis strip calibration module (DASC) and parallel CNN branches to achieve efficient and high-precision crack segmentation. Attached Figure Description

[0048] The technical solution and other beneficial effects of the present invention will become apparent from the following detailed description of specific embodiments of the invention, in conjunction with the accompanying drawings.

[0049] Figure 1 A flowchart of a semantic segmentation method for thin cracks based on a hybrid architecture provided in an embodiment of the present invention.

[0050] Figure 2 This is a schematic diagram of the HECS-Ne model provided in an embodiment of the present invention.

[0051] Figure 3 The image shows the segmentation result provided in an embodiment of the present invention.

[0052] Figure 4 This is a schematic diagram of the structure of a thin-slit semantic segmentation system based on a hybrid architecture provided in an embodiment of the present invention.

[0053] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] This invention provides a method, system, storage medium, and electronic device for semantic segmentation of thin, elongated cracks based on a hybrid architecture. The thin, elongated crack semantic segmentation system provided by this invention can be integrated into an electronic device, such as a terminal or server. The terminal can include tablet computers, laptops, personal computers (PCs), microprocessor boxes, or other devices.

[0056] Please see Figure 1 , Figure 1 The flowchart illustrates a semantic segmentation method for elongated cracks based on a hybrid architecture, provided in an embodiment of the present invention. This method, applied in an electronic device, includes the following steps:

[0057] S1: Acquire crack image data and perform preprocessing.

[0058] Image acquisition was conducted using high-resolution industrial cameras (at least 4K resolution), drones equipped with gimbal cameras, and handheld smart terminal devices. Drones were primarily used to capture images of cracks in hard-to-reach areas such as the base of bridges and the exterior walls of high-rise buildings, while handheld devices were used to capture close-up details of minute cracks. Sliding window cropping and data augmentation were then applied to the crack images.

[0059] S2, the preprocessed crack image is input into the hybrid encoder, which includes a convolution branch and a Transformer branch. The convolution branch outputs convolutional features, and the Transformer branch outputs transformed features.

[0060] Steps S2-S4 can be applied to the HECS-Net model (Hybrid Encoder Crack SegmentationNet, a semantic segmentation network for thin cracks based on an anisotropic feature calibration hybrid architecture). Figure 2This is a schematic diagram of the structure of the HECS-Ne model provided in an embodiment of the present invention, as shown below. Figure 2 As shown, the HECS-Ne model includes a hybrid encoder, a frequency-spatial dual-gated fusion module, and a decoder.

[0061] The convolutional branch is an improved ResNet network, consisting of sequentially connected convolutional and pooling layers, two central difference convolutional layers, and two Res convolutional layers. The Transformer branch includes four coding layers, each with a lightweight dual-axis stripe calibration module cascaded at its output. The number of frequency-spatial dual-gated fusion modules is the same as the number of coding layers in the Transformer branch.

[0062] Specifically, to address the problem that traditional convolutions struggle to distinguish crack edges from noise on rough concrete surfaces, the convolution branch includes a Structured Differential Aware Network (SD-CNN encoder). The SD-CNN is an improved ResNet network that replaces the standard convolutions in the first two stages of ResNet with Center Differential Convolutions (CDC). By aggregating the gradient differences between the center pixel of the convolution kernel and its neighboring pixels, the network's ability to respond to subtle edge gradients is significantly enhanced.

[0063] The processing procedure of the convolutional branch is as follows: The crack image is input into the convolutional branch. First, it is processed through convolution and pooling layers to obtain initial convolutional features. Then, the initial convolutional features are input into the first central difference convolutional layer to obtain the first convolutional feature. The first convolutional feature is input into the second central difference convolutional layer to obtain the second convolutional feature. The second convolutional feature is input into the first Res convolutional layer to obtain the third convolutional feature. The third convolutional feature is input into the second Res convolutional layer to obtain the fourth convolutional feature. The first, second, third, and fourth convolutional features are convolutional features at different levels output by the convolutional branch.

[0064] The processing procedure of the central difference convolutional layer is represented by the following formula:

[0065]

[0066] in, The output features of the central difference convolutional layer, For learnable weights, The input feature map is the center difference convolutional layer. Center pixel coordinates To indicate the sensory field area Neighboring pixel coordinates within, For the sensory field area.

[0067] This formula enables the network to keenly capture intensity variations between pixels rather than absolute intensity values.

[0068] The Transformer branch is specifically a Transformer encoder, which includes four coding layers. Each coding layer's output is connected in series with a lightweight biaxial strip calibration module. Specifically, in constructing the Transformer branch, this embodiment abandons the traditional full-parameter fine-tuning and instead adopts a non-intrusive "frozen backbone combined with anisotropic feature calibration" strategy. All pre-trained weights of the Swin Transformer backbone network are maintained. It is currently in a frozen state and serves only as a high-order semantic extractor. To enhance the perception of slender cracks, a lightweight dual-axis strip calibration module (DASC) is added in parallel at the output of each layer.

[0069] The Transformer branch processes the following: The crack image is input into the Transformer branch. First, the crack image is divided into image patches. Then, linear projection is performed to obtain patch embedding vectors, and position embeddings are added, transforming the crack image into a matrix. This matrix is ​​then input into the first encoding layer to obtain the first transformation feature. The first transformation feature is input into the second encoding layer to obtain the second transformation feature. The second transformation feature is input into the third encoding layer to obtain the third transformation feature. The third transformation feature is input into the fourth encoding layer to obtain the fourth transformation feature. Here, the first, second, third, and fourth transformation features are transformation features at different levels output by the Transformer branch. These four transformation features are then input into four corresponding lightweight dual-axis strip calibration modules to obtain four corrected transformation features.

[0070] In one embodiment, the processing procedure of the biaxial strip calibration module includes:

[0071] (1) Utilizing asymmetric large receptive field convolution kernels (size: and Extract linear features in orthogonal directions respectively, and calculate horizontal and vertical masks.

[0072] Specifically, it is expressed by the following formula:

[0073]

[0074]

[0075] in, For horizontal masks, For vertical masks, It is the Sigmoid activation function. For average pooling operation, This is the input feature (i.e., conversion feature) for a lightweight biaxial strip calibration module.

[0076] (2) Correct the input features based on the horizontal mask and the vertical mask to obtain the corrected transformation features.

[0077] Specifically, it is expressed by the following formula:

[0078]

[0079] in, For Hadama accumulation, These are the corrected transformation features.

[0080] This design of the dual-axis strip calibration module enables the network to adaptively enhance the crack region and suppress nonlinear background noise without updating the backbone parameters. This avoids the risk of overfitting caused by small sample fine-tuning and effectively complements the gradient-aware features of the CNN branches.

[0081] S3 inputs the aligned convolutional features and transformed features into the frequency-spatial dual-gated fusion module to obtain the fused feature map.

[0082] The frequency-spatial dual-gated fusion module includes parallel spatial and frequency domain paths; the processing steps of the frequency-spatial dual-gated fusion module include:

[0083] S31, the spatial reliability map is obtained by calculating the spatial path through channel attention and pixel absolute difference.

[0084] To address the issue of spatial confusion between complex backgrounds (such as oil stains, watermarks, and shadows) and cracks, this invention proposes a frequency-spatial dual-gated fusion module. This module no longer relies solely on spatial pixel similarity but introduces frequency domain analysis as a supplementary criterion. The module comprises two parallel processing paths: a spatial path and a frequency domain path.

[0085] First, calculate the absolute pixel difference between the aligned convolutional features and the (corrected) transformed features:

[0086]

[0087] in, For aligned convolutional features, This refers to the aligned transformation feature (the transformation feature refers to the corrected transformation feature).

[0088] Spatial path calculation uses channel attention and pixel absolute difference to compute spatial reliability map To preserve spatial details;

[0089]

[0090] in, For activation function, For convolution operations, It is a multilayer perceptron. This is for average pooling.

[0091] S32, the frequency domain path obtains the convolution feature amplitude spectrum and the transformation feature amplitude spectrum by performing a two-dimensional fast Fourier transform on the aligned convolution features and (corrected) transformation features.

[0092] Specifically, it is expressed by the following formula:

[0093]

[0094] in, The convolution feature amplitude spectrum, To convert the characteristic amplitude spectrum, This is a Fourier transform.

[0095] S33, calculate the high-frequency component differences between the convolution feature amplitude spectrum and the transformation feature amplitude spectrum, and calculate the high-frequency difference map based on the high-frequency component differences.

[0096] Specifically, the difference in high-frequency components between the convolutional feature amplitude spectrum and the transformed feature amplitude spectrum is calculated and multiplied by a high-pass filter mask. (High-Pass Filter) is used to filter out low-frequency components (such as smoothing the background) and retain only high-frequency differences (such as crack edges or high-frequency noise), expressed by the following formula:

[0097]

[0098] in, This is a high-frequency difference graph.

[0099] S34 performs Fourier transformation on the high-frequency difference map and obtains a frequency domain saliency mask through convolution operations and activation functions.

[0100] The high-frequency difference map is restored back to the spatial domain through inverse Fourier transform, and then... After convolution and channel adjustment, a frequency domain saliency mask is generated using the Sigmoid activation function. .

[0101]

[0102] in, For activation function, To perform the operation of taking the real part, This is the inverse Fourier transform.

[0103] S35, the fusion weights are calculated based on the frequency domain saliency mask and the spatial reliability map, and the aligned convolutional features and (corrected) transformation features are fused based on the fusion weights to obtain the fusion feature map.

[0104] First, calculate the fusion weight. It is determined by gating in two dimensions:

[0105]

[0106]

[0107] in, and These are learnable balance coefficients. This mechanism effectively filters high-frequency noise interference using frequency domain information, resulting in fused features. It retains clear crack boundaries and has robust global context consistency, significantly reducing the false detection rate.

[0108] S4. Input the fused feature map into the decoder to obtain the semantic segmentation result of the narrow crack.

[0109] like Figure 2 As shown, the decoder adopts a multi-level upsampling structure and uses skip connections to gradually restore the fused features generated by the encoder to the original image resolution, so as to make up for the spatial details lost during the downsampling process and finally obtain the semantic segmentation result of the narrow crack.

[0110] The training process of the HECS-Net network is described below:

[0111] (1) Obtain the dataset and perform data annotation and preprocessing.

[0112] Multi-device collaborative image acquisition utilizes high-resolution industrial cameras (at least 4K resolution), drones equipped with gimbal cameras, and handheld smart terminal devices. Drones are primarily used to capture images of cracks in hard-to-reach areas such as the underside of bridges and the exterior walls of high-rise buildings, while handheld devices are used to capture close-up details of minute cracks. Multi-scene coverage encompasses a variety of complex engineering environments, including but not limited to concrete bridge surfaces, asphalt pavements, concrete linings inside tunnels, and cement exterior walls of buildings. To ensure the model's generalization ability, extreme environmental conditions are intentionally included during the acquisition process. Lighting conditions include front lighting, backlighting, shadow occlusion (such as tree or vehicle shadows), and artificial lighting at night. Surface conditions include dry surfaces, wet surfaces, oil-stained surfaces, and complex backgrounds such as traces left by road repairs. A basic database containing thousands of original images will be built, covering various crack morphologies, such as transverse cracks, longitudinal cracks, network cracks, and micro-hairline cracks.

[0113] To address the characteristics of the original images—high resolution, small crack proportions, and complex backgrounds—the following preprocessing operations were performed: Refined pixel-level annotation: Pixel-level semantic annotation was performed on the acquired images using open-source annotation tools (such as LabelMe). Since cracks typically exhibit extremely fine linear structures, strict specifications were established for annotation: the crack centerline must be meticulously delineated to ensure that the annotation error for crack width is controlled within 1-2 pixels. Two label categories were set: Background and Crack.

[0114] A sliding window cropping technique is applied to the original data. Considering the high resolution of the original images (e.g., 4000×3000), directly inputting them into the network would lead to memory overflow or loss of small object features. Therefore, a sliding window strategy is used to crop the high-resolution images into fixed-size patches (e.g., 512×512 or 768×768). The sliding step size is set to be smaller than the cropping size (i.e., retaining 10%-15% of the overlap area) to prevent cracks from being truncated at the image boundaries and to ensure the integrity of the cracks. Simultaneously, targeted data augmentation is performed. To enhance the model's robustness to complex environments and prevent overfitting, various data augmentation operations are implemented, such as random rotation (90°, 180°, 270°), random horizontal / vertical flipping, and random scaling (0.8-1.2x), simulating different shooting angles and distances, as well as lighting and color changes. Random color jitter (adjusting brightness, contrast, saturation, and hue) is applied to simulate imaging effects under different weather and lighting conditions. To mitigate overfitting, noise simulation is performed by adding Gaussian noise or Gaussian blur to simulate sensor noise or motion blur, improving the model's adaptability to low-quality images. Finally, standardization is applied, normalizing image pixel values ​​to the [0, 1] interval and performing Z-Score Normalization based on the mean and variance of the ImageNet dataset to accelerate network convergence.

[0115] The constructed dataset is divided into training, validation, and test sets according to a certain ratio. A total of 10,000 images are collected and allocated in an 8:1:1 ratio, with 8,000 images used as the training dataset, 1,000 images used for validation, and 1,000 images used for testing.

[0116] (2) Input the preprocessed dataset into the HECS-Net network for processing to obtain the predicted segmentation results. Construct the total loss function based on the predicted segmentation results and the corresponding real images, and train the HECS-Net network based on the total loss function.

[0117] The processing procedure for the HECS-Net network is the same as steps S2-S4, and will not be described in detail here.

[0118] Furthermore, this application's embodiments design a composite supervision strategy, abandoning the conventional combination of cross-entropy and Dice loss, and proposing a joint optimization objective that combines Focal Tversky Loss (classification loss function) and Gradient Direction Consistency Loss. First, to address the severe class imbalance problem and focus on mining hard-to-classify samples, Focal Tversky Loss (…) is introduced. It replaces the traditional Dice loss. Compared to the Dice coefficient, the Tversky coefficient introduces... and Parameters are used to control the penalty weights for false positives (FP) and false negatives (FN) separately, which is crucial for detecting minute cracks that are easily missed. This is achieved by introducing focusing parameters. To further reduce the weight of simple background samples, the following definition is used:

[0119]

[0120] in, It is a true positive. It was a false negative. The Tversky index. As a parameter, it is usually taken as To enhance attention to difficult samples, and They can be set separately and To strengthen the penalty for missed crack detection. Secondly, to address the common problems of edge jaggedness and directional breakage in crack detection, a gradient direction consistency loss is introduced. Traditional pixel-level loss mechanisms often ignore the geometric derivative information of the shape, while Forced Constraint Prediction Graph With truth graph The gradient direction remains consistent at the edges. The Sobel operator is used to calculate the gradient directions for both. and gradient of direction and And the directional difference is measured by cosine similarity:

[0121]

[0122] Where P represents the predicted semantic segmentation result, and G represents the corresponding ground truth graph. and Let be the gradients of the Sobel operator in the x and y directions. This is a very small constant used to prevent the denominator from being zero, ensuring the stability of numerical calculations.N This represents the total number of pixels involved in the calculation, used to normalize the summation result.

[0123] The final total loss function is defined as:

[0124]

[0125] in, These are the weight parameters.

[0126] By introducing gradient direction constraints, this joint loss function forces the model to focus not only on the accuracy of pixel classification, but also on the continuity and smoothness of crack edge direction, thereby fundamentally solving the problems of discontinuity and "dashed lines" in the detection of slender structures.

[0127] Furthermore, to verify the effectiveness of the proposed HECS-Net model in the task of segmenting long and narrow cracks, a detailed comparative experiment was conducted with it and current mainstream semantic segmentation networks. The comparison models included classic convolutional networks (such as FCN-8s, U-Net, and DeepLabV3+) and advanced Transformer networks (such as SegFormer and Swin-UNet). The constructed crack dataset was divided into training, validation, and test sets, which were then fed into the aforementioned models for training and inference, respectively. In the quantitative analysis, precision, recall, intersection-over-union (IoU), and F1 score were used as the core evaluation metrics, and their calculation formulas are as follows:

[0128]

[0129]

[0130]

[0131]

[0132] Among them, TP is a true positive, FP is a false positive, and FN is a false negative.

[0133] Figure 3 The segmentation result diagram provided in the embodiment of the present invention, such as Figure 3As shown in Table 1, the first column represents the original crack image, the second column represents the corresponding ground truth image, and the third column represents the predicted semantic segmentation image obtained using the HECS-Net network. It can be seen that HECS-Net performs excellently on the crack segmentation task. Specific experimental results are shown in Table 1. HECS-Net outperforms the comparison models on all key metrics, especially achieving a high Recall metric of 91.46% reflecting crack structural integrity, and a CrackIoU of 83.24%, demonstrating the significant advantages of skeleton recall loss and hybrid architecture in maintaining crack connectivity. Furthermore, compared to other hybrid models, HECS-Net maintains high accuracy while exhibiting lower computational complexity.

[0134] Table 1. Performance comparison of different models on the task of slender crack segmentation.

[0135]

[0136] Based on the method described in the above embodiments, this embodiment will further describe it from the perspective of a thin-slit semantic segmentation system supported by a hybrid architecture. This thin-slit semantic segmentation system supported by a hybrid architecture can be implemented as an independent entity or integrated into an electronic device. The electronic device can be a terminal, server, or other device. The terminal can include a tablet computer, a laptop computer, a personal computer (PC), a microprocessor box, or other devices.

[0137] Please see Figure 4 , Figure 4 This invention specifically describes a semantic segmentation system for elongated cracks based on a hybrid architecture, applicable to electronic devices. The system may include:

[0138] The acquisition and preprocessing module is used to acquire crack image data and perform preprocessing.

[0139] The encoding module is used to input the preprocessed crack image into the hybrid encoder, which includes a convolution branch and a Transformer branch. The convolution branch outputs convolutional features, and the Transformer branch outputs transformation features.

[0140] The fusion module is used to input the aligned convolutional features and transformed features into the frequency domain-spatial domain dual-gated fusion module to obtain the fused feature map;

[0141] The decoding module is used to input the fused feature map into the decoder to obtain the semantic segmentation result of the narrow crack.

[0142] In specific implementation, the above modules and / or units can be implemented as independent entities, or they can be arbitrarily combined and implemented as the same or several entities. For the specific implementation of the above modules and / or units, please refer to the previous method embodiments. For the specific beneficial effects that can be achieved, please also refer to the beneficial effects in the previous method embodiments, which will not be repeated here.

[0143] In addition, this embodiment of the invention also provides an electronic device, which may be a computer, tablet computer, or other such device. This electronic device can implement the steps of any embodiment of the elongated crack semantic segmentation method based on hybrid architecture support provided in this embodiment of the invention. Therefore, it can achieve the beneficial effects that any elongated crack semantic segmentation method based on hybrid architecture support provided in this embodiment of the invention can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0144] Figure 5 A specific structural block diagram of an electronic device provided in an embodiment of the present invention is shown. This electronic device can be used to implement the thin-slit semantic segmentation method based on hybrid architecture support provided in the above embodiments. The electronic device 500 can be a terminal, server, or other device. The terminal can include a tablet computer, laptop computer, personal computer (PC), microprocessor box, or other devices.

[0145] The memory 520 can be used to store software programs and modules, such as the program instructions / modules corresponding to those in the above embodiments. The processor 580 executes various functional applications and data processing by running the software programs and modules stored in the memory 520. The memory 520 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 520 may further include memory remotely located relative to the processor 580, and these remote memories can be connected to the electronic device 500 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0146] The input unit 530 can be used to receive input numeric or character information, and to generate a keyboard and mouse related to user settings and function control.

[0147] Display unit 540 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces, which can be composed of graphics, text, icons, video, and any combination thereof. Display unit 540 may include display panel 541, which may optionally be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or other similar forms.

[0148] Electronic device 500, through transmission module 570 (e.g., Wi-Fi module), can help users receive requests, send information, etc., providing users with wireless broadband internet access. Although transmission module 570 is shown in the figure, it is understood that it is not an essential component of electronic device 500 and can be omitted as needed without changing the essence of the invention.

[0149] The processor 580 is the control center of the electronic device 500. It connects to various parts of the phone via various interfaces and lines, and performs various functions and processes data of the electronic device 500 by running or executing software programs and / or modules stored in the memory 520, and by calling data stored in the memory 520, thereby providing overall monitoring of the electronic device. Optionally, the processor 580 may include one or more processing cores; in some embodiments, the processor 580 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 580.

[0150] Electronic device 500 also includes a power supply 590 (such as a battery) that supplies power to various components. In some embodiments, the power supply may be logically connected to processor 580 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 590 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0151] Although not shown, the electronic device 500 also includes cameras (such as front-facing cameras and rear-facing cameras), Bluetooth modules, etc., which will not be described in detail here. Specifically, in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors. One or more programs contain instructions for performing the following operations:

[0152] Acquire crack image data and perform preprocessing;

[0153] The preprocessed crack image is input into a hybrid encoder, which includes a convolutional branch and a Transformer branch. The convolutional branch outputs convolutional features, and the Transformer branch outputs transformed features.

[0154] The aligned convolutional features and transformed features are input into the frequency domain-spatial domain dual-gated fusion module to obtain the fused feature map;

[0155] The fused feature map is input into the decoder to obtain the semantic segmentation result of the narrow crack.

[0156] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities. For the specific implementation of the above modules, please refer to the previous method implementation examples, which will not be repeated here.

[0157] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. Therefore, embodiments of the present invention provide a storage medium storing multiple instructions that can be loaded by a processor to execute the steps of any embodiment of the elongated crack semantic segmentation method based on hybrid architecture support provided by the present invention.

[0158] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0159] Since the instructions stored in the storage medium can execute the steps in any embodiment of the long and thin crack semantic segmentation method based on hybrid architecture support provided in the embodiments of the present invention, the beneficial effects that any long and thin crack semantic segmentation method based on hybrid architecture support provided in the embodiments of the present invention can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0160] The foregoing has provided a detailed description of a semantic segmentation method, system, storage medium, and electronic device for elongated cracks based on a hybrid architecture, as provided in the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A semantic segmentation method for elongated cracks based on a hybrid architecture, characterized in that, The method includes: Acquire crack image data and perform preprocessing; The preprocessed crack image is input into a hybrid encoder, which includes a convolutional branch and a Transformer branch. The convolutional branch outputs convolutional features, and the Transformer branch outputs transformed features. The convolutional branch is an improved ResNet network, which includes a convolutional and pooling layer, two central difference convolutional layers, and two Res convolutional layers connected in sequence. The Transformer branch includes four coding layers, and the output of each coding layer is connected in series with a lightweight dual-axis strip calibration module. The number of frequency-spatial dual-gated fusion modules is the same as the number of coding layers in the Transformer branch. The lightweight dual-axis strip calibration module's processing includes: extracting linear features in orthogonal directions using asymmetric large receptive field convolution kernels, calculating horizontal and vertical masks; and correcting the input features based on the horizontal and vertical masks to obtain the corrected transformed features. The aligned convolutional features and transformed features are input into the frequency domain-spatial domain dual-gated fusion module to obtain the fused feature map; The frequency-spatial dual-gated fusion module includes parallel spatial and frequency paths. The processing steps of the frequency-spatial dual-gated fusion module include: the spatial path calculates a spatial reliability map using channel attention and pixel absolute differences; the frequency path performs a two-dimensional fast Fourier transform on the aligned convolutional and transformed features to obtain the convolutional feature amplitude spectrum and the transformed feature amplitude spectrum; calculates the high-frequency component differences between the convolutional and transformed feature amplitude spectra, and calculates a high-frequency difference map based on these differences; performs Fourier transformation on the high-frequency difference map, and obtains a frequency domain saliency mask through convolution operations and activation functions; calculates fusion weights based on the frequency domain saliency mask and the spatial reliability map, and fuses the aligned convolutional and transformed features based on these fusion weights to obtain a fused feature map. The fused feature map is input into the decoder to obtain the semantic segmentation result of the narrow crack.

2. The semantic segmentation method for elongated cracks based on a hybrid architecture as described in claim 1, characterized in that, The processing procedure of the central difference convolutional layer is expressed by the following formula: in, The output features of the central difference convolutional layer, For learnable weights, The input feature map is the center difference convolutional layer. The center pixel coordinates, To experience the wild area Neighboring pixel coordinates, For the sensory field area.

3. The semantic segmentation method for elongated cracks based on a hybrid architecture as described in claim 1, characterized in that, The horizontal and vertical masks are calculated using the following formula: in, For horizontal masks, For vertical masks, It is the Sigmoid activation function. For average pooling operation, Input characteristics for a lightweight biaxial strip calibration module. and For the size of the large sensory field; The process of correcting input features to obtain corrected transformed features is represented by the following formula: in, For Hadama accumulation, These are the corrected transformation features.

4. The semantic segmentation method for elongated cracks based on a hybrid architecture as described in claim 1, characterized in that, The method is applied to the HECS-Net model, which is trained based on a total loss function, which is: in, For the total loss function, For classification loss function, For weight parameters, Let be the gradient direction consistency loss function. It is a true positive. It was a false negative. The Tversky index. All are parameters, where P is the predicted semantic segmentation result and G is the corresponding ground truth graph. and Let be the gradients of the Sobel operator in the x and y directions. This is a very small constant smoothing term used to prevent the denominator from being zero, ensuring the stability of numerical calculations. N This represents the total number of pixels involved in the calculation, used to normalize the summation result.

5. A semantic segmentation system for elongated cracks based on a hybrid architecture, wherein the system is used to implement the semantic segmentation method for elongated cracks based on a hybrid architecture as described in claim 1, characterized in that... include: The acquisition and preprocessing module is used to acquire crack image data and perform preprocessing. The encoding module is used to input the preprocessed crack image into the hybrid encoder, which includes a convolution branch and a Transformer branch. The convolution branch outputs convolutional features, and the Transformer branch outputs transformation features. The fusion module is used to input the aligned convolutional features and transformed features into the frequency domain-spatial domain dual-gated fusion module to obtain the fused feature map; The decoding module is used to input the fused feature map into the decoder to obtain the semantic segmentation result of the narrow crack.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to execute the elongated crack semantic segmentation method based on a hybrid architecture supported by any one of claims 1 to 4.

7. An electronic device, characterized in that, It includes a processor and a memory, the processor being electrically connected to the memory, the memory being used to store instructions and data, and the processor being used to perform the steps in the long, thin crack semantic segmentation method based on a hybrid architecture as described in any one of claims 1 to 4.

Images