Intelligent detection method and system for appearance defects of multi-biological attachment pier

Abstract

The invention belongs to the technical field of underwater detection, and discloses an intelligent detection method and system for appearance defects of a multi-biological attachment pier, and the method comprises the steps: obtaining multi-modal image data, carrying out the feature-level registration, splicing the registered multi-modal images, and obtaining a fusion feature; obtaining a high-level semantic feature based on the fusion feature, obtaining a first feature map based on the high-level semantic feature and the fusion feature, and generating a pier surface biological attachment mask based on the first feature map; based on the advanced semantic features and the attachment mask, reconstructing a clean pier surface image without biological attachments; on the basis of the cleaned pier surface image, identifying and segmenting apparent defects of the pier surface; and defining a loss function, and training the intelligent detection model to obtain a trained intelligent detection model. According to the method, the accuracy and robustness of defect detection are remarkably improved, meanwhile, the dependence on manual annotation data is reduced, and a feasible intelligent solution is provided for structure detection in an underwater complex scene.

Description

Technical Field

[0001] This invention relates to the field of underwater inspection technology, and in particular to an intelligent inspection method and system for surface defects on bridge piers with multiple biological attachments. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Bridge piers are the core load-bearing structure of a bridge, and their health directly affects the stability and safety of the entire bridge. Once damaged, repairs are extremely difficult and costly, and may even lead to catastrophic consequences. Therefore, they are key parts that need to be monitored and protected during operation and maintenance.

[0004] However, bridge piers have been in service in complex and harsh aquatic environments for a long time, and their surfaces are extremely prone to the adhesion of organisms such as mussels, barnacles, and algae, forming a covering layer. This not only conceals structural defects such as cracks and peeling, but its complex texture and color also cause serious visual interference, making it impossible for mainstream underwater technical facility inspection methods such as ROV optical inspection and manual diving inspection to detect them accurately.

[0005] Manual underwater inspection is high-risk, subjective, inefficient, and difficult to quantify. While ROV optical inspection improves data acquisition capabilities, the biological attachments in the captured two-dimensional images are highly similar in texture and morphology to real structural defects such as cracks and pitting, forming a strong visual "camouflage." This makes it difficult for conventional image processing algorithms to effectively distinguish them, resulting in low accuracy in defect identification. Acoustic detection technology, while not limited by optics, has insufficient resolution and cannot capture minute damage or distinguish microorganisms. In summary, existing technologies cannot effectively solve the detection challenges in complex scenarios where "biological organisms" and "defects" coexist, and there is a lack of robust, high-precision intelligent detection solutions specifically designed for this scenario. Summary of the Invention

[0006] To address the aforementioned issues, this invention proposes an intelligent detection method and system for apparent defects on bridge piers with multiple biological attachments. This method effectively overcomes visual interference from biological attachments, significantly improves the accuracy and robustness of defect detection, and reduces reliance on manually labeled data. It provides a feasible intelligent solution for structural inspection in complex underwater scenarios.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides an intelligent detection method for apparent defects in bridge piers with multiple biological attachments, comprising the following steps: Multimodal data of the bridge pier surface is acquired and feature-level registration is performed. The registered multimodal data is then stitched together to obtain fused features. Hierarchical downsampling is performed on the fused features to obtain high-level semantic features. Hierarchical upsampling is performed on the high-level semantic features. The downsampled features and upsampled features of the same scale are spliced ​​and fused to obtain the first feature map. Based on the first feature map, a mask of biological attachments on the surface of the bridge pier is generated. Based on advanced semantic features and attachment masks, a clean bridge pier surface image without biological attachments is reconstructed. Based on images of clean bridge pier surfaces, identify and segment apparent defects on the bridge pier surface; Define a loss function, train the intelligent detection model, and obtain the trained intelligent detection model.

[0008] As an alternative implementation, the multimodal data includes RGB images, polarization information, and depth information. Feature-level registration employs an attention mechanism to weightedly fuse features from different modalities.

[0009] As an alternative implementation, the loss function includes reconstruction loss and adversarial loss. The reconstruction loss uses a decontamination generator to synthesize a clean surface image with an attachment mask into a fake image with attachments, and compares it with the original input image. The adversarial loss uses a discriminator to distinguish between the real image and the generated image, and performs adversarial training against the generator.

[0010] As an alternative implementation, the attachment parsing branch includes a decoder structure and a mask output layer for generating probabilistic and binary masks of biological attachments, and introduces a temporal consistency constraint based on optical flow warping to improve the stability of the mask.

[0011] As an alternative implementation, the surface reconstruction branch is a U-Net structure, which takes as input the stitching result of high-level semantic features and attachment masks, and outputs a reconstructed clean pier surface image, and introduces temporal consistency loss to smooth the reconstruction result.

[0012] As an alternative implementation, the defect detection branch adopts an encoder-decoder architecture and integrates shallow detail features during the decoding process to output a pixel-level category label map of the defect.

[0013] Secondly, the present invention provides an intelligent detection system for apparent defects in bridge piers with multiple biological attachments, comprising: The data acquisition and preprocessing module is configured to: acquire multimodal image data of the bridge pier surface and perform feature-level registration; then stitch the registered multimodal image data together to obtain fused features. The attachment analysis module is configured to: perform hierarchical downsampling on the fused features to obtain high-level semantic features, perform hierarchical upsampling on the high-level semantic features, and splice and fuse the downsampled features and upsampled features of the same scale to obtain a first feature map, and generate a biological attachment mask on the surface of the bridge pier based on the first feature map; The surface reconstruction module is configured to reconstruct clean bridge pier surface images free of biological attachments based on high-level semantic features and attachment masks. The defect detection module is configured to identify and segment apparent defects on the surface of the bridge pier based on images of the clean bridge pier surface. The model training module is configured to: define a loss function, train the intelligent detection model, and obtain a trained intelligent detection model.

[0014] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0015] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0016] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention presents an intelligent detection method for surface defects on bridge piers with multiple biological attachments, effectively overcoming interference from biological attachments and improving defect identification accuracy. Through an innovative "decomposition-detection" process, it first utilizes multimodal information fusion and feature decoupling technology to accurately separate biological attachments from the bridge pier body at the pixel level, generating an attachment distribution mask. Then, based on deep learning, it reconstructs an image of the occluded clean surface, providing uncontaminated input for defect identification. This method eliminates visual confusion caused by attachments at the data source, fundamentally avoiding false detections, allowing subsequent algorithms to focus on genuine surface defects, thereby significantly improving the accuracy and reliability of identifying minor damage such as cracks and spalling.

[0018] This invention presents an intelligent detection method for apparent defects on bridge piers with multiple biological attachments. It enhances the robustness of the algorithm by utilizing multimodal data and a self-supervised mechanism. The method effectively integrates multi-source information such as RGB, polarization, and depth, fully leveraging the complementarity of different modal data. Polarization information helps distinguish surface materials, while depth information provides geometric features. Combined with RGB texture information, this provides a more comprehensive basis for the model to distinguish between attachments and the structural body. Furthermore, the design incorporates a temporal self-supervised signal. By applying smoothness constraints to the prediction results of consecutive video frames, it reduces flickering and jitter in the output results, improving the stability and reliability of the algorithm in complex underwater environments.

[0019] This invention provides an intelligent detection method for surface defects on bridge piers with multiple biofouling deposits, offering a quantifiable assessment of surface condition to support precise engineering decisions. Compared to conventional methods that only output defect categories, this invention simultaneously generates biofouling coverage area and distribution, clean surface restoration images, and precise pixel-level contours and classification information of defects. This structured data enables a quantitative assessment of the bridge pier's surface condition, allowing for the calculation of key indicators such as biofouling coverage, defect area ratio, and spatial distribution. Based on these objective quantitative results, engineers can accurately grasp the degree of biofouling and structural damage, providing direct data support for developing targeted cleaning and maintenance plans and repair solutions, effectively improving the scientific rigor and accuracy of operation and maintenance decisions.

[0020] This invention presents an intelligent detection method for apparent defects on bridge piers with multiple biological attachments. This method reduces reliance on labeled data and enhances cross-scene transferability. The invention constructs a self-supervised learning loop through reconstruction loss and adversarial training, enabling the model to generate supervisory signals from the raw data itself, significantly reducing dependence on costly manual annotation. This design allows the algorithm to maintain stable performance in real-world scenarios lacking labeled samples, greatly improving the model's adaptability across different water bodies and bridge pier types. This makes the technology effectively adaptable to detection tasks in different water bodies and bridge pier types, demonstrating its potential for large-scale application.

[0021] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0022] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0023] Figure 1 This is a flowchart illustrating the framework of the intelligent detection method for apparent defects on bridge piers with multiple biological attachments according to the present invention. Figure 2 This is a schematic diagram of the neural network of the present invention. Detailed Implementation

[0024] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0025] It should be noted that the following detailed description is exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0026] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but includes other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0028] Terminology Explanation: 1. Feature-level registration: This is an image processing algorithm that aims to align semantic features corresponding to different images in spatial location, allowing for some deformation and not requiring strict pixel-level matching. Compared to pixel-level registration, this method reduces hardware requirements, allows for slight differences in sensor position or viewpoint, and fuses refined, more meaningful information rather than raw, potentially noisy data. The algorithm can autonomously decide how to most effectively combine these features, making it more intelligent.

[0029] 2. Receptive Field: This refers to the size of the region in the original input image corresponding to a point on the feature map of a convolutional neural network. In other words, it measures how much of the original input image a neuron in the feature map represents. A larger receptive field means the neuron can capture more and more comprehensive contextual information; a smaller receptive field means it captures more localized information and focuses on finer details.

[0030] 3. Mask: In computer vision and image processing, a mask is a matrix or image used to define the interest of a specific region in an image. Its core functions are "selection" and "masking". Taking this scheme as an example, each value in the matrix can represent the confidence or probability that the corresponding pixel in the input image belongs to the semantic category of "biological attachment". The binary image is the result of thresholding the attachment mask, and is a special type of mask. Each element in its matrix takes only two discrete values, usually represented by 0 and 1 (or 0 and 255). By setting a decision threshold τ on the probability mask M, binarization can be performed to generate a binary image B: ; ; in: (Often visualized as white) indicates that this pixel is identified as the foreground, i.e., the biological attachment in this scheme; (Often visualized as black) indicates that the pixel is determined to be the background, that is, the surface of the bridge pier without any attachments in this scheme.

[0031] 4. Warping: This is an image transformation technique. Its core is to resample the source image pixel by pixel according to a given deformation field in order to map it to the target coordinate system.

[0032] 5. Bilinear interpolation: This is a core method in deep learning and image processing for estimating pixel values ​​at non-integer coordinates. The idea is to perform linear interpolation twice in two different directions.

[0033] Example 1 like Figure 1 As shown, this embodiment provides an intelligent detection method for apparent defects on bridge piers with multiple biological attachments, including the following steps: Multimodal data of the bridge pier surface is acquired and feature-level registration is performed. The registered multimodal data is then stitched together to obtain fused features. Hierarchical downsampling is performed on the fused features to obtain high-level semantic features. Hierarchical upsampling is performed on the high-level semantic features. The downsampled features and upsampled features of the same scale are spliced ​​and fused to obtain the first feature map. Based on the first feature map, a mask of biological attachments on the surface of the bridge pier is generated. Based on advanced semantic features and attachment masks, a clean bridge pier surface image without biological attachments is reconstructed. Based on images of clean bridge pier surfaces, identify and segment apparent defects on the bridge pier surface; Define a loss function, train the intelligent detection model, and obtain the trained intelligent detection model.

[0034] The specific solution of the present invention is as follows: (1) Obtain multimodal data of the bridge pier surface and perform data registration. Then, stitch together the registered multimodal data to obtain fusion features.

[0035] This invention uses an underwater robot to collect multimodal data from the surface of bridge piers, and then performs feature-level registration and stitching on the data. Specific implementation steps include: S1: The underwater robot is equipped with key sensors such as a polarization camera and a structured light scanner that can record information such as color, light intensity and light wave polarization direction. It collects continuous video sequence images of the underwater bridge pier surface and obtains polarization information such as standard RGB images, polarization degree and polarization angle, structured light 3D point cloud information and depth information.

[0036] S2: The collected RGB, polarization, depth and other multimodal data are precisely registered at the feature level to ensure that different types of information correspond to the same position on the surface of the bridge pier. The registered multimodal data are then stitched together to obtain fused features.

[0037] Furthermore, the steps of S2 are as follows: Taking bimodal data as an example: RGB image; Polarization image.

[0038] S21: Initially extract features from each modality's data using a shared encoder E (CNN): (1); and For feature maps of the same dimension, let their shape be denoted as . .

[0039] S22: Use attention maps to weightedly fuse features from different modalities, laying the foundation for the network to learn to assign higher weights to features of the appropriate modality at different locations in the image while registering data.

[0040] use The function will use RGB features and polarization characteristics Concatenating them along the channel dimension, the output is a feature block containing information from both modalities, with a shape of [shape missing]. Use one more Convolutional layer Processing this feature block involves scanning the fused features, and after comprehensive analysis, outputting a preliminary weight value A, i.e., a single-channel feature map, with the shape of... .

[0041] Each value output from the convolutional layer is mapped using the Sigmoid function, which compresses any numerical value to between 0 and 1. In this case, we define a value closer to 1 as indicating that the network considers RGB features at that location to be strong. More importantly, the closer the value is to 0, the more the network believes that the polarization characteristics are present at that location. More importantly, the final attention weight values ​​will also be obtained after data registration and training are completed: (2); Final features The result of weighted fusion: (3); S3: Indirectly guides the alignment of modal data through a series of loss functions and learning tasks in the network.

[0042] Further preferred, the specific steps of S3 are as follows: S31: Using the reconstruction loss function as a self-supervised signal to promote data registration.

[0043] In a self-supervised closed-loop system, the decontamination generator G reassembles the clean surface image output from the surface reconstruction branch. ) and the attachment mask output by the attachment parsing branch ( This generates a fake image with attached objects. The fake image is then compared to the original acquired image. Losses: (4); With the goal of minimizing this loss, the encoder E learns to attach objects. and cleaning surfaces Precise registration is performed to allow the network to reconstruct the original image. During this step, the network also learns that in areas with biological attachments, polarization features that better distinguish the material have a higher weight; while in areas with distinct geometric structures, RGB or structured light features have a higher weight. S32: Use the adversarial loss function as another self-supervised signal to facilitate data registration.

[0044] In the self-supervised closed-loop system, the discriminator D distinguishes between real bridge pier images and images generated by the decontamination generator G. D returns 1 if it determines the image to be "true," and 0 otherwise. This is the design... : (5); The stronger the discrimination ability of D, the better. The smaller. Ideally, it should be. , ,at this time It is 0. To minimize With the goal of training, the discriminator D is trained to label the images generated by the generator G as "fake" and the real images as "real", and the parameters of D are continuously updated to enhance its discrimination ability.

[0045] After the discriminator D is trained, its parameters are fixed, and an adversarial loss function is designed. : (6); By minimizing To promote characteristics and The image is well-aligned so that the generator G can reconstruct an approximate image, with the goal of generating an image that the discriminator D will classify as "true". Ideally, this would be... That is, the discriminator judges the image generated by G as true, and this term is 0 at this time.

[0046] S4: Organize consecutive video frames into a time series to provide a data foundation for subsequent time series constraints.

[0047] (2) Perform hierarchical downsampling on the fused features to obtain high-level semantic features, perform hierarchical upsampling on the high-level semantic features, and splice and fuse the downsampled features and upsampled features of the same scale to obtain the first feature map. Based on the first feature map, generate a mask of biological attachments on the surface of the bridge pier.

[0048] First, the shared encoder extracts multimodal features based on hierarchical downsampling. Using a deep convolutional neural network as the shared backbone, the shared encoder progressively extracts and condenses multimodal information from the input image (i.e., fuses features) through a hierarchical structure including convolution, nonlinear activation, and downsampling, ultimately outputting a general high-level feature map rich in semantics. (i.e., high-level semantic features). The specific implementation steps are as follows: S1: Shallow feature extraction and preliminary downsampling.

[0049] The early layers of the encoder achieve initial spatial downsampling through convolutions and convolutional layers with a stride greater than 1, expanding the receptive field while maintaining high spatial resolution, and focusing on extracting local low-level features. (8); in, This represents the first group of convolution, activation, and downsampling operations. Output feature map. The size is It includes shallow information such as primary textures, including edges, corners, basic color patches, and the roughness of concrete.

[0050] S2: Intermediate feature combination and further downsampling.

[0051] Within the larger receptive field obtained through further downsampling, the intermediate layer combines shallow features through convolutional operations to form more complex intermediate patterns: (9); At this stage, the feature map The size of the feature continues to decrease, while the semantic features are improved. The network adaptively integrates multimodal information again to form structured representations such as "circular outline" corresponding to barnacles, "crack structure" corresponding to cracks, and "layered surface" corresponding to mussel communities, thus initially decoupling the abstract information of components such as attachments and defects.

[0052] S3: High-level semantic abstraction and global context integration.

[0053] After multiple downsampling operations, the feature map size of deep networks is significantly reduced, but each neuron possesses a large receptive field covering most of the original image, thus enabling the integration of global contextual information and the combination of intermediate features into high-level semantic concepts. (10); The high-level feature map obtained at this level This is the final output of the shared encoder, where This represents the total downsampling factor. This tensor achieves high semanticization of features in the channel dimension (C), with different channels corresponding to advanced concepts such as "biological attachments," "structural defects," and "intact concrete surfaces," respectively; in the spatial dimension... The above encodes the coarse location information of each semantic concept. This feature tensor provides a shared and robust feature foundation for subsequent parallel attachment parsing, surface reconstruction, and defect detection branches.

[0054] Secondly, hierarchical upsampling is performed on the high-level semantic features. Downsampled and upsampled features of the same scale are then concatenated and fused to obtain a first feature map. Based on this first feature map, a mask of biological attachments on the bridge pier surface is generated. The specific implementation steps are as follows: S1: The decoder layer restores resolution based on high-level feature maps.

[0055] The decoder in this branch is composed of multiple upsampling modules stacked together. (The sentence is incomplete and requires more context to translate accurately.) Taking a layer as an example, input a high-level feature map. Each module first performs upsampling, then uses transposed convolution to double the size of the input feature map: (11); in, .

[0056] Then combine the encoder features of the same scale Features after upsampling To splice: (12); Finally, feature extraction is performed. Regular convolutional layers are used to process the concatenated features, fusing information and reducing the number of channels. (13) The decoder's final output feature map (First feature image) Shape is The image has been restored to the same size as the input image. This feature map contains rich spatial and semantic information fused from the encoding and decoding processes.

[0057] S2: Output mask and attached texture.

[0058] Use two independent The convolutional layers generate two types of outputs: masks and textures.

[0059] A linear classifier specifically designed for binary segmentation is applied to the decoder's output feature map. At each pixel location, by integrating information from all channels at that location, a raw score representing the confidence level of "is an attachment" is calculated: (14); The Sigmoid function is used again to compress the original Logits score, which is in any range, into the (0,1) interval, thus representing the probability that the pixel is an attachment: (15); Received The number of output channels is 1. The final threshold is set. The binary mask of the attachment is obtained: (16); Here This is an indicator function that returns 1 if the condition within the parentheses is true, and 0 otherwise.

[0060] This is performed in parallel with texture decomposition prediction: (17); Received The output channel number is K. The K-dimensional vector of each pixel is an abstract representation of the texture features of that point. Different types of textures will be mapped to different regions in this feature space.

[0061] S3: Improve prediction accuracy by using loss functions and temporal consistency constraints based on optical flow warping.

[0062] During training, the mask output This provides important spatial constraints for texture decomposition. The loss function here is designed as follows: (18); in This is a hyperparameter used to balance the weights of the two loss parameters.

[0063] The binary cross-entropy loss is used to supervise the binary segmentation of the attachment region. (19); Multi-class cross-entropy loss is used to supervise fine-grained texture classification within the attachment and the background. (20); Where FG represents the foreground region, that is, the region consisting of... Predicted areas of attachment. This represents the number of foreground pixels.

[0064] This means that the gradient of the texture loss will only propagate back from the attachment region. When training the texture branch, the network is only required to optimize the classification accuracy within the attachment region. For the background region (concrete), the output of the texture branch is not evaluated by the loss function, thus avoiding unnecessary interference.

[0065] To improve the temporal stability of mask prediction, this method also applies a temporal consistency constraint based on optical flow warping to the predicted attachment mask, taking advantage of the physical property that the distribution and morphology of the attachments do not change drastically in a short period of time. This forces the model to output smooth, progressive and stable results on consecutive video frames.

[0066] First, an optical flow estimation network is used to calculate the optical flow field from frame T to frame T+1. This describes the specific position of each pixel in frame T when it moves to frame T+1.

[0067] Using the calculated optical flow field The attachment mask predicted in frame T. By inversely warping to the viewpoint of frame T+1, a predicted mask is obtained using bilinear interpolation. : (twenty one); The above warped prediction The model's actual prediction result for frame T+1 For comparison, if the predictions are time-stationary, the two should be very similar; therefore, the time-series loss function is defined as follows: (twenty two); This loss function will penalize The greater the difference between the result inferred from the previous frame and the result inferred from the previous frame, the greater the penalty.

[0068] Through the above multi-task decoder design, the network will learn to locate attachments and describe texture characteristics at the pixel level. This is an end-to-end, loss function-driven, complex mapping process from high-level features to pixel-level predictions.

[0069] (3) Based on high-level semantic features and attachment masks, reconstruct clean bridge pier surface images free of biological attachments. The specific implementation steps are as follows: S1: The reconstruction decoder R uses information provided by shared features and attachment branches to reconstruct a virtual image of a clean bridge pier surface free of any biological attachments.

[0070] This decoder typically consists of transposed or upsampled convolutional layers and regular convolutional layers; it is a parameterized neural network with a U-Net-like structure. (twenty three); Mask M provides explicit spatial guidance for this process: in the area marked by the mask as the background ( The learning objective of reconstructing the network is to retain and restore as much as possible the data from the original network. The original characteristics are preserved to ensure the fidelity of information in areas not covered by the mask. In areas marked as attachments by the mask ( The learning objective of reconstructing the network is to suppress and discard [illegible characters]. The system detects the characteristic responses of areas contaminated by biological deposits and, based on contextual information provided by the surrounding clean area covered by its receptive field (including but not limited to texture structure, spectral characteristics, and geometry), infers and generates an image that matches the surrounding environment in terms of visual coherence, material consistency, and physical plausibility, thereby achieving a reasonable reconstruction of the occluded area. This mechanism ensures that its output is not an exact restoration of the original occluded pixels, but rather an optimal visual estimate based on contextual awareness and prior knowledge.

[0071] S2: Utilize temporal consistency constraints based on optical flow warping to ensure the temporal stability of the reconstruction results.

[0072] This branch also introduces a temporal consistency loss function, which forces the clean surfaces reconstructed from adjacent frames to maintain consistency in content, effectively eliminating the flickering and jitter of the reconstructed surface caused by random occlusion by attachments, ensuring the temporal stability of the reconstruction results, and providing reliable input for defect detection.

[0073] (twenty four); Furthermore, the reconstruction loss function mentioned above It will also drive the reconstruction decoder R to generate a high-quality, realistic... .

[0074] (4) Based on clean bridge pier surface images, identify and segment the apparent defects on the bridge pier surface.

[0075] Using the biodegradable, repaired bridge pier surface image generated by the surface reconstruction branch as the sole input, the system accurately identifies and locates surface defects such as cracks, spalling, holes, and exposed rebar, and outputs their category, pixel-level contour, and location information. The specific implementation steps are as follows: S1: Pixel-level prediction is performed using an encoder-decoder network architecture.

[0076] The encoder uses a pre-trained convolutional neural network, ResNet, as the backbone network to extract discriminative multi-level features from the input clean surface image.

[0077] (25); The decoder gradually restores spatial resolution and fuses deep semantic information with shallow detail information through upsampling and skip connections.

[0078] (26); Here It is a computational unit that integrates core steps such as up-adoption and jump connection.

[0079] Jump connections inject shallow details: (27); Finally, a 1×1 convolutional layer is used to map the number of channels to the total number of categories C.

[0080] (28); in: And here , where K is the number of defect types to be detected (such as cracks, peeling, etc.), and 1 represents the background class.

[0081] S2: Output a label map containing defect categories.

[0082] The Logits are processed using a pixel-by-pixel Sigmoid function to obtain a probability map of each pixel belonging to each category.

[0083] For each pixel position The category with the highest probability is selected as the final predicted label: (29); The final output is a label image with the same resolution as the input clean surface. Each pixel value in the image is a category index label, which represents the defect category that the network determines the pixel belongs to. For example, 0 represents the background, 1 represents a crack, 2 represents spalling, and 3 represents exposed rebar.

[0084] S3: Introduce a loss function to improve the accuracy of defect prediction.

[0085] To accurately guide the training of the defect detection branch and enable it to effectively handle challenges such as class imbalance (e.g., background pixels far outnumber defect pixels) and complex boundaries (e.g., thin cracks), this branch employs a joint loss function that combines weighted cross-entropy loss and Dice loss: (30); Among them, weighted cross-entropy loss Higher weights are assigned to defect categories with lower frequency of occurrence to alleviate class imbalance, forcing the model to pay more attention to these easily overlooked difficulties. The mathematical expression is: (31); Where c is a pixel The true category label. The weight coefficients are pre-set for category c, and their magnitudes are inversely proportional to the frequency of that category in the training data. Dice loss Directly optimize the defect area predicted by the model Compared to the actual defect area The size of the intersection between the parameters optimizes the segmentation of the foreground target, i.e., the defect. Its mathematical expression is: (32); in and These represent the total number of pixels in the foreground region of the true label and the predicted result, respectively.

[0086] It is a smoothing constant with a very small value, used to avoid the denominator being zero and to ensure numerical stability.

[0087] Joint loss function Weighted cross-entropy loss To ensure pixel-level classification accuracy, while using Dice loss... This directly optimizes the spatial overlap between the predicted and ground truth regions, improving the overall segmentation integrity of defects. These two approaches complement each other, jointly ensuring the model's comprehensive performance in terms of pixel accuracy and region consistency.

[0088] (5) Self-supervised signal generation and adversarial training.

[0089] The decontamination generator combines "clean surface" with "attachment mask and texture" information to inversely synthesize the original observed image with attachments, constructing a self-supervised closed loop. By comparing the synthesized image with the real original input image, the image reconstruction loss and adversarial loss are calculated, forcing the decontamination generator to produce results that are extremely realistic in both geometry and texture, ensuring the geometric consistency and realism of the entire decomposition-reconstruction process.

[0090] Example 2 This embodiment provides an intelligent detection system for apparent defects on bridge piers with multiple biological attachments, including: The data acquisition and preprocessing module is configured to: acquire multimodal image data of the bridge pier surface and perform feature-level registration; then stitch the registered multimodal image data together to obtain fused features. The attachment analysis module is configured to: perform hierarchical downsampling on the fused features to obtain high-level semantic features, perform hierarchical upsampling on the high-level semantic features, and splice and fuse the downsampled features and upsampled features of the same scale to obtain a first feature map, and generate a biological attachment mask on the surface of the bridge pier based on the first feature map; The surface reconstruction module is configured to reconstruct clean bridge pier surface images free of biological attachments based on high-level semantic features and attachment masks. The defect detection module is configured to identify and segment apparent defects on the surface of the bridge pier based on images of the clean bridge pier surface. The model training module is configured to: define a loss function, train the intelligent detection model, and obtain a trained intelligent detection model.

[0091] It should be noted that the above modules correspond to the steps in Embodiment 1, and the examples and application scenarios implemented by the above modules and their corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules can be executed in a computer system as part of the system.

[0092] In further embodiments, the following is also provided: An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0093] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0094] A computer-readable storage medium for storing computer instructions that, when executed by a processor, perform the method of Embodiment 1.

[0095] The method in Example 1 can be directly executed by a hardware processor, or it can be executed by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0096] A computer program product includes a computer program that, when executed by a processor, implements the method in Embodiment 1.

[0097] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0098] The computer program code used to implement the methods of the present invention may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0099] In the context of this invention, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0100] Those skilled in the art will recognize that the units and algorithm steps described in conjunction with the embodiments herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0101] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. An intelligent detection method for apparent defects on bridge piers with multiple biological attachments, characterized in that, Includes the following steps: Multimodal data of the bridge pier surface is acquired and feature-level registration is performed. The registered multimodal data is then stitched together to obtain fused features. Hierarchical downsampling is performed on the fused features to obtain high-level semantic features. Hierarchical upsampling is performed on the high-level semantic features. The downsampled features and upsampled features of the same scale are spliced ​​and fused to obtain the first feature map. Based on the first feature map, a mask of biological attachments on the surface of the bridge pier is generated. Based on advanced semantic features and attachment masks, a clean bridge pier surface image without biological attachments is reconstructed. Based on images of clean bridge pier surfaces, identify and segment apparent defects on the bridge pier surface; Define a loss function, train the intelligent detection model, and obtain the trained intelligent detection model.

2. The intelligent detection method for apparent defects in bridge piers with multiple biological attachments as described in claim 1, characterized in that, The multimodal data includes RGB images, polarization information, and depth information. The feature-level registration uses an attention mechanism to weightedly fuse features from different modalities.

3. The intelligent detection method for apparent defects in bridge piers with multiple biological attachments as described in claim 1, characterized in that, The loss function includes reconstruction loss and adversarial loss. The reconstruction loss uses a decontamination generator to synthesize a clean surface image with an attachment mask into a fake image with attachments, and compares it with the original input image. The adversarial loss uses a discriminator to distinguish between real images and generated images, and performs adversarial training against the generator.

4. The intelligent detection method for apparent defects in bridge piers with multiple biological attachments as described in claim 1, characterized in that, The attachment parsing branch includes a decoder structure and a mask output layer, which are used to generate probabilistic and binary masks of biological attachments, and introduce temporal consistency constraints based on optical flow warping to improve the stability of the masks.

5. The intelligent detection method for apparent defects of bridge piers with multiple biological attachments as described in claim 1, characterized in that, The surface reconstruction branch is a U-Net structure. Its input is the stitching result of high-level semantic features and attachment masks, and its output is the reconstructed clean bridge pier surface image. Temporal consistency loss is introduced to smooth the reconstruction result.

6. The intelligent detection method for apparent defects in bridge piers with multiple biological attachments as described in claim 1, characterized in that, The defect detection branch adopts an encoder-decoder architecture and integrates shallow detail features during the decoding process to output a pixel-level category label map of defects.

7. An intelligent detection system for apparent defects in bridge piers with multiple biological attachments, characterized in that, include: The data acquisition and preprocessing module is configured to: acquire multimodal image data of the bridge pier surface and perform feature-level registration; then stitch the registered multimodal image data together to obtain fused features. The attachment analysis module is configured to: perform hierarchical downsampling on the fused features to obtain high-level semantic features, perform hierarchical upsampling on the high-level semantic features, and splice and fuse the downsampled features and upsampled features of the same scale to obtain a first feature map, and generate a biological attachment mask on the surface of the bridge pier based on the first feature map; The surface reconstruction module is configured to reconstruct clean bridge pier surface images free of biological attachments based on high-level semantic features and attachment masks. The defect detection module is configured to identify and segment apparent defects on the surface of the bridge pier based on images of the clean bridge pier surface. The model training module is configured to: define a loss function, train the intelligent detection model, and obtain a trained intelligent detection model.

8. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the method described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the method described in any one of claims 1-6.

Images