Mold and salt mist defect recognition method and system based on dense-cbam-unet++ double branch fusion

Abstract

The application discloses a mold and salt mist defect recognition method and system based on Dense-CBAM-UNet++ double-branch fusion. The mold and salt mist defect recognition method based on Dense-CBAM-UNet++ double-branch fusion comprises image acquisition, image preprocessing, shared encoder feature extraction, double-branch decoding parallel detection, standard rating mapping, detection report generation, data storage and backtracking. The mold and salt mist defect recognition system based on Dense-CBAM-UNet++ double-branch fusion comprises an image acquisition module, a preprocessing module, a shared feature extraction module, a double-branch parallel module and a report generation module. On the basis of UNet++, the application fuses dense connection (Dense), channel attention and spatial attention (CBAM) and a double-task decoupling structure to form a lightweight high-precision deep learning model specially adapted to mold and salt mist double-scene defect recognition, and realizes synchronous mold grade classification and salt mist corrosion segmentation rating through one forward inference.

Description

Technical Field

[0001] This invention relates to the field of material environmental reliability testing technology, specifically to a method and system for identifying mold and salt spray defects based on Dense-CBAM-UNet++ dual-branch fusion. Background Technology

[0002] In the field of military assembly environmental testing, mold testing and salt spray corrosion testing are two key environmental testing items for evaluating the product's resistance to damp heat and salt corrosion. They are widely used for environmental testing of shipborne equipment, military components used in jungle environments, coastal outdoor communication equipment, and anti-corrosion coating materials. Military assembly operates in actual service environments such as coastal areas and humid warehouses, where salty moisture coexists with high temperature and humidity. Salt spray corrosion causes pinholes and blistering damage to the coating, and the humid environment easily fosters mold growth. The interaction between these two factors means that mold hyphae further damage the protective layer and accelerate substrate corrosion. Therefore, during product type approval testing and factory sampling inspection, a large number of samples need to undergo both mold and salt spray environmental tests sequentially. Often, the surface of the same sample is simultaneously covered with mold spots and various corrosion defects.

[0003] Currently, most testing personnel employ a combination of manual visual inspection and traditional machine vision testing. Mold testing follows GJB 150.10A-2009 "Environmental Testing Methods for Military Assembly Laboratories - Part 10: Mold Testing," with a 0-4 graded standard to determine the degree of mold growth on the test surface. Salt spray corrosion testing is based on GB / T 6461-2002 "Rating of Specimens and Test Pieces with Metallic and Other Inorganic Coatings on Metallic Substrates After Corrosion Testing," using a dual-dimensional assessment of "Protection Rating (RP) + Appearance Rating (RA)," including defect area calculation and formula R. P =3(2-logA), defect type A~J coding, relying on dual indicators to achieve quantitative judgment of corrosion degree.

[0004] As the mass production scale of military equipment continues to expand and the batch size of samples to be tested continues to increase, mold growth and salt spray corrosion problems frequently occur on the surface of the samples. Relying on the existing manual visual inspection combined with traditional machine testing mode has the following technical defects: Manual visual inspection and rating are highly subjective, with inspectors relying on personal experience to judge the level of mold and salt spray corrosion. Different inspectors may have different ratings for the same sample, resulting in inconsistent ratings and low testing efficiency. Furthermore, prolonged manual visual inspection can easily lead to visual fatigue. Testing large batches of samples requires significant labor costs, and tracing and controlling the test results is difficult.

[0005] Traditional machine learning relies on manually designed features, which suffers from poor stability in extracting features of irregular small targets such as tiny mold spots, pinhole corrosion, and bulges. These small defects are prone to being missed or misjudged, making it difficult to adapt to complex sample imaging conditions. Furthermore, existing machine mold and salt spray detection technologies are independent of each other and cannot be integrated for identification.

[0006] In view of this, how to overcome the shortcomings of the existing technology has become the subject of study and solution of this invention. Summary of the Invention

[0007] The purpose of this invention is to provide a method and system for identifying mold and salt spray defects based on Dense-CBAM-UNet++ dual-branch fusion.

[0008] To achieve the above objectives, on the one hand, the technical solution adopted by this invention is: a method for identifying mold and salt spray defects based on Dense-CBAM-UNet++ dual-branch fusion, comprising: Step 1, Image Acquisition: Acquire images of the sample surface using a multi-dimensional image sensor to establish an original image sequence; Step 2, Image Preprocessing: The original image is subjected to size normalization, Gaussian denoising, white balance correction, and image edge masking in sequence to obtain the preprocessed image; Step 3, Feature Extraction by Shared Encoder: The preprocessed image is input into the shared encoder. The DenseBlock dense connection module in the shared encoder enhances feature reuse and alleviates gradient vanishing. The CBAM channel and spatial attention module in the shared encoder adaptively enhance the features of the defect region from both channel and spatial dimensions, and outputs multi-scale shared features. Step 4, Dual-branch parallel decoding and reasoning: Input the multi-scale shared features into the mold classification branch and the salt spray segmentation branch respectively, and output the two types of recognition results simultaneously after one forward reasoning; Mold classification branch: The multi-scale shared features are successively processed through global average pooling, fully connected layer and Softmax normalization to obtain the probability distribution of mold growth level, and the 0~4 level mold rating is obtained by mapping according to the GJB 150.10A-2009 grading standard. Salt spray segmentation branch: The UNet++ deep supervised segmentation network is used to perform semantic segmentation on multi-scale shared features to obtain segmentation results of multiple types of corrosion defects. The output includes substrate corrosion area, blistering, pitting, corrosion products, peeling, and anodic / cathode coating defects. The corrosion area is calculated, and the corrosion ratio is converted according to the corresponding standard GB / T 6461-2002. The salt spray corrosion test results of the sample are then output. Step 5, Standard Result Output: Integrate mold growth level and salt spray corrosion rating to generate test results conforming to the corresponding standard format. Through built-in corresponding national and military standard grading mapping standards, the mold level, mold description, salt spray protection rating, salt spray appearance rating, and comprehensive salt spray rating are standardized and organized.

[0009] In the above scheme, mold / salt spray samples are photographed to establish a high-resolution original image sequence. Multi-dimensional collection of complete sample surface defect information is carried out. After the images are captured, image preprocessing is performed. The acquired original images are sequentially normalized in size, Gaussian denoising, white balance correction, and image edge masking to obtain preprocessed images.

[0010] The preprocessed image is input into the shared encoder and the DenseBlock dense connection module to extract mold colony, hyphae, color, and texture features, suppress the background, and enhance tiny mold spots, thus fully preserving shallow and subtle defect features, alleviating the gradient vanishing problem during training, and improving the model's feature extraction effect and training stability. The CBAM attention module enhances the model through channel and spatial attention and multi-scale feature fusion, effectively reducing background interference and improving the extraction accuracy of tiny defects.

[0011] The shared features output by the shared encoder are split into a mold recognition branch and a salt spray corrosion recognition branch by a dual-branch decoder. In the mold recognition branch, the shared features are processed sequentially through global average pooling, a fully connected layer, and Softmax to obtain the mold growth probability. The mold is then mapped to a 0-4 level mold rating according to the GJB 150.10A-2009 grading standard. Global average pooling is used to compress the feature dimension, reduce the number of parameters, and improve the recognition accuracy of actual sample images. The fully connected layer obtains the feature matching scores corresponding to the 5 levels, which are then converted into the predicted probabilities of each category by Softmax. The category number corresponding to the highest probability is selected, and the mold rating is mapped to a 0-4 level according to the GJB 150.10A-2009 grading standard. This eliminates the need for manual comparison standards, unifies the grading standards, effectively avoids subjective bias caused by reliance on human experience, and effectively reduces mold grading errors. The salt spray corrosion identification branch of UNet++ accurately segments various corrosion defects, automatically calculates the corrosion ratio, and automatically calculates the protection rating and appearance rating according to the GB / T 6461-2002 formula, replacing manual area measurement and rating, resulting in more accurate detection results.

[0012] This solution employs a shared encoder and dual-branch parallel detection. A single set of image features can simultaneously complete both the national military standard grading for mold and the national standard grading for salt spray, effectively reducing the number of image acquisitions and feature extractions, significantly improving detection efficiency, reducing manual visual judgment, and minimizing detection errors caused by human subjectivity. Furthermore, it organizes defect rating data according to national and military standards, automatically generating detection reports, saving time spent on manual data processing and report writing. The entire process is archived, facilitating subsequent retrieval of original images, intermediate data, and results for easy verification.

[0013] In a further technical solution, the edge shielding treatment in step 2 strictly matches the effective detection area requirements of GB / T 6461-2002 and GJB150.10A-2009, retaining only the main detection surface of the sample for subsequent identification calculations.

[0014] Preferably, in step 2, non-effective detection areas such as hanger marks, contact holes, and brackets are removed from the original image to reduce interference factors in the image and improve the accuracy of subsequent image recognition.

[0015] In a further technical solution, in the salt spray segmentation branch of step 4, the area ratio calculation and protection rating R are... P The specific rules are as follows: The percentage of pixels with corrosion defects in the total number of pixels in the effective detection area of ​​the sample is denoted as corrosion area percentage A. If A=0, then the protection rating R P =10; If A > 0, then according to formula R P =3(2-logA) is used for calculation. After rounding the result, it is limited to the range of 0 to 10 to obtain the protection rating R. P .

[0016] In a further technical solution, in the salt spray segmentation branch of step 4, the appearance rating R A It consists of a combination of area level, severity indicator, and defect type code; The defect type code corresponds to Class AJ defects in GB / T 6461-2002, and the severity is divided into four levels: very mild, mild, moderate and severe. When multiple types of defects exist, multiple sets of appearance rating combinations will be output accordingly.

[0017] In a further technical solution, in the shared encoder, a DenseBlock dense connection module is connected in series after each downsampling layer, and the output of each DenseBlock is connected to a CBAM attention module; the DenseBlock realizes the reuse and transmission of shallow texture features and deep semantic features through cross-layer dense connections, and the CBAM attention module completes feature weight calibration in sequence through the channel attention submodule and the spatial attention submodule.

[0018] A further technical solution is that the method also includes a model pre-training step, specifically: A sample dataset was constructed, in which mold samples were labeled with growth levels 0 to 4 according to GJB 150.10A-2009, and salt spray samples were labeled with defect areas, defect types and area proportions according to GB / T 6461-2002. Data augmentation processing is performed on the sample dataset, including rotation, flipping, Gaussian noise, brightness transformation, and blurring. The network was trained using a multi-task joint loss function, where the mold classification branch used cross-entropy loss and the salt spray segmentation branch used a combination of Dice loss, cross-entropy loss and area regression loss. After training, the model is lightweighted through channel pruning and knowledge distillation.

[0019] Further technical solutions include training using the AdamW optimizer with an initial learning rate of 1e-4; model evaluation metrics include mAP@0.5, rank accuracy, and area calculation error, with the area calculation error controlled within ±0.05%.

[0020] The network was optimized using the AdamW optimizer with a learning rate of 1e-4. After training, the model was lightweighted by channel pruning and knowledge distillation to adapt it to embedded deployment. The model evaluation metrics were mAP@0.5, rank accuracy, and area error controlled within ±0.05%.

[0021] A further technical solution is that the method also includes generating test reports and storing and tracing data; The output data from step 5 will be automatically used to generate a sample testing report. The system stores the original image, preprocessed image, feature data, rating results, and detection report in a unified manner, supporting subsequent data retrieval.

[0022] On the other hand, the technical solution adopted in this invention is: a mold and salt spray defect identification system based on Dense-CBAM-UNet++ dual-branch fusion, including an image acquisition module, a preprocessing module, a shared feature extraction module, a dual-branch parallel decoding module, a standard result output module, and a report storage and backtracking module; The image acquisition module is used to acquire sample images through a multi-dimensional image sensor and establish an original image sequence; The preprocessing module is connected to the image acquisition module and is used to perform size normalization, Gaussian noise reduction, white balance correction and edge masking on the original image in sequence, and output the preprocessed image. The shared feature extraction module is connected to the preprocessing module and has a built-in shared encoder for receiving the preprocessed image and extracting multi-scale shared features. The shared encoder has a DenseBlock dense connection module and a CBAM channel-space joint attention module. The DenseBlock dense connection module realizes multi-layer feature reuse and alleviates gradient vanishing. The CBAM channel-space joint attention module adaptively enhances the features of the defect region from the channel dimension and the spatial dimension. The dual-branch parallel decoding module includes a mold classification branch unit and a salt spray segmentation branch unit. The mold classification branch unit and the salt spray segmentation branch unit are respectively connected to the shared feature extraction module, which can simultaneously output two types of recognition results based on multi-scale shared feature input to the dual-branch decoder inference. The standard result output module is connected to the mold classification branch unit and the salt spray segmentation branch unit respectively, and is used to integrate the mold growth level and the salt spray corrosion rating to generate test results that conform to the corresponding standard format. The report storage and traceability module is connected to the standard result output module and is used to automatically generate sample test reports, uniformly store original images, preprocessed images, feature data, rating results and test reports, and support the retrieval and traceability of test data.

[0023] A further technical solution is that a DenseBlock dense connection module is connected in series after each downsampling layer in the shared encoder, and the output end of each DenseBlock dense connection module is configured with a CBAM attention module; the DenseBlock dense connection module achieves the reuse and transfer of shallow texture features and deep semantic features through cross-layer dense connections, and the CBAM attention module completes feature weight calibration in sequence through the channel attention submodule and the spatial attention submodule.

[0024] A further technical solution is that the mold classification branch unit is configured with a global average pooling unit, a fully connected layer unit, and a Softmax normalization unit; the global average pooling unit and the fully connected layer unit perform feature mapping on multi-scale shared features, and the Softmax normalization unit outputs the probability distribution corresponding to each mold growth level, and outputs a mold rating of 0 to 4 according to the GJB 150.10A-2009 grading rules.

[0025] A further technical solution is that the salt spray segmentation branch unit is configured with a UNet++ deep supervised segmentation network, which is used to perform semantic segmentation on the multi-scale shared features output by the shared feature extraction module to obtain corrosion defect segmentation results; The salt spray segmentation branch unit is also equipped with a corrosion rating calculation unit and an appearance rating unit. The corrosion rating calculation unit is used to calculate the percentage of corrosion defect pixels in the total number of pixels in the effective detection area of ​​the sample, and obtain the corrosion area ratio A. If A=0, then the protection rating R P =10; If A > 0, then according to formula R P =3(2-logA) is used for calculation. After rounding the result, it is limited to the range of 0 to 10 to obtain the protection rating R. P .

[0026] The appearance rating unit is used to combine area grade, severity label, and defect type code to generate an appearance rating R. A The defect type code corresponds to the AJ class defect specified in GB / T 6461-2002, and the severity is divided into four levels: very minor, minor, moderate and severe. If there are multiple types of defects, multiple sets of appearance rating combinations will be output.

[0027] In a further technical solution, the system also includes a model training module, which includes a sample labeling subunit, a data augmentation subunit, a multi-task loss training subunit, and a model lightweighting subunit. The sample labeling subunit is used to construct the sample dataset. Mold samples are labeled with growth levels 0 to 4 according to GJB 150.10A-2009, and salt spray samples are labeled with defect areas, defect types and area proportions according to GB / T 6461-2002. The data augmentation subunit is used to perform data augmentation processing on the sample dataset, including rotation, flipping, Gaussian noise, brightness transformation, and blurring. The multi-task loss training subunit uses a multi-task joint loss function to train the network, wherein the mold classification branch uses cross-entropy loss, and the salt spray segmentation branch uses a combination of Dice loss, cross-entropy loss and area regression loss. The model lightweighting subunit is used to perform model lightweighting through channel pruning and knowledge distillation after training.

[0028] Due to the application of the above-mentioned solution, the technical solution of this application has the following advantages and effects compared with the prior art: This invention employs a Dense-CBAM-UNet++ architecture, utilizing dense connections (DenseBlock) to enhance feature reuse, CBAM attention to focus on defective regions, and UNet++ deep supervision to improve segmentation accuracy. It constructs a shared encoder and a dual-branch decoder structure, enabling simultaneous mold level assessment (GJB 150.10A-2009) and salt spray corrosion rating (GB / T6461-2002) in a single inference iteration. The algorithm incorporates national standard rating formulas and edge masking rules, directly outputting results in a standard format.

[0029] It effectively reduces the number of image acquisitions and the number of repeated feature extractions, improves detection efficiency, automates the entire process, reduces subjective errors caused by manual visual judgment, and improves the stability and accuracy of detection results.

[0030] The DenseBlock module enhances feature reuse, enabling multi-layer defect feature reuse and mitigating the gradient vanishing problem in deep networks. This is beneficial for preserving features of minute mold spots and fine pitting. CBAM channel attention and spatial dual attention automatically focus on defect locations, weakening useless background interference and improving the accuracy of defect feature extraction. One set of features is used to supply two detection peaks, effectively avoiding redundant feature extraction and saving computational power.

[0031] The system operates in parallel with two branches, organizing defect rating data according to national and military standards, and automatically generating inspection reports, saving the time spent on manual data processing and report writing. Furthermore, the entire process is archived, facilitating the retrieval of original images, intermediate data, and results for subsequent data verification. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the method for identifying mold and salt spray defects based on Dense-CBAM-UNet++ dual-branch fusion according to the present invention. Figure 1 ; Figure 2 This is a flowchart illustrating the method for identifying mold and salt spray defects based on Dense-CBAM-UNet++ dual-branch fusion according to the present invention. Figure 2 ; Figure 3 This is a schematic diagram of the module composition of the mold and salt spray defect identification system based on the Dense-CBAM-UNet++ dual-branch fusion of the present invention. Detailed Implementation

[0033] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0034] The terms "first," "second," etc., used in this article do not specifically refer to order or sequence, nor are they intended to limit this case; they are merely used to distinguish components or operations described using the same technical terms.

[0035] The terms "connection" or "positioning" as used in this article can refer to two or more components or devices making direct physical contact with each other, or making indirect physical contact with each other, or to two or more components or devices operating or moving with each other.

[0036] The terms “include,” “including,” and “have” used in this article are all open-ended, meaning they include but are not limited to.

[0037] Unless otherwise specified, the terms used herein generally have their ordinary meaning in the context of the art, the subject matter, and the specific context. Certain terms used to describe this case will be discussed below or elsewhere in this specification to provide additional guidance to those skilled in the art in describing this case.

[0038] The terms “front,” “back,” “up,” “down,” “left,” and “right” used in this article are directional terms. In this case, they are only used to describe the positional relationship between the structures and are not intended to limit the specific direction of the protection scheme or its actual implementation.

[0039] The terminology used herein is for the purpose of describing specific embodiments only and is not intended to limit the scope of this work. Singular forms such as “a,” “this,” “this,” “the,” and “the” as used herein also include plural forms.

[0040] This application aims to address the problems of existing methods that rely on manual visual assessment to determine the level of mold and salt spray, which are subject to the influence of the testers' experience, have high subjectivity, inconsistent rating results for the same sample, and are prone to visual fatigue for manual testing in large-scale testing, with fragmented raw data that is not conducive to traceability; and the independent use of mold and salt spray detection equipment, which requires repeated image acquisition, feature extraction, and repeated calculations, resulting in low equipment utilization.

[0041] The Dense-CBAM-UNet++ dual-branch fusion architecture in this patent application is described as follows: Dense is a densely connected module, CBAM is a convolutional block attention module, and UNet++ is an improved U-shaped image segmentation network with a deep supervision mechanism. These modules together constitute the backbone network used in this application for extracting image features. A feature extraction backbone integrating the Dense densely connected module and the CBAM attention module is adopted, with two parallel processing branches for mold level recognition and salt spray defect segmentation. The two branches share the multi-scale image features extracted by the backbone, achieving simultaneous recognition of mold and salt spray defects.

[0042] See Figure 1 and Figure 2 As shown, the mold and salt spray defect identification method based on Dense-CBAM-UNet++ dual-branch fusion includes: Step 1, Image Acquisition: Acquire images of the sample surface using a multi-dimensional image sensor to establish an original image sequence; take pictures of the mold / salt spray test samples to establish a high-resolution original image sequence.

[0043] Step 2, Image Preprocessing: The acquired original image is subjected to size normalization, Gaussian noise reduction, white balance correction, and image edge masking in sequence to obtain a preprocessed image; the edge masking process strictly matches the effective detection area requirements of GB / T 6461-2002 and GJB150.10A-2009, and only the main detection surface of the sample is retained for subsequent identification calculations.

[0044] The original image was scaled to 512×512 pixels, Gaussian noise reduction was applied, color difference was corrected by automatic white balance, and the edge area of ​​the sample was shielded according to national standards. Non-effective detection areas such as fixture marks, bracket indentations, control strips, and contact holes were removed to obtain a pre-processed image of 512×512 pixels.

[0045] Step 3, Feature Extraction by Shared Encoder: The preprocessed image is input into the shared encoder. The DenseBlock dense connection module in the shared encoder enhances feature reuse and alleviates gradient vanishing. The CBAM channel and spatial attention modules in the shared encoder adaptively enhance the features of the defect region from both channel and spatial dimensions, outputting multi-scale shared features. In the shared encoder, a DenseBlock dense connection module is connected in series after each downsampling layer, and the output of each DenseBlock is connected to a CBAM attention module. The DenseBlock achieves the reuse and transfer of shallow texture features and deep semantic features through cross-layer dense connections. The CBAM attention module completes feature weight calibration in sequence through the channel attention sub-module and the spatial attention sub-module.

[0046] After the image is downsampled and compressed at each layer, it is first fed into DenseBlock to reuse multi-layer shallow and deep textures and defect details, alleviating the gradient vanishing problem caused by the increase in network depth. The feature image output by DenseBlock is directly fed into the CBAM module, which first filters and suppresses invalid background channels through the channel dimension, and then locates the areas where mold spots and corrosion points are located through the spatial dimension. After completing the double feature enhancement, it is then fed into the next downsampling layer to continuously extract multi-scale features.

[0047] The shared encoder extracts mold colony, hyphae, color, and texture features, suppresses the background, enhances tiny mold spots, and outputs multi-scale shared features.

[0048] Step 4, Dual-branch parallel decoding and reasoning: Input the multi-scale shared features into the mold classification branch and the salt spray segmentation branch respectively, and output the two types of recognition results simultaneously after one forward reasoning; Mold classification branch: The multi-scale shared features are processed sequentially through global average pooling, fully connected layer and Softmax normalization to obtain the probability distribution of mold growth level, and then mapped to the 0~4 level mold rating according to the GJB 150.10A-2009 grading standard; Grade 0: No mold growth; Level 1: Trace growth (scattered, sparse); Level 2: Mild growth (discontinuous spread on the surface); Level 3: Moderate growth (extensive growth, visible changes in material). Level 4: Severe growth (heavy cover); Output mold grade: 0 / 1 / 2 / 3 / 4 and growth description.

[0049] Salt spray segmentation branch: A UNet++ deep supervised segmentation network is used to perform semantic segmentation on multi-scale shared features, obtaining segmentation results for various types of corrosion defects. Outputs include substrate corrosion areas, blistering, pitting, corrosion products, peeling, and anodic / cathode coating defects. The corrosion area is calculated, and the corrosion percentage is converted according to the corresponding standard GB / T 6461-2002, outputting the salt spray corrosion detection results for the sample. UNet++ employs a multi-layer deep supervision mechanism. Specifically, the shared encoder outputs multi-scale features, which are then fed into the UNet++ decoder for layer-by-layer upsampling and fusion. The decoder progressively enlarges the feature image size from shallow to deep layers and fuses the subtle texture information from the encoder's corresponding layers. Each layer of the decoder generates a defect prediction image independently after upsampling. During the training phase, the prediction image generated by each layer participates in the loss calculation independently, and the weighted sum of multiple loss layers jointly participates in the network parameter iteration. The shallow processing results capture the outlines of tiny pinholes and sporadic pitting corrosion, while the deep processing results complete the overall segmentation of large-area corrosion areas. In the actual detection and inference stage, only the defect area output by the deepest decoder is used as the final segmentation result, balancing the accuracy of tiny defect recognition and the inference speed.

[0050] The salt spray corrosion identification branch performs pixel segmentation of the corrosion area based on the shared features output by the shared encoder, and calculates the percentage of corrosion pixels in the total effective area of ​​the sample to obtain the corrosion ratio A. And calculate R according to GB / T 6461-2002 formula P =3(2-logA) to calculate the protection rating R P Based on the calculation results, protection ratings of 0 to 10 are obtained. R P Salt spray protection rating, the calculated result is rounded to the final national standard protection level of 0~10; A: the percentage of corrosion area to the effective surface area of ​​the sample; log: the common logarithm with base 10; Appearance rating R A The judgment is based on the area grade 0~10, severity (vs / s / m / x), and defect type (AJ) in GB / T 6461-2002, which are combined sequentially to output the appearance rating (R). A .

[0051] Salt spray protection rating R P Calculations show that if A=0, the sample shows no corrosion, and the protection rating R is directly determined. P =10; When A > 0, according to formula R P The calculation is performed using 3(2-logA), and the calculated value is rounded to the nearest integer. The rounded result is then limited to the range of 0-10; values ​​less than 0 are rounded to 0, and values ​​greater than 10 are rounded to 10. This yields the standard protection rating R for levels 0-10. P .

[0052] Appearance rating: R A Rating and Determination: Based on the defect segmentation results, defect types are classified and identified, and a preset defect type AJ is matched. Then, based on the defect morphology and damage range, the severity of the defect is determined, using four labels: vs (very mild), s (mild), m (moderate), and x (severe). Combined with the corrosion percentage A, an area rating of 0-10 is calculated; appearance rating R. A Appearance ratings are generated by combining area level, severity indicator, and defect type code. Example annotation format: such as 2mA, 6sC.

[0053] Finally, output in the standard format, according to R. P / R A Outputs, such as 9 / 2mA, 10 / 6sC.

[0054] Step 5, Standard Result Output: Integrate mold growth level and salt spray corrosion rating to generate test results conforming to the corresponding standard format. Through built-in corresponding national and military standard grading mapping standards, the mold level, mold description, and salt spray protection rating (R) are mapped. P Salt spray appearance rating R A Standardize and organize the comprehensive salt spray rating. Step 6: Generate test report: Automatically generate sample test report from the output data of Step 5; Step 7, Data Storage and Backtracking: Store the original image, preprocessed image, feature data, rating results, and detection report in a unified manner to support subsequent data backtracking and retrieval.

[0055] Step 8, Model Training and Optimization: Construct a sample dataset, in which mold samples are labeled with growth levels 0 to 4 according to GJB 150.10A-2009, and salt spray samples are labeled with defect areas, defect types and area proportions according to GB / T 6461-2002; Data augmentation processing is performed on the sample dataset, including rotation, flipping, Gaussian noise, brightness transformation, and blurring. The network was trained using a multi-task joint loss function, where the mold classification branch used cross-entropy loss and the salt spray segmentation branch used a combination of Dice loss, cross-entropy loss and area regression loss. After training, the model is lightweighted through channel pruning and knowledge distillation.

[0056] The network was trained using an annotated sample dataset, including mold and salt spray samples. Mold samples were labeled according to GJB 150.10A (grades 0-4), while salt spray samples were labeled according to GB / T 6461 (defect areas, types, and areas). Before training, the sample datasets were augmented using rotation, flipping, Gaussian noise, brightness transformation, and blurring. After training, the network was lightweighted. In the model training and optimization steps, the mold recognition branch used cross-entropy loss to optimize network parameters and ensure classification accuracy; the salt spray corrosion recognition branch was trained using a combination of Dice loss, cross-entropy loss, and area regression loss to ensure accurate corrosion region segmentation and area calculation.

[0057] Training was performed using the AdamW optimizer with an initial learning rate of 1e-4. Model evaluation metrics included mAP@0.5, rank accuracy, and area calculation error, with the area calculation error controlled within ±0.05%.

[0058] The AdamW optimizer employs a decoupled independent weight decay mechanism. Gradient calculation uses only the original loss gradient to complete adaptive first-order and second-order momentum updates. The weight decay operation is executed independently of the gradient update process, avoiding interference between weight decay and adaptive learning rate. This effectively suppresses overfitting in deep networks and improves the generalization ability to identify minor defects.

[0059] mAP@0.5 is a common accuracy evaluation index in the fields of image segmentation and defect recognition. Its core judgment logic is as follows: calculate the intersection-union ratio (IOU) between the predicted defect region and the actual labeled defect region. When the IOU is greater than or equal to 50%, it is considered a valid detection. This application simultaneously identifies multiple types of corrosion defects (AJ) and also needs to distinguish between different degrees of mold. mAP@0.5 is used to quantify the model's recognition effect. A 50% overlap ratio between the predicted and actual labeled defect regions is used as the correct detection threshold. The average accuracy for each type of defect is calculated separately, and then the average accuracy for all categories is obtained to objectively measure the overall detection capability for various defects such as micro-pitting corrosion, large-area blistering, and sporadic mold.

[0060] Complete analysis process for mold identification: Input the image of the sample after the mold test; Preprocessing (size normalization, de-reflection, white balance, noise reduction, automatic masking of the 5mm edge area, removal of reference strips and bracket interference areas); Feature extraction (shared encoder extracts mold colony, hyphae, color, and texture features, suppresses background, and enhances tiny mold spots); Mold branch inference (multi-scale feature fusion, global pooling, fully connected layer, softmax, outputting mold growth probability). Level mapping (levels 0-4); Output (mold grade, growth description).

[0061] Complete analysis process for salt spray identification: Input an image of the salt spray corrosion sample; Pretreatment (removal of salt crystals, stains, and reflections; standardized lighting; automatic shielding of the 5 mm edge area of ​​the sample; removal of fixture marks and contact holes; preservation of the main surface according to standards); Salt spray branch segmentation (UNet++ deep supervised segmentation, outputting substrate corrosion areas, blistering, pitting, peeling, corrosion products, and anodic / cathonic coating defects). Area calculation (pixel statistics: percentage of eroded area A, calculated according to standard formula R) P =3(2-logA) is calculated, which is mapped to protection ratings of 0 to 10. Appearance rating judgment (defect type code AJ, severity label: vs=very minor, s=minor, m=moderate, x=severe, area level 0~10, multiple defect combination output); Performance Rating Output (R) P / R A Output examples, such as 9 / 2mA, 10 / 6sC).

[0062] See Figure 3 As shown, the mold and salt spray defect identification system based on Dense-CBAM-UNet++ dual-branch fusion includes: an image acquisition module, a preprocessing module, a shared feature extraction module, a dual-branch parallel decoding module, a standard result output module, and a report storage and backtracking module.

[0063] The image acquisition module is used to acquire sample images through a multi-dimensional image sensor and establish a high-resolution raw image sequence.

[0064] The preprocessing module is connected to the image acquisition module and is used to perform size normalization, Gaussian denoising, white balance correction, and edge masking on the original image in sequence, and output the preprocessed image.

[0065] The shared feature extraction module is connected to the preprocessing module and includes a built-in shared encoder. This encoder receives the preprocessed image output from the preprocessing module and extracts multi-scale shared features, enhancing small defect features. The shared encoder contains a DenseBlock dense connection module and a CBAM channel-spatial joint attention module. The DenseBlock dense connection module enables multi-layer feature reuse and mitigates gradient vanishing, while the CBAM channel-spatial joint attention module adaptively enhances defect region features from both channel and spatial dimensions. Each downsampling layer in the shared encoder is followed by a DenseBlock dense connection module, and the output of each DenseBlock dense connection module is configured with a CBAM attention module. The DenseBlock dense connection module achieves the reuse and transfer of shallow texture features and deep semantic features through cross-layer dense connections. The CBAM attention module sequentially performs feature weight calibration through channel attention sub-modules and spatial attention sub-modules. The dual-branch parallel decoding module includes a mold classification branch unit and a salt spray segmentation branch unit. Both the mold classification branch unit and the salt spray segmentation branch unit are connected to the shared feature extraction module, enabling the dual-branch decoder to infer and output two types of recognition results simultaneously based on multi-scale shared feature input.

[0066] The standard result output module is connected to the mold classification branch unit and the salt spray segmentation branch unit respectively, and is used to integrate the mold growth level and salt spray corrosion rating to generate test results that conform to the corresponding standard format.

[0067] The mold classification branch unit is configured with a global average pooling unit, a fully connected layer unit, and a Softmax normalization unit. The global average pooling unit and the fully connected layer unit perform feature mapping on the multi-scale shared features. The Softmax normalization unit outputs the probability distribution corresponding to each mold growth level and outputs a mold rating of 0 to 4 according to the GJB 150.10A-2009 grading rules.

[0068] The salt spray segmentation branch unit is configured with a UNet++ deep supervised segmentation network to perform semantic segmentation on the multi-scale shared features output by the shared feature extraction module, thereby obtaining the corrosion defect segmentation result, i.e., defect area calculation. R is then calculated in accordance with GB / T 6461-2002. P / R A .

[0069] The salt spray segmentation branch unit is equipped with a corrosion rating calculation unit and an appearance rating unit. The corrosion rating calculation unit is used to calculate the percentage of corrosion defect pixels in the total number of pixels in the effective detection area of ​​the sample to obtain the corrosion area ratio A. If A=0, then the protection rating R P =10; If A > 0, then according to formula R P=3(2-logA) is used for calculation. After rounding the result, it is limited to the range of 0 to 10 to obtain the protection rating R. P .

[0070] The appearance rating unit is used to combine area rating, severity label, and defect type code to generate an appearance rating R. A The defect type code corresponds to the AJ class defect specified in GB / T 6461-2002, and the severity is divided into four levels: very minor, minor, moderate and severe. If there are multiple types of defects, multiple sets of appearance rating combinations will be output.

[0071] The report storage and traceability module is connected to the standard result output module to automatically generate sample test reports, uniformly store original images, preprocessed images, feature data, rating results and test reports, and support the retrieval and traceability of test data.

[0072] This embodiment of the system also includes a model training and optimization unit, which comprises a sample labeling subunit, a data augmentation subunit, a multi-task loss training subunit, a model lightweighting subunit, a network optimization subunit, and an evaluation index subunit. The sample labeling subunit includes mold samples and salt spray samples. Mold samples are labeled according to GJB 150.10A with levels 0-4, and salt spray samples are labeled with defect areas, types, and areas according to GB / T 6461. The data augmentation subunit augments the samples using rotation, flipping, Gaussian noise, brightness transformation, and blurring. The multi-task loss training subunit trains the network using a multi-task joint loss function, where the mold classification branch uses cross-entropy loss, and the salt spray segmentation branch uses a combination of Dice loss, cross-entropy loss, and area regression loss. The model lightweighting subunit lightweights the model through channel pruning and knowledge distillation to adapt it for embedded deployment. The network optimization sub-unit uses the AdamW optimizer with a learning rate of 1e-4 to optimize the network; the evaluation metric sub-unit uses mAP@0.5, rank accuracy, and area error <±0.05% as model evaluation metrics; and implements functions such as dataset labeling, data augmentation, network iterative optimization, model lightweighting, and performance testing.

[0073] Based on UNet++, this invention integrates dense connections, channel attention, and spatial attention (CBAM) with a dual-task decoupling structure to form a lightweight, high-precision deep learning model specifically adapted for defect recognition in both mold and salt spray scenarios. This model enables mold level classification and salt spray corrosion segmentation and rating to be completed simultaneously in a single forward inference.

[0074] By improving the backbone network, DenseBlock dense connection embedding is used. Dense connection blocks are added between downsampling layers of the encoder, enabling direct reuse and transmission of shallow texture features and deep semantic features. This significantly improves the utilization rate of weak target features such as tiny mold spots, pinhole corrosion, and minor bubbles, alleviates gradient vanishing, and improves the detection rate of small defects.

[0075] The attention mechanism has been improved, with enhanced joint attention of CBAM channels and space. A CBAM attention module is connected after each DenseBlock output. First, key defect feature dimensions are filtered through channel attention, and then the defect area is located through spatial attention. This enables adaptive focusing on mold colonies, corrosion spots, and corrosion product areas, while suppressing background and reflection interference.

[0076] The architecture has been improved with a decoupled dual-branch decoding structure. A shared encoder and dual-branch independent decoder structure are adopted. The shared encoder extracts common defect features, reducing the number of parameters and computational load. The mold branch uses a classification head to specifically output growth levels 0-4. The salt spray branch uses a UNet++ deep-supervised segmentation head to specifically perform defect segmentation, area calculation, and rating calculation. The two tasks do not interfere with each other and output synchronously, achieving integrated identification of both defects.

[0077] The loss function is improved, and the joint loss across multiple tasks is optimized. The mold branch uses cross-entropy loss to ensure classification accuracy; the salt spray branch uses Dice loss. Figure 3 Dice), cross-entropy loss ( Figure 3 The system incorporates CE (Central Emissions and Coefficients) and area regression loss (for monitoring purposes, ensuring accurate segmentation of the corroded region) to guarantee the precision of the corroded region segmentation and the accuracy of the area calculation, thus ensuring that the rating results strictly conform to the national standard formula.

[0078] The post-processing module has been improved, featuring hard mapping to national standards. The algorithm incorporates the GB / T 6461-2002 protection rating formula and the GJB150.10A-2009 mold grading rules, automatically completing: Salt spray: R P =3(2-logA) calculation, defect type A~J coding, degree vs / s / m / x labeling; mold: level 0~4 judgment; directly output standard format results, no manual conversion required.

[0079] Improved pretreatment with adaptive standard shielding. In accordance with GB / T 6461-2002, the sample edge (5mm), fixture marks, contact holes, and support area are automatically shielded to ensure the identification range and rating rules fully comply with national standards.

[0080] This embodiment employs a shared encoder paired with a dual-branch parallel detection architecture, leveraging DenseBlock and CBAM for efficient defect feature extraction. A single set of extracted features is used to simultaneously complete mold grading and salt spray corrosion rating, eliminating the need for two separate imaging and feature extraction processes, effectively reducing equipment computational overhead and testing time. The entire process automatically completes quantitative judgments according to current national and military standards, and image processing automatically calculates the corrosion percentage, converting it into protection and appearance ratings. This effectively reduces human error associated with traditional visual judgment, improving rating consistency and detection accuracy. Furthermore, this embodiment uniformly archives and stores the original images, feature data, and generated reports, facilitating subsequent data tracing and review.

[0081] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for identifying mold and salt spray defects based on Dense-CBAM-UNet++ dual-branch fusion, characterized in that, include: Step 1, Image Acquisition: Acquire images of the sample surface using a multi-dimensional image sensor to establish an original image sequence; Step 2, Image Preprocessing: The original image is subjected to size normalization, Gaussian denoising, white balance correction, and image edge masking in sequence to obtain the preprocessed image; Step 3, Feature Extraction by Shared Encoder: The preprocessed image is input into the shared encoder. The DenseBlock dense connection module in the shared encoder enhances feature reuse and alleviates gradient vanishing. The CBAM channel and spatial attention module in the shared encoder adaptively enhance the features of the defect region from both channel and spatial dimensions, and outputs multi-scale shared features. Step 4, Dual-branch parallel decoding and reasoning: Input the multi-scale shared features into the mold classification branch and the salt spray segmentation branch respectively, and output the two types of recognition results simultaneously after one forward reasoning; Mold classification branch: The multi-scale shared features are successively processed through global average pooling, fully connected layer and Softmax normalization to obtain the probability distribution of mold growth level, and the 0~4 level mold rating is obtained by mapping according to the GJB 150.10A-2009 grading standard. Salt spray segmentation branch: The UNet++ deep supervised segmentation network is used to perform semantic segmentation on multi-scale shared features to obtain segmentation results of multiple types of corrosion defects. The output includes substrate corrosion area, blistering, pitting, corrosion products, peeling, and anodic / cathode coating defects. The corrosion area is calculated, and the corrosion ratio is converted according to the corresponding standard GB / T 6461-2002. The salt spray corrosion detection results of the sample are then output. Step 5, Standard Result Output: Integrate mold growth level and salt spray corrosion rating to generate test results that conform to the corresponding standard format. Through the built-in corresponding national and military standard grading mapping standards, the mold level, mold description, salt spray protection rating, salt spray appearance rating, and salt spray comprehensive rating are standardized and organized.

2. The method for identifying mold and salt spray defects based on Dense-CBAM-UNet++ dual-branch fusion according to claim 1, characterized in that: The edge shielding process in step 2 strictly matches the effective detection area requirements of GB / T 6461-2002 and GJB 150.10A-2009, retaining only the main detection surface of the sample for subsequent identification calculations.

3. The method for identifying mold and salt spray defects based on Dense-CBAM-UNet++ dual-branch fusion according to claim 1, characterized in that: In step 4, the salt spray segmentation branch involves area percentage calculation and protection rating R. P The specific rules are as follows: The percentage of pixels with corrosion defects in the total number of pixels in the effective detection area of ​​the sample is denoted as corrosion area percentage A. If A=0, then the protection rating R P =10; If A > 0, then according to formula R P =3(2-logA) is used for calculation. After rounding the result, it is limited to the range of 0 to 10 to obtain the protection rating R. P .

4. The method for identifying mold and salt spray defects based on Dense-CBAM-UNet++ dual-branch fusion according to claim 1, characterized in that: In the salt spray segmentation branch of step 4, the appearance rating R A It consists of a combination of area level, severity indicator, and defect type code; The defect type code corresponds to Class AJ defects in GB / T 6461-2002, and the severity is divided into four levels: very mild, mild, moderate and severe. When multiple types of defects exist, multiple sets of appearance rating combinations will be output accordingly.

5. The method for identifying mold and salt spray defects based on Dense-CBAM-UNet++ dual-branch fusion according to claim 1, characterized in that: In the shared encoder, a DenseBlock dense connection module is connected in series after each downsampling layer, and the output of each DenseBlock is connected to a CBAM attention module. The DenseBlock realizes the reuse and transfer of shallow texture features and deep semantic features through cross-layer dense connections. The CBAM attention module completes feature weight calibration in sequence through the channel attention submodule and the spatial attention submodule.

6. The method for identifying mold and salt spray defects based on Dense-CBAM-UNet++ dual-branch fusion according to claim 1, characterized in that: The method also includes a model pre-training step, specifically: A sample dataset was constructed, in which mold samples were labeled with growth levels 0 to 4 according to GJB 150.10A-2009, and salt spray samples were labeled with defect areas, defect types and area proportions according to GB / T 6461-2002. Data augmentation processing is performed on the sample dataset, including rotation, flipping, Gaussian noise, brightness transformation, and blurring. The network was trained using a multi-task joint loss function, where the mold classification branch used cross-entropy loss and the salt spray segmentation branch used a combination of Dice loss, cross-entropy loss and area regression loss. After training, the model is lightweighted through channel pruning and knowledge distillation.

7. The method for identifying mold and salt spray defects based on Dense-CBAM-UNet++ dual-branch fusion according to claim 6, characterized in that: Training was performed using the AdamW optimizer with an initial learning rate set to 1e-4. The model evaluation metrics include mAP@0.5, rank accuracy, and area calculation error, with the area calculation error controlled within ±0.05%.

8. The method for identifying mold and salt spray defects based on Dense-CBAM-UNet++ dual-branch fusion according to claim 1, characterized in that: The method also includes generating test reports and storing and tracing data; The output data from step 5 will be automatically used to generate a sample testing report. The system stores the original image, preprocessed image, feature data, rating results, and detection report in a unified manner, supporting subsequent data retrieval.

9. A mold and salt spray defect identification system based on Dense-CBAM-UNet++ dual-branch fusion, characterized in that: It includes an image acquisition module, a preprocessing module, a shared feature extraction module, a dual-branch parallel decoding module, a standard result output module, and a report storage and backtracking module; The image acquisition module is used to acquire sample images through a multi-dimensional image sensor and establish an original image sequence; The preprocessing module is connected to the image acquisition module and is used to perform size normalization, Gaussian noise reduction, white balance correction and edge masking on the original image in sequence, and output the preprocessed image. The shared feature extraction module is connected to the preprocessing module and has a built-in shared encoder for receiving the preprocessed image and extracting multi-scale shared features. The shared encoder has a DenseBlock dense connection module and a CBAM channel-space joint attention module. The DenseBlock dense connection module realizes multi-layer feature reuse and alleviates gradient vanishing. The CBAM channel-space joint attention module adaptively enhances the features of the defect region from the channel dimension and the spatial dimension. The dual-branch parallel decoding module includes a mold classification branch unit and a salt spray segmentation branch unit. The mold classification branch unit and the salt spray segmentation branch unit are respectively connected to the shared feature extraction module, which can simultaneously output two types of recognition results based on multi-scale shared feature input to the dual-branch decoder inference. The standard result output module is connected to the mold classification branch unit and the salt spray segmentation branch unit respectively, and is used to integrate the mold growth level and the salt spray corrosion rating to generate test results that conform to the corresponding standard format. The report storage and traceability module is connected to the standard result output module and is used to automatically generate sample test reports, uniformly store original images, preprocessed images, feature data, rating results and test reports, and support the retrieval and traceability of test data.

10. The mold and salt spray defect identification system based on Dense-CBAM-UNet++ dual-branch fusion according to claim 9, characterized in that: Each downsampling layer in the shared encoder is followed by a DenseBlock dense connection module, and the output of each DenseBlock dense connection module is configured with a CBAM attention module. The DenseBlock dense connection module achieves the reuse and transfer of shallow texture features and deep semantic features through cross-layer dense connections. The CBAM attention module completes feature weight calibration in sequence through the channel attention submodule and the spatial attention submodule.

11. The mold and salt spray defect identification system based on Dense-CBAM-UNet++ dual-branch fusion according to claim 9, characterized in that: The mold classification branch unit is configured with a global average pooling unit, a fully connected layer unit, and a Softmax normalization unit. The global average pooling unit and the fully connected layer unit perform feature mapping on multi-scale shared features, and the Softmax normalization unit outputs the probability distribution corresponding to each mold growth level, and outputs a mold rating of 0 to 4 according to the GJB 150.10A-2009 grading rules.

12. The mold and salt spray defect identification system based on Dense-CBAM-UNet++ dual-branch fusion according to claim 9, characterized in that: The salt spray segmentation branch unit is configured with a UNet++ deep supervised segmentation network, which is used to perform semantic segmentation on the multi-scale shared features output by the shared feature extraction module to obtain corrosion defect segmentation results. The salt spray segmentation branch unit is also equipped with a corrosion rating calculation unit and an appearance rating unit. The corrosion rating calculation unit is used to calculate the percentage of corrosion defect pixels in the total number of pixels in the effective detection area of ​​the sample, and obtain the corrosion area ratio A. If A=0, then the protection rating R P =10; If A > 0, then according to formula R P =3(2-logA) is used for calculation. After rounding the result, it is limited to the range of 0 to 10 to obtain the protection rating R. P ; The appearance rating unit is used to combine area grade, severity label, and defect type code to generate an appearance rating R. A The defect type code corresponds to the AJ class defect specified in GB / T 6461-2002, and the severity is divided into four levels: very minor, minor, moderate and severe. If there are multiple types of defects, multiple sets of appearance rating combinations will be output.

13. The mold and salt spray defect identification system based on Dense-CBAM-UNet++ dual-branch fusion according to claim 9, characterized in that: The system also includes a model training module, which includes a sample labeling subunit, a data augmentation subunit, a multi-task loss training subunit, and a model lightweighting subunit. The sample labeling subunit is used to construct the sample dataset. Mold samples are labeled with growth levels 0 to 4 according to GJB 150.10A-2009, and salt spray samples are labeled with defect areas, defect types and area proportions according to GB / T 6461-2002. The data augmentation subunit is used to perform data augmentation processing on the sample dataset, including rotation, flipping, Gaussian noise, brightness transformation, and blurring. The multi-task loss training subunit uses a multi-task joint loss function to train the network, wherein the mold classification branch uses cross-entropy loss, and the salt spray segmentation branch uses a combination of Dice loss, cross-entropy loss and area regression loss. The model lightweighting subunit is used to perform model lightweighting through channel pruning and knowledge distillation after training.

Images