A quantitative identification method of wear type based on image segmentation

Abstract

This invention discloses a quantitative identification method for wear types based on image segmentation, belonging to the field of image recognition technology. The method includes acquiring several wear mark images to obtain a dataset containing wear region mask information; training a semantic segmentation network based on the dataset, and combining hyperparameter optimization and cross-validation strategies to obtain a globally optimal wear image semantic segmentation model. The semantic segmentation network includes a feature extraction module embedded with a CBAM composite attention unit, a hollow spatial pyramid pooling module, a feature fusion module, and an output layer; using the trained semantic segmentation network to detect the wear mark SEM image to be detected, outputting wear type detection results, and labeling the regions of each wear type in the detection results. Simultaneously, based on pixel statistics, the proportion of different wear types is output. This invention can quickly, objectively, and quantitatively identify and determine various wear categories.

Description

Technical Field

[0001] This invention discloses a method for quantitative identification of wear type based on image segmentation, which relates to the fields of image recognition and wear mechanism analysis technology. Background Technology

[0002] Identification and quantitative analysis of wear mechanisms are crucial aspects of materials tribology research. Traditional wear track analysis primarily relies on manual observation and experience-based judgment using scanning electron microscopy (SEM) images. Researchers infer wear types and mechanisms through subjective identification of wear track morphology features (such as grooves, delamination, tearing, and adhesion). While this method is feasible for qualitative analysis, in mixed wear systems, different wear types often occur simultaneously and overlap within the same wear track area. Manual identification is not only time-consuming, labor-intensive, and inefficient, but also highly susceptible to the influence of operator experience and visual judgment, leading to highly subjective results, poor repeatability, and difficulty in objectively quantifying the wear process.

[0003] To improve recognition efficiency, some studies have attempted to introduce image processing and statistical methods to classify, perform grayscale analysis, or threshold segmentation of wear track morphology features. However, these methods based on traditional image processing algorithms are highly sensitive to image quality and parameter settings, making it difficult to accurately distinguish regions with similar morphological features, such as abrasive wear, tribochemical reactions, and delamination wear. Furthermore, traditional methods typically only achieve a rough classification of wear features, lacking an understanding of mixed wear track regions and failing to effectively address the recognition needs of complex mixed wear.

[0004] In recent years, with the development of deep learning and computer vision technologies, semantic segmentation models have demonstrated excellent feature extraction and pixel-level classification capabilities in fields such as medical imaging, remote sensing recognition, and material microscopic image analysis. This technology can simultaneously consider the global structure and local details of an image, achieving automatic segmentation and precise recognition of complex microscopic morphologies. However, existing research mainly focuses on qualitative segmentation of wear areas or automatic classification and detection of wear mark morphology, with limited research on quantitative recognition and mechanism determination under conditions of multiple wear types coexisting. Especially in mixed wear systems, how to accurately divide regions of different wear types using semantic segmentation technology and achieve automated statistical and quantitative assessment of wear ratios still lacks a systematic and effective solution.

[0005] Therefore, there is an urgent need for a quantitative identification method for wear types that can balance identification accuracy and judgment objectivity. Summary of the Invention

[0006] In view of this, the purpose of this invention is to overcome the problems of strong subjectivity of manual judgment, low recognition efficiency, and difficulty in quantitative analysis of mixed wear mechanisms in the traditional wear mark recognition process, and to provide a quantitative recognition method for wear type based on image segmentation.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] This application provides a method for quantitative identification of wear type based on image segmentation, including the following steps:

[0009] S1. Acquire several wear marks images, and divide and label the wear marks images to form a dataset containing wear area mask information;

[0010] S2. Divide the dataset into a training set and a validation set, and use the training set to train the semantic segmentation network. During the training process, use hyperparameter optimization and cross-validation strategies to optimize the semantic segmentation network to obtain the globally optimal wear image semantic segmentation model. The semantic segmentation network includes a feature extraction module with embedded CBAM composite attention units, a hollow spatial pyramid pooling module, a feature fusion module, and an output layer.

[0011] S3. The trained semantic segmentation network is used to detect the wear marks in the SEM image to be detected, and the segmentation mask image of the wear marks in the SEM image to be detected is output. The regions of each wear type in the segmentation mask image are marked, and the proportion of different wear types is output based on pixel statistics.

[0012] Furthermore, obtaining the wear mark image in step S1 specifically includes:

[0013] Scanning electron microscope images of the worn sample were selected as the original wear marks images;

[0014] The original wear marks images were then filtered to remove those that were contaminated, abnormally exposed, or whose typical wear types could not be distinguished, and these were used as the final wear marks images.

[0015] Furthermore, the process of dividing and labeling the wear mark image in step S1 specifically includes:

[0016] Based on the preset wear type labeling standard and using the deep learning image labeling software labelme, the wear mark image is divided into regions and semantically labeled to obtain a dataset containing wear region mask information. The wear types include: abrasive wear region, delamination wear region, tribochemical reaction wear region and non-wear background region.

[0017] Furthermore, step S1 also includes:

[0018] Data augmentation is performed on the labeled images in the dataset using rotation, cropping, Gaussian blur, and / or contrast enhancement algorithms.

[0019] Furthermore, in S2, the MobileNet backbone network is used to extract features from different levels of the input scratch image to obtain corresponding convolutional features, including shallow features, mid-level features, and deep features;

[0020] The intermediate-layer features are input into the first CBAM composite attention unit for feature enhancement to obtain the enhanced intermediate-layer features;

[0021] The deep features are input into the second CBAM composite attention unit for feature enhancement to obtain enhanced deep features;

[0022] The enhanced deep features are fused and enhanced through a hollow spatial pyramid pooling module to obtain enhanced multi-scale deep features;

[0023] The enhanced multi-scale features output by the void space pyramid pooling module are input into the feature fusion module for upsampling. The sampling results are then fused with the mid-layer features enhanced by the first CBAM composite attention unit. The fused features are further upsampled, and the further sampling results are fused with the shallow features to obtain the final fused features.

[0024] The output layer obtains the segmentation mask result based on the final fusion features.

[0025] Furthermore, both the first CBAM composite attention unit and the second CBAM composite attention unit include channel attention subunits and spatial attention subunits connected in series.

[0026] The channel attention subunit takes the convolutional features output by the feature extraction module as input, performs max pooling and average pooling operations on the convolutional features to obtain the corresponding channel description vectors; the obtained channel description vectors are input into the multilayer perceptron module for feature mapping, the feature mapping results are summed and then the channel weight coefficients are generated through the activation function; the channel weight coefficients are multiplied by the convolutional features channel by channel to obtain the channel weighted features;

[0027] The spatial attention subunit takes the channel-weighted features output by the channel attention subunit as input, performs average pooling and max pooling operations on each channel, and concatenates the obtained average pooling and max pooling results. The concatenated result is then used to generate a spatial attention weight map through convolution. The spatial attention weight map is then multiplied pixel-by-pixel with the input channel-weighted features to obtain the enhanced features after spatial-channel dual weighting.

[0028] Furthermore, in S2, the semantic segmentation network is optimized using hyperparameter optimization and cross-validation strategies, specifically including:

[0029] Identify the hyperparameters to be optimized;

[0030] Set a hyperparameter optimization target and use a pruning tool to prune hyperparameter combinations Trial whose optimization effect is below the threshold.

[0031] After each round of training, the performance metrics of the validation set corresponding to the combination are recorded. The hyperparameter optimization model automatically selects the optimal hyperparameter combination for subsequent cross-validation training.

[0032] Each search experiment performs 5-fold cross-validation. The labeled dataset obtained in step S1 is randomly divided according to distribution constraints. On each fold dataset, a semantic segmentation model is independently trained based on the model parameters optimized by hyperparameters. The training results of each fold are evaluated according to the preset model performance index, and the model with the best performance is selected as the global optimal model.

[0033] Furthermore, in the step of determining the hyperparameters to be optimized, the hyperparameters to be optimized specifically include: model downsampling factor, number of training epochs frozen, batch size during the freeze and unfreeze phases, total number of training epochs, initial learning rate, learning rate decay method, and optimizer type.

[0034] Furthermore, in the step of evaluating the training results of each fold according to preset model performance metrics, the model performance metrics include average intersection-over-union ratio, training set and validation set loss, and global pixel-level classification accuracy. Each model performance metric is obtained as follows:

[0035] The formula for calculating the average crossover ratio is as follows:

[0036] ;

[0037] ;

[0038] in, This represents the number of wear type categories; in one specific implementation, C=4. For type The intersection and union ratio, Class C The average value, , , These represent the results of the model prediction and the actual annotation in the validation set, respectively. The number of true positive pixels, false positive pixels, and false negative pixels for each wear type;

[0039] Training set and validation set loss:

[0040] Both the training and validation set losses use a hybrid loss function, the formula of which is:

[0041] ;

[0042] in The cross-entropy loss function represents the difference between predicted and true label values. Let represent the similarity loss function between the predicted value and the true label value;

[0043] Accuracy: Global pixel-level classification accuracy, the formula is:

[0044] ;

[0045] in, The total number of true positive pixels across all categories. This represents the total number of pixels in a single image.

[0046] After each fold of training, record the mIoU, accuracy, and loss curves of the validation set, and select the model with the highest mIoU as the optimal model for that fold.

[0047] Furthermore, in S3, marking the wear type regions in the segmented mask image specifically includes:

[0048] Based on the obtained pixel-level wear type segmentation mask image, regions of different wear types are marked by differential colors;

[0049] The color-marked segmentation mask image is overlaid and fused with the original SEM image at a preset transparency to generate an original image-segmentation mask fusion visualization image used to display the model recognition results.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] This invention utilizes a wear image semantic segmentation model containing a CBAM composite attention unit to detect wear marks, thereby improving the accuracy of wear detection.

[0052] This invention can quickly distinguish various wear morphologies such as abrasive wear, delamination wear, and tribochemical reactions, and can also achieve quantitative statistics on the proportion of various wear regions, thus providing a more efficient, objective, and repeatable means of wear mechanism analysis for tribological experiments. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0054] Figure 1 This is a flowchart of the quantitative identification method in Embodiment 1 of the present invention.

[0055] Figure 2 The recognition method of the present invention, provided for implementation of the invention, outputs a recognition result image for a mixed wear scenario in SEM images, which is abrasive wear and tribochemical reaction.

[0056] Figure 3 This is a schematic diagram illustrating the identification results of the identification method of the present invention for another mixed wear condition of abrasive wear and tribochemical reaction in SEM images, provided in an embodiment of the present invention.

[0057] Figure 4 The identification method of the present invention, provided in the embodiments of the present invention, presents the identification result of a typical scene of large-area layered wear in SEM images.

[0058] Figure 5 This is a schematic diagram illustrating the identification results of the identification method of the present invention for mixed wear conditions of layered wear and tribochemical reaction in SEM images, as provided in the embodiments of the present invention. Detailed Implementation

[0059] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0060] The term "embodiment" used herein, as an example, is not necessarily to be construed as superior to or better than other embodiments. Performance testing in the embodiments of this application, unless otherwise specified, employs conventional testing methods in the art. It should be understood that the terminology used in this application is merely for describing particular implementations and is not intended to limit the scope of this disclosure.

[0061] Unless otherwise stated, the technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; other experimental methods and technical means not specifically mentioned herein refer to experimental methods and technical means commonly used by one of ordinary skill in the art.

[0062] To better illustrate the content of this application, numerous specific details are provided in the following detailed embodiments. Those skilled in the art should understand that this application can be implemented even without certain specific details. In the embodiments, some methods, means, instruments, and devices well-known to those skilled in the art are not described in detail in order to highlight the main points of this application.

[0063] Without conflict, the technical features disclosed in the embodiments of this application can be combined arbitrarily, and the resulting technical solution belongs to the content disclosed in the embodiments of this application.

[0064] To better understand the present invention, the following embodiments are provided for further detailed description of the present invention, but they should not be construed as limiting the present invention. Any non-essential improvements and adjustments made by those skilled in the art based on the above-described invention are also considered to fall within the protection scope of the present invention.

[0065] A method for quantitative identification of wear type based on image segmentation includes the following steps:

[0066] S1. Acquire several wear marks images, and divide and label the wear marks images to form a dataset containing wear area mask information;

[0067] S2. Divide the above dataset into a training set and a validation set, and use the training set to train the semantic segmentation network. Combine automated hyperparameter optimization and cross-validation strategies to obtain the trained globally optimal wear image semantic segmentation model. The semantic segmentation network includes a feature extraction module with embedded CBAM composite attention units, a hollow spatial pyramid pooling module, a feature fusion module, and an output layer.

[0068] S3. Use the trained semantic segmentation network to detect wear marks in the SEM image and output the wear type detection results.

[0069] Furthermore, step S1 specifically includes:

[0070] S11. SEM Image Acquisition and Screening: Scanning electron microscope (SEM) images of the worn samples are selected as the data source. To ensure that the images fully reflect the morphological characteristics and mechanistic differences of the worn areas, the magnification of the selected images is preferably 1000x-5000x to ensure clear wear details and discernible surface textures. The acquired images are screened to remove image samples that are contaminated, abnormally exposed, or cannot distinguish typical wear types, in order to improve the effectiveness and accuracy of subsequent semantic annotation and model training.

[0071] S12. Wear Image Classification and Labeling: Common wear types in wear scenes are classified into three categories: abrasive wear, delamination wear, and tribochemical wear. Areas without obvious wear marks or outside of wear marks are classified as background. There are a total of four categories, and the specific discrimination criteria for each category are shown in Table 1.

[0072] Table 1. Criteria for Determining Wear Type

[0073]

[0074] Based on the aforementioned criteria, the selected dataset was divided into regions and semantically labeled using the deep learning image annotation software LabelMe, resulting in a dataset containing wear region mask information. The annotation rules were as follows: abrasive wear regions were labeled as 1, delamination wear regions as 2, tribochemical reaction regions as 3, and other regions were not labeled and were considered background with a default value of 0. During the annotation process, three researchers with expertise in materials wear independently annotated the data. Discrepancies in annotation were resolved through collective discussion to determine the final annotation results, ensuring accuracy and professional consistency.

[0075] Furthermore, step S2 specifically includes:

[0076] S21. Dataset Augmentation and Partitioning: The dataset containing wear area mask information obtained in step S1 is augmented using methods such as rotation, cropping, Gaussian blurring, and contrast enhancement. The augmented dataset is divided into training and validation sets in an 8:2 ratio and organized according to the VOC semantic segmentation standard format to form a standardized dataset that can be used for semantic segmentation tasks. The training set is used for model training, and the validation set is used for hyperparameter optimization and performance evaluation.

[0077] S22. Network Architecture and Parameter Configuration: A semantic segmentation network is used as the baseline architecture, with a fixed input sub-image resolution of 512×512. The network output layer is set as a four-channel convolutional structure, corresponding to three wear types and background categories. The network consists of a feature extraction module, a dilated spatial pyramid pooling module, a feature fusion module, and an output layer. To achieve accurate extraction of key material wear features and suppress background noise, a CBAM composite attention module is embedded in the backbone network of the feature extraction module. The network finally uses the Softmax activation function to output the category probability of each pixel.

[0078] In the semantic segmentation network disclosed in this invention, an RGB wear sub-image with a resolution of 512×512 is input. First, a lightweight MobileNet backbone network, such as MobileNetV3-Large, is used to extract features from the RGB wear sub-image, outputting feature maps at multiple scales. In this embodiment, three different scales of features are mainly utilized: shallow features with high spatial resolution, fewer channels, and containing detailed information; mid-level features with medium spatial resolution, a moderate number of channels, and containing moderate semantic information; and deep features with low spatial resolution, a large number of channels, and containing rich semantic information.

[0079] The first CBAM module is embedded after the output layer of the MobileNet backbone network that outputs mid-level features to enhance the representation of mid-level features, and the second CBAM module is embedded after the output layer that outputs deep features to enhance the representation of deep features.

[0080] The Spatial Pyramid Pooling (ASPP) module receives deep features enhanced by the second CBAM. The ASPP captures multi-scale contextual information through multiple parallel convolutional layers with different dilation rates and fuses the outputs of multiple parallel branches to obtain enhanced multi-scale features.

[0081] The feature fusion module upsamples the enhanced multi-scale features output by the ASPP module and fuses them with the mid-level features enhanced by the first CBAM. The fused features are then further upsampled and fused with low-level features to recover spatial details. Convolutional operations may be performed after each fusion to integrate the features.

[0082] The output layer upsamples the output of the feature fusion module to the input image size (512×512), uses a four-channel convolutional layer, followed by a Softmax activation function, and outputs the confidence score of each pixel belonging to one of the four categories.

[0083] The following section provides a more detailed introduction to each module of the semantic segmentation network.

[0084] The MobileNetV3-Large backbone takes a 512×512×3 RGB wear sub-image as input and performs three-stage feature extraction. The first stage is shallow feature extraction. In the shallow feature extraction unit, the first layer uses a 3×3 convolution with a stride of 2 and an output of 16 channels to perform a convolution operation on the input image. The result of the convolution operation is then processed through batch normalization and h-swish activation to obtain a 256×256×16 feature map. The second layer, while maintaining the same resolution and number of channels, uses an inverse residual bottleneck block to perform nonlinear transformation and enhancement of the features. The 256×256×16 feature map is then processed using the first inverse residual bottleneck block. The first layer expands the 256×256×16 extended features using depthwise separable convolutions (including depthwise and pointwise convolutions) and a Squeeze-and-Excitation (SE) attention module. The second layer uses a second inverse residual bottleneck block to downsample and expand the channels of the 256×256×16 extended features. First, pointwise convolutions expand the number of channels (expansion ratio of 4, i.e., to 64 channels). Then, depthwise convolutions with a stride of 2 halve the spatial resolution (from 256×256 to 128×128). Finally, pointwise convolutions compress the number of channels to 24, resulting in a shallow feature layer with a resolution of 128×128. Because the stride is 2 and the input and output channels are different, the second inverse residual bottleneck block does not use residual connections.

[0085] Similarly, in the second stage of the MobileNetV3-Large backbone network, the shallow features from the first stage are passed through a series of inverted residual bottleneck blocks to gradually increase the number of channels. The number of output channels increases from 24 to 40. After downsampling with a step size of 2, the resolution is reduced to 64×64, and the number of output channels is 40, resulting in a mid-level feature with a resolution of 64×64.

[0086] In the third stage, the 8th to 11th inverse residual bottleneck blocks of MobileNetV3-Large are all activated using the h-swish activation function. Through multiple channel upscaling and downscaling adjustments, the number of channels is gradually increased to 160. To maintain an output stride of 16, the last few blocks use dilated convolutions (dilation rate 2), resulting in a final output of 160 channels and a deep feature resolution of 32×32.

[0087] In this invention, the backbone network embeds two CBAM (Concurrent Attention Units). The first CBAM unit module is embedded after the mid-level features, performing a composite attention operation on the mid-level features with a resolution of 64×64, and outputting enhanced mid-level features. The second CBAM unit module is embedded after the high-level features, performing a composite attention operation on the deep features with a resolution of 32×32, and outputting enhanced deep features. To retain more details, the shallow features with higher spatial resolution are not subjected to composite attention operations in this invention.

[0088] Each CBAM composite attention unit comprises channel attention subunits and spatial attention subunits connected in series. The channel attention subunit takes the convolutional features output from the feature extraction module as input, performs max pooling and average pooling operations on these features to obtain corresponding channel description vectors. These channel description vectors are then input into a multilayer perceptron module for feature mapping. The feature mapping results are summed and then processed through an activation function to generate channel weight coefficients. These channel weight coefficients are then multiplied with the convolutional features channel-wise to obtain channel-weighted features. The spatial attention subunit takes the channel-weighted features output from the channel attention subunits as input, performs average pooling and max pooling operations on each channel, and concatenates the average pooling and max pooling results. The concatenated result is then processed through convolution to generate a spatial attention weight map. The spatial attention weight map is then multiplied pixel-wise with the input channel-weighted features to obtain the enhanced features after spatial-channel dual weighting.

[0089] In the Spatial Pyramid Pooling (ASPP) module, the deep features enhanced by CBAM are processed by multiple parallel dilated convolutional branches with different dilation rates to obtain global information of different receptive field sizes for the input deep features. Each branch uses a 3×3 convolution followed by batch normalization and ReLU activation. Specifically, the Spatial Pyramid Pooling module includes: Branch 1, which uses a 1×1 convolutional layer to output 256 channels of original scale features at 32×32 resolution; Branch 2, which uses a 3×3 dilated convolution with a dilation rate of 6 to output 256 channels of medium-scale features at 32×32 resolution to extract contextual information of medium receptive fields; Branch 3, which uses a 3×3 dilated convolution with a dilation rate of 12 to output 256 channels of features at 32×32 resolution to extract contextual information of large receptive fields; and Branch 4, which uses a 3×3 dilated convolution with a dilation rate of 18 to output 256 channels of features at 32×32 resolution to extract global information of extremely large receptive fields.

[0090] Simultaneously, a global average pooling branch is used to pool the feature map into a 1x1 format. Then, a 1x1 convolution is used to adjust the number of channels, and the image is upsampled back to its original size. Finally, the features output from multiple dilated convolution branches and the global pooling branch are concatenated and fused along the channel dimension. A 1×1 convolutional layer is then used to reduce the dimensionality of the channels, followed by batch normalization and ReLU activation to obtain 256 channels and 32×32 resolution enhanced multi-scale features.

[0091] In semantic segmentation networks, the feature fusion module fuses the output of the dilated spatial pyramid pooling module and the features of the backbone network. Specifically, firstly, the enhanced multi-scale features output by ASPP are upsampled by a factor of 2 to 64×64×256. Then, the upsampled features are concatenated with the 64×64 mid-layer features enhanced by the first CBAM, resulting in a 64×64×368 feature map. The concatenated features are then subjected to 1x1 convolutional dimensionality reduction (reduced to 128 channels) and 3×3 convolutional refinement, resulting in a 64×64×128 feature map. Next, the fused 64×64×128 feature map is upsampled by a factor of 2 to 128×128×128. Finally, the upsampled result is concatenated with the bottom-level features to obtain a 128×128×152 feature map. This feature map is also subjected to 1×1 convolutional dimensionality reduction and 3×3 convolutional refinement, resulting in a final fused feature map of 128×128×64.

[0092] In the semantic segmentation network output layer of this embodiment, the feature map output by the feature fusion module is upsampled by a factor of 4 to 512×512×64, and then passed through a 4-channel 1×1 convolutional layer to obtain a 512×512×4 feature map. A Softmax activation function is then applied to calculate the class probability of each pixel along the channel dimension, outputting a four-channel probability map, with each channel corresponding to the probability of one class.

[0093] The improved semantic segmentation network described above can fully explore and integrate the multi-level features extracted by the backbone network, and enhance key features and suppress noise through the attention mechanism, thereby improving the segmentation accuracy of worn areas.

[0094] S23. Perform model training: Based on the set network structure, number of categories, and hyperparameter settings, conduct model training and optimization on the standardized dataset.

[0095] During training, the system monitors and outputs the core performance metrics of the training and validation sets in real time, while setting the following quantitative standards: 1. Performance metric threshold: The overall recognition accuracy (mIoU) of each wear type on the validation set must be ≥80%; 2. Loss value stability threshold: The training loss value must converge to 0.3, and the validation set loss value must not decrease for 5 consecutive rounds to avoid overfitting or underfitting the model. When the model meets the above standards, the system automatically saves the corresponding training weight file and its performance data, and comprehensively selects the globally optimal model based on the evaluation results.

[0096] S24. Model Optimization: During model training, a Bayesian optimization-based hyperparameter search algorithm is employed. This algorithm optimizes the model's hyperparameters within a predefined search space comprised of hyperparameter categories and their value ranges, using the average intersection-union ratio (mIoU) of the validation set as the objective function. A pruning mechanism is introduced to further optimize the hyperparameters. Through multiple rounds of automated iterative search, the globally optimal hyperparameter combination is ultimately selected.

[0097] Preferably, the predefined hyperparameter categories in this invention include: model downsampling factor (Downsample_Factor), feature extractor freeze training epoch (Freeze_Epoch), freeze phase batch size (Freeze_Batch_Size), total training epochs (UnFreeze_Epoch), unfreeze phase batch size (Unfreeze_Batch_Size), initial learning rate (Init_Lr), learning rate decay method (Lr_Decay_Type), and optimizer type (Optimizer_Type).

[0098] The value ranges of the predefined hyperparameters are shown in Table 2:

[0099] Table 2. Range of Hyperparameter Values

[0100]

[0101] Each search experiment performs 5-fold cross-validation, and uses the mIoU of each fold as the objective function to dynamically update the search space and determine the globally optimal combination of hyperparameters.

[0102] The above strategies yield the globally optimal semantic segmentation model for wear images.

[0103] S25. Hybrid Loss Function and Performance Evaluation:

[0104] Loss function: The loss function is used to calculate the difference between the predicted value and the true value, which guides the training process of the model.

[0105] As a preferred approach, "labeled smooth cross-entropy loss" is adopted. ) + Weighted Dice loss ( The mixed loss of ")" is used as the final loss function, and the formula is:

[0106] ;

[0107] in, Mitigating class imbalance through a label smoothing loss function:

[0108] ;

[0109] in, This represents the smoothed label distribution, where C is the total number of categories. The probability that the i-th sample belongs to class x as predicted by the model; the smoothed label distribution can be obtained by the following formula:

[0110]

[0111] in, Let c represent the true label of the i-th sample, and c represent the class index. This represents the smoothing factor, which can be between 0.1 and 0.2.

[0112] Weighted Dice loss Strengthen the learning of small percentages of wear samples by using category weights:

[0113]

[0114] in, Indicates the category weight.

[0115] Performance Evaluation: Key performance metrics are monitored in real time during training, including: a 4×4 confusion matrix (providing a pixel-level classification basis for subsequent metric calculations), mean intersection-over-union ratio (mIoU, a core global segmentation metric reflecting overall class segmentation accuracy), global pixel accuracy (reflecting the overall correctness of pixel classification across the entire image), and training and validation set loss curves (reflecting the model's convergence trend and generalization ability). After each fold of training, the model weights with the best mIoU on the validation set are saved. After all folds of training are completed, the model with the highest mIoU among the 5 folds is selected as the globally optimal semantic segmentation model.

[0116] The specific calculation methods for average intersection-over-union ratio and global pixel accuracy during performance evaluation are as follows:

[0117] The formula for calculating the average intersection-union ratio (mIoU) is as follows:

[0118] ;

[0119] ;

[0120] in, This represents the number of wear type categories; in one specific implementation, C=4. For type The intersection and union ratio, Class C The average value, , , These represent the results of the model prediction and the actual annotation in the validation set, respectively. The number of true positive pixels, false positive pixels, and false negative pixels for each wear type;

[0121] Accuracy: Global pixel-level classification accuracy, the formula is:

[0122]

[0123] in, The total number of true positive pixels across all categories. This represents the total number of pixels in a single image.

[0124] After each fold of training is completed, record the mIoU, accuracy, and training / validation set loss curves on the validation set, and select the model with the highest mIoU as the optimal model for that fold.

[0125] Furthermore, step S3 specifically includes:

[0126] After the SEM image to be detected is filtered and preprocessed in step S1, it is input into the global optimal model obtained in step S2 to obtain the pixel-level segmentation mask image of the SEM image. The image uses three types of differential colors to mark three types of wear, and the background area is not marked with color. The segmentation mask image is then superimposed and fused with the original SEM image with 50% transparency to form the "original SEM image-mask fusion visualization image".

[0127] Data labels are embedded in the upper left corner of the above image. The label content is the percentage of each wear type calculated based on pixel statistics: the number of pixels of abrasive wear (P1), delamination wear (P2), and tribochemical reaction (P3) in the segmented mask image. The total number of wear pixels P_total = P1 + P2 + P3 is calculated. The percentages are calculated according to the formulas: abrasive wear % = (P1 / P_total) × 100%, delamination wear % = (P2 / P_total) × 100%, tribochemical reaction % = (P3 / P_total) × 100%.

[0128] The final output is a "raw image-mask hybrid visualization image (including quantitative data labels)," which enables intuitive identification and accurate quantification of various categories under mixed wear scenarios, providing a "visualization + numerical" basis for material wear state assessment and performance analysis.

[0129] Example 1

[0130] This embodiment addresses the scenario of quantitative identification of mixed wear patterns in wear-marked SEM images, covering the entire process from data preparation to result output. It is suitable for rapid analysis of single samples in experiments. Figure 1 This is a flowchart illustrating the operation of the quantitative identification method of the present invention.

[0131] The core parameters and operations are as follows:

[0132] Model training environment: CPU is Intel i9-14900K, GPU is NVIDIA RTX4090, deep learning framework is PyTorch 1.10.0, CUDA version 11.3;

[0133] Dataset preparation: Wear surfaces under different wear conditions were photographed using a scanning electron microscope at magnifications of 1000x-5000x, resulting in images with a resolution of 512. Wear images of 512 were used; the wear regions of the augmented SEM images were labeled using LabelMe, with three researchers with expertise in materials wear independently labeling the images to ensure accuracy and consistency; the image set was augmented using image rotation, random cropping, and random Gaussian blur; the labeled dataset containing wear region mask information was randomly divided into training and validation sets in an 8:2 ratio and organized according to the VOC semantic segmentation format;

[0134] Model Training and Optimization: A semantic segmentation model for wear-related SEM images was constructed using a dilated convolution-based semantic segmentation network as the baseline architecture. Input images were preprocessed and uniformly adjusted to a 512×512 pixel RGB three-channel format. A CBAM composite attention module was embedded after the output layer of the feature extraction backbone network to enhance wear-related features and suppress background interference. The network adopted a four-channel convolutional structure (corresponding to three wear types and background) and output the class probability of each pixel through a Softmax activation function.

[0135] The parameters of the trained model are shown in Table 3:

[0136] Table 3 Training Model Parameter Table

[0137]

[0138] During model training, a Bayesian optimization-based hyperparameter search algorithm is used to iteratively search for preset hyperparameter value combinations: the model performance under different hyperparameter combinations is calculated through multiple rounds of trials, the trial combination with the best performance on the validation set is selected and its hyperparameters are saved; at the same time, combined with a 5-fold cross-validation mechanism, the dataset is divided into 5 folds, each fold is trained and validated independently, and the stability of the performance between folds is evaluated. Finally, the global model is trained based on the optimal hyperparameters to ensure the model's generalization ability and the reliability of quantitative results in mixed wear recognition scenarios.

[0139] After model training and optimization, the performance of the globally optimal model corresponding to the final saved globally optimal hyperparameter combination is tested: independent validation set data is loaded, and core evaluation indicators such as average intersection-union ratio (mIoU), average accuracy (mAccuracy), and pixel-level confusion matrix of each type of wear are calculated through inference to comprehensively quantify the segmentation performance and quantitative accuracy of the model in the mixed wear recognition task.

[0140] Model Application: The system runs the model inference script, loads the globally optimal semantic segmentation model saved during training and optimization; after selecting the wear-mark SEM image to be analyzed, the system automatically standardizes the image to a size of 512×512 and performs pixel-level category recognition on the image through the model; subsequently, the system performs pixel-level category recognition on the preprocessed image through the model to generate a segmentation mask image containing abrasive wear (red marker), delamination wear (green marker), tribochemical reaction wear (yellow marker) and / or background. The system automatically counts the number of pixels for each type of wear, calculates the proportion of each type of wear based on the total number of wear pixels, and finally outputs a "mixed visualization image of original image and mask", with a quantitative proportion label embedded in the upper left corner, providing intuitive and accurate technical support for the mechanism analysis and performance evaluation of mixed wear.

[0141] To further demonstrate the beneficial effects of the present invention and to better understand it, the following experimental examples and comparative examples further illustrate the technical features disclosed in the present invention, but should not be construed as limiting the present invention. Other improvements made by those skilled in the art based on the above-described invention, without inventive effort, are also considered to fall within the protection scope of the present invention.

[0142] This embodiment focuses on the quantitative identification task of a single mixed wear SEM image, and finally obtains the core results shown in the attached figure, which fully verifies the effectiveness of the method of the present invention in the quantitative identification of mixed wear mechanisms:

[0143] Results Output: The globally optimal model obtained through training is applied to the quantitative identification of wear SEM images under different experimental conditions. The identification results under different conditions are visualized as follows: Figures 2-5 As shown, the specific manifestations are as follows:

[0144] Figure 2 The SEM image shown depicts a mixed wear scenario involving abrasive wear and tribochemical reactions. Semantic segmentation by the model identifies abrasive wear as a large red area, with pixel-level annotations precisely covering typical features such as the furrows of abrasive wear. The tribochemical reaction area is clearly presented in yellow, with a clear and unambiguous boundary between the two types of wear. Quantitative annotations in the upper left corner of the image show that abrasive wear accounts for 85.3%, the tribochemical reaction area accounts for 13.5%, and the background is extremely low, consistent with researchers' expectations of "significant dominant wear areas and minimal background interference." This identification result visually demonstrates the spatial distribution of mixed wear and, through precise proportion data, clarifies the mechanism conclusion that "abrasive wear is the main failure mode, accompanied by localized tribochemical reactions," fully validating the model's accuracy and quantitative reliability in identifying mixed wear scenarios.

[0145] Figure 3 The SEM image shown presents a mixed wear scenario of abrasive wear and tribochemical reaction. Semantic segmentation and identification by the model revealed that tribochemical reaction is dominated by a large yellow area in the result image, with pixel-level annotations accurately matching the typical microscopic features of tribochemical reactions, such as oxide layer fracture and fragmentation. Abrasive wear areas are clearly distinguished by red, showing a clear boundary between the two types of wear without category confusion. Quantitative annotations in the upper left corner of the result image show that tribochemical reaction accounts for 63.4%, while abrasive wear accounts for 35.0%. This identification result not only intuitively reconstructs the spatial distribution of mixed wear, but its quantitative accuracy and feature matching degree also meet researchers' expectations for mixed wear analysis, further validating the semantic segmentation accuracy and quantitative reliability of the model in mixed wear scenarios.

[0146] pass Figure 2 and Figure 3As can be seen from the comparison, the wear type quantitative identification method proposed in this invention can still achieve pixel-level accurate segmentation and quantitative statistics of each wear type even in mixed wear scenarios where the wear type composition is the same (both are abrasive wear + tribochemical reaction wear) but the proportion of the two is significantly different. This verifies the identification accuracy and robustness of the method under complex mixed wear conditions.

[0147] Figure 4 The SEM image shown presents a typical scenario of extensive layered wear. Through semantic segmentation and identification by the model, the layered wear area is dominated by large green regions. This not only achieves clear separation between the worn area and the unworn background, but the green-annotated areas also accurately cover characteristic regions such as the peeling of layers in the SEM image, with clear and unambiguous boundary delineation. Quantitative annotation of the results image shows that layered wear accounts for 78.6%, a precise figure that clearly confirms the conclusion that "layered wear is the main failure mode." The segmentation accuracy and quantitative performance meet the researchers' expectations for layered wear analysis, fully validating the model's reliability in scenarios dominated by a single wear category.

[0148] Figure 5 The SEM images shown present a mixed wear condition of layered wear and tribochemical reaction. Semantic segmentation by the model identifies layered wear as being dominated by large green areas, accurately covering its typical characteristics such as flaking. The yellowish-brown areas of tribochemical wear clearly demarcate from the layered wear boundary, without any category confusion. Quantitative annotation of the results shows that layered wear accounts for 50.7% and tribochemical wear accounts for 40.6%, clearly confirming the mechanism of "mixed layered wear and tribochemical reaction." The model accurately identifies the semantic boundaries of the two types of wear, and the proportion statistics are reliable, fully meeting the requirements of "category differentiation and quantitative accuracy" for mixed wear analysis in scientific research scenarios. This fully verifies the model's accuracy and quantitative reliability in identifying mixed wear scenarios.

[0149] In summary, the globally optimal semantic segmentation model of this invention can achieve pixel-level accurate identification of abrasive wear, delamination wear, and tribochemical reaction wear in mixed wear scenarios. Its labeled areas precisely correspond to the typical microscopic features of each type of wear. Through a visualization method of "original image-masked hybrid visualization image (including quantitative data labels)," the reliability of the model in identifying mixed wear scenarios is intuitively verified. The model exhibits high feature matching and clear boundary segmentation, fully meeting the requirements of scientific research-level analysis. For mixed wear regions with multiple coexisting mechanisms, the model can accurately define the semantic boundaries of various types of wear without issues such as category confusion or boundary ambiguity. It successfully achieves the core technical goal of "clearly distinguishing mixed wear regions," providing a stable and reliable segmentation foundation for the analysis of complex wear scenarios with multiple coupled mechanisms.

[0150] The top left corner of each fusion result image contains quantitative percentage data of various types of wear, realizing direct correlation and mutual verification between "quantitative data and visual features". This makes it easy for experimental personnel to quickly and accurately determine the dominant wear mechanism under different working conditions, significantly reducing the technical threshold for mixed wear mechanism analysis. At the same time, it provides intuitive and scientific technical support for the wear resistance assessment and working condition optimization of different materials.

[0151] In the scenario of batch wear SEM image recognition, the model's recognition efficiency reaches 40 images / second, which is significantly improved compared to the efficiency of manual analysis by human experts. It can efficiently complete the automated quantitative recognition of hundreds of wear samples under multiple working conditions, greatly shorten the data analysis cycle of mixed wear experiments, and fully meet the high-efficiency analysis needs of "large sample statistics and multi-working condition comparison" in experiments.

[0152] In summary, this application provides a method for quantitative identification of wear types based on image segmentation. Using wear mark SEM images as the core data source, it forms a complete hybrid wear identification technical solution by constructing wear type labeling standards, optimizing the dataset construction process, improving the semantic segmentation model, defining segmentation mask image specifications and quantitative result output methods. Ultimately, it achieves pixel-level accurate identification and quantitative calculation of different wear mechanisms in hybrid wear scenarios, effectively solving the technical defects of traditional identification methods such as low segmentation accuracy, difficulty in distinguishing hybrid wear types, and inaccurate quantitative calculation.

[0153] In terms of data processing and model training, this application performs pixel-level annotation on images according to preset annotation rules and designs a suitable model training process. Combined with hyperparameter optimization and cross-validation strategies, it significantly improves the stability and generalization performance of the model in complex wear scenarios, providing high-quality data and training support for subsequent accurate segmentation.

[0154] At the level of network structure improvement, this application optimizes the semantic segmentation network and introduces the CBAM composite attention module. By strengthening the key texture features of wear marks and suppressing background noise interference, it effectively improves the recognition defects of traditional models in terms of blurred wear boundaries and weakened features, and further improves the pixel-level accuracy of wear mark image segmentation and the reliability of wear area recognition.

[0155] Ultimately, this application outputs a "mixed visualization image of the original image and mask" along with the pixel percentage results for each wear type, achieving rapid, objective, and quantitative identification of mixed wear. This provides an efficient and reliable technical means for wear mechanism determination, working condition analysis, and material performance evaluation. In summary, this application effectively overcomes the key pain points of existing technologies in mixed wear identification, demonstrating significant technological advancement and practical application value.

[0156] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It is obvious to those skilled in the art that this application is not limited to the details of the above exemplary embodiments, and that this application can be implemented in other specific forms without departing from the spirit or basic characteristics of this application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects. The scope of this application is defined by the appended claims rather than the foregoing description. Therefore, all variations falling within the meaning and scope of equivalent elements of the claims are intended to be included within this application, and no reference numerals in the claims should be considered as limiting the scope of the claims.

[0157] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for quantitatively identifying a wear type based on image segmentation, characterized by, Includes the following steps: S1. Acquire several wear marks images, and divide and label the wear marks images to form a dataset containing wear area mask information; S2. Divide the dataset into a training set and a validation set, and use the training set to train the semantic segmentation network. During the training process, use hyperparameter optimization and cross-validation strategies to optimize the semantic segmentation network to obtain the globally optimal wear image semantic segmentation model. The semantic segmentation network includes a feature extraction module with embedded CBAM composite attention units, a hollow spatial pyramid pooling module, a feature fusion module, and an output layer. The feature extraction module in the semantic segmentation network adopts the MobileNet backbone network. The MobileNet backbone network is used to extract features at different levels of the input scratch image to obtain corresponding convolutional features, including shallow features, medium features and deep features. The intermediate-layer features are input into the first CBAM composite attention unit for feature enhancement to obtain the enhanced intermediate-layer features; The deep features are input into the second CBAM composite attention unit for feature enhancement to obtain enhanced deep features; The enhanced deep features are fused and enhanced through a hollow spatial pyramid pooling module to obtain enhanced multi-scale deep features; The enhanced multi-scale features output by the void space pyramid pooling module are input into the feature fusion module for upsampling. The sampling results are then fused with the mid-layer features enhanced by the first CBAM composite attention unit. The fused features are further upsampled, and the further sampling results are fused with the shallow features to obtain the final fused features. The output layer obtains the segmentation mask result based on the final fusion features; S3. The trained semantic segmentation network is used to detect the wear marks in the SEM image to be detected, and the segmentation mask image of the wear marks in the SEM image to be detected is output. The regions of each wear type in the segmentation mask image are marked, and the proportion of different wear types is output based on pixel statistics.

2. The method of claim 1, wherein the method comprises: The specific steps in step S1 of obtaining the wear mark image include: Scanning electron microscope images of the worn sample were selected as the original wear marks images; The original wear marks images were then filtered to remove those that were contaminated, abnormally exposed, or whose typical wear types could not be distinguished, and these were used as the final wear marks images.

3. The method of claim 1, wherein the method comprises: The specific steps of dividing and labeling the wear mark image in step S1 include: Based on the preset wear type labeling standard and using the deep learning image labeling software labelme, the wear mark image is divided into regions and semantically labeled to obtain a dataset containing wear region mask information. The wear types include: abrasive wear region, delamination wear region, tribochemical reaction wear region and non-wear background region.

4. The method of claim 1, wherein the method is characterized by, Step S1 also includes: Data augmentation is performed on the labeled images in the dataset using rotation, cropping, Gaussian blur, and / or contrast enhancement algorithms.

5. The method of claim 1, wherein the method is characterized by, Both the first CBAM composite attention unit and the second CBAM composite attention unit include a channel attention subunit and a spatial attention subunit connected in series. The channel attention subunit takes the convolutional features output by the feature extraction module as input, performs max pooling and average pooling operations on the convolutional features to obtain the corresponding channel description vectors; the obtained channel description vectors are input into the multilayer perceptron module for feature mapping, the feature mapping results are summed and then the channel weight coefficients are generated through the activation function; the channel weight coefficients are multiplied by the convolutional features channel by channel to obtain the channel weighted features; The spatial attention subunit takes the channel-weighted features output by the channel attention subunit as input, performs average pooling and max pooling operations on each channel, and concatenates the obtained average pooling and max pooling results. The concatenated result is then used to generate a spatial attention weight map through convolution. The spatial attention weight map is then multiplied pixel-by-pixel with the input channel-weighted features to obtain the enhanced features after spatial-channel dual weighting.

6. The method of claim 1, wherein the method is characterized by, In S2, the semantic segmentation network is optimized using hyperparameter optimization and cross-validation strategies, specifically including: The hyperparameters to be optimized are determined, and a Bayesian optimization-based hyperparameter search algorithm is used to optimize them. Set a hyperparameter optimization target and use a pruning tool to prune hyperparameter combinations Trial whose optimization effect is below the threshold. After each round of training, the performance metrics of the validation set corresponding to the combination are recorded. The hyperparameter optimization model automatically selects the optimal hyperparameter combination for subsequent cross-validation training. Each search experiment performs 5-fold cross-validation. The labeled dataset obtained in step S1 is randomly divided according to distribution constraints. On each fold dataset, a semantic segmentation model is independently trained based on the model parameters optimized by hyperparameters. The training results of each fold are evaluated according to the preset model performance index, and the model with the best performance is selected as the global optimal model.

7. The method of claim 6, wherein the method comprises: In the step of determining the hyperparameters to be optimized, the hyperparameters to be optimized specifically include: model downsampling factor, number of training epochs frozen, batch size during the freeze and unfreeze phases, total number of training epochs, initial learning rate, learning rate decay method, and optimizer type.

8. The method of claim 6, wherein the method is characterized by, In the step of evaluating the training results of each fold according to preset model performance metrics, the model performance metrics include: Average intersection-union ratio, training set and validation set loss, and global pixel-level classification accuracy.

9. The method of claim 1, wherein the method is characterized by, S3 specifically includes marking the wear type regions in the segmented mask image as follows: Based on the obtained pixel-level wear type segmentation mask image, regions of different wear types are marked by differential colors; The color-marked segmentation mask image is overlaid and fused with the original SEM image at a preset transparency to generate an original image-segmentation mask fusion visualization image used to display the model recognition results.

Images