Automatic recognition and learning method of fracture morphology based on multimodal fusion learning

Abstract

The present invention provides a method for automatic recognition and learning of fracture morphology based on multimodal fusion learning, which relates to the field of metal fracture morphology analysis, including: step S1, constructing a data set; step S2, taking fracture images and annotating the images, after the annotation is completed, dividing the data set into a training set, a validation set and a test set; step S3, multimodal fusion learning; step S4, feature importance analysis and SHAP analysis. The method combines image recognition with machine learning to achieve accurate prediction of dimple fracture and brittle fracture areas of aluminum alloys, and can also effectively obtain key factors affecting metal fracture morphology, thereby providing an important reference basis for the effective optimization of material properties.

Description

Technical Field

[0001] The present invention relates to the technical field of metal fracture morphology analysis, and in particular to a fracture morphology automatic recognition and learning method based on multi-modal fusion learning. Background Art

[0002] With the rapid development of modern industry, the requirements for material properties in the aerospace field are becoming more and more stringent; the plasticity and toughness of materials play a vital role in aerospace structures, which directly affect the safety, durability and overall performance of aerospace vehicles. As a typical high-strength aluminum alloy, 7050 aluminum alloy is widely used in aerospace structures due to its excellent strength-to-weight ratio, high corrosion resistance and good machinability. Optimizing the strength, plasticity and toughness of 7050 aluminum alloy is crucial, but also challenging, and it requires finding a proper balance between various material properties. Studies have shown that the higher the proportion of ductile dimple fracture in the material fracture, the better its plasticity and toughness, and vice versa, the worse its plasticity and toughness. Therefore, exploring the proportion of ductile dimple fracture and brittle fracture areas in the fracture of 7050 aluminum alloy can provide a deeper understanding of the microscopic fracture mechanism, thereby optimizing the plasticity and toughness of 7050 aluminum alloy.

[0003] However, in the actual fracture process, dimples and brittle fractures do not occur in isolation, they can occur simultaneously or alternately; in addition, quantitative analysis of dimple fractures and brittle fractures requires a lot of manpower and resources, and the internal microstructure of the material is diverse and complex, which increases the difficulty of accurately quantifying the ratio of dimples to brittle fractures in material failure. Traditionally, the prediction of material strength can be achieved through mathematical and physical modeling of different strengthening mechanisms, mainly including solid solution strengthening, precipitation strengthening, dislocation strengthening and grain refinement; however, traditional mathematical and physical models are difficult to accurately describe the complex relationship between material microstructure parameters and the ratio of dimples to brittle fractures, which further increases the difficulty of quantifying the ratio of dimples to brittle fractures. Summary of the invention

[0004] In view of the problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a method for automatic recognition and learning of fracture morphology based on multimodal fusion learning. This method combines image recognition with machine learning to achieve accurate prediction of dimple fracture and brittle fracture areas of aluminum alloy, thereby providing an important reference basis for the effective optimization of material properties.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] A method for automatic recognition and learning of fracture morphology based on multimodal fusion learning, comprising:

[0007] Step S1, constructing a data set: performing heat treatment, tensile testing, and morphology analysis steps on the sample in sequence, and constructing a data set containing six key features: first aging time, second aging time, dislocation density, and grain size in three directions;

[0008] Step S2: capture and annotate the fracture images. After the annotation is completed, the data set is divided into a training set, a validation set, and a test set.

[0009] Step S3, multimodal fusion learning: semantic segmentation of the fracture image is performed using the improved YOLOv5 model, and target detection is performed based on the segmented image; then, a machine learning model is used to construct a regression prediction model between the dimple fracture and brittle fracture ratios using six key features as input;

[0010] Step S4, feature importance analysis and SHAP analysis: The importance of each key feature in predicting the dimple fracture and brittle fracture ratio is calculated through feature importance analysis, and the influence of each key feature on the crack morphology is calculated through SHAP analysis; the accuracy of the influence of key features on the model is verified by combining feature importance analysis and SHAP analysis.

[0011] Based on further optimization of the above scheme, the sample in step S1 is a 7050 aluminum alloy forging, and the 7050 aluminum alloy forging is cut along the L, T, and S directions respectively.

[0012] Based on further optimization of the above scheme, the heat treatment in step S1 is specifically as follows: first, the sample is solution treated at a temperature of 477°C for 4 hours; then, it is water quenched at a temperature of 66°C; subsequently, a two-stage aging process is performed in sequence: the sample is first subjected to a first aging at a temperature of 121°C for 4 to 6 hours, and then the temperature is increased to 177°C at a rate of 5°C / min, and then a second aging is performed for 4 to 7 hours.

[0013] Based on further optimization of the above scheme, the tensile test in step S1 is specifically as follows: an electronic universal mechanical testing machine is used for the tensile test, the cross-sectional area of ​​the tensile specimen is 2×2 mm², the gauge length is 20 mm, the tensile rate is set to 1.0 mm / min, and a telescope with a gauge length of 12.5 mm is used; after the heat treatment is completed, the tensile test is immediately repeated 3 times.

[0014] Based on the further optimization of the above scheme, the morphology analysis in step S1 is specifically as follows: after the tensile test is completed, an X-ray diffractometer is first used for analysis, and the instrument is set to a voltage of 45 kV, a current of 40 mA, a step length of 0.01313°, an instrument scanning range of 2θ= 35°~140°, and a scanning speed of 0.05° / s; a pseudo-Voigt function is used to fit each diffraction peak to obtain the half-peak full width, and then the modified Williamson-Hall diagram is used to calculate the dislocation density, specifically:

[0015]

[0016] Where: represents the FWHM of strain broadening in reciprocal space; D g represents the average grain size; A represents the area of ​​the grain; b represents the Burgers vector; represents the dislocation density; K represents the diffraction vector; Average contrast factor of different diffraction peaks;

[0017]

[0018] Where: represents the diffraction angle, represents the half-height width (FWHM) of the diffraction peak, represents the wavelength of X-rays;

[0019] Then, the grain size of the samples on different surfaces was analyzed by electron backscatter diffraction, and the equivalent circle diameter was used. D Is the representation of granularity:

[0020]

[0021] Where: A represents the area of ​​the grain;

[0022] The grain sizes in the L, T, and S directions were measured respectively, and these measurements were designated as grain size 1, grain size 2, and grain size 3, respectively, corresponding to the actual tensile directions of the sample in the right-handed Cartesian coordinate system, that is, the grain sizes in three directions were obtained.

[0023] Based on further optimization of the above solution, step S2 is specifically as follows:

[0024] Image acquisition: Fracture surface imaging was performed by scanning electron microscopy. The image acquisition process followed the standard material characterization procedure: imaging was performed under a scanning electron microscope at a fixed magnification of 1000 times, and an acceleration voltage of 3kV and a working distance were used to optimize the contrast and clarity of the image; for each sample, 10 images of non-overlapping areas were randomly captured. In order to ensure high-quality image data, imaging was performed under stable vacuum environment conditions to minimize the impact of external factors on image quality;

[0025] Image annotation: Labelme software was used to accurately annotate dimples and brittle fracture areas, with red and green polygons marking dimples and brittle fractures, respectively. To ensure the consistency and accuracy of the annotation, a detailed annotation guide was developed to clearly define the microscopic features of dimples and brittle fractures: dimples are generally circular or elliptical pits with smooth edges and uniform distribution, while brittle fractures are irregular linear or angular mesh structures. Before final confirmation, each image underwent multiple rounds of review to ensure that there were no incorrect labels or omissions.

[0026] Based on further optimization of the above scheme, the ratio of the training set, the validation set and the test set in step S2 is 7:1:2.

[0027] Based on the further optimization of the above scheme, the specific steps of using the improved YOLOv5 model to perform semantic segmentation on the fracture image in step S3 and performing target detection based on the segmented image are as follows:

[0028] Step S31, processing the input image by mosaic data enhancement and normalization, thereby optimizing the data input;

[0029] Step S32, the improved YOLOv5 model includes a trunk, a neck and a head: the trunk adopts the architecture of CSPDarknet53 to extract spatial and semantic information from the image. In the process of trunk optimization, more front convolutional layers are added to capture fine texture details, and the use of rear convolutional layers is reduced to prevent excessive abstraction of these details; the neck uses FPN and PANet structures for feature fusion. In the optimization process, lateral connections are added and an adaptive weighting mechanism is introduced to ensure that detailed information is not lost. In addition, the bottom-up feature propagation and convolution operations of key nodes in PANet are enhanced, which improves the sensitivity and accuracy of identifying subtle structural changes; the head integrates bounding box regression, target classification and confidence scoring functions. In order to optimize performance, a dynamic anchor mechanism is implemented to adaptively adjust the size of the anchor box;

[0030] Step S33: First, for each anchor box, use the softmax activation function to predict the probability of belonging to each category to complete the category prediction; then, predict the bounding box regression parameters of each detection box through the model, and introduce the IoU loss function to improve the accuracy of the bounding box, so as to accurately fit the target position; then, apply Non-Maximum Suppression (NMS) post-processing to the prediction output part to select the optimal bounding box (i.e., remove overlapping low-confidence boxes and retain high-confidence boxes), so as to achieve the purpose of predicting the position and category of the target box; finally, set the confidence threshold to filter low-confidence predictions, so as to reduce the false positive rate;

[0031] Step S34: Use precision (Precision, P), recall (Recall, R) and F1 score (F1-score) to evaluate the image recognition results.

[0032] Based on the further optimization of the above scheme, the machine learning model in step S3 constructs a regression prediction model between the ratio of dimple fracture and brittle fracture as follows: six key features of the first aging time, the second aging time, dislocation density and grain size in three directions are used as input, and are randomly divided into training data and test data, training data: test data = 7:3; the training data is used to construct an XGBoost model in a python environment to establish the relationship between input features and output features, and the percentage of dimple fracture and brittle fracture is used as output, and the key factors affecting the ratio of dimple fracture and brittle fracture on the fracture are analyzed and explored, so as to provide a reference for optimizing the plasticity and toughness of the material; the constructed XGBoost model is used to verify the test data, and the mean absolute error MAE, square error MSE, root mean square error RMSE and determination coefficient R² are used to evaluate the accuracy of the model.

[0033] Based on the further optimization of the above scheme, the feature importance analysis in step S4 calculates the key features that play an important role in predicting the ratio of dimple fracture and brittle fracture in the XGBoost model through the three factors of Gain, Cover and Frequency.

[0034] Based on the further optimization of the above scheme, the SHAP analysis in step S4 is specifically as follows:

[0035] First, we use SHAP in Python to analyze the model and calculate the Shapley value of each key feature. :

[0036]

[0037] Where: represents the set of all input features, p Indicates the number of input features; Indicates that the collection is not included The set of all input features of ; Representing feature subsets S predictions;

[0038] In: The denominator represents p In any order of features, The situation of a combination of types; the meaning of the molecule: in determining the subset S After that, the set of features is , subset S , which itself contains A sequential combination followed by the remaining features of feature j contain Combinations of molecules, that is, molecules represent certain subsets S After p The features have Combination of situations;

[0039] Then, we visualize the overall impact of the features by summarizing the Shapley values ​​of all samples;

[0040] The importance of each feature is measured based on the average absolute Shapley value of each feature:

[0041]

[0042] The following are the technical effects of the technical solution of the present invention:

[0043] The present invention adopts a multimodal fusion learning framework, which can quickly and accurately identify brittle fractures and dimple fractures in images through an improved YOLOv5 model, thereby significantly improving the efficiency of acquiring quantitative data of fracture morphology; then, a machine learning model of the XGBoost model is adopted, combined with the input of six key features of the first aging time, the second aging time, the dislocation density, and the grain size in three directions, to effectively describe the relationship between microstructural parameters and crack morphology, and through feature importance analysis and SHAP analysis, the key factors affecting the ratio of dimples and brittle fractures on the fracture, that is, the fracture morphology, are analyzed and explored, and the key factors affecting the fracture morphology of metals can be accurately obtained, thereby providing a new way to optimize the plasticity and toughness of materials.

[0044] Compared with traditional mathematical and physical model prediction descriptions, the method adopted by the present invention has high prediction efficiency and accuracy, and can describe the complex relationship between material microstructure parameters and the ratio of dimples and brittle fractures, thereby providing accurate and effective data support for material optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 The improved YOLOv5 model in the embodiment of the present invention performs semantic segmentation on the fracture image.

[0046] Figure 2 This is a comparison diagram of the aluminum alloy fracture before and after recognition by the improved YOLOv5 model in the embodiment of the present invention; wherein, Figure 2 (a) is the original image before recognition. Figure 2 (b) is the image after recognition.

[0047] Figure 3 This is a diagram of the image recognition evaluation results of the improved YOLOv5 model in an embodiment of the present invention.

[0048] Figure 4 This is a graph of the prediction results of the XGBoost model in an embodiment of the present invention.

[0049] Figure 5 Schematic diagram of the result of feature importance analysis in an embodiment of the present invention.

[0050] Figure 6 Schematic diagram of the results of SHAP analysis in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present invention will be described clearly and completely below. In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures and technologies are proposed to facilitate a thorough understanding of the embodiments of the present invention.

[0052] Embodiment 1:

[0053] An automatic recognition and learning method for fracture morphology based on multimodal fusion learning is proposed. A 7050 aluminum alloy forging is used as a sample, and the 7050 aluminum alloy forging is cut along the L, T, and S directions respectively, including:

[0054] Step S1, constructing a data set: performing heat treatment, tensile testing, and morphology analysis steps on the sample in sequence, and constructing a data set containing six key features: first aging time, second aging time, dislocation density, and grain size in three directions;

[0055] Step S11, heat treatment: first, the sample is solution treated at a temperature of 477°C for 4 hours; then, it is water quenched at a temperature of 66°C; subsequently, a two-stage aging process is performed in sequence: the sample is first subjected to a first aging at a temperature of 121°C for 4 to 6 hours (obtaining the first aging time, in this embodiment, the first aging time can be 4 hours, 5 hours, or 6 hours), and then the temperature is increased to 177°C at a rate of 5°C / min, and then a second aging is performed for 4 to 7 hours (obtaining the second aging time, in this embodiment, the second aging time can be 4 hours, 5 hours, 6 hours, or 7 hours).

[0056] Step S12, the tensile test is performed by using an electronic universal mechanical testing machine (for example, a 10 kN MTS E43 electronic universal mechanical testing machine), the cross-sectional area of ​​the tensile specimen is 2×2 mm², the gauge length is 20 mm, the tensile rate is set to 1.0 mm / min, and a telescope with a gauge length of 12.5 mm is used; after the heat treatment is completed, the tensile test is immediately repeated 3 times.

[0057] Step S13, morphology analysis: After the tensile test is completed, an X-ray diffractometer (for example, a Uranus X-ray diffractometer equipped with a Cu Kα12 radiation source) is first used for analysis. The instrument is set to a voltage of 45 kV, a current of 40 mA, a step length of 0.01313°, an instrument scanning range of 2θ= 35° to 140°, and a scanning speed of 0.05° / s. Each diffraction peak is fitted using a pseudo-Voigt function to obtain the half-peak full width, and then the dislocation density is calculated using a modified Williamson-Hall diagram, specifically:

[0058]

[0059] Where: represents the FWHM of strain broadening in reciprocal space; D g represents the average grain size; A represents the area of ​​the grain; b represents the Burgers vector; represents the dislocation density; K represents the diffraction vector; Average contrast factor of different diffraction peaks;

[0060]

[0061] Where: represents the diffraction angle, represents the half-height width (FWHM) of the diffraction peak, represents the wavelength of X-rays;

[0062] Then, the grain size of the sample on different surfaces is analyzed using an electron backscatter diffraction instrument (e.g., a Zeiss 300 field emission scanning electron microscope equipped with an Oxford Symmetry S1 detector), and the equivalent circle diameter is used. D Is the representation of granularity:

[0063]

[0064] Where: A represents the area of ​​the grain;

[0065] The grain sizes in the L, T, and S directions were measured respectively, and these measurements were designated as grain size 1, grain size 2, and grain size 3, respectively, corresponding to the actual tensile directions of the sample in the right-handed Cartesian coordinate system, that is, the grain sizes in three directions were obtained.

[0066] Step S2: taking fracture images and labeling them:

[0067] Image acquisition: Fracture surface imaging was performed by scanning electron microscopy. The image acquisition process followed the standard material characterization procedure: imaging was performed under a scanning electron microscope at a fixed magnification of 1000 times, and an acceleration voltage of 3kV and a working distance were used to optimize the contrast and clarity of the image; for each sample, 10 images of non-overlapping areas were randomly captured. In order to ensure high-quality image data, imaging was performed under stable vacuum environment conditions to minimize the impact of external factors on image quality;

[0068] Image annotation: Labelme software was used to accurately annotate dimples and brittle fracture areas (this software is an open source image annotation tool developed by MIT), with red and green polygons marked as dimples and brittle fractures, respectively. To ensure the consistency and accuracy of the annotation, a detailed annotation guide was developed to clearly define the microscopic features of dimples and brittle fractures: dimples are generally circular or elliptical pits with smooth edges and uniform distribution, while brittle fractures are irregular linear or angular mesh structures. Before final confirmation, each image underwent multiple rounds of review to ensure that there were no incorrect labels or omissions.

[0069] After the labeling is completed, the dataset is divided into training set, validation set and test set, and the ratio of training set, validation set and test set is 7:1:2.

[0070] Step S3, multimodal fusion learning: use the improved YOLOv5 model to perform semantic segmentation on the fracture image, and perform target detection based on the segmented image, such as Figure 1 As shown, the specific steps are:

[0071] Step S31, using mosaic data enhancement and normalization processing to process the input image, thereby optimizing the data input; specifically: first, using mosaic data enhancement technology, four original images (each with a size of 640×640) are scaled and cropped, and then combined into a 640×640 composite image, so that the model can see more diverse features in a single image; then, in order to effectively process the pixel value differences of different images, normalization processing is performed to scale the pixel values ​​from 0 to 255 to between 0 and 1, that is, the pixel value 255 becomes 1 after normalization processing, while 0 remains 0;

[0072] The purpose of image normalization is to scale the input pixel values ​​to a uniform range (usually 0 to 1) to eliminate the differences in pixel value ranges between images and enhance the stability of model training. The specific formula is:

[0073]

[0074] Where: I represents the original image matrix, max(I), min(I) Respectively represent the maximum and minimum values ​​of the original image matrix; Represents the normalized image matrix.

[0075] Step S32, the improved YOLOv5 model includes a trunk, a neck and a head:

[0076] The backbone uses the CSPDarknet53 architecture to extract spatial and semantic information from images. During backbone optimization, more front convolutional layers are added to capture fine texture details and fewer back convolutional layers are used to prevent over-abstraction of these details. Figure 1As shown in the figure: In the input part of the backbone network, the initial convolution block (32x320x320 CBS) is responsible for extracting low-level features, increasing the number of channels to 32 while retaining a high spatial resolution (320x320), where CBS stands for convolution, batch normalization, and activation function (ReLU); in the feature extraction process, the CSP module (Cross Stage Partial) optimizes the amount of calculation through a branching structure while maintaining feature expression capabilities; CSP1_1 and CSP1_2 modules enhance the network's feature learning capabilities by splitting and merging features. These modules focus on optimizing gradient flow, which helps the network learn complex features more efficiently; as the network deepens, the spatial size of the feature map gradually decreases, while the number of channels gradually increases (for example: from 64x160x160 to 128x80x80), which is achieved by downsampling (convolution or pooling with a stride of 2); in addition, subsequent CBS and CSP modules (for example: CSP1_3) continue to strengthen this, and finally increase the number of channels of the feature map to 512. The network also introduces the SSPF module (Spatial Pyramid Pooling Fast) for multi-scale feature fusion. This module enhances the network's detection ability for objects of different sizes by pooling and merging features of different scales, ensuring the generation of fixed-size feature maps and improving the robustness of detection.

[0077] The neck uses FPN and PANet structures for feature fusion. During the optimization process, lateral connections are added and an adaptive weighting mechanism is introduced to ensure that detailed information is not lost. In addition, the bottom-up feature propagation and convolution operations of key nodes in PANet are enhanced to improve the sensitivity and accuracy of identifying subtle structural changes. Figure 1 As shown in the figure: In the feature fusion process of the neck network (Neck), the neck network uses upsampling and concatenation (Concat) operations to process feature maps: upsampling is used to adjust the low-resolution feature map to a higher resolution to facilitate fusion with the high-resolution feature map; the concatenation operation integrates the upsampled feature map with the previous feature map, so that the model can simultaneously use high-level semantic information and underlying detail information, thereby enhancing the descriptive ability of the features. In addition, the small CSP module continues to play a role in the neck network, maintaining the efficiency and diversity of the feature flow through further convolution and normalization operations. This design makes the neck network richer in feature representation and helps to improve the accuracy of target detection; by combining high-level semantics with low-level spatial information, the neck network significantly enhances the multi-scale detection capability of the model, which enables the model to have better detection performance and robustness when facing targets of different sizes, especially small targets.

[0078] Among them, the FPN structure is used to fuse features between different scales to enhance the detection capability of multi-scale targets. Its core idea is bottom-up multi-level feature aggregation; the PANet structure further improves the FPN structure by adding top-down paths to enhance information flow and better aggregate features.

[0079] The head integrates bounding box regression, target classification and confidence scoring functions. In order to optimize performance, a dynamic anchor mechanism is implemented to adaptively adjust the size of the anchor box. Figure 1 As shown: First, the head network (Head) usually contains multiple prediction layers, each of which operates on different feature scales; the head network contains two main branches: classification branch and regression branch, where the classification branch is responsible for determining which category each candidate region in the feature map belongs to, and generates predictions corresponding to the number of categories through a series of convolution operations; the regression branch is responsible for predicting the location and size of the target, and usually adjusts the preset anchor box (or anchor) through the bounding box regression technology to better fit the location of the target. The head network uses an anchor box mechanism, which is predefined with different shapes and sizes to identify targets at specific locations and help the model deal with targets of different scales and aspect ratios. In order to train the model, the head network uses a loss function to consider both the classification error and the bounding box regression error, such as cross entropy loss and smooth L1 loss.

[0080] Step S33: First, for each anchor box, use the softmax activation function to predict the probability of belonging to each category to complete the category prediction; then, predict the bounding box regression parameters of each detection box through the model, and introduce the IoU loss function to improve the accuracy of the bounding box, so as to accurately fit the target position; then, apply Non-Maximum Suppression (NMS) post-processing to the prediction output part to select the optimal bounding box (i.e., remove overlapping low-confidence boxes and retain high-confidence boxes), so as to achieve the purpose of predicting the position and category of the target box; finally, set the confidence threshold to filter low-confidence predictions, thereby reducing the false detection rate.

[0081] Step S34, using precision (Precision, P), recall (Recall, R) and F1 score (F1-score) to evaluate the results of image recognition (wherein, precision is used to evaluate the accuracy of the classification model, recall is used to evaluate the ability of the model to identify positive samples, and F1 score is used to comprehensively evaluate the performance of the model. Precision, recall and F1 score adopt conventional methods in the field); refer to Figure 3 As shown: the light blue line corresponds to the dimple fracture category, the yellow line corresponds to the brittle fracture category, and the thicker dark blue line represents the average of all categories; Figure 3As shown in (a): When the confidence level is 0.497, the F1 score of each category is 0.98, which shows that the improved YOLOv5 model performs well in predicting dimple fracture and brittle fracture, with high accuracy and recall. Figure 3 As shown in (b), the overall trend shows that as the confidence increases, the precision also increases; when the confidence reaches 0.845, the precision reaches 1. Figure 3 (c) shows the recall rates at different confidence levels. The recall rates of dimple fracture and brittle fracture remain unchanged at first and then drop rapidly. Initially, the average recall rate is slightly lower than that of brittle fracture and slightly higher than that of dimple fracture. Finally, the three rates are close, especially when the confidence index exceeds 0.8. Figure 3 (d) shows the accuracy and recall of the classifier at different thresholds: the dimple fracture line remains stable at around 0.990, while the brittle fracture line remains at 0.995, and the average precision-recall curve hovers around 0.993; these three lines show that as the recall rate increases, the accuracy decreases slightly, but generally remains at a satisfactory level, indicating that the improved YOLOv5 model performs very well in identifying dimple fractures and brittle fractures, with high stability and accuracy.

[0082] Subsequently, a machine learning model was used to construct a regression prediction model between the proportion of dimple fracture and brittle fracture through six key features as input; specifically, the six key features of the first aging time, the second aging time, the dislocation density and the grain size in three directions were used as input, and they were randomly divided into training data and test data, with training data: test data = 7:3; the training data was used to construct an XGBoost model in a python environment to establish the relationship between input features and output features, and the percentage of dimple fracture and brittle fracture was used as output to analyze and explore the key factors affecting the proportion of dimple fracture and brittle fracture on the fracture, thereby providing a reference for optimizing the plasticity and toughness of the material; the constructed XGBoost model was used to verify the test data, and the mean absolute error MAE, square error MSE, root mean square error RMSE and determination coefficient R² were used to evaluate the accuracy of the model:

[0083]

[0084] Where: x i Represents the true value of the sample; y i Represents the predicted value of the sample; Represents the average of all true values; MAE calculates the average absolute difference between the predicted value and the true value: the smaller the MAE value, the closer the model's predicted value is to the actual value, indicating that the model is more accurate; MSE calculates the average of the squared differences between the predicted value and the true value: the smaller the MSE value, the smaller the prediction error and the higher the prediction accuracy; RMSE is the square root of MSE, which is used to measure the deviation between the predicted value and the true value: the smaller the RMSE value, the more consistent the model's predicted value is with the actual value; R² is used to evaluate the performance of the regression model: the closer the R² value is to 1, the stronger the model's ability to explain data variability, indicating that the model performance is better.

[0085] The mean absolute error MAE, square error MSE, root mean square error RMSE and determination coefficient R² thresholds are preset respectively. If the MAE, MSE, RMSE and R² values ​​of the XGBoost model are all within the preset value range, the XGBoost model is judged to be qualified; otherwise, the XGBoost model is retrained until it is qualified.

[0086] In this embodiment, the MAE, MSE, RMSE and R² of the dimple fracture and brittle fracture ratio prediction of the XGBoost model using the training data are 5.05, 2.25, 1.91 and 0.73 respectively; the prediction results of the XGBoost model and the experimental results are shown in Figure 4 As shown in the figure: In the prediction of dimple fracture, the R² values ​​of training data and test data are 0.98 and 0.71 respectively; in the prediction of brittle fracture, the R² values ​​of training data and test data are 0.99 and 0.75 respectively; the experimental data show that the XGBoost model has high accuracy in predicting crack morphological characteristics and can effectively capture the relationship between crack morphological characteristics and microstructural parameters.

[0087] Step S4, feature importance analysis and SHAP analysis: The importance of each key feature in predicting the dimple fracture and brittle fracture ratio is calculated through feature importance analysis. In this embodiment, the key features that play an important role in predicting the dimple fracture and brittle fracture ratio in the XGBoost model are calculated through the three factors of Gain, Cover and Frequency; wherein, Gain is used to indicate the contribution of a certain feature to the improvement of the model prediction performance when used for segmentation, Cover is used to indicate the sum of the number of data samples used for splitting a feature when building a tree, and Frequency indicates the number of times a feature is used as a segmentation point. The more times a feature is used, the greater its contribution to the model prediction (Gain, Cover and Frequency all adopt conventional means in this field).

[0088] After performing the importance feature analysis, visualize the results, refer to Figure 5As shown, according to the importance feature analysis: the ratio of dimple fracture and brittle fracture is mainly affected by dislocation density and the first aging time, and the effect of grain size on dimple fracture and brittle fracture is relatively small.

[0089] The influence of each key feature on the crack morphology is calculated through SHAP analysis (the input is the trained XGBoost and training data, and the output is the SHAP value of each key factor). The specific SHAP analysis is as follows:

[0090] First, we use SHAP in Python to analyze the model and calculate the Shapley value of each key feature. :

[0091]

[0092] Where: represents the set of all input features, p Indicates the number of input features; Indicates that the collection is not included The set of all input features of ; Representing feature subsets S predictions;

[0093] In: The denominator represents p In any order of features, The situation of a combination of types; the meaning of the molecule: in determining the subset S After that, the set of features is , subset S , which itself contains A sequential combination followed by the remaining features of feature j contain Combinations of molecules, that is, molecules represent certain subsets S After p The features have Combination of situations;

[0094] Then, we visualize the overall impact of the features by summarizing the Shapley values ​​of all samples;

[0095] The importance of each feature is measured based on the average absolute Shapley value of each feature:

[0096]

[0097] Figure 6This is a summary diagram of the SHAP values ​​in this example, where the position on the x-axis represents the impact of the feature value on the prediction, red points represent features with greater impact, and blue points represent features with less impact. According to the SHAP analysis, the key features that have the greatest impact on dimple fracture and brittle fracture are dislocation density and first aging time, which is consistent with the results of the feature importance analysis.

[0098] The feature importance analysis is combined with the SHAP analysis to verify the accuracy of the impact of key features on the model, that is, to determine that dislocation density and the first aging time are the key features with the greatest impact on dimple fracture and brittle fracture.

[0099] Embodiment 2:

[0100] As another preferred embodiment of the present invention, based on the solution in Example 1, the bounding box parameters predicted in the head network include the center position offset (t x ,t y )、width and height(t w ,t h ) and confidence .

[0101]

[0102] In the formula, Represents the center point coordinates, length, and width of the predicted bounding box in the feature map; Indicates the coordinates of the upper left point of each unit; Represents the length and width of the prior box relative to the feature map.

[0103] Embodiment 3:

[0104] A multimodal fusion learning system for alloy fracture, using the fracture morphology automatic recognition and learning method based on multimodal fusion learning in any one of Embodiment 1 or Embodiment 2, including: a data collection module, an image collection module, a target detection module, a prediction module, and a feature analysis module.

[0105] The data collection module is electrically connected to the heat treatment timer, electronic universal mechanical testing machine, X-ray diffractometer, and electron backscatter diffractometer to obtain key characteristic parameters such as the first aging time, the second aging time, dislocation density, and grain size in three directions;

[0106] The image collection module is connected to an external camera device and is used to collect images of different fracture surfaces of the alloy;

[0107] The target detection module uses the improved YOLOv5 model to perform semantic segmentation on the fracture image and realizes target detection at the fracture based on the segmented image;

[0108] The prediction module uses the XGBoost model to predict the proportional relationship between dimple fracture and brittle fracture at the fracture surface and complete the output;

[0109] The feature analysis module, feature importance analysis module and SHAP analysis module perform interpretable analysis on each key feature in the XGBoost model, so as to screen the most important key features that affect the ratio of dimple fracture and brittle fracture at the fracture surface, and provide data support for the research of alloy materials.

Claims

1. A method for automatic recognition and learning of fracture morphology based on multimodal fusion learning, characterized in that: include: Step S1, constructing a data set: performing heat treatment, tensile testing, and morphology analysis steps on the sample in sequence, and constructing a data set containing six key features: first aging time, second aging time, dislocation density, and grain size in three directions; The morphology analysis is as follows: after the tensile test is completed, an X-ray diffractometer is first used for analysis. The instrument is set to a voltage of 45 kV, a current of 40 mA, a step size of 0.01313°, an instrument scanning range of 2θ = 35° to 140°, and a scanning speed of 0.05° / s; a pseudo-Voigt function is used to fit each diffraction peak to obtain the half-peak full width, and then the modified Williamson-Hall diagram is used to calculate the dislocation density, specifically: Where: represents the FWHM of strain broadening in reciprocal space; D g represents the average grain size; A represents the area of ​​the grain; b represents the Burgers vector; represents the dislocation density; K represents the diffraction vector; Average contrast factor of different diffraction peaks; Where: represents the diffraction angle, represents the half-height width of the diffraction peak, represents the wavelength of X-rays; Then, the grain size of the samples on different surfaces was analyzed by electron backscatter diffraction, and the equivalent circle diameter was used. D Is the representation of granularity: Where: A represents the area of ​​the grain; The grain sizes in the L, T, and S directions were measured respectively, and these measurements were designated as grain size 1, grain size 2, and grain size 3, which corresponded to the actual tensile directions of the sample in the right-handed Cartesian coordinate system, that is, the grain sizes in three directions were obtained; Step S2: capture and annotate the fracture images. After the annotation is completed, the data set is divided into a training set, a validation set, and a test set. Step S3, multimodal fusion learning: use the improved YOLOv5 model to perform semantic segmentation on the fracture image, and perform target detection based on the segmented image. The specific steps are as follows: Step S31, processing the input image by mosaic data enhancement and normalization, thereby optimizing the data input; Step S32, the improved YOLOv5 model includes a trunk, a neck and a head: the trunk adopts the architecture of CSPDarknet53 to extract spatial and semantic information from the image; the neck uses FPN and PANet structures for feature fusion; The head integrates bounding box regression, object classification and confidence scoring functions; Step S33: First, for each anchor box, the softmax activation function is used to predict the probability of belonging to each category to complete the category prediction; then, the bounding box regression parameters of each detection box are predicted by the model, and the IoU loss function is introduced to improve the accuracy of the bounding box, so as to accurately fit the target position; then, the prediction output part applies Non-Maximum Suppression post-processing to select the optimal bounding box; finally, the confidence threshold is set to filter low-confidence predictions, thereby reducing the false positive rate; Step S34, using precision, recall and F1 score to evaluate the image recognition results; Subsequently, a machine learning model was used to construct a regression prediction model between the ratio of dimple fracture and brittle fracture using six key features as input; Step S4, feature importance analysis and SHAP analysis: The importance of each key feature in predicting the dimple fracture and brittle fracture ratio is calculated through feature importance analysis, and the influence of each key feature on the crack morphology is calculated through SHAP analysis.

2. The method for automatic recognition and learning of fracture morphology based on multimodal fusion learning according to claim 1 is characterized in that: The sample in step S1 is a 7050 aluminum alloy forging, and the 7050 aluminum alloy forging is cut along the L, T, and S directions respectively.

3. The method for automatic recognition and learning of fracture morphology based on multimodal fusion learning according to claim 1 or 2, characterized in that: The heat treatment in step S1 is specifically as follows: first, the sample is subjected to solution treatment at a temperature of 477°C for 4 hours; then, water quenching is performed at a temperature of 66°C; Subsequently, a two-stage aging process was performed in sequence: the sample was first subjected to the first aging at a temperature of 121°C for 4 to 6 hours, and then the temperature was increased to 177°C at a rate of 5°C / min, and then subjected to the second aging for 4 to 7 hours.

4. The method for automatic recognition and learning of fracture morphology based on multimodal fusion learning according to claim 3 is characterized in that: The tensile test in step S1 is specifically performed as follows: an electronic universal mechanical testing machine is used, the cross-sectional area of ​​the tensile specimen is 2×2 mm², the gauge length is 20 mm, the tensile rate is set to 1.0 mm / min, and a telescope with a gauge length of 12.5 mm is used; after the heat treatment is completed, the tensile test is immediately repeated 3 times.

5. The method for automatic recognition and learning of fracture morphology based on multimodal fusion learning according to claim 1 is characterized in that: The step S2 is specifically as follows: Image acquisition: Fracture surface imaging was performed by scanning electron microscopy. The image acquisition process followed the standard material characterization procedure: imaging was performed under a scanning electron microscope at a fixed magnification of 1000 times, using an accelerating voltage of 3 kV and a working distance to optimize the contrast and clarity of the image; for each sample, 10 images of non-overlapping areas were randomly captured, and imaging was performed under stable vacuum environment conditions; Image annotation: Labelme software was used to accurately annotate the dimples and brittle fracture areas, with red and green polygons marking dimples and brittle fractures, respectively; the microscopic features of dimples and brittle fractures were defined: dimples were circular or elliptical pits with smooth edges and uniform distribution, while brittle fractures were irregular linear or angular mesh structures.

6. The method for automatic recognition and learning of fracture morphology based on multimodal fusion learning according to claim 1 is characterized in that: The specific steps of constructing a regression prediction model between the ratio of dimple fracture and brittle fracture by the machine learning model in step S3 are as follows: taking six key features of the first aging time, the second aging time, the dislocation density and the grain size in three directions as input, and randomly dividing them into training data and test data, with the ratio of training data to test data being 7:3; constructing an XGBoost model with the training data in a python environment to establish the relationship between the input features and the output features, taking the percentage of dimple fracture and brittle fracture as output, analyzing and exploring the key factors affecting the ratio of dimple fracture and brittle fracture on the fracture; using the constructed XGBoost model to verify the test data, and using the mean absolute error MAE, square error MSE, root mean square error RMSE and determination coefficient R² to evaluate the accuracy of the model.

Images