Metal material microscopic image performance evaluation method based on convolutional neural network

Abstract

The invention discloses a metal material microscopic image performance evaluation method based on a convolutional neural network, and belongs to the technical field of metal material performance testing. The method comprises the following steps: acquiring a metallographic image of a metal material sample, performing preprocessing operation, constructing a multi-modal data set, and performing sample expansion by using a generative adversarial network (GAN); constructing a double-branch residual network to obtain an enhanced feature map; constructing an attention guiding feature fusion network to obtain a fusion feature vector; and finally, completing performance evaluation by using a multi-task output head. According to the method, a double-branch residual convolution module is combined with cavity convolution and frequency domain analysis, local details of crystal grain morphology and global structure features of a phase interface are considered, optimal complementation of image and component features is realized through dynamic weight distribution based on a cross-modal fusion strategy of an attention mechanism, and the optimal complementation of the image and component features is realized. And the model can still rapidly converge and maintain excellent prediction stability and generalization performance under the condition of a small number of real samples.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of metal material performance testing, and in particular to a metal material microscopic image performance evaluation method based on a convolutional neural network. BACKGROUND Metal material microscopic image analysis and performance evaluation is to reveal the material microscopic structure characteristics such as grain morphology, phase distribution, defect type and distribution, etc. through high-resolution microscopic imaging technology such as transmission electron microscope, scanning electron microscope, metallographic microscope, etc., and to establish the quantitative correlation between microscopic structure and macroscopic performance by combining mechanical property testing and corrosion behavior research. Transmission electron microscopy can analyze nanoscale precipitates and dislocation networks, metallographic analysis can predict the plastic deformation ability of materials through grain size statistics and phase proportion calculation, and artificial intelligence driven image stitching and segmentation technology realizes high-throughput intelligent characterization of complex microstructure. This technology system provides key basis for material design optimization and service performance prediction, covering the core needs of high-end manufacturing fields such as aerospace and energy equipment.

[0002] Traditional metal material microscopic analysis, such as metallographic structure identification and inclusion rating, relies on manual observation or simple image processing, which requires technical personnel to manually label, measure and count, and the process is tedious and labor-intensive. The microscopic performance of metal materials is highly dependent on their microscopic structure characteristics, but these characteristics are often subtle, complex and overlapping. In addition, existing technologies lack effective strategies to deeply integrate numerical data such as composition and process with image data, resulting in low utilization efficiency of multi-modal information; and in small sample actual industrial scenarios, the model has poor generalization ability and insufficient prediction stability due to data scarcity. These limitations have seriously restricted the application of this technology in high-precision and automated material performance evaluation. SUMMARY

[0003] To solve the above technical problems, the present application provides a metal material microscopic image performance evaluation method based on a convolutional neural network.

[0004] To achieve the above technical scheme, the steps include: S1, collect the metallographic image of the metal material sample and perform a preprocessing operation to obtain a multi-modal data set containing the metallographic image, composition data and mechanical property data; The way to collect the metallographic image of the metal material sample is to use a metallographic microscope, a scanning electron microscope or a transmission electron microscope to obtain it; While collecting the metallographic image of the metal material sample, record the composition data; The mechanical property data is obtained from actual measurement experiments, including yield strength YS, tensile strength TS and elongation EL; The preprocessing method includes illumination correction and noise suppression.

[0005] S2, sample expansion is performed using a generative adversarial network (GAN); In the generative adversarial network (GAN), the generator is based on random noise Z, and adopts a progressive training strategy, that is, gradually performs upsampling operations from low resolution until an image with the same size as the original pixel size of 256*256 is obtained; In the generative adversarial network (GAN), the discriminator is based on the original image with the original pixel size of 256*256 and the generated image, and a feature matching loss is introduced, and when the feature matching loss is less than a threshold value, the random noise Z is adjusted until the feature matching loss is greater than the threshold value; The expression of the feature matching loss is as follows: In the formula, denotes the discriminator; denotes the real vector; denotes the vector generated by the generator.

[0006] S3, a double-branch residual network is constructed, and the metallographic image in the expanded data set is input to obtain an enhanced feature map; The double-branch residual network includes: Branch 1: adopts a dilated convolution and a deformable convolution, and performs an activation function and a batch normalization operation to capture the morphology of the grains in the metallographic image; Branch 2: adopts a short-time Fourier transform (STFT) to extract image frequency domain features; The results output by branch 1 and branch 2 are added element by element, and then a residual connection operation is performed to obtain an enhanced feature map; The steps include: S3.1, a metallographic image with an original pixel size of 256*256*1 is input into an initial convolution layer as an original image for preliminary feature extraction; The initial convolution layer is set to a 7*7 convolution layer, the step size is set to 2, the output channel is set to 64, and the ReLU activation function is used to output the preliminary feature extraction result after down-sampling; S3.2, the preliminary feature extraction result is input into branch 1 to output a 128*128*64 feature map with geometric morphology; In branch 1, the image sequentially passes through a dilated convolution, a deformable convolution, and an activation and normalization operation; The dilated convolution uses a 3*3 convolution kernel, and the expansion rate is set to 2, so as to expand the perception boundary; The input of deformable convolution is the output of the dilated convolution, in order to improve the calculation speed, the deformable convolution is set as a 3x3 convolution layer, and there are 2x9=18 offset fields of channel, wherein 2 represents the horizontal and vertical coordinates (x, y) of each sampling point, that is, when 9 sampling points are set in the deformable convolution for the output of the dilated convolution, the sampling points are subjected to bilinear interpolation and convolution with the weight to extract the features of the curved grain boundary, and finally sequentially through the batch normalization operation and the ReLU activation function, a 128x128x64 feature map with the morphology features of the grains in the metallographic image is obtained; The expression of convolution with the weight is: In the formula, N represents a sampling point, and the present application takes 9; The sampling point weight is represented by Wn; The bias term is represented by b; The value of the nth sampling point is represented by Xn; S3.3, input the preliminary feature extraction result into branch 2, and output a 128x128x64 feature map with periodic texture; In branch 2, the image sequentially undergoes short-time Fourier transform (STFT) and inverse transform, and a 128x128x64 feature map with periodic texture information in the metallographic image is obtained; S3.4, element-by-element addition is performed on the results obtained by branch 1 and branch 2 to obtain a fused image; S3.5, residual connection is performed on the fused image and the preliminary feature extraction result to obtain an enhanced feature map.

[0007] S4, an attention-guided feature fusion network is constructed, and the enhanced feature map and the component data in the multi-modal data set are input to obtain a fused feature vector; The steps include: S4.1, input the enhanced feature map into two fully connected layers; In the first fully connected layer, the 128x128x64 enhanced feature map is subjected to global average pooling operation and then projected through the fully connected layer to obtain a 512-dimensional vector; In the second fully connected layer, the 512-dimensional vector is projected to obtain a 128-dimensional vector ; S4.2, use 1D-CNN to model the time sequence of the component data; The modeling manner is as follows: an 8-dimensional sequence with the same length as the number of component data is constructed, and after sliding convolution, a vector with the same dimension as the output of the two fully connected layers in S3.1 is output through the fully connected layer , that is, 128 dimensions; S4.3, the outputs of the two full connection layers and the 1D-CNN are spliced, input into a multi-head attention layer, and a vector with semantic features is obtained; The spliced sequence is represented as: , ] Attention calculation is performed through the query Q, key K and value V to obtain image features with component context and component features with image information, represented as , ]; S4.4, the results of the multi-head attention layer are merged and input into a full connection network, and two weights are obtained through a Softmax function, as follows: In the formula, represents the weight of the image; represents the weight of the component; represents the full connection network; S4.5, based on the obtained weights, the two full connection layers and the 1D-CNN are assigned weights, and a fusion feature vector is output, as follows: Based on the weight assignment, redundant information is suppressed. The weight assignment enables the model to autonomously weigh the importance of image and component features, avoids the subjectivity of manually presetting weights, and improves the robustness and generalization ability of feature fusion.

[0008] S5, based on the fusion feature vector, a multi-task output head is used to predict the yield strength, tensile strength and elongation; In the multi-task output head, the batch size is set to 16; the fusion feature vector first passes through two full connection layers in turn to reduce the dimension from 128 to 32 to improve the operation speed; and the yield strength YS, tensile strength TS and elongation EL prediction results are output through the activation function respectively; Combined with the mechanical property data collected by the multi-modal data set, the mean square error MSE is calculated to judge the gap between each prediction value and the true value; In the multi-task output head, dynamic loss weight adjustment is used to adjust the weights between the three output heads to ensure that there is no fluctuation during task calculation; The calculation method of dynamic loss weight adjustment is: Among them, is the dynamic loss weight, is is the yield strength, is the tensile strength, or is the elongation, is the dynamic weight, is the first The true value of each performance metric Set dynamic weights for model predictions. ∈ [0.3~0.4]; By capturing the implicit correlations between performance metrics through multi-task learning, dynamic weight allocation prioritizes and optimizes high-uncertainty samples, thereby improving prediction accuracy and enhancing the model's robustness to industrial noise.

[0009] Beneficial effects of the present invention This invention combines a dual-branch residual convolution module with hole convolution and frequency domain analysis, taking into account both the local details of grain morphology and the global structural features of phase interfaces. The deformable convolution layer further adapts to complex micromorphologies, significantly improving the accuracy of key feature characterization. This solves the problem that the microscopic properties of traditional metallic materials are highly dependent on their microstructural features, but these features are subtle, complex, and overlapping.

[0010] This invention proposes a cross-modal fusion strategy based on an attention mechanism. By dynamically allocating weights, it achieves optimal complementarity between image and component features, avoids interference from redundant information, and enhances the model's robustness in representing heterogeneous data.

[0011] For industrial scenarios with small sample sizes, this invention integrates progressive generative adversarial networks for data augmentation. By synthesizing high-resolution, high-fidelity microscopic images, the training dataset is significantly expanded, enabling the model to converge quickly and maintain excellent prediction stability and generalization performance even with a small number of real samples. Attached Figure Description

[0012] Figure 1 This is a flowchart of the steps of the present invention. Detailed Implementation

[0013] The present invention will be further described in detail below with reference to specific embodiments.

[0014] like Figure 1 As shown, a method for evaluating the performance of microscopic images of metallic materials based on convolutional neural networks includes the following steps: S1. Acquire metallographic images of metallic material samples and perform preprocessing operations to obtain a multimodal dataset containing metallographic images, composition data, and mechanical property data. The way of collecting the metallographic image of the metal material sample is: using a metallographic microscope, a scanning electron microscope or a transmission electron microscope to obtain; such images are presented in grayscale or color form and contain rich information levels: from macro metallographic structure to micro grain morphology, phase interface characteristics, and defect structure; modern microtechniques combined with digital image processing can clearly distinguish nanoscale features, providing direct basis for analyzing the correlation between material mechanical properties and microstructure; in the industrial field, micro images are not only the core tool for material research and quality control, but also can automatically extract features through deep learning models to realize rapid prediction and optimization of material performance, becoming the key bridge connecting the micro world and macro performance; The metallographic image of the metal material sample is collected at the same time, and composition data is recorded; The mechanical property data is obtained by actual measurement experiment, including: yield strength YS, tensile strength TS and elongation EL; The pre-processing method includes: light correction and noise suppression; The light correction is used to eliminate the uneven brightness of the whole image caused by uneven light source, optical path defect or difference in camera sensor sensitivity; the light correction method can use the conventional flat field correction method, that is, first collect the original image of the sample, and under the same optical setting, shoot a standard sample with flat surface and uniform features, cover the camera lens cap, and collect a pure black image, through the correction formula: In the formula, represents the original image; represents the pure black image; represents the image shot under the same optical setting; The noise suppression is used to eliminate random noise in the image, and the noise suppression can be median filtering method or non-local mean filtering method; The image is subjected to light correction and noise suppression to realize the standardized data acquisition process to ensure the high quality of the input information, the pre-processing technology improves the image contrast and signal-to-noise ratio, provides a reliable basis for subsequent feature extraction, and reduces the interference of noise on the analysis result.

[0015] S2, using a generative adversarial network (GAN) to expand the sample; In the generative adversarial network (GAN), the generator is based on random noise Z, and adopts a progressive training strategy, that is, gradually performs upsampling operation from low resolution (4x4) until the same image as the original pixel size of 256x256 is obtained; In the generative adversarial network (GAN), the discriminator is based on the original image with the original pixel size of 256x256 and the generated image, and introduces a feature matching loss, when the feature matching loss is less than a threshold value, the random noise Z is adjusted, until the feature matching loss is greater than the threshold value; The expression for feature matching loss is as follows: In the formula, Indicates the discriminator; Represents the true vector; This represents the vector generated by the generator; In this invention, in order to better preserve information about key microstructures such as grain boundaries and phase regions, the discriminator is required to make strict judgments so that the generator can generate realistic images of grain structures with different morphologies, clear grain boundaries, and possible simulated different phase distributions. Therefore, the threshold is set to 0.9.

[0016] S3. Construct a dual-branch residual network, using the metallographic images in the expanded dataset as input, to obtain enhanced feature maps; Two-branch residual networks include: Branch 1: Employs dilated convolution and deformable convolution, using activation functions and batch normalization operations to capture the morphological features of grains in metallographic images; Branch 2: Extract image frequency domain features using Short Time Fourier Transform (STFT); After adding the outputs of branch 1 and branch 2 element by element, a residual connection operation is performed to obtain the enhanced feature map. The steps include: S3.1. Input the metallographic image with an original pixel size of 256×256×1 into the initial convolutional layer for preliminary feature extraction. The initial convolutional layer is set to a 7×7 convolutional layer with a stride of 2 and an output channel of 64. After passing through the ReLU activation function, the output is the preliminary feature extraction result after downsampling. S3.2 Input the preliminary feature extraction results into branch 1 and output a 128×128×64 feature map with geometric shape; In branch 1, the image undergoes dilated convolution, deformable convolution, and activation and normalization operations in sequence. Dilated convolution uses a 3×3 kernel with a dilation rate of 2 to expand the perceptual limit. The input of deformable convolution is the output of dilated convolution. To improve the computation speed, the deformable convolution is set to a 3×3 convolutional layer, which has a total of 2×9=18 channels of offset field, where 2 represents the x and y coordinates of each sampling point. That is, when the output of dilated convolution is set to 9 sampling points in deformable convolution, bilinear interpolation is applied to the sampling points, and then convolution is performed with weights to extract the features of curved grain boundaries. Finally, batch normalization and ReLU activation function are applied sequentially to obtain a 128×128×64 feature map with the morphological features of grains in metallographic images. The expression for convolution with weights is: In the formula, N represents the sampling points, which is 9 in this invention; Indicates the weight of the sampling points; Indicates the bias term; This represents the value of the nth sampling point; S3.3 Input the preliminary feature extraction results into branch 2 and output a 128×128×64 feature map with periodic texture; In branch 2, the image undergoes short-time Fourier transform (STFT) and inverse transform in sequence to obtain a 128×128×64 feature map with periodic texture information in the metallographic image. S3.4 Add the results from branch 1 and branch 2 element by element to obtain the fused image; S3.5. Perform residual connection between the fused image and the preliminary feature extraction results to obtain an enhanced feature map; Compared to traditional convolutional networks, this method reduces information loss and improves the accuracy of key feature representation. While ordinary convolutional layers may transform features layer by layer, residual connections retain the original information through a "short-circuit" mechanism and dynamically adjust weights to adapt to irregular structures. The spatial branch captures grain morphology features at different scales through dilated convolution, while the frequency domain branch extracts image frequency domain features using short-time Fourier transform. Residual connections are embedded in deformable convolutional layers, dynamically adjusting channel weights to adapt to irregular grain boundary morphologies. By fusing multi-scale features to take into account both local details and global structure, deformable convolution enhances the modeling ability for complex micromorphologies, significantly improving the model's accuracy in representing key features such as grain distribution and phase interfaces.

[0017] S4. Construct an attention-guided feature fusion network, using the enhanced feature map and component data from the multimodal dataset as inputs, to obtain the fused feature vector; The steps include: S4.1 Input the enhanced feature map into two fully connected layers; In the first fully connected layer, the 128×128×64 enhanced feature map is subjected to global average pooling for each channel and then projected through the fully connected layer to obtain a 512-dimensional vector. In the second fully connected layer, the 512-dimensional vector is projected to obtain a 128-dimensional vector. ; S4.2. Use 1D-CNN to perform time series modeling of component data; The modeling method is as follows: construct an 8-dimensional sequence of the same length as the number of component data, pass it through a sliding convolution, and then output a vector of the same dimension as the output of the two fully connected layers in S3.1 through a fully connected layer. That is, 128 dimensions; S4.3. Concatenate the outputs of the two fully connected layers and the output of the 1D-CNN, and input the result into the multi-head attention layer to obtain a vector with semantic features. The concatenated sequence is represented as: [ , ]; This embodiment uses an 8-head attention mechanism, which performs attention calculation by querying Q, key K, and value V to obtain image features with component context and component features with image information, represented as […]. , ]; S4.4. Merge the results of the multi-head attention layer, input them into the fully connected network, and obtain two weights through the Softmax function, as shown in the following expression: In the formula, Represents the weights of the image; Indicates the weight of the components; Indicates a fully connected network; S4.5. Based on the obtained weights, weights are assigned to the two fully connected layers and the 1D-CNN, and the fused feature vector is output as follows: In this invention, image information is the primary driver because it directly contains the microstructure of the crystal. Compositional information merely corrects and supplements visual judgment; therefore, image information is given a higher weight, set to 0.7. Thus, the final fused feature vector of this invention is expressed as: Based on weight allocation, redundant information is suppressed. Weight allocation enables the model to autonomously weigh the importance of image and component features, avoiding the subjectivity of manually preset weights and improving the robustness and generalization ability of feature fusion.

[0018] S5. Based on the fused feature vector, use a multi-task output head to predict yield strength, tensile strength and elongation; In the multi-task output header, the batch size is set to 16; the fused feature vector first passes through two fully connected layers to reduce the 128 dimensions to 32 dimensions to improve the computation speed; the predicted yield strength YS, tensile strength TS, and elongation EL are output through activation functions respectively; By combining the mechanical performance data collected from the multimodal dataset, the mean square error (MSE) is calculated to determine the difference between each predicted value and the true value. In the multi-task output head, dynamic loss weights are used to adjust the weights between the three output heads to ensure that there are no fluctuations during task calculation; The calculation method for dynamic loss weight adjustment is as follows: in, For dynamic loss weights, for For yield strength, For tensile strength or For elongation, For dynamic weights, For the first The true value of each performance metric Set dynamic weights for model predictions. ∈ [0.3~0.4]; By capturing the implicit correlations between performance metrics through multi-task learning, dynamic weight allocation prioritizes and optimizes high-uncertainty samples, thereby improving prediction accuracy and enhancing the model's robustness to industrial noise.

[0019] This method for microscopic image analysis and performance evaluation of metallic materials based on convolutional neural networks first collects metallographic images, compositional data, and mechanical property data to construct a multimodal dataset, and preprocesses the images with illumination correction and noise suppression. For small sample scenarios, a progressive adversarial network is used to generate synthetic data and meta-learning is introduced to optimize the initial parameters of the model. Then, a dual-branch residual convolution module is designed. The spatial branch uses dilated convolution to capture grain morphology features, and the frequency domain branch extracts multi-scale features from the image frequency domain. Adaptive fusion of image and compositional features is achieved through a cross-modal fusion network with dynamic weight allocation. Finally, a multi-task output head is constructed to simultaneously predict yield strength, tensile strength, and elongation, and multi-objective optimization is balanced by adjusting the dynamic loss weights. This invention effectively solves the problems of feature dependence on manual design, low efficiency of multimodal fusion, and poor adaptability to small sample scenarios in traditional technologies, and significantly improves the robustness and real-time inference capability of the model while ensuring prediction accuracy.

[0020] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for evaluating performance of a metal material micrograph based on a convolutional neural network, characterized in that, The method comprises the following steps: S1, collecting a metal material sample metallographic image and performing a pretreatment operation to obtain a multi-modal data set containing the metallographic image, composition data and mechanical property data; The mechanical property data includes: yield strength YS, tensile strength TS and elongation EL; S2, using a generative adversarial network GAN for sample expansion; S3, constructing a double-branch residual network, taking the metallographic image in the expanded data set as input, and obtaining an enhanced feature map; The double-branch residual network comprises: Branch 1: adopts a cavity convolution and a deformable convolution, and performs an activation function and a batch normalization operation to capture the morphology of the grains in the metallographic image; Branch 2: adopts a short-time Fourier transform to extract image frequency domain features; After element-by-element addition of the results output by Branch 1 and Branch 2, a residual connection operation is performed to obtain an enhanced feature map; S4, constructing an attention-guided feature fusion network, taking the enhanced feature map and the composition data in the multi-modal data set as input, and obtaining a fusion feature vector; The attention-guided feature fusion network comprises two fully connected layers, a 1D-CNN, an input multi-head attention layer and a fully connected network; S5, based on the fusion feature vector, using a multi-task output head to predict the yield strength, tensile strength and elongation, and completing performance evaluation.

2. The metal material micrograph performance evaluation method based on a convolutional neural network according to claim 1, wherein: In the generative adversarial network GAN, the generator is based on random noise Z and adopts a progressive training strategy, that is, gradually performs an upsampling operation from a resolution of 4x4 until an image with the same size as the original pixel is obtained; In the generative adversarial network GAN, the discriminator is based on the original image with the original pixel size and the generated image, and a feature matching loss is introduced, when the feature matching loss is less than a threshold value, the random noise Z is adjusted until the feature matching loss is greater than the threshold value; The threshold value is set to 0.

9.

3. The metal material micrograph performance evaluation method based on a convolutional neural network according to claim 1, wherein: The step of obtaining an enhanced feature map comprises: S3.1, taking a metallographic image with an original pixel size of 256x256x1 as an original image to input an initial convolutional layer for preliminary feature extraction; The initial convolutional layer is set to a 7x7 convolutional layer, the step size is set to 2, the output channel is set to 64, and the ReLU activation function is used to output the preliminary feature extraction result after down-sampling; S3.2, inputting the preliminary feature extraction result into Branch 1 to output a 128x128x64 feature map with geometric morphology; In Branch 1, the image sequentially passes through a cavity convolution, a deformable convolution and an activation and normalization operation; The cavity convolution uses a 3x3 convolution kernel and the dilation rate is set to 2; The input of deformable convolution is the output of the empty convolution, in order to improve the calculation speed, the deformable convolution is set as a 3x3 convolution layer, with a total of 9 sampling points, after bilinear interpolation of the sampling points, convolution with the weight is used to extract the features of the curved grain boundary, and finally the batch normalization operation and the ReLU activation function are sequentially passed through, so that a 128x128x64 feature map with the morphology features of the grains in the metallographic image is obtained; The expression of convolution with the weight is:

4. where N represents the sampling points; represents the sampling point weight; represents the bias term; represents the value of the nth sampling point; S3.3, input the preliminary feature extraction result into branch 2, and output a 128x128x64 feature map with periodic texture; In branch 2, the image sequentially passes through the short-time Fourier transform STFT and the inverse transform, and a 128x128x64 feature map with periodic texture information in the metallographic image is obtained; S3.4, the results obtained by branch 1 and branch 2 are added element by element, and a fused image is obtained; S3.5, the fused image and the preliminary feature extraction result are connected by residual, and an enhanced feature map is obtained.

5. The metal material micrograph performance evaluation method based on the convolutional neural network according to claim 3, characterized in that: The step of obtaining the fused feature vector comprises: S4.1, input the enhanced feature map into two fully connected layers; In the first fully connected layer, the 128x128x64 enhanced feature map is subjected to a global average pooling operation in each channel and then projected through a fully connected layer to obtain a 512-dimensional vector; In the second layer full connection layer, a 512-dimensional vector is obtained and projected to obtain a 128-dimensional vector ; S4.2, use 1D-CNN to model the time sequence of the composition data; The modeling manner is: the same number of 8-dimensional sequences is constructed according to the number of ingredient data, and after sliding convolution, a vector with the same dimension as the output of the two full connection layers in S3.1 is output through a full connection layer, that is, 128 dimensions ​ S4.3, splice the outputs of the two fully connected layers and the output of the 1D-CNN, input a multi-head attention layer, and obtain a vector with semantic features; The spliced sequence is represented as: , ]; attention calculation is performed by querying Q, key K and value V to obtain image features with component context and component features with image information, represented as , ]; S4.4, merge the results of the multi-head attention layer, input a fully connected network, and obtain two weights through a Softmax function, as follows:

6. In the formula, representing the weight of the image; representing the weight of the component; representing the fully connected network; S4.5, based on the obtained weights, weight allocation is performed on the two fully connected layers and the 1D-CNN, and a fused feature vector is output, as follows: 。 7. The metal material micrograph performance evaluation method based on the convolutional neural network according to claim 1, characterized in that: In the multi-task output head, the batch size is set to 16; the fused feature vector is sequentially subjected to two fully connected layers to reduce the dimension from 128 to 32, so as to improve the operation speed; and the yield strength YS, the tensile strength TS and the elongation EL prediction results are respectively output through the activation function; Combined with the mechanical property data collected by the multi-modal data set, the mean square error MSE is calculated to judge the gap between each prediction value and the true value; In the multi-task output head, dynamic loss weight adjustment is used to adjust the weights among the three output heads, so as to ensure that there is no fluctuation during task calculation; The calculation method of dynamic loss weight adjustment is:

8. wherein, is a dynamic loss weight, belongs to the yield strength or tensile strength or elongation , is a dynamic weight, is the true value of the th performance index, is the model predicted value; set the dynamic weight ∈ [0.3~0.4] between.

Images