Nickel-based superalloy grain segmentation extraction method based on VGG and U-Net fusion model

Abstract

The invention relates to the technical field of nickel-based superalloy grain segmentation and extraction, in particular to a nickel-based superalloy grain segmentation and extraction method based on a VGG and U-Net fusion model. The method comprises the following steps: preparing an Inconel 740H high-temperature alloy sample; carrying out an in-situ tensile experiment according to the manufactured sample, and observing the microstructure by using a scanning electron microscope in the experiment process; acquiring image data according to different stages of the experiment; performing data enhancement on the acquired image according to the brightness contrast; marking according to the characteristics of the crystal grains to be segmented; obtaining a reference image data set according to the annotation file; performing normalization processing according to the reference image data set to obtain a data set; designing a feature extraction module according to the VGG network model; designing a feature fusion module according to the U-Net network model; carrying out fusion according to the two designed modules to obtain a V-Unet fusion model; designing a loss function according to gray information and structural information of the grain boundary area of the reference image grain relative to other areas, and obtaining a loss function weight of the grain boundary area of the reference image grain; calculating according to the weight of the loss function to obtain model feedback; training according to the V-Unet fusion model to obtain a model; carrying out grain segmentation extraction on an Inconel 740H high-temperature alloy scanning electron microscope image according to the model; performing morphological post-processing according to a segmentation extraction result to obtain a grain image result; and performing statistics and data analysis according to a grain image result. The method can be used for segmenting, extracting and counting the same type of grains at different positions of the nickel-based superalloy and refining the grains to improve the mechanical property of the superalloy, and has great significance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of nickel-based superalloy grain segmentation extraction, and particularly relates to a nickel-based superalloy grain segmentation extraction method based on a VGG and U-Net fusion model. BACKGROUND

[0002] The nickel-based superalloy grain refers to a large number of nickel atoms and alloy element atoms arranged according to a certain crystal structure in a nickel-based superalloy material, and has specific grain boundaries, dislocations and other microstructure characteristics. The size, morphology and orientation of the nickel-based superalloy grain significantly affect the basic structural unit of the functional characteristics of the alloy, such as high-temperature strength, creep resistance and fatigue resistance. The grain boundary contains a large number of dislocations and vacancies, and reducing the grain size is beneficial to improving the mechanical properties of the high-temperature alloy, especially the application performance at high temperature. Therefore, the grain extraction analysis is of great significance for grain refinement to improve the mechanical properties of the high-temperature alloy.

[0003] In the existing grain boundary extraction technology, although the Canny, Sobel and Kirsch edge detection operators can effectively locate the edges, they are easily disturbed by many factors. Due to the target metal, the internal grain itself will produce a false boundary during preparation and data acquisition, and part of the grain boundary will be lost, and thus scratches, unclear grain boundaries and uneven internal grain gray distribution often exist in the collected images, which will interfere with the segmentation and extraction of the grain. SUMMARY

[0004] In order to solve the technical problems that the existing technology cannot overcome the scratches in the collected image, the unclear grain boundary, the uneven internal gray distribution of the grain, and the like, and it is difficult to better segment and extract the grains, the purpose of the present application is to provide a nickel-based high-temperature alloy grain segmentation and extraction method based on a VGG and U-Net fusion model, and the technical scheme adopted is specifically as follows: A nickel-based high-temperature alloy grain segmentation and extraction method based on a VGG and U-Net fusion model, the method comprises the following steps: preparing an Inconel 740H high-temperature alloy sample; performing an in-situ tensile test according to the preparation, observing the microstructure during the experiment by using a scanning electron microscope; collecting image data according to different stages of the experiment; performing data enhancement on the collected images according to brightness contrast; labeling according to the features of the grains to be segmented; obtaining a reference image dataset according to the labeling file; performing normalization processing on the reference image dataset to obtain a dataset; designing a feature extraction module according to a VGG network model; designing a feature fusion module according to a U-Net network model; fusing the two modules designed to obtain a V-UNet fusion model; designing a loss function according to the gray information and structure information of the grain boundary region of the reference image grain compared with other regions to obtain the loss function weight of the grain boundary region of the reference image grain; calculating the loss function weight to obtain model feedback; training the V-UNet fusion model to obtain a model; performing grain segmentation and extraction on the Inconel 740H high-temperature alloy scanning electron microscope image according to the model; performing morphological post-processing on the segmentation and extraction results to obtain a grain image result; and performing statistics and analysis according to the grain image result.

[0005] Further, the method for preparing the Inconel 740H high-temperature alloy sample comprises: solid solution treatment on the sample; water quenching; air cooling; and using wire cutting processing.

[0006] Further, the in-situ tensile test method comprises: performing a tensile test on the prepared sample until fracture at room temperature by using an in-situ tensile test system, and observing the microstructure and collecting image data during the experiment by using a scanning electron microscope.

[0007] Further, the dataset obtaining method comprises: performing data enhancement according to the contrast brightness of the image, and the data enhancement principle formula is as follows: In the formula, I out represents the gray value result of the data enhanced image; I in represents the input image pixel point gray value; and ε represents the related index of image enhancement.

[0008] Further, the V-UNet fusion model obtaining method comprises the following steps: a feature extraction module is designed based on a VGG model, input image information is converted into a tensor, and feature maps are extracted from shallow to deep in the tensor; a feature fusion module is designed based on a U-Net model, convolution blocks are combined, and the extracted feature maps are subjected to special fusion, and original image information and sizes are gradually recovered from deep information; a skip connection is added to corresponding layers of the feature extraction module and the feature fusion module, and is used for image feature information transmission.

[0009] Further, the model feedback obtaining method comprises the following steps: a loss function is designed according to gray information and structure information of a grain boundary area of a reference image grain compared with other areas, and the model feedback is calculated by using the loss function, and the loss function calculation formula is as follows: In the formula, A represents a pixel value of a grain reference image, B represents a pixel value of a prediction result, |A∩B| is approximately a dot product between the prediction result and the pixel of the grain reference image, and |A|+|B| is approximately pixels in corresponding images of them.

[0010] Further, the model file is obtained by training based on the V-UNet fusion model, comprising the following steps: each grain is divided into a segmentation target along a grain boundary, and the remaining area of the image is taken as a background image; the model is trained according to the loss function until the loss value reaches a preset value, and a trained model is obtained; and the trained model is used to segment and extract the grains.

[0011] Further, the morphological post-processing method comprises the following steps: the model segmentation and extraction result image is subjected to thresholding; a closing operation is used for smoothing processing, and the principle formula is as follows: C(A,B)=E(D(A,B),B), wherein A represents an input image, B represents a structure element used for corrosion, D(A,B) represents a dilated result image, E(A,B) represents a corroded result image, and C(A,B) represents a closing operation result image; grain boundary detection is performed according to morphological edge detection, extraction is performed along a grain boundary contour, and a grain image result is obtained.

[0012] The present application has the following excellent effects: the present application uses heat treatment and processing means to manufacture Inconel 740H high-temperature alloy samples; in-situ tensile experiments are carried out according to the manufactured samples, and the microstructure is observed by a scanning electron microscope during the experiment process; since stress is continuously applied during the experiment process until the material breaks, image data is collected at different stages of the experiment, ensuring that there is sufficient sample for deep learning model training; since the contrast and brightness of the collected image data change little, the image data is subjected to data enhancement, which improves the model generalization ability while enriching the data; the image data is marked along the grain boundary profile and converted into a grain reference image; since only extreme gray scale information is needed to segment and extract grains, the image data is subjected to normalization processing to obtain a data set, reducing computer operation and improving efficiency; since the grains have special image features, the down-sampling of the U-Net network model is optimized, i.e., the feature extraction part, the VGG network model is used as a feature extraction module to fuse the up-sampling part of the U-Net network model to obtain a V-Unet fusion model; since the grain boundary region of the grain has differences in gray scale information and structure information compared to other regions, a loss function is designed to calculate the weight to obtain model feedback; the model is trained according to the V-Unet fusion model; the model is used to segment and extract grains from the Inconel 740H high-temperature alloy scanning electron microscope image; since the model segmentation result may have incomplete or incorrect segmentation of the grain boundary, morphological post-processing is performed according to the segmentation and extraction result to obtain a grain image result; the grain image result is subjected to statistical and data analysis. The present application takes into account the scratch and grain boundary loss of the grain in the scanning electron microscope image, and designs a unique feature extraction and feature fusion module, which increases the segmentation and generalization ability of the deep learning segmentation network model and improves the completeness and accuracy of the segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the schemes in the embodiments of the present application or the prior art, the drawings needed in the implementation examples or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0014] Figure 1 FIG. 1 is a flow chart of a nickel-based high-temperature alloy grain segmentation and extraction method based on a VGG and U-Net fusion model according to an embodiment of the present application. DETAILED DESCRIPTION

[0015] In order to further illustrate the technical means and effects taken by the present application to achieve the predetermined inventive purpose, the specific implementation, structure, features and effects of a high-temperature alloy precipitated phase segmentation and analysis method and system based on deep learning according to the present application are described in detail as follows. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0016] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0017] The specific scheme of the nickel-based high-temperature alloy grain segmentation extraction method based on the VGG and U-Net fusion model provided by the present application is described in detail below in combination with the drawings.

[0018] Please refer to Figure 1 which shows a nickel-based high-temperature alloy grain segmentation extraction method based on a VGG and U-Net fusion model provided by an embodiment of the present application. The method comprises the following steps: step S1: preparing an Inconel 740H high-temperature alloy sample; performing in-situ tensile test according to the prepared sample, observing the microstructure during the test process by using a scanning electron microscope; and collecting image data according to different stages of the test.

[0019] The embodiment of the present application is mainly applied to the nickel-based high-temperature alloy grain image segmentation scene. In order to obtain the nickel-based high-temperature alloy grain image data, first, an Inconel 740H high-temperature alloy sample is prepared, an in-situ tensile test is performed, and image data during the test process is collected. In the embodiment of the present application, a deep learning model is used to segment the nickel-based high-temperature alloy grain, and the model network needs a large number of data set samples, i.e. scanning electron micrographs containing nickel-based high-temperature alloy grains, to ensure that the data set samples of the model network are sufficient.

[0020] Preferably, in an embodiment of the present application, the method for preparing an Inconel 740H high-temperature alloy sample comprises: solid solution treatment of the sample at 1150℃ for 30 minutes, then water quenching, followed by air cooling at 800℃ for 4 hours, using wire cutting discharge machining to process the tubular material into a tensile specimen with a length of 42mm, and obtaining the Inconel 740H high-temperature alloy sample.

[0021] Preferably, in one embodiment of the present application, the in-situ tensile test method comprises: using an in-situ tensile test system at room temperature to perform a tensile test on an Inconel 740H sample until fracture, with the displacement being positively correlated with the stress, ranging from 0 to 500 microns, as the stress gradually increases from 0 MPa to 630 MPa, during which the sample undergoes elastic deformation, and in the later stage, the sample begins to deform plastically; at different stages of the experiment, scanning electron microscope images of the microstructure are collected.

[0022] Step S2: data enhancement is performed on the collected images according to brightness contrast; the grain features to be segmented are labeled; reference image data sets are obtained according to the labeled files; and data sets are obtained by normalizing the reference image data sets.

[0023] In one embodiment of the present application, the original data set is a scanning electron microscope image with a resolution of 1024x948 and a magnification range of 70 to 1500 times. Since the contrast and brightness of the collected image data change little, the image data is enhanced to improve the model generalization ability and enrich the data. Since the pixel gray value of the segmentation target object is greatly different from other regions, the pixel points of the grain boundaries are labeled manually. Since only extreme gray information is needed for grain segmentation and extraction, the image data is normalized to obtain the data set, thereby reducing the computer operation amount and improving the efficiency.

[0024] Preferably, in one embodiment of the present application, the data enhancement method comprises: data enhancement is realized according to a data enhancement principle formula, and the data enhancement principle formula is as follows: In the formula, I out represents the gray value result of the data enhanced image; I in represents the input image pixel gray value; and epsilon represents the related index of image enhancement.

[0025] Step S3: a feature extraction module is designed according to a VGG network model; a feature fusion module is designed according to a U-Net network model; and a V-UNet fusion model is obtained by fusing the two designed modules.

[0026] In one embodiment of the present application, the V-UNet fusion model obtaining method comprises: a feature extraction module is designed based on a VGG model, the input image information is converted into a tensor, and feature maps are extracted from shallow to deep in the tensor; a feature fusion module is designed based on a U-Net model, convolution blocks are combined, and the extracted feature maps are fused, so that the deep information gradually restores the original image information and size; and a skip connection is added to the corresponding layers of the feature extraction module and the feature fusion module, and is used for image feature information transmission.

[0027] Preferably, in one embodiment of the present application, the feature extraction module design method comprises: the input layer reshapes the input scanning electron microscope image into a tensor with a size of 512*512*3, and extracts the grain image features based on the pre-trained VGG16 network as the backbone.

[0028] Preferably, in one embodiment of the present application, the feature fusion module design method comprises: aligning and converting the grain image features by using a 1*1 convolution block, constantly fusing the grain image features by using different convolution blocks, and gradually restoring the image information and size, and adding a skip connection corresponding to the level of the feature extraction module in the process.

[0029] Step S4: Designing a loss function according to the gray scale information and structural information of the grain boundary region of the reference image grain compared with other regions, obtaining the loss function weight of the grain boundary region of the reference image grain, and calculating the model feedback according to the loss function weight.

[0030] According to the above process, the gray scale information and structural information of the grain boundary region of each grain are obtained. Since the grain boundary region of the grain is similar to the features of other regions of the image, and the grain boundary region information is easily disturbed by scratch noise, the loss function is designed to calculate according to the feature information to obtain the model feedback.

[0031] Preferably, in one embodiment of the present application, the model feedback obtaining method comprises: obtaining the loss weight according to the loss function formula, and calculating the model feedback, and the calculation formula is as follows: In the formula, A represents the pixel value of the grain reference image, B represents the pixel value of the prediction result, |A∩B| is approximately the dot product between the prediction result and the pixel of the grain reference image, and |A|+|B| is approximately the pixel of the corresponding image.

[0032] Step S5: Training the model according to the V-Unet fusion model; performing grain segmentation and extraction on the Inconel 740H high-temperature alloy scanning electron microscope image according to the model; performing morphological post-processing according to the segmentation and extraction result to obtain the grain image result; and performing statistical and data analysis according to the grain image result.

[0033] Preferably, in one embodiment of the present application, the model is obtained by training the V-Unet fusion model, which comprises: freezing the backbone network for training, and then unfreezing the backbone network for fine-tuning, so as to quickly adjust the model, reduce the consumption of computing resources, and avoid overfitting; the initial learning rate is set to 0.0001, and is dynamically adjusted according to the cosine smoothing decay, so as to achieve better convergence.

[0034] The model obtained through training is used for segmentation processing of the nickel-based superalloy grain image, so that the problem of incomplete segmentation result and category confusion caused by similar features of the target and the background image is avoided to the greatest extent, the accuracy of segmentation is improved, and the segmentation of the nickel-based superalloy grain is completed.

[0035] Preferably, in an embodiment of the present application, the morphological post-processing method comprises: thresholding the model segmentation extraction result image; and performing smoothing processing using morphological closing operation, and the principle formula is as follows:

[0036] C(A,B)=E(D(A,B),B), wherein A represents an input image; B represents a structure element for corrosion; D(A,B) represents a dilation result image; E(A,B) represents a corrosion result image; C(A,B) represents a closing operation result image; grain boundary detection is performed according to morphological edge detection, extraction is performed along the grain boundary profile, and a grain image result is obtained.

[0037] Thus far, the segmentation and extraction of the nickel-based superalloy grain are completed.

[0038] To sum up, an Inconel 740H superalloy sample is made; in-situ tensile experiments are performed according to the made sample, the microstructure is observed during the experiments by using a scanning electron microscope; image data is collected according to different stages of the experiments; the collected images are subjected to data enhancement according to brightness contrast; the features of the to-be-segmented grains are labeled; reference image data sets are obtained according to the labeled files; data sets are obtained by performing normalization processing according to the reference image data sets; a feature extraction module is designed according to a VGG network model; a feature fusion module is designed according to a U-Net network model; a V-Unet fusion model is obtained by fusing the two designed modules; a loss function weight of the grain boundary region of the reference image grain is obtained by designing a loss function according to the gray information and structure information of the grain boundary region of the reference image grain compared with other regions; a model feedback is obtained by calculating the loss function weight; a model is obtained by training the V-Unet fusion model; the grains of the Inconel 740H superalloy scanning electron microscope image are segmented and extracted according to the model; a grain image result is obtained by performing morphological post-processing according to the segmentation and extraction result; and statistics and data analysis are performed according to the grain image result.

[0039] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are also possible or can be advantageous.

[0040] The various embodiments described in this specification are presented by way of example, and each embodiment is not inherently more important than any other embodiment.

Claims

1. A nickel-based superalloy grain segmentation and extraction method based on VGG and U-Net fusion model, characterized in that: The method includes: preparing an Inconel 740H high-temperature alloy sample; conducting an in-situ tensile test on the prepared sample, and observing the microstructure with a scanning electron microscope during the experiment; collecting image data according to different stages of the experiment; performing data enhancement on the collected image according to brightness contrast; annotating according to features of grains to be segmented; obtaining a reference image data set according to an annotation file; performing normalization processing on the reference image data set to obtain a data set; designing a feature extraction module according to a VGG network model; designing a feature fusion module according to a U-Net network model; fusing the two designed modules to obtain a V-UNet fusion model; designing a loss function according to grayscale information and structural information of a grain boundary region of a reference image grain compared with other regions, and obtaining a loss function weight of the grain boundary region of the reference image grain; obtaining model feedback according to calculation of the loss function weight; training the V-UNet fusion model to obtain a model; performing grain segmentation and extraction on an Inconel 740H high-temperature alloy scanning electron microscope image according to the model; performing morphological post-processing on the segmentation and extraction results to obtain a grain image result; and performing statistics and analysis on the grain image result.

2. The nickel-based high-temperature alloy grain segmentation and extraction method based on the VGG and U-Net fusion model according to claim 1 is characterized in that: The method for preparing an Inconel 740H high-temperature alloy sample comprises: performing a solution treatment on the sample; performing water quenching; performing air cooling; and performing wire cutting processing.

3. The nickel-based high-temperature alloy grain segmentation and extraction method based on the VGG and U-Net fusion model according to claim 1 is characterized in that: The in-situ tensile test method includes: performing a tensile test on a prepared sample at room temperature using an in-situ tensile test system until it breaks, and observing the microstructure and collecting image data using a scanning electron microscope during the test.

4. The method for segmenting and extracting nickel-based high-temperature alloy grains based on the VGG and U-Net fusion model according to claim 1, characterized in that: The data set acquisition method includes: performing data enhancement according to image contrast and brightness, and the data enhancement principle formula is as follows: Where, I out Represents the image grayscale value result of data enhancement; I in represents the grayscale value of the input image pixel; ε represents the correlation index of image enhancement; the enhanced image data is annotated along the grain boundary of the grain; the grain reference image data is converted according to the annotation file; the data set is obtained by normalization based on the grayscale value information of the grain reference image.

5. The method for segmenting and extracting nickel-based high-temperature alloy grains based on the VGG and U-Net fusion model according to claim 1, characterized in that: The V-UNet fusion model acquisition method includes: designing a feature extraction module based on the VGG model, converting input image information into a tensor, and extracting feature maps from the tensor from shallow to deep; designing a feature fusion module based on the U-Net model, combining convolution blocks to perform special fusion on the extracted feature maps, and gradually restoring the original image information and size from deep information; and adding skip layer connections between corresponding layers of the feature extraction module and the feature fusion module for transmitting image feature information.

6. The method for segmenting and extracting nickel-based high-temperature alloy grains based on the VGG and U-Net fusion model according to claim 1, characterized in that: The model feedback obtaining method includes: designing a loss function based on the grayscale information and structural information of the grain boundary region of the reference image grain compared with other regions, and calculating the model feedback using the loss function. The loss function calculation formula is as follows: Where A represents the pixel value of the grain reference image, B represents the pixel value of the prediction result, and |A∩B| is approximately the dot product between the prediction result and the grain reference image pixel. |A|+|B| is approximately the sum of the pixels in their respective corresponding images.

7. The method for segmenting and extracting nickel-based high-temperature alloy grains based on the VGG and U-Net fusion model according to claim 1, characterized in that: The model file is obtained by training based on the V-UNet fusion model, including: dividing each grain along the grain boundary into a segmentation target, and the rest of the image area is used as the background image; training the model according to the loss function until the loss value reaches a preset value, and then obtaining the trained model; and segmenting and extracting the grains according to the trained model.

8. The method for segmenting and extracting nickel-based high-temperature alloy grains based on the VGG and U-Net fusion model according to claim 1, characterized in that: The morphological post-processing method includes: thresholding a model segmentation extraction result image; performing smoothing using a morphological closing operation, the principle formula of which is as follows: C(A, B)=E(D(A, B), B), where A represents an input image; B represents a structural element used for corrosion; D(A, B) represents an expansion result image; E(A, B) represents an corrosion result image; and C(A, B) represents a closing operation result image; performing grain boundary detection based on morphological edge detection, extracting along the grain boundary contour, and obtaining a grain image result.

Images