Method for quantitatively counting second phase proportion of alloy based on image recognition technology

Abstract

The invention relates to the field of alloy analysis, in particular to a method for quantitatively counting the second phase proportion of alloy based on an image recognition technology, which comprises the following steps: acquiring an alloy sample image; inputting the alloy sample image into an alloy analysis model, and outputting the proportion of the second phase of the alloy in the alloy sample image; the training of the alloy analysis model comprises the following steps: acquiring a first sample image of an alloy; converting the first sample image into a gray scale histogram; querying the number of target gray peaks in the gray level histogram, wherein the target gray peaks refer to the gray peaks except the gray peak of the base phase; according to the number, whether separation processing needs to be carried out on the gray level histogram is judged, and a training sample set is correspondingly obtained; and inputting the training sample set into the deep network learning model to obtain an alloy analysis model.

Description

Technical Field

[0001] The present invention relates to the technical field of alloy analysis, and in particular to a method for quantitatively counting the proportion of a second phase in an alloy based on image recognition technology. Background Art

[0002] The second phase in alloys is crucial to the study of the physical and chemical properties of alloys. At present, in order to improve the efficiency of the second phase statistical proportion, the second phase can be identified, extracted and quantitatively counted in an automated way.

[0003] Among them, the traditional method mainly uses mixed phase images as training samples to complete model construction. For example, CN111696632A discloses a method for characterizing the full-field quantitative statistical distribution of γ' phase microstructure in metal materials. The characterization method includes: step a: labeling the γ' phase, cloud interference, and γ matrix by Labelme to produce standard feature training samples; step b, using BD U-Net to establish a feature recognition and extraction model based on deep learning; step c, collecting the γ' feature map in the metal material to be tested; step d, automatic identification and extraction of the γ' phase; step e, in-situ quantitative statistical distribution characterization of γ' in a large range and full field of view.

[0004] In addition, CN114972300A also discloses a material image segmentation and recognition method based on computer vision and deep learning. The method uses computer vision methods to segment the Trip steel image, and then uses deep learning to train on the segmented results to identify the microstructure of the Trip steel.

[0005] However, the above traditional network training methods have two main defects:

[0006] 1) Poor versatility

[0007] Since the morphology of alloys (e.g., volume fraction of each phase, grain size, morphology and distribution, etc.) is very complex, especially with the change of processing conditions, the microscopic morphology of alloys also changes significantly. However, the existing network models have low accuracy and reliability when counting the second phase data of complex alloy images.

[0008] 2) Low training efficiency

[0009] Due to the complexity of the alloy phase diagram morphology, samples currently need to be manually labeled, so the process of constructing a training sample set is very time-consuming. Summary of the invention

[0010] The purpose of the present invention is to provide a method for quantitatively counting the proportion of the second phase in an alloy based on image recognition technology, which partially solves or alleviates the above-mentioned deficiencies in the prior art and can coordinate the single-phase pretreatment method with the semi-automatic labeling method to further improve the accuracy and reliability of the alloy analysis model on the basis of improving the efficiency of model training.

[0011] In order to solve the above-mentioned technical problems, the present invention specifically adopts the following technical solutions: a method for quantitatively counting the proportion of the second phase of an alloy based on image recognition technology, comprising:

[0012] S100 acquires an image of an alloy sample to be tested;

[0013] S101 inputs the alloy sample image into a pre-trained alloy analysis model, and outputs the proportion of the second phase of the alloy in the alloy sample image; wherein the training step of the alloy analysis model includes:

[0014] S200 acquires a first sample image of the alloy;

[0015] S201 converts the first sample image into a grayscale histogram;

[0016] S202 queries the number of target grayscale peaks in the grayscale histogram, wherein the target grayscale peaks refer to grayscale peaks other than the grayscale peaks of the base phase;

[0017] S203: judging whether the grayscale histogram needs to be separated according to the number, and correspondingly acquiring a training sample set, the training sample set including: a plurality of second sample images converted from the grayscale histogram;

[0018] S204: inputting the training sample set into a deep network learning model to obtain the alloy analysis model;

[0019] Among them, S203 includes:

[0020] S300 determines whether the number of the current grayscale histogram is greater than a preset target number, if so, proceeds to S301, if not, proceeds to S302;

[0021] S301 controls the binarization threshold of the current grayscale histogram to obtain a first separated grayscale histogram including a phase, wherein the phase in the first separated grayscale histogram corresponds to one of the target grayscale peaks, and the phase includes: at least one sub-phase of a shape;

[0022] S302 determines whether the phase in the currently corresponding separated grayscale histogram has a sub-phase with a different shape; if so, proceeds to S303;

[0023] S303: marking the type of the sub-phase in the currently corresponding separated grayscale histogram, and using the separated grayscale histogram as the second sample image.

[0024] In some embodiments, the method further comprises the steps of:

[0025] S304: Subtract the first separated grayscale histogram from the current grayscale histogram to obtain a corresponding second separated grayscale histogram; return to S300.

[0026] In some embodiments, S301 includes the steps of:

[0027] Obtain the left boundary point of the rightmost target grayscale peak of the current grayscale histogram;

[0028] Calculate the binarization threshold according to the left boundary point, wherein the binarization threshold=L / 255, wherein L is the value of the left boundary point;

[0029] The first grayscale histogram corresponding to the current target grayscale peak is separated from the grayscale histogram according to the binarization threshold.

[0030] In some embodiments, the step of marking the type of the subphase comprises:

[0031] (1) obtaining a shape of at least one of the subphases;

[0032] (2) searching for subphase information matching the subphase in a database according to a preset first query rule; wherein the database includes: at least one subphase graphic having a typical geometric structure, and the subphase graphic is associated with the subphase information, and the subphase information includes: a main phase type corresponding to the subphase, and a subphase type of the subphase; wherein the first query rule requires that the shape of the subphase matches the subphase graphic corresponding to the subphase information, and the main phase type of the subphase matches the main phase type of the subphase graphic;

[0033] (3) If the sub-phase information that meets the first query rule is found in (2), the sub-phase is automatically marked according to the sub-phase information.

[0034] In some embodiments, the step of marking the type of the subphase further comprises:

[0035] (4) if the sub-phase graphic that meets the first query rule cannot be found in (2), then searching the database for sub-phase information that matches the sub-phase according to the second query rule; wherein the second query rule requires that the shape of the sub-phase matches the sub-phase graphic corresponding to the sub-phase information;

[0036] (5) If the sub-phase information that meets the second query rule is found in (4), a first prompt signal is sent to the user, wherein the first prompt signal includes: an image of the sub-phase to be marked, the found sub-phase image, and the corresponding sub-phase information;

[0037] (6) Marking the sub-phase in response to a first marking signal input by the user, wherein the first marking signal includes: a sub-phase type.

[0038] In some embodiments, the step of marking the type of the subphase further comprises:

[0039] If the sub-phase graph is not found in either (2) or (4), the sub-phase is marked in response to a second marking signal input by a user, where the second marking signal includes: a sub-phase type.

[0040] In some embodiments, the typical geometric structure includes one or more of the following: a circle, a quasi-circle, a polygon, and a quasi-polygon.

[0041] In some embodiments, the deep network learning model includes: U-net resent34 model and U-netresent18 semantic segmentation model.

[0042] In some embodiments, S200 includes:

[0043] Select multiple alloy samples in different solid solution and aging states;

[0044] performing a polishing process on a plurality of the alloy samples;

[0045] An image acquisition device is used to scan and obtain a high-throughput first sample image of the alloy sample.

[0046] In some embodiments, the alloy includes: an aluminum alloy.

[0047] Beneficial technical effects:

[0048] Contrary to the traditional model training idea, the present invention proposes a sample selection mechanism of step-by-step separation and data screening (that is, a single-phase processing mode). On the one hand, the valid information in the sample is quickly marked through step-by-step processing and multi-level data screening. On the other hand, the invalid information is quickly screened out through the selection mechanism, thereby improving the effectiveness and reliability of the training process while reducing the amount of training sample data.

[0049] Furthermore, for the above single-phase processing mode, the present invention also provides a semi-automatic sample marking method of human-machine collaboration. Among them, double query through main phase type and sub-phase graph can effectively improve the sample marking efficiency in the single-phase processing mode, and the human-machine collaborative mode has higher flexibility and reliability, which helps to improve the accuracy of batch sample marking. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, each element or part is not necessarily drawn according to the actual scale. Obviously, the drawings described below are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without paying creative labor.

[0051] Figure 1 is a flow chart of a model training method in an exemplary embodiment of the present invention;

[0052] Figure 2 It is a schematic diagram of a single-phase extraction process in an exemplary embodiment of the present invention;

[0053] Figure 3 Schematic diagram of the distribution of phases in 2024 alloy;

[0054] Figure 4 Schematic diagram of the segmentation result of the sample image A of the alloy;

[0055] Figure 5 Schematic diagram of the segmentation result of the sample image B of the alloy;

[0056] Figure 6 Schematic diagram of the segmentation result of the sample image C of the alloy;

[0057] Figure 7 is a first calculation result diagram obtained according to the alloy analysis model;

[0058] Figure 8 is a second calculation result diagram obtained according to the alloy analysis model;

[0059] Fig. 9 is a third calculation result diagram obtained according to the alloy analysis model;

[0060] Fig.10 It is a schematic diagram of the module structure of a system in an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0062] Herein, suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present invention, and have no specific meanings by themselves. Therefore, "module", "component" or "unit" can be used mixedly.

[0063] In this document, the terms "upper", "lower", "inner", "outer", "front", "back", "one end", "the other end" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0064] In this document, unless otherwise clearly specified and limited, the terms "installed", "provided with", "connected", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0065] Herein "and / or" includes any and all combinations of one or more of the associated listed items.

[0066] Herein, "plurality" means two or more than two, ie, it includes two, three, four, five, etc.

[0067] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0068] As used in this specification, the term "about" typically means + / - 5% of the stated value, more typically + / - 4% of the stated value, more typically + / - 3% of the stated value, more typically + / - 2% of the stated value, even more typically + / - 1% of the stated value, and even more typically + / - 0.5% of the stated value.

[0069] In this specification, some embodiments may be disclosed in a format of being within a certain range. It should be understood that such description of "being within a certain range" is only for convenience and brevity, and should not be interpreted as a rigid limitation on the disclosed range. Therefore, the description of the range should be considered to have specifically disclosed all possible sub-ranges and independent numerical values ​​within this range. For example, the range The description of should be considered to have specifically disclosed sub-ranges such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6 etc., as well as individual numbers within this range, for example, 1, 2, 3, 4, 5, and 6. The above rules apply regardless of the breadth of the range.

[0070] In metal alloys, in addition to the main matrix phase (primary phase), there are also some relatively small non-matrix phases in different forms. These non-matrix phases are called second phases. The second phase is an important component of the alloy and can cause a variety of physical and chemical effects, giving the alloy more excellent properties.

[0071] Herein, "secondary phase" refers to a general term for all other phases in a material that are different from the matrix phase, and are generally discontinuously distributed in the matrix phase.

[0072] For example, the second phase particles in 2024 aluminum alloy in the aging state are mainly: nano-precipitated phase S phase (Al2CuMg) and larger T phase (Al20Cu2Mn3), see Figure 3 shown.

[0073] Embodiment 1

[0074] like Figure 1 , Figure 2 As shown, the present invention provides a method for quantitatively counting the proportion of the second phase of an alloy based on image recognition technology, comprising:

[0075] S100 acquires an image of an alloy sample to be tested;

[0076] For example, the sample image may be a high-throughput image scanned by a SEM.

[0077] S101: input the alloy sample image into a pre-trained alloy analysis model, and output the second phase of the alloy in the alloy sample image;

[0078] For example, in some embodiments, the alloy analysis model can automatically identify the second phase in the sample image and calculate the content of the second phase.

[0079] Among them, see Figure 1 , Figure 2 As shown, the training steps of the alloy analysis model include:

[0080] S200 acquires a first sample image of the alloy;

[0081] S201 converts the first sample image into a grayscale histogram;

[0082] S202 queries the number of target grayscale peaks in the grayscale histogram, wherein the target grayscale peaks refer to grayscale peaks other than the grayscale peaks of the base phase;

[0083] S203: judging whether the grayscale histogram needs to be separated according to the number, and correspondingly acquiring a training sample set, the training sample set including: a plurality of second sample images converted from the grayscale histogram;

[0084] S204 inputs the training sample set into a deep network learning model to obtain the alloy analysis model.

[0085] Preferably, in this embodiment, a single-phase extraction method is used to extract, mark and filter sample data from sample images.

[0086] In order to optimize the network training process, the present invention proposes a sample selection mechanism of step-by-step separation and data screening. For example, in some embodiments, S203 includes:

[0087] S300 determines whether the number of the current grayscale histogram is greater than a preset target number, if so, proceeds to S301, if not, proceeds to S302;

[0088] S301 controls the binarization threshold of the current grayscale histogram to obtain a first separated grayscale histogram including a phase, wherein the phase in the first separated grayscale histogram corresponds to one of the target grayscale peaks, and the phase includes: at least one sub-phase of a shape;

[0089] S302 determines whether the phase in the current corresponding grayscale histogram has the sub-phase with different shapes; if so, proceed to S303; otherwise, it is considered that the current separated grayscale histogram does not conform to the sample selection mechanism and is removed from the sample data.

[0090] S303: marking the type of the sub-phase in the currently corresponding separated grayscale histogram, and using the separated grayscale histogram as the second sample image.

[0091] For example, in some embodiments, the currently corresponding grayscale histogram may be the first separated grayscale histogram acquired in S301. Alternatively, the currently corresponding grayscale histogram may also be a grayscale histogram whose number is less than the target number in S300.

[0092] In some embodiments, S300 further includes the steps of:

[0093] S304: Subtract the first separated grayscale histogram from the current grayscale histogram to obtain a corresponding second separated grayscale histogram; return to S300.

[0094] The first separated grayscale histogram A refers to the grayscale histogram obtained by controlling the binarization threshold. The second separated grayscale histogram B refers to the grayscale histogram obtained by image algebraic calculation, that is, the image obtained by subtracting the first separated grayscale histogram from the latest grayscale histogram (wherein the latest grayscale histogram refers to the original grayscale histogram, or the second separated grayscale histogram obtained by the last calculation).

[0095] In some embodiments, a phase (also referred to as a main phase, where one main phase corresponds to one grayscale peak) may be composed of at least one sub-phase.

[0096] In some embodiments, when the number of target grayscale peaks in the grayscale histogram is 0 or 1, the shape differences between the phases are mainly considered. If there is no second phase with shape difference in the graph, the graph is deleted from the sample data. Otherwise, the shape information of the relevant second phase can be directly obtained by consulting the data or expert experience, and then the image can be imported into Roboflow for classification and annotation.

[0097] On the contrary, in some embodiments, when the grayscale histogram has at least two peaks in addition to the matrix phase, it is considered that there are at least two second phases with different contrasts. In this case, the binarization threshold can be controlled to achieve a separate segmentation of a certain phase, and then image algebraic operations can be used to segment another phase with no difference in contrast (the phase may also be a mixed phase, composed of different types of second phases with the same contrast).

[0098] In some embodiments, when the first grayscale histogram includes two or more sub-phases of different shapes, the current first grayscale histogram is used as the second sample image, and the second sample image marked with the sub-phase type is input into the semantic segmentation model. Otherwise, the current first grayscale histogram is deleted (i.e., the image is removed from the sample data).

[0099] Taking alloy A as an example, the sample selection mechanism of the above-mentioned step-by-step separation and data screening is explained below:

[0100] Alloy A includes a matrix phase and multiple second phases, such as a, b, and c phases. Among them, the a phase contains two sub-phases, α and β, the b phase contains two sub-phases, θ and η, and the c phase further contains three sub-phases, γ, s, and s'. Different sub-phases often have different geometric shapes. In this embodiment, the target number is set to 1.

[0101] At this time, it is first found that the number of grayscale peaks in the grayscale histogram of alloy A is 3 (ie, the number is greater than the target number), and then the binarization threshold of the grayscale histogram is controlled to separate and obtain the first separated grayscale histogram A1 including only the a phase.

[0102] Subsequently, it is determined whether the first separated grayscale histogram A1 includes two sub-phases of different shapes. If the result is yes (i.e., the histogram A1 includes two sub-phases of α and β of different shapes), it is considered that the first separated grayscale histogram A1 meets the current sample selection mechanism and can be used as training data. On the contrary, if the result is no, it is considered that the current histogram A1 does not meet the selection mechanism, and the current first separated grayscale histogram A1 is deleted from the sample data.

[0103] At the same time, the second separated grayscale histogram B1 is obtained by subtracting the first separated grayscale histogram A1 from the original grayscale histogram, and the same judgment rule is used to continue to judge that the number of target grayscale peaks of the second separated grayscale histogram B1 is still greater than 1. At this time, the binarization threshold of the second separated grayscale histogram B1 is continued to be controlled to separate and obtain a new first separated grayscale histogram A2 (i.e., a histogram including only the b-phase). Then, the new first separated grayscale histogram A2 is processed through S302.

[0104] Finally, the current grayscale histogram (which is the second separated grayscale histogram B1 obtained last time) is subtracted from the current first separated grayscale histogram A2 to obtain a new second separated grayscale histogram B2.

[0105] Different from the traditional training method (the traditional training idea is to obtain a variety of different types of sample images to increase the data set type and thus improve the accuracy of model training), the present invention proposes a sample selection mechanism of step-by-step separation and data screening, which selects single-phase images with mixed subphases as training data sets and screens out the remaining sample data that do not conform to the selection mechanism. Moreover, after experimental verification, this sample selection mechanism can effectively improve the effectiveness of the training data set, that is, improve the accuracy and versatility of the obtained alloy analysis model.

[0106] In some embodiments, S301 includes the steps of:

[0107] Obtain the left boundary point of the rightmost target grayscale peak of the current grayscale histogram;

[0108] Calculate the binarization threshold according to the left boundary point, wherein the binarization threshold=L / 255, wherein L is the value of the left boundary point;

[0109] The first grayscale histogram corresponding to the current target grayscale peak is separated from the grayscale histogram according to the binarization threshold.

[0110] In this embodiment, a unidirectional sequence from right to left is used to perform single-phase processing step by step to improve the accuracy and reliability of the obtained training data.

[0111] Next, taking alloy A as an example, the above-mentioned unidirectional step-by-step treatment method is described in detail:

[0112] The grayscale histogram rgb ranges of a, b, and c are respectively mixed phase a (78-96), mixed phase b (123-166), and mixed phase c (203-244), where the mixed phase refers to a main phase that includes two or more sub-phases. At this time, the following steps need to be followed:

[0113] (1) The binarization threshold of the mixed phase c obtained from the original grayscale histogram is 203. After normalization (calculation formula: 203 / 255), the threshold is selected as 0.91. When the binarization threshold is set to 0.91, only the mixed phase c is bright, and the other two mixed phases and the substrate are converted to dark colors. At this time, the separate segmentation result of the mixed phase c is obtained.

[0114] (2) Perform image algebraic operations on the original grayscale histogram and the mixed phase c to obtain an image containing only two mixed phases, a and b.

[0115] (3) Repeat step (1), import the image in (2), calculate the second binarization threshold, and control the second binarization threshold to obtain the mixed phase b extraction result.

[0116] (4) Repeat step (2) to obtain an image containing only the a-mixed phase.

[0117] Furthermore, in order to improve the efficiency of sample labeling, the present invention also provides a semi-automatic labeling method for human-machine collaboration suitable for a single-phase processing mode, so as to improve the efficiency of sample labeling while effectively ensuring the reliability and accuracy of automatic labeling.

[0118] In some embodiments, the step of marking the type of the subphase comprises:

[0119] (1) obtaining a shape of at least one of the subphases;

[0120] (2) searching for subphase information matching the subphase in a database according to a preset first query rule; wherein the database includes: at least one subphase graphic having a typical geometric structure, and the subphase graphic is associated with the subphase information, and the subphase information includes: the main phase type corresponding to the subphase (such as a, b, c), and the subphase type of the subphase (such as α, β); wherein the first query rule requires that the shape of the subphase matches the subphase graphic corresponding to the subphase information, and the main phase type of the subphase matches the main phase type of the subphase graphic;

[0121] (3) If the sub-phase information that meets the first query rule is found in (2), the sub-phase is automatically marked according to the sub-phase information, that is, the automatic marking is completed by default at this time.

[0122] In some embodiments, the step of marking the type of the subphase further comprises:

[0123] (4) if the sub-phase graphic that meets the first query rule cannot be found in (2), then searching the database for sub-phase information that matches the sub-phase according to the second query rule; wherein the second query rule requires that the shape of the sub-phase matches the sub-phase graphic corresponding to the sub-phase information;

[0124] (5) If the sub-phase information that meets the second query rule is found in (4), a first prompt signal is sent to the user, wherein the first prompt signal includes: an image of the sub-phase to be marked, the found sub-phase image, and the corresponding sub-phase information;

[0125] (6) In response to the first marking signal input by the user, the sub-phase is marked, wherein the first marking signal includes: a sub-phase type. That is, a manual verification is prompted at this time, and the manual verification can be manually selected. The sub-phase type given by the prompt signal can be marked after the verification is passed. Alternatively, the user can manually re-enter the sub-phase type to be marked.

[0126] In some embodiments, the step of marking the type of the subphase further comprises:

[0127] If the sub-phase graph is not found in either (2) or (4), the sub-phase is marked in response to a second marking signal input by a user, where the second marking signal includes: a sub-phase type.

[0128] In some embodiments, for a subphase with a typical geometric structure, a computer is used to automatically identify the difference between the actual subphase shape and the standard subphase shape in the database. When the difference between the two is less than a preset difference threshold, the two are considered to match.

[0129] In some embodiments, the typical geometric structure includes one or more of the following: a circle, a quasi-circle (eg, an ellipse), a polygon, and a quasi-polygon.

[0130] For example, the quasi-polygon may be a geometric shape between a square and a circle, such as a quasi-polygon whose corner is an arc corner, or a quasi-polygon whose adjacent line segment near the corner is an arc line segment. Typical quasi-polygons may include: a quasi-rectangle, a quasi-triangle, and the like.

[0131] In this embodiment, the dual query rules are used in conjunction with the single-phase processing mode to quickly perform semi-automatic labeling on typical sub-phases. Among them, the three types of methods, default automatic labeling, manual verification labeling, and manual labeling, are used in coordination, which can greatly reduce the workload of manual labeling through automation, and at the same time, the accuracy and reliability of sample labeling can be ensured by using human-machine collaboration.

[0132] In some embodiments, the deep network learning model includes: U-net resent34 model and U-netresent18 semantic segmentation model.

[0133] For example, in some embodiments, the model training process includes:

[0134] Build the U-net resent34 model, which adds a 34-layer residual network to the Unet classic semantic segmentation model, including: Building the basic structure of the Unet model. The Unet model consists of an encoder (downsampling path) and a decoder (upsampling path), and the feature maps of the encoder and decoder are merged through jump connections. Here, ResNet34 is used as the encoder to extract image features. Build the encoder and decoder parts of Unet. Subsequently, the training sample set (specifically, before the training sample set is input into the semantic segmentation network, it can also be pre-processed by cropping, flipping, edge processing, random rotation, etc.) is imported into the U-net resent34 semantic segmentation network.

[0135] Prepare the loss function and optimizer, where the loss function uses the cross entropy function to calculate the loss value, uses the Adam optimizer to update the model parameters, and calls the optimizer's backward() and step() methods to perform backpropagation and parameter update operations.

[0136] The training is performed for 50 epochs. In each epoch, it trains, calculates the loss values ​​on the training set and the test set in turn, and stores the loss values ​​in the corresponding lists. Through the iterative training process, the changes in training and test losses can be tracked through the training process visualization to evaluate the performance and progress of the model.

[0137] Use evaluate to evaluate miou. The module named evaluate is imported. The load function is called from the evaluate module and the returned result is assigned to the mean_iou variable. The model is set to evaluation mode. Iterate each batch of the test dataset and move the image data and label data to the specified device (such as GPU) for calculation. Calculate the output of the model through forward propagation. Get the prediction results from the output of the model. Find the maximum value and its index in each sample output. Convert the prediction results and labels to NumPy arrays and store them in a list. Add the prediction results and labels of the batch to the mean_iou object. Calculate the results of the evaluation indicators. The num_labels parameter is the number of categories, and the ignore_index parameter is used to specify the label index to be ignored. The result here is the calculation result of the evaluation indicator. Finally, result contains the results calculated by the evaluation indicator, which can be used to evaluate the performance of the model on the test set.

[0138] Visualize the semantic segmentation results, observe the difference between the predicted results and the actual results in the visualization, and display them in a visual effect.

[0139] In some embodiments, S200 includes:

[0140] Select multiple alloy samples in different solid solution and aging states;

[0141] performing a polishing process on a plurality of the alloy samples;

[0142] An image acquisition device is used to scan and obtain a high-throughput first sample image of the alloy sample.

[0143] The following takes 2024 aluminum alloy as an example (see Figure 3-Figure 7 As shown), the second phase statistical method of the present invention is described:

[0144] Standard aluminum alloy specimens in different solid solution and aging states were selected, polished, and SEM scanning high-throughput images were obtained. The input RGB true color image was read and converted into a grayscale image, and its grayscale histogram was obtained accordingly. Normalization and binarization were performed, the binarization threshold was controlled, the segmentation of the second phase based on different contrasts was achieved, and the number of pixels of the second phase Al2Cu in the binary image was traversed pixel by pixel to statistically calculate the proportion of the second phase Al2Cu. After obtaining the proportion of the Al2Cu phase, a series of morphological processing can be performed on the image, such as corrosion opening operation, distance transformation of the binary image (watershed transformation), and counting the number of connected components. After obtaining all connected domains, the bounding box attributes of the region are extracted, and the rectangle function is used to draw a red rectangle on the image to represent the bounding box of each region. At the same time, the region number is displayed next to the centroid coordinates of each region. Using image algebraic operations, A is removed on the basis of the original image. l2 Cu phase, and the organizational morphology of two second phases, Al2CuMg and iron-containing phase, were obtained. The SEM scanning electron microscope images of the two second phases, Al2CuMg and iron-containing phase, were annotated with roboflow software, a label was added to the iron-containing phase, and 184 labeled images were exported.

[0145] Finally, a series of operations are performed on the exported 184 images, such as cropping and image enhancement, so that the sample is expanded from 184 to 1472. The 1472 images are divided according to the principle of equal-proportional stratified sampling, where the ratio of the training set, test set, and validation set is 6:3:1. After the dataset is imported, the model is generated, the loss function and optimizer are prepared, and training and evaluation are performed. Evaluate is used to evaluate miou, and the results are visualized.

[0146] The present invention also attempts to test the generalization ability of multiple models, including U-net resent18 and U-net resent34 models, import the training set data into two different segmentation models to obtain two training models, and use the trained models for prediction, and compare the prediction accuracy of each model. After running the training results of each model, traverse pixel by pixel, count the number of pixels of the iron-containing phase, and calculate the phase ratio. Binarize the image, control the binarization threshold, and realize the separation of the iron-containing phase. Perform image algebraic operations on the original image, the separated iron-containing phase and the Al2Cu phase to obtain the Al2CuMg phase.

[0147] Furthermore, the present invention also verifies the reliability of the network training process by using Al2CuMg alloy. Figure 4-Figure 6 The single-phase processing process of some training sample images is shown. Specifically, Figure 4-Figure 6Figures (a), (b), (c) and (d) are the original sample image, the predicted image after the semantic segmentation network, and two separated single-phase images. Furthermore, the alloy analysis model is obtained by training the samples after the single-phase treatment. Figure 7-Figure 9 The results of automatic labeling of different sample photos of Al2CuMg alloy by the alloy analysis model are shown in sequence, namely Figure 7-Figure 9 This is an illustration of the training effect. After analyzing the original sample images and verifying and comparing the existing alloy data, it can be seen that Figure 7-Figure 9 The obtained automatic marking results have high accuracy and meet the user's analysis needs for the second phase of alloys.

[0148] Embodiment 2

[0149] like Fig.10 As shown, the present invention also provides a system for quantitatively counting the proportion of the second phase of an alloy based on image recognition technology, corresponding to the above-mentioned embodiment 1, including:

[0150] An image acquisition module 10 is configured to acquire an image of an alloy sample to be detected;

[0151] The alloy analysis module 11 is configured to input the alloy sample image into a pre-trained alloy analysis model, and output the proportion of the second phase of the alloy in the alloy sample image;

[0152] A sample acquisition module 20, configured to acquire a first sample image of the alloy;

[0153] A grayscale conversion module 21 is configured to convert the first sample image into a grayscale histogram;

[0154] A grayscale peak query module 22 is configured to query the number of target grayscale peaks in the grayscale histogram, wherein the target grayscale peak refers to a grayscale peak other than a grayscale peak of a base phase;

[0155] The sample separation processing module 23 is configured to determine whether the grayscale histogram needs to be separated according to the quantity, and correspondingly obtain a training sample set, the training sample set including: a plurality of second sample images converted from the grayscale histogram;

[0156] A network training module 24 is configured to input the training sample set into a deep network learning model to obtain the alloy analysis model;

[0157] The sample separation processing module 23 further includes:

[0158] The first judging unit 30 is configured to judge whether the number of the current grayscale histogram is greater than a preset target number, and if so, input the current grayscale histogram to the first separation unit 32, and if not, input the current grayscale histogram to the second judging unit 34;

[0159] The first separation unit 31 is configured to control a binarization threshold of the current grayscale histogram to obtain a first separated grayscale histogram including a phase, wherein the phase in the first separated grayscale histogram corresponds to one of the target grayscale peaks, and the phase includes: at least one sub-phase of a shape;

[0160] The second judgment unit 32 is configured to judge whether the phase in the currently corresponding separated grayscale histogram has a sub-phase with a different shape; if so, input the current grayscale histogram into the sample marking unit;

[0161] The sample marking unit 33 is configured to mark the type of the sub-phase in the currently corresponding grayscale histogram, and use the separated grayscale histogram as the second sample image.

[0162] In some embodiments, the sample separation processing module 23 includes:

[0163] The second separation unit 34 is configured to obtain a second separated grayscale histogram by subtracting the first separated grayscale histogram from the current grayscale histogram; and input the second separated grayscale histogram to the first judgment unit 30 .

[0164] In some embodiments, the first separation unit 31 is further configured to obtain the left boundary point of the rightmost target grayscale peak of the grayscale histogram currently corresponding to the current grayscale level;

[0165] Calculate the binarization threshold according to the left boundary point, wherein the binarization threshold=L / 255, wherein L is the value of the left boundary point;

[0166] The first separated grayscale histogram corresponding to the current target grayscale peak is separated from the grayscale histogram according to the binarization threshold.

[0167] In some embodiments, the sample labeling unit 33 includes:

[0168] The sub-phase acquisition sub-unit 33a is configured to acquire the shape of at least one of the sub-phases;

[0169] The first query subunit 33b is configured to retrieve subphase information matching the subphase in a database according to a preset first query rule; wherein the database includes: at least one subphase graphic having a typical geometric structure, and the subphase graphic is associated with the subphase information, and the subphase information includes: a main phase type corresponding to the subphase, and a subphase type of the subphase; wherein the first query rule requires that the shape of the subphase matches the subphase graphic corresponding to the subphase information, and the main phase type of the subphase matches the main phase type of the subphase graphic;

[0170] The first marking subunit 33c automatically marks the subphase according to the subphase information if the subphase information that meets the first query rule is found in the first query subunit 33b.

[0171] In some embodiments, it also includes:

[0172] The second query subunit 33d is configured to retrieve subphase information matching the subphase in the database according to a second query rule if the subphase graphic that meets the first query rule is not found in the first query subunit 33b; wherein the second query rule requires that the shape of the subphase matches the subphase graphic corresponding to the subphase information;

[0173] The second marking sub-unit 33e is configured to send a first prompt signal to the user if the sub-phase information that meets the second query rule is queried in the second query sub-unit 33d, and the first prompt signal includes: an image of the sub-phase to be marked, the queried sub-phase image and the corresponding sub-phase information; and mark the sub-phase in response to the first marking signal input by the user, and the first marking signal includes: sub-phase type.

[0174] In some embodiments, it also includes:

[0175] The third marking subunit 33f is configured to mark the subphase in response to a second marking signal input by a user if the subphase graph is not found in either the first query subunit 33b or the second query subunit 33d, wherein the second marking signal includes: a subphase type.

[0176] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0177] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a computer terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0178] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

[0179] References

[0180] [1] Wan Weihao et al. "Identification, extraction and quantitative statistical analysis of second phase in large-size aluminum alloy based on deep learning algorithm." Rare Metal Materials and Engineering 002(2022):051.

Claims

1. A method for quantitatively counting the proportion of the second phase of an alloy based on image recognition technology, characterized in that: include: S100 acquires an image of an alloy sample to be tested; S101 inputs the alloy sample image into a pre-trained alloy analysis model, and outputs the proportion of the second phase of the alloy in the alloy sample image; wherein the training step of the alloy analysis model includes: S200 acquires a first sample image of the alloy; S201 converts the first sample image into a grayscale histogram; S202 queries the number of target grayscale peaks in the grayscale histogram, wherein the target grayscale peaks refer to grayscale peaks other than the grayscale peaks of the base phase; S203: judging whether the grayscale histogram needs to be separated according to the number, and correspondingly acquiring a training sample set, the training sample set including: a plurality of second sample images converted from the grayscale histogram; S204: inputting the training sample set into a deep network learning model to train and obtain the alloy analysis model; Among them, S203 includes: S300 determines whether the number of the current grayscale histogram is greater than a preset target number, if so, proceeds to S301, if not, proceeds to S302; S301 controls the binarization threshold of the current grayscale histogram to obtain a first separated grayscale histogram including a phase, wherein the phase in the first separated grayscale histogram corresponds to one of the target grayscale peaks, and the phase includes: at least one sub-phase of a shape; S302 determines whether the phase in the current corresponding grayscale histogram has a sub-phase with a different shape; if so, proceeds to S303; S303: marking the type of the sub-phase in the currently corresponding grayscale histogram, and using the grayscale histogram as the second sample image.

2. The method according to claim 1, characterized in that Also includes the steps: S304: Subtract the first separated grayscale histogram from the current grayscale histogram to obtain a corresponding second separated grayscale histogram; return to S300.

3. The method according to claim 2, characterized in that S301 includes the steps of: Obtain the left boundary point of the rightmost target grayscale peak of the grayscale histogram corresponding to the current grayscale level; Calculating the binarization threshold according to the left boundary point, wherein the binarization threshold=L / 255, wherein L is the value of the left boundary point; The first separated grayscale histogram corresponding to the current target grayscale peak is separated from the grayscale histogram according to the binarization threshold.

4. The method according to claim 2, characterized in that: The step of marking the type of the subphase comprises: (1) obtaining a shape of at least one of the subphases; (2) searching for subphase information matching the subphase in a database according to a preset first query rule; wherein the database includes: at least one subphase graphic having a typical geometric structure, and the subphase graphic is associated with the subphase information, and the subphase information includes: a main phase type corresponding to the subphase, and a subphase type of the subphase; wherein the first query rule requires that the shape of the subphase matches the subphase graphic corresponding to the subphase information, and the main phase type of the subphase matches the main phase type of the subphase graphic; (3) If the sub-phase information that meets the first query rule is found in (2), the sub-phase is automatically marked according to the sub-phase information.

5. The method according to claim 4, characterized in that The step of marking the type of the subphase further includes: (4) if the sub-phase graphic that meets the first query rule cannot be found in (2), searching the database for the sub-phase information that matches the sub-phase according to the second query rule; wherein the second query rule requires that the shape of the sub-phase matches the sub-phase graphic corresponding to the sub-phase information; (5) If the sub-phase information that meets the second query rule is found in (4), a first prompt signal is sent to the user, wherein the first prompt signal includes: an image of the sub-phase to be marked, the found sub-phase graphic, and the corresponding sub-phase information; (6) Marking the sub-phase in response to a first marking signal input by the user, wherein the first marking signal includes: a sub-phase type.

6. The method according to claim 5, characterized in that The step of marking the type of the subphase further includes: If the corresponding sub-phase graph is not found in either (2) or (4), the sub-phase is marked in response to a second marking signal input by the user, where the second marking signal includes: a sub-phase type.

7. The method according to claim 4, characterized in that The typical geometric structures include one or more of the following: circle, quasi-circle, polygon and quasi-polygon.

8. The method according to claim 1, characterized in that The deep network learning model includes: U-netresent34 model and U-net resent18 semantic segmentation model.

9. The method according to claim 1, characterized in that: The alloy includes: aluminum alloy.

10. The method according to claim 1, characterized in that S200 includes: Select multiple alloy samples in different solid solution and aging states; performing a polishing process on a plurality of the alloy samples; An image acquisition device is used to scan and obtain a plurality of high-throughput first sample images of the alloy sample.

Images