Metallographic impurity identification and impurity identification method, equipment, system and storage medium

Abstract

The embodiment of the invention provides a metallographic impurity recognition method, device and system, an impurity recognition method, device and system and a storage medium. According to the method, impurity surface phase feature extraction is carried out on microscopic images, with different magnification times, of a metal sample, so that impurity surface phase features, with different particle sizes, of the same impurity can be obtained, and the surface phase features of the impurities are presented with different finenesses. And due to the impurity surface phase characteristics with different finesses, the distinguishing degree of the surface phase characteristics of the metallographic inclusions and the surface phase characteristics of the surface pollutants can be more obvious, and whether the impurities are the metallographic inclusions or the surface pollutants can be distinguished more conveniently. Therefore, whether the impurities contained in the metal sample are metallographic inclusions or surface pollutants can be distinguished according to the surface phase characteristics of the impurities contained in the microscopic images with different magnification times, and the types of the impurities contained in the metal sample can be identified. Therefore, when metallographic grading is carried out on the metallographic inclusions, interference of surface pollutants can be eliminated, and improvement of the accuracy of metallographic grading of the inclusions is facilitated.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a metallographic impurity identification and impurity identification method, device, system and storage medium. Background Art

[0002] Metallographic analysis is one of the important means of experimental research on metal materials. It adopts the principle of quantitative metallography and determines the three-dimensional spatial morphology of the alloy structure by measuring and calculating the metallographic microstructure of the ground surface of a two-dimensional metallographic sample or a thin film, thereby establishing a quantitative relationship between alloy composition, structure and performance.

[0003] Metallographic inclusions have many effects on the properties of metal materials. Therefore, the study of metallographic inclusions in metal materials has always been an important topic in metal smelting. Accurate metallographic analysis results can make the production and application of metal products more efficient and reliable.

[0004] The inventors of this application have found that the impurities contained in metal materials during metallographic analysis generally include metallographic inclusions and surface contaminants. Metallographic inclusions have many effects on the quality and performance of metal materials, while surface contaminants are brought in during the metallographic analysis process and do not affect the quality and performance of the original metal materials. Therefore, accurately distinguishing between metallographic inclusions and surface contaminants has an important impact on the accuracy of metallographic analysis results. Summary of the invention

[0005] Various aspects of the present application provide a metallographic impurity identification and impurity identification method, device, system and storage medium for identifying the type of metallographic impurities, which helps to improve the accuracy of subsequent metallographic grading of inclusions.

[0006] The present application embodiment provides a metallographic impurity identification method, comprising:

[0007] Acquire multiple microscopic images of the metal sample; the multiple microscopic images have multiple magnifications;

[0008] Extracting impurity surface phase features from the plurality of microscopic images to obtain impurity surface phase features contained in the plurality of microscopic images;

[0009] According to the impurity surface phase characteristics contained in the multiple microscopic images, the impurity type of the impurities contained in the metal sample is identified; the impurity type is metallographic inclusions or surface contaminants.

[0010] The present application also provides an impurity identification method, including:

[0011] Acquire multiple microscopic images of the target object; the multiple microscopic images have multiple magnifications;

[0012] Extracting impurity surface phase features from the plurality of microscopic images to obtain impurity surface phase features contained in the plurality of microscopic images;

[0013] According to the impurity surface characteristics contained in the multiple microscopic images, the impurity type of the impurities contained in the target object is identified.

[0014] The embodiment of the present application also provides a metallographic impurity identification system, comprising: a microscope and a computing device; the microscope is used to collect microscopic images of metal samples at different magnifications to obtain multiple microscopic images with multiple magnifications; and the multiple microscopic images are provided to the computing device;

[0015] The computing device is used to execute the steps in the above-mentioned metallographic impurity identification method.

[0016] The embodiment of the present application further provides a computing device, comprising: a memory and a processor; wherein the memory is used to store a computer program;

[0017] The processor is coupled to the memory and is used to execute the computer program to execute the steps in the above-mentioned metallographic impurity identification method and / or impurity identification method.

[0018] An embodiment of the present application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the above-mentioned metallographic impurity identification method and / or steps in the impurity identification method.

[0019] In an embodiment of the present application, by extracting the impurity surface characteristics of the microscopic images of the metal sample at different magnifications, the impurity surface characteristics of different particle sizes of the same impurity can be obtained, so that the impurity surface characteristics have different fineness. The impurity surface characteristics with different fineness can make the surface characteristics of metallographic inclusions and the surface characteristics of surface contaminants more clearly distinguishable, and it is easier to distinguish whether the impurities are metallographic inclusions or surface contaminants. Therefore, according to the impurity surface characteristics contained in the microscopic images of different magnifications, it can be distinguished whether the impurities contained in the metallographic inclusions are metallographic inclusions or surface contaminants, and the type of impurities contained in the metal sample can be identified. In this way, when metallographic inclusions are metallographically graded, the interference of surface contaminants can be eliminated, which helps to improve the accuracy of the metallographic rating of inclusions. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0021] Figure 1A schematic diagram of the structure of a metallographic impurity identification system provided in an embodiment of the present application;

[0022] Figure 2 A schematic diagram of a microscopic image acquisition process of a metal sample provided in an embodiment of the present application;

[0023] Figure 3 A schematic diagram of a specific process of metallographic impurity identification provided in an embodiment of the present application;

[0024] Figure 4a A schematic diagram of a process flow of a metallographic impurity identification method provided in an embodiment of the present application;

[0025] Figure 4b A schematic diagram of a flow chart of an impurity identification method provided in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0028] In some embodiments of the present application, by extracting the impurity surface characteristics of the microscopic images of the metal sample at different magnifications, the impurity surface characteristics of different particle sizes of the same impurity can be obtained, so that the impurity surface characteristics have different fineness. The impurity surface characteristics with different fineness can make the surface characteristics of metallographic inclusions and the surface characteristics of surface contaminants more clearly distinguishable, and it is easier to distinguish whether the impurities are metallographic inclusions or surface contaminants. Therefore, according to the impurity surface characteristics contained in the microscopic images of different magnifications, it can be distinguished whether the impurities contained in the metallographic inclusions are metallographic inclusions or surface contaminants, and the type of impurities contained in the metal sample can be identified. In this way, when metallographic inclusions are metallographically graded, the interference of surface contaminants can be eliminated, which helps to improve the accuracy of the metallographic rating of inclusions.

[0029] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0030] It should be noted that the same reference numerals denote the same objects in the following drawings and embodiments, and therefore, once an object is defined in one drawing or embodiment, it does not need to be further discussed in the subsequent drawings and embodiments.

[0031] Figure 1 This is a schematic diagram of the structure of the metallographic impurity identification system provided in the embodiment of the present application. Figure 1 As shown, the system mainly includes: a microscope 10 and a computing device 20.

[0032] In the present embodiment, the microscope 10 refers to a device that can scan and capture images of the microscopic morphology of a substance. The microscope 10 can be configured with a communication component and has communication capabilities. Of course, the microscope 10 can also have certain computing capabilities, etc. In the present embodiment, the microscope 10 can scan and capture the metallographic phase of a metal material. Metallography refers to the chemical composition of a metal or alloy and the physical and chemical states of various components inside the alloy. In the embodiment of the present application, the specific implementation form of the microscope 10 is not limited. Optionally, the microscope 10 can be a metallographic microscope or a scanning electron microscope (SEM for short), etc.

[0033] Among them, the metallographic microscope is a product that combines optical microscope technology, photoelectric conversion technology, and computer image processing technology. It can observe metallographic images on the computer, thereby analyzing and rating the metallographic maps.

[0034] Scanning electron microscopy is an observation method between transmission electron microscopy and optical microscopy. It uses a focused narrow high-energy electron beam to scan samples, and stimulates various physical information through the interaction between the beam and the material. This information is collected, amplified, and re-imaged to achieve the purpose of characterizing the microscopic morphology of the material.

[0035] The computing device 20 refers to a computer device with computing, communication and other functions. The computing device 20 can be a single server device, a cloud server array, or a virtual machine (VM) running in a cloud server array. In addition, the computing device 20 can also refer to other computing devices with corresponding computing service capabilities, such as computers, mobile phones, wearable devices and other terminal devices.

[0036] In this embodiment, the computing device 20 can be integrated with the microscope 10 as the same device, or can be an independent device. The computing device 20 is connected to the microscope 10 for communication. The computing device 20 and the microscope 10 can be connected wirelessly or by wire. Optionally, the computing device 20 can be connected to the microscope 10 via a mobile network communication. Accordingly, the network standard of the mobile network can be any one of 2G (such as Global System for Mobile Communications (GSM)), 2.5G (such as General Packet Radio Service (GPRS)), 3G (such as Wideband Code Division Multiple Access (WCDMA), Time Division-Synchronous Code Division Multiple Access (TD-SCDMA), Code Division Multiple Access 2000 (CDMA2000), Universal Mobile Telecommunications System (UTMS)), 4G (such as Long Term Evolution (LTE)), 4G+ (such as upgraded version of Long Term Evolution (LTE-Advanced, LTE-A)), 5G, World Interoperability for Microwave Access (WiMax), etc. Optionally, the computing device 20 may be communicatively connected to the microscope 10 via Bluetooth, Wireless Fidelity (WiFi), infrared, or the like.

[0037] In this embodiment, the observation position and magnification of the microscope 10 can be adjusted dynamically. The microscope 10 collects microscopic images of the metal sample at a certain magnification. The metal sample refers to a metal sample obtained by cutting and sampling the metal material to be analyzed; the metal sample is mechanically ground and polished; then, the ground and polished metal sample can be metallographically etched to obtain a metal sample that meets the detection requirements of the microscope 10.

[0038] Among them, metallographic etching is one of the processes in metallographic sample preparation, including chemical etching and electrolytic etching. Chemical etching is to corrode the polished surface of the sample in a chemical corrosive for a certain period of time to show the sample's microstructure. The equipment used for electrolytic etching is the same as that for electrolytic polishing, but the working voltage and working current are smaller than those for electrolytic polishing. At this time, a thin film is generally not formed on the grinding surface of the sample. Due to the different potentials between the phases and between the grains and the grain boundaries, the corrosion degree of each phase is different under the action of weak current, thus showing the microstructure.

[0039] In the embodiments of the present application, the specific element composition of the metal is not limited. The metal may be steel, aluminum, copper or other metals or alloys. In actual industrial production, metal materials inevitably contain impurities. In the present embodiment, the impurity types of the impurities contained in the metal material mainly include: metallographic inclusions and surface contaminants.

[0040] Among them, metallographic inclusions refer to inclusions in metal materials, which are brought by the smelting process of metal materials and affect the quality and performance of metal materials. Metallographic inclusions can be divided into two categories: metal inclusions and non-metallic inclusions. Metal inclusions are also called special-shaped metals. It is an inclusion caused by foreign unmelted metal. There are many reasons for the formation of metal inclusions in steel. For example, when smelting metal materials, it is necessary to use the raw material alloy corresponding to the metal material for smelting. For example, when smelting steel, ferroalloys need to be added for smelting. When adding the raw material alloy of the metal material, sometimes it is not completely melted due to the time of addition or the inappropriate addition position, and other metals in the raw material alloy will remain in the metal material, forming metal inclusions in the metal material.

[0041] For example, in the process of smelting steel, a large amount of ferroalloy is added or the ferroalloy block is too large, especially those ferroalloys with high melting points, such as tungsten iron and metallic nickel. Sometimes they are not completely melted due to the time of addition or the inappropriate addition location, and they will remain in the steel to form metal inclusions in the steel.

[0042] Of course, metal inclusions in metal materials may also be caused by other reasons. For example, when pouring steel, thermite (a mixture of aluminum powder and iron oxide) is added to the cap of the steel ingot. If the amount is too much, it will accumulate and not melt in time, or because the iron oxide particles are large, segregation occurs, causing local molten steel to be oxidized and decarburized, and the crystallization temperature rises and solidifies first and sinks into the molten steel, which may form metal inclusions in the steel. The carbon content of this inclusion will be lower than that of the base steel. After the ingot is poured, in order to facilitate demolding, a round steel with a different composition from the ingot is often inserted into the cap. If the cap is inserted too deep into the ingot body, or the insertion time is too early, the molten steel has not yet solidified and sinks into the ingot body, and part of it will be pressed into the steel during pressure processing, forming metal inclusions. When the molten steel is poured, splashing occurs, and the small particles formed fail to melt and disperse in the steel, which will also form metal inclusions in the steel. Due to operational negligence during steel tapping, other metals that fall into the steel ladle may become metal inclusions in the steel if they are not completely melted by the molten steel and have a uniform composition.

[0043] Metal inclusions in metal materials are mostly caused by accidental external factors. Therefore, as long as effective measures are taken and every detail of the operation is paid close attention to during the smelting and pouring of metal materials, so as to prevent the occurrence of the above-mentioned situations, most metal inclusions in steel can be completely avoided.

[0044] Non-metallic inclusions in metal materials are non-metallic substances formed when a small amount of slag, refractory materials and reaction products in smelting enter the metal liquid (such as molten steel) during the metal smelting process. They are an inevitable component in metal materials. Non-metallic inclusions can include: oxides, sulfides, carbides, nitrides and borides. The presence of these non-metallic inclusions in metal materials will affect their performance and quality. The defects of metal materials caused by non-metallic inclusions seriously affect the performance and quality of metal materials. Therefore, metallographic analysis mainly focuses on the analysis of non-metallic inclusions in metal materials.

[0045] Metallographic inclusions can be tested and graded according to the testing standards, such as GB / T10561-2005. According to the morphology and distribution characteristics of metallographic inclusions, metallographic inclusions can be mainly divided into five categories: (1) Category A (sulfide): single gray inclusions with high ductility and a wide range of morphology ratios, generally with rounded ends; (2) Category B (alumina): most of them are not deformed, angular, with a small morphology ratio (generally <3), black or bluish particles, arranged in a row along the rolling direction, with at least 3 particles; (3) Category C (silicate): single black or dark gray inclusions with high ductility and a wide range of morphology ratios (generally ≥3), generally with sharp ends; (4) Category D (spherical oxide): non-deformed, angular or rounded, with a small morphology ratio (generally <3), black or bluish, and irregularly distributed particles; (5) Category DS (single-particle spherical): single-particle inclusions that are round or nearly round and have a diameter ≥13 microns (μm).

[0046] There is no unified testing standard for surface contaminants. Surface contaminants are formed during the production process of the above-mentioned metal samples, that is, the metal undergoes various surface processing in the air, especially the chemical reactions that occur during heat treatment or thermal processing and storage. Since surface contaminants are formed during the production process of metal samples, the metal material itself does not contain surface contaminants. Therefore, surface contaminants will not affect the performance and quality of the metal material.

[0047] For metal samples with surface contaminants attached to their surfaces, since the surface phase characteristics of the surface contaminants and metallographic inclusions are similar, it may not be possible to directly distinguish whether the impurities contained in the metal samples are surface contaminants or metallographic inclusions, which may lead to errors in subsequent metallographic ratings.

[0048] In the embodiment of the present application, in order to improve the accuracy of metallographic grading, a method for identifying metallographic impurities is proposed, which is mainly used to identify the impurity types of metallographic impurities. Among them, the impurity types of metallographic impurities include: metallographic inclusions and surface contaminants. The metallographic impurity identification in the embodiment of the present application mainly refers to identifying whether the impurities contained in the metal sample are metallographic inclusions or surface contaminants.

[0049] Specifically, the microscope 10 collects microscopic images of the metal sample at different magnifications to obtain multiple microscopic images with multiple magnifications. Among them, multiple refers to 2 or more. Multiple refers to 2 or more. There is at least one microscopic image of one magnification. Further, the microscope 10 can provide multiple microscopic images with multiple magnifications to the computing device 20.

[0050] Since the microscope 10 has a limited field of view, when collecting a microscopic image of a metal sample at a magnification A, the microscope can scan the metal sample in a certain order, and collect a local area of ​​the metal sample each time during the scanning process until the scanning of the metal sample is completed, thereby obtaining multiple microscopic images with a magnification A. For example, Figure 2 As shown, it is assumed that the acquisition field of view of the microscope 10 can cover Figure 2 If there is a small rectangle (such as D00-D24) in the metal sample, the metal sample can be scanned in a certain order, and the area corresponding to a small rectangle of the metal sample is collected each time.

[0051] The microscope 10 may collect microscopic images of the metal sample under the control of the computing device 20. In some embodiments, the multiple magnifications may include: a first magnification and a second magnification, wherein the first magnification is smaller than the second magnification.

[0052] The microscope 10 may first collect a microscopic image of the metal sample at a first magnification to obtain a first microscopic image with a first magnification as a partial microscopic image of the multiple microscopic images with various magnifications. The first microscopic image may be one or more. Multiple refers to 2 or more, which is specifically determined by the size of the metallographic sample and the collection field of view of the microscope 10 at the first magnification. The number of the first microscopic images is less than the number of the multiple microscopic images with various magnifications.

[0053] Further, the microscope 10 may provide the first microscopic image to the computing device 20. The computing device 20 may perform impurity detection on the first microscopic image to determine the position information of the impurities contained in the metal sample on the metal sample (defined as target position information). The position information of the impurities contained in the metal sample in the metal sample (i.e., target position information) may be represented by the point coordinates of the impurities contained in the metal sample on the metal sample. For example, the boundary contour of the impurities contained in the metal sample may be represented by the point coordinates on the metal sample; or, the point coordinates of the center of mass of the impurities contained in the metal sample may be represented by the point coordinates on the metal sample, and so on.

[0054] Specifically, the computing device 20 can perform impurity detection on the first microscopic image to determine the position information of impurities contained in the metal sample in the first microscopic image; further, the position information of impurities contained in the metal sample in the collection area corresponding to the first microscopic image can be determined based on the first magnification and the position information of impurities contained in the metal sample in the first microscopic image; thereafter, the position information of impurities contained in the metal sample on the metal sample can be determined based on the position information of impurities contained in the metal sample in the collection area corresponding to the first microscopic image and the position information of the collection area corresponding to the first microscopic image on the metal sample.

[0055] The position information of the image of the impurities contained in the metal sample in the first microscopic image can be represented by the pixel coordinates of the image of the impurities contained in the metal sample in the first microscopic image. For example, the pixel coordinates of the boundary contour of the image of the impurities contained in the metal sample in the first microscopic image are used to represent it; or the pixel coordinates of the centroid of the image of the impurities contained in the metal sample in the first microscopic image are used to represent it, and so on.

[0056] In some embodiments, the computing device 20 may run a pre-trained impurity detection model, and use the impurity detection model to perform impurity detection on the first microscopic image to obtain the position information of the image of impurities contained in the metal sample in the first microscopic image. The impurity detection model may be a neural network model. In the embodiment of the present application, the specific implementation form of the impurity detection model is not limited. Optionally, the impurity detection model may be a convolutional neural network (CNN), a deep learning neural network (DNN), a recurrent neural network (RNN) or a convolutional neural tensor network (CNTN), etc., but is not limited thereto.

[0057] The impurity detection model is trained using microscopic images of image positions of known impurities as training samples. Specifically, microscopic image samples of image positions with impurities pre-annotated can be obtained. Afterwards, the initial model corresponding to the impurity detection model is trained with the loss function minimization as the training target and the microscopic image samples of image positions with impurities pre-annotated as training samples to obtain the impurity detection model.

[0058] The loss function may be determined based on the difference between the predicted image position of the impurity output by the model training and the image position of the impurity marked in advance for the same microscopic image sample. For example, the loss function may be expressed as the mean square error or distance variance between the predicted image position of the impurity corresponding to the same microscopic image sample and the image position of the impurity marked in advance, or the cross entropy between the predicted image position of the impurity corresponding to the same microscopic image sample and the image position of the impurity marked in advance.

[0059] The process of training the impurity detection model may be completed by the computing device 20 or by other computing devices, and the trained impurity detection model may be provided to the computing device 20 or the like.

[0060] After the computing device 20 determines the target position information of the impurities contained in the metal sample on the metal sample, the microscope can be controlled to adjust the magnification to a second magnification. The second magnification is greater than the first magnification. For example, the first magnification may be 100 times magnification; the second magnification may be 200 times magnification, etc.

[0061] Furthermore, the computing device 20 can control the microscope 10 to adjust the observation position to the target observation position according to the target position information of the impurities contained in the metal sample on the metal sample. When the microscope 10 collects the microscopic image at the target observation position with the second magnification, the collected field of view covers the area corresponding to the target position information, thereby realizing local magnification of the impurities contained in the metal sample.

[0062] Furthermore, the microscope 10 can be controlled to collect a microscopic image of the metal sample at a second magnification at the target observation position to obtain a second microscopic image with a second magnification as a partial microscopic image of the plurality of microscopic images with a plurality of magnifications. Furthermore, the microscope 10 can also provide the second microscopic image to the computing device 20.

[0063] The above only takes the multiple magnifications including two magnifications, namely, the first magnification and the second magnification, as an example to illustrate the implementation method of obtaining microscopic images at multiple magnifications, which does not constitute a limitation. Of course, the multiple magnifications can also be implemented as more than two magnifications, such as three magnifications, four magnifications, etc., but is not limited thereto.

[0064] For multiple magnifications, the microscope can adjust the observation position of the microscope according to the order of magnification from small to large, and the position information of the impurities in the metal sample obtained by impurity detection based on the microscopic image of the previous magnification and the adjusted next magnification, and collect the microscopic image of the metal sample at the next magnification at the adjusted observation position. Among them, the next magnification is greater than the previous magnification. Each time the microscopic image collected is provided to the computing device 20, the computing device 20 obtains multiple microscopic images with multiple magnifications.

[0065] Furthermore, the computing device 20 may extract impurity surface phase features from the multiple microscopic images to obtain the impurity surface phase features contained in the multiple microscopic images.

[0066] Furthermore, the computing device 20 may identify the impurity type of the impurities contained in the metal sample according to the surface characteristics of the impurities contained in the plurality of microscopic images, that is, identify whether the impurities contained in the metal sample are metallographic inclusions or surface contaminants.

[0067] In this embodiment, by extracting the impurity surface characteristics of the microscopic images of the metal sample at different magnifications, the impurity surface characteristics of different particle sizes of the same impurity can be obtained, so that the impurity surface characteristics have different fineness. The impurity surface characteristics with different fineness can make the surface characteristics of metallographic inclusions and the surface characteristics of surface contaminants more clearly distinguishable, and it is easier to distinguish whether the impurities are metallographic inclusions or surface contaminants. Therefore, according to the impurity surface characteristics contained in the microscopic images of different magnifications, it can be distinguished whether the impurities contained in the metallographic inclusions are metallographic inclusions or surface contaminants, and the type of impurities contained in the metal sample can be identified. In this way, when metallographic inclusions are graded, the interference of surface contaminants can be eliminated, which helps to improve the accuracy of the metallographic rating of inclusions.

[0068] In the embodiment of the present application, when extracting impurity surface features and classifying impurities in microscopic images, a neural network model can be used to extract impurity surface features and classify impurities. In the embodiment of the present application, the neural network model used for extracting impurity surface features and classifying impurities in microscopic images is defined as a metallographic impurity recognition model. The specific implementation form of the metallographic impurity recognition model can be CNN, RNN, DNN or CNTN, etc. Alternatively, the metallographic impurity recognition model can also be a classification model, such as a random forest model, a gradient boosting decision tree (Gradient Boosting Decision Tree, GBDT) or a boosting tree model (XGBoost), etc.

[0069] Before using the metallographic impurity recognition model to extract impurity surface features and classify impurities from the microscopic image, the metallographic impurity recognition model needs to be trained. Specifically, multiple microscopic image samples of the metal sample specimen can be obtained. The multiple microscopic image samples have multiple magnifications. Among them, the number of magnifications of the multiple microscopic image samples can be the same as the number of magnifications of the multiple microscopic images actually used online.

[0070] For example, the input of the metallographic impurity recognition model in the training stage is: two microscopic image samples of the same metal sample with different magnifications. When used online, the input of the metallographic impurity recognition model is also: two microscopic images of the same metal sample with different magnifications.

[0071] The metal sample specimen may be one or more. Multiple refers to two or more. The microscopic image samples of the metal sample specimen are pre-labeled with: image position information of impurities contained in the metal sample specimen in each microscopic image sample and impurity type of impurities contained in the metal sample specimen. For the representation of the image position information of impurities contained in the metal sample specimen in each microscopic image sample, please refer to the relevant content of the image position information of impurities contained in the metal sample specimen in the microscopic image, which will not be repeated here.

[0072] Furthermore, taking minimization of the loss function as a training objective and a plurality of microscopic image samples as training samples, the initial model corresponding to the metallographic impurity recognition model is trained to obtain the metallographic impurity recognition model.

[0073] The loss function is determined at least according to the difference between the predicted impurity type of the impurities contained in the metal sample sample output by the model and the impurity type of the impurities contained in the pre-labeled metal sample sample. For example, in some embodiments, the loss function can be implemented as: the mean square error, variance or standard deviation, etc. between the predicted impurity type of the impurities contained in the metal sample sample output by the model and the impurity type of the impurities contained in the pre-labeled metal sample sample.

[0074] In some embodiments, assuming that the impurity type of metallographic inclusions is pre-marked as 1, and the impurity type of surface contaminants is pre-marked as 0, the loss function can be expressed as:

[0075]

[0076] Wherein, in the above formula (1), N represents the total number of microscopic image samples, and i represents the i-th microscopic image sample. i=1, 2, ..., N. i represents the impurity type pre-labeled in the i-th microscopic image sample, p i Equal to 0 or 1. It represents the value corresponding to the predicted impurity type of the i-th microscopic image sample, which is generally any value between 0 and 1.

[0077] Of course, in some embodiments, the loss function can also be jointly determined based on the difference between the predicted impurity type of impurities contained in the metal sample specimen output by the model and the impurity type of impurities contained in the pre-labeled metal sample specimen, and the difference between the predicted image position of impurities contained in the metal sample specimen output by the model in each microscopic image sample and the image position information of impurities contained in the pre-labeled metal sample specimen in each microscopic image sample.

[0078] Among them, the difference between the image predicted position of impurities contained in the metal sample sample output by the model in each microscopic image sample and the image position information of impurities contained in the metal sample sample labeled in advance in each microscopic image sample can be determined by the mean square error, variance or standard deviation of the distance between the image predicted position of impurities contained in the metal sample sample output by the model in each microscopic image sample and the image position information of impurities contained in the metal sample sample labeled in advance in each microscopic image sample.

[0079] For example, in some embodiments, the difference between the image prediction position of the impurities contained in the metal sample sample output by the model in each microscopic image sample and the image position information of the impurities contained in the metal sample sample in each microscopic image sample that is pre-labeled can be expressed as:

[0080]

[0081] In formula (2), N represents the total number of microscopic image samples, and i represents the i-th microscopic image sample. i = 1, 2, ..., N. (x i ,y i ) represents the image position information of the impurities contained in the pre-marked metal sample specimen in each i-th microscopic image sample. It represents the image predicted position of the impurities contained in the metal sample sample output by the model in each microscopic image sample of the i-th image.

[0082] Accordingly, the loss function can be expressed as: L = L1 + L2. Accordingly, the input of the metallographic impurity recognition model can be multiple microscopic images with multiple magnifications, and the output can be: image position information of impurities contained in the metal sample on each microscopic image, and the impurity type of the impurities on the image position information (metallographic inclusions or surface contaminants).

[0083] The implementation forms of the loss functions shown in the above embodiments are only exemplary and do not constitute limitations.

[0084] Based on the trained metallographic impurity recognition model, the computing device 20 can use the metallographic impurity recognition model to perform impurity detection on multiple microscopic images respectively to obtain image position information of impurities contained in the metal sample in each microscopic image.

[0085] Further, for any microscopic image X in the plurality of microscopic images, a local microscopic image Y of the image containing the impurities contained in the metal sample can be determined from the microscopic image X according to the image position information of the impurities contained in the metal sample in the microscopic image X. Further, the metallographic impurity recognition model can be used to extract the impurity surface phase characteristics of the local microscopic image Y corresponding to the microscopic image X to obtain the impurity surface phase characteristics contained in the microscopic image X.

[0086] Furthermore, in the metallographic impurity identification model, the impurity type of the impurities contained in the metal sample can be identified according to the impurity surface characteristics contained in multiple microscopic images. The same metal sample can contain impurities of one or more impurity types.

[0087] For example, Figure 3 As shown, the multiple magnifications are two magnifications, namely, a first magnification and a second magnification. The second magnification is greater than the first magnification, and the second microscopic image of the second magnification is obtained by collecting a microscopic image of the metal sample at the second magnification after adjusting the observation position based on the position information of the impurities in the metal sample determined by the first microscopic image of the first magnification.

[0088] In the metallographic impurity recognition model, the first microscopic image and the second microscopic image may be respectively subjected to impurity detection to obtain image position information of impurities contained in the metal sample in the first microscopic image and the second microscopic image, respectively.

[0089] Furthermore, based on the image position information of the impurities contained in the metal sample in the first microscopic image and the second microscopic image, respectively, local microscopic images of the impurities contained in the metal sample can be determined from the first microscopic image and the second microscopic image, respectively; and the impurity surface phase characteristics are extracted from the local microscopic image corresponding to the first microscopic image and the local microscopic image corresponding to the second microscopic image, respectively, to obtain the impurity surface phase characteristics contained in the first microscopic image and the impurity surface phase characteristics contained in the second microscopic image.

[0090] Furthermore, in the metallographic impurity identification model, the impurity type of the impurities contained in the metal sample can be identified based on the impurity surface characteristics contained in the first microscopic image and the impurity surface characteristics contained in the second microscopic image. Figure 3 The metallographic impurity identification model can determine that the impurities contained in the metal sample are metallographic inclusions and surface contaminants. Among them, metallographic inclusions correspond to Figure 3 The large circle in the figure corresponds to the surface contaminant Figure 3 The small circle in the middle.

[0091] The metallographic impurity identification model can also output the image position information of the impurities contained in the metal sample in the microscopic image. Accordingly, the computing device 20 can determine the position information of the impurities contained in the metal sample in the metal sample based on the image position information of the impurities contained in the metal sample in the microscopic image. Among them, regarding the specific implementation method of determining the position information of the impurities contained in the metal sample in the metal sample based on the image position information of the impurities contained in the metal sample in the microscopic image, please refer to the relevant content of the above embodiment, which will not be repeated here.

[0092] Specifically, the metallographic impurity recognition model may include: an impurity detection module, a feature extraction module, and an impurity classification module. The impurity detection module may perform impurity detection on multiple microscopic images respectively to obtain image position information of impurities contained in the metal sample in each microscopic image; and, for any microscopic image X in the multiple microscopic images, the local microscopic image Y containing the impurities contained in the metal sample may be determined from the microscopic image X according to the image position information of the impurities contained in the metal sample in the microscopic image X. Furthermore, the feature extraction module may extract impurity surface phase features from the local microscopic image Y corresponding to the microscopic image X using the metallographic impurity recognition model to obtain the impurity surface phase features contained in the microscopic image X.

[0093] Furthermore, the impurity classification module can identify the impurity types of impurities contained in the metal sample based on the surface characteristics of the impurities contained in multiple microscopic images. Among them, the same metal sample may contain impurities of one or more impurity types. In some embodiments, the impurity types of impurities contained in the metal sample include: metallographic inclusions, and of course, surface contaminants. Since the metallographic impurity identification method provided in the embodiment of the present application can distinguish between metallographic inclusions and surface contaminants, when metallographic inclusions are metallographically graded, the interference of surface contaminants can be excluded, and the metallographic grade can be performed only based on the morphological characteristics of the metallographic inclusions contained in the metal sample, which helps to improve the accuracy of the metallographic inclusion rating.

[0094] Based on this, the computing device 20 can also extract the morphological characteristics of the metallographic inclusions in the metal sample according to the position information of the metallographic inclusions in the metal sample. The morphological characteristics of the metallographic inclusions in the metal sample include but are not limited to: one or more of the contour shape, color, size, particle number and particle distribution form of the metallographic inclusions in the metal sample, and multiple refers to 2 or more.

[0095] Furthermore, the metallographic inclusions contained in the metal sample can be metallographically graded according to the morphological characteristics of the metallographic inclusions in the metal sample. Specifically, the morphological characteristics of the metallographic inclusions in the metal sample can be compared with the morphological characteristics of the inclusions corresponding to each metallographic grade in the metallographic grading standard to determine the metallographic grade of the metallographic inclusions contained in the metal sample. The metallographic grading standard can be the above-mentioned metallographic detection standard, such as GB / T10561-2005, etc.

[0096] In addition to the metallographic impurity identification system embodiment provided in the above embodiment, the present application also provides a metallographic impurity identification method. The metallographic impurity identification method can be applied to any electronic device with computing power, such as the above computing device, and of course can also be applied to microscopes with strong computing power.

[0097] Figure 4aThe following is a schematic diagram of the process of metallographic impurity identification method provided in the embodiment of the present application. Figure 4a As shown, the metallographic impurity identification method mainly includes:

[0098] 401. Obtain multiple microscopic images of the metal sample; the multiple microscopic images have multiple magnifications.

[0099] 402. Extract impurity surface phase features from the multiple microscopic images to obtain the impurity surface phase features contained in the multiple microscopic images.

[0100] 403. Identify the impurity type of the impurities contained in the metal sample based on the impurity surface characteristics contained in multiple microscopic images; the impurity type is metallographic inclusions or surface contaminants.

[0101] In the embodiment of the present application, in order to improve the accuracy of metallographic grading, a method for identifying metallographic impurities is proposed, which is mainly used to identify the impurity types of metallographic impurities. Among them, the impurity types of metallographic impurities include: metallographic inclusions and surface contaminants. The metallographic impurity identification in the embodiment of the present application mainly refers to identifying whether the impurities contained in the metal sample are metallographic inclusions or surface contaminants.

[0102] Specifically, the microscope collects microscopic images of the metal sample at different magnifications to obtain multiple microscopic images with multiple magnifications. Among them, multiple refers to 2 or more. Multiple refers to 2 or more. There is at least one microscopic image of one magnification. Further, the microscope can provide multiple microscopic images with multiple magnifications to a computing device. Regarding the implementation form of the microscope, please refer to the relevant content of the above embodiment, which will not be repeated here.

[0103] Accordingly, the device for metallographic impurity identification may obtain a plurality of microscopic images of the metal sample with various magnifications in step 401 .

[0104] In some embodiments, the multiple magnifications may include: a first magnification and a second magnification, wherein the first magnification is smaller than the second magnification.

[0105] The microscope may be controlled to collect a microscopic image of the metal sample at a first magnification to obtain a first microscopic image with a first magnification as a partial microscopic image of the plurality of microscopic images with various magnifications. The first microscopic image may be one or more. Multiple refers to two or more, which is specifically determined by the size of the metallographic sample and the collection field of view of the microscope at the first magnification. The number of the first microscopic images is less than the number of the plurality of microscopic images with various magnifications.

[0106] Furthermore, the first microscopic image may be subjected to impurity detection to determine the position information of impurities contained in the metal sample on the metal sample (defined as target position information). For the representation of the target position information, please refer to the relevant contents of the above embodiment, which will not be repeated here.

[0107] Specifically, the first microscopic image can be subjected to impurity detection to determine position information of impurities contained in the metal sample in the first microscopic image; further, the position information of impurities contained in the metal sample in the collection area corresponding to the first microscopic image can be determined based on the first magnification and the position information of impurities contained in the metal sample in the first microscopic image; thereafter, the position information of impurities contained in the metal sample on the metal sample can be determined based on the position information of impurities contained in the metal sample in the collection area corresponding to the first microscopic image and the position information of the collection area corresponding to the first microscopic image on the metal sample.

[0108] In some embodiments, a pre-trained impurity detection model can be run, and the first microscopic image can be detected for impurities using the impurity detection model to obtain position information of the image of impurities contained in the metal sample in the first microscopic image. The specific implementation form and model training method of the impurity detection model can be found in the relevant content of the above embodiment, and will not be repeated here.

[0109] After determining the target position information of the impurities contained in the metal sample on the metal sample, the microscope can be controlled to adjust the magnification to a second magnification. The second magnification is greater than the first magnification. Further, according to the target position information of the impurities contained in the metal sample on the metal sample, the microscope can be controlled to adjust the observation position to the target observation position. When the microscope collects a microscopic image at the target observation position with the second magnification, the collected field of view covers the area corresponding to the target position information, thereby realizing local amplification of the impurities contained in the metal sample.

[0110] Furthermore, the microscope can be controlled to capture a microscopic image of the metal sample at a second magnification at a target observation position to obtain a second microscopic image with a second magnification as a partial microscopic image of the multiple microscopic images with multiple magnifications.

[0111] The above only takes the multiple magnifications including two magnifications, namely, the first magnification and the second magnification, as an example to illustrate the implementation method of obtaining microscopic images at multiple magnifications, which does not constitute a limitation. Of course, the multiple magnifications can also be implemented as more than two magnifications, such as three magnifications, four magnifications, etc., but is not limited thereto.

[0112] For various magnifications, the microscope can be controlled to follow the order of magnification from small to large, and the position information of impurities in the metal sample obtained by impurity detection based on the microscopic image of the previous magnification and the adjusted next magnification, the observation position of the microscope is adjusted, and at the adjusted observation position, the microscopic image of the metal sample is collected at the next magnification to obtain multiple microscopic images with various magnifications.

[0113] Further, in step 402, the impurity surface phase characteristics of the multiple microscopic images can be extracted to obtain the impurity surface phase characteristics contained in the multiple microscopic images; and in step 403, the impurity type of the impurities contained in the metal sample is identified based on the impurity surface phase characteristics contained in the multiple microscopic images, that is, it is identified whether the impurities contained in the metal sample are metallographic inclusions or surface contaminants.

[0114] In this embodiment, by extracting the impurity surface characteristics of the microscopic images of the metal sample at different magnifications, the impurity surface characteristics of different particle sizes of the same impurity can be obtained, so that the impurity surface characteristics have different fineness. The impurity surface characteristics with different fineness can make the surface characteristics of metallographic inclusions and the surface characteristics of surface contaminants more clearly distinguishable, and it is easier to distinguish whether the impurities are metallographic inclusions or surface contaminants. Therefore, according to the impurity surface characteristics contained in the microscopic images of different magnifications, it can be distinguished whether the impurities contained in the metallographic inclusions are metallographic inclusions or surface contaminants, and the type of impurities contained in the metal sample can be identified. In this way, when metallographic inclusions are graded, the interference of surface contaminants can be eliminated, which helps to improve the accuracy of the metallographic rating of inclusions.

[0115] In the embodiments of the present application, when extracting impurity surface features and classifying impurities in microscopic images, a neural network model can be used to extract impurity surface features and classify impurities. In the embodiments of the present application, the neural network model used for extracting impurity surface features and classifying impurities in microscopic images is defined as a metallographic impurity recognition model. The specific implementation form of the metallographic impurity recognition model can be found in the relevant content of the above embodiments, which will not be repeated here.

[0116] Before using the metallographic impurity recognition model to extract impurity surface features and classify impurities from the microscopic image, the metallographic impurity recognition model needs to be trained. Specifically, multiple microscopic image samples of the metal sample specimen can be obtained. The multiple microscopic image samples have multiple magnifications. Among them, the number of magnifications of the multiple microscopic image samples can be the same as the number of magnifications of the multiple microscopic images actually used online.

[0117] For example, the input of the metallographic impurity recognition model in the training stage is: two microscopic image samples of the same metal sample with different magnifications. When used online, the input of the metallographic impurity recognition model is also: two microscopic images of the same metal sample with different magnifications.

[0118] The metal sample specimen may be one or more. Multiple refers to two or more. The microscopic image samples of the metal sample specimen are pre-labeled with: image position information of impurities contained in the metal sample specimen in each microscopic image sample and impurity type of impurities contained in the metal sample specimen. For the representation of the image position information of impurities contained in the metal sample specimen in each microscopic image sample, please refer to the relevant content of the image position information of impurities contained in the metal sample specimen in the microscopic image, which will not be repeated here.

[0119] Furthermore, with the loss function minimization as the training goal and multiple microscopic image samples as training samples, the initial model corresponding to the metallographic impurity recognition model is trained to obtain the metallographic impurity recognition model. The implementation form of the loss function can be found in the relevant content of the above embodiment, which will not be repeated here.

[0120] Based on the trained metallographic impurity recognition model, the metallographic impurity recognition model can be used to perform impurity detection on multiple microscopic images respectively to obtain image position information of impurities contained in the metal sample in each microscopic image.

[0121] Further, for any microscopic image X in the plurality of microscopic images, a local microscopic image Y containing impurities contained in the metal sample can be determined from the microscopic image X according to the image position information of the impurities contained in the metal sample in the microscopic image X. Further, the metallographic impurity recognition model can be used to extract the impurity surface phase characteristics of the local microscopic image Y corresponding to the microscopic image X to obtain the impurity surface phase characteristics contained in the microscopic image X.

[0122] Furthermore, in the metallographic impurity identification model, the impurity type of the impurities contained in the metal sample can be identified according to the impurity surface characteristics contained in multiple microscopic images. The same metal sample can contain impurities of one or more impurity types.

[0123] The metallographic impurity identification model can also output the image position information of the impurities contained in the metal sample in the microscopic image. Accordingly, the position information of the impurities contained in the metal sample in the metal sample can be determined based on the image position information of the impurities contained in the metal sample in the microscopic image. Among them, regarding the specific implementation method of determining the position information of the impurities contained in the metal sample in the metal sample based on the image position information of the impurities contained in the metal sample in the microscopic image, please refer to the relevant content of the above embodiment, which will not be repeated here.

[0124] In some embodiments, the impurity types of the impurities contained in the metal sample include: metallographic inclusions, and of course, surface contaminants. Since the metallographic impurity identification method provided in the embodiment of the present application can distinguish between metallographic inclusions and surface contaminants, when metallographic inclusions are graded, the interference of surface contaminants can be excluded, and the metallographic grade is only performed based on the morphological characteristics of the metallographic inclusions contained in the metal sample, which helps to improve the accuracy of the metallographic inclusion grade.

[0125] Based on this, the morphological characteristics of the metallographic inclusions in the metal sample can also be extracted according to the position information of the metallographic inclusions in the metal sample. The morphological characteristics of the metallographic inclusions in the metal sample include but are not limited to: one or more of the contour shape, color, size, particle number and particle distribution form of the metallographic inclusions in the metal sample, and multiple refers to 2 or more.

[0126] Furthermore, the metallographic inclusions contained in the metal sample can be metallographically graded according to the morphological characteristics of the metallographic inclusions in the metal sample. Specifically, the morphological characteristics of the metallographic inclusions in the metal sample can be compared with the morphological characteristics of the inclusions corresponding to each metallographic grade in the metallographic grading standard to determine the metallographic grade of the metallographic inclusions contained in the metal sample. The metallographic grading standard can be the above-mentioned metallographic detection standard, such as GB / T10561-2005, etc.

[0127] The metallographic impurity identification method provided in the embodiment of the present application can also be extended to the impurity identification in other application scenarios, for example, the impurity type identification of glass, plastic or other organic polymers, as exemplified below.

[0128] Figure 4b The following is a flow chart of the identification method provided in the embodiment of the present application. Figure 4b As shown, the impurity identification method mainly includes:

[0129] 41. Acquire multiple microscopic images of the target object; the multiple microscopic images have multiple magnifications.

[0130] 42. Extract impurity surface phase features from multiple microscopic images to obtain the impurity surface phase features contained in the multiple microscopic images.

[0131] 43. Identify the impurity type of the impurities contained in the target object based on the impurity surface characteristics contained in multiple microscopic images.

[0132] In this embodiment, the target object refers to any material that needs to distinguish the impurity type, including but not limited to the above-mentioned metal sample, or glass, plastic or other organic polymers.

[0133] In this embodiment, the impurity types of impurities contained in the target object may include: inclusions and surface contaminants. Inclusions refer to impurities mixed into the target object during the manufacturing process of the target object; surface contaminants refer to impurities attached to the surface of the target object. Inclusions affect the quality of the target object, while surface contaminants have no effect on the quality of the target object. However, when rating the target object, surface contaminants cause certain interference to the rating and affect the rating results.

[0134] In an embodiment of the present application, in order to improve the accuracy of the rating of the target object, the microscope collects microscopic images of the target object at different magnifications to obtain multiple microscopic images with multiple magnifications. Among them, multiple refers to 2 or more. Multiple refers to 2 or more. There is at least one microscopic image of one magnification. Further, the microscope can provide multiple microscopic images with multiple magnifications to a computing device. Regarding the implementation form of the microscope, please refer to the relevant content of the above embodiment, which will not be repeated here.

[0135] Accordingly, the device for impurity identification can obtain multiple microscopic images of the target object with multiple magnifications in step 41. For the specific implementation of how the microscope acquires multiple microscopic images of the target object with multiple magnifications, please refer to the relevant content of acquiring multiple microscopic images of metal samples with multiple magnifications, which will not be repeated here.

[0136] Further, in step 42, the impurity surface phase characteristics of the multiple microscopic images can be extracted to obtain the impurity surface phase characteristics contained in the multiple microscopic images; and in step 43, the impurity type of the impurities contained in the target object is identified based on the impurity surface phase characteristics contained in the multiple microscopic images, that is, it is identified whether the impurities contained in the target object are inclusions or surface contaminants.

[0137] In this embodiment, by extracting the impurity surface characteristics of the target object at different magnifications of the microscopic images, the impurity surface characteristics of the same impurity with different particle sizes can be obtained, so that the impurity surface characteristics have different fineness. The impurity surface characteristics with different fineness can make the distinction between the surface characteristics of inclusions and the surface characteristics of surface contaminants more obvious, and it is easier to distinguish whether the impurity is an inclusion or a surface contaminant. Therefore, according to the impurity surface characteristics contained in the microscopic images of different magnifications, it can be distinguished whether the impurities contained in the target object are inclusions or surface contaminants, and the type of impurities contained in the target object can be identified. In this way, when the target object is graded for quality, the interference of surface contaminants can be eliminated, which helps to improve the accuracy of the rating results of the target object.

[0138] In the embodiment of the present application, when extracting the impurity surface characteristics and classifying the impurities in the microscopic image, a neural network model can be used to extract the impurity surface characteristics and classify the impurities. In the embodiment of the present application, the neural network model used for extracting the impurity surface characteristics and classifying the impurities in the microscopic image is defined as an impurity recognition model. The specific implementation form of the impurity recognition model can be found in the above-mentioned metallographic impurity recognition module, which will not be repeated here.

[0139] Before using the impurity recognition model to extract impurity surface features and classify impurities from microscopic images, the impurity recognition model needs to be trained. For the training process of the impurity recognition model, please refer to the above-mentioned content on the model training of the metallographic impurity recognition model, which will not be repeated here.

[0140] Based on the trained impurity recognition model, the impurity recognition model can be used to perform impurity detection on multiple microscopic images respectively to obtain image position information of impurities contained in the target object in each microscopic image.

[0141] Further, for any microscopic image X among the multiple microscopic images, a local microscopic image Y containing impurities contained in the target object can be determined from the microscopic image X according to the image position information of the impurities contained in the target object in the microscopic image X. Further, the impurity surface phase feature extraction can be performed on the local microscopic image Y corresponding to the microscopic image X using the impurity recognition model to obtain the impurity surface phase feature contained in the microscopic image X.

[0142] Furthermore, in the metallographic impurity identification model, the impurity type of the impurities contained in the target object can be identified based on the impurity surface characteristics contained in multiple microscopic images. The same target object can contain impurities of one or more impurity types.

[0143] The impurity identification model can also output the image position information of the impurities contained in the target object in the microscopic image. Accordingly, the position information of the impurities contained in the metal sample in the metal sample can be determined based on the image position information of the impurities contained in the target object in the microscopic image. Among them, regarding the specific implementation method of determining the position information of the impurities contained in the metal sample in the metal sample based on the image position information of the impurities contained in the metal sample in the microscopic image, please refer to the relevant content of the above embodiment, which will not be repeated here.

[0144] In some embodiments, the impurity types of impurities contained in the target object include: inclusions, and of course, surface contaminants. Since the impurity identification method provided in the embodiment of the present application can distinguish between inclusions and surface contaminants, when rating inclusions, the interference of surface contaminants can be excluded, and only the morphological characteristics of the inclusions contained in the target object are rated, which helps to improve the accuracy of inclusion rating.

[0145] Based on this, the morphological characteristics of the inclusions in the target object can also be extracted according to the location information of the inclusions in the target object. The morphological characteristics of the inclusions in the metal sample include but are not limited to: one or more of the contour shape, color, size, particle number and particle distribution form of the inclusions in the target object, and multiple refers to 2 or more.

[0146] Furthermore, the inclusions contained in the target object may be rated according to the morphological characteristics of the inclusions in the target object. Specifically, the morphological characteristics of the inclusions in the target object may be compared with the morphological characteristics of the inclusions corresponding to each level in the rating standard of the target object to determine the level of the inclusions contained in the target object.

[0147] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 401 and 402 can be device A; for another example, the execution subject of step 401 can be device A, and the execution subject of step 402 can be device B; and so on.

[0148] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations appearing in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel, and the sequence numbers of the operations, such as 401, 402, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel.

[0149] Accordingly, an embodiment of the present application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the above-mentioned metallographic impurity identification method and / or the steps in the impurity identification method.

[0150] Figure 5 This is a schematic diagram of the structure of a computing device provided in an embodiment of the present application. Figure 5 As shown, the computing device mainly includes: a memory 50a and a processor 50b; wherein the memory 50a is used to store computer programs.

[0151] The processor 50b is coupled to the memory 50a, and is used to execute a computer program for: acquiring multiple microscopic images of a metal sample; the multiple microscopic images have multiple magnifications; extracting impurity surface phase characteristics from the multiple microscopic images to obtain the impurity surface phase characteristics contained in the multiple microscopic images; identifying the impurity type of impurities contained in the metal sample based on the impurity surface phase characteristics contained in the multiple microscopic images; the impurity type is a metallographic inclusion or a surface contaminant.

[0152] In some embodiments, the multiple magnifications include: a first magnification and a second magnification. Accordingly, when the processor 50b acquires multiple microscopic images of the metal sample, it is specifically used to: control the microscope to collect microscopic images of the metal sample at a first magnification to obtain a first microscopic image with a first magnification as a partial microscopic image in the multiple microscopic images; perform impurity detection on the first microscopic image to determine the target position information of impurities contained in the metal sample on the metal sample; control the microscope to adjust the magnification to a second magnification; wherein the second magnification is greater than the first magnification; and, according to the target position information, control the microscope to adjust the observation position to the target observation position; when the microscope collects microscopic images at the target observation position at the second magnification, the collection field of view covers the area corresponding to the target position information; and control the microscope to collect microscopic images of the metal sample at the second magnification at the target observation position to obtain a second microscopic image with a second magnification as a partial microscopic image in the multiple microscopic images.

[0153] In some embodiments, when the processor 50b extracts impurity features from multiple microscopic images, it is specifically used to: use a pre-trained metallographic impurity recognition model to perform impurity detection on the multiple microscopic images to obtain image position information of impurities contained in the metal sample in the multiple microscopic images; for any microscopic image among the multiple microscopic images, determine a local microscopic image containing impurities contained in the metal sample from any microscopic image based on the image position information of impurities contained in the metal sample in any microscopic image; use the metallographic impurity recognition model to extract impurity surface phase features from the local microscopic image corresponding to any microscopic image to obtain the impurity surface phase features contained in any microscopic image.

[0154] Correspondingly, when the processor 50b identifies the impurity type of impurities contained in a metal sample based on the impurity surface phase characteristics contained in multiple microscopic images, it is specifically used to: in a metallographic impurity identification model, identify the impurity type of impurities contained in a metal sample based on the impurity surface phase characteristics contained in multiple microscopic images.

[0155] Furthermore, the processor 50b is also used for: obtaining multiple microscopic image samples of a metal sample specimen before respectively extracting impurity surface features from multiple microscopic images using a pre-trained metallographic impurity recognition model; the multiple microscopic image samples have multiple magnification factors; the multiple microscopic image samples are pre-labeled with: image position information of impurities contained in the metal sample specimen in each microscopic image sample and impurity types of impurities contained in the metal sample specimen; training an initial model corresponding to the metallographic impurity recognition model with minimization of the loss function as a training goal and multiple microscopic image samples as training samples to obtain a metallographic impurity recognition model; wherein the loss function is at least determined based on the difference between the predicted impurity type of impurities contained in the metal sample specimen output by the model and the impurity type of impurities contained in the metal sample specimen labeled in advance.

[0156] Alternatively, the loss function is jointly determined based on the difference between the predicted impurity type of impurities contained in the metal sample sample output by the model and the impurity type of impurities contained in the pre-labeled metal sample sample, and the difference between the image predicted position of impurities contained in the metal sample sample output by the model in each microscopic image sample and the image position information of impurities contained in the pre-labeled metal sample sample in each microscopic image sample.

[0157] In some embodiments of the present application, the impurity types of the impurities contained in the metal sample include: metallographic inclusions. The processor 50b is also used to: determine the position information of the metallographic inclusions in the metal sample according to the image position information of the metallographic inclusions in each microscopic image; identify the metallographic inclusions contained in the metal sample from the impurities contained in the metal sample according to the position information of the metallographic inclusions in the metal sample, and extract the morphological characteristics of the metallographic inclusions in the metal sample; perform metallographic rating on the metallographic inclusions contained in the metal sample according to the morphological characteristics of the metallographic inclusions in the metal sample.

[0158] Optionally, when the processor 50b performs metallographic grading on the metallographic inclusions contained in the metal sample according to the morphological characteristics of the metallographic inclusions in the metal sample, the processor 50b is specifically used to: compare the morphological characteristics of the metallographic inclusions in the metal sample with the morphological characteristics of the inclusions corresponding to each metallographic grade in the metallographic grading standard to determine the metallographic grade of the metallographic inclusions contained in the metal sample.

[0159] In some embodiments of the present application, the processor 50b is also used to: obtain multiple microscopic images of the target object; the multiple microscopic images have multiple magnifications; extract impurity surface phase features from the multiple microscopic images to obtain the impurity surface phase features contained in the multiple microscopic images; and identify the impurity type of impurities contained in the target object based on the impurity surface phase features contained in the multiple microscopic images.

[0160] In some optional embodiments, such as Figure 5 As shown, the computing device may also include: a communication component 50c and a power component 50d and other components. In some embodiments, the computing device may be implemented as a terminal device such as a computer, a workstation, a mobile phone, etc., and may also include: a display component 50e and an audio component 50f and other components. Figure 5 The components are shown schematically only and do not necessarily include Figure 5 The components shown do not necessarily mean that the computing device can only include Figure 5 Components shown.

[0161] The computing device provided in this embodiment can extract the impurity surface phase characteristics of the same impurity with different particle sizes by performing extraction on the microscopic images of the metal sample at different magnifications, so that the impurity surface phase characteristics can be presented with different fineness. The impurity surface phase characteristics with different fineness can make the distinction between the surface phase characteristics of metallographic inclusions and the surface phase characteristics of surface contaminants more obvious, and it is easier to distinguish whether the impurities are metallographic inclusions or surface contaminants. Therefore, according to the impurity surface phase characteristics contained in the microscopic images of different magnifications, it can be distinguished whether the impurities contained in the metallographic inclusions are metallographic inclusions or surface contaminants, and the type of impurities contained in the metal sample can be identified. In this way, when metallographic inclusions are metallographically graded, the interference of surface contaminants can be eliminated, which helps to improve the accuracy of the metallographic rating of inclusions.

[0162] In an embodiment of the present application, the memory is used to store a computer program and can be configured to store various other data to support operations on the device where it is located. Among them, the processor can execute the computer program stored in the memory to implement the corresponding control logic. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random-Access Memory, SRAM), electrically erasable programmable read only memory (Electrically Erasable Programmable Read Only Memory, EEPROM), erasable programmable read only memory (Electrical Programmable Read Only Memory, EPROM), programmable read only memory (Programmable Read Only Memory, PROM), read only memory (Read Only Memory, ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0163] In the embodiment of the present application, the processor can be any hardware processing device that can execute the logic of the above method. Optionally, the processor can be a central processing unit (CPU), a graphics processing unit (GPU) or a microcontroller unit (MCU); it can also be a field programmable gate array (FPGA), a programmable array logic device (PAL), a general array logic device (GAL), a complex programmable logic device (CPLD) and other programmable devices; or an application specific integrated circuit (ASIC) chip; or an advanced reduced instruction set (RISC) processor (Advanced RISC Machines, ARM) or a system on chip (SoC), etc., but not limited to this.

[0164] In an embodiment of the present application, the communication component is configured to facilitate wired or wireless communication between the device in which it is located and other devices. The device in which the communication component is located can access a wireless network based on a communication standard, such as Wireless Fidelity (WiFi), 2G or 3G, 4G, 5G or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component can also be based on Near Field Communication (NFC) technology, Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology or other technologies.

[0165] In an embodiment of the present application, the display component may include a liquid crystal display (LCD) and a touch panel (TP). If the display component includes a touch panel, the display component may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0166] In an embodiment of the present application, a power supply component is configured to provide power to various components of the device in which it is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component is located.

[0167] In an embodiment of the present application, the audio component may be configured to output and / or input audio signals. For example, the audio component includes a microphone (Microphone, MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal may be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal. For example, for a device with a language interaction function, voice interaction with a user can be achieved through an audio component.

[0168] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0169] It should also be noted that the descriptions such as "first" and "second" in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit "first" and "second" to different types.

[0170] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage, etc.) that contain computer-usable program code.

[0171] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (or systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0172] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0173] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0174] In a typical configuration, a computing device includes one or more processors (such as a CPU, etc.), an input / output interface, a network interface, and a memory.

[0175] Memory may include non-permanent storage in a computer-readable medium, random-access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0176] The storage medium of a computer is a readable storage medium, which may also be referred to as a readable medium. The readable storage medium includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be a computer-readable instruction, a data structure, a module of a program, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0177] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the above elements.

[0178] The above contents are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A metallographic impurity identification method, characterized in that: include: Acquire multiple microscopic images of metal specimens; The plurality of microscopic images have various magnifications; Extracting impurity surface phase features from the plurality of microscopic images to obtain impurity surface phase features contained in the plurality of microscopic images; According to the impurity surface phase characteristics contained in the multiple microscopic images, the impurity type of the impurities contained in the metal sample is identified; the impurity type is metallographic inclusions or surface contaminants.

2. The method according to claim 1, characterized in that The multiple magnifications include: a first magnification and a second magnification; the step of obtaining multiple microscopic images of the metal sample includes: Controlling the microscope to collect a microscopic image of the metal sample at a first magnification to obtain a first microscopic image with the first magnification as a partial microscopic image of the plurality of microscopic images; Performing impurity detection on the first microscopic image to determine target position information of impurities contained in the metal sample on the metal sample; Controlling the microscope to adjust the magnification to the second magnification; wherein the second magnification is greater than the first magnification; According to the target position information, the microscope is controlled to adjust the observation position to the target observation position; when the microscope acquires a microscopic image at the target observation position with a second magnification, the acquisition field of view covers the area corresponding to the target position information; The microscope is controlled to be at the target observation position to collect a microscopic image of the metal sample at the second magnification to obtain a second microscopic image with the second magnification as a partial microscopic image of the multiple microscopic images.

3. The method according to claim 1, characterized in that The extracting impurity features from the plurality of microscopic images to obtain the impurity surface features contained in the plurality of microscopic images includes: Using a pre-trained metallographic impurity recognition model to perform impurity detection on the plurality of microscopic images to obtain image position information of impurities contained in the metal sample in the plurality of microscopic images; For any one of the plurality of microscopic images, according to image position information of the impurities contained in the metal sample in the any one microscopic image, determine a local microscopic image containing the impurities contained in the metal sample from the any one microscopic image; Using the metallographic impurity recognition model, extract impurity surface phase features from the local microscopic image corresponding to any microscopic image to obtain the impurity surface phase features contained in any microscopic image; The step of identifying the impurity type of the impurities contained in the metal sample according to the impurity surface phase characteristics contained in the plurality of microscopic images comprises: In the metallographic impurity identification model, the impurity type of the impurities contained in the metal sample is identified based on the impurity surface phase characteristics contained in the multiple microscopic images.

4. The method according to claim 3, characterized in that Before respectively extracting impurity surface features from the plurality of microscopic images using the pre-trained metallographic impurity recognition model, the method further comprises: Acquire multiple microscopic image samples of the metal sample; the multiple microscopic image samples have multiple magnifications; the multiple microscopic image samples are pre-labeled with: image position information of impurities contained in the metal sample in each microscopic image sample and impurity types of impurities contained in the metal sample; Taking minimization of the loss function as a training objective and the plurality of microscopic image samples as training samples, model training is performed on an initial model corresponding to the metallographic impurity recognition model to obtain the metallographic impurity recognition model; The loss function is determined at least based on the difference between the predicted impurity type of the impurities contained in the metal sample output by the model and the impurity type of the impurities contained in the metal sample marked in advance.

5. The method according to claim 4, characterized in that The loss function is jointly determined based on the difference between the predicted impurity type of the impurities contained in the metal sample sample output by the model and the impurity type of the impurities contained in the metal sample sample marked in advance, and the difference between the image predicted position of the impurities contained in the metal sample sample output by the model in each microscopic image sample and the image position information of the impurities contained in the metal sample sample marked in advance in each microscopic image sample.

6. The method according to any one of claims 3 to 5, characterized in that: The impurities contained in the metal sample include: metallographic inclusions; the method further includes: Determining the position information of the metallographic inclusions in the metal sample according to the image position information of the metallographic inclusions in each microscopic image; Extracting morphological features of the metallographic inclusions in the metal sample according to position information of the metallographic inclusions in the metal sample; The metallographic inclusions contained in the metal sample are metallographically graded according to the morphological characteristics of the metallographic inclusions in the metal sample.

7. An impurity identification method, characterized in that: include: Acquire multiple microscopic images of the target object; The plurality of microscopic images have various magnifications; Extracting impurity surface phase features from the plurality of microscopic images to obtain impurity surface phase features contained in the plurality of microscopic images; According to the impurity surface characteristics contained in the multiple microscopic images, the impurity type of the impurities contained in the target object is identified.

8. A metallographic impurity identification system, characterized in that: include: A microscope and a computing device; the microscope is used to collect microscopic images of metal samples at different magnifications to obtain multiple microscopic images with multiple magnifications; and providing the plurality of microscopic images to the computing device; The computing device is used to execute the steps in the method according to any one of claims 1 to 6.

9. A computing device, characterized in that include: A memory and a processor; wherein the memory is used to store a computer program; The processor is coupled to the memory and configured to execute the computer program to perform the steps in the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the method according to any one of claims 1 to 7.

Images