A carbide metallographic image acquisition method and device and electronic equipment
By acquiring multiple field-of-view images using a metallographic imaging microscope and employing a neural network model for detection and stitching, the problem of inaccurate detection caused by the limitation of the microscope's field of view was solved, and high-accuracy acquisition of carbide metallographic images was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 河钢数字技术股份有限公司
- Filing Date
- 2023-09-27
- Publication Date
- 2026-06-16
Smart Images
Figure CN117237942B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of carbide metallographic structure technology, and in particular to a method, apparatus and electronic device for acquiring carbide metallographic images. Background Technology
[0002] In the steel industry, carbide inhomogeneity testing is an important indicator for evaluating metallurgical quality. Currently, the main testing method in the field of carbide testing is metallographic analysis. Metallographic analysis requires observing the steel sample under a microscope to obtain the test results.
[0003] The field of view of a microscope represents the maximum area that the microscope can observe, with the national standard being 710 micrometers × 710 micrometers. However, for ribbon-like carbides, the metallographic structure may extend beyond the microscope's field of view. Performing metallographic analysis using a single image from the microscope's field of view may result in the metallographic structure being segmented, leading to inaccurate metallographic analysis. Summary of the Invention
[0004] This invention provides a method, apparatus, and electronic device for acquiring carbide metallographic images, which can improve the accuracy of carbide metallographic image detection.
[0005] In a first aspect, the present invention provides a method for acquiring metallographic images of carbides, applied to a metallographic image microscope. The method includes: acquiring multiple field-of-view images of a target region of a carbide captured by the metallographic image microscope; detecting the multiple field-of-view images based on a pre-set metallographic detection model to obtain a primary detection result for the multiple field-of-view images; the metallographic detection model is obtained by training a neural network on the metallographic images of the carbide; stitching the multiple field-of-view images together to obtain multiple stitched images; detecting the multiple stitched images based on the metallographic detection model to obtain a secondary detection result for the multiple field-of-view images; and determining the metallographic detection result based on the primary and secondary detection results, thereby realizing the acquisition of the metallographic image of the carbide.
[0006] In one possible implementation, multiple field-of-view images are detected based on a pre-set metallographic detection model to obtain a single detection result for the multiple field-of-view images. This includes: detecting multiple field-of-view images based on the pre-set metallographic detection model to obtain multiple detection boxes; determining the orientation of the multiple detection boxes based on their length and width; filtering the multiple detection boxes based on the prior orientation and their orientations to obtain filtered detection boxes; and determining a single detection result based on the filtered detection boxes.
[0007] In one possible implementation, before stitching together multiple field-of-view images to obtain multiple stitched images, the method further includes: for the filtered detection boxes, if the filtered detection boxes in a certain field-of-view image are located at the edge of the field-of-view image, then it is determined that the field-of-view image has a cross-field-of-view problem; the cross-field-of-view problem means that the metallographic structure corresponding to the filtered detection boxes in the field-of-view image spans the field-of-view image and the field-of-view image adjacent to the field-of-view image.
[0008] In one possible implementation, multiple field-of-view images are stitched together to obtain multiple stitched images, including: identifying field-of-view images with cross-field-of-view problems among the multiple field-of-view images; constructing a group of images to be stitched based on the field-of-view images with cross-field-of-view problems and the field-of-view images adjacent to the field-of-view images with cross-field-of-view problems; determining feature points in each field-of-view image in the group of images to be stitched based on the ORB algorithm; calculating feature descriptors based on the feature points in each field-of-view image in the group of images to be stitched; matching the field-of-view images in the group of images to be stitched based on the feature points and feature descriptors to determine the matching similarity; determining multiple pairs of field-of-view images in the group of images to be stitched based on the matching similarity; and stitching each pair of field-of-view images together to obtain multiple stitched images.
[0009] In one possible implementation, each pair of field-of-view images is stitched together to obtain multiple stitched images, including: for each pair of field-of-view images, determining key point pairs based on the similarity of each feature point in each pair of field-of-view images; determining the horizontal and vertical offsets of the two field-of-view images in each pair of field-of-view images based on the key point pairs; calculating the mode of the horizontal and vertical offsets; and stitching the two field-of-view images in each pair of field-of-view images based on the mode of the horizontal and vertical offsets.
[0010] In one possible implementation, before detecting multiple field-of-view images based on a pre-set metallographic detection model and obtaining a single detection result for multiple field-of-view images, the method further includes: acquiring multiple carbide images labeled with metallographic structures; generating training samples based on the multiple carbide images labeled with metallographic structures; and training a new neural network model based on the training samples to obtain a metallographic detection model.
[0011] In one possible implementation, multiple stitched images are detected based on a metallographic detection model to obtain secondary detection results for multiple field-of-view images. This includes: detecting multiple stitched images based on the metallographic detection model to obtain multiple detection boxes; determining the orientation of multiple detection boxes based on their length and width; filtering the multiple detection boxes based on their prior orientation and orientation to obtain filtered detection boxes; and determining the secondary detection results based on the filtered detection boxes.
[0012] In one possible implementation, the first detection result includes a field-of-view image containing the metallographic structure and the size of the metallographic structure; the second detection result includes a stitched image containing the metallographic structure and the size of the metallographic structure; the metallographic detection result includes the size of the largest metallographic structure and the size distribution of the metallographic structure; based on the first and second detection results, the metallographic detection result is determined to achieve the acquisition of metallographic images of carbides, including: sorting the metallographic structures according to their size based on the first and second detection results; and determining the metallographic detection result for the sorted metallographic structures.
[0013] Secondly, embodiments of the present invention provide a metallographic image acquisition device for carbides, characterized in that it is applied to a metallographic image microscope, the metallographic image microscope including a three-dimensional motion control console and a stage; the motion control device includes: a communication module for acquiring multiple field-of-view images captured by the metallographic image microscope on a target area of a carbide; a processing module for detecting the multiple field-of-view images based on a pre-set metallographic detection model to obtain a primary detection result of the multiple field-of-view images; the metallographic detection model is obtained by training a neural network on the metallographic images of the carbide; stitching the multiple field-of-view images to obtain multiple stitched images; detecting the multiple stitched images based on the metallographic detection model to obtain a secondary detection result of the multiple field-of-view images; and determining the metallographic detection result based on the primary detection result and the secondary detection result, thereby realizing the acquisition of metallographic images of carbides.
[0014] Thirdly, embodiments of the present invention provide an electronic device including a memory and a processor. The memory stores a computer program, and the processor is configured to call and run the computer program stored in the memory to perform the steps of the method as described in the first aspect and any possible implementation thereof.
[0015] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements the steps of the method as described in the first aspect and any possible implementation thereof.
[0016] This invention provides a method, apparatus, and electronic device for acquiring metallographic images of carbides. The invention involves capturing multiple field-of-view images of a target area of a carbide using a metallographic imaging microscope. These images are then inspected using a metallographic detection model to determine a primary detection result. The multiple field-of-view images are then stitched together, and the resulting stitched image is inspected using the metallographic detection model to obtain a secondary detection result. Because the stitched image can completely display the metallographic structure, avoiding fragmentation of the metallographic structure, the metallographic detection result determined by combining the primary and secondary detection results can accurately reflect the metallographic structure of the carbide, thus improving the accuracy of carbide metallographic image detection. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a schematic flowchart of a method for acquiring carbide metallographic images provided in an embodiment of the present invention;
[0019] Figure 2 This is a schematic diagram of the structure of a carbide metallographic image acquisition device provided in an embodiment of the present invention;
[0020] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0021] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.
[0022] In the description of this invention, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" and "more than one" refer to two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.
[0023] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner to facilitate understanding.
[0024] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not limited to the steps or modules listed, but may optionally include other steps or modules not listed, or may optionally include other steps or modules inherent to such process, method, product, or device.
[0025] To make the objectives, technical solutions, and advantages of the present invention clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.
[0026] like Figure 1 As shown, this embodiment of the invention provides a method for acquiring carbide metallographic images. Applied to a metallographic image microscope, the acquisition method includes steps S101-S104.
[0027] S101. Obtain multiple field-of-view images of the target area of the carbide taken by a metallographic imaging microscope.
[0028] It should be noted that the field of view of a metallographic imaging microscope represents the maximum area that the microscope can observe. For banded carbides, the detection field of view is 710 μm × 710 μm under the national standard. For samples in actual production scenarios, we usually acquire images of the entire steel sample and select the field of view with the highest grade of banded carbides. For manual grading, the microscope's field of view can be flexibly adjusted to find the field of view with the highest grade of banded carbides. However, for automated metallographic acquisition devices, because the image acquisition position is fixed and movable, and the entire sample is acquired from all directions, there may be a problem where the maximum field of view is split into two images in the acquired images. However, the algorithm of the metallographic inspection model is performed on a single image, and the problem of the maximum grade metallographic structure being split needs to be considered.
[0029] S102. Based on the pre-set metallographic detection model, multiple field-of-view images are detected to obtain a single detection result for multiple field-of-view images.
[0030] In this embodiment of the application, the metallographic detection model is obtained by training a neural network on the metallographic image of the carbide.
[0031] In some embodiments, a single detection result includes a field image of the metallographic structure and the size of the metallographic structure.
[0032] As one possible implementation, step S102 can be specifically implemented as S1021-S1024.
[0033] S1021. Based on the pre-set metallographic detection model, multiple field-of-view images are detected to obtain multiple detection boxes.
[0034] S1022. Determine the orientation of multiple detection boxes based on their length and width.
[0035] S1023. Based on the prior direction and the direction of multiple detection boxes, filter the multiple detection boxes to obtain the filtered detection boxes.
[0036] It should be noted that for the samples to be collected, the image of each sample will be stored in a folder, and the user can initially input the placement orientation of the sample as the prior orientation.
[0037] S1024. Based on the filtered detection boxes, determine a detection result.
[0038] S103. Stitch together multiple field-of-view images to obtain multiple stitched images.
[0039] As a possible implementation, before step S103, the acquisition method provided in this embodiment of the invention further includes: for the filtered detection box, if the filtered detection box in a certain field of view image is located at the edge of the field of view image, then it is determined that the field of view image has a cross-field of view problem; the cross-field of view problem means that the metallographic structure corresponding to the filtered detection box in the field of view image spans the field of view image and the field of view image adjacent to the field of view image.
[0040] As one possible implementation, step S103 can be specifically implemented as S1031-S1037.
[0041] S1031. Identify the field-of-view image that has a cross-field-of-view problem among multiple field-of-view images.
[0042] S1032. Based on the field-of-view image with cross-field-of-view problem and the field-of-view image adjacent to the field-of-view image with cross-field-of-view problem, construct a group of images to be stitched together.
[0043] S1033. Based on the ORB algorithm, determine the feature points in each field of view image in the image group to be stitched.
[0044] The ORB (oriented fAST and rotated BRIEF) algorithm is a fast and robust feature extraction algorithm, an improvement on the FAST (Features from Accelerated Segment Test) algorithm. Compared with operators such as SIFT and SURF, the ORB operator has significantly improved speed.
[0045] S1034. Calculate feature descriptors based on feature points in each field of view image in the image group to be stitched.
[0046] The feature descriptor is determined by a set of locally invariant feature points in the field of view image and is used to represent the points of metallographic structure in the field of view image.
[0047] S1035. Based on feature points and feature descriptors, match the field images in the image group to be stitched together and determine the matching similarity.
[0048] This invention employs a distance-based brute-force matching method, which compares the feature descriptors of two images one by one and stores the similarity between the descriptors. Based on the matching similarity, keypoint pairs matching between the two images are identified.
[0049] To stitch the images together, we need to align the two images. We use the mode to determine the alignment. Specifically, for each matching keypoint pair, we calculate its horizontal and vertical offsets, and then take the mode of the calculated results as the alignment.
[0050] S1036. Based on matching similarity, determine multiple pairs of field-of-view images in the image group to be stitched together.
[0051] S1037. Stitch together each pair of field images to obtain multiple stitched images.
[0052] For example, step S1037 can also be implemented as A1-A7.
[0053] A1. For each pair of field-of-view images, determine the key point pairs based on the similarity of each feature point in each pair of field-of-view images.
[0054] Here, a keypoint pair is a set of multiple pairs of feature points with a similarity greater than a set threshold, or a set of the top N pairs of feature points sorted by similarity from highest to lowest. Here, N is a positive integer.
[0055] A2. Based on key point pairs, determine the horizontal and vertical offsets of the two field images in each pair of field images.
[0056] For example, for each keypoint pair, the horizontal and vertical offsets can be determined based on the x and y coordinates of the two feature points in the keypoint pair. The horizontal offset is the difference between the x-coordinates, and the vertical offset is the difference between the y-coordinates.
[0057] A3. Calculate the mode of the horizontal offset and the mode of the vertical offset.
[0058] A4. Based on the mode of the horizontal offset and the mode of the vertical offset, stitch together the two field images in each pair of field images.
[0059] For example, in embodiments of the present invention, the mode of the horizontal offset can be determined as the horizontal offset of the two field-of-view images, and the mode of the vertical offset can be determined as the vertical offset of the two field-of-view images. Thus, embodiments of the present invention can move the two field-of-view images based on their horizontal and vertical offsets to achieve field-of-view image stitching.
[0060] It should be noted that the specific splicing process is as follows.
[0061]
[0062] in, For feature point set 1, For descriptor set 1, For feature point set 2, For descriptor set 2, This is the horizontal offset. This is the vertical offset.
[0063] S104. Based on the metallographic detection model, multiple stitched images are detected to obtain secondary detection results for multiple field-of-view images.
[0064] In some embodiments, the secondary detection results include a stitched image containing a metallographic structure, and the size of the metallographic structure.
[0065] As one possible implementation, step S104 can be specifically implemented as S1041-S1044.
[0066] S1041. Based on the metallographic detection model, multiple stitched images are detected to obtain multiple detection boxes.
[0067] S1042. Determine the orientation of multiple detection boxes based on their length and width.
[0068] S1043. Based on the prior direction and the direction of multiple detection boxes, multiple detection boxes are filtered to obtain the filtered detection boxes.
[0069] It should be noted that the orientation of a sample determines the orientation of the carbides, that is, the orientation of the metallographic structure within the carbides. Therefore, in the test results of the same sample, the orientation of the carbides can be determined a priori, and test results that do not conform to the correct orientation can be filtered out accordingly.
[0070] Specifically, for each detection box output by the model, the algorithm first determines the height and width of the detection box. If the height is greater than the width, it is a vertical detection box; if the width is greater than the height, it is a horizontal detection box. The direction of the detection box is compared with the prior direction. If the two directions are the same, the detection box is retained; otherwise, the detection box is deleted from the result set.
[0071] For example, if the direction of a detection box is consistent with the prior direction, the detection box is retained; if the direction of a detection box is inconsistent with the prior direction, the detection box is deleted.
[0072] The orientation detection and filtering algorithm is shown below.
[0073]
[0074] Where c is the category, w is the detection box width, h is the detection box length, x is the horizontal axis, y is the vertical axis, and s is the confidence score.
[0075] S1044. Based on the filtered detection boxes, determine the secondary detection results.
[0076] S105. Based on the results of the first and second tests, determine the metallographic test results and acquire metallographic images of carbides.
[0077] In some embodiments, the metallographic test results include the size of the largest metallographic structure and the size distribution of the metallographic structure.
[0078] As one possible implementation, embodiments of the present invention can sort the metallographic structures according to their size based on the results of a first and second test; and determine the metallographic test results for the sorted metallographic structures.
[0079] This invention provides a method for acquiring metallographic images of carbides. Multiple field-of-view images are captured on the target area of the carbide using a metallographic imaging microscope. These images are then inspected using a metallographic detection model to determine a primary detection result. The multiple field-of-view images are then stitched together, and the stitched image is inspected using the metallographic detection model to obtain a secondary detection result. Because the stitched image can completely display the metallographic structure, avoiding fragmentation of the metallographic structure, the metallographic detection result determined by combining the primary and secondary detection results can accurately reflect the metallographic structure of the carbide, thus improving the accuracy of carbide metallographic image detection.
[0080] Optionally, the method for acquiring carbide metallographic images provided in this embodiment of the invention further includes steps S201-S203.
[0081] S201. Obtain multiple carbide images marked with metallographic structures.
[0082] S202. Generate training samples based on multiple carbide images labeled with metallographic structures.
[0083] S203. Based on the training samples, train the new neural network model to obtain the metallographic detection model.
[0084] Thus, embodiments of the present invention can pre-train a metallographic detection model before metallographic image acquisition, providing a foundation for carbide metallographic detection.
[0085] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0086] The following are device embodiments of the present invention. For details not described in detail, please refer to the corresponding method embodiments described above.
[0087] Figure 2 A schematic diagram of a carbide metallographic image acquisition device according to an embodiment of the present invention is shown. The acquisition device 300 includes a communication module 301 and a processing module 302.
[0088] The communication module 301 is used to acquire multiple field-of-view images of the target area of the carbide taken by the metallographic image microscope.
[0089] The processing module 302 is used to detect multiple field-of-view images based on a pre-set metallographic detection model to obtain a primary detection result for the multiple field-of-view images; the metallographic detection model is obtained by training a neural network on the metallographic images of carbides; the multiple field-of-view images are stitched together to obtain multiple stitched images; based on the metallographic detection model, the multiple stitched images are detected to obtain a secondary detection result for the multiple field-of-view images; based on the primary detection result and the secondary detection result, the metallographic detection result is determined, thereby realizing the acquisition of metallographic images of carbides.
[0090] In one possible implementation, the processing module 302 is specifically used to detect multiple field images based on a pre-set metallographic detection model to obtain multiple detection boxes; determine the orientation of the multiple detection boxes based on their length and width; filter the multiple detection boxes based on the prior orientation and their orientation to obtain filtered detection boxes; and determine a detection result based on the filtered detection boxes.
[0091] In one possible implementation, the processing module 302 is further configured to, for the filtered detection box, if the filtered detection box in a certain field of view image is located at the edge of the field of view image, then determine that the field of view image has a cross-field of view problem; the cross-field of view problem means that the metallographic structure corresponding to the filtered detection box in the field of view image spans the field of view image and the field of view image adjacent to the field of view image.
[0092] In one possible implementation, the processing module 302 is specifically used to: identify field-of-view images with cross-field-of-view problems among multiple field-of-view images; construct a group of images to be stitched based on the field-of-view images with cross-field-of-view problems and the field-of-view images adjacent to the field-of-view images with cross-field-of-view problems; determine feature points in each field-of-view image in the group of images to be stitched based on the ORB algorithm; calculate feature descriptors based on the feature points in each field-of-view image in the group of images to be stitched; match the field-of-view images in the group of images to be stitched based on the feature points and feature descriptors, and determine the matching similarity; determine multiple pairs of field-of-view images in the group of images to be stitched based on the matching similarity; and stitch each pair of field-of-view images to obtain multiple stitched images.
[0093] In one possible implementation, the processing module 302 is specifically used to determine key point pairs for each pair of field images based on the similarity of each feature point in each pair of field images; determine the horizontal and vertical offsets of the two field images in each pair of field images based on the key point pairs; calculate the mode of the horizontal and vertical offsets; and stitch the two field images in each pair of field images together based on the mode of the horizontal and vertical offsets.
[0094] In one possible implementation, the communication module 301 is further configured to acquire multiple carbide images labeled with metallographic structures; the processing module 302 is further configured to generate training samples based on the multiple carbide images labeled with metallographic structures; and to train a new neural network model based on the training samples to obtain a metallographic detection model.
[0095] In one possible implementation, the processing module 302 is specifically used to detect multiple stitched images based on a metallographic detection model to obtain multiple detection boxes; determine the orientation of the multiple detection boxes based on their length and width; filter the multiple detection boxes based on the prior orientation and their orientation to obtain filtered detection boxes; and determine the secondary detection result based on the filtered detection boxes.
[0096] In one possible implementation, the first detection result includes a field-of-view image containing the metallographic structure and the size of the metallographic structure; the second detection result includes a stitched image containing the metallographic structure and the size of the metallographic structure; the metallographic detection result includes the size of the largest metallographic structure and the size distribution of the metallographic structure; the processing module 302 is specifically used to sort the metallographic structures according to their size based on the first and second detection results; and to determine the metallographic detection result for the sorted metallographic structures.
[0097] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. For example... Figure 3 As shown, the electronic device 400 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, it implements the steps in the above-described method embodiments, for example... Figure 1 The steps S101-S105 are shown. Alternatively, when the processor 401 executes the computer program 403, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 2 The functions of the communication module 301 and the processing module 302 shown are illustrated.
[0098] For example, the computer program 403 can be divided into one or more modules / units, which are stored in the memory 402 and executed by the processor 401 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 403 in the electronic device 400. For example, the computer program 403 can be divided into... Figure 2 The communication module 301 and the processing module 302 are shown.
[0099] The processor 401 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.
[0100] The memory 402 can be an internal storage unit of the electronic device 400, such as a hard disk or memory of the electronic device 400. The memory 402 can also be an external storage device of the electronic device 400, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD) card, flash card, etc., equipped on the electronic device 400. Furthermore, the memory 402 can include both internal and external storage units of the electronic device 400. The memory 402 is used to store the computer program and other programs and data required by the terminal. The memory 402 can also be used to temporarily store data that has been output or will be output.
[0101] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0102] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0103] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.
[0104] In the embodiments provided by this invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0105] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0106] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0107] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.
[0108] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A method for acquiring carbide metallographic images, characterized in that, The acquisition method, applied to metallographic imaging microscopes, includes: The metallographic imaging microscope captures multiple field-of-view images of the target area of the carbide. Based on a pre-set metallographic detection model, multiple field-of-view images are detected to obtain a single detection result for the multiple field-of-view images. This includes: detecting multiple field-of-view images based on the pre-set metallographic detection model to obtain multiple detection boxes; determining the orientation of the multiple detection boxes based on their length and width; filtering the multiple detection boxes based on prior orientation and their orientation to obtain filtered detection boxes; and determining the single detection result based on the filtered detection boxes. The metallographic detection model is obtained by training a neural network on the metallographic images of the carbide. The process of stitching together multiple field-of-view images to obtain multiple stitched images includes: identifying field-of-view images with cross-field-of-view problems; constructing a group of images to be stitched based on the field-of-view images with cross-field-of-view problems and adjacent field-of-view images; determining feature points in each field-of-view image in the group of images to be stitched based on the ORB algorithm; calculating feature descriptors based on the feature points in each field-of-view image in the group of images to be stitched; matching the field-of-view images in the group of images to be stitched based on the feature points and the feature descriptors to determine the matching similarity; determining multiple pairs of field-of-view images in the group of images to be stitched based on the matching similarity; and stitching together each pair of field-of-view images to obtain the multiple stitched images. Based on the metallographic detection model, the multiple stitched images are detected to obtain the secondary detection results of the multiple field-of-view images; Based on the first and second detection results, the metallographic detection results are determined, and the metallographic image of the carbide is acquired. Before stitching together the multiple field-of-view images to obtain multiple stitched images, the method further includes: for the filtered detection boxes, if the filtered detection boxes in a certain field-of-view image are located at the edge of the field-of-view image, then it is determined that the field-of-view image has a cross-field-of-view problem; the cross-field-of-view problem means that the metallographic structure corresponding to the filtered detection boxes in the field-of-view image spans the field-of-view image and the field-of-view image adjacent to the field-of-view image.
2. The method for acquiring carbide metallographic images according to claim 1, characterized in that, The process of stitching together each pair of field-of-view images to obtain the plurality of stitched images includes: For each pair of field-of-view images, key point pairs are determined based on the similarity of each feature point in each pair of field-of-view images; Based on the key point pairs, determine the horizontal and vertical offsets of the two field images in each pair of field images; Calculate the mode of the horizontal offset and the mode of the vertical offset; Based on the mode of the horizontal offset and the mode of the vertical offset, the two field images in each pair of field images are stitched together.
3. The method for acquiring carbide metallographic images according to claim 1, characterized in that, Before obtaining a single detection result for the multiple field-of-view images based on a pre-set metallographic detection model, the method further includes: Acquire multiple carbide images marked with metallographic structures; Training samples are generated based on the multiple carbide images labeled with metallographic structures. Based on the training samples, a new neural network model is trained to obtain the metallographic detection model.
4. The method for acquiring carbide metallographic images according to claim 1, characterized in that, The step of detecting the multiple stitched images based on the metallographic detection model to obtain secondary detection results for the multiple field-of-view images includes: Based on the metallographic detection model, multiple stitched images are detected to obtain multiple detection boxes; The orientation of the multiple detection frames is determined based on their length and width. Based on the prior direction and the direction of the multiple detection boxes, the multiple detection boxes are filtered to obtain the filtered detection boxes; Based on the filtered detection box, the secondary detection result is determined.
5. The method for acquiring carbide metallographic images according to claim 1, characterized in that, The first detection result includes a field-of-view image containing the metallographic structure and the size of the metallographic structure; the second detection result includes a stitched image containing the metallographic structure and the size of the metallographic structure; the metallographic detection result includes the size of the largest metallographic structure and the size distribution of the metallographic structure. The process of determining the metallographic detection result based on the first and second detection results, and acquiring the metallographic image of the carbide, includes: Based on the results of the first and second tests, the metallographic structures are sorted according to their size. For the sorted metallographic structures, the metallographic test results are determined.
6. A device for acquiring carbide metallographic images, characterized in that, An apparatus for use in a metallographic imaging microscope, the metallographic imaging microscope comprising a three-dimensional motion control console and a stage; the apparatus includes: The communication module is used to acquire multiple field-of-view images of the target area of the carbide taken by the metallographic imaging microscope; The processing module is used to detect multiple field-of-view images based on a pre-set metallographic detection model to obtain a primary detection result for the multiple field-of-view images; the metallographic detection model is obtained by training a neural network on the metallographic image of the carbide; the multiple field-of-view images are stitched together to obtain multiple stitched images; based on the metallographic detection model, the multiple stitched images are detected to obtain a secondary detection result for the multiple field-of-view images; based on the primary detection result and the secondary detection result, the metallographic detection result is determined, thereby realizing the acquisition of the metallographic image of the carbide; The processing module is specifically used to: identify field-of-view images with cross-field-of-view problems among the plurality of field-of-view images; construct a group of images to be stitched based on the field-of-view images with cross-field-of-view problems and the field-of-view images adjacent to them; determine feature points in each field-of-view image in the group of images to be stitched based on the ORB algorithm; calculate feature descriptors based on the feature points in each field-of-view image in the group of images to be stitched; match the field-of-view images in the group of images to be stitched based on the feature points and the feature descriptors to determine the matching similarity; determine multiple pairs of field-of-view images in the group of images to be stitched based on the matching similarity; and stitch each pair of field-of-view images to obtain the plurality of stitched images. The processing module is specifically used to detect multiple field images based on a pre-set metallographic detection model to obtain multiple detection boxes; determine the orientation of the multiple detection boxes based on their length and width; filter the multiple detection boxes based on the prior orientation and their orientation to obtain filtered detection boxes; and determine the first detection result based on the filtered detection boxes. The processing module is further configured to determine that the field of view has a cross-field problem if the filtered detection box is located at the edge of the field of view in a certain field of view image; the cross-field problem means that the metallographic structure corresponding to the filtered detection box in the field of view image spans the field of view image and the field of view image adjacent to the field of view image.
7. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor being configured to invoke and run the computer program stored in the memory to perform the method as described in any one of claims 1 to 5.