Crystal grain detection and analysis method and related equipment

Abstract

The invention discloses a crystal grain detection and analysis method and related equipment, effective crystal grains are obtained through corrosion expansion pretreatment denoising, OfficientNet and bidirectional feature pyramid network detection, contour obtaining through region segmentation and three-step filtering, qualification is judged in combination with a threshold value, a report is generated, and the existing technical problems are solved in a targeted mode. On the aspect of reducing calculation complexity, preprocessing reduces model data volume and interference, the model abandons complex modules, parameters are few, calculation cost is low, hardware occupation can be reduced, and detection and training time can be shortened; on the aspect of improving generalization ability, features are preprocessed and purified, interference of imaging conditions and material types is reduced, the number of layers of the model is simplified, multi-scale fusion ability is achieved, and detection in different fields can be adapted without optimization of a specific data set; on the aspect of reducing data dependence, invalid and pseudo crystal grains are accurately removed through three-step filtering, defects of a small-scale or low-quality data set are cooperatively made up, dependence on a high-quality large-scale data set is greatly reduced, and robustness is improved.

Description

Technical Field

[0001] This application relates to the field of intelligent detection technology, and more specifically, to a grain detection and analysis method and related equipment. Background Technology

[0002] Grain structure is the core factor determining the mechanical properties of materials. Accurately detecting the location, type, outline, and conformity of grains is a key link in material research and development and industrial production quality control, directly affecting the reliability of products in aerospace, automobile manufacturing and other fields. Therefore, there is an urgent need for efficient and accurate grain detection and analysis technology.

[0003] Current mainstream grain detection methods largely rely on complex deep learning architectures. For example, some schemes introduce query set self-reconstruction modules and cross-attention modules to enhance feature extraction. The training phase also requires a large number of data augmentation and complex feature encoding operations, and has stringent requirements for datasets, requiring large-scale, high-quality labeled metallographic images to support model training. However, existing technologies have significant drawbacks: First, they have high computational complexity. Complex modules significantly increase the computational load, requiring high-performance hardware support when processing large-scale datasets, and the training and detection times are lengthy, making it difficult to meet the real-time requirements of industry. Second, they have weak generalization ability. Models are mostly optimized on specific benchmark datasets, and when applied to fine-grained detection tasks with different materials or imaging conditions, the accuracy drops sharply, making them unable to adapt to diverse scenarios. Third, they are highly dependent on data. High-quality labeled datasets are scarce in actual production. If the dataset is small or contains noise or labeling errors, it will directly lead to poor model training results, making it difficult to guarantee detection accuracy and severely limiting the application of the technology.

[0004] Therefore, a new grain detection and analysis scheme is urgently needed to avoid the shortcomings of existing technologies. Summary of the Invention

[0005] This application provides a grain detection and analysis method and related equipment. It uses corrosion expansion preprocessing to remove noise, EfficientNet and bidirectional feature pyramid network detection, region segmentation to obtain contours, and three-step filtering to obtain effective grains. Combined with threshold judgment to generate a pass / fail report, it reduces the amount of data and interference through preprocessing, simplifies the model and eliminates complex modules to reduce computational complexity, reduce hardware usage and time. It also improves the generalization ability of the model by preprocessing to purify features and the multi-scale fusion capability of the model to adapt to detection in different fields. At the same time, it reduces data dependence, ensures detection accuracy and improves robustness.

[0006] A method for grain detection and analysis, comprising:

[0007] Etching and dilation operations are performed on the acquired raw metallographic image to obtain a pre-processed filtered image with impurities and noise removed.

[0008] The preprocessed and filtered image is input into a grain detection model that includes an EfficientNet backbone network and a bidirectional feature pyramid network. The model performs feature extraction and target recognition to generate a set of detection boxes that include grain location and category.

[0009] Based on the detection box set, the corresponding region in the preprocessed image is segmented to extract the contour of each grain and form a grain contour set.

[0010] The set of grain contours is sequentially filtered using a grain area screening method, a multi-feature fusion scoring method, and a density clustering method to remove invalid grains and pseudo grains, thereby obtaining a set of valid grains.

[0011] Based on the size information of each grain in the effective grain set, and combined with the user-preset judgment threshold, a pass / fail determination is made, and a test report containing statistical results and judgment conclusions is generated.

[0012] Optionally, both the corrosion operation and the expansion operation use a core with a 5×5 structural element;

[0013] The calculation formulas for the corrosion operation and the expansion operation are as follows:

[0014]

[0015]

[0016] Where I is the input image and K is the structuring element. These are the pixel coordinates within the structuring element. To output the image after the erosion operation, This is the output image after the dilation operation.

[0017] Optionally, the grain detection model includes:

[0018] The EfficientNet backbone network is used to extract multi-scale features from the input preprocessed and filtered image and output feature maps at multiple levels.

[0019] A bidirectional feature pyramid network is used to perform bidirectional weighted fusion of feature maps from multiple levels, and output an enhanced multi-scale feature map.

[0020] The detection box prediction network, based on the enhanced multi-scale feature map, regresses and generates the localization detection box of the grain;

[0021] A category prediction network predicts the category of the target within the localization detection box based on the enhanced multi-scale feature map.

[0022] Optionally, based on the detection box set, image segmentation is performed on the corresponding regions in the preprocessed image to extract the contour of each grain, forming a grain contour set, including:

[0023] Convert the image region corresponding to the set of detection boxes into a grayscale image;

[0024] The grayscale image is binarized to obtain a binary image;

[0025] The contours of connected regions are extracted from the binary image to form the set of grain contours.

[0026] Optionally, the grain area screening method includes:

[0027] Calculate the mean area and standard deviation of the area of ​​all grains in the grain profile set;

[0028] Based on the mean area and the standard deviation of the area, the upper limit and lower limit of the effective area are determined to form the effective area range of the grain;

[0029] Select and retain grains in the set of grain profiles whose grain area falls within the effective area range of the grains;

[0030] The formulas for determining the upper limit and lower limit of the effective area are as follows:

[0031]

[0032]

[0033] in, This is the upper limit of the effective area. This is the lower limit of the effective area. The average area of ​​all grains. The area standard deviation of all grains. This is an adjustable coefficient.

[0034] Optionally, the scoring calculation formula used in the multi-feature fusion scoring method is as follows:

[0035]

[0036]

[0037]

[0038] in, For multi-feature fusion scoring, For grain area, For roundness, For compactness, For the perimeter, It is a linear normalized function. These are the corresponding weighting coefficients.

[0039] Optionally, the processing steps of the density clustering method include:

[0040] For each grain in the set of grain profiles, the number of other grain center coordinates within the neighborhood radius is calculated based on its center coordinates.

[0041] If the number is less than the preset minimum number of neighborhood points, the grain is determined to be a pseudo grain and is removed.

[0042] A grain detection and analysis device, comprising:

[0043] The filtering module is used to perform erosion and dilation operations on the acquired raw metallographic image to obtain a pre-processed filtered image with impurities and noise removed.

[0044] The grain detection module is used to input the preprocessed filtered image into a grain detection model containing an EfficientNet backbone network and a bidirectional feature pyramid network. The model performs feature extraction and target recognition to generate a set of detection boxes containing grain locations and categories.

[0045] The contour recognition module is used to perform image segmentation on the corresponding region in the preprocessed image based on the detection box set, extract the contour of each grain, and form a grain contour set.

[0046] The effective screening module is used to sequentially perform filtering processing on the set of grain contours, including grain area screening, multi-feature fusion scoring and density clustering, to remove invalid grains and pseudo grains and obtain an effective set of grains.

[0047] The pass / fail determination module is used to determine the pass / fail status based on the size information of each grain in the effective grain set and the user-preset determination threshold, and generate a test report containing statistical results and determination conclusions.

[0048] A grain detection and analysis device, comprising a memory and a processor;

[0049] The memory is used to store programs;

[0050] The processor is configured to execute the program to implement the various steps of the grain detection and analysis method as described in any of the preceding claims.

[0051] A readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the grain detection and analysis method as described in any of the preceding claims.

[0052] As can be seen from the above technical solutions, the grain detection and analysis method and related equipment provided in this application first perform corrosion and dilation operations on the original metallographic image to obtain a preprocessed filtered image, and then input it into a detection model containing an EfficientNet backbone network and a bidirectional feature pyramid network. After feature extraction and target recognition, a set of detection boxes is generated. Subsequently, the corresponding region is segmented to obtain a set of grain contours. Through area screening, multi-feature fusion scoring and density clustering filtering, an effective set of grains is obtained. Finally, the passability is determined by combining the grain size and a preset threshold, and a detection report is generated.

[0053] This application addresses the shortcomings of existing technologies in several ways: Firstly, it reduces computational complexity and resource consumption. Preprocessing removes noise and impurities through erosion and dilation algorithms, reducing the amount of data and interference in subsequent model processing. The model employs an efficient EfficientNet backbone network and a bidirectional feature pyramid network, eliminating complex modules such as query set self-reconstruction, significantly reducing computational load and minimizing hardware usage when processing large-scale data, thus shortening detection and training time. Secondly, it enhances generalization ability. Preprocessing purifies input features, reducing interference from different imaging conditions and material types. Furthermore, the simplified EfficientNet backbone network, combined with the multi-scale fusion capability of the bidirectional feature pyramid, allows for adaptation to fine-grained detection in different fields without relying on specific datasets for optimization, resulting in significantly improved generalization performance. Thirdly, it reduces data dependency. The three-step filtering accurately removes invalid and pseudo-crystals. Even with small or low-quality training datasets, preprocessing and postprocessing can collaboratively compensate for data deficiencies, ensuring detection accuracy and significantly reducing reliance on high-quality, large-scale datasets, thereby improving the model's robustness for industrial applications. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0055] Figure 1 This is a flowchart of a grain detection and analysis method disclosed in an embodiment of this application;

[0056] Figure 2 This is a schematic diagram comparing the corrosion and expansion operations before and after the treatment disclosed in an embodiment of this application;

[0057] Figure 3 This is a schematic diagram of an image containing a set of detection boxes, as disclosed in an embodiment of this application;

[0058] Figure 4This is a schematic diagram of a grain detection and analysis device disclosed in an embodiment of this application;

[0059] Figure 5 This is a hardware structure block diagram of a grain detection and analysis device disclosed in an embodiment of this application. Detailed Implementation

[0060] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] This application can be used in a wide variety of general-purpose or special-purpose computing device environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor devices, distributed computing environments including any of the above devices, etc.

[0062] The following section introduces the solution proposed in this application. The technical solution is as follows, and details are provided below.

[0063] Figure 1 This is a flowchart of a grain detection and analysis method disclosed in an embodiment of this application.

[0064] like Figure 1 As shown, the method may include:

[0065] Step S1: Perform erosion and dilation operations on the acquired original metallographic image to obtain a pre-processed filtered image with impurities and noise removed.

[0066] Specifically, etching and dilation operations are sequentially performed on the acquired raw metallographic image to remove impurities and noise, preventing these impurities from interfering with subsequent grain detection and measurement. The result is a pre-processed, filtered image with effectively removed impurities and noise, such as... Figure 2 The image shown is the input image and the output image after processing with erosion and dilation operations.

[0067] Furthermore, both the corrosion operation and the expansion operation employ a core with a 5×5 structural element.

[0068] The calculation formulas for the corrosion operation and the expansion operation are as follows:

[0069]

[0070]

[0071] Where I is the input image and K is the structuring element. These are the pixel coordinates within the structuring element. To output the image after the erosion operation, This is the output image after the dilation operation.

[0072] Specifically, both the corrosion and expansion operations use a 5×5 core as the structural element. This size of structural element has been verified to be suitable, as it can accurately target tiny impurities in the original metallographic image while avoiding damage to the main grain outline, thus balancing the requirements of noise reduction effect and grain structure integrity.

[0073] The core function of the erosion operation is to shrink the foreground region (i.e., the potential grain region) in the original metallographic image, thereby specifically removing small-sized foreground noise. The processing logic is as follows: using a 5×5 structuring element as the traversal unit, it moves point-by-point across the original metallographic image (denoted as the input image). For each pixel position in the input image, the minimum value of all pixels within the area covered by the structuring element at that position is taken. This minimum value is used as the output value for the corresponding pixel position after the erosion operation. In this way, foreground noise smaller than 5×5 will disappear due to erosion, achieving initial noise reduction.

[0074] The dilation operation is performed after the erosion operation. Its core function is to expand the foreground region after erosion, fill in the tiny holes that may have been created in the foreground region (i.e., the grain region) due to the erosion operation, and further solidify the main outline of the grain. The processing logic is as follows: using a 5×5 structuring element as the traversal unit, it moves point by point across the eroded image. For each pixel position in the image, it takes the maximum value of all pixels within the area covered by the structuring element at that position, and uses this maximum value as the output value for the corresponding pixel position after the dilation operation. In this way, tiny defects in the grain region are filled, ensuring the integrity of the grain outline.

[0075] By combining etching and expansion, impurities and noise in the original metallographic image can be systematically filtered out, laying a clear and reliable image foundation for the subsequent input of the image into the grain detection model. This preprocessing method is completely consistent with the technical design for image impurity removal in the document and conforms to the technical logic of the overall detection process.

[0076] Step S2: Input the preprocessed filtered image into a grain detection model that includes an EfficientNet backbone network and a bidirectional feature pyramid network. The model performs feature extraction and target recognition to generate a set of detection boxes that include grain location and category.

[0077] Specifically, the preprocessed and filtered image obtained in step S1 is input into a preset grain detection model. This model is specifically designed for grain detection scenarios and consists of four parts working together: an EfficientNet backbone network, a Bidirectional Feature Pyramid Network (BiFPN), a category prediction network, and a bounding box prediction network. Through multi-module division of labor, feature extraction and target recognition are achieved, ultimately generating a set of bounding boxes containing grain locations and categories, such as... Figure 3 As shown.

[0078] The EfficientNet backbone network, based on a composite scaling strategy, can extract multi-scale semantic feature maps with low computational cost while balancing network depth, width, and input resolution. These feature maps encompass information from low-order (e.g., grain edges, texture) to high-order (e.g., grain category semantics), providing comprehensive feature support for subsequent recognition. Secondly, these multi-scale feature maps are input into a Bidirectional Feature Pyramid Network (BiFPN). This network uses bidirectional connections (top-down and bottom-up) to weightedly fuse feature maps of different resolutions, enhancing the detailed information of low-resolution feature maps while supplementing the semantic information of high-resolution feature maps, thus solving the information loss problem that easily occurs in traditional feature pyramids.

[0079] Step S3: Based on the detection box set, perform image segmentation on the corresponding region in the preprocessed image, extract the contour of each grain, and form a grain contour set.

[0080] Specifically, based on the generated set of detection boxes, the region corresponding to each potential grain in the preprocessed filtered image is located. A three-step image processing workflow is then used to extract the complete contour of each grain, forming a set of grain contours. This provides a precise morphological basis for subsequent filtering and determination. The specific operations are as follows:

[0081] The first step is grayscale conversion. For each detection box in the detection box set, the image region corresponding to that region (i.e., the region of a single potential grain) is converted into a grayscale image. The conversion process assigns weights based on the differences in human eye sensitivity to red, green, and blue colors, focusing on preserving the brightness difference between the grain and the background. This ensures that the converted grayscale image can clearly distinguish the grain from the background, laying the foundation for subsequent segmentation operations.

[0082] The second step is binarization. Binarization is performed on the grayscale image, setting a preset threshold (this threshold can be calibrated according to the grayscale characteristics of metallographic images of different materials). Regions with grayscale values ​​below the threshold (i.e., grain regions) are marked as white, and regions with grayscale values ​​above the threshold (i.e., background regions) are marked as black. This operation achieves a clear separation between grains and the background, presenting the grain regions as solid-color connected blocks, facilitating subsequent contour extraction.

[0083] The third step is contour extraction. Contour detection is performed on the binarized image. By traversing the white connected regions in the image, the boundary pixels of each connected region are traced to form the complete boundary trajectory of each region. This trajectory is the contour of the corresponding grain. The contours of all grains together constitute the grain contour set, achieving accurate capture of the morphology of each grain.

[0084] In other words, based on the detection box set, image segmentation is performed on the corresponding regions in the preprocessed image to extract the contour of each grain, forming a grain contour set, further including:

[0085] Convert the image region corresponding to the set of detection boxes into a grayscale image;

[0086] The grayscale image is binarized to obtain a binary image;

[0087] The contours of connected regions are extracted from the binary image to form the set of grain contours.

[0088] Step S4: The set of grain contours is sequentially filtered using a grain area screening method, a multi-feature fusion scoring method, and a density clustering method to remove invalid grains and pseudo grains, thereby obtaining a set of valid grains.

[0089] Specifically, for the set of grain contours, three rounds of processing are performed sequentially: grain area screening, multi-feature fusion scoring, and density clustering filtering. Through layer-by-layer verification, invalid grains (such as grains with abnormal size or shape) and spurious grains (such as false grains formed by impurity agglomeration or isolated noise) are eliminated, ultimately yielding a true and valid set of grains. The specific process is as follows:

[0090] The first round is grain area screening. First, the actual area of ​​each grain is calculated based on its outline. Second, the mean and standard deviation of the area of ​​all grains in the current image frame are calculated, and combined with a preset adjustable coefficient, a dynamic area threshold range is calculated. Finally, the area of ​​each grain is compared with the threshold range, and only grains with an area within the threshold range are retained, while invalid grains with an area that is too large or too small are removed.

[0091] The second round involves multi-feature fusion scoring. For the grains retained after area screening, three core features are calculated: grain area, roundness, and compactness. These three features are normalized individually, and then pre-defined weights are assigned based on the influence of each feature on the grain's effectiveness to calculate a comprehensive score for each grain. A scoring threshold is set, and only grains with scores above the threshold are retained, while grains with excessively low scores are discarded.

[0092] The third round is density clustering filtering. First, the center coordinates of each qualified grain are calculated; second, the neighborhood radius and the minimum number of neighborhood points are set; then, the distance between any two grain centers is calculated, and the number of other grains contained within the set neighborhood radius of each grain center is counted; finally, if the number of grains in the neighborhood of a certain grain center is less than the preset minimum number of neighborhood points, the grain is determined to be an outlier pseudo-grain and is removed.

[0093] Step S5: Based on the size information of each grain in the effective grain set, and combined with the user-preset judgment threshold, perform a qualification judgment and generate a test report containing statistical results and judgment conclusions.

[0094] Specifically, the effective grain set obtained in step S4 is used as the core data source. Based on the size information of each grain in the set, and combined with the judgment threshold preset by the user based on the detection requirements, a pass / fail judgment is performed, and a test report containing statistical results and judgment conclusions is generated, thus realizing a closed loop of grain detection.

[0095] The specific process is as follows: First, extract the size information of the effective dies. Based on the outline of each die in the effective die set and the previously calculated data, obtain key size parameters, including but not limited to die length, width, and area. Second, perform a pass / fail judgment on a single die, obtaining the judgment threshold preset by the user for the testing scenario. This threshold must cover all extracted size parameters, including but not limited to the upper and lower limits of die length, width, and area. Compare the size parameters of each die with the corresponding preset threshold one by one. If all size parameters of the die are within the preset threshold range, the die is judged as qualified; if any size parameter exceeds the preset threshold, the die is judged as unqualified. Then, conduct statistical analysis of the overall test results. Summarize and statistically analyze the pass / fail judgment results of all effective dies, and calculate the core statistical indicators: the total number of effective dies, the number of qualified dies, and the number of unqualified dies. Then, according to the yield calculation logic in this document, obtain the yield of this test, quantify the overall die pass / fail status, and provide data support for subsequent overall judgment. Finally, generate a standardized test report.

[0096] As can be seen from the above technical solutions, the grain detection and analysis method and related equipment provided in this application first perform corrosion and dilation operations on the original metallographic image to obtain a preprocessed filtered image, and then input it into a detection model containing an EfficientNet backbone network and a bidirectional feature pyramid network. After feature extraction and target recognition, a set of detection boxes is generated. Subsequently, the corresponding region is segmented to obtain a set of grain contours. Through area screening, multi-feature fusion scoring and density clustering filtering, an effective set of grains is obtained. Finally, the passability is determined by combining the grain size and a preset threshold, and a detection report is generated.

[0097] This application addresses the shortcomings of existing technologies in several ways: Firstly, it reduces computational complexity and resource consumption. Preprocessing removes noise and impurities through erosion and dilation algorithms, reducing the amount of data and interference in subsequent model processing. The model employs an efficient EfficientNet backbone network and a bidirectional feature pyramid network, eliminating complex modules such as query set self-reconstruction, significantly reducing computational load and minimizing hardware usage when processing large-scale data, thus shortening detection and training time. Secondly, it enhances generalization ability. Preprocessing purifies input features, reducing interference from different imaging conditions and material types. Furthermore, the simplified EfficientNet backbone network, combined with the multi-scale fusion capability of the bidirectional feature pyramid, allows for adaptation to fine-grained detection in different fields without relying on specific datasets for optimization, resulting in significantly improved generalization performance. Thirdly, it reduces data dependency. The three-step filtering accurately removes invalid and pseudo-crystals. Even with small or low-quality training datasets, preprocessing and postprocessing can collaboratively compensate for data deficiencies, ensuring detection accuracy and significantly reducing reliance on high-quality, large-scale datasets, thereby improving the model's robustness for industrial applications.

[0098] In some embodiments of this application, the structure of the grain detection model in step S2 is described, and the grain detection model may specifically include:

[0099] The EfficientNet backbone network is used to extract multi-scale features from the input preprocessed and filtered image and output feature maps at multiple levels.

[0100] A bidirectional feature pyramid network is used to perform bidirectional weighted fusion of feature maps from multiple levels, and output an enhanced multi-scale feature map.

[0101] The detection box prediction network, based on the enhanced multi-scale feature map, regresses and generates the localization detection box of the grain;

[0102] A category prediction network predicts the category of the target within the localization detection box based on the enhanced multi-scale feature map.

[0103] Specifically, the core function of the EfficientNet backbone network is to perform multi-scale feature extraction on the pre-processed and filtered input image, outputting feature maps at multiple levels to provide basic feature support for subsequent feature fusion. This network adopts a composite scaling design approach, balancing network depth, width, and input image resolution to significantly reduce computational resource consumption while ensuring feature extraction accuracy. Furthermore, it allows for flexible selection of an appropriate network size version based on the accuracy and speed requirements of the actual detection scenario.

[0104] In the specific processing, after the preprocessed and filtered image is input into the EfficientNet backbone network, it is processed step by step through multiple convolutional and pooling operations: convolutional operations are used to capture local features in the image (such as grain edges and texture details), while pooling operations are used to reduce the size of the feature maps, retain key features, and improve the network's generalization ability. After multiple rounds of processing, the network outputs multiple feature maps of different resolutions (i.e., feature maps at multiple levels). The low-resolution feature maps contain high-order semantic information (which can reflect the overall category features of the grains), while the high-resolution feature maps contain low-order detail information (which can reflect local features such as grain boundaries and textures). These feature maps completely cover the key information of the grains from local details to the overall category, laying the foundation for the subsequent fusion processing of the bidirectional feature pyramid network.

[0105] In addition, the training data of this backbone network is obtained by sampling metallographic samples using suitable equipment during the data acquisition stage and by labeling the grains in the sampled images using open source tools during the data labeling stage, forming a dataset with labeled information, ensuring that the network has feature extraction adaptability for actual metallographic images after training.

[0106] The core function of the bidirectional feature pyramid network is to perform bidirectional weighted fusion of feature maps from multiple levels output by the EfficientNet backbone network, producing an enhanced multi-scale feature map. This addresses the information loss problem common in traditional feature pyramid networks, improving the semantic expressiveness and detail preservation capabilities of features, aligning with the design goal of enhancing grain detection performance. The network achieves feature fusion through a top-down and bottom-up bidirectional connection structure: the top-down connection path transfers high-order semantic information from low-resolution feature maps to high-resolution feature maps, compensating for the semantic discriminative power of high-resolution feature maps; the bottom-up connection path feeds back low-order detail information from high-resolution feature maps to low-resolution feature maps, supplementing the lack of detail capture in low-resolution feature maps. During the fusion process, the network weights the contribution of feature maps from different levels, ensuring that both semantic and detail information are effectively preserved in the fused feature map. The final output enhanced multi-scale feature map possesses both clear grain semantic category information and accurate grain boundary detail information, providing high-quality feature input for subsequent bounding box prediction and category prediction.

[0107] The core function of the bounding box prediction network is to generate localization bounding boxes for each grain in the image by using the enhanced multi-scale feature map output by the bidirectional feature pyramid network and regression calculations. In practice, after the enhanced multi-scale feature map is input into the bounding box prediction network, the network processes it through multiple layers of convolution and regression calculations: convolution operations further extract key information related to the grain position from the feature map, while regression calculations predict the specific location parameters of the bounding box based on this information. Because the enhanced multi-scale feature map contains feature information at different resolutions, the network can generate suitable localization bounding boxes for grains of different sizes, ensuring the accuracy of large-sized grain localization while avoiding localization errors caused by insufficient feature information for small-sized grains. The generated localization bounding boxes directly correspond to the potential grain regions in the image, providing clear locational basis for subsequent category prediction and contour extraction.

[0108] The core function of the category prediction network is to predict the category of targets within a localization detection box based on the enhanced multi-scale feature map output by the bidirectional feature pyramid network. This determines whether a target within each localization detection box is a grain and its specific grain type, thus achieving grain classification and recognition. Specifically, the enhanced multi-scale feature map is simultaneously input into the category prediction network, which processes it through convolution operations and activation functions: convolution operations extract semantic information related to grain categories from the feature map, while the activation function transforms this information into a category probability distribution. Based on this probability distribution, the network can determine whether a target within a localization detection box is a valid grain and further identify the specific category of the valid grain.

[0109] In addition, after the output of the bounding box prediction network and the category prediction network, the implicit optimization logic in this document will be used for filtering: through confidence threshold filtering and deduplication, a set of bounding boxes containing grain positions and categories will be generated in the end, providing accurate target region localization for contour extraction.

[0110] In some embodiments of this application, the filtering process of grain area screening, multi-feature fusion scoring, and density clustering in step S4 is described, which may specifically include:

[0111] Grain area screening method:

[0112] ① Calculate the mean area and standard deviation of the area of ​​all grains in the grain profile set;

[0113] ② Based on the mean area and the standard deviation of the area, determine the upper limit and lower limit of the effective area to form the effective area range of the grain;

[0114] ③ Screen and retain grains whose grain area is within the effective area range of the grain profile set;

[0115] The formulas for determining the upper limit and lower limit of the effective area are as follows:

[0116]

[0117]

[0118] in, This is the upper limit of the effective area. This is the lower limit of the effective area. The average area of ​​all grains. The area standard deviation of all grains. This is an adjustable coefficient.

[0119] Specifically, firstly, based on the contour data of each grain in the grain contour set, the mean area and standard deviation of the area of ​​all grains in the set are calculated. The grain area is obtained by relying on the grain contours extracted in the previous image segmentation step, combined with the calibration parameters of the detection equipment, to convert the pixel-level area corresponding to the contour into the actual physical area, ensuring the physical validity of the area data. The calculated mean area reflects the overall level of the grain area under the current field of view, while the standard deviation reflects the dispersion of the grain area; both serve as the core basis for subsequently determining the effective area range. Secondly, based on the calculated mean area and standard deviation, combined with a preset adjustable coefficient, the upper and lower limits of the effective area are determined, thus constituting the effective grain area range. The adjustable coefficient can be calibrated according to the material properties of the sample being tested and the actual testing scenario requirements. Its function is to flexibly adapt to the size distribution characteristics of different types of grains, ensuring that the effective area range covers most normal-sized grains while excluding abnormal grains of extreme sizes. Finally, the actual area of ​​each grain in the grain profile set is compared with the effective area range of the grains constructed above. Grains with areas within this range are selected and retained, while grains with areas exceeding the upper limit of the effective area or below the lower limit of the effective area are removed. This initially achieves the filtering of invalid grains with abnormal sizes, laying the foundation for subsequent refined screening.

[0120] Multi-feature fusion scoring method:

[0121] The scoring formula used in the multi-feature fusion scoring method is as follows:

[0122]

[0123]

[0124]

[0125] in, For multi-feature fusion scoring, For grain area, For roundness, For compactness, For the perimeter, It is a linear normalized function. These are the corresponding weighting coefficients.

[0126] Specifically, the multi-feature fusion scoring method comprehensively considers and quantifies multiple morphological features of the grains to further refine the screening of grains with abnormal shapes. This method uses grains retained after area screening as the processing object, and its core lies in constructing a comprehensive scoring system by combining multiple key features of the grains. These features include grain area, roundness, and compactness. The grain area is consistent with the actual physical area used in the aforementioned area screening to ensure data consistency; roundness reflects the degree to which the grain shape approximates a standard circle, and its calculation relies on the grain's perimeter and area. The perimeter is obtained by statistically analyzing the edge pixels of the grain's outline; compactness reflects the regularity of the grain's outline, and is also related to the grain's area and perimeter. Together, these two features supplement the shortcomings of area features in determining the shape. In the specific scoring process, the three features of grain area, roundness, and compactness are first linearly normalized. By mapping the values ​​of each feature to a unified numerical range, the interference caused by differences in the dimensions of different features is eliminated, ensuring that each feature has a comparable basis in the scoring system. Subsequently, based on the influence weight of each feature on the determination of grain validity, a corresponding weight coefficient is assigned to each normalized feature. The setting of the weight coefficients needs to be calibrated based on a large number of sample tests and industry testing experience to reflect the differences in the importance of different features in distinguishing valid grains from abnormally shaped grains. Based on the above normalized features and corresponding weight coefficients, a multi-feature fusion score for each grain is obtained through weighted calculation. Finally, this score is compared with a preset scoring threshold. Grains with scores that reach or exceed the threshold are retained, while abnormally shaped grains with scores below the threshold are removed, further improving the effectiveness and regularity of the selected grain set.

[0127] Density clustering method:

[0128] ① For each grain in the set of grain profiles, using its center coordinates as a reference, calculate the number of other grain center coordinates within the neighborhood radius;

[0129] ②If the number is less than the preset minimum number of neighborhood points, the grain is determined to be a pseudo grain and is removed.

[0130] Specifically, this method uses the grains retained after multi-feature fusion scoring as the processing object. The core logic is that real grains usually exhibit clustered distribution characteristics in the field of view, while pseudo grains mostly exist in an isolated form, which can be distinguished through spatial density analysis. First, for each grain in the processing object, its center coordinates need to be determined. These center coordinates are the geometric center of the grain outline, obtained by geometric calculation of the pixel coordinates of the grain outline, ensuring that the center coordinates can accurately reflect the spatial position of the grain in the image. Based on these center coordinates, a neighborhood radius is set. The setting of this neighborhood radius needs to be adapted to the average distribution density of grains in the current field of view, ensuring that it can effectively cover the adjacent grains around the real grains. Then, the number of other grain center coordinates within this neighborhood radius is calculated. This number directly reflects the density of the current grains in spatial distribution. Second, a minimum number of neighborhood points is preset. The setting of this number of points needs to be based on the statistical law of the clustered distribution of real grains, ensuring that it can distinguish between normally clustered grains and isolated pseudo grains. The number of grain centers within the calculated neighborhood is compared with a preset minimum number of neighborhood points. If the number is less than the minimum number of neighborhood points, it indicates that the grain is spatially isolated and does not conform to the cluster distribution characteristics of real grains; therefore, it is identified as a pseudo-grain and removed from the grain set. If the number reaches or exceeds the minimum number of neighborhood points, it indicates that the grain belongs to a normal cluster and is retained for subsequent processing. This method effectively removes isolated pseudo-grains formed by impurities, noise, and other factors, ultimately yielding a real, effective grain set with a spatial distribution consistent with actual conditions.

[0131] The following describes a grain detection and analysis device provided in the embodiments of this application. The grain detection and analysis device described below can be referred to in correspondence with the grain detection and analysis method described above.

[0132] See Figure 4 , Figure 4 This is a schematic diagram of a grain detection and analysis device disclosed in an embodiment of this application.

[0133] like Figure 4 As shown, the grain detection and analysis device may include:

[0134] The filtering module 110 is used to perform corrosion and dilation operations on the acquired original metallographic image to obtain a pre-processed filtered image with impurities and noise removed.

[0135] The grain detection module 120 is used to input the preprocessed filtered image into a grain detection model containing an EfficientNet backbone network and a bidirectional feature pyramid network, and to perform feature extraction and target recognition through the model to generate a set of detection boxes containing grain positions and categories.

[0136] The contour recognition module 130 is used to perform image segmentation on the corresponding region in the preprocessed image based on the detection box set, extract the contour of each grain, and form a grain contour set.

[0137] The effective screening module 140 is used to sequentially perform filtering processing on the set of grain contours, including grain area screening, multi-feature fusion scoring and density clustering, to remove invalid grains and pseudo grains and obtain an effective set of grains.

[0138] The pass / fail judgment module 150 is used to judge the pass / fail status based on the size information of each grain in the effective grain set and the judgment threshold preset by the user, and generate a test report containing statistical results and judgment conclusions.

[0139] As can be seen from the above technical solutions, the grain detection and analysis method and related equipment provided in this application first perform corrosion and dilation operations on the original metallographic image to obtain a preprocessed filtered image, and then input it into a detection model containing an EfficientNet backbone network and a bidirectional feature pyramid network. After feature extraction and target recognition, a set of detection boxes is generated. Subsequently, the corresponding region is segmented to obtain a set of grain contours. Through area screening, multi-feature fusion scoring and density clustering filtering, an effective set of grains is obtained. Finally, the passability is determined by combining the grain size and a preset threshold, and a detection report is generated.

[0140] This application addresses the shortcomings of existing technologies in several ways: Firstly, it reduces computational complexity and resource consumption. Preprocessing removes noise and impurities through erosion and dilation algorithms, reducing the amount of data and interference in subsequent model processing. The model employs an efficient EfficientNet backbone network and a bidirectional feature pyramid network, eliminating complex modules such as query set self-reconstruction, significantly reducing computational load and minimizing hardware usage when processing large-scale data, thus shortening detection and training time. Secondly, it enhances generalization ability. Preprocessing purifies input features, reducing interference from different imaging conditions and material types. Furthermore, the simplified EfficientNet backbone network, combined with the multi-scale fusion capability of the bidirectional feature pyramid, allows for adaptation to fine-grained detection in different fields without relying on specific datasets for optimization, resulting in significantly improved generalization performance. Thirdly, it reduces data dependency. The three-step filtering accurately removes invalid and pseudo-crystals. Even with small or low-quality training datasets, preprocessing and postprocessing can collaboratively compensate for data deficiencies, ensuring detection accuracy and significantly reducing reliance on high-quality, large-scale datasets, thereby improving the model's robustness for industrial applications.

[0141] Optionally, both the corrosion operation and the expansion operation use a core with a 5×5 structural element;

[0142] The calculation formulas for the corrosion operation and the expansion operation are as follows:

[0143]

[0144]

[0145] Where I is the input image and K is the structuring element. These are the pixel coordinates within the structuring element. To output the image after the erosion operation, This is the output image after the dilation operation.

[0146] Optionally, the grain detection model includes:

[0147] The EfficientNet backbone network is used to extract multi-scale features from the input preprocessed and filtered image and output feature maps at multiple levels.

[0148] A bidirectional feature pyramid network is used to perform bidirectional weighted fusion of feature maps from multiple levels, and output an enhanced multi-scale feature map.

[0149] The detection box prediction network, based on the enhanced multi-scale feature map, regresses and generates the localization detection box of the grain;

[0150] A category prediction network predicts the category of the target within the localization detection box based on the enhanced multi-scale feature map.

[0151] Optionally, based on the detection box set, image segmentation is performed on the corresponding regions in the preprocessed image to extract the contour of each grain, forming a grain contour set, including:

[0152] Convert the image region corresponding to the set of detection boxes into a grayscale image;

[0153] The grayscale image is binarized to obtain a binary image;

[0154] The contours of connected regions are extracted from the binary image to form the set of grain contours.

[0155] Optionally, the grain area screening method includes:

[0156] Calculate the mean area and standard deviation of the area of ​​all grains in the grain profile set;

[0157] Based on the mean area and the standard deviation of the area, the upper limit and lower limit of the effective area are determined to form the effective area range of the grain;

[0158] Select and retain grains in the set of grain profiles whose grain area falls within the effective area range of the grains;

[0159] The formulas for determining the upper limit and lower limit of the effective area are as follows:

[0160]

[0161]

[0162] in, This is the upper limit of the effective area. This is the lower limit of the effective area. The average area of ​​all grains. The area standard deviation of all grains. This is an adjustable coefficient.

[0163] Optionally, the scoring calculation formula used in the multi-feature fusion scoring method is as follows:

[0164]

[0165]

[0166]

[0167] in, For multi-feature fusion scoring, For grain area, For roundness, For compactness, For the perimeter, It is a linear normalized function. These are the corresponding weighting coefficients.

[0168] Optionally, the processing steps of the density clustering method include:

[0169] For each grain in the set of grain profiles, the number of other grain center coordinates within the neighborhood radius is calculated based on its center coordinates.

[0170] If the number is less than the preset minimum number of neighborhood points, the grain is determined to be a pseudo grain and is removed.

[0171] The grain detection and analysis device provided in this application embodiment can be applied to grain detection and analysis equipment. Figure 5 The hardware structure block diagram of the grain detection and analysis equipment is shown below. Figure 5 The hardware structure of the grain detection and analysis equipment may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;

[0172] In this embodiment of the application, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4;

[0173] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0174] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0175] The memory stores a program, which the processor can call. The program is used for:

[0176] Etching and dilation operations are performed on the acquired raw metallographic image to obtain a pre-processed filtered image with impurities and noise removed.

[0177] The preprocessed and filtered image is input into a grain detection model that includes an EfficientNet backbone network and a bidirectional feature pyramid network. The model performs feature extraction and target recognition to generate a set of detection boxes that include grain location and category.

[0178] Based on the detection box set, the corresponding region in the preprocessed image is segmented to extract the contour of each grain and form a grain contour set.

[0179] The set of grain contours is sequentially filtered using a grain area screening method, a multi-feature fusion scoring method, and a density clustering method to remove invalid grains and pseudo grains, thereby obtaining a set of valid grains.

[0180] Based on the size information of each grain in the effective grain set, and combined with the user-preset judgment threshold, a pass / fail determination is made, and a test report containing statistical results and judgment conclusions is generated.

[0181] Optionally, the refined and extended functions of the program can be referred to the above description.

[0182] This application embodiment also provides a readable storage medium that can store a program suitable for execution by a processor, the program being used for:

[0183] Etching and dilation operations are performed on the acquired raw metallographic image to obtain a pre-processed filtered image with impurities and noise removed.

[0184] The preprocessed and filtered image is input into a grain detection model that includes an EfficientNet backbone network and a bidirectional feature pyramid network. The model performs feature extraction and target recognition to generate a set of detection boxes that include grain location and category.

[0185] Based on the detection box set, the corresponding region in the preprocessed image is segmented to extract the contour of each grain and form a grain contour set.

[0186] The set of grain contours is sequentially filtered using a grain area screening method, a multi-feature fusion scoring method, and a density clustering method to remove invalid grains and pseudo grains, thereby obtaining a set of valid grains.

[0187] Based on the size information of each grain in the effective grain set, and combined with the user-preset judgment threshold, a pass / fail determination is made, and a test report containing statistical results and judgment conclusions is generated.

[0188] Optionally, the refined and extended functions of the program can be referred to the above description.

[0189] This application also provides a computer program product, including a computer program, wherein the computer program is executed by a processor using the following method:

[0190] Etching and dilation operations are performed on the acquired raw metallographic image to obtain a pre-processed filtered image with impurities and noise removed.

[0191] The preprocessed and filtered image is input into a grain detection model that includes an EfficientNet backbone network and a bidirectional feature pyramid network. The model performs feature extraction and target recognition to generate a set of detection boxes that include grain location and category.

[0192] Based on the detection box set, the corresponding region in the preprocessed image is segmented to extract the contour of each grain and form a grain contour set.

[0193] The set of grain contours is sequentially filtered using a grain area screening method, a multi-feature fusion scoring method, and a density clustering method to remove invalid grains and pseudo grains, thereby obtaining a set of valid grains.

[0194] Based on the size information of each grain in the effective grain set, and combined with the user-preset judgment threshold, a pass / fail determination is made, and a test report containing statistical results and judgment conclusions is generated.

[0195] Optionally, the refined and extended functions of the program can be referred to the above description.

[0196] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0197] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0198] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for grain detection and analysis, characterized in that, include: Etching and dilation operations are performed on the acquired raw metallographic image to obtain a pre-processed filtered image with impurities and noise removed. The preprocessed and filtered image is input into a grain detection model that includes an EfficientNet backbone network and a bidirectional feature pyramid network. The model performs feature extraction and target recognition to generate a set of detection boxes that include grain location and category. Based on the detection box set, the corresponding region in the preprocessed image is segmented to extract the contour of each grain and form a grain contour set. The set of grain contours is sequentially filtered using a grain area screening method, a multi-feature fusion scoring method, and a density clustering method to remove invalid grains and pseudo grains, thereby obtaining a set of valid grains. Based on the size information of each grain in the effective grain set, and combined with the user-preset judgment threshold, a pass / fail determination is made, and a test report containing statistical results and judgment conclusions is generated.

2. The method according to claim 1, characterized in that, Both the corrosion operation and the expansion operation use a core with a structural element of 5×5; The calculation formulas for the corrosion operation and the expansion operation are as follows: Where I is the input image and K is the structuring element. These are the pixel coordinates within the structuring element. To output the image after the erosion operation, This is the output image after the dilation operation.

3. The method according to claim 1, characterized in that, The grain detection model includes: The EfficientNet backbone network is used to extract multi-scale features from the input preprocessed and filtered image and output feature maps at multiple levels. A bidirectional feature pyramid network is used to perform bidirectional weighted fusion of feature maps from multiple levels, and output an enhanced multi-scale feature map. The detection box prediction network, based on the enhanced multi-scale feature map, regresses and generates the localization detection box of the grain; A category prediction network predicts the category of the target within the localization detection box based on the enhanced multi-scale feature map.

4. The method according to claim 1, characterized in that, Based on the detection box set, image segmentation is performed on the corresponding regions in the preprocessed image to extract the contour of each grain, forming a grain contour set, including: Convert the image region corresponding to the set of detection boxes into a grayscale image; The grayscale image is binarized to obtain a binary image; The contours of connected regions are extracted from the binary image to form the set of grain contours.

5. The method according to claim 1, characterized in that, The grain area screening method includes: Calculate the mean area and standard deviation of the area of ​​all grains in the grain profile set; Based on the mean area and the standard deviation of the area, the upper limit and lower limit of the effective area are determined to form the effective area range of the grain; Select and retain grains in the set of grain profiles whose grain area falls within the effective area range of the grains; The formulas for determining the upper limit and lower limit of the effective area are as follows: in, This is the upper limit of the effective area. This is the lower limit of the effective area. The average area of ​​all grains. The area standard deviation of all grains. This is an adjustable coefficient.

6. The method according to claim 1, characterized in that, The scoring formula used in the multi-feature fusion scoring method is as follows: in, For multi-feature fusion scoring, For grain area, For roundness, For compactness, For the perimeter, It is a linear normalized function. These are the corresponding weighting coefficients.

7. The method according to claim 1, characterized in that, The processing steps of the density clustering method include: For each grain in the set of grain profiles, the number of other grain center coordinates within the neighborhood radius is calculated based on its center coordinates. If the number is less than the preset minimum number of neighborhood points, the grain is determined to be a pseudo grain and is removed.

8. A grain detection and analysis device, characterized in that, include: The filtering module is used to perform erosion and dilation operations on the acquired raw metallographic image to obtain a pre-processed filtered image with impurities and noise removed. The grain detection module is used to input the preprocessed filtered image into a grain detection model containing an EfficientNet backbone network and a bidirectional feature pyramid network. The model performs feature extraction and target recognition to generate a set of detection boxes containing grain locations and categories. The contour recognition module is used to perform image segmentation on the corresponding region in the preprocessed image based on the detection box set, extract the contour of each grain, and form a grain contour set. The effective screening module is used to sequentially perform filtering processing on the set of grain contours, including grain area screening, multi-feature fusion scoring and density clustering, to remove invalid grains and pseudo grains and obtain an effective set of grains. The pass / fail determination module is used to determine the pass / fail status based on the size information of each grain in the effective grain set and the user-preset determination threshold, and generate a test report containing statistical results and determination conclusions.

9. A grain detection and analysis device, characterized in that, Including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the grain detection and analysis method as described in any one of claims 1-7.

10. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the grain detection and analysis method as described in any one of claims 1-7.

Images