A method, apparatus, device, and medium for tracing a metal microstructure defect

Abstract

The application discloses a metal microstructure defect tracing method and device, equipment and medium, and relates to the technical field of computer vision, and comprises the following steps: determining the size relationship between the native output dimension of a target self-supervised visual model and the target dimension; inputting a pretreated historical metal microstructure defect image into the target self-supervised visual model to obtain an initial feature vector, determining whether to construct a linear projection layer based on the size relationship, and if yes, writing a first feature vector, metadata and a model version number determined based on the linear projection layer and the initial feature vector into a preset vector database; determining a second feature vector of a metal microstructure defect image to be detected; arranging candidate feature vectors according to the similarity between the second feature vector and the candidate feature vectors, and obtaining and outputting a target historical metal microstructure defect image and target metadata according to the image ID of the sorted candidate feature vectors. High-quality metal microstructure image features can be extracted without fine tuning.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method, apparatus, equipment and medium for tracing the source of defects in the microstructure of metals. Background Technology

[0002] In the production and processing of metal materials, metallurgical microscopy (JMS) and scanning electron microscopy (SEM) are core tools for detecting microstructural defects in metals. When defects are discovered, engineers need to trace their causes by comparing historical defect images. With the popularization of digital inspection, image retrieval technology based on computer vision has been introduced into this field, but existing technologies have the following shortcomings: 1. Supervised learning models lack the ability to represent features in images of metal microstructures; 2. The application of self-supervised visual models in the field of industrial inspection faces adaptation challenges: difficulty in selecting model specifications, incompatibility of feature vector dimensions, inconsistency in feature spaces of different models leading to retrieval distortion, and the effectiveness of self-supervised model features in the field of metal microstructure has not been verified. 3. The search results lack intelligent analysis methods; 4. The defect tracing dimensions are too limited.

[0003] Therefore, existing technologies lack a method and system that can efficiently extract the image features of metal microstructures without fine-tuning, adaptively select model specifications according to hardware conditions, solve the compatibility and isolation problems of multi-model feature vectors, intelligently analyze the retrieval results, and realize multi-dimensional correlation and tracing. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for tracing the source of defects in metal microstructures. This method can efficiently extract image features of metal microstructures without fine-tuning, adaptively select model specifications based on hardware conditions, and solve the problems of incompatibility between feature vector dimensions and inconsistencies in feature space among multiple models without rebuilding a vector database. It also achieves accurate and reliable defect retrieval and source tracing. The specific solution is as follows: In a first aspect, this application discloses a method for tracing the source of defects in the microstructure of metals, including: The target self-supervised visual model is determined based on the available memory on the server, and the relationship between the native output dimension of the target self-supervised visual model and the target dimension is determined. The preprocessed historical metal microstructure defect image is input into the target self-supervised visual model to obtain an initial feature vector. Based on the size relationship, it is determined whether to construct a linear projection layer without bias terms. If constructed, a first feature vector is determined based on the linear projection layer and the initial feature vector. Write the first feature vector, the metadata of the historical metal microstructure defect image, and the version number of the target self-supervised visual model into a preset vector database. A second feature vector corresponding to the microstructural defect image of the metal to be detected is determined, a target filtering condition including model version filtering conditions is constructed, and candidate feature vectors in the preset vector database are filtered based on the target filtering conditions; the model version filtering conditions include model version equal to the version number of the current model or model version being empty; Determine the similarity between the second feature vector and the candidate feature vector, and sort the candidate feature vectors in descending order according to the similarity to obtain the sorted candidate feature vectors; Based on the image ID corresponding to the sorted candidate feature vector, the target historical metal microstructure defect image and the corresponding target metadata are obtained from the preset vector database. The target historical metal microstructure defect image and the target metadata are determined as the retrieval results and output.

[0005] Optionally, determining the target self-supervised visual model based on available server memory includes: Based on the priority of feature extraction accuracy from high to low, several self-supervised visual models of different specifications are traversed in turn to determine whether the available memory of the server meets the minimum memory requirements of the target specification of the self-supervised visual model. If the conditions are met, the self-supervised visual model of the target specification is loaded, and the successfully loaded self-supervised visual model of the target specification is determined as the target self-supervised visual model. If the target specification is not met, skip the self-supervised visual model and continue iterating. Accordingly, the method further includes: If all self-supervised vision models fail to load, then load the pre-trained convolutional neural network model and remove the fully connected layers of the pre-trained convolutional neural network model. The output of the average pooling layer of the pre-trained convolutional neural network model is determined as the original feature vector, and the original feature vector is L2 normalized to obtain the first feature vector. Wherein, the output dimension of the pre-trained convolutional neural network model is equal to the target dimension, and the linear projection layer is not constructed.

[0006] Optionally, determining whether to construct a linear projection layer without a bias term based on the size relationship includes: If the native output dimension is not equal to the target dimension, a linear projection layer without bias terms is constructed; wherein, the weight matrix of the linear projection layer is initialized based on the Xavier normal distribution initialization method so that each element of the weight matrix satisfies a normal distribution; If the native output dimension is equal to the target dimension, then a linear projection layer without bias terms is not constructed, and the initial feature vector is L2 normalized to obtain the first feature vector. Accordingly, determining the first feature vector based on the linear projection layer and the initial feature vector includes: The initial feature vector is input into the linear projection layer to obtain the projected feature vector of the target dimension; The projected feature vector is L2 normalized to obtain the first feature vector.

[0007] Optionally, before inputting the preprocessed historical metal microstructure defect image into the target self-supervised visual model, the method further includes: The historical metal microstructure defect image is scaled to a preset pixel size to obtain a scaled image; If the scaled image is a grayscale image, then the scaled image is converted into an RGB three-channel image by channel copying to obtain the image to be standardized; If the scaled image is an RGB three-channel image, then the scaled image is determined as the image to be standardized; Based on the preset mean and preset standard deviation, channel standardization is performed on each channel of the image to be standardized to obtain a preprocessed historical metal microstructure defect image. Accordingly, the step of inputting the preprocessed historical metal microstructure defect image into the target self-supervised visual model to obtain an initial feature vector includes: The preprocessed historical metal microstructure defect image is input into the target self-supervised vision model based on the Vision Transformer architecture, and the CLS token feature vector is extracted as the initial feature vector; the CLS token feature vector is the classification label vector used in Vision Transformer to aggregate global image information.

[0008] Optionally, determining the similarity between the second feature vector and the candidate feature vector includes: The similarity is determined by the inner product of the second feature vector and the candidate feature vector. Accordingly, the method further includes: Based on the similarity sequence corresponding to the sorted candidate feature vectors, calculate the difference between two adjacent similarities; The target threshold is determined based on the standard deviation of the similarity sequences; The positions where the difference is greater than or equal to the target threshold are defined as significant gaps; The search results are divided into a first probability group, a second probability group, and a third probability group based on the number and location of the salience gaps; wherein the first probability is greater than the second probability; and the second probability is greater than the third probability.

[0009] Optionally, the target metadata includes material composition information, defect type labels, and attribution to a spectral library; the material composition information, the defect type labels, and the attribution to the spectral library are associated with the target historical metal microstructure defect image through the image ID; Accordingly, the method further includes: The search results are grouped based on defect labels, attribution to an image library, and filename prefixes to obtain multiple defect clusters. Specifically, when grouping by defect label dimension, search results with the same defect label are classified into one cluster; when grouping by attribution to an image library dimension, search results belonging to the same image library are classified into one cluster; when grouping by filename prefix dimension, the letter or Chinese character prefix is ​​extracted from the original filename of each historical metal microstructure defect image in the storage system, and search results with the same prefix are classified into one cluster. Calculate the number of samples for each defect cluster; Based on the sample quantity sorted in descending order, the first preset number of defect clusters are determined as the defect type clustering results and output.

[0010] Optionally, the method further includes: If there are multiple images of the microstructure defects of the metal to be detected, then determine the search result set corresponding to each image of the microstructure defects of the metal to be detected. The search result sets are aggregated according to the target image dimension, and the average similarity, maximum similarity, minimum similarity, number of matches, and matching rate of each aggregated target historical metal microstructure defect image are calculated; the matching rate is the ratio of the number of matches to the total number of metal microstructure defect images to be detected; The aggregation results are sorted based on the descending order of the number of matches and the descending order of the average similarity to obtain the sorted aggregation results; Generate a similarity matrix with the images of metal microstructure defects to be detected as rows and the images of target historical metal microstructure defects as columns; wherein, the elements in the similarity matrix are the similarity between the corresponding images of metal microstructure defects to be detected and the corresponding images of target historical metal microstructure defects; if the search result set of any image of metal microstructure defects to be detected does not contain any image of target historical metal microstructure defects, then the corresponding element in the similarity matrix is ​​set to zero; The similarity matrix and the sorted aggregation results are used to determine the batch retrieval results and output them.

[0011] Secondly, this application discloses a device for tracing the source of defects in the microstructure of metals, comprising: The size relationship determination module is used to determine the target self-supervised visual model based on the available memory of the server, and to determine the size relationship between the native output dimension of the target self-supervised visual model and the target dimension. The first feature vector determination module is used to input the preprocessed historical metal microstructure defect image into the target self-supervised visual model to obtain an initial feature vector, and determine whether to construct a linear projection layer without bias terms based on the size relationship. If constructed, the first feature vector is determined based on the linear projection layer and the initial feature vector. The writing module is used to write the first feature vector, the metadata of the historical metal microstructure defect image, and the version number of the target self-supervised visual model into a preset vector database. The candidate feature vector filtering module is used to determine the second feature vector corresponding to the image of the metal microstructure defect to be detected, construct a target filtering condition that includes model version filtering conditions, and filter candidate feature vectors in the preset vector database based on the target filtering conditions; the model version filtering conditions include the model version being equal to the version number of the current model or the model version being empty; The sorting module is used to determine the similarity between the second feature vector and the candidate feature vector, and to sort the candidate feature vectors in descending order according to the similarity, so as to obtain the sorted candidate feature vectors. The retrieval result determination module is used to obtain the target historical metal microstructure defect image and the corresponding target metadata corresponding to the image ID from the preset vector database according to the image ID corresponding to the sorted candidate feature vector, and determine the target historical metal microstructure defect image and the target metadata as the retrieval result and output them.

[0012] Thirdly, this application discloses an electronic device, including: Memory, used to store computer programs; A processor is used to execute computer programs to implement the steps of the aforementioned method for tracing the source of defects in the metal microstructure.

[0013] Fourthly, this application discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the aforementioned method for tracing the source of defects in metal microstructures.

[0014] This application first determines the target self-supervised visual model based on the available memory on the server, and determines the size relationship between the native output dimension and the target dimension of the target self-supervised visual model; it then inputs a preprocessed historical metal microstructure defect image into the target self-supervised visual model to obtain an initial feature vector; based on the size relationship, it determines whether to construct a linear projection layer without a bias term; if so, it determines a first feature vector based on the linear projection layer and the initial feature vector; it then writes the first feature vector, the metadata of the historical metal microstructure defect image, and the version number of the target self-supervised visual model into a preset vector database; finally, it determines the second feature vector corresponding to the metal microstructure defect image to be detected, and constructs a model containing... The target filtering conditions for version filtering are used to filter candidate feature vectors in the preset vector database. The model version filtering conditions include the model version being equal to the current model version number or the model version being empty. The similarity between the second feature vector and the candidate feature vectors is determined, and the candidate feature vectors are sorted in descending order according to the similarity from high to low to obtain sorted candidate feature vectors. Based on the image ID corresponding to the sorted candidate feature vectors, the target historical metal microstructure defect image and the corresponding target metadata corresponding to the image ID are obtained from the preset vector database, and the target historical metal microstructure defect image and the target metadata are determined as the retrieval results and output. As can be seen, this application selects the target self-supervised visual model by dynamically adapting to server memory, and flexibly constructs a linear projection layer without bias terms based on the relationship between the model's native output dimension and the target dimension. This reduces computational overhead and memory usage while ensuring the accuracy of feature representation. Simultaneously, it structurally stores historical image features, metadata, and model version numbers in a vector database, and introduces model version filtering conditions (exactly matching the current version or compatible with empty versions) during retrieval. This effectively avoids mismatches caused by inconsistencies in the feature spaces of different model versions, significantly improving the accuracy and stability of similar defect retrieval in cross-version scenarios. Finally, through similarity ranking and metadata backtracking of associated image IDs, it achieves efficient, accurate, and interpretable historical case recommendations for metal microstructure defects, providing a low-resource-consumption, highly robust intelligent retrieval solution for practical quality inspection scenarios. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0016] Figure 1 This is a flowchart of a method for tracing the source of defects in the microstructure of metals disclosed in this application; Figure 2 This is a schematic diagram of a metal microstructure defect tracing device disclosed in this application. Figure 3 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] In the production and processing of metal materials, metallurgical microscopes and scanning electron microscopes are core tools for detecting defects in the microstructure of metals. When defects are found, engineers need to trace the cause of the defects by comparing historical defect images. Existing technologies, image retrieval schemes based on supervised pre-training convolutional neural networks on natural image datasets lack sufficient feature representation capabilities for metal microstructure images. Directly applying self-supervised visual models to industrial inspection faces a series of technical challenges, including difficulties in selecting model specifications, incompatibility of feature vector dimensions, inconsistencies in feature spaces between different models leading to retrieval distortion, a lack of intelligent analysis methods for retrieval results, and a single dimension for defect tracing. Therefore, this application provides a method, apparatus, device, and medium for tracing the source of metal microstructure defects. This method can efficiently extract features from metal microstructure images without fine-tuning, adaptively select model specifications based on hardware conditions, solve the problems of incompatibility of feature vector dimensions and inconsistencies in feature spaces among multiple models without rebuilding the vector database, and achieve accurate and reliable defect retrieval and tracing.

[0019] See Figure 1 As shown in the figure, an embodiment of the present invention discloses a method for tracing the source of defects in the microstructure of metals, including: Step S11: Determine the target self-supervised visual model based on the available memory of the server, and determine the size relationship between the native output dimension of the target self-supervised visual model and the target dimension.

[0020] In this embodiment, considering the varying server hardware conditions in industrial settings, the system first detects the available server memory upon startup, and then adaptively determines the target self-supervised visual model based on the available memory. In a specific implementation, the current available server memory can be obtained through system interfaces provided by the operating system or third-party libraries.

[0021] After obtaining the available server memory, several self-supervised visual models of different specifications are traversed sequentially based on feature extraction accuracy from high to low priority. It is determined whether the available server memory meets the minimum memory requirement of the target specification self-supervised visual model. If it does, the target specification self-supervised visual model is loaded, and the successfully loaded target specification self-supervised visual model is identified as the target self-supervised visual model. If it does not meet the requirement, the target specification self-supervised visual model is skipped, and the traversal continues. For example, the self-supervised visual model can be the self-supervised visual Transformer model DinoV2 based on a self-distillation mechanism. Correspondingly, the model candidate list, ordered from high to low feature extraction accuracy, can be defined as [(dinov2_vitl14,1024,8),(dinov2_vitb14,768,4),(dinov2_vits14,384,2)], where the elements of the tuple are the model name, the native output dimension, and the minimum memory requirement (in GB). In other words: the first priority is dinov2_vitl14 (ViT-Large specification, approximately 300M parameters, native output dimension of 1024), which requires more than or equal to 8GB of available memory to provide the highest precision feature extraction; the second priority is dinov2_vitb14 (ViT-Base specification, approximately 86M parameters, native output dimension of 768), which requires more than or equal to 4GB of available memory to achieve a balance between accuracy and speed; the third priority is dinov2_vits14 (ViT-Small specification, approximately 22M parameters, native output dimension of 384), which requires more than or equal to 2GB of available memory, has the lowest resource requirements, and ensures that it can still run in low-configuration environments.

[0022] During the process of traversing the candidate model list, for each model specification, it is first determined whether the currently available memory meets the minimum memory requirements of the model: if not, the model is skipped and the next specification is tried; if it meets the requirements, the pre-trained weights of the model are loaded and set to evaluation mode, and the traversal stops when the loading is successful for the first time.

[0023] In one specific implementation, during model loading, pre-trained weights are first read from the local cache directory. If the local cache does not exist, the model is downloaded from the remote model repository and cached locally. The strategy for determining the local cache directory can be as follows: first, check the `.torch / hub / checkpoints` path in the user's home directory; if this path does not exist, check the system temporary directory; if neither exists, download the model from the remote model repository and cache it in the `torch` cache path in the user's home directory. This eliminates the need to repeatedly download model weights upon system restart, significantly improving startup speed. Furthermore, in industrial intranet environments with limited network access, model loading can be completed as long as the local cache exists, improving system availability in offline environments.

[0024] It should be noted that potential failures during model loading include: network anomalies, i.e., the remote model repository is unreachable and there is no local cache; and corrupted model files, i.e., the locally cached model files are incomplete or have incorrect formats. For each failure scenario, record the detailed cause of failure and log information, and then continue to try the next model specification so that operations and maintenance personnel can troubleshoot the problem afterward.

[0025] Furthermore, if all self-supervised vision models fail to load, a pre-trained convolutional neural network (CNN) model is loaded, and the fully connected layers of the pre-trained CNN model are removed. The output of the average pooling layer of the pre-trained CNN model is determined as the original feature vector, and the original feature vector is L2 normalized to obtain the first feature vector. The output dimension of the pre-trained CNN model is equal to the target dimension, and the linear projection layer is not constructed. For example, the pre-trained CNN model can be a ResNet50. Pre-trained weights are loaded using `torchvision.models.resnet50(pretrained=True)`, and after removing the last fully connected layer, the output of its average pooling layer is exactly 2048 dimensions, equal to the target dimension D=2048. The advantages of ResNet50 as a fallback solution are: First, the model size is small (approximately 25M parameters), with low hardware requirements, and it can be loaded and run even on servers with very low configurations; Second, the output dimension is exactly 2048 dimensions, eliminating the need for additional linear projection layers and simplifying the implementation complexity of the fallback path; Third, ResNet50 has high stability in the PyTorch ecosystem, with an extremely low probability of loading failure, making it a reliable fallback solution.

[0026] After determining the target self-supervised vision model (or the pre-trained convolutional neural network model after rollback), the successfully loaded model identifier is recorded as the model version number. For example, if a DinoV2 model is loaded, the model version number can be the model name, such as "dinov2_vitl14", "dinov2_vitb14", or "dinov2_vits14"; if rolling back to ResNet50, the model version number can be "resnet50". This model version number will be used for feature vector version management in subsequent vector storage and vector retrieval.

[0027] In this way, since the traversal order is arranged from high to low feature extraction accuracy, and the system only tries the next specification when the available memory does not meet the minimum requirements or loading fails, the target self-supervised visual model determined by the system is necessarily the model with the highest accuracy that can be successfully loaded under the current hardware conditions, thus achieving the optimal match between hardware resources and feature accuracy. Furthermore, due to the existence of the fallback path of the pre-trained convolutional neural network model, even if all self-supervised visual models are unavailable (e.g., low-configuration servers or model repositories are unreachable), the system can still extract feature vectors consistent with the target dimension to maintain basic retrieval functions.

[0028] After determining the target self-supervised visual model, the relationship between the native output dimension *d* and the target dimension *D* of the target self-supervised visual model is further determined, i.e., whether the native output dimension *d* is equal to the target dimension *D*. Here, the target dimension *D* is a fixed feature vector dimension required by a pre-defined vector database. For example, the target dimension *D* can be 2048 dimensions, consistent with the output dimension of the ResNet50 average pooling layer. This ensures that the historical feature vectors extracted by the pre-trained convolutional neural network model and the feature vectors extracted and projected by the self-supervised visual model maintain dimensional consistency, laying the foundation for the coexistence of feature vectors from multiple models in the same vector database. The size relationship is used in subsequent steps to determine whether a linear projection layer needs to be built: taking DinoV2 as an example, the native output dimension of dinov2_vitl14 is 1024 dimensions, dinov2_vitb14 is 768 dimensions, and dinov2_vits14 is 384 dimensions, none of which are equal to the target dimension of 2048 dimensions, so a linear projection layer needs to be built; while the output dimension of ResNet50 under the fallback path is exactly 2048 dimensions, which is equal to the target dimension, so no linear projection layer needs to be built.

[0029] Step S12: Input the preprocessed historical metal microstructure defect image into the target self-supervised visual model to obtain an initial feature vector. Based on the size relationship, determine whether to construct a linear projection layer without bias terms. If constructed, determine the first feature vector based on the linear projection layer and the initial feature vector.

[0030] In this embodiment, before inputting the historical metal microstructure defect image into the target self-supervised visual model, the image needs to be preprocessed, specifically including: scaling the historical metal microstructure defect image to a preset pixel size to obtain a scaled image; if the scaled image is a grayscale image, converting it into an RGB three-channel image through channel copying to obtain an image to be standardized; if the scaled image is an RGB three-channel image, determining it as the image to be standardized; and performing channel standardization on each channel of the image to be standardized based on a preset mean and a preset standard deviation to obtain a preprocessed historical metal microstructure defect image.

[0031] For example, the preset pixel size can be 224×224 pixels, and bilinear interpolation can be used during scaling. Since the metal microstructure images acquired by metallographic microscopes and scanning electron microscopes are mostly single-channel grayscale images, while the input of the pre-trained visual model is usually a three-channel image, the grayscale image is converted into an RGB three-channel image by copying the single channel three times. During channel standardization, the pixel values ​​are first normalized from [0, 255] to [0, 1], and then each channel is standardized according to the preset mean μ=[0.485, 0.456, 0.406] and the preset standard deviation σ=[0.229, 0.224, 0.225] (i.e., the statistical mean and standard deviation of the ImageNet dataset) to obtain the preprocessed image tensor. Using the same standardization parameters as in the model pre-training stage can align the pixel distribution of the input image with the input distribution during model pre-training, thereby ensuring that the feature extraction capability of the pre-trained model is fully utilized.

[0032] It should be noted that in other embodiments, the preset pixel size is not limited to 224×224 pixels, and can also be set to other resolutions divisible by the selected model's patch size, such as 336×336 pixels, 518×518 pixels, etc. Higher input resolution is beneficial for preserving finer texture information in the metal microstructure, but the computational load increases accordingly. A compromise can be made based on the server's computing power and retrieval latency requirements. The scaling interpolation method can also use other interpolation methods such as bicubic interpolation and nearest neighbor interpolation. In addition to using ImageNet statistical values, the preset mean and preset standard deviation can also use the mean and standard deviation obtained statistically from a metal microstructure image dataset. This application does not limit these aspects.

[0033] After preprocessing, the preprocessed historical metal microstructure defect image is input into the target self-supervised visual model based on the VisionTransformer architecture, and the CLS token feature vector is extracted as the initial feature vector; the CLS token feature vector is the classification label vector used in VisionTransformer to aggregate global image information. Specifically, taking DinoV2 as an example, the preprocessed image tensor x∈R... 1×3×224×224 The input model, DinoV2, is based on the Vision Transformer architecture. It segments a 224×224 pixel image into 14×14 pixel patches, generating (224 / 14)×(224 / 14)=16×16=256 patches, forming a patch sequence. This sequence, along with an additional CLS token, is input into a multi-layer Transformer encoder for processing. Through self-attention, the CLS token interacts fully with all patch tokens, incorporating information from all patches. Therefore, the CLS token contains the global semantic features of the image. The initial feature vector v=model(x)∈R is extracted. 1×d , where d is the native output dimension of the model.

[0034] After obtaining the initial feature vector, a decision is made based on the size relationship to determine whether to construct a linear projection layer without bias terms: if the native output dimension d is not equal to the target dimension D, then a linear projection layer without bias terms is constructed, the initial feature vector is input into the linear projection layer to obtain the projected feature vector of the target dimension, and then the projected feature vector is L2 normalized to obtain the first feature vector; if the native output dimension d is equal to the target dimension D, then a linear projection layer without bias terms is not constructed, and the initial feature vector is directly L2 normalized to obtain the first feature vector.

[0035] In one specific implementation, the linear projection layer can be constructed as `projection=nn.Linear(d,D,bias=False)`, where `bias=False` indicates that the linear projection layer does not contain a bias term. The purpose of not containing a bias term is to avoid introducing additional translational degrees of freedom: since the feature vectors will subsequently undergo L2 normalization, the semantic information of the normalized feature vectors is carried by their orientation on the unit hypersphere. If a bias term is introduced, the same fixed offset component will be injected into all projected vectors, destroying the purely linear nature of the projection, perturbing the relative relationship between vector orientations, and thus affecting the effectiveness of the similarity measurement. Omitting the bias term reduces the number of parameters while ensuring that the projection is a purely linear mapping, thus preserving the relative geometric relationship between vectors.

[0036] Furthermore, the weight matrix W∈R of the linear projection layer is initialized based on the Xavier normal distribution method. D×d Initialization is performed so that each element of the weight matrix W follows an N(0, 2 / (d+D)) distribution. Xavier normal distribution initialization ensures the stability of the variance of the feature vectors before and after projection, avoiding feature distribution distortion or information loss caused by random initialization. The linear projection layer maintains unchanged weights during system operation without any training or fine-tuning.

[0037] In this way, this embodiment achieves smooth replacement of models of different dimensions without the need to rebuild the vector database or conduct joint training, thus balancing upgrade costs, business continuity, and retrieval accuracy.

[0038] In one specific implementation, the method for L2 normalizing the projected feature vector (or the initial feature vector when the native output dimension is exactly equal to the target dimension, or the original feature vector under the fallback path) is as follows: ,in For vectors The L2 norm. After normalization, the inner product of any two eigenvectors is equivalent to the cosine similarity, i.e. In this way, subsequent vector retrieval does not require dividing the inner product result by the vector magnitude. Similarity calculation can be performed directly using the highly optimized inner product index of the pre-defined vector database, balancing retrieval accuracy and efficiency.

[0039] Understandably, for the fallback path—where the self-supervised visual model fails to load all targets and a pre-trained convolutional neural network (CNN) model is used—the pre-processed image is input into the pre-trained CNN model with fully connected layers removed. The output of the average pooling layer is extracted as the original feature vector. In this case, the dimension of the original feature vector is exactly the target dimension D=2048. Without going through a linear projection layer, L2 normalization is performed directly to obtain the first feature vector. Therefore, the first feature vector output by both the self-supervised visual model path and the CNN fallback path is an L2-normalized feature vector of the target dimension. The two paths are completely unified in their output interface, and the downstream vector storage and retrieval logic does not need to be aware of the difference in feature extraction paths, reducing the complexity of the system implementation.

[0040] Step S13: Write the first feature vector, the metadata of the historical metal microstructure defect image, and the version number of the target self-supervised visual model into a preset vector database.

[0041] In this embodiment, a preset vector database is used to store feature vectors and their metadata of historical metal microstructure defect images, and supports scalar field filtering and vector similarity retrieval. For example, the preset vector database can be a Milvus vector database; in other embodiments, other databases supporting scalar filtering and vector retrieval, such as Faiss, Weaviate, Qdrant, Elasticsearch's dense vector retrieval function, or a PostgreSQL database with the pgvector extension installed, can also be used. This application does not limit this to any particular database.

[0042] In one specific implementation, upon system startup, the system first checks whether the target set already exists in the preset vector database. If it does not exist, a schema containing a model version field is created. The model version field identifies the extraction model corresponding to each feature vector, and its data type is a variable-length string with a maximum length of 64 characters. For example, the schema may contain fields as shown in Table 1: Table 1 Schema can contain fields

[0043] The model_version field is the model version field; the feature_vector field is used to store the floating-point feature vector of the target dimension (2048 dimensions), i.e., the first feature vector; the id field is the primary key, used to uniquely identify each historical metal microstructure defect image; the gallery field and gallery_name field are used to record the image's library affiliation, with each library typically corresponding to a batch, process, or material system; the tags field stores the image's defect type tags in comma-separated form; the url, file_name, original_name, mime_type, size, and created_at fields record the image's access address, storage file name, original file name, MIME type, file size (bytes), and creation timestamp, respectively. These fields together constitute the metadata of the historical metal microstructure defect image.

[0044] Furthermore, if the target collection already exists, the current schema's field list is obtained through the `describeCollection` API (Application Programming Interface), and each field is checked to see if it contains the `model_version` field. If it is missing, the `model_version` field (VarChar type, `max_length=64`) is automatically added through the `alterCollection` operation. Historical data uses this field as an empty string by default. The technical effect of this automatic compatibility mechanism can be deduced as follows: For vector databases upgraded from older systems without version management, their existing data does not contain a model version field. Manually modifying the schema and backfilling the data requires downtime and is prone to errors. However, this embodiment automatically detects and adds missing fields upon system startup. The model version field in historical data is an empty string by default. Combined with the compatibility filtering branch for "empty model version" during subsequent searches, old data can continue to be retrieved after the upgrade without any backfilling. Therefore, system upgrades do not require manual modification of the database schema or downtime for data migration, achieving a smooth upgrade with zero downtime.

[0045] After completing the schema preparation, the first feature vector, the metadata of the historical metal microstructure defect image, and the version number of the target self-supervised visual model are written into a preset vector database. When writing, the `model_version` field is set to the identifier of the current feature extraction model, such as "dinov2_vitb14" or "resnet50".

[0046] In one specific implementation, a delete-then-insert strategy is used during writing to ensure data consistency. This involves first deleting existing records (if any) by image ID, and then inserting new records, ensuring that only the latest model-extracted feature vector version is retained for the same image. The technical effect of this strategy can be derived as follows: Without the delete-then-insert strategy, if the same image is repeatedly entered into the database before and after a model upgrade, the preset vector database will contain both old and new versions of the same image's feature vector records. During retrieval, the same image may appear repeatedly in the results list, or the old model's features may be used for sorting, causing result confusion. With the delete-then-insert strategy, the feature vector of the same image in the database is always the unique version extracted by the latest model, ensuring the determinism and consistency of retrieval results from a data perspective.

[0047] It should be noted that in other implementations, if the vector database natively supports upsert (update insertion) operations based on primary keys, upsert operations can be used to replace the deletion and insertion strategy. Both can ensure that only the latest feature vector record is retained for the same image, and are equivalent implementations.

[0048] Step S14: Determine the second feature vector corresponding to the microstructure defect image of the metal to be detected, construct a target filtering condition that includes model version filtering conditions, and filter candidate feature vectors in the preset vector database based on the target filtering conditions; the model version filtering conditions include the model version being equal to the version number of the current model or the model version being empty.

[0049] In this embodiment, after receiving the image of the metal microstructure defect to be detected (i.e., the query image), the second feature vector corresponding to the image of the metal microstructure defect to be detected is determined according to the same preprocessing, feature extraction, linear projection (if necessary), and L2 normalization process as in step S12. Since the second feature vector and the first feature vector already in the database are extracted by the same target self-supervised visual model and have undergone the same linear projection layer and the same L2 normalization process, they are located in the same feature space, which is a prerequisite for the effectiveness of subsequent similarity measurement.

[0050] In one specific implementation, the target filter condition including model version filtering is constructed as follows: Let the current model version number be V (e.g., "dinov2_vitb14"), and the scalar filter condition of the graph library be G (optional condition, e.g., gallery=="gallery_001"). The construction rule for the target filter condition is: if the graph library filter condition G exists, then filter=G AND (model_version==V OR model_version==""); if the graph library filter condition G does not exist, then filter=model_version==V OR model_version=="". The branch `model_version==""` is used to ensure compatibility with historical data without version identifiers, guaranteeing that old data can still be retrieved after system upgrades. The logical meaning of the target filter condition is: among the candidate feature vectors participating in similarity calculation, only feature vectors extracted from the current model version or old feature vectors without version identifiers extracted before system upgrades are retained, while feature vectors extracted from other model versions are excluded.

[0051] It should be noted that in other embodiments, the model version filtering condition can be extended to a version whitelist, i.e., model_version IN {V1, V2, …}, to include multiple model versions that have been verified offline to be compatible in feature space within the search scope. Furthermore, in addition to the graph library filtering condition, other scalar filtering conditions such as defect label filtering and creation time range filtering can be superimposed to form more complex hybrid filtering conditions together with the model version filtering condition, further narrowing the candidate set and improving the targeting and efficiency of the search. All the above variations fall within the scope of protection of this application.

[0052] Step S15: Determine the similarity between the second feature vector and the candidate feature vector, and sort the candidate feature vectors in descending order according to the similarity to obtain the sorted candidate feature vectors.

[0053] In this embodiment, the inner product of the second feature vector and the candidate feature vector is determined as the similarity. In a specific implementation, based on the target filtering condition and the inner product (IP, Inner Product) distance metric, a vector search is performed in a preset vector database. For example, search(collection, data=[q], limit=TopK, filter=filter, metric_type=IP, output_fields=[…]) can be called, where q is the second feature vector, TopK is the number of most similar results returned (e.g., 50 or 100, which can be configured according to business needs), filter is the target filtering condition, and output_fields specifies the metadata fields that need to be returned along with the search results. Since both the second feature vector and the candidate feature vectors in the database are L2 normalized, the inner product is equivalent to cosine similarity, i.e., sim(q, v)=q·v=cos(q, v), and the similarity value ranges from [-1, 1], with a larger value indicating greater similarity. The candidate feature vectors are sorted in descending order of similarity from high to low, resulting in a similarity sequence s1≥s2≥…≥s n .

[0054] This embodiment can also perform significant gap detection and intelligent grouping based on the similarity sequence corresponding to the sorted candidate feature vectors, specifically including the following steps: First, the statistical characteristics of the similarity sequences are calculated, including the mean μ, standard deviation σ, median, and quantiles (P25, P50, P75, P90, P95). These statistical characteristics reflect the overall quality distribution of the search results. The mean and standard deviation are used for subsequent gap threshold calculation, while the quantiles are used as the grouping criteria when there are no significant gaps.

[0055] Secondly, based on the similarity sequence corresponding to the sorted candidate feature vectors, the difference between two adjacent similarities, i.e., gap, is calculated. i =s i -s (i+1) , i=1, 2, …, n-1. gap i This reflects the degree of similarity jump between the i-th and i+1-th results.

[0056] Next, the target threshold is determined based on the standard deviation of the similarity sequence. In one specific implementation, the target threshold is an adaptive threshold T = max(σ × 0.5, 0.03). The design principle of this adaptive threshold is as follows: when the standard deviation σ of the similarity distribution is large (indicating that the quality of the search results is uneven), the threshold is increased accordingly to avoid misjudging normal fluctuations as significant gaps; when the standard deviation σ is small (indicating that the quality of the search results is relatively uniform), the threshold is reduced to a minimum value of 0.03 to ensure that subtle but significant similarity jumps can still be identified. The selection of the coefficient 0.5 is based on the following considerations: under the assumption of a normal distribution, approximately 68% of the data falls within the range of μ ± σ, and the expected difference between adjacent results is approximately on the order of 0.5σ. Therefore, setting the coefficient to 0.5 can distinguish significant jumps beyond normal fluctuations. The selection of the minimum value 0.03 is based on the actual distribution characteristics of cosine similarity. In metal microstructure image retrieval, experience shows that similarity differences below 0.03 are usually measurement noise rather than true semantic differences. The difference gap is then... i The positions that are greater than or equal to the target threshold T are defined as the saliency gaps.

[0057] Finally, the retrieval results are divided into a first possibility group, a second possibility group and a third possibility group according to the number and positions of the saliency gaps, wherein the first possibility is greater than the second possibility, and the second possibility is greater than the third possibility, that is, the first possibility group, the second possibility group, and the third possibility group correspond to a high possibility group, a medium possibility group, and a low possibility group respectively. In a specific embodiment, the grouping rule is: if there is no saliency gap, it indicates that the similarity of the retrieval results presents a smooth decreasing distribution, and the grouping is performed according to quantiles at this time: when P75 ≥ 0.7 (0.7 is an empirical threshold, which indicates that high semantic correlation is considered to exist when the cosine similarity reaches above 0.7), all results are classified into the first possibility group; when P50 ≥ 0.5, results with similarity greater than or equal to P75 are classified into the first possibility group, results with similarity in the interval [P50, P75) are classified into the second possibility group, and results with similarity less than P50 are classified into the third possibility group; when P50 < 0.5, results with similarity greater than or equal to P75 are classified into the second possibility group, results with similarity less than P75 are classified into the third possibility group, and the first possibility group is not set. If there is one saliency gap (located at position k), the first k results are classified into the first possibility group; among the remaining results, those with similarity greater than or equal to 0.5 are classified into the second possibility group, and those with similarity less than 0.5 are classified into the third possibility group, where 0.5 is an empirical threshold, indicating that a certain degree of semantic correlation is considered to exist when the cosine similarity reaches above 0.5. If there are two or more saliency gaps (assuming the first two saliency gaps are located at positions k1 and k2 respectively, and k1 < k2), the first k1 results are classified into the first possibility group, the (k1+1)-th to k2-th results are classified into the second possibility group, and the (k2+1)-th and subsequent results are classified into the third possibility group.

[0058] In this way, the saliency gap corresponds to the natural breakpoint in the similarity distribution, and the similarity between the results before the breakpoint and the query image is significantly higher than that of the results after the breakpoint, so they are more likely to be valid matches; through the adaptive threshold, the gap judgment standard is automatically adjusted according to the discreteness of the retrieval result distribution: when σ is large (the result quality is uneven), the threshold is increased to avoid misjudging normal fluctuations as breakpoints; when σ is small (the result quality is uniform), the threshold is reduced to 0.03, which can still capture subtle but significant jumps. Thus, breakpoints can be stably and objectively identified in different query scenarios. The division of the three groups of first / second / third possibility realized accordingly provides engineers with an objective and reproducible basis for defect matching judgment. Engineers can preferentially review a small number of results in the first possibility group instead of reviewing all results one by one, which can significantly reduce the workload of manual judgment when the number of retrieval results is large, and at the same time eliminate the subjective difference of manual judgment.

[0059] Step S16: Based on the image ID corresponding to the sorted candidate feature vector, obtain the target historical metal microstructure defect image and the corresponding target metadata from the preset vector database, determine the target historical metal microstructure defect image and the target metadata as the retrieval result and output them.

[0060] In this embodiment, based on the image ID corresponding to the sorted candidate feature vectors, the corresponding target historical metal microstructure defect image (e.g., obtaining the image access address through the URL (uniform resource locator) field) and corresponding target metadata are obtained from a preset vector database. These two elements are then used as the retrieval results and output. The output retrieval results can be presented to engineers through a web front-end interface. Each result simultaneously displays a thumbnail of the target historical metal microstructure defect image, a similarity score, and target metadata.

[0061] In one specific implementation, the target metadata includes material composition information, defect type tags, and image library attribution. The material composition information, defect type tags, and image library attribution are associated with the target historical metal microstructure defect image via the image ID. This embodiment associates data from four dimensions through the image ID, forming a four-dimensional traceability evidence chain of image-composition-tag-source. Specifically: the image feature dimension stores images acquired by metallurgical microscopes and scanning electron microscopes, along with their target dimension (2048 dimensions) L2 normalized feature vectors. Image-level similarity matching is achieved through vector retrieval. The image feature dimension is the core dimension for traceability, answering the question "Which historical images are most similar to the current defect image?"; the composition data dimension stores the material composition information corresponding to the image, including Fe, Cr, Ni, Cu, Zn, P, Al, Si, Mn, Mo, Na, V, Mg, S, K, Ca, W, Co, C, O, ... The mass percentage content of elements such as N, F, Ti, Nb, and B is used to assist in defect tracing by filtering the composition range. The composition data dimension answers the question of "whether the defect appears in the same material system". The label data dimension stores defect type labels for images, such as "crack", "inclusion", "segregation", "porosity", "decarburization", "overheating", etc. Label clustering identifies similar defects. The label data dimension answers the question of "whether the defects belong to the same type". The image library data dimension is organized into image libraries according to the image source. Each image library corresponds to a batch, process or material system. The image library filtering narrows the search scope. The image library data dimension answers the question of "whether the defect comes from the same production batch or process".

[0062] The workflow of four-dimensional traceability is as follows: First, a set of similar images is retrieved through the image feature dimension. Then, it is determined whether the similar images belong to the same material system through the component data dimension. It is determined whether the defect types are consistent through the label data dimension. It is determined whether they come from the same production batch through the spectral library data dimension. The evidence from the four dimensions corroborates each other to form a complete conclusion on the cause of the defect.

[0063] In this way, by associating image features, material composition, defect labels, and image library affiliation through image IDs, when similar defect cases are retrieved, engineers can not only see the image similarity, but also simultaneously obtain material composition information (to determine whether they are the same material system), defect labels (to determine whether they are the same type of defect), and image library affiliation (to determine whether they are from the same batch or process). When the evidence from these four dimensions corroborates each other, the confidence of the tracing conclusion is significantly improved. If any dimension is contradictory, it can also prompt engineers to conduct further verification. Thus, isolated image similarity matching is upgraded to a complete, interpretable, and more reliable chain of evidence for tracing the cause of defects, breaking through the limitations of traditional single image similarity matching.

[0064] Furthermore, in one specific implementation, the search results can be grouped based on defect tags, image library affiliation, and filename prefixes to obtain multiple defect clusters, thereby achieving defect type clustering. Specifically: when grouping based on the defect tag dimension, search results with the same defect tag are classified into one cluster. Defect tags include, but are not limited to, "cracks," "inclusions," "segregation," "voids," "decarburization," and "overheating." When grouping based on the image library affiliation dimension, search results belonging to the same image library are classified into one cluster. The same image library usually corresponds to the detection data of the same batch or the same process. When grouping based on the filename prefix dimension, the letter or Chinese character prefix (e.g., material grade prefixes such as "Q235-" and "304SS-") in the original filename of each historical metal microstructure defect image in the storage system is extracted, and search results with the same prefix are classified into one cluster. Then, the number of samples, average similarity, and maximum similarity are calculated for each defect cluster. Based on the number of samples, they are sorted in descending order, and the top preset number of defect clusters are determined as the defect type clustering results and output. For example, the preset number can be 10, that is, the top 10 clusters with the largest number of samples are taken as the defect type clustering results; in other embodiments, the preset number can also be 5, 20 or other values, which are not limited in this application. In this way, after clustering the search results according to three orthogonal dimensions of label, graph library and file name prefix, engineers can quickly identify the defect type distribution characteristics in the search results at the cluster level. For example, if the top few defect clusters with the largest number of samples all correspond to the "mixed" label and are concentrated in the same graph library, it can be preliminarily inferred that the current defect is homologous to the mixed defects in this batch, thereby quickly focusing the investigation direction and further improving the efficiency of defect tracing.

[0065] Furthermore, this embodiment also supports batch retrieval and comparison of multiple images of microstructural defects in metals to be detected. In a specific implementation, if there are multiple (let's say N) images of microstructural defects in metals to be detected {I1, I2, …, I…} N Then, for each image of the metal microstructure defect to be detected, the aforementioned feature extraction and version isolation retrieval process is performed independently to determine the retrieval result set {R1, R2, …, R} corresponding to each image of the metal microstructure defect to be detected. N}, where R i ={r i,1 , r i,2 , …, r i,K}, r i,jIt includes the target image identifier `target_id` and the similarity score `similarity`. It should be noted that the retrieval process for each image of the metal microstructure defect to be detected is independent. This process can be performed in parallel using thread pools, asynchronous I / O (Input / Output), etc., to improve batch retrieval efficiency, or it can be performed serially; this application does not limit this approach.

[0066] Then, the search result sets are aggregated according to the target image dimension. In one specific implementation, an aggregation mapping table M:target_id→AggregatedResult is constructed, and all search result sets R are traversed. i For each result r i,j If `target_id` is not in mapping table M, a new entry `{averageSimilarity:similarity,maxSimilarity:similarity,minSimilarity:similarity,matchCount:1,details:[{sourceIndex:i,similarity: similarity}]}` is created. If `target_id` is already in mapping table M, the corresponding entry is updated: `matchCount` is incremented by 1, `maxSimilarity` and `minSimilarity` are updated, the current `{sourceIndex:i,similarity: similarity}` is appended to the `details` list, and `averageSimilarity` is recalculated as the average of all similarities in `details`. After all images of metal microstructure defects to be detected have been processed, the matching rate `matchRate` = `matchCount / N` is calculated for each record in the aggregated mapping table M. This matching rate is the ratio of the number of matches to the total number of images of metal microstructure defects to be detected. Thus, the average similarity, maximum similarity, minimum similarity, number of matches, and matching rate for each target historical metal microstructure defect image are obtained.

[0067] Next, the aggregation results are sorted based on the descending order of the number of matches and the descending order of the average similarity to obtain the sorted aggregation results, that is, the target historical metal microstructure defect images that are simultaneously matched by multiple images of metal microstructure defects to be detected and have high average similarity are displayed first.

[0068] Finally, a similarity matrix Matrix∈R is generated, with the image of the microstructure defect of the metal to be detected as the row and the image of the target historical metal microstructure defect as the column. N×MWhere M is the number of target historical metal microstructure defect images obtained through aggregation, and the matrix element Matrix[i][j] is the similarity between the i-th metal microstructure defect image to be detected and the j-th target historical metal microstructure defect image; if the retrieval result set of any metal microstructure defect image to be detected does not contain any target historical metal microstructure defect image, then the corresponding element in the similarity matrix is ​​set to zero. The similarity matrix and the sorted aggregation result are determined as the batch retrieval result and output. In a specific implementation, the similarity matrix supports heatmap visualization, where darker colors in the matrix indicate higher similarity, allowing engineers to intuitively identify the correspondence between the image to be detected and the target historical image.

[0069] In one specific implementation, to verify the quality of the feature vectors extracted by the method of this application, the following evaluation metrics can also be used for quantitative evaluation: intra-class similarity, i.e., for all images of the same defect type, calculate the cosine similarity between each pair and take the average as the intra-class similarity of the defect type. The higher the intra-class similarity, the stronger the consistency of the feature extraction model in representing the same type of defect; inter-class discrimination, i.e. for different defect types, calculate the cosine similarity between the center vectors of each class and take the average as the inter-class discrimination. The lower the inter-class discrimination, the stronger the feature extraction model's ability to distinguish different defect types; clustering quality metrics, i.e., using Silhouette Score and Davies-Bouldin Index (DBI) to evaluate the clustering quality of the feature vectors, where the Silhouette Score ranges from [-1, ... [1] The larger the value, the better the clustering quality; the smaller the DBI value, the better the clustering quality. Retrieval metrics, namely mAP@K (mean precision), Recall@K (recall rate), and Precision@K (precision rate), are used to evaluate the accuracy of vector retrieval. Based on manually labeled defect types, the ranking of similar defects in the retrieval results is evaluated. Feature distribution visualization, namely, using the t-SNE (t-Distributed Stochastic Neighbor Embedding) dimensionality reduction algorithm to reduce the target dimension (2048 dimensions) feature vector to 2 dimensions for visualization, observe the distribution of feature vectors of different defect types in the dimensionality reduction space. The ideal feature distribution should show that defects of the same type are clustered and defects of different types are separated. Using the above evaluation system, experiments were conducted with DinoV2 as the self-supervised visual model and ResNet50 as the benchmark. The results show that DinoV2 significantly improves the intra-class similarity of defects of the same type in metal microstructure images compared with ResNet50, and also significantly improves the inter-class discrimination of different types of defects. This verifies that the self-supervised visual model can obtain feature representation capabilities superior to supervised learning models without fine-tuning in the low-variance, high-texture-repetition image domain of metal microstructures.

[0070] As can be seen, this application selects the target self-supervised visual model by dynamically adapting to server memory, and flexibly constructs a linear projection layer without bias terms based on the relationship between the model's native output dimension and the target dimension. This reduces computational overhead and memory usage while ensuring the accuracy of feature representation. Simultaneously, it structurally stores historical image features, metadata, and model version numbers in a vector database, and introduces model version filtering conditions (exactly matching the current version or compatible with empty versions) during retrieval. This effectively avoids mismatches caused by inconsistencies in the feature spaces of different model versions, significantly improving the accuracy and stability of similar defect retrieval in cross-version scenarios. Finally, through similarity ranking and metadata backtracking of associated image IDs, it achieves efficient, accurate, and interpretable historical case recommendations for metal microstructure defects, providing a low-resource-consumption, highly robust intelligent retrieval solution for practical quality inspection scenarios.

[0071] In one specific implementation, the above-mentioned method for tracing the source of metal microstructure defects can be deployed as a system for tracing the source of metal microstructure defects. This system includes: an image acquisition and uploading module for receiving metal microstructure images acquired by metallurgical microscopes and scanning electron microscopes, supporting single and batch uploads, automatically identifying image formats and performing preprocessing, including image compression, format conversion, and size normalization; a feature extraction module for extracting feature vectors using a self-supervised visual model and a linear projection layer, including a model loading unit (loading self-supervised visual models of various specifications according to priority), a linear projection unit (dimensional mapping initialized by Xavier), an L2 normalization unit, and a model fallback unit (automatically switching to a pre-trained convolutional neural network model when loading fails); and a vector storage and retrieval module that uses a Milvus vector database to achieve model version-aware vector storage and isolation. The retrieval system includes a model version isolation unit (version filtering retrieval), a schema compatibility unit (automatically adding missing fields), and a hybrid retrieval unit (scalar + vector hybrid query); a search analysis module performs saliency gap detection and defect type clustering on the retrieval results, including a similarity grouping unit (adaptive threshold gap detection) and a defect type clustering unit (3D clustering of tags / map library / filename prefixes); a batch comparison module supports batch retrieval of multiple images, result aggregation, and similarity matrix generation, including a multi-image retrieval unit, a result aggregation unit, and a similarity matrix generation unit; and a multi-dimensional source tracing module performs four-dimensional association between image features and components, tags, and map libraries, including a component association unit, a tag association unit, and a map library association unit.

[0072] The data flow between modules is as follows: the image acquisition and upload module transmits the preprocessed image to the feature extraction module; the feature extraction module transmits the extracted feature vectors to the vector storage and retrieval module for storage, or transmits the query feature vectors to the vector storage and retrieval module for retrieval; the vector storage and retrieval module transmits the retrieval results to the search analysis module and the batch comparison module; the analysis results of the search analysis module and the batch comparison module, together with the associated information from the multi-dimensional tracing module, constitute the final output.

[0073] See Figure 2 As shown, an embodiment of the present invention discloses a device for tracing the source of defects in the microstructure of metals, comprising: The size relationship determination module 11 is used to determine the target self-supervised visual model based on the available memory of the server, and to determine the size relationship between the native output dimension of the target self-supervised visual model and the target dimension. The first feature vector determination module 12 is used to input the preprocessed historical metal microstructure defect image into the target self-supervised visual model to obtain an initial feature vector, and determine whether to construct a linear projection layer without bias terms based on the size relationship. If constructed, the first feature vector is determined based on the linear projection layer and the initial feature vector. The writing module 13 is used to write the first feature vector, the metadata of the historical metal microstructure defect image, and the version number of the target self-supervised visual model into a preset vector database. The candidate feature vector filtering module 14 is used to determine the second feature vector corresponding to the image of the metal microstructure defect to be detected, construct a target filtering condition that includes model version filtering conditions, and filter candidate feature vectors in the preset vector database based on the target filtering conditions; the model version filtering conditions include the model version being equal to the version number of the current model or the model version being empty; The sorting module 15 is used to determine the similarity between the second feature vector and the candidate feature vector, and to sort the candidate feature vectors in descending order according to the similarity from high to low to obtain the sorted candidate feature vectors. The retrieval result determination module 16 is used to obtain the target historical metal microstructure defect image and the corresponding target metadata corresponding to the image ID from the preset vector database according to the image ID corresponding to the sorted candidate feature vector, and determine the target historical metal microstructure defect image and the target metadata as the retrieval result and output them.

[0074] Since the embodiments of the device part correspond to the embodiments described above, please refer to the embodiments described in the method part for the embodiments of the device part, and will not be repeated here.

[0075] As can be seen, this application selects the target self-supervised visual model by dynamically adapting to server memory, and flexibly constructs a linear projection layer without bias terms based on the relationship between the model's native output dimension and the target dimension. This reduces computational overhead and memory usage while ensuring the accuracy of feature representation. Simultaneously, it structurally stores historical image features, metadata, and model version numbers in a vector database, and introduces model version filtering conditions (exactly matching the current version or compatible with empty versions) during retrieval. This effectively avoids mismatches caused by inconsistencies in the feature spaces of different model versions, significantly improving the accuracy and stability of similar defect retrieval in cross-version scenarios. Finally, through similarity ranking and metadata backtracking of associated image IDs, it achieves efficient, accurate, and interpretable historical case recommendations for metal microstructure defects, providing a low-resource-consumption, highly robust intelligent retrieval solution for practical quality inspection scenarios.

[0076] Furthermore, embodiments of this application also disclose an electronic device, Figure 3 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0077] Figure 3 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the metal microstructure defect tracing method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0078] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0079] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0080] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the metal microstructure defect tracing method disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0081] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned method for tracing the source of defects in the metal microstructure. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0082] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0083] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0084] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0085] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0086] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for tracing the source of microstructural defects in metals, characterized in that, include: The target self-supervised visual model is determined based on the available memory on the server, and the relationship between the native output dimension of the target self-supervised visual model and the target dimension is determined. The preprocessed historical metal microstructure defect image is input into the target self-supervised visual model to obtain an initial feature vector. Based on the size relationship, it is determined whether to construct a linear projection layer without bias terms. If constructed, a first feature vector is determined based on the linear projection layer and the initial feature vector. Write the first feature vector, the metadata of the historical metal microstructure defect image, and the version number of the target self-supervised visual model into a preset vector database. A second feature vector corresponding to the microstructural defect image of the metal to be detected is determined, a target filtering condition including model version filtering conditions is constructed, and candidate feature vectors in the preset vector database are filtered based on the target filtering conditions; the model version filtering conditions include model version equal to the version number of the current model or model version being empty; Determine the similarity between the second feature vector and the candidate feature vector, and sort the candidate feature vectors in descending order according to the similarity to obtain the sorted candidate feature vectors; Based on the image ID corresponding to the sorted candidate feature vector, the target historical metal microstructure defect image and the corresponding target metadata are obtained from the preset vector database. The target historical metal microstructure defect image and the target metadata are determined as the retrieval results and output.

2. The method for tracing the source of defects in the microstructure of metals according to claim 1, characterized in that, The step of determining the target self-supervised visual model based on the available memory of the server includes: Based on the priority of feature extraction accuracy from high to low, several self-supervised visual models of different specifications are traversed in turn to determine whether the available memory of the server meets the minimum memory requirements of the target specification of the self-supervised visual model. If the conditions are met, the self-supervised visual model of the target specification is loaded, and the successfully loaded self-supervised visual model of the target specification is determined as the target self-supervised visual model. If the target specification is not met, skip the self-supervised visual model and continue iterating. Accordingly, the method further includes: If all self-supervised vision models fail to load, then load the pre-trained convolutional neural network model and remove the fully connected layers of the pre-trained convolutional neural network model. The output of the average pooling layer of the pre-trained convolutional neural network model is determined as the original feature vector, and the original feature vector is L2 normalized to obtain the first feature vector. Wherein, the output dimension of the pre-trained convolutional neural network model is equal to the target dimension, and the linear projection layer is not constructed.

3. The method for tracing the source of defects in the microstructure of metals according to claim 1, characterized in that, The step of determining whether to construct a linear projection layer without bias terms based on the size relationship includes: If the native output dimension is not equal to the target dimension, a linear projection layer without bias terms is constructed; wherein, the weight matrix of the linear projection layer is initialized based on the Xavier normal distribution initialization method so that each element of the weight matrix satisfies a normal distribution; If the native output dimension is equal to the target dimension, then a linear projection layer without bias terms is not constructed, and the initial feature vector is L2 normalized to obtain the first feature vector. Accordingly, determining the first feature vector based on the linear projection layer and the initial feature vector includes: The initial feature vector is input into the linear projection layer to obtain the projected feature vector of the target dimension; The projected feature vector is L2 normalized to obtain the first feature vector.

4. The method for tracing the source of defects in the microstructure of metals according to claim 1, characterized in that, Before inputting the preprocessed historical metal microstructure defect image into the target self-supervised visual model, the method further includes: The historical metal microstructure defect image is scaled to a preset pixel size to obtain a scaled image; If the scaled image is a grayscale image, then the scaled image is converted into an RGB three-channel image by channel copying to obtain the image to be standardized; If the scaled image is an RGB three-channel image, then the scaled image is determined as the image to be standardized; Based on the preset mean and preset standard deviation, channel standardization is performed on each channel of the image to be standardized to obtain a preprocessed historical metal microstructure defect image. Accordingly, the step of inputting the preprocessed historical metal microstructure defect image into the target self-supervised visual model to obtain an initial feature vector includes: The preprocessed historical metal microstructure defect image is input into the target self-supervised vision model based on the Vision Transformer architecture, and the CLS token feature vector is extracted as the initial feature vector; the CLS token feature vector is the classification label vector used in Vision Transformer to aggregate global image information.

5. The method for tracing the source of defects in the microstructure of metals according to claim 1, characterized in that, Determining the similarity between the second feature vector and the candidate feature vector includes: The similarity is determined by the inner product of the second feature vector and the candidate feature vector. Accordingly, the method further includes: Based on the similarity sequence corresponding to the sorted candidate feature vectors, calculate the difference between two adjacent similarities; The target threshold is determined based on the standard deviation of the similarity sequences; The positions where the difference is greater than or equal to the target threshold are defined as significant gaps; The search results are divided into a first probability group, a second probability group, and a third probability group based on the number and location of the salience gaps; wherein the first probability is greater than the second probability; and the second probability is greater than the third probability.

6. The method for tracing the source of defects in the microstructure of metals according to claim 1, characterized in that, The target metadata includes material composition information, defect type labels, and image library affiliation; the material composition information, defect type labels, and image library affiliation are associated with the target's historical metal microstructure defect images through the image ID; Accordingly, the method further includes: The search results are grouped based on defect labels, attribution to an image library, and filename prefixes to obtain multiple defect clusters. Specifically, when grouping by defect label dimension, search results with the same defect label are classified into one cluster; when grouping by attribution to an image library dimension, search results belonging to the same image library are classified into one cluster; when grouping by filename prefix dimension, the letter or Chinese character prefix is ​​extracted from the original filename of each historical metal microstructure defect image in the storage system, and search results with the same prefix are classified into one cluster. Calculate the number of samples for each defect cluster; Based on the sample quantity sorted in descending order, the first preset number of defect clusters are determined as the defect type clustering results and output.

7. The method for tracing the source of defects in the microstructure of metals according to any one of claims 1 to 6, characterized in that, Also includes: If there are multiple images of the microstructure defects of the metal to be detected, then determine the search result set corresponding to each image of the microstructure defects of the metal to be detected. The search result sets are aggregated according to the target image dimension, and the average similarity, maximum similarity, minimum similarity, number of matches, and matching rate of each aggregated target historical metal microstructure defect image are calculated; the matching rate is the ratio of the number of matches to the total number of metal microstructure defect images to be detected; The aggregation results are sorted based on the descending order of the number of matches and the descending order of the average similarity to obtain the sorted aggregation results; Generate a similarity matrix with the images of metal microstructure defects to be detected as rows and the images of target historical metal microstructure defects as columns; wherein, the elements in the similarity matrix are the similarity between the corresponding images of metal microstructure defects to be detected and the corresponding images of target historical metal microstructure defects; if the search result set of any image of metal microstructure defects to be detected does not contain any image of target historical metal microstructure defects, then the corresponding element in the similarity matrix is ​​set to zero; The similarity matrix and the sorted aggregation results are used to determine the batch retrieval results and output them.

8. A device for tracing the source of microstructural defects in metals, characterized in that, include: The size relationship determination module is used to determine the target self-supervised visual model based on the available memory of the server, and to determine the size relationship between the native output dimension of the target self-supervised visual model and the target dimension. The first feature vector determination module is used to input the preprocessed historical metal microstructure defect image into the target self-supervised visual model to obtain an initial feature vector, and determine whether to construct a linear projection layer without bias terms based on the size relationship. If constructed, the first feature vector is determined based on the linear projection layer and the initial feature vector. The writing module is used to write the first feature vector, the metadata of the historical metal microstructure defect image, and the version number of the target self-supervised visual model into a preset vector database. The candidate feature vector filtering module is used to determine the second feature vector corresponding to the image of the metal microstructure defect to be detected, construct a target filtering condition that includes model version filtering conditions, and filter candidate feature vectors in the preset vector database based on the target filtering conditions; the model version filtering conditions include the model version being equal to the version number of the current model or the model version being empty; The sorting module is used to determine the similarity between the second feature vector and the candidate feature vector, and to sort the candidate feature vectors in descending order according to the similarity, so as to obtain the sorted candidate feature vectors. The retrieval result determination module is used to obtain the target historical metal microstructure defect image and the corresponding target metadata corresponding to the image ID from the preset vector database according to the image ID corresponding to the sorted candidate feature vector, and determine the target historical metal microstructure defect image and the target metadata as the retrieval result and output them.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing a computer program to implement the steps of the method for tracing the source of defects in the metal microstructure as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on a computer-readable storage medium, and when executed by a processor, the computer program implements the steps of the method for tracing the source of defects in the metal microstructure as described in any one of claims 1 to 7.

Images