Weld inspection method and system for hand mold

Abstract

A welding detection method and system for a hand mold, which acquires a welding detection image of a hand mold to be detected, uses deep learning-based image processing technology to mine and capture features of the welding detection image of the hand mold to be detected, and realizes hand mold welding detection through classification processing. In this way, non-destructive testing of the welding quality of the hand mold can be realized, the quality and efficiency of hand mold manufacturing can be improved, and costs and risks can be reduced.

Description

Technical Field

[0001] This application relates to the field of intelligent inspection technology, and more specifically, to a welding inspection method and system for hand molds. Background Technology

[0002] Metal hand models, made from alloys, have a lifespan 16 to 24 times longer than ordinary ceramic hand models.

[0003] During the welding process of metal hand molds, the following problems often occur: 1. Defects such as cracks, porosity, and slag inclusions may exist in the weld, leading to insufficient weld strength or even failure; 2. Thermal stress and cooling shrinkage can cause deformation of the weldment, thus affecting assembly accuracy and appearance quality; 3. Contamination on the metal surface, such as grease, oxides, and rust, can affect the weld quality. Therefore, welding inspection is a very important task.

[0004] Traditional welding inspection requires visual or destructive testing methods, which are time-consuming, labor-intensive, and prone to repetitive testing. Therefore, an optimized welding inspection solution for hand molds is needed. Summary of the Invention

[0005] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide a welding inspection method and system for hand molds, which acquires welding inspection images of the hand mold to be inspected; utilizes deep learning-based image processing technology to perform feature mining and capture on the welding inspection images of the hand mold to be inspected, and achieves welding inspection of the hand mold through classification processing. This enables non-destructive testing of the welding quality of hand molds, improves the quality and efficiency of hand mold production, and reduces costs and risks.

[0006] In a first aspect, a welding inspection method for a hand mold is provided, comprising: acquiring a welding inspection image of the hand mold to be inspected; passing the welding inspection image through a welding area target detection network to obtain multiple welding regions of interest; passing each of the multiple welding regions of interest through a welding feature extractor based on a convolutional neural network model to obtain multiple welding feature vectors; calculating the cosine similarity between every two welding feature vectors to form a similarity correlation matrix; passing the similarity correlation matrix through a correlation feature extractor to obtain a similarity correlation feature matrix; arranging the multiple welding feature vectors in a two-dimensional manner to form a welding feature matrix; inputting the welding feature matrix and the similarity correlation feature matrix into a graph neural network model to obtain a welding topology global correlation feature matrix; performing feature affinity space affine learning on the welding topology global correlation feature matrix to obtain an optimized welding topology global correlation feature matrix; and passing the optimized welding topology global correlation feature matrix through a classifier to obtain a classification result, wherein the classification result is used to indicate whether the welding quality of the hand mold to be inspected meets predetermined requirements.

[0007] In the above-mentioned welding inspection method for hand molds, the target detection network for the welding area is YOLOv1, FCOS, or CenterNet.

[0008] In the above-mentioned welding detection method for hand molds, the multiple welding regions of interest are respectively processed through a welding feature extractor based on a convolutional neural network model to obtain multiple welding feature vectors. This includes: using each layer of the welding feature extractor based on the convolutional neural network model to process the input data during the forward pass of each layer, performing the following operations on the input data: convolution processing to obtain a convolutional feature map; mean pooling processing based on the feature matrix on the convolutional feature map to obtain a pooled feature map; and nonlinear activation on the pooled feature map to obtain an activated feature map. The output of the last layer of the welding feature extractor based on the convolutional neural network model is the multiple welding feature vectors, and the input of the first layer of the welding feature extractor based on the convolutional neural network model is the multiple welding regions of interest.

[0009] In the above-mentioned welding inspection method for hand molds, calculating the cosine similarity between every two welding feature vectors to form a similarity association matrix from the multiple cosine similarities includes: calculating multiple cosine similarities between every two welding feature vectors using the following cosine formula; wherein, the formula is: ,in, and This represents every two welding feature vectors among the plurality of welding feature vectors. and This represents the feature value at each position in the plurality of welding feature vectors. The cosine distance between any two welding feature vectors is represented by the plurality of welding feature vectors; and the plurality of cosine similarities are arranged in two dimensions to obtain the similarity association matrix.

[0010] In the above-mentioned welding detection method for hand molds, the similarity correlation matrix is ​​processed by a correlation feature extractor to obtain a similarity correlation feature matrix. This includes: using each layer of the correlation feature extractor to perform convolution processing, pooling processing along the channel dimension, and nonlinear activation processing on the input data during the forward pass of the layer, so that the output of the last layer of the correlation feature extractor is the similarity correlation feature matrix, wherein the input of the first layer of the correlation feature extractor is the similarity correlation matrix.

[0011] In the above-mentioned welding detection method for hand molds, the welding feature matrix and the similarity association feature matrix are input into a graph neural network model to obtain a global welding topology association feature matrix. This includes: using the graph neural network model with learnable neural network parameters to process the welding feature matrix and the similarity association feature matrix to obtain the global welding topology association feature matrix containing irregular spatial topology features and welding cooperative features.

[0012] In the above-mentioned welding inspection method for hand molds, performing feature affinity space affine learning on the welding topology global correlation feature matrix to obtain an optimized welding topology global correlation feature matrix includes: performing feature affinity space affine learning on the welding topology global correlation feature matrix using the following optimization formula to obtain an optimized welding topology global correlation feature matrix; wherein, the optimization formula is: ,in, It is the global correlation feature matrix of the welding topology. It is the transpose of the global correlation feature matrix of the welding topology. Describes the 2-norm of a matrix. Let denote the nuclear norm of the matrix, and It is the scale of the matrix. It is the global correlation feature matrix of the optimized welding topology.

[0013] In the above-mentioned welding inspection method for hand molds, the optimized welding topology global correlation feature matrix is ​​passed through a classifier to obtain a classification result. The classification result is used to indicate whether the welding quality of the hand mold to be inspected meets the predetermined requirements. The method includes: expanding the optimized welding topology global correlation feature matrix into a classification feature vector according to row vectors or column vectors; using multiple fully connected layers of the classifier to fully connect and encode the classification feature vector to obtain an encoded classification feature vector; and passing the encoded classification feature vector through the Softmax classification function of the classifier to obtain the classification result.

[0014] Secondly, a welding inspection system for a hand mold is provided, comprising: an image acquisition module for acquiring a welding inspection image of the hand mold to be inspected; a region of interest (ROI) generation module for passing the welding inspection image through a welding region target detection network to obtain multiple welding ROIs; a welding feature extraction module for passing each of the multiple ROIs through a welding feature extractor based on a convolutional neural network model to obtain multiple welding feature vectors; a cosine similarity calculation module for calculating the cosine similarity between every two welding feature vectors to form a similarity correlation matrix from the multiple cosine similarities; a correlation feature extraction module for passing the similarity correlation matrix through a correlation feature extractor to obtain a similarity correlation feature matrix; a two-dimensional arrangement module for arranging the multiple welding feature vectors in a two-dimensional manner to form a welding feature matrix; a graph neural network module for inputting the welding feature matrix and the similarity correlation feature matrix into a graph neural network model to obtain a welding topology global correlation feature matrix; and an optimization module for performing feature affinity space affine learning on the welding topology global correlation feature matrix to obtain an optimized welding topology global correlation feature matrix; and The welding quality result generation module is used to pass the optimized welding topology global correlation feature matrix through a classifier to obtain a classification result, which is used to indicate whether the welding quality of the hand mold to be inspected meets the predetermined requirements.

[0015] In the aforementioned welding inspection system for hand molds, the target detection network for the welding area is YOLOv1, FCOS, or CenterNet.

[0016] Compared with existing technologies, the welding inspection method and system for hand molds provided in this application acquire welding inspection images of the hand mold to be inspected; utilize deep learning-based image processing technology to perform feature mining and capture on the welding inspection images of the hand mold to be inspected, and achieve hand mold welding inspection through classification processing. This enables non-destructive testing of hand mold welding quality, improves the quality and efficiency of hand mold production, and reduces costs and risks. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of a welding inspection method for a hand mold according to an embodiment of this application.

[0019] Figure 2 This is a flowchart of a welding inspection method for a hand mold according to an embodiment of this application.

[0020] Figure 3 This is a schematic diagram of the architecture of a welding inspection method for a hand mold according to an embodiment of this application.

[0021] Figure 4 This is a flowchart of a sub-step in step 190 of the welding inspection method for a hand mold according to an embodiment of this application.

[0022] Figure 5 This is a block diagram of a welding inspection system for a hand mold according to an embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0024] Unless otherwise stated, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application.

[0025] In the embodiments described in this application, it should be noted that, unless otherwise stated and limited, the term "connection" should be interpreted broadly. For example, it can be an electrical connection, or a connection between two internal components. It can be a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above term according to the specific circumstances.

[0026] It should be noted that the terms "first," "second," and "third" used in the embodiments of this application are merely used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first," "second," and "third" can be interchanged in a specific order or sequence where permitted. It should be understood that the objects distinguished by "first," "second," and "third" can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in an order other than those illustrated or described herein.

[0027] To address the aforementioned technical problems, the technical concept of this application is to utilize deep learning-based image processing technology to perform feature mining and capture on the welding inspection image of the hand mold to be inspected, and to achieve hand mold welding inspection through classification processing.

[0028] Specifically, in the technical solution of this application, firstly, a welding inspection image of the hand mold to be inspected is obtained. Here, a welding inspection image refers to a digital image obtained by taking a picture or scanning the welded part, which can reflect information such as the shape, size, and defects of the weld.

[0029] Next, the welding inspection image is processed through a welding area target detection network to obtain multiple welding regions of interest. This network can automatically locate and identify different welding areas in the image, such as weld seams, weld points, and weld bends. This effectively extracts the welding areas and distinguishes them from other background areas, thereby reducing the complexity and error of the inspection.

[0030] Then, the multiple regions of interest (ROIs) are processed by a welding feature extractor based on a convolutional neural network (CNN) model to obtain multiple welding feature vectors. Here, a CNN is a deep learning model capable of effectively extracting image features. It captures local information by sliding convolutional kernels across the image, then uses pooling layers for dimensionality reduction and abstraction to obtain the high-dimensional latent features of the image. Specifically, the ROIs refer to the portions of the image containing the weld seam; these are key areas for judging welding quality. The CNN-based welding feature extractor transforms each ROI into a welding feature vector, which contains information such as the weld seam shape, color, and texture—information crucial for identifying defects or deformations in the weld seam.

[0031] Furthermore, the cosine similarity between every two feature vectors in the plurality of welding feature vectors is calculated to form a similarity correlation matrix. Here, cosine similarity can be used to measure the similarity between two vectors. Specifically, in the welding inspection process, cosine similarity can be used to measure the degree of similarity between two welding feature vectors, or in other words, the relationship between them.

[0032] As mentioned earlier, the similarity correlation matrix is ​​a symmetric matrix that reflects the degree of similarity between different welding regions of interest. This matrix can be used to describe the topological structure of the welding areas, i.e., which areas are adjacent, which are separated, which are similar, and which are different. This information is helpful in judging the welding quality because, generally speaking, areas with good welding quality should have high similarity to their surrounding areas, while areas with poor welding quality should have low similarity to their surrounding areas.

[0033] However, the similarity correlation matrix itself is only a numerical matrix and cannot be directly used in a classifier. Therefore, in the technical solution of this application, the similarity correlation matrix is ​​processed by an association feature extractor to obtain a similarity correlation feature matrix. That is, the association feature extractor extracts the feature information contained in the similarity correlation matrix. The association feature extractor can use convolution kernels to capture the feature information in the similarity correlation matrix. The size and number of convolution kernels can be adjusted according to the actual situation to achieve the best results.

[0034] Subsequently, the multiple welding feature vectors are arranged in a two-dimensional pattern to form a welding feature matrix. This preserves the dimensionality of the original feature vectors while facilitating subsequent model calculations.

[0035] Furthermore, the welding feature matrix and the similarity association feature matrix are input into a graph neural network model to obtain a global welding topology association feature matrix. Here, the graph neural network model is a deep learning model capable of processing graph-structured data. It can extract global information using the topological relationships and attribute information between nodes in the graph. Specifically, the welding feature matrix and the similarity association feature matrix can form a graph structure, where each welding region of interest corresponds to a node, and each cosine similarity corresponds to an edge. By inputting these two matrices into the graph neural network model, a global welding topology association feature matrix can be obtained, which integrates the local features of the welding regions of interest and the global topological relationships between them, thereby better reflecting the feature information of the welding quality of the hand mold to be inspected.

[0036] After obtaining the global correlation feature matrix of the welding topology, it is passed through a classifier to obtain a classification result. The classification result is used to indicate whether the welding quality of the hand mold to be inspected meets the predetermined requirements. Here, the classifier can learn a discriminant function based on the labels in the training data to classify the data in the inference stage. In this way, the welding quality of the hand mold can be automatically detected, improving detection efficiency and quality.

[0037] Here, when the welding feature matrix and the similarity association feature matrix are input into the graph neural network model to obtain the welding topology global association feature matrix, each corresponding welding topology global association feature vector in the welding topology global association feature matrix represents the local correlation expression of the image semantics of a single welding region of interest under the global semantic similarity topology of the welding detection image, and the various welding topology global association feature vectors have a certain degree of independence. Thus, when the various welding topology global association feature vectors are directly arranged into the welding topology global association feature matrix, the correlation between the local distributions of the welding topology global association feature matrix corresponding to each welding topology global association feature vector will be insufficient, affecting the accuracy of the classification results obtained by the classifier from the welding topology global association feature matrix.

[0038] Based on this, in the technical solution of this application, it is preferable to perform feature affinity space affine learning on the global correlation feature matrix of the welding topology for optimization, as expressed as: ,in, It is the global correlation feature matrix of the welding topology. It is the transpose of the global correlation feature matrix of the welding topology. Describes the 2-norm of a matrix. Let denote the nuclear norm of the matrix, and It is the scale of the matrix. It is the global correlation feature matrix of the optimized welding topology.

[0039] Here, the feature affinity spatial affine learning performs affine transfer based on spatial transformation by expressing detailed structured information in a low-dimensional eigenspace within the high-resolution information representation of the feature distribution space of the global correlation feature matrix of the welding topology. This is achieved by performing detailed structured information representation in a low-dimensional eigenspace within the feature distribution space of the global matrix with high-resolution information representation. This enables super-resolution activation of the feature distribution in each local part of the global matrix feature distribution (i.e., per-eigenvector) based on the dense simulation of affinity between image semantic topological representations. Then, the optimized diagonal matrix is ​​further transformed by the inverse transformation corresponding to the linear transformation. By restoring the welding topology global correlation feature matrix, the correlation between the various local feature distributions of the optimized welding topology global correlation feature matrix can be improved, thereby improving the accuracy of the classification results obtained by the classifier from the welding topology global correlation feature matrix.

[0040] This application has the following technical advantages: 1. It provides an intelligent welding inspection solution for hand molds. 2. This solution enables non-destructive testing of the welding quality of hand molds, improving the quality and efficiency of hand mold production while reducing costs and risks.

[0041] Figure 1 This is a schematic diagram illustrating a scenario of a welding inspection method for a hand mold according to an embodiment of this application. Figure 1 As shown, in this application scenario, firstly, the hand model to be detected is obtained (e.g., ...). Figure 1 The weld inspection image of M as shown (e.g., as shown) Figure 1 (as shown in C); then, the acquired weld inspection image is input to a server that deploys a weld inspection algorithm based on a hand mold (e.g., as shown in C). Figure 1 In the S shown, the server is able to process the welding detection image based on the welding detection algorithm of the hand mold to generate a classification result indicating whether the welding quality of the hand mold to be detected meets the predetermined requirements.

[0042] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0043] In one embodiment of this application, Figure 2 This is a flowchart of a welding inspection method for a hand mold according to an embodiment of this application. Figure 2 As shown, a welding detection method 100 for a hand mold according to an embodiment of this application includes: 110, acquiring a welding detection image of the hand mold to be detected; 120, passing the welding detection image through a welding area target detection network to obtain multiple welding regions of interest; 130, passing each of the multiple welding regions of interest through a welding feature extractor based on a convolutional neural network model to obtain multiple welding feature vectors; 140, calculating the cosine similarity between every two welding feature vectors to form a similarity correlation matrix from the multiple cosine similarities; 150, passing the similarity correlation matrix through correlation features... The extractor obtains a similarity association feature matrix; 160, the plurality of welding feature vectors are arranged in two dimensions to form a welding feature matrix; 170, the welding feature matrix and the similarity association feature matrix are input into a graph neural network model to obtain a welding topology global association feature matrix; 180, the welding topology global association feature matrix is ​​subjected to feature affinity space affine learning to obtain an optimized welding topology global association feature matrix; and 190, the optimized welding topology global association feature matrix is ​​passed through a classifier to obtain a classification result, the classification result being used to indicate whether the welding quality of the hand mold to be detected meets the predetermined requirements.

[0044] Figure 3 This is a schematic diagram of the architecture of a welding inspection method for a hand mold according to an embodiment of this application. Figure 3As shown, in this network architecture, firstly, a welding inspection image of the hand mold to be inspected is acquired; then, the welding inspection image is passed through a welding area target detection network to obtain multiple welding regions of interest; next, the multiple welding regions of interest are passed through a welding feature extractor based on a convolutional neural network model to obtain multiple welding feature vectors; then, the cosine similarity between every two welding feature vectors is calculated to form a similarity association matrix; next, the similarity association matrix is ​​passed through an association feature extractor to obtain a similarity association feature matrix; then, the multiple welding feature vectors are arranged in two dimensions to form a welding feature matrix; next, the welding feature matrix and the similarity association feature matrix are input into a graph neural network model to obtain a welding topology global association feature matrix; then, feature affinity space affine learning is performed on the welding topology global association feature matrix to obtain an optimized welding topology global association feature matrix; and finally, the optimized welding topology global association feature matrix is ​​passed through a classifier to obtain a classification result, which is used to indicate whether the welding quality of the hand mold to be inspected meets the predetermined requirements.

[0045] Specifically, in step 110, a welding inspection image of the hand mold to be inspected is acquired. To address the aforementioned technical problems, the technical concept of this application is to utilize deep learning-based image processing technology to perform feature mining and capture on the welding inspection image of the hand mold to be inspected, and to achieve hand mold welding inspection through classification processing.

[0046] Specifically, in the technical solution of this application, firstly, a welding inspection image of the hand mold to be inspected is obtained. Here, a welding inspection image refers to a digital image obtained by taking a picture or scanning the welded part, which can reflect information such as the shape, size, and defects of the weld.

[0047] Specifically, in step 120, the welding inspection image is passed through a welding area target detection network to obtain multiple welding regions of interest. Then, the welding inspection image is passed through the welding area target detection network again to obtain multiple welding regions of interest. The welding area target detection network can automatically locate and identify different welding areas in the image, such as weld seams, weld points, and weld bends. This effectively extracts the welding areas and distinguishes them from other background areas, thereby reducing the complexity and error of the inspection.

[0048] The target detection network for the welding area is YOLOv1, FCOS, or CenterNet.

[0049] Specifically, in step 130, the multiple welding regions of interest are respectively processed through a welding feature extractor based on a convolutional neural network model to obtain multiple welding feature vectors. Then, the multiple welding regions of interest are respectively processed through the welding feature extractor based on a convolutional neural network model to obtain multiple welding feature vectors. Here, the convolutional neural network model (CNN) is a deep learning model capable of effectively extracting image features. It captures local information of the image by sliding convolutional kernels across the image, and then performs dimensionality reduction and abstraction through pooling layers to finally obtain the high-dimensional latent features of the image.

[0050] Specifically, the region of interest (ROI) in welding refers to the portion of an image containing the weld seam, which is a key area for judging the welding quality. Using a welding feature extractor based on a convolutional neural network model, each ROI can be transformed into a welding feature vector. This vector contains information such as the weld seam shape, color, and texture, which is crucial for identifying whether the weld seam has defects or deformation.

[0051] The process involves passing the multiple regions of interest (ROIs) through a welding feature extractor based on a convolutional neural network (CNN) model to obtain multiple welding feature vectors. This includes: each layer of the CNN model performing the following operations on the input data during the forward pass of each layer: convolving the input data to obtain a convolutional feature map; performing mean pooling on the convolutional feature map based on the feature matrix to obtain a pooled feature map; and performing nonlinear activation on the pooled feature map to obtain an activated feature map. The output of the last layer of the CNN model-based welding feature extractor is the multiple welding feature vectors, and the input of the first layer of the CNN model-based welding feature extractor is the multiple ROIs.

[0052] A Convolutional Neural Network (CNN) is an artificial neural network with wide applications in fields such as image recognition. A CNN can consist of an input layer, hidden layers, and an output layer. Hidden layers can include convolutional layers, pooling layers, activation layers, and fully connected layers. Each layer performs calculations on the input data and outputs the results to the next layer. The initial input data undergoes multiple layers of computation to arrive at a final result.

[0053] Convolutional neural network models utilize convolutional kernels as feature filtering factors, exhibiting excellent performance in local image feature extraction. Compared to traditional image feature extraction algorithms based on statistics or feature engineering, these models possess stronger feature extraction generalization and fitting capabilities.

[0054] Specifically, in step 140, the cosine similarity between every two welding feature vectors is calculated to form a similarity association matrix. Further, the cosine similarity between every two feature vectors is calculated to form a similarity association matrix. Here, cosine similarity can be used to measure the similarity between two vectors. Specifically, in the welding inspection process, cosine similarity can be used to measure the degree of similarity between two welding feature vectors, or in other words, their relationship.

[0055] As mentioned earlier, the similarity correlation matrix is ​​a symmetric matrix that reflects the degree of similarity between different welding regions of interest. This matrix can be used to describe the topological structure of the welding areas, i.e., which areas are adjacent, which are separated, which are similar, and which are different. This information is helpful in judging the welding quality because, generally speaking, areas with good welding quality should have high similarity to their surrounding areas, while areas with poor welding quality should have low similarity to their surrounding areas.

[0056] The calculation of the cosine similarity between every two welding feature vectors to form a similarity association matrix includes: calculating multiple cosine similarities between every two welding feature vectors using the following cosine formula; wherein the formula is: ,in, and This represents every two welding feature vectors among the plurality of welding feature vectors. and This represents the feature value at each position in the plurality of welding feature vectors. The cosine distance between any two welding feature vectors is represented by the plurality of welding feature vectors; and the plurality of cosine similarities are arranged in two dimensions to obtain the similarity association matrix.

[0057] Specifically, in step 150, the similarity correlation matrix is ​​processed by an association feature extractor to obtain a similarity correlation feature matrix. However, the similarity correlation matrix itself is only a numerical matrix and cannot be directly used in a classifier. Therefore, in the technical solution of this application, the similarity correlation matrix is ​​processed by an association feature extractor to obtain a similarity correlation feature matrix. That is, the feature information contained in the similarity correlation matrix is ​​extracted by the association feature extractor. The association feature extractor can use convolution kernels to capture the feature information in the similarity correlation matrix. The size and number of convolution kernels can be adjusted according to the actual situation to achieve the best results.

[0058] The process of obtaining a similarity association feature matrix by passing the similarity association matrix through an association feature extractor includes: using each layer of the association feature extractor to perform convolution processing, pooling processing along the channel dimension, and non-linear activation processing on the input data during the forward pass of the layer, so that the output of the last layer of the association feature extractor is the similarity association feature matrix, wherein the input of the first layer of the association feature extractor is the similarity association matrix.

[0059] Specifically, in step 160, the plurality of welding feature vectors are then arranged in a two-dimensional manner into a welding feature matrix. This preserves the dimensionality of the original feature vectors while facilitating subsequent model calculations.

[0060] Specifically, in step 170, the welding feature matrix and the similarity association feature matrix are input into a graph neural network model to obtain a global welding topology association feature matrix. Furthermore, the welding feature matrix and the similarity association feature matrix are input into the graph neural network model to obtain the global welding topology association feature matrix. Here, the graph neural network model is a deep learning model capable of processing graph-structured data; it can extract global information using the topological relationships and attribute information between nodes in the graph.

[0061] Specifically, the welding feature matrix and the similarity association feature matrix can form a graph structure, where each welding region of interest corresponds to a node, and each cosine similarity corresponds to an edge. By inputting these two matrices into a graph neural network model, a global welding topology association feature matrix can be obtained. This matrix integrates the local features of the welding regions of interest with the global topological relationships between them, thus better reflecting the feature information of the welding quality of the hand mold to be inspected.

[0062] Specifically, inputting the welding feature matrix and the similarity association feature matrix into a graph neural network model to obtain a global welding topology association feature matrix includes: using the graph neural network model with learnable neural network parameters to process the welding feature matrix and the similarity association feature matrix to obtain the global welding topology association feature matrix containing irregular spatial topology features and welding cooperative features.

[0063] Research on graph neural networks is closely related to graph embedding or network embedding, which is another topic of increasing interest in data mining and machine learning. Graph embedding aims to represent vertices in a graph as low-dimensional vectors by preserving the network topology and node content information, making them easier to process using simple machine learning algorithms (e.g., support vector machine classification). Many graph embedding algorithms are typically unsupervised and can be broadly categorized into three types: matrix factorization, random walks, and deep learning methods. Deep learning methods for graph embedding also fall under the category of graph neural networks, including algorithms based on graph autoencoders (such as DNGR and SDNE) and unsupervised trained graph convolutional neural networks (such as GraphSage). Graph neural networks are classified into five main categories: Graph Convolution Networks (GCN), Graph Attention Networks, Graph Autoencoders, Graph Generative Networks, and Graph Spatial-temporal Networks.

[0064] There are two main methods for performing convolution operations in Graph Convolutional Networks (GCNs): one is based on spectral decomposition, i.e., spectral decomposition graph convolution; the other is based on node space transformation, i.e., spatial graph convolution. Bruna et al. were the first to generalize convolutional neural networks to graph data, proposing two parallel graph convolution models—spectral decomposition graph convolution and spatial graph convolution. GNNs based on autoencoders are called Graph Auto-encoders (GAEs), which can learn graph node information in a semi-supervised or unsupervised manner. In the field of deep learning, autoencoders (AEs) are a type of artificial neural network that learns representations of input information. Graph Generative Networks (GGNs) are a type of GNN used to generate graph data. They use certain rules to recombine nodes and edges, ultimately generating target graphs with specific attributes and requirements. Graph Recurrent Networks (GRNs) are the earliest type of GNN model. Compared to other GNN algorithms, GRN typically transforms graph data into sequences, which continuously evolve and change recursively during training. The attention mechanism allows a neural network to focus only on the information needed for task learning, enabling it to select specific inputs. Introducing the attention mechanism into GNNs allows the neural network to focus on nodes and edges that are more relevant to the task, improving training effectiveness and testing accuracy, thus forming Graph Attention Network (GAT).

[0065] Specifically, in step 180, feature affinity space affine learning is performed on the welding topology global correlation feature matrix to obtain an optimized welding topology global correlation feature matrix. Here, when the welding feature matrix and the similarity correlation feature matrix are input into the graph neural network model to obtain the welding topology global correlation feature matrix, each corresponding welding topology global correlation feature vector of the welding topology global correlation feature matrix represents the local correlation expression of the image semantics of a single welding region of interest under the global semantic similarity topology of the welding detection image, and the various welding topology global correlation feature vectors have a certain degree of independence. Thus, when the various welding topology global correlation feature vectors are directly arranged into the welding topology global correlation feature matrix, the correlation between the local distributions of the welding topology global correlation feature matrix corresponding to the various welding topology global correlation feature vectors will be insufficient, affecting the accuracy of the classification results obtained by the classifier from the welding topology global correlation feature matrix.

[0066] Based on this, in the technical solution of this application, it is preferable to perform feature affinity space affine learning on the global correlation feature matrix of the welding topology for optimization, expressed as follows: The optimized global correlation feature matrix of the welding topology is obtained by performing feature affinity space affine learning on the global correlation feature matrix of the welding topology using the following optimization formula; wherein, the optimization formula is: ,in, It is the global correlation feature matrix of the welding topology. It is the transpose of the global correlation feature matrix of the welding topology. Describes the 2-norm of a matrix. Let denote the nuclear norm of the matrix, and It is the scale of the matrix. It is the global correlation feature matrix of the optimized welding topology.

[0067] Here, the feature affinity spatial affine learning performs affine transfer based on spatial transformation by expressing detailed structured information in a low-dimensional eigenspace within the high-resolution information representation of the feature distribution space of the global correlation feature matrix of the welding topology. This is achieved by performing detailed structured information representation in a low-dimensional eigenspace within the feature distribution space of the global matrix with high-resolution information representation. This enables super-resolution activation of the feature distribution in each local part of the global matrix feature distribution (i.e., per-eigenvector) based on the dense simulation of affinity between image semantic topological representations. Then, the optimized diagonal matrix is ​​further transformed by the inverse transformation corresponding to the linear transformation. By restoring the welding topology global correlation feature matrix, the correlation between the various local feature distributions of the optimized welding topology global correlation feature matrix can be improved, thereby improving the accuracy of the classification results obtained by the classifier from the welding topology global correlation feature matrix.

[0068] Specifically, in step 190, the optimized welding topology global correlation feature matrix is ​​passed through a classifier to obtain a classification result. This classification result indicates whether the welding quality of the hand mold to be inspected meets predetermined requirements. After obtaining the welding topology global correlation feature matrix, it is passed through a classifier to obtain a classification result, which indicates whether the welding quality of the hand mold to be inspected meets predetermined requirements. Here, the classifier can learn a discriminant function based on the labels in the training data to classify the data in the inference stage. This method automates the detection of the welding quality of the hand mold, improving both detection efficiency and quality.

[0069] Figure 4 The flowchart below shows the sub-steps of step 190 in the welding inspection method for a hand mold according to an embodiment of this application. Figure 4As shown, the optimized welding topology global correlation feature matrix is ​​passed through a classifier to obtain a classification result, which is used to indicate whether the welding quality of the hand mold to be inspected meets the predetermined requirements. The classification result includes: 191, expanding the optimized welding topology global correlation feature matrix into a classification feature vector according to row vectors or column vectors; 192, using multiple fully connected layers of the classifier to fully connect and encode the classification feature vector to obtain an encoded classification feature vector; and 193, passing the encoded classification feature vector through the Softmax classification function of the classifier to obtain the classification result.

[0070] In summary, the welding inspection method 100 for hand molds based on the embodiments of this application is explained, which acquires a welding inspection image of the hand mold to be inspected; utilizes deep learning-based image processing technology to perform feature mining and capture on the welding inspection image of the hand mold to be inspected, and achieves welding inspection of the hand mold through classification processing. In this way, non-destructive testing of the welding quality of hand molds can be achieved, improving the quality and efficiency of hand mold production, and reducing costs and risks.

[0071] In one embodiment of this application, Figure 5 This is a block diagram of a welding inspection system for a hand mold according to an embodiment of this application. Figure 5 As shown, the welding inspection system 200 for a hand mold according to an embodiment of this application includes: an image acquisition module 210 for acquiring a welding inspection image of the hand mold to be inspected; a region of interest generation module 220 for passing the welding inspection image through a welding region target detection network to obtain multiple welding regions of interest; a welding feature extraction module 230 for passing the multiple welding regions of interest through a welding feature extractor based on a convolutional neural network model to obtain multiple welding feature vectors; a cosine similarity calculation module 240 for calculating the cosine similarity between every two welding feature vectors to form a similarity association matrix from the multiple cosine similarities; and an association feature extraction module 250 for processing the similarity association matrix. The system employs a correlation feature extractor to obtain a similarity correlation feature matrix; a two-dimensional arrangement module 260 is used to arrange the multiple welding feature vectors into a two-dimensional welding feature matrix; a graph neural network module 270 is used to input the welding feature matrix and the similarity correlation feature matrix into a graph neural network model to obtain a welding topology global correlation feature matrix; an optimization module 280 is used to perform feature affinity space affine learning on the welding topology global correlation feature matrix to obtain an optimized welding topology global correlation feature matrix; and a welding quality result generation module 290 is used to pass the optimized welding topology global correlation feature matrix through a classifier to obtain a classification result, the classification result being used to indicate whether the welding quality of the hand mold to be detected meets the predetermined requirements.

[0072] In a specific example, in the welding inspection system for the aforementioned hand mold, the target detection network for the welding area is YOLOv1, FCOS, or CenterNet.

[0073] In a specific example, in the welding inspection system for the aforementioned hand mold, the welding feature extraction module is used to: perform the following operations on the input data during the forward propagation of the layers of the welding feature extractor based on the convolutional neural network model: convolving the input data to obtain a convolutional feature map; performing mean pooling based on the feature matrix on the convolutional feature map to obtain a pooled feature map; and performing nonlinear activation on the pooled feature map to obtain an activated feature map; wherein the output of the last layer of the welding feature extractor based on the convolutional neural network model is the plurality of welding feature vectors, and the input of the first layer of the welding feature extractor based on the convolutional neural network model is the plurality of welding regions of interest.

[0074] In a specific example, in the welding inspection system for the aforementioned hand mold, the cosine similarity calculation module is used to: calculate multiple cosine similarities between every two welding feature vectors using the following cosine formula; wherein, the formula is: ,in, and This represents every two welding feature vectors among the plurality of welding feature vectors. and This represents the feature value at each position in the plurality of welding feature vectors. The cosine distance between any two welding feature vectors is represented by the plurality of welding feature vectors; and the plurality of cosine similarities are arranged in two dimensions to obtain the similarity association matrix.

[0075] In a specific example, in the welding inspection system for the aforementioned hand mold, the associated feature extraction module is used to: perform convolution processing, pooling processing along the channel dimension, and nonlinear activation processing on the input data in the forward pass of each layer of the associated feature extractor so that the output of the last layer of the associated feature extractor is the similarity associated feature matrix, wherein the input of the first layer of the associated feature extractor is the similarity associated matrix.

[0076] In a specific example, in the welding inspection system for the aforementioned hand mold, the graph neural network module is used to: process the welding feature matrix and the similarity association feature matrix with learnable neural network parameters using the graph neural network model to obtain the welding topology global association feature matrix containing irregular spatial topology features and welding cooperative features.

[0077] In a specific example, in the welding inspection system for the aforementioned hand mold, the optimization module is used to: perform feature affinity space affine learning on the global correlation feature matrix of the welding topology using the following optimization formula to obtain an optimized global correlation feature matrix of the welding topology; wherein, the optimization formula is: ,in, It is the global correlation feature matrix of the welding topology. It is the transpose of the global correlation feature matrix of the welding topology. Describes the 2-norm of a matrix. Let denote the nuclear norm of the matrix, and It is the scale of the matrix. It is the global correlation feature matrix of the optimized welding topology.

[0078] In a specific example, in the aforementioned hand mold welding inspection system, the welding quality result generation module includes: an expansion unit, used to expand the optimized welding topology global correlation feature matrix into a classification feature vector according to row vectors or column vectors; an encoding unit, used to perform fully connected encoding on the classification feature vector using multiple fully connected layers of the classifier to obtain an encoded classification feature vector; and a classification unit, used to pass the encoded classification feature vector through the Softmax classification function of the classifier to obtain the classification result. Here, those skilled in the art will understand that the specific functions and operations of each unit and module in the above-mentioned welding inspection system for hand molds have been referenced above. Figures 1 to 4 The welding inspection method for hand molds is described in detail in the description, and therefore, its repeated description will be omitted.

[0079] As described above, the welding inspection system 200 for hand molds according to embodiments of this application can be implemented in various terminal devices, such as servers for welding inspection of hand molds. In one example, the welding inspection system 200 for hand molds according to embodiments of this application can be integrated into a terminal device as a software module and / or a hardware module. For example, the welding inspection system 200 for hand molds can be a software module in the operating system of the terminal device, or it can be an application developed for the terminal device; of course, the welding inspection system 200 for hand molds can also be one of many hardware modules of the terminal device.

[0080] Alternatively, in another example, the welding inspection system 200 for the hand mold and the terminal device can also be separate devices, and the welding inspection system 200 for the hand mold can be connected to the terminal device via wired and / or wireless networks, and transmit interactive information in accordance with an agreed data format.

[0081] This application also provides a computer program product including instructions that, when executed, cause a device to perform operations corresponding to the methods described above.

[0082] In one embodiment of this application, a computer-readable storage medium is also provided, which stores a computer program that performs the methods described above.

[0083] It should be understood that embodiments of this application may be provided as methods, systems, or computer program products. Therefore, they may take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, they may take the form of computer program products implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0084] This application describes methods, systems, and computer program products using flowcharts and / or block diagrams. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the flowcharts and / or block diagrams. Figure 1 One or more processes and / or frames Figure 1 A device that provides the functions specified in one or more boxes.

[0085] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or frames Figure 1 The function specified in one or more boxes.

[0086] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or frames Figure 1 The steps of the function specified in one or more boxes.

[0087] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0088] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0089] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0090] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0091] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0092] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method of detecting a weld of a hand mold, characterized by, include: Acquire welding inspection images of the hand mold to be inspected; The welding inspection image is passed through a welding area target detection network to obtain multiple welding regions of interest. The multiple regions of interest in welding are respectively processed through a welding feature extractor based on a convolutional neural network model to obtain multiple welding feature vectors. This includes: using each layer of the welding feature extractor based on the convolutional neural network model to process the input data during the forward pass of each layer, performing the following operations on the input data: convolution processing to obtain a convolutional feature map; mean pooling processing based on the feature matrix on the convolutional feature map to obtain a pooled feature map; and nonlinear activation on the pooled feature map to obtain an activated feature map. The output of the last layer of the welding feature extractor based on the convolutional neural network model is the multiple welding feature vectors, and the input of the first layer of the welding feature extractor based on the convolutional neural network model is the multiple regions of interest in welding. Calculating the cosine similarity between every two welding feature vectors in the plurality of welding feature vectors to form a similarity association matrix composed of the plurality of cosine similarities includes: calculating multiple cosine similarities between every two welding feature vectors in the plurality of welding feature vectors using the following cosine formula; wherein, the formula is: ； in, and This represents every two welding feature vectors among the plurality of welding feature vectors. and This represents the feature value at each position in the plurality of welding feature vectors. The cosine similarity between any two welding feature vectors is represented by the plurality of welding feature vectors; the plurality of cosine similarities are arranged in two dimensions to obtain the similarity association matrix; The similarity association matrix is ​​processed by an association feature extractor to obtain a similarity association feature matrix; The plurality of welding feature vectors are arranged in two dimensions to form a welding feature matrix; The welding feature matrix and the similarity association feature matrix are input into a graph neural network model to obtain a global association feature matrix of welding topology; The global correlation feature matrix of the welding topology is subjected to affine learning in feature affinity space to obtain an optimized global correlation feature matrix of the welding topology; and The optimized welding topology global correlation feature matrix is ​​passed through a classifier to obtain a classification result, which is used to indicate whether the welding quality of the hand mold to be inspected meets the predetermined requirements.

2. The method of claim 1, wherein The target detection network for the welding area is YOLOv1, FCOS, or CenterNet.

3. The method of claim 2, wherein The similarity association matrix is ​​processed by an association feature extractor to obtain a similarity association feature matrix, which includes: using each layer of the association feature extractor to perform convolution processing, pooling processing along the channel dimension and non-linear activation processing on the input data during the forward pass of the layer, so that the output of the last layer of the association feature extractor is the similarity association feature matrix, wherein the input of the first layer of the association feature extractor is the similarity association matrix.

4. The method of claim 3, wherein Inputting the welding feature matrix and the similarity association feature matrix into a graph neural network model to obtain a global welding topology association feature matrix includes: using the graph neural network model with learnable neural network parameters to process the welding feature matrix and the similarity association feature matrix to obtain the global welding topology association feature matrix containing irregular spatial topology features and welding cooperative features.

5. The method of claim 4, wherein the welding of the glove is detected by a change in the color of the glove. To obtain an optimized global correlation feature matrix of welding topology by performing feature affinity space affine learning on the global correlation feature matrix of welding topology, the following optimization formula is used: to perform feature affinity space affine learning on the global correlation feature matrix of welding topology to obtain an optimized global correlation feature matrix of welding topology; The optimization formula is as follows: ； in, It is the global correlation feature matrix of the welding topology. It is the transpose of the global correlation feature matrix of the welding topology. Describes the 2-norm of a matrix. Let represent the nuclear norm of the matrix, and It is the scale of the matrix. It is the global correlation feature matrix of the optimized welding topology.

6. The method of claim 5, wherein the welding of the glove is detected by a change in the color of the glove. The optimized welding topology global correlation feature matrix is ​​passed through a classifier to obtain a classification result. The classification result is used to indicate whether the welding quality of the hand mold to be inspected meets the predetermined requirements, including: The optimized welding topology global correlation feature matrix is ​​expanded into a classification feature vector according to row vectors or column vectors; The classification feature vector is fully encoded using multiple fully connected layers of the classifier to obtain an encoded classification feature vector; and The encoded classification feature vector is passed through the Softmax classification function of the classifier to obtain the classification result.

7. A system for detecting a weld in a hand mold, the system comprising: include: The image acquisition module is used to acquire welding inspection images of the hand mold to be inspected; The region of interest generation module is used to pass the welding inspection image through a welding area target detection network to obtain multiple welding regions of interest. A welding feature extraction module is used to pass the multiple welding regions of interest (ROIs) through a welding feature extractor based on a convolutional neural network (CNN) model to obtain multiple welding feature vectors. The module includes: using each layer of the CNN model-based welding feature extractor to process the input data during the forward pass of each layer, performing the following operations on the input data: convolution processing to obtain a convolutional feature map; mean pooling processing on the convolutional feature map based on the feature matrix to obtain a pooled feature map; and nonlinear activation on the pooled feature map to obtain an activated feature map. The output of the last layer of the CNN model-based welding feature extractor is the multiple welding feature vectors, and the input of the first layer of the CNN model-based welding feature extractor is the multiple welding ROIs. The cosine similarity calculation module is used to calculate the cosine similarity between every two welding feature vectors in the plurality of welding feature vectors to form a similarity association matrix composed of the plurality of cosine similarities, including: calculating multiple cosine similarities between every two welding feature vectors in the plurality of welding feature vectors using the following cosine formula; wherein, the formula is: ； in, and This represents every two welding feature vectors among the plurality of welding feature vectors. and This represents the feature value at each position in the plurality of welding feature vectors. The cosine similarity between any two welding feature vectors is represented by the plurality of welding feature vectors; the plurality of cosine similarities are arranged in two dimensions to obtain the similarity association matrix; The association feature extraction module is used to pass the similarity association matrix through the association feature extractor to obtain a similarity association feature matrix; A two-dimensional arrangement module is used to arrange the multiple welding feature vectors into a welding feature matrix in two dimensions. The graph neural network module is used to input the welding feature matrix and the similarity association feature matrix into the graph neural network model to obtain the global association feature matrix of welding topology; An optimization module is used to perform feature affinity space affine learning on the global correlation feature matrix of the welding topology to obtain an optimized global correlation feature matrix of the welding topology; and The welding quality result generation module is used to pass the optimized welding topology global correlation feature matrix through a classifier to obtain a classification result, which is used to indicate whether the welding quality of the hand mold to be inspected meets the predetermined requirements.

8. The system for detecting a weld of a glove mold according to claim 7, wherein The target detection network for the welding area is YOLOv1, FCOS, or CenterNet.

Images