Sealing nail defect detection method and system based on improved graph attention mechanism, computer readable storage medium and computer program product
By improving the semantic segmentation network of the graph attention mechanism, the problems of poor generalization performance and low detection accuracy of small-size defects in sealed nail defect detection are solved, and higher defect detection accuracy and effective geometric features of point cloud data are achieved. use.
Patent Information
- Application Number
- CN202411095112.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-11-29
AI Technical Summary
The existing sealing nail defect detection algorithm has poor generalization performance when detecting multiple defects, and has low detection accuracy for small-size defects, making it difficult to effectively utilize the geometric characteristics of point cloud data, resulting in low defect detection accuracy.
采用基于改进图注意力机制的语义分割网络,通过向量注意力算子聚合局部特征,生成相关系数矩阵,并根据空间距离关系约束,获取语义分割网络的注意力权重矩阵,为边缘卷积运算过程 Additional normal vector feature encoding, semantic segmentation model is constructed and trained to identify defects in snails.
It improves the accuracy of defect detection, can identify potential defects that cannot be identified by traditional algorithms, enhances the geometric relationship capture ability of point cloud data, and improves the effect of small-size defect detection.
Smart Images

Figure CN119048449A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of defect detection technology, and in particular to a method, system, computer-readable storage medium and computer program product for sealing nail defect detection based on an improved graph attention mechanism. Background Art
[0002] When using traditional algorithms for existing sealing nail defect detection, the basis for determining defects is variable, and parameters need to be adjusted frequently according to working conditions and actual production needs. This brings a lot of difficulties to the machine vision detection algorithm in detecting and classifying multiple types of defects. For example, the Chinese patent with application number "202211166623.7" and name "A Sealing Nail Defect Detection Method and System" proposes a sealing nail defect detection algorithm based on the combination of YOLOv5 and 3D point cloud. When the detection model of this patented technology updates a certain type of defect in the defect detection model, it will affect the detection accuracy of the defect detection model for other types of defects, resulting in low accuracy in detecting sealing nail welding defects; therefore, the generalization performance of the existing algorithm is poor, which limits the deployment and application of machine vision detection methods.
[0003] At present, in the actual production process, there are several problems in the sealing nail defect detection project:
[0004] (1) During the sealing process, the probability of defective products is low due to the improvement of processing technology; this leads to a small number of samples of missing sealing nails, making it difficult to collect large-scale data;
[0005] (2) The sizes of various defects of existing sealing nails vary greatly, and some defects are small in size. A small number of defect detection methods that use 2D images alone, such as the Chinese patent with application number "202310581810.X" and titled "Detection method, storage medium and electronic device for sealing nail welding defects", may miss small defects;
[0006] (3) In order to achieve high-precision detection, the point cloud data of the sealing nails is collected during the detection process. However, the current classic deep learning methods cannot be directly applied to non-Euclidean data such as point clouds.
[0007] Therefore, how to extract useful features from point cloud pairs whose geometric features are not obvious or whose overlap is low and construct effective correspondences has become an urgent problem to be solved by those skilled in the art. Summary of the invention
[0008] The object of the present invention is to provide a method, system, computer-readable storage medium and computer program product for sealing nail defect detection based on an improved graph attention mechanism, so as to solve or at least partially solve the technical problems mentioned in the above background technology.
[0009] To achieve this object, the present invention adopts the following technical solutions:
[0010] In a first aspect, the present invention provides a method for sealing nail defect detection based on an improved graph attention mechanism, comprising:
[0011] Build a semantic segmentation network for segmenting defect areas;
[0012] Use the vector attention operator to aggregate local features, generate a correlation coefficient matrix, and constrain the correlation coefficient matrix according to the spatial distance relationship to obtain the attention weight matrix of the semantic segmentation network;
[0013] Add normal vector feature encoding to the edge convolution operation process to obtain the edge convolution operator of the semantic segmentation network;
[0014] Building a semantic segmentation model based on the semantic segmentation network, and using a pre-built sealing nail defect training set to train the semantic segmentation model to obtain network parameters of the semantic segmentation model;
[0015] The trained semantic segmentation model is used to detect or identify defects in sealing nails.
[0016] Optionally, before aggregating local features using a vector attention operator to generate a correlation coefficient matrix and constraining the correlation coefficient matrix according to a spatial distance relationship to obtain an attention weight matrix of a semantic segmentation network, the method further includes:
[0017] Construct a graph structure for the point cloud data of the sealing nail; specifically including:
[0018] Constructing a directed graph To represent the local features of point cloud data;
[0019] For each x i ∈X is the center point. According to the relative position relationship between the points, Construct the k-nearest neighbor graph of X;
[0020] Among them, υ={1,2,......,N} is the set of vertices, is a set of edges, k is the k value of the k nearest neighbor algorithm, and X is a set of point features in the point cloud data.
[0021] Optionally, the vector attention operator is used to aggregate local features, generate a correlation coefficient matrix, and the correlation coefficient matrix is constrained according to the spatial distance relationship to obtain the attention weight matrix of the semantic segmentation network, specifically including:
[0022] According to the k nearest neighbor graph, the center point x i and its neighboring points x j The characteristic difference (xj -x i ) as input, and use the subtraction relationship to generate the correlation coefficient matrix; the correlation coefficient matrix is calculated as follows: Among them, φ and ψ are two independent linear layers;
[0023] Constrain the correlation coefficient matrix according to the spatial distance relationship to obtain the attention weight matrix of the semantic segmentation network;
[0024] Among them, the spatial distance relationship expression is: p i For x i The spatial coordinates of j For x j The spatial coordinates of
[0025] The attention weight matrix of the semantic segmentation network is calculated as follows:
[0026]
[0027] Optionally, the process of adding normal vector feature encoding to the edge convolution operation process to obtain the edge convolution operator of the semantic segmentation network specifically includes:
[0028] Design the parameters of the convolution kernel of the graph convolution module and calculate the edge core operator of the graph convolution module; the calculation formula of the edge core operator is:
[0029] e ij =α m ·x i +β m ·(x j -x i ), α m and β m is the parameter of the convolution kernel;
[0030] Add normal vector feature encoding to the edge core operator; the calculation formula of the edge core operator is updated to
[0031] e ij’ =α m ·x i +β m ·(x j -x i )+δ(n j -n i );
[0032] Among them, δ is a nonlinear function, where n i and n j are the center points x i and neighboring points x j The normal vector of .
[0033] Optionally, constructing a semantic segmentation model based on the semantic segmentation network, and using a pre-constructed sealing nail defect training set to train the semantic segmentation model to obtain network parameters of the semantic segmentation model specifically includes:
[0034] According to the defect size in the sealing nail defect training set, the model training parameters are adjusted accordingly;
[0035] The constructed semantic segmentation model is trained using the sealing nail defect training set, and the most suitable network parameters are obtained through multiple trainings as the network parameters of the semantic segmentation model.
[0036] Optionally, adjusting the model training parameters accordingly according to the defect size in the sealing nail defect training set includes:
[0037] For the training samples with smaller defect sizes in the sealing nail defect training set, the weight l of the label ratio is added to the loss function n ; Weight l n Expressed as:
[0038]
[0039] Where C is the number of categories, x n is the prediction result output by the network, y is the label, and w c is the label weight; N is the total number of points in the dataset, n c is the total number of points of one type in the data set.
[0040] Optionally, before constructing a graph structure for the point cloud data of the sealing nail, the method further includes:
[0041] Collecting multiple sealing nail point cloud data and annotating them to construct a data set required for training and testing; the data set required for training and testing includes a sealing nail defect training set and a sealing nail defect detection test set;
[0042] Before using the trained semantic segmentation model to detect or identify defects of the sealing nail, the method further includes:
[0043] The trained semantic segmentation model is tested using the sealing nail defect detection test set. The test results are saved in the form of txt text and visualized using point cloud rendering.
[0044] Optionally, the collecting and labeling of a plurality of sealing nail point cloud data to construct a data set required for training and testing specifically includes:
[0045] Use 3D line scanning cameras to collect point cloud data of two types of sealing nails: flat nails and concave nails;
[0046] Use the RANSAC algorithm to segment the welding area of the point cloud data of the concave nail to obtain data samples;
[0047] Perform data enhancement on welding area samples, expand the number of data samples, and form a data set;
[0048] Use annotation tools to annotate data samples in the dataset and build the dataset required for training and testing.
[0049] Optionally, constructing a semantic segmentation network for segmenting defect areas specifically includes:
[0050] Based on the 2D image semantic segmentation network U-Net, a semantic segmentation network for segmenting defect areas is constructed;
[0051] The semantic segmentation network is an encoding-decoding structure, including: a graph attention module, a downsampling module, a graph convolution module and an upsampling module;
[0052] Among them, the graph attention module is the core feature extraction structure, which is used to encode the input features; the graph convolution module in the decoding layer can add the normal vector feature encoding of the point to enhance the features extracted by the encoding layer.
[0053] In a second aspect, the present invention provides a system for sealing nail defect detection based on an improved graph attention mechanism, comprising:
[0054] A network architecture module, used to build a semantic segmentation network for segmenting defect areas;
[0055] A data processing module, electrically connected to the network architecture module, for aggregating local features with a vector attention operator to generate a correlation coefficient matrix, and constraining the correlation coefficient matrix according to a spatial distance relationship to obtain an attention weight matrix of a semantic segmentation network; and for adding a normal vector feature encoding to an edge convolution operation process to obtain an edge convolution operator of a semantic segmentation network;
[0056] A model training module, electrically connected to the data processing module, for building a semantic segmentation model based on the semantic segmentation network, and training the semantic segmentation model using a pre-built sealing nail defect training set to obtain network parameters of the semantic segmentation model;
[0057] The defect detection terminal is electrically connected to the model training module and is used to detect or identify defects of the sealing nails using the trained semantic segmentation model.
[0058] In a third aspect, the present invention provides a computer-readable storage medium storing at least one instruction, characterized in that the instruction is loaded and executed by a processor to implement a method for sealing nail defect detection based on an improved graph attention mechanism as described above.
[0059] In a fourth aspect, the present invention provides a computer program product, including a computer program / instructions, characterized in that when the computer program / instructions are executed by a processor, a method for sealing nail defect detection based on an improved graph attention mechanism as described above is implemented.
[0060] Compared with the prior art, the present invention has the following beneficial effects:
[0061] The semantic segmentation network used in the present invention uses a subtraction relationship instead of a multiplication relationship to generate a correlation coefficient matrix, and uses a spatial distance relationship to constrain the correlation coefficient matrix, which can better capture the local geometric relationship of the point cloud and accurately segment the defect area; by adding normal vector feature encoding to the edge convolution operation process and capturing the deviation between the normal vectors of adjacent points, potential defects that traditional algorithms cannot identify can be identified, effectively improving the accuracy of defect detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0063] Figure 1 A flow chart of a method for sealing nail defect detection based on an improved graph attention mechanism provided in an embodiment of the present invention.
[0064] Figure 2 An architectural schematic diagram of a system for sealing nail defect detection based on an improved graph attention mechanism provided in an embodiment of the present invention.
[0065] In the figure:
[0066] 10. Network architecture module; 20. Data processing module; 30. Model training module; 40. Defect detection terminal. DETAILED DESCRIPTION
[0067] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0068] Embodiment 1:
[0069] Please refer to Figure 1 , Figure 1 A flow chart of a method for sealing nail defect detection based on an improved graph attention mechanism provided in an embodiment of the present invention, the method specifically includes:
[0070] Step 110: construct a semantic segmentation network for segmenting defect areas. Specifically:
[0071] Based on the 2D image semantic segmentation network U-Net, a semantic segmentation network for segmenting defect areas is constructed;
[0072] The semantic segmentation network is an encoding-decoding structure, including: graph attention module, downsampling module, graph convolution module and upsampling module;
[0073] Among them, the graph attention module is the core feature extraction structure, which is used to encode the input features; the graph convolution module in the decoding layer can add the normal vector feature encoding of the point to enhance the features extracted by the encoding layer.
[0074] It should be noted that the U-Net network is a paper published by Ronneberger et al. in MICCAI, a top journal of computer medical imaging in 2015. The title of the paper is "U-Net: Convolutional Networks for Biomedical Image Segmentation". The paper first proposed a U-shaped structure for semantic segmentation of images. The specific principle can be found in the original paper and will not be repeated here.
[0075] Before step 110, a preset step 101 is also included.
[0076] Step 101: constructing a graph structure for the point cloud data of the sealing nail; specifically comprising:
[0077] Constructing a directed graph To represent the local features of point cloud data;
[0078] For each x i ∈X is the center point. According to the relative position relationship between the points, Construct the k-nearest neighbor graph of X;
[0079] In step 101, υ = {1, 2, ..., N} is a set of vertices, is a set of edges, k is the k value of the k nearest neighbor algorithm, and X is a set of point features in the point cloud data.
[0080] The method for obtaining point cloud data in step 101 is specifically as follows:
[0081] Collect and annotate multiple sealing nail point cloud data to build the data set required for training and testing;
[0082] Use 3D line scanning cameras to collect point cloud data of two types of sealing nails: flat nails and concave nails;
[0083] Use the RANSAC algorithm to segment the welding area of the point cloud data of the concave nail to obtain data samples;
[0084] Perform data enhancement on welding area samples, expand the number of data samples, and form a data set;
[0085] Use annotation tools to annotate data samples in the dataset and build the dataset required for training and testing.
[0086] In this embodiment, the data sets required for training and testing include a data set for sealing nail defect training and a data set for sealing nail defect detection testing; for ease of description, the data set for sealing nail defect training is defined as a sealing nail defect training set, and the data set for sealing nail defect detection testing is defined as a sealing nail defect detection test set.
[0087] It is understandable that each data set should contain multiple different data. For example, the sealing nail defect training set should include point cloud data of various defects that may occur in actual production. Because when the data set is rich enough, the defect recognition accuracy of the model trained on it is higher.
[0088] Step 120: Use the vector attention operator to aggregate local features, generate a correlation coefficient matrix, and constrain the correlation coefficient matrix according to the spatial distance relationship to obtain the attention weight matrix of the semantic segmentation network.
[0089] Step 120 specifically includes:
[0090] According to the k nearest neighbor graph, the center point x i and its neighboring points x j The characteristic difference (x j -x i ) as input, and use the subtraction relationship to generate the correlation coefficient matrix; the correlation coefficient matrix is calculated as follows: Among them, φ and ψ are two independent linear layers;
[0091] Constrain the correlation coefficient matrix according to the spatial distance relationship to obtain the attention weight matrix of the semantic segmentation network;
[0092] Among them, the spatial distance relationship expression is: p i For x i The spatial coordinates of j For x j The spatial coordinates of
[0093] The attention weight matrix of the semantic segmentation network is calculated as:
[0094]
[0095] In this step, what is the K-nearest neighbor algorithm, that is, the K-Nearest Neighbor algorithm, referred to as the KNN algorithm, that is, given a training data set, for a new input instance, find the K instances closest to the instance in the training data set, that is, the K neighbors. If most of these K instances belong to a certain class, the input instance is classified into this class;
[0096] When it is impossible to determine which category the current point to be classified belongs to among the known categories, we can look at its location characteristics based on statistical theory, measure the weights of its neighbors, and classify or assign it to the category with a larger weight.
[0097] In this step, the vector attention operator is used to aggregate local features, the correlation coefficient matrix is generated using a subtraction relationship instead of a multiplication relationship, and the correlation coefficient matrix is constrained according to the spatial distance relationship, and finally the attention matrix is generated through Softmax activation, where Softmax is the activation function.
[0098] In this step, the linear layer is a basic layer in the neural network. Each node in this layer is connected to each node in the previous layer. Each connection has a weight to adjust the influence of the input data. The linear layer can help the network better understand the input data. It is usually used interchangeably with nonlinear activation functions (such as ReLU) and constitutes the basic component of the neural network.
[0099] Step 130: Add normal vector feature encoding to the edge convolution operation process to obtain the edge convolution operator of the semantic segmentation network. Specifically, it includes:
[0100] Design the parameters of the convolution kernel of the graph convolution module and calculate the edge core operator of the graph convolution module; the calculation formula of the edge core operator is:
[0101] e ij =α m ·x i +β m ·(x j -x i ), α m and β m is the parameter of the convolution kernel;
[0102] Add normal vector feature encoding to the edge core operator; the calculation formula of the edge core operator is updated to
[0103] e ij’ =αm ·x i +β m ·(x j -x i )+δ(n j -n i );
[0104] Among them, δ is a nonlinear function, where n i and n j are the center points x i and neighboring points x j The normal vector of .
[0105] This step uses a multi-layer perceptron (MLP), and then uses maximum pooling to aggregate the edge features extracted by edge convolution. Unlike other semantic segmentation tasks, in the sealing nail defect detection task, the normal vectors of the defective area to be segmented vary significantly. Ideally, the normal vectors of the deepest points in the defect-free area and the defective area are usually perpendicular to the XZ plane. By capturing the deviation between adjacent normal vectors, potential defects can be identified. Therefore, normal vector feature encoding is added to the edge convolution operation process to more efficiently extract local geometric information.
[0106] It is understandable that there are many nonlinear activation functions in MLP, which can be selected according to different application requirements. For example, the nonlinear activation function selected in this embodiment is ReLU, which has the following advantages:
[0107] (1) The calculation is relatively simple, just set the negative values in the input to zero and keep the positive values unchanged. This is faster than activation functions that involve complex calculations (such as sigmoid and tanh);
[0108] (2) ReLU does not generate exponential operations during the calculation process like sigmoid and tanh, so it converges faster;
[0109] (3) ReLU can effectively alleviate the gradient vanishing problem;
[0110] (4) Since ReLU will output all negative inputs as zero, it means that it can produce sparse activation (that is, some neurons remain inactive at the same time). Sparse activation can improve the efficiency and generalization ability of the network and alleviate overfitting.
[0111] In this embodiment, maximum pooling is a downsampling technique commonly used in convolutional neural networks. Its main function is to retain the most important features by selecting the maximum value in the local area while reducing the dimension and amount of calculation of the data. This technology helps to improve the robustness and generalization ability of the model, making the model less sensitive to small input changes. During the processing, the maximum pooling layer selects the maximum value in each local area of the input feature map as the output value. This process is similar to a sliding window moving on the input data and selecting the maximum value at each position. The maximum pool is used to reduce the spatial size of the data while maintaining the significance of the features, and improve the efficiency and performance of the entire network by simplifying the data representation and reducing the computational complexity.
[0112] In this embodiment, the basic network architecture of the semantic segmentation model is constructed with reference to the 2D image semantic segmentation network U-Net, which is a typical encoding-decoding structure. The network includes a graph attention module, a downsampling module, a graph convolution module and an upsampling module. Among them, the core feature extraction structure is the graph attention module, which encodes the input features. Then, the normal vector feature encoding of the additional point of the graph convolution module in the decoding layer is used to strengthen the features extracted by the encoding layer and improve the network's ability to extract local geometric information.
[0113] Step 140: construct a semantic segmentation model based on the semantic segmentation network, and use the pre-constructed sealing nail defect training set to train the semantic segmentation model to obtain network parameters of the semantic segmentation model.
[0114] Step 140 specifically includes:
[0115] According to the defect size in the sealing nail defect training set, the model training parameters are adjusted accordingly;
[0116] The constructed semantic segmentation model is trained using the sealing nail defect training set, and the most suitable network parameters are obtained through multiple trainings as the network parameters of the semantic segmentation model.
[0117] According to the defect size in the sealing nail defect training set, the model training parameters are adjusted accordingly, including:
[0118] For the training samples with smaller defect sizes in the sealing nail defect training set, the weight l of the label ratio is added to the loss function n The loss function is used to measure the difference between the predicted result and the actual result. In this embodiment, l n Expressed as:
[0119]
[0120] Among them, C is the number of categories, xn is the prediction result output by the network, y is the label, and w c is the label weight; N is the total number of points in the dataset, nc is the total number of points of one type in the data set.
[0121] After the semantic segmentation network constructed in step 110 is improved in steps 120 and 130, the new model is trained using the pre-constructed training set, the training parameters are adjusted, and label weights are added to the loss function for the problem of small defect sizes. The appropriate network parameters are found through multiple trainings to optimize the performance of the trained model.
[0122] It should be noted that since the defect area in the sealing nail data set is very small, in step 140, a weight network needs to be added to the loss function. Otherwise, during training, all points will tend to be predicted as defect-free to reduce losses, resulting in failure of network convergence.
[0123] Step 150: Test the trained semantic segmentation model using the sealing nail defect detection test set, save the test results in the form of txt text, and use point cloud rendering to visualize the test results.
[0124] The model trained in step 140 is used to perform testing using a sealing nail defect detection test set; the recognition result is saved in the form of a txt text, and can be visualized using point cloud rendering software.
[0125] In this step, tests are performed to determine whether the trained semantic segmentation model has achieved the expected goal. The sampling visualization is designed to facilitate staff to more intuitively inspect the test conditions and results.
[0126] Step 160 , judging whether the accuracy of defect detection based on the semantic segmentation model reaches the expectation according to the test results; if so, executing step 170 ; if not, executing step 180 .
[0127] Step 170: Use the trained semantic segmentation model to detect or identify defects of the sealing nails.
[0128] After the trained semantic segmentation model is tested using the sealing nail defect detection test set, the accuracy of defect detection can be used to determine whether the training of the semantic segmentation model has achieved the expected effect. The following Table 1 compares the recognition results of the defect detection algorithm of this embodiment and several other defect detection algorithms based on the same scene:
[0129] Table 1
[0130] algorithm Overall accuracy Average Class Precision Average Intersection Union PointNet++ 87.54 76.23 50.03 GACNet 89.97 83.26 56.44 PointTrans 98.66 91.74 73.13 Our 99.36 94.53 80.04
[0131] It can be seen from Table 1 that the semantic segmentation network used in this embodiment has higher recognition accuracy, can accurately segment the defect area, and effectively improve the accuracy of defect detection.
[0132] Step 180: reconstruct the sealing nail defect training set based on the test results.
[0133] When the training effect is not good, it indicates that there are omissions in the construction of the training set or the training volume is insufficient. At this time, the training set needs to be further enriched. For example, the defective content that cannot be recognized or recognized incorrectly in the test set can be added to the training set, and the samples can be further enhanced; then continue to train the semantic segmentation model.
[0134] It is understood that neural networks make predictions or classifications by learning patterns and regularities in data. During the training phase, the network receives input data, calculates the output through forward propagation, and then compares it with the actual results, using the backpropagation algorithm to adjust the weights and biases to reduce the prediction error. This process is repeated until the network can accurately predict or classify new, unseen data.
[0135] Embodiment 2:
[0136] Please refer to Figure 2 , Figure 2 An architectural schematic diagram of a system for sealing nail defect detection based on an improved graph attention mechanism provided in an embodiment of the present invention, the system comprising:
[0137] A network architecture module 10, for constructing a semantic segmentation network for segmenting defect areas;
[0138] The data processing module 20 is electrically connected to the network architecture module 10, and is used to aggregate local features using a vector attention operator to generate a correlation coefficient matrix, and constrain the correlation coefficient matrix according to the spatial distance relationship to obtain an attention weight matrix of the semantic segmentation network; and is also used to add a normal vector feature code to the edge convolution operation process to obtain an edge convolution operator of the semantic segmentation network;
[0139] The model training module 30 is electrically connected to the data processing module 20 and is used to construct a semantic segmentation model based on the semantic segmentation network, and train the semantic segmentation model using a pre-constructed sealing nail defect training set to obtain network parameters of the semantic segmentation model;
[0140] The defect detection terminal 40 is electrically connected to the model training module 30, and is used to detect or identify defects of the sealing pins using the trained semantic segmentation model.
[0141] As the construction method or process of the semantic segmentation network and the semantic segmentation model in the network architecture module 10, the data processing module 20 and the model training module 30 have been explained in detail in the first embodiment, they will not be repeated in this embodiment.
[0142] Based on the constructed and trained semantic segmentation model, the defect area can be accurately segmented and potential defects that traditional algorithms cannot identify can be identified, effectively improving the accuracy of defect detection.
[0143] Based on the same concept, an embodiment of the present invention also provides a computer-readable storage medium, in which at least one instruction is stored. The instruction is loaded and executed by a processor to implement a method for sealing nail defect detection based on an improved graph attention mechanism provided by an embodiment of the present invention.
[0144] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used or used in conjunction with an instruction execution system, device, or device.
[0145] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0146] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
[0147] Computer program code for performing the operations of the present invention may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
[0148] Based on the same concept, an embodiment of the present invention also provides a computer program product, including a computer program / instructions. When the computer program / instructions are executed by a processor, a method for sealing nail defect detection based on an improved graph attention mechanism provided by an embodiment of the present invention is implemented.
[0149] The computer program product may be loaded on a computer device, the components of which may include but are not limited to: one or more processors or processing units, a system memory, and a bus connecting different system components (including the system memory and the processing unit).
[0150] Computer devices typically include a variety of computer system readable media. These media can be any available media that can be accessed by the computer device, including volatile and non-volatile media, removable and non-removable media.
[0151] The system memory may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The computer device may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system may be used to read and write non-removable, non-volatile magnetic media. The computer program product has a set (e.g., at least one) of program modules that are configured to perform the functions of various embodiments of the present invention.
[0152] A program / utility having a set (at least one) of program modules may be stored, for example, in a memory, such program modules including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules generally perform the functions and / or methods of the embodiments described herein.
[0153] The computer device can also communicate with one or more external devices (such as keyboards, pointing devices, displays, etc.), and can also communicate with one or more devices that enable users to interact with the computer device, and / or communicate with any device (such as a network card, a modem, etc.) that enables the computer device to communicate with one or more other computing devices. This communication can be carried out through an input / output (I / O) interface. In addition, the computer device can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN) and / or a public network, such as the Internet) through a network adapter. As shown in the figure, the network adapter communicates with other modules of the computer device through a bus. It should be understood that other hardware and / or software modules can be used in conjunction with the computer device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.
[0154] The processing unit executes various functional applications and data processing by running programs stored in the system memory, such as implementing a method for sealing nail defect detection based on an improved graph attention mechanism provided in an embodiment of the present invention.
[0155] Those skilled in the art can understand that all or part of the steps of the above embodiments can be completed by hardware, or by a program to instruct the relevant hardware to complete, and the program can be stored in a computer-readable storage medium, and the storage medium mentioned above can be a read-only memory, a disk or an optical disk, etc. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0156] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for sealing nail defect detection based on an improved graph attention mechanism, characterized in that: include: Build a semantic segmentation network for segmenting defect areas; Use the vector attention operator to aggregate local features, generate a correlation coefficient matrix, and constrain the correlation coefficient matrix according to the spatial distance relationship to obtain the attention weight matrix of the semantic segmentation network; Add normal vector feature encoding to the edge convolution operation process to obtain the edge convolution operator of the semantic segmentation network; Building a semantic segmentation model based on the semantic segmentation network, and using a pre-built sealing nail defect training set to train the semantic segmentation model to obtain network parameters of the semantic segmentation model; The trained semantic segmentation model is used to detect or identify defects in sealing nails.
2. According to claim 1, a method for sealing nail defect detection based on an improved graph attention mechanism is characterized in that: The method of aggregating local features with a vector attention operator to generate a correlation coefficient matrix and constraining the correlation coefficient matrix according to the spatial distance relationship to obtain the attention weight matrix of the semantic segmentation network also includes: Construct a graph structure for the point cloud data of the sealing nail; specifically including: Constructing a directed graph To represent the local features of point cloud data; For each x i ∈X is the center point. According to the relative position relationship between the points, Construct the k-nearest neighbor graph of X; Among them, υ={1,2,......,N} is the set of vertices, is a set of edges, k is the k value of the k nearest neighbor algorithm, and X is a set of point features in the point cloud data.
3. According to claim 2, a method for sealing nail defect detection based on an improved graph attention mechanism is characterized in that: The vector attention operator is used to aggregate local features, generate a correlation coefficient matrix, and constrain the correlation coefficient matrix according to the spatial distance relationship to obtain the attention weight matrix of the semantic segmentation network, which specifically includes: According to the k nearest neighbor graph, the center point x i and its neighboring points x j The characteristic difference (x j -x i ) as input, and use the subtraction relationship to generate the correlation coefficient matrix; the correlation coefficient matrix is calculated as follows: Among them, φ and ψ are two independent linear layers; Constrain the correlation coefficient matrix according to the spatial distance relationship to obtain the attention weight matrix of the semantic segmentation network; Among them, the spatial distance relationship expression is: p i For x i The spatial coordinates of j For x j The spatial coordinates of The attention weight matrix of the semantic segmentation network is calculated as follows:
4. The method for sealing nail defect detection based on improved graph attention mechanism according to claim 3 is characterized in that: The process of adding normal vector feature encoding to the edge convolution operation process to obtain the edge convolution operator of the semantic segmentation network specifically includes: Design the parameters of the convolution kernel of the graph convolution module and calculate the edge core operator of the graph convolution module; the calculation formula of the edge core operator is: e ij =α m ·x i +β m ·(x j -x i ), α m and β m is the parameter of the convolution kernel; Add normal vector feature encoding to the edge core operator; the calculation formula of the edge core operator is updated to e ij’ =a m ·x i +b m ·(x j -x i )+δ(n j -n i );; Among them, δ is a nonlinear function, where n i and n j are the center points x i and neighboring points x j The normal vector of .
5. The method for sealing nail defect detection based on improved graph attention mechanism according to claim 4, characterized in that: The step of constructing a semantic segmentation model based on the semantic segmentation network, and training the semantic segmentation model using a pre-constructed sealing nail defect training set, and obtaining network parameters of the semantic segmentation model specifically includes: According to the defect size in the sealing nail defect training set, the model training parameters are adjusted accordingly; The constructed semantic segmentation model is trained using the sealing nail defect training set, and the most suitable network parameters are obtained through multiple trainings as the network parameters of the semantic segmentation model.
6. The method for sealing nail defect detection based on improved graph attention mechanism according to claim 5, characterized in that: The method of adjusting the model training parameters according to the defect size in the sealing nail defect training set includes: For the training samples with smaller defect sizes in the sealing nail defect training set, the weight l of the label ratio is added to the loss function n ; Weight l n Expressed as: Among them, C is the number of categories, xn is the prediction result output by the network, y is the label, and w c is the label weight; N is the total number of points in the dataset, n c is the total number of points of one type in the data set.
7. The method for sealing nail defect detection based on improved graph attention mechanism according to claim 5, characterized in that: Before constructing the graph structure for the point cloud data of the sealing nail, the method further includes: Collecting multiple sealing nail point cloud data and annotating them to construct a data set required for training and testing; the data set required for training and testing includes a sealing nail defect training set and a sealing nail defect detection test set; Before using the trained semantic segmentation model to detect or identify defects of the sealing nail, the method further includes: The trained semantic segmentation model is tested using the sealing nail defect detection test set. The test results are saved in the form of txt text and visualized using point cloud rendering.
8. The method for sealing nail defect detection based on improved graph attention mechanism according to claim 7, characterized in that: The collecting and labeling of multiple sealing nail point cloud data to construct a data set required for training and testing specifically includes: Use 3D line scanning cameras to collect point cloud data of two types of sealing nails: flat nails and concave nails; Use the RANSAC algorithm to segment the welding area of the point cloud data of the concave nail to obtain data samples; Perform data enhancement on welding area samples, expand the number of data samples, and form a data set; Use annotation tools to annotate data samples in the dataset and build the dataset required for training and testing.
9. The method for sealing nail defect detection based on improved graph attention mechanism according to claim 1, characterized in that: The step of constructing a semantic segmentation network for segmenting defect areas specifically includes: Based on the 2D image semantic segmentation network U-Net, a semantic segmentation network for segmenting defect areas is constructed; The semantic segmentation network is an encoding-decoding structure, including: a graph attention module, a downsampling module, a graph convolution module and an upsampling module; Among them, the graph attention module is the core feature extraction structure, which is used to encode the input features; the graph convolution module in the decoding layer can add the normal vector feature encoding of the point to enhance the features extracted by the encoding layer.
10. A system for sealing nail defect detection based on an improved graph attention mechanism, comprising: A network architecture module, used to build a semantic segmentation network for segmenting defect areas; A data processing module, electrically connected to the network architecture module, for aggregating local features with a vector attention operator to generate a correlation coefficient matrix, and constraining the correlation coefficient matrix according to a spatial distance relationship to obtain an attention weight matrix of a semantic segmentation network; and for adding a normal vector feature encoding to an edge convolution operation process to obtain an edge convolution operator of a semantic segmentation network; A model training module, electrically connected to the data processing module, for building a semantic segmentation model based on the semantic segmentation network, and training the semantic segmentation model using a pre-built sealing nail defect training set to obtain network parameters of the semantic segmentation model; The defect detection terminal is electrically connected to the model training module and is used to detect or identify defects of the sealing nails using the trained semantic segmentation model.
11. A computer-readable storage medium, wherein at least one instruction is stored in the storage medium, characterized in that: The instructions are loaded and executed by the processor to implement a method for sealing nail defect detection based on an improved graph attention mechanism as described in any one of claims 1-9.
12. Computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, a method for sealing nail defect detection based on an improved graph attention mechanism as described in any one of claims 1 to 9 is implemented.
Citation Information
Patent Citations
Point cloud semantic segmentation method and device, electronic equipment and storage medium
CN113516663A
Plate surface defect detection method and system based on image and point cloud data fusion
CN115496746A
Point cloud segmentation network based on combination of attention mechanism and double-graph convolution
CN117541795A
Large-scale point cloud semantic segmentation method based on density self-adaption and attention mechanism
CN117765258A
Communication equipment twin point cloud completion and defect detection method based on regional reasoning
CN122200138A
Cited By
Defect aggregation identification method and system based on point cloud semantic segmentation
CN120953265A
Nuclear power plant equipment appearance corrosion defect segmentation method based on attention mechanism and edge detection operator
CN122089721A
A nuclear power plant equipment appearance rust defect segmentation method based on attention mechanism and edge detection operator
CN122089721B
