A method for intelligent recognition of images of weld seams with the same joint

Abstract

This invention aims to provide an intelligent method for identifying identical weld seam images, belonging to the field of pipeline welding inspection technology, to accurately and efficiently prevent identical weld seam detection and falsification in welding films. It utilizes a deep learning model to identify and extract features from weld defect areas, then compares the feature data to identify films with identical weld seams. Simultaneously, it combines neural network architecture search technology to find the most suitable neural network structure for film deduplication. To optimize the network architecture search, this invention improves the autoencoder structure, adopting a strategy of adaptively adjusting the distribution of decoding results based on posterior distribution, effectively solving the problem of unequal-length encoding and further improving the accuracy and robustness of identical weld seam detection. This invention effectively combines deep learning and neural network architecture search to solve the problem of identical weld seam detection.

Description

Technical Field

[0001] This invention belongs to the field of pipeline welding inspection technology, and specifically relates to an intelligent recognition method for images of weld seams at the same joint. Background Technology

[0002] Welding technology is a critical component of engineering projects, and its quality control is essential for project success. X-ray films play a vital role in monitoring weld quality, effectively revealing potential weld defects such as porosity and slag inclusions. However, in actual engineering practice, due to cost-cutting efforts by some contractors, falsification is common. For example, multiple X-ray films of the same weld joint are submitted as films of different weld joints, severely hindering accurate weld quality assessment. Furthermore, the assessment process requires examining films from the same weld joint; although these films may not necessarily originate from falsification, accurate identification is still necessary.

[0003] Currently, traditional weld film duplication detection techniques heavily rely on manual inspection, resulting in low efficiency and a high risk of misjudgment. While technologies have emerged in recent years capable of identifying film forgery by analyzing the characteristics of adjacent weld areas, these methods are not ideal when dealing with full-weld forgery. Some researchers have attempted to perform film duplication by comparing the image similarity of the entire weld joint; however, this method faces a challenge: defect-free weld sections are often visually very similar, potentially leading to misjudgments when comparing the entire weld. Summary of the Invention

[0004] To address these challenges, this invention achieves several key innovations. First, the method focuses on comparing defective weld areas, rather than the entire weld, to improve the accuracy of duplicate detection. Second, the method is adaptable to different types and sizes of films; by adjusting the target detection algorithm, it can accurately locate defective weld areas on various films. Furthermore, regarding the problem of falsifying the entire weld joint on circumferential weld films, the method effectively handles the issue of closed-loop weld shapes. Even if the starting point of the image is anywhere on the loop, the method can effectively align and compare two circumferential weld films, especially when their starting points may be inconsistent.

[0005] To further improve the accuracy and efficiency of plagiarism detection, this invention utilizes Neural Architecture Search (NAS) technology to optimize the application of deep learning in the identification of weld seam films. NAS is an automated machine learning technique that optimizes network structure and parameters to achieve superior performance on specific tasks. By using NAS, the optimal neural network structure best suited for a specific weld seam film plagiarism detection task can be found, thereby improving the accuracy and robustness of plagiarism detection.

[0006] In traditional neural network architecture search, to enable optimization algorithms to handle discrete, discontinuous network structure parameters, they need to be encoded into a continuous form. Since different network architectures have varying numbers and lengths of parameters, an "unequal length" encoding problem arises. Handling strategies include zero-padding and autoencoders. Zero-padding is simple but may introduce invalid information, affecting model learning; autoencoders can effectively encode deep features of the original data, but may encounter problems such as range overflow or uneven result distribution during decoding. To address this issue, this invention employs a strategy based on posterior distribution to adaptively adjust the distribution of decoding results to improve the autoencoder. This improvement aims to adaptively adjust and optimize the distribution of decoding results, making it closer to the distribution of actual data, thereby effectively solving the problems of range overflow and uneven distribution of decoding results, and further improving the accuracy and robustness of neural network architecture search results.

[0007] In the process of implementing the technical solution of this invention, the inventors discovered that: on the one hand, by using deep learning technology, a metric model can be created that can perform in-depth comparison of weld areas with key points, thereby improving the efficiency of weld quality assessment. On the other hand, by using neural architecture search technology, combined with an improved autoencoder and a multi-fidelity evaluation strategy, it can more efficiently find the optimal network architecture in a vast potential design space. These two key discoveries not only solve the problem of film duplication detection but also improve the efficiency of network architecture search, greatly enhancing the application value of this invention in preventing welding film fraud and improving the accuracy of weld quality assessment.

[0008] The purpose of this invention is to provide a welding film deduplication and quality assessment method based on deep learning and neural architecture search, addressing the problems of high manpower consumption, high false positive rate, and insufficient accuracy of existing deep learning methods in film deduplication scenarios in practical applications. This method can accurately identify and compare films taken on the same weld joint, preventing welding film forgery and effectively improving the accuracy of weld quality assessment. Furthermore, by combining improved autoencoder results and a low-fidelity evaluation strategy, this method can efficiently search and evaluate the optimal network architecture in a large-scale potential design space, thereby improving the efficiency of network architecture search.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0010] This invention provides an intelligent recognition method for images of weld seams with the same joint, the method comprising:

[0011] Step 1: Build the dataset;

[0012] Construct a weld area detection dataset: Use weld X-ray images to annotate the weld area information and organize it into the corresponding dataset format;

[0013] A key point detection dataset is constructed, and weld feature instance information with memory characteristics in the weld area is annotated using weld X-ray film images and organized into the corresponding dataset format.

[0014] A keypoint measurement dataset is constructed using paired original films. Pairs of weld seam region images containing keypoints are extracted, and each pair of images is stored as a category. These categories are then divided into training and testing categories in an 8:2 ratio. A triplet dataset is constructed based on these categories; where, for each category c... i Choose an image as anchor point A, then choose another image in the same category as positive sample P, and randomly select an image from other categories as negative sample N; then construct triplet data {(A, P, N)}.

[0015] Step 2: Train the weld area detection model: Use the target detection algorithm to train on the weld area detection dataset to obtain the weld area detection model;

[0016] Step 3: Train the keypoint detection model: Use the object detection algorithm to train on the keypoint detection dataset to obtain the keypoint detection model;

[0017] Step 4: Keypoint Feature Extraction Model Backbone Network Architecture Search: Design a neural network architecture search strategy and use an efficient evaluation strategy to obtain a feature extraction model suitable for measuring the similarity of keypoints on the negative. The specific steps are as follows:

[0018] Step 4-1: Define the search space Ω: Define the set of possible network structures and hyperparameters, where the search space Ω includes, but is not limited to, the number of network layers L, the type of each layer T, and the size of the filter F, i.e., Ω = {L, T, F, ...};

[0019] Step 4-2: Design the architecture encoding method E: Based on the definition of the search space Ω, use an encoding strategy to transform the network architecture into a computer-processable form E(Ω), wherein the encoding strategy includes, but is not limited to, integer-based encoding methods and autoencoder-based encoding methods;

[0020] Step 4-3: Select a search strategy: Determine a strategy for searching the architecture space, wherein the search strategy includes, but is not limited to, grid search, random search, evolutionary algorithms, and reinforcement learning;

[0021] Step 4-4: Define a performance evaluation metric: Define a performance evaluation metric to measure the performance of each network architecture. The evaluation metric includes, but is not limited to, accuracy, loss function, and F1 score.

[0022] Steps 4-5: Training and evaluating network architectures: For each network architecture, train the model using the triplet dataset of the training categories obtained in Step 1, and evaluate it using the triplet dataset of the test categories.

[0023] Steps 4-6: Update the search space Ω′: Based on the network architecture evaluation results, update the search space Ω and retain the network architectures or hyperparameter combinations that perform well; if the predetermined stopping condition is met, stop the experiment and select the best architecture found as the search result to obtain the optimal architecture; otherwise, continue to execute steps 4-5 and 4-6.

[0024] Step 5: Keypoint similarity measurement model training: Using the selected optimal architecture, train the model on the keypoint similarity dataset to obtain the keypoint similarity measurement model;

[0025] Step 6: Identifying Films with the Same Image: Select a film to be tested as the query film and compare it sequentially with each film target in the film database. Based on the calculated overall similarity, recommend the films with the highest similarity to the query film for the evaluator's reference. If films with the same image are found, further processing is performed; otherwise, the current query film is added to the film database, and its key point detection information is saved. The specific steps are as follows:

[0026] Step 6-1: Weld seam area detection on film: Using the model obtained in step 2, the weld seam area is extracted from the query film image to obtain the weld seam area image;

[0027] Step 6-2: Key Point Detection on Film: Using the model obtained in Step 3, key point instances in the query film image are detected to obtain the coordinate information of key point instances in the weld area image; whereby the key points detected in the query film are represented as {q1, q2, ..., q...} n};

[0028] Step 6-3: Obtain the image information of the target film from the film library: This includes the key point information of the target film, represented as {t1, t2, ..., t...}. n}

[0029] Step 6-4: Organize Keypoints: Following a specific strategy, perform a binary match between the keypoints detected in the query film and the keypoints detected in the target film; each keypoint in the query film is matched with at most one keypoint in the target film, and each keypoint in the target film can only be matched with at most one keypoint in the query film; the matching combination can be represented as a set, where each matching combination contains one keypoint in the query film and one keypoint in the target film, denoted as {(q i, t j )}, where i represents the index of the key point of the query negative, and j represents the index of the key point of the target negative; the set of matching combinations can be represented as: Where j1, j2, ..., j k It is the index in the target negative that matches the key point of the corresponding query negative, and each key point of the target negative matches at most one key point of the query negative;

[0030] Step 6-5: Keypoint Pair Similarity Calculation: Calculate the similarity S of the matched keypoint pairs using the keypoint similarity comparison model obtained in Step 5. similarity The formula for calculating image similarity is as follows:

[0031]

[0032] in, Let i be the feature vector of the i-th pair of matching keypoints in the query. Let be the feature vector of the i-th pair of matching keypoints to target;

[0033] Step 6-6: Overall Similarity Calculation of Negatives: The similarity scores of each pair of keypoints are integrated and statistically analyzed to obtain the overall similarity between the query negative and the target negative. The calculation formula is as follows:

[0034] C total =∑S similarity

[0035] Finally, based on the overall similarity, negatives with high similarity to the search negatives are recommended for the film critics to use as a reference and for further processing.

[0036] Furthermore, the "key points" include, but are not limited to, weld beads, fish scale patterns, and other significant weld features.

[0037] Furthermore, the target detection model includes, but is not limited to, neural networks used for target detection such as Faster-rcnn and YOLO.

[0038] Furthermore, the metric model used to measure image similarity is a deep learning-based triplet network. This network structure consists of three parallel convolutional neural networks that share the same parameters. Each network corresponds to one member of a triplet: an anchor sample, a positive sample, and a keypoint from a negative sample. This network receives three input images, performs forward propagation on each image, and then calculates their feature vectors. The similarity between images is represented by the Euclidean distance between their feature vectors, calculated as follows:

[0039] distance = ||feature i -feature j || 2

[0040] The training of the metric model is performed by calculating the triplet loss function. Specifically, the triplet loss function is defined as:

[0041] L(A,P,N)=max(||f(A)-f(P)|| 2 -||f(A)-f(N)|| 2 +margin, 0)

[0042] Here, f(x) represents the features extracted by the convolutional neural network, and margin is a threshold used to ensure that there is at least a certain distance between positive and negative samples.

[0043] Furthermore, the design of the backbone network of the key point feature extraction model includes, but is not limited to, methods such as using automatic search and optimization techniques like Neural Architecture Search (NAS), manually designing or combining existing neural network architectures; wherein,

[0044] Furthermore, the efficient evaluation strategies include, but are not limited to: using parameter sharing strategies, using multi-fidelity evaluation strategies, and using learning curve prediction strategies.

[0045] Furthermore, the method for detecting weld seam areas on film includes, but is not limited to, target detection technology, image segmentation technology, etc.

[0046] Furthermore, the key point similarity calculation method includes, but is not limited to, using cosine similarity, Euclidean distance, etc.

[0047] Based on the above technical solution, the beneficial effects of the present invention are as follows:

[0048] This invention proposes an intelligent image recognition method for weld seams with the same joint, which not only improves the accuracy and efficiency of weld seam film detection but also effectively prevents the falsification of welding films, ensuring the welding quality of engineering projects. Furthermore, by using autoencoder encoding and multi-fidelity evaluation methods, the efficiency of network architecture search is improved, reducing manual intervention and saving significant human and material resources. This method has high practical value and broad application prospects. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 It is a schematic diagram of the weld seam, including the weld seam text area, the weld seam area, and examples of key points.

[0051] Figure 2 This is a schematic diagram of key points extracted from the film of the same weld seam;

[0052] Figure 3 This is a schematic diagram of an intelligent image recognition method for weld seams with the same joint provided in an embodiment of the present invention;

[0053] Figure 4 This is a flowchart of a neural network architecture search method based on an improved autoencoder provided in an embodiment of the present invention;

[0054] Figure 5 This is a schematic diagram of an improved self-encoder structure provided in an embodiment of the present invention;

[0055] Figure 6 This is a flowchart of an intelligent image recognition method for weld seams at the same joint provided by an embodiment of the present invention; Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0057] This embodiment provides a method for inspecting radiographs of weld seams at the same joint, the process of which is as follows: Figure 3 As shown, the specific implementation includes the following steps:

[0058] Step 1: Building the Dataset; In the initial stage of building the dataset, the primary task is to carefully analyze and understand the complexity of the background environment of industrial weld film. For example... Figure 1 As shown, weld film can be divided into two main areas: the type area and the weld area. The type area usually contains some fixed, identifiable markings or text, while the weld area is the focus of attention. This area contains several key features with significant memorability characteristics, which will become important criteria for determining and analyzing whether two films are from the same plate.

[0059] Construct a weld area detection dataset, use weld X-ray images to annotate the weld area information, and organize it into COCO data format;

[0060] A key point detection dataset is constructed, and weld feature instances with memory characteristics, such as weld beads, fish scale patterns and other significant weld features, are annotated in weld X-ray film images and organized into COCO data format.

[0061] A keypoint measurement dataset is constructed using paired films. Pairs of weld seam region images containing keypoints are extracted, and each pair of images is stored as a category. Some paired weld seam region images containing keypoints are shown below. Figure 2 As shown, these categories are divided into training and testing categories in an 8:2 ratio. A triplet dataset is then constructed based on these categories.

[0062] Specifically, for each category c i (Where i represents the category index), select an image as anchor point A, then select another image from the same category as positive sample P, and randomly select an image from other categories as negative sample N. Then construct the triplet data {(A, P, N)}.

[0063] Step 2: Train the weld area detection model;

[0064] A weld area detection model is obtained by training an object detection algorithm on a weld area detection dataset.

[0065] Specifically, the weld area dataset is divided into training and test sets in an 8:2 ratio. The Faster-R-CNN object detection algorithm is trained on the training set, and the generalization ability of the model is improved by randomly cropping and flipping the training set. The SGD optimizer is used to continuously train and iterate. After each generation of training is completed, it is tested on the test set. The model with the highest Map value on the test set is saved, and the weld area detection model is obtained.

[0066] Step 3: Training the key point detection model.

[0067] A keypoint detection model is obtained by training an object detection algorithm on a keypoint instance detection dataset.

[0068] Specifically, firstly, local histogram enhancement is performed on all data in the keypoint instance detection dataset. Then, the dataset is divided into training and test sets in an 8:2 ratio. The Faster R-CNN object detection algorithm is trained on the training set, and the model's generalization ability is improved by randomly pruning and flipping the training set. Using the SGD optimizer, the model is continuously trained iteratively. After each generation of training, it is tested on the test set. The model with the highest Map value on the test set is saved, resulting in the keypoint detection model.

[0069] Step 4: Keypoint Feature Extraction Model Backbone Network Architecture Search. Using neural network architecture methods, the optimal architecture for the feature extraction model suitable for measuring the similarity of keypoints on the negative is obtained. The neural network architecture search method is as follows: Figure 4 As shown, the specific steps are as follows:

[0070] Step 4-1: Define the search space Ω: In the search process of the backbone network architecture of the key point feature extraction model, the search space Ω is defined first. In this embodiment, the search space depends on the DenseNet network structure and the parameters such as the kernel size K of the search convolutional layer, the number of dense blocks B, and the number of dense layers L. Wherein, K∈{3, 5, 7, 9, 11}; B∈{3, 4, 5}; L∈{3, 4, ..., 64}.

[0071] Step 4-2, Design Architecture Encoding Method E: This embodiment uses an autoencoder to encode the network architecture. The autoencoder consists of two parts: an encoder and a decoder. The encoder is a four-layer fully connected neural network, with the last layer using the Sigmoid activation function to encode the input data into a fixed-length hidden layer of length 4, with values ​​ranging from 0 to 1. The decoder has a similar structure to the encoder, transforming the encoded data back to the original data dimension.

[0072] Training the autoencoder: 100,000 architecture parameters are generated as training samples by sampling in the defined search space Ω using Monte Carlo methods. The mean squared error (MSE) is used as the loss function and the AdamW optimizer is used for optimization. The MSE loss between the input data and the decoded data is calculated, and the parameters of the autoencoder are updated through backpropagation. The autoencoder consists of two parts, namely the encoder E and the decoder D. The training is iterated until the MSE of the autoencoder is ≤ 0.1.

[0073] Adjust the decoder's output distribution using posterior probabilities: Randomly generate 1 million latent vectors in the hidden layer, and represent these vectors as... Then, these latent vectors are input into the trained decoder D to obtain a series of decoding results. Where x i =D(z) i ), i = 1, 2, ..., 10 6 In order to adjust the output distribution of the decoder, a binning operation is performed, which converts the continuous values ​​of the decoding result into discrete bins.

[0074] Specifically, suppose that the value range of element x1 in the decoding result X is [x min x max This interval can be divided into B boxes, and the range of each box is...

[0075] [(xmax -x min ) / B] j +x min ≤x≤[(x max -x min ) / B] j+1 +x min

[0076] Where j∈{0, 1, ..., B-1}; This method converts the continuous values ​​of the decoding result into discrete bins for effective adjustment of the data distribution. The improved autoencoder structure based on the above steps is as follows: Figure 5 As shown.

[0077] Step 4-3: Select search strategy: Select the Particle Swarm Optimization (PSO) algorithm as the strategy for network architecture search, search for the optimal architecture in the latent space, and decode the searched latent vectors into network architecture parameters through the decoder.

[0078] Specifically, using the particle search network architecture parameters from the particle swarm optimization (PSO) algorithm, the coordinate information of each particle can represent a vector in the latent space during the PSO algorithm iteration. This vector can be transformed into candidate network architecture parameters through the aforementioned decoding process. Let the position of particle i in the particle swarm be p. i (t), velocity v i The individual extreme value is pbest(t), and the population extreme value is gbest(t), where t represents the number of iterations in the search. In each iteration, the PSO algorithm updates the particle's position and velocity according to the following formula:

[0079] v i (t+1)=ωv i (t)+c1r1(pbest i (t)-p i (t))+c2r2(gbest(t)-p i (t))

[0080] p i (t+1)=p i (t)+v i (t+1)

[0081] Where ω is the inertia weight, c1 and c2 are learning factors, and r1 and r2 are random numbers between (0, 1).

[0082] Step 4-4: Training and Evaluating the Network Architecture: The model's accuracy is used as the performance evaluation metric. During the search process, the error rate is used as the fitness function for PSO, and this error rate is then used to guide the PSO algorithm's search process. The fitness function calculation formula is as follows:

[0083] Fitness(x) = 1 - accuracy(x)

[0084] Here, accuracy(x) represents the accuracy of the candidate network architecture evaluated on the test set after training. Specifically, 20% of the data in the training portion of the keypoint metric dataset in step 1 is randomly selected as the training set, and all the data in the test portion of the keypoint metric dataset in step 1 is used as the test set. All candidate architectures are trained on the newly constructed metric dataset using the SGD optimizer and triplet loss function. Each candidate model is trained for 20 epochs, and its prediction accuracy is tested on the test set after each epoch: For each pair of triples (A, P, N) in the test set, features are extracted using the trained model, and the extracted feature vectors are denoted as f(A), f(P), and f(N). The Euclidean distance between f(A) and f(P) and the Euclidean distance between f(A) and f(N) are calculated. If the distance from f(A) to f(P) is less than the distance from f(A) to f(N), the prediction is considered correct; otherwise, it is considered incorrect. The result of the epoch with the highest accuracy out of 20 epochs is used as the final result. The formula for calculating the Euclidean distance is as follows:

[0085] distance = ||feature i -feature j || 2

[0086] Steps 4-5: Update the search space Ω′: Based on the evaluation results, update the search space Ω, retaining well-performing network architectures or hyperparameter combinations. If the number of iterations reaches 20, stop the experiment; otherwise, train and evaluate the network architecture again, adjusting the particles in the search space based on the evaluation results. The best-performing architecture found is ultimately selected as the optimal architecture for subsequent training of the keypoint similarity comparison model.

[0087] Step 5: Training the key point similarity comparison model.

[0088] Specifically, using the optimal architecture selected in step 4, the model is trained on the keypoint similarity dataset. The training set consists of triples representing training categories, and the test set consists of triples representing test categories. The model's generalization ability is improved by randomly pruning and flipping the training set. Using the SGD optimizer and triplet loss function, the model is trained iteratively. After each generation of training, it is tested on the test set, and the model with the highest accuracy on the test set is saved. The accuracy calculation is the same as described in steps 4-4. This yields the keypoint similarity comparison model. The keypoint similarity measurement model is obtained through training iterations.

[0089] Step 6: Identification of Negatives with Similar Gravity. Select one negative to be tested as the query negative and compare it sequentially with each negative in the negative database. Based on the calculated overall similarity, recommend the three negatives with the highest similarity to the query negative for the evaluator's reference. The flowchart for identifying negatives with similar gravity is shown below. Figure 6 As shown.

[0090] Specifically, the determination of the same-hole film needs to go through the weld area inspection stage, the key point inspection stage, the key point organization stage, and the key point measurement stage.

[0091] Step 6-1: In the weld area detection stage, using the weld detection model obtained in Step 2, input the query image into the model to extract the weld area and obtain the weld area image of the query image.

[0092] Step 6-2: In the key point detection stage, using the key point detection model obtained in Step 3, the weld area image of the query image is input into the model to detect key point instances, obtaining the coordinate information of key point instances in the weld area image. The key points detected in the query image are represented as {q1, q2, ..., q...} n};

[0093] Step 6-3: Obtain the image information of the target film from the film library, thereby obtaining the key point instance coordinate information in the target image;

[0094] Step 6-4: In the keypoint organization stage, based on the keypoint location information in the query and target images, perform binary matching on all keypoints in the query and target to obtain the optimal matching result. During the matching process, a pair of keypoints is used as anchor points, and all other keypoint pairs are traversed sequentially. For each pair of keypoints, the matching algorithm calculates the distance and angle from the center point of all keypoints in the query to the center point of the anchor point, and calculates the distance and angle from the center point of all keypoints in the target to the center point of their corresponding anchor point. Let the coordinates of the center point of the keypoint be (x1, y1), and the coordinates of the center point of the anchor point be (x2, y2), then the formulas for calculating the distance and angle between them are as follows:

[0095]

[0096]

[0097] If the difference between the distance and angle from the first keypoint in the query to its anchor point and the distance and angle from a keypoint in the target to its corresponding anchor point meets a pre-defined threshold, then this is considered a possible match. The algorithm iterates through all possible anchor point combinations and calculates their matching degree. Finally, it selects the group with the largest number of matches as the final matching result and returns it as the algorithm's output.

[0098] Step 6-5: In the keypoint pair similarity calculation stage, the similarity S of the matched keypoint pairs is calculated using the keypoint similarity comparison model obtained in Step 5. similarity The formula for calculating image similarity is as follows:

[0099]

[0100] in, Let i be the feature vector of the i-th pair of matching keypoints in the query. Let be the feature vector of the i-th pair of matching keypoints to target.

[0101] Step 6-6: Calculate the overall similarity between the two films: Sum the similarity scores of each pair of keypoints to obtain the overall similarity between the query film and the target film. Let C be the function for calculating this overall similarity, then the overall similarity can be expressed as:

[0102] C total =∑S similarity

[0103] Finally, based on the overall similarity, the three negatives with the highest similarity to the queried negative are recommended for the reviewer's reference. If a negative with the same aperture is found, further processing is performed. Otherwise, the current queried negative is added to the negative library, and its keypoint detection information is saved.

[0104] The above description is merely a specific embodiment of the present invention. Any feature disclosed in this specification may be replaced by other equivalent or similar features unless otherwise specified. All disclosed features, or steps in all methods or processes, may be combined in any way except for mutually exclusive features and / or steps.

Claims

1. A method for intelligent recognition of images of weld seams, characterized in that, Includes the following steps: Step 1: Build the dataset; Construct a weld area detection dataset: Use weld X-ray images to annotate the weld area information and organize it into the corresponding dataset format; A key point detection dataset is constructed, and weld feature instance information with memory characteristics in the weld area is annotated using weld X-ray film images and organized into the corresponding dataset format. A keypoint measurement dataset is constructed using paired original films. Pairs of weld seam region images containing keypoints are extracted, and each pair of images is stored as a category. These categories are then divided into training and testing categories in an 8:2 ratio. A triplet dataset is constructed based on these categories; where, for each category c... i Choose an image as anchor point A, then choose another image from the same category as positive sample P, and randomly select an image from other categories as negative sample N; then construct triplet data {(A,P,N)}; Step 2: Train the weld area detection model: Use the target detection algorithm to train on the weld area detection dataset to obtain the weld area detection model; Step 3: Train the keypoint detection model: Use the object detection algorithm to train on the keypoint detection dataset to obtain the keypoint detection model; Step 4: Keypoint Feature Extraction Model Backbone Network Architecture Search: Design a neural network architecture search strategy and use an efficient evaluation strategy to obtain a feature extraction model suitable for measuring the similarity of keypoints on the negative. The specific steps are as follows: Step 4-1: Define the search space Ω: Define the set of network structure and hyperparameters, where the search space Ω includes, but is not limited to, the number of network layers L, the type of each layer T, and the size of the filter F, i.e., Ω = {L, T, F, ...}; Step 4-2: Design the architecture encoding method E: Based on the definition of the search space Ω, use an encoding strategy to transform the network architecture into a computer-processable form E(Ω), wherein the encoding strategy includes, but is not limited to, integer-based encoding methods and autoencoder-based encoding methods; Step 4-3: Select a search strategy: Determine the strategy for searching the architecture space, wherein the search strategy includes, but is not limited to, grid search, random search, evolutionary algorithms, and reinforcement learning; Step 4-4: Define a performance evaluation metric: Define a performance evaluation metric to measure the performance of each network architecture. The evaluation metric includes, but is not limited to, accuracy, loss function, and F1 score. Steps 4-5: Training and evaluating network architectures: For each network architecture, train the model using the triplet dataset of the training categories obtained in Step 1, and evaluate it using the triplet dataset of the test categories. Steps 4-6: Update the search space Ω′: Based on the network architecture evaluation results, update the search space Ω and retain the network architectures or hyperparameter combinations that perform well; if the predetermined stopping condition is met, stop the experiment and select the best architecture found as the search result to obtain the optimal architecture; otherwise, continue to execute steps 4-5 and 4-6. Step 5: Keypoint similarity measurement model training: Using the selected optimal architecture, train the model on the keypoint similarity dataset to obtain the keypoint similarity measurement model; Step 6: Identifying Films with the Same Image: Select a film to be tested as the query film and compare it sequentially with each film target in the film database. Based on the calculated overall similarity, recommend the films with the highest similarity to the query film for the evaluator's reference. If films with the same image are found, further processing is performed; otherwise, add the current query film to the film database and save its key point detection information. The specific steps are as follows: Step 6-1: Weld seam area detection on film: Using the model obtained in step 2, the weld seam area is extracted from the query film image to obtain the weld seam area image; Step 6-2: Key Point Detection on Film: Using the model obtained in Step 3, key point instances in the query film image are detected to obtain the coordinate information of key point instances in the weld area image; whereby the key points detected in the query film are represented as {q1,q2,…,q n }; Step 6-3: Obtain the image information of the target film from the film library: This includes the key point information of the target film, represented as {t1, t2, ..., t...} n } Step 6-4: Organize Keypoints: Following a specific strategy, perform a binary match between the keypoints detected in the query film and the keypoints detected in the target film; each keypoint in the query film is matched with at most one keypoint in the target film, and each keypoint in the target film can only be matched with at most one keypoint in the query film; the matching combination is represented as a set, where each matching combination contains one keypoint in the query film and one keypoint in the target film, denoted as {(q i ,t j )}, where i represents the index of the key point of the query negative, and j represents the index of the key point of the target negative; the set of matching combinations is represented as: Where j1, j2, ..., j k It is the index in the target negative that matches the key point of the corresponding query negative, and each key point of the target negative matches at most one key point of the query negative; Step 6-5: Keypoint Pair Similarity Calculation: Calculate the similarity S of the matched keypoint pairs using the keypoint similarity comparison model obtained in Step 5. similarity The formula for calculating image similarity is as follows: in, Let i be the feature vector of the i-th pair of matching keypoints in the query. Let be the feature vector of the i-th pair of matching keypoints to target; Step 6-6: Overall Similarity Calculation of Negatives: The similarity scores of each pair of keypoints are integrated and statistically analyzed to obtain the overall similarity between the query negative and the target negative. The calculation formula is as follows: C total =∑s similarity Finally, based on the overall similarity, negatives with high similarity to the search negatives are recommended for the film critics to use as a reference and for further processing. The search space Ω in step 4-1 uses the DenseNet network structure, and searches for the kernel size K of the convolutional layer, the number of dense blocks B, the number of dense layers L, and other parameters; where K∈{3,5,7,9,11}; B∈{3,4,5}; L∈{3,4,…,64}. In step 4-2, the architecture encoding method is to use an autoencoder to encode the network architecture. The specific steps of the network architecture encoding are as follows: the autoencoder consists of two parts: an encoder and a decoder; the encoder is a four-layer fully connected neural network, and the last layer uses the Sigmoid activation function to encode the input data into a fixed-length hidden layer with a value range of 0 to 1; the structure of the decoder is similar to that of the encoder, and it inversely transforms the encoded data back to the dimension of the original data. Training the autoencoder: 100,000 architecture parameters are generated by sampling in the defined search space Ω using Monte Carlo methods as training samples. The mean squared error is used as the loss function and the AdamW optimizer is used for optimization. The MSE loss between the input data and the decoded data is calculated, and the parameters of the autoencoder are updated through backpropagation. The autoencoder consists of two parts, namely encoder E and decoder D. The training is iterated until the MSE of the autoencoder is <0.

1. Adjust the decoder's output distribution using posterior probabilities: Randomly generate 1 million latent vectors in the hidden layer, and represent these vectors as... Then, these latent vectors are input into the trained decoder D to obtain a series of decoding results. Where x i =D(z) i ), i = 1, 2, ..., 10 6 In order to adjust the output distribution of the decoder, a binning operation is performed to convert the continuous values ​​of the decoding result into discrete bins; Specifically, suppose that the value range of element x1 in the decoding result X is [x min ,x max Divide this interval into B boxes, then the range of each box is... [(x max -x min ) / B] j +x min ≤x≤[(x max -x min ) / B] j+1 +x min Where j∈{0,1,…,B-1}; this method converts the continuous values ​​of the decoding results into discrete bins to effectively adjust the data distribution; The search strategy in step 4-3 uses the particle swarm optimization algorithm. The specific steps are as follows: The network architecture parameters are searched using particles in the particle swarm. During the iteration of the particle swarm optimization algorithm, the coordinate information of each particle represents a vector in the latent space. This vector is transformed into candidate network architecture parameters through the above decoding process. Let the position of particle i in the particle swarm be p. i (t), velocity v i The individual extreme value is pbest(t), and the swarm extreme value is gbest(t), where t represents the number of iterations in the search. In each iteration, the particle swarm optimization algorithm updates the position and velocity of the particles according to the following formula: v i (t+1)=ωv i (t)+c1r1(pbest i (t)-p i (t))+c2r2(gbest(t)-p i (t)) p i (t+1)=p i (t)+v i (t+1) Where ω is the inertia weight, c1 and c2 are learning factors, and r1 and r2 are random numbers between (0,1); In step 4-4, the model's accuracy is used as the performance evaluation metric during the training and evaluation of the network architecture. During the search process, the error rate is used as the fitness function of the particle swarm optimization algorithm. This error rate is then used as the fitness function to guide the search process of the particle swarm optimization algorithm. The fitness function is calculated as follows: Fitness(x) = 1 - accuracy(x) Wherein, accuracy(x) represents the accuracy of the candidate network architecture evaluated on the test set after training. Specifically, 20% of the data in the training part of the keypoint metric dataset is randomly selected as the training set, and all the data in the test part of the keypoint metric dataset is used as the test set. All candidate architectures are trained on the newly constructed metric dataset using the SGD optimizer and the ternary loss function. Each candidate model is trained for 20 epochs, and its prediction accuracy is tested on the test set after each epoch. For each pair of triples (A, P, N) in the test set, feature extraction is performed using the trained model. The extracted feature vectors are denoted as f(A), f(P), and f(N). The Euclidean distance between f(A) and f(P) and the Euclidean distance between f(A) and f(N) are calculated. If the distance from f(A) to f(P) is less than the distance from f(A) to f(N), it is considered a correct prediction; otherwise, it is considered a wrong prediction. The result of the epoch with the highest accuracy among the 20 epochs is used as the final result. The formula for calculating the Euclidean distance is as follows: distance=||feature i -feature j || 2 The ternary loss function is defined as follows: L(A,P,N)=max(‖f(A)-f(P)‖ 2 -‖f(A)-f(N)‖ 2 +margin,0) Here, f(x) represents the features extracted by the convolutional neural network, and margin is a threshold used to ensure that there is at least a certain distance between positive samples P and negative samples N.

2. The intelligent recognition method for same-joint weld images according to claim 1, characterized in that, The key points include, but are not limited to, weld beads, fish scale patterns, and other significant weld features.

3. The intelligent recognition method for same-joint weld images according to claim 1, characterized in that, Steps 2 and 3 are specifically as follows: Step 2: Divide the weld area detection dataset into training and test sets in an 8:2 ratio. Use the Faster-RCN n object detection algorithm to train on the training set. Improve the generalization ability of the model by randomly cropping and flipping the training set. Use the SGD optimizer to continuously train and iterate. After each generation of training is completed, test on the test set. Save the model with the highest Mapping value on the test set to obtain the weld area detection model. Step 3: First, perform local histogram enhancement on all data in the keypoint detection dataset. Then, divide the keypoint detection dataset into training and test sets in an 8:2 ratio. Use the Faster-R-CNN object detection algorithm to train on the training set. Improve the model's generalization ability by randomly cropping and flipping the training set. Use the SGD optimizer to continuously train and iterate. After each generation of training is completed, test on the test set. Save the model with the highest Mapping score on the test set to obtain the keypoint detection model.

4. The intelligent recognition method for same-joint weld images according to claim 3, characterized in that, The Faster-rcnn object detection algorithm can be replaced with YOLO or other neural networks used for object detection.

5. The intelligent recognition method for same-joint weld images according to claim 4, characterized in that, The target detection algorithm can be replaced by image segmentation techniques or other methods for detecting weld seam areas.

6. The intelligent recognition method for same-joint weld images according to claim 5, characterized in that, In step 4, strategies are adopted to accelerate the evaluation process of the network architecture. These strategies include, but are not limited to, using a parameter sharing strategy, a multi-fidelity evaluation strategy, and a learning curve prediction strategy.

7. The intelligent recognition method for same-joint weld images according to claim 6, characterized in that, Step 5 specifically involves: using the optimal architecture selected in step 4, training the model on the keypoint measurement dataset, with the training set consisting of triples of training categories and the test set consisting of triples of test categories; improving the model's generalization ability by randomly pruning and flipping the training set; using the SGD optimizer and the ternary loss function, continuously training and iterating, testing on the test set after each generation of training, and saving the model with the highest accuracy on the test set to obtain the keypoint similarity comparison model, and obtaining the keypoint similarity measurement model through training iterations.

8. The intelligent recognition method for same-joint weld images according to claim 7, characterized in that, The binary matching process in step 6 is as follows: During the matching process, a keypoint pair is used as an anchor point, and all other keypoint pairs are traversed sequentially. For each keypoint pair, the matching algorithm calculates the distance and angle from the center point of all keypoints in the query to the center point of the corresponding anchor point, and calculates the distance and angle from the center point of all keypoints in the target to the center point of their corresponding anchor point. Let the coordinates of the center point of the keypoint be (x1, y1) and the coordinates of the center point of the anchor point be (x2, y2), then the formulas for calculating the distance and angle between them are as follows: If the difference between the distance and angle from the first keypoint in the query to the anchor point and the distance and angle from a keypoint in the target to its corresponding anchor point meets a pre-set threshold condition, then this is considered a possible match. The algorithm will traverse all possible anchor point combinations and calculate their matching degree. Finally, the set with the largest number of matches will be selected as the final matching result and returned as the output of the algorithm.

9. The intelligent recognition method for same-joint weld images according to claim 8, characterized in that, The key point similarity calculation method has been replaced with cosine similarity and Euclidean distance.

Images