Real-time defect detection system for 3D printed assembled ring beams based on Transformer recognition
The real-time defect detection system for 3D printed prefabricated ring beams uses Transformer recognition to convert geometric structures into graph neural networks, enabling comprehensive and rapid defect identification, reducing manual reliance and enhancing automation.
Patent Information
- Application Number
- CN202411337631.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-25
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-09-25
AI Technical Summary
Existing real-time defect detection systems for 3D printed prefabricated ring beams are inadequate in identifying defects caused by structural irregularities or connection problems, lack comprehensive defect identification, and rely heavily on manual labels and expert knowledge, reducing automation and recognition accuracy.
A real-time defect detection system utilizing Transformer recognition, comprising modules for data acquisition, segmentation and extraction, maintenance support, potential inference, defect prediction, classification and recognition, modeling and simulation, and report generation, which converts the ring beam's geometric structure into a graph neural network, employs expert knowledge and historical data, and uses Transformer models for deep learning and pattern recognition to identify defects.
The system enhances defect identification comprehensiveness and accuracy, allows for real-time defect detection, reduces reliance on manual labels, and improves automation by quickly identifying and locating defects, providing forward-looking maintenance suggestions.
Smart Images

Figure CN118864458B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of defect detection, and in particular to a real-time defect detection system for a 3D printed assembled ring beam based on Transformer recognition. Background Art
[0002] With the advancement of construction technology, 3D printed prefabricated structures have become an important trend in modern architecture. This technology enables the efficient production of complex structures and significantly improves the design freedom of buildings. However, defects generated during the 3D printing process, such as surface unevenness, internal holes or cracks, may seriously affect the safety and service life of the structure. Therefore, timely and accurate detection of these defects is key to ensuring the quality of construction. Traditional defect detection methods often rely on manual inspection and sensor-based monitoring, which are not only time-consuming but also costly. To solve this problem, computer vision and automated detection technologies have been gradually introduced into the defect detection process. These technologies can use image processing and analysis techniques to automatically identify and classify defects, thereby improving detection efficiency and accuracy.
[0003] The existing real-time defect detection system for 3D printed prefabricated ring beams is not conducive to discovering defects caused by structural irregularities or connection problems, which reduces the comprehensiveness of defect identification and cannot quickly identify and locate defects while processing the data of 3D printed prefabricated ring beams; in addition, the existing real-time defect detection system for 3D printed prefabricated ring beams cannot identify the evolution process and type of defects, which reduces the recognition accuracy, and has a high reliance on manual labels and expert knowledge, which reduces the automation level and universality of the system. For this reason, we propose a real-time defect detection system for 3D printed prefabricated ring beams based on Transformer recognition. Summary of the invention
[0004] The purpose of the present invention is to solve the defects in the prior art and to propose a real-time defect detection system for a 3D printed assembled ring beam based on Transformer recognition.
[0005] In order to achieve the above object, the present invention adopts the following technical solutions:
[0006] The real-time defect detection system for 3D printed assembled ring beams based on Transformer recognition includes an acquisition preprocessing module, a segmentation and extraction module, a maintenance support module, a potential inference module, a defect prediction module, a classification and recognition module, a modeling and simulation module, a monitoring and adjustment module, and a report generation module;
[0007] The acquisition preprocessing module is used to acquire raw data from the 3D printing assembly site through a high-resolution camera and sensor, and perform preliminary processing on the acquired data;
[0008] The segmentation and extraction module is used to segment the preprocessed image, extract the key parts of the ring beam, and extract geometric and texture features from the segmented image.
[0009] As a further solution of the present invention, the segmentation and extraction module extracts geometric and texture features in the following specific steps:
[0010] S1.1: Collect and collect the image data of each group after the preprocessing module, and adjust the color image of each group of image data to a grayscale image through grayscale conversion. According to the image size and noise conditions, set Filter size and standard deviation , calculate the weight matrix of the Gaussian filter, and use the Gaussian filter to convolve with each image data, calculate the weighted average of each pixel to smooth each image data;
[0011] S1.2: By as well as Calculate the horizontal gradient of an image And vertical gradient ,in Represents the rate of change of the image intensity value relative to the x-axis or y-axis direction. represents the intensity value of the image, Represents the horizontal and vertical coordinates of the pixel value of the image, and then according to the horizontal gradient And vertical gradient Calculate the gradient magnitude And the gradient direction ;
[0012] S1.3: Remove non-edge points in the gradient amplitude map, retain the local maximum, and set two thresholds, high and low. Pixels above the high threshold are marked as strong edges, and pixels between the high and low thresholds are marked as weak edges. Pixels below the low threshold are directly discarded, and then weak edges and strong edges are connected to form the final edge map;
[0013] S1.4: Count the number of pixels at each gray level in the image to form a gray histogram, take each gray level as a threshold in turn, take the image gray value greater than the threshold as the foreground, and take the image gray value less than or equal to the threshold as the background, obtain the inter-class variance of the foreground and background, traverse all thresholds, take the threshold with the largest inter-class variance as the optimal threshold, and divide the image foreground and background according to the optimal threshold;
[0014] S1.5: Use the findContours function in OpenCV to extract the contours in the image after threshold segmentation. Then sort the detected contours according to their area or perimeter, extract the main contours, and identify the various parts of the ring beam by analyzing the geometric features of the contours. Draw the detected contours on the original image for visualization.
[0015] S1.6: Calculate the geometric moments of the image to obtain geometric moments of various orders, use the geometric moments to calculate the central moments, and normalize the calculated central moments. Then, use the normalized central moments to calculate the Hu invariant moments to extract the shape invariance features of the image, and extract the texture features through GLCM.
[0016] S1.7: For each pixel in the image, the difference between it and the surrounding pixels is calculated to generate the corresponding LBP value. When the LBP values of all pixels are calculated, the corresponding LBP image is generated, and the LBP image is subjected to histogram statistics to obtain a texture feature histogram.
[0017] The maintenance support module is used to construct and maintain a knowledge graph of 3D printed assembled ring beams and provide background knowledge in the field of ring beam defect detection.
[0018] As a further solution of the present invention, the maintenance support module constructs and maintains the knowledge graph of the 3D printed assembled ring beam in the following specific steps:
[0019] S2.1: Collect various industry standards, design specifications and historical data related to prefabricated ring beams, extract various concepts of prefabricated ring beams from various industry standards, design specifications and historical data, and classify the probabilities to form nodes of different categories;
[0020] S2.2: According to industry standards and specifications, determine the relationship types between different concepts, establish relationship mappings for each pair of related concepts, determine the specific association between the two, and then initialize the graph structure in the Neo4j graph database;
[0021] S2.3: Add the defined concepts as nodes in the graph database and add edges for each pair of related concepts to represent the relationship between them. Collect detailed text and data information from industry standards, design specifications, and historical data, and then map the collected data to the nodes and edges of the knowledge graph.
[0022] S2.4: Add attribute information to each node and edge, and annotate each node and edge in detail based on the specific content of the data, provide contextual information, check the accuracy of the integrated and annotated data automatically or manually, and regularly calibrate and update the data and annotation information in the knowledge graph according to the latest industry standards and specifications.
[0023] As a further solution of the present invention, the historical data in S2.1 includes technical documents, specifications and standards, engineering design drawings, historical defect records and maintenance logs.
[0024] The potential inference module is used to analyze and infer the potential defect locations and types of the ring beam.
[0025] As a further solution of the present invention, the potential inference module analyzes and infers the potential defect position and type of the ring beam in the following specific steps:
[0026] S3.1: Obtain the design drawings or 3D modeling data of the ring beam, and convert the geometric structure and connection relationship of the ring beam into a graph structure, where the nodes represent the key parts of the ring beam and the edges represent the connection relationship between the nodes, and then set the architecture of the graph neural network;
[0027] S3.2: Set the initial features of nodes and edges in the graph structure based on the texture features and geometric features extracted by the segmentation and extraction module, load expert knowledge and historical data about ring beam structure and defect detection from the knowledge graph, and generate an initial population with an initial set of belief spaces. Each group of individuals in the population represents a parameter configuration or training strategy of a graph neural network.
[0028] S3.3: Evaluate the accuracy of each group of individuals in the graph neural network model, and select multiple groups of individuals as parent individuals through the roulette wheel selection method, then randomly exchange the genes of any two groups of parent individuals at multiple crossover points to generate new individuals, then mutate each group of new individuals, and add the individuals generated by crossover and mutation to the original population;
[0029] S3.4: Evaluate the performance of each individual in the current population, select individuals or sets of individuals whose accuracy exceeds the preset threshold, extract the characteristics of each excellent individual, and then perform weighted average of the new knowledge and the old knowledge in the belief space, and adjust the content and strategy of the belief space according to domain knowledge and optimization requirements, and output the optimal individual as the training strategy of the graph neural network;
[0030] S3.5: Construct a set of ring beam training data sets, aggregate the features of the neighbor nodes of each node in the data set, and update the features of the current node through a nonlinear activation function based on the aggregated neighbor node information. Repeat the aggregation and update steps in multiple graph convolutional layers in the graph neural network to capture higher-order graph structure features until the preset number of iterations is reached, and then output the features of the nodes and edges;
[0031] S3.6: Evaluate model performance through the cross entropy function, update model parameters through back propagation, infer the test data of the ring beam through the trained graph neural network, output the feature representation of nodes and edges, parse the feature representation of nodes and edges, and identify potential defects.
[0032] As a further solution of the present invention, the key parts of the ring beam specifically include the end points of the circular or arc-shaped contour, joint positions, bending points, support points, key points of the central axis, holes or opening positions, corners or transition areas, and defective areas.
[0033] The defect prediction module is used to perform deep learning and pattern recognition on the characteristics of the ring beam through the Transformer model to identify various defects of the ring beam.
[0034] As a further solution of the present invention, the specific steps of the defect prediction module for identifying various defects of the ring beam are as follows:
[0035] S4.1: Serialize the extracted geometric features and texture features, and then input them into the Transformer model, and add position encoding to preserve the position relationship in the feature sequence. According to the multi-head attention mechanism, each set of feature data is converted into query, key and value matrices, and then the attention weight matrix is obtained by multiplying the query matrix with the key matrix;
[0036] S4.2: Scale the weight matrix, and then use the softmax function to calculate the attention score of each feature, divide the input feature vector into multiple subspaces, and calculate the attention for each subspace independently. Based on the calculation results, a weighted feature representation is generated for each space;
[0037] S4.3: Concatenate the outputs of all subspaces and generate the final output through linear transformation. Then, the output of the multi-head self-attention mechanism is processed by a feedforward neural network. At the same time, the feedforward neural network introduces nonlinear characteristics through the ReLU activation function. By stacking multiple encoder layers, the Transformer model extracts a high-level representation of the input feature vector;
[0038] S4.4: Based on the output features of the Transformer model, construct the corresponding state set and observation set, define the probability of each state at the initial time, and construct the state transition probability matrix and observation probability matrix;
[0039] S4.5: In time When , the initial probability of each state is calculated by the initial state probability and the observation probability, and Initially, for each state at time , calculated from the time The probability of a random state transitioning to any state, and at the last time point in the sequence , select the state with the maximum probability;
[0040] S4.6: Start from the selected state and trace back to the starting point of the sequence, and determine the state sequence with the highest probability. At the same time, use a tracking table to record the optimal predecessor state at each time point. Based on the traceback path, calculate the probability of each state sequence in the traceback path, determine the defect type and location, and generate a final report or warning based on the identified defect type and location.
[0041] As a further solution of the present invention, the specific calculation formula of the position code in S4.1 is as follows:
[0042] ;
[0043] ;
[0044] In the formula, represents positional encoding; Represents location; Represents the dimension index; Represents the dimension of the Transformer encoder.
[0045] The classification and recognition module is used to verify and classify the output results and determine the final defect type and location.
[0046] As a further solution of the present invention, the classification recognition module verifies and classifies the output results in the following specific steps:
[0047] S5.1: Collect all test results output by the potential inference module and the defect prediction module, and collect each set of additional information related to the test results, clean each set of collected data, and remove outliers and noise data;
[0048] S5.2: Calculate the mean, standard deviation and median of the test results. According to the test result data, classify all test results according to defect types. Each defect type forms a category. Calculate the number of occurrences of each defect type. And obtain the frequency of occurrence of each defect type based on the number of occurrences of each defect type and the total number of data points.
[0049] S5.3: Initialize the prior probability of the Bayesian model, set the prior distribution based on historical data and domain knowledge, obtain the observed data based on the output of the detection system, calculate the likelihood function of the observed data under different defect types, and quantify the matching degree between the observed data and each defect type;
[0050] S5.4: Calculate the posterior probability of each defect type by combining the prior probability and the likelihood function. Based on the calculated posterior probability, evaluate the confidence of each test result, identify defect types with high confidence, and mark the results with low confidence and the preset threshold as pending review.
[0051] The modeling and simulation module is used to perform 3D modeling and simulation on the detected defects and evaluate the impact of the defects on the overall structure of the ring beam;
[0052] The monitoring and adjustment module is used to monitor the 3D printing process in real time and to alarm and handle detected defects;
[0053] The report generation module is used to generate a detailed test report, recording the test results and analysis process.
[0054] Compared with the prior art, the present invention has the following beneficial effects:
[0055] 1. The present invention converts the geometric structure and connection relationship of the ring beam into a graph structure, and then sets the architecture of the graph neural network, loads expert knowledge and historical data about the ring beam structure and defect detection from the knowledge graph, and generates an initial population with an initial set of belief spaces. Each group of individuals in the population represents a parameter configuration or training strategy of the graph neural network. The accuracy of each group of individuals in the graph neural network model is evaluated, and multiple groups of individuals are selected as parent individuals through a roulette wheel selection method. New individuals are generated by performing crossover operations on each group of parent individuals, and then each group of new individuals is mutated, the population information is updated, the performance of each individual in the current population is evaluated, and the characteristics of each excellent individual are extracted. The belief space is updated, and the optimal individual is used as the training strategy of the graph neural network to construct a set of training data sets for ring beams. The characteristics of the neighbor nodes of each node in the data set are Aggregation is performed, and based on the aggregated neighbor node information, the features of the current node are updated through a nonlinear activation function. The aggregation and update steps are repeated in multiple graph convolutional layers in the graph neural network to capture higher-order graph structure features. After reaching the preset number of iterations, the features of the nodes and edges are output, and the model performance is evaluated through the cross entropy function. The model parameters are updated through back propagation. The test data of the ring beam is inferred through the trained graph neural network, and the feature representation of the nodes and edges is output. The feature representation of the nodes and edges is analyzed to identify potential defects, which is helpful to discover defects caused by structural irregularities or connection problems, can better identify potential defects, and improve the comprehensiveness of defect identification. At the same time, when processing the data of 3D printed prefabricated ring beams, defects can be quickly identified and located, thereby meeting the needs of real-time detection.
[0056] 2. The present invention extracts geometric features and texture features through serialization processing, and then inputs them into the Transformer model, and adds position encoding. According to the multi-head attention mechanism, the input feature vector is divided into multiple subspaces, and the attention is calculated independently for each subspace. According to the calculation results, a weighted feature representation is generated for each space, the outputs of all subspaces are spliced, and the final output is generated through linear transformation. After that, the output of the multi-head self-attention mechanism is introduced into the nonlinear characteristics through the ReLU activation function of the feedforward neural network. By stacking multiple encoder layers, the high-level representation of the input feature vector is extracted, the corresponding state set and observation set are constructed, the probability of each state at the initial moment is defined, and the state transition probability matrix and the observation probability matrix are constructed. When , the initial probability of each state is calculated by the initial state probability and the observation probability, and Initially, for each state at time Calculate from time The probability of a random state transitioning to any state, and at the last time point in the sequence , select the state with the highest probability, trace back from the selected state to the starting point of the sequence, and determine the state sequence with the highest probability. At the same time, use a tracking table to record the optimal predecessor state at each time point. According to the backtracking path, calculate the probability of each state sequence in the backtracking path, determine the defect type and location, and then evaluate the confidence of each detection result through Bayesian optimization, identify the defect type with high confidence, and mark the results with low confidence and preset threshold as pending review, and generate a final report or warning based on the identified defect type and location. It can identify the evolution process and type of defects, improve recognition accuracy, provide forward-looking maintenance suggestions, improve the accuracy of the system's defect judgment, reduce dependence on manual labels and expert knowledge, and improve the system's automation level and universality. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.
[0058] Figure 1 This is a system block diagram of the real-time defect detection system for 3D printed assembled ring beams based on Transformer recognition proposed in the present invention. DETAILED DESCRIPTION
[0059] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.
[0060] Example 1
[0061] Reference Figure 1 , a real-time defect detection system for 3D printed assembled ring beams based on Transformer recognition, including an acquisition preprocessing module, a segmentation and extraction module, a maintenance support module, a potential inference module, a defect prediction module, a classification and recognition module, a modeling and simulation module, a monitoring and adjustment module, and a report generation module;
[0062] The acquisition and preprocessing module is used to acquire raw data from the 3D printing assembly site through high-resolution cameras and sensors, and to perform preliminary processing on the acquired data; the segmentation and extraction module is used to segment the preprocessed image, extract the key parts of the ring beam, and extract geometric and texture features from the segmented image.
[0063] Specifically, the image data of each group processed by the acquisition preprocessing module are collected, and the color image of each group of image data is adjusted to a grayscale image through grayscale conversion. According to the image size and noise conditions, the Filter size and standard deviation , calculate the weight matrix of the Gaussian filter, and use the Gaussian filter to convolve with each image data, calculate the weighted average of each pixel to smooth each image data, respectively. ,as well as Calculate the horizontal gradient of an image And vertical gradient ,in Represents the rate of change of the image intensity value relative to the x-axis or y-axis direction. represents the intensity value of the image, Represents the horizontal and vertical coordinates of the pixel value of the image, and then according to the horizontal gradient And vertical gradient Calculate the gradient magnitude And the gradient direction , remove non-edge points in the gradient amplitude map, retain the local maximum, and set two high and low thresholds. Pixels above the high threshold are marked as strong edges, pixels between the high and low thresholds are marked as weak edges, and pixels below the low threshold are directly discarded. Then connect weak edges and strong edges to form the final edge map, count the number of pixels of each gray level in the image, form a grayscale histogram, and take each gray level as the threshold in turn. The image grayscale value greater than the threshold is taken as the foreground, and the image grayscale value less than or equal to the threshold is taken as the background. Obtain the inter-class variance of the foreground and background, traverse all thresholds, and take the threshold with the largest inter-class variance as the optimal threshold. The image foreground and background are divided according to the optimal threshold. Use the findContours function in OpenCV to extract the image after threshold segmentation. , and then sort the detected contours according to the area or perimeter of the contours, extract the main contours, and identify the various parts of the ring beam by analyzing the geometric features of the contours, and draw the detected contours on the original image for visual display, calculate the geometric moments of the image to obtain geometric moments of various orders, use the geometric moments to calculate the central moments, and normalize the calculated groups of central moments, and then use the normalized central moments to calculate the Hu invariant moments to extract the shape invariance features of the image, extract the texture features through GLCM, and for each pixel in the image, calculate the difference between it and the surrounding pixels to generate the corresponding LBP value. When the LBP values of all pixels are calculated, generate the corresponding LBP image, and perform histogram statistics on the LBP image to obtain the texture feature histogram.
[0064] The maintenance support module is used to build and maintain the knowledge graph of 3D printed assembled ring beams and provide background knowledge in the field of ring beam defect detection.
[0065] Specifically, collect various industry standards, design specifications and historical data related to prefabricated ring beams, extract various concepts of prefabricated ring beams from various industry standards, design specifications and historical data, and classify the probabilities to form nodes of different categories. According to industry standards and specifications, determine the relationship type between different concepts, establish a relationship mapping for each pair of related concepts, and determine the specific association between the two. Then initialize the graph structure in the Neo4j graph database, add the defined concepts as nodes in the graph database, and add edges for each pair of related concepts to represent the relationship between them. Collect detailed text and data information from industry standards, design specifications and historical data, and then map the collected data to the nodes and edges of the knowledge graph, add attribute information to each node and edge, and annotate each node and edge in detail according to the specific content of the data to provide contextual information. Check whether the integrated and annotated data is accurate by machine automatic or manual methods, and regularly calibrate and update the data and annotation information in the knowledge graph according to the latest industry standards and specifications.
[0066] In this embodiment, the historical data includes technical documents, specifications and standards, engineering design drawings, historical defect records, and maintenance logs.
[0067] The potential inference module is used to analyze and infer the potential defect locations and types of ring beams.
[0068] Specifically, the design drawings or 3D modeling data of the ring beam are obtained, and the geometric structure and connection relationship of the ring beam are converted into a graph structure, where the nodes represent the key parts of the ring beam and the edges represent the connection relationship between the nodes. Then, the architecture of the graph neural network is set, and the initial features of the nodes and edges in the graph structure are set according to the texture features and geometric features extracted by the segmentation and extraction module. Expert knowledge and historical data about the ring beam structure and defect detection are loaded from the knowledge graph. An initial set of belief spaces is used to regenerate an initial population, and each group of individuals in the population represents a parameter configuration or training strategy of a graph neural network. The accuracy of each group of individuals in the graph neural network model is evaluated, and multiple groups of individuals are selected as parent individuals through a roulette wheel selection method. Then, the genes of any two groups of parent individuals are randomly exchanged at multiple intersections to generate new individuals. Then, each group of new individuals is mutated, and the individuals generated by the crossover and mutation are added to the original population. The performance of each individual in the current population is evaluated, and Individuals or sets of individuals whose accuracy exceeds the preset threshold are identified, and the characteristics of each excellent individual are extracted. The new knowledge is then weighted averaged with the old knowledge in the belief space. The content and strategy of the belief space are adjusted according to domain knowledge and optimization requirements, and the optimal individual is output as the training strategy of the graph neural network. A set of training data sets for ring beams is constructed, and the characteristics of the neighboring nodes of each node in the data set are aggregated. According to the aggregated neighboring node information, the characteristics of the current node are updated through a nonlinear activation function. The aggregation and update steps are repeated in multiple graph convolutional layers in the graph neural network to capture higher-order graph structure features. After reaching the preset number of iterations, the characteristics of the nodes and edges are output, the model performance is evaluated through the cross entropy function, and the model parameters are updated through back propagation. The test data of the ring beam is inferred through the trained graph neural network, and the feature representations of the nodes and edges are output. The feature representations of the nodes and edges are analyzed to identify potential defects.
[0069] It should be further explained that the key parts of the ring beam specifically include the end points of the circular or arc-shaped contour, the joint positions, the bending points, the supporting points, the key points of the central axis, the holes or opening positions, the corners or transition areas and the defective areas.
[0070] Example 2
[0071] Reference Figure 1The real-time defect detection system of 3D printed assembled ring beam based on Transformer recognition includes acquisition preprocessing module, segmentation and extraction module, maintenance support module, potential inference module, defect prediction module, classification and recognition module, modeling and simulation module, monitoring and adjustment module and report generation module.
[0072] The defect prediction module is used to perform deep learning and pattern recognition on the characteristics of the ring beam through the Transformer model to identify various defects of the ring beam.
[0073] Specifically, the extracted geometric features and texture features are serialized and then input into the Transformer model, and position encoding is added to retain the positional relationship in the feature sequence. According to the multi-head attention mechanism, each group of feature data is converted into query, key and value matrices, and then the attention weight matrix is obtained by multiplying the query matrix with the key matrix. The weight matrix is scaled, and then the attention score of each feature is calculated through the softmax function. The input feature vector is divided into multiple subspaces, and the attention is calculated independently for each subspace. According to the calculation results, a weighted feature representation is generated for each space, the outputs of all subspaces are concatenated, and the final output is generated through linear transformation. After that, the output of the multi-head self-attention mechanism is processed by a feedforward neural network. At the same time, the feedforward neural network introduces nonlinear characteristics through the ReLU activation function. By stacking multiple encoder layers, the Transformer model extracts a high-level representation of the input feature vector. According to the output characteristics of the Transformer model, the corresponding state set and observation set are constructed, the probability of each state at the initial moment is defined, and the state transition probability matrix and observation probability matrix are constructed. When , the initial probability of each state is calculated by the initial state probability and the observation probability, and Initially, for each state at time , calculated from the time The probability of a random state transitioning to any state, and at the last time point in the sequence , select the state with the maximum probability, trace back from the selected state to the starting point of the sequence, and determine the state sequence with the maximum probability. At the same time, use a tracking table to record the optimal predecessor state at each time point. According to the backtracking path, calculate the probability of each state sequence in the backtracking path, determine the defect type and location, and generate a final report or warning based on the identified defect type and location.
[0074] In this embodiment, the specific calculation formula of the position code is as follows:
[0075] ;
[0076] ;
[0077] In the formula, represents positional encoding; Represents location; Represents the dimension index; Represents the dimension of the Transformer encoder.
[0078] The classification and recognition module is used to verify and classify the output results and determine the final defect type and location.
[0079] Specifically, all test results output by the potential inference module and the defect prediction module are collected, and each group of additional information related to the test results is collected, each group of collected data is cleaned, and outliers and noise data are removed, and the mean, standard deviation and median of the test results are calculated. According to the test result data, all test results are classified according to defect type, and each defect type forms a category. The number of occurrences of each defect type is calculated, and the frequency of occurrence of each defect type is obtained based on the number of occurrences of each defect type and the total number of data points. The prior probability of the Bayesian model is initialized, and the prior distribution is set according to historical data and domain knowledge. According to the output results of the detection system, the observed data is obtained, and the likelihood function of the observed data under different defect types is calculated. The degree of matching between the observed data and each defect type is quantified. The posterior probability of each defect type is calculated by combining the prior probability and the likelihood function. According to the calculated posterior probability, the confidence of each test result is evaluated, and the defect type with high confidence is identified, and the results with low confidence and preset threshold are marked as pending review.
[0080] The modeling and simulation module is used to perform 3D modeling and simulation of the detected defects and evaluate the impact of the defects on the overall structure of the ring beam; the monitoring and adjustment module is used to monitor the 3D printing process in real time and alarm to handle the detected defects; the report generation module is used to generate a detailed test report to record the test results and analysis process.
Claims
1. A real-time defect detection system for 3D printed assembled ring beams based on Transformer recognition, characterized in that: It includes acquisition preprocessing module, segmentation and extraction module, maintenance support module, potential inference module, defect prediction module, classification and recognition module, modeling and simulation module, monitoring and adjustment module and report generation module; The acquisition preprocessing module is used to acquire raw data from the 3D printing assembly site through a high-resolution camera and sensor, and perform preliminary processing on the acquired data; The segmentation and extraction module is used to segment the pre-processed image, extract the key parts of the ring beam, and extract geometric and texture features from the segmented image; The maintenance support module is used to construct and maintain a knowledge graph of 3D printed assembled ring beams and provide background knowledge in the field of ring beam defect detection; The potential inference module is used to analyze and infer the potential defect locations and types of the ring beam; The defect prediction module is used to perform deep learning and pattern recognition on the characteristics of the ring beam through the Transformer model to identify various defects of the ring beam; The classification and identification module is used to verify and classify the output results and determine the final defect type and location; The modeling and simulation module is used to perform 3D modeling and simulation on the detected defects and evaluate the impact of the defects on the overall structure of the ring beam; The monitoring and adjustment module is used to monitor the 3D printing process in real time and to alarm and handle detected defects; The report generation module is used to generate a detailed test report, recording the test results and analysis process; The specific steps of extracting geometric and texture features by the segmentation and extraction module are as follows: S1.1: Collect and collect the image data of each group after the preprocessing module, and adjust the color image of each group of image data to a grayscale image through grayscale conversion. According to the image size and noise conditions, set Filter size and standard deviation , calculate the weight matrix of the Gaussian filter, and use the Gaussian filter to convolve with each image data, and calculate the weighted average of each pixel to smooth each image data; S1.2: By as well as Calculate the horizontal gradient of an image And vertical gradient ,in Represents the rate of change of the image intensity value relative to the x-axis or y-axis direction. represents the intensity value of the image, Represents the horizontal and vertical coordinates of the pixel value of the image, and then according to the horizontal gradient And vertical gradient Calculate the gradient magnitude And the gradient direction ; S1.3: Remove non-edge points in the gradient amplitude map, retain the local maximum, and set two thresholds, high and low. Pixels above the high threshold are marked as strong edges, and pixels between the high and low thresholds are marked as weak edges. Pixels below the low threshold are directly discarded, and then weak edges and strong edges are connected to form the final edge map; S1.4: Count the number of pixels at each gray level in the image to form a gray histogram, take each gray level as a threshold in turn, take the image gray value greater than the threshold as the foreground, and take the image gray value less than or equal to the threshold as the background, obtain the inter-class variance of the foreground and background, traverse all thresholds, take the threshold with the largest inter-class variance as the optimal threshold, and divide the image foreground and background according to the optimal threshold; S1.5: Use the findContours function in OpenCV to extract the contours in the image after threshold segmentation. Then sort the detected contours according to their area or perimeter, extract the main contours, and identify the various parts of the ring beam by analyzing the geometric features of the contours. Draw the detected contours on the original image for visualization. S1.6: Calculate the geometric moments of the image to obtain geometric moments of various orders, use the geometric moments to calculate the central moments, and normalize the calculated central moments. Then, use the normalized central moments to calculate the Hu invariant moments to extract the shape invariance features of the image, and extract the texture features through GLCM. S1.7: For each pixel in the image, the difference between it and the surrounding pixels is calculated to generate the corresponding LBP value. When the LBP values of all pixels are calculated, the corresponding LBP image is generated, and the LBP image is subjected to histogram statistics to obtain a texture feature histogram. The maintenance support module constructs and maintains the knowledge graph of the 3D printed assembled ring beam in the following specific steps: S2.1: Collect various industry standards, design specifications and historical data related to prefabricated ring beams, extract various concepts of prefabricated ring beams from various industry standards, design specifications and historical data, and classify the probabilities to form nodes of different categories; S2.2: According to industry standards and specifications, determine the relationship types between different concepts, establish relationship mappings for each pair of related concepts, determine the specific association between the two, and then initialize the graph structure in the Neo4j graph database; S2.3: Add the defined concepts as nodes in the graph database and add edges for each pair of related concepts to represent the relationship between them. Collect detailed text and data information from industry standards, design specifications, and historical data, and then map the collected data to the nodes and edges of the knowledge graph. S2.4: Add attribute information to each node and edge, and annotate each node and edge in detail according to the specific content of the data, provide contextual information, check the accuracy of the integrated and annotated data automatically or manually, and regularly calibrate and update the data and annotation information in the knowledge graph according to the latest industry standards and specifications; The specific steps of the potential inference module for analyzing and inferring the potential defect locations and types of the ring beam are as follows: S3.1: Obtain the design drawings or 3D modeling data of the ring beam, and convert the geometric structure and connection relationship of the ring beam into a graph structure, where the nodes represent the key parts of the ring beam and the edges represent the connection relationship between the nodes, and then set the architecture of the graph neural network; S3.2: Set the initial features of nodes and edges in the graph structure based on the texture features and geometric features extracted by the segmentation and extraction module, load expert knowledge and historical data about ring beam structure and defect detection from the knowledge graph, and generate an initial population with an initial set of belief spaces. Each group of individuals in the population represents a parameter configuration or training strategy of a graph neural network. S3.3: Evaluate the accuracy of each group of individuals in the graph neural network model, and select multiple groups of individuals as parent individuals through the roulette wheel selection method, then randomly exchange the genes of any two groups of parent individuals at multiple crossover points to generate new individuals, then mutate each group of new individuals, and add the individuals generated by crossover and mutation to the original population; S3.4: Evaluate the performance of each individual in the current population, select individuals or sets of individuals whose accuracy exceeds the preset threshold, extract the characteristics of each excellent individual, and then perform weighted average of the new knowledge and the old knowledge in the belief space, and adjust the content and strategy of the belief space according to domain knowledge and optimization requirements, and output the optimal individual as the training strategy of the graph neural network; S3.5: Construct a set of ring beam training data sets, aggregate the features of the neighbor nodes of each node in the data set, and update the features of the current node through a nonlinear activation function based on the aggregated neighbor node information. Repeat the aggregation and update steps in multiple graph convolutional layers in the graph neural network to capture higher-order graph structure features until the preset number of iterations is reached, and then output the features of the nodes and edges; S3.6: Evaluate model performance through the cross entropy function, update model parameters through back propagation, infer the test data of the ring beam through the trained graph neural network, output the feature representation of nodes and edges, parse the feature representation of nodes and edges, and identify potential defects; The specific steps of the defect prediction module for identifying various defects of the ring beam are as follows: S4.1: Serialize the extracted geometric features and texture features, and then input them into the Transformer model, and add position encoding to preserve the position relationship in the feature sequence. According to the multi-head attention mechanism, each set of feature data is converted into query, key and value matrices, and then the attention weight matrix is obtained by multiplying the query matrix with the key matrix; S4.2: Scale the weight matrix, and then use the softmax function to calculate the attention score of each feature, divide the input feature vector into multiple subspaces, and calculate the attention for each subspace independently. Based on the calculation results, a weighted feature representation is generated for each space; S4.3: Concatenate the outputs of all subspaces and generate the final output through linear transformation. Then, the output of the multi-head self-attention mechanism is processed by a feedforward neural network. At the same time, the feedforward neural network introduces nonlinear characteristics through the ReLU activation function. By stacking multiple encoder layers, the Transformer model extracts a high-level representation of the input feature vector; S4.4: Based on the output features of the Transformer model, construct the corresponding state set and observation set, define the probability of each state at the initial time, and construct the state transition probability matrix and observation probability matrix; S4.5: In time When , the initial probability of each state is calculated by the initial state probability and the observation probability, and Initially, for each state at time , calculated from time The probability of a random state transitioning to any state, and at the last time point in the sequence , select the state with the maximum probability; S4.6: Start from the selected state and trace back to the starting point of the sequence, and determine the state sequence with the highest probability. At the same time, use a tracking table to record the optimal predecessor state at each time point. Based on the traceback path, calculate the probability of each state sequence in the traceback path, determine the defect type and location, and generate a final report or warning based on the identified defect type and location.
2. The real-time defect detection system for 3D printed assembled ring beams based on Transformer recognition according to claim 1 is characterized in that: S2.1 The historical data referred to in the document include technical documents, specifications and standards, engineering design drawings, historical defect records and maintenance logs.
3. The real-time defect detection system for 3D printed assembled ring beams based on Transformer recognition according to claim 1 is characterized in that: The specific calculation formula for the position code described in S4.1 is as follows: ; ; In the formula, represents positional encoding; Represents location; Represents the dimension index; Represents the dimension of the Transformer encoder.
4. The real-time defect detection system for 3D printed assembled ring beams based on Transformer recognition according to claim 1 is characterized in that: The classification and recognition module verifies and classifies the output results in the following specific steps: S5.1: Collect all test results output by the potential inference module and the defect prediction module, and collect each set of additional information related to the test results, clean each set of collected data, and remove outliers and noise data; S5.2: Calculate the mean, standard deviation and median of the test results. According to the test result data, classify all test results according to defect types. Each defect type forms a category. Calculate the number of occurrences of each defect type. And obtain the frequency of occurrence of each defect type based on the number of occurrences of each defect type and the total number of data points. S5.3: Initialize the prior probability of the Bayesian model, set the prior distribution based on historical data and domain knowledge, obtain the observed data based on the output of the detection system, calculate the likelihood function of the observed data under different defect types, and quantify the matching degree between the observed data and each defect type; S5.4: Calculate the posterior probability of each defect type by combining the prior probability and the likelihood function. Based on the calculated posterior probability, evaluate the confidence of each test result, identify defect types with high confidence, and mark the results with low confidence and the preset threshold as pending review.
