A Feature Recognition Method for Micro-Structural Parts Based on the Constraint of Assembly Attention Mechanism

Abstract

The present invention discloses a method for identifying microstructural part features based on the constraint of an assembly attention mechanism, which relates to the technical field of assembly detection and includes: collecting a certain number of two-dimensional pictures from the surface of the microstructural part assembly structure through an industrial camera; dividing the two-dimensional pictures into a training set, a validation set, and a test set; calibrating pipeline features as constraint factor labels in the training dataset; annotating clamp features as training labels; constructing a microfeature recognition neural network with the constraint of an assembly attention mechanism; and S6, training through the dataset to obtain a network model that can accurately identify the features of microstructural parts. The present invention can quickly, efficiently, and accurately detect the assembly quality of microstructural parts.

Description

Technical Field

[0001] The present invention relates to the technical field of assembly detection, and particularly to a method for identifying microstructural component features based on assembly attention mechanism constraints. Background Art

[0002] The pipeline system of an aeroengine is distributed on the outer contour of the engine, with large distributed space dimensions and intricate structures. The microstructural components in the aeroengine pipeline system are mainly composed of microstructural components such as clamps and safety devices. The parts are small in volume, large in quantity, and distributed at multiple positions in the pipeline system. For example, the main functions of the clamps in the aeroengine pipeline system include supporting the pipeline, ensuring the gap between pipelines, preventing friction and collision between pipelines, and at the same time, they can disperse the internal stress in the pipeline structure. The supporting forces in different positions and directions play a damping role in pipeline resonance during engine operation. Thus, it can be seen that the clamps are crucial in the aeroengine pipeline system, and their assembly reliability is related to the operation safety of the engine. As the "safety belts" of the aeroengine pipeline system, due to their small volume and large quantity, misassembly and missing assembly of microstructural accessories are likely to occur during the assembly process of the engine pipeline system. Therefore, during the assembly process of the aeroengine pipeline system, detecting the assembly pose state of the clamps is an important task. Due to the complexity of the assembly structure, the assembly state of the microstructural components is mainly detected manually for their assembly categories and positions, resulting in low detection accuracy and poor reliability. Whether the assembly state of the microstructural components meets the requirements of the pipeline system assembly design cannot be guaranteed. The anti-misassembly detection of the microstructural components is both difficult and important in the entire engine assembly detection process, and it is an urgent problem to be solved. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for identifying microstructural component features based on assembly attention mechanism constraints, which can quickly, efficiently, and accurately detect the assembly quality of microstructural components.

[0004] To solve the above key technical problems, the present invention provides the following technical solutions:

[0005] A method for identifying microstructural component features based on assembly attention mechanism constraints of the present invention includes the following steps:

[0006] S1. Collect 200 - 300 two-dimensional pictures from the surface of the assembly structure of the aeroengine pipeline system containing the microstructural component clamps through an industrial camera;

[0007] S2. Select the pictures with clamp features from them, and randomly divide them into a training data set, a validation data set, and a test data set. The picture grouping ratio is 5:1:1. Randomly select 80 pictures from the training data set and render them onto 800 randomly selected pictures in the VOC data set as the support set, which is used for training the backbone network parameters of the neural network model. The remaining pictures are used as the query set. The validation data set is used to evaluate the model effect and adjust the hyperparameters at the same time. The test data set is used to evaluate the segmentation effect of the backbone model.

[0008] S3. Calibrate the pipeline features in the training data set as the constraint factors.

[0009] S4. Label the clamp assembly structure features as the features to be recognized and extracted.

[0010] S5. Build a microstructure feature recognition neural network model with an attention mechanism and assembly relationship constraints. The micro-feature recognition neural network model includes a backbone feature extraction network module, a candidate extraction network module, and an adaptive classifier module based on the classifier weight transfer network model. Among them, the backbone feature extraction network module includes convolutional pooling and parameter update calculations. Through training the microstructure feature recognition neural network model with an attention mechanism and assembly relationship constraints, feature extraction and expression are realized, and the generalization performance of the convolutional neural network is ensured. The Swin-transformer is used as the backbone network to extract the clamp features and background features.

[0011] The candidate extraction network module retrieves the anchor box positions from the feature map output by the backbone network based on the sliding window method, and then projects the anchor box onto the feature map to detect the target features.

[0012] The adaptive classifier module based on the classifier weight transfer network model is based on the pre-trained encoder and decoder of the backbone feature extraction network. It extracts feature vectors from the image pixels in the training data and trains the classifier model using the cross-entropy loss function. For the construction of the classifier model, first freeze the weight parameters of the encoder and decoder modules trained under a large number of general samples, and use the query set to retrain the classifier module with the attention mechanism to adjust and calibrate the network model parameters.

[0013] Train the micro-feature recognition neural network model with the attention mechanism and assembly relationship constraints through the query set, so as to obtain a network model that can accurately recognize the microstructure features of the aero-engine pipeline system.

[0014] The above-mentioned method for identifying the features of micro-structural components based on the constraint of the assembly attention mechanism, wherein: in step S2, 140 pictures with clamp features are preferably selected and randomly divided into a training data set of 100 pictures, a validation data set of 20 pictures, and a test data set of 20 pictures. Among them, the 20 pictures in the validation data set are used to calibrate the trained network model again, and the 20 pictures in the test data set are used to test the accuracy of the network model in identifying the clamp of the micro-structural features after training.

[0015] Compared with the prior art, the present invention has obvious beneficial effects. From the above technical solutions, it can be seen that: in the present invention, the clamp in the aero-engine pipeline system is used to fix the pipeline, and there is an assembly relationship between the micro-structural component clamp and the pipeline. The feature size of the pipeline is large and easy to identify. Therefore, first, it is marked in the training data set as a constraint factor, which can accurately and quickly identify micro-structural components such as clamps and provide data support for subsequent assembly quality analysis. Secondly, by using the assembly feature attributes of micro-structural components such as clamps in the engine pipeline system, a combined attention mechanism is added to the backbone network module and the classifier network module, and the feature recognition of small samples and small targets such as micro-structural components in the aero-engine pipeline system is accurately and reliably realized. The present invention constructs a network model with an attention mechanism by using the constraint of the micro-structural component assembly relationship, realizes the accurate detection of the features of micro-structural components in complex assembly structures, and can be widely applied to the small target detection tasks of complex assembly structures. Brief Description of the Drawings

[0016] Figure 1 It is a schematic diagram of the principle of the feature attention mechanism association method;

[0017] Figure 2 It is an architecture diagram of a neural network model based on the attention mechanism. Detailed Embodiments

[0018] In order to make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the drawings and specific embodiments (taking the identification of the clamp of typical micro-structural components in the aero-engine pipeline system as an example). Thereby, the implementation process of how the present application uses technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.

[0019] A method for identifying the features of micro-structural components based on the constraint of the assembly attention mechanism includes the following steps:

[0020] S1. Collect 200 two-dimensional pictures from the surface of the clamp assembly structure of the aero-engine pipeline system through an industrial camera;

[0021] S2. Select 140 valid photos with clamp features from different positions and perspectives, and randomly divide them into a training dataset of 100 photos, a validation dataset of 20 photos, and a test dataset of 20 photos. Randomly select 80 photos from the 100 training photos and render them onto 800 randomly selected photos in the VOC dataset (a commonly used dataset for neural network training) as the support set for training the backbone network parameters of the neural network model. The remaining 20 photos are used as the query set. Among them, the 20 photos in the validation dataset are used to recalibrate the trained network model, and the 20 photos in the test dataset are used to test the accuracy of the clamp recognition of the microstructural features after the network model is trained.

[0022] S3. Find the vector relationship between the pixel values of the pipeline and the clamp in the two-dimensional image (as shown in the schematic diagram of the principle of the feature attention mechanism association method). By calculation, obtain the vector relationship of the pixel change between the pipeline feature and the clamp feature. Through the pipeline feature constraint, the position of the clamp feature can be quickly identified and located. Therefore, the pipeline feature is calibrated as a constraint factor in the training dataset. Figure 1 As shown in the schematic diagram of the principle of the feature attention mechanism association method, by calculating the vector relationship of the pixel change between the pipeline feature and the clamp feature, the position of the clamp feature can be quickly identified and located through the pipeline feature constraint. Therefore, the pipeline feature is calibrated as a constraint factor in the training dataset.

[0023] S4. Label the clamp assembly structure features as the features to be recognized and extracted: Manually label the clamp features in the training dataset (so that the training set contains both the pipeline feature constraint factor label and the final clamp label to be recognized), which is used to train the neural network model for recognizing the microstructural features of aero-engine with an assembly attention mechanism and assembly relationship constraint; label the clamp features in the test dataset for testing the recognition accuracy of the microstructural feature recognition neural network model.

[0024] S5. Construct a neural network for recognizing microstructural features with an attention mechanism module and assembly relationship constraint; the microstructural feature recognition neural network model (such as Figure 2 the architecture diagram of the neural network model based on the attention mechanism) includes a backbone feature extraction network module, a candidate extraction network module (Region Proposal Network, RPN), and an adaptive classifier module based on the Classifier Weight Transformer (CWT) network model. Among them, the backbone feature extraction network module includes convolutional pooling and neural network parameter update calculations. Through training the micro-feature neural network model with an attention mechanism and assembly relationship constraint, feature extraction and expression are realized, and the generalization performance of the convolutional neural network is ensured. The Swin-transformer is used as the backbone network to extract the clamp features and background features (to obtain efficient and accurate feature expressions).

[0025] The candidate extraction network module (RPN) retrieves the anchor box positions from the feature map output by the backbone network based on the sliding window method, and then projects the anchor boxes onto the feature map to detect target features;

[0026] The adaptive classifier module (CWT) based on the classifier weight transfer network model is based on the encoder and decoder obtained after training the backbone feature extraction network. It extracts feature vectors from the image pixels in the training data and trains the classifier model using the cross-entropy loss function. For the construction of the classifier model, first, the weights of the encoder and decoder modules trained under a large number of general samples are frozen. The classifier module with the attention mechanism is retrained using the query set to adjust and calibrate the network model parameters and classify the feature maps output by the encoder and decoder. After training is completed, the construction of the backbone feature extraction network model is completed. At this time, this backbone feature extraction network has a certain generalization ability and has a relatively stable feature extraction ability for general data. The support set s in the labeled training data is used to optimize the classifier weight μ. Through the pre-trained feature extraction network, first, feature vectors f∈R d are extracted for the image pixels in the support set of each training data, and the feature dimension is denoted as d. During the pre-training process, the cross-entropy loss function is used to train the classifier model. However, the generalization performance of the pre-trained model still has deficiencies because it does not show good generalization performance for the data in the query set. To solve this problem, the present invention introduces a meta-learning module, which is a classifier weight converter (CWT). The network module structure is as Figure 2 the classifier adaptation layer module based on the attention mechanism in it, which is used for the adaptive recognition of query objects in small-sample data.

[0027] In the classifier training, CWT is used to learn how to adapt the classifier weights to a sampled class in each scenario. Formally, the input of the transformer is in the form of a (Query, Key, Value) triple. First, self-attention is used to extract features for all n pixels of the query image. To learn discriminative query condition information, the input parameters are:

[0028]

[0029] where F is the feature extracted from each image, μ is the fully connected classification parameter, W q 、W k and W vare learnable parameters (each parameter is represented by a fully connected layer), which project the classifier weights and query features into a high-dimensional space based on the attention mechanism, and dynamically adapt the weights of the classifier trained on the support set to each query image in an inductive manner, forming a classifier with the function of the attention mechanism for the query image.

[0030]

[0031] where ω * is the final weight, and softmax(*) is the row-wise softmax function for attention normalization. is a linear layer with an input dimension of d a and an output dimension of d o .

[0032] This method adapts the classifier weights to the query image through pairwise similarity and attention learning. By weighted aggregation and adjustment of the classifier weights, the within-class variation can be alleviated. The training objective is to apply the classifier weights adapted to the query samples to the segmentation prediction using the cross-entropy loss. In the meta-learning test phase, this method can be directly applied to new tasks or categories, applying the classifier weights optimized by the support set to the query image for adaptation to unseen categories. The support set and query image are used as inputs to the transformer to update the foreground / background classifier.

[0033] After meta-learning training, the parameters of the classifier weight transformer are fixed, and the adaptation for each query image is performed independently. Here, similar to other few-shot segmentation methods, by leveraging the assembly feature attributes of the clamp in the engine pipeline system, a combined attention mechanism is added to the backbone network module and the classifier network module, accurately and reliably realizing the feature recognition of few-shot clamp fasteners and micro-structural parts.

[0034] By training the micro-feature recognition neural network model with attention mechanism and assembly relationship constraints on the query set, a network model that can accurately recognize the micro-structural part features of the aero-engine pipeline system is obtained.

Claims

1. A micro-structure feature recognition method based on assembly attention mechanism constraints, comprising the following steps: S1. Collect 200-300 two-dimensional images from the assembly structure surface of the aircraft engine pipeline system including the micro-structure clamp through an industrial camera; S2, select pictures with clamp features, randomly divide them into training data set, verification data set and test data set, and the picture grouping ratio is 5:1:1; randomly select 80 pictures from the training data set and render them on 800 pictures randomly selected from the VOC data set as the support set for training the backbone network parameters of the neural network model, and the rest are used as the query set; The validation dataset is used to evaluate the model effect and adjust the hyperparameters. The test dataset is used to evaluate the segmentation effect of the backbone model. S3, calibrate pipeline features in the training data set as constraint factors; S4, marking the clamp assembly structural features as features that need to be identified and extracted; S5. Construct a neural network model for microstructure feature recognition with attention mechanism and assembly relationship constraints; The micro-feature recognition neural network model includes a backbone feature extraction network module, a candidate extraction network module, and an adaptive classifier module based on a classifier weight transfer network model, wherein the backbone feature extraction network module includes convolution pooling and parameter update calculation, and the feature extraction and expression are realized by training a micro-structure feature recognition neural network model with an attention mechanism and assembly relationship constraints to ensure the generalization performance of the convolutional neural network, and the Swin-transformer is used as the backbone network to extract the clamp features and background features; The candidate extraction network module retrieves the anchor box position from the feature map output by the backbone network in a sliding window manner, and then projects the anchor box onto the feature map to detect the target feature; The adaptive classifier module based on the classifier weight transfer network model is an encoder and decoder pre-trained based on the backbone feature extraction network, extracts feature vectors from image pixels in the training data, and trains the classifier model using a cross entropy loss function; the classifier model is constructed by first freezing the weight parameters of the encoder and decoder modules trained under a large number of general samples, retraining the classifier module with an attention mechanism using a query set, and adjusting and calibrating the network model parameters; The neural network model for micro-feature recognition based on the attention mechanism and assembly relationship constraints is trained through the query set, thus obtaining a network model that can accurately identify the features of micro-structure parts of the aero-engine piping system.

2. A micro-structure feature recognition method based on assembly attention mechanism constraints as claimed in claim 1, wherein: In step S2, 140 pictures with clamp features are selected and randomly divided into 100 training data sets, 20 verification data sets, and 20 test data sets, wherein the 20 verification data sets are used to recalibrate the trained network model, and the 20 test data sets are used to test the accuracy of microstructure feature clamp recognition after network model training.

Images