A composite material defect ultrasonic detection method based on convolution network and trajectory tracking

Abstract

The application discloses a composite material defect ultrasonic detection method based on a convolution network and trajectory tracking, and comprises the following steps: step 1, establishing an ultrasonic wave database; step 2, obtaining a data set from the ultrasonic wave database, constructing a deep learning neural network algorithm model, and determining neural network model parameters; step 3, training the constructed neural network deep learning algorithm model by using the collected data set; and step 4, embedding the trained neural network model parameters into an ultrasonic device, and performing real-time diagnosis classification and defect visualization labeling on ultrasonic wave signals collected by a current probe position. Real-time composite material defect diagnosis and labeling can be realized. The method is different from defect visualization of ultrasonic C scanning, and can solve the problem that existing composite material defect or damage detection relies on expert prior knowledge such as understanding of material characteristics of a tested component or extraction of predetermined signal characteristics based on physics.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of composite material defect detection, and particularly relates to a composite material defect ultrasonic detection method based on a convolution network and trajectory tracking. BACKGROUND

[0002] Composite materials have high specific strength and specific stiffness, can effectively reduce the weight of aircraft structures, and have the advantages of corrosion resistance and high damage safety. Composite materials have a higher and higher application proportion in aircraft structures due to excellent performance, and have developed from secondary structures to main structures and complex load-bearing structures. Composite components may have defects in the production process, and various types of damage may occur during use, including cracks, scratches, ablation, lightning strikes, dents, perforations, delamination and debonding. Detection and analysis of defects and damage are crucial. There are many methods for evaluating composite materials or components, and non-destructive testing is one of the important categories. Non-destructive testing refers to evaluating and testing materials or components without changing the original properties or damaging the tested objects, so as to characterize or find defects and damage. Non-destructive testing technology provides a cost-effective testing method, which can be used for individual investigation of samples or inspection of entire materials in the production process, and inspection of entire materials in the production quality control system. At present, the main methods for inspecting damage of aircraft composite structures include visual inspection, knock inspection, ultrasonic inspection, infrared thermal wave method and X-ray inspection.

[0003] Non-destructive testing using instruments is mainly used for in-field maintenance practice detection and production process quality evaluation, and ultrasonic testing technology is widely used in composite material detection because of the strong penetrating power and high sensitivity of ultrasonic waves, and the light and low-cost detection equipment, and is suitable for detecting delamination, debonding, adhesive pores and other damage or defects of composite material structures.

[0004] At present, ultrasonic testing method is a widely used non-destructive testing technology for composite materials. Traditional ultrasonic non-destructive testing technology relies on understanding of material properties of test components or extraction of predetermined signal features based on physics, and it is difficult to obtain expert prior knowledge of ultrasonic testing of different composite component damage, and manual signal interpretation has the problems of high labor cost and low detection efficiency. SUMMARY

[0005] In order to solve the problems in the prior art, the application provides a composite material defect ultrasonic detection method based on a convolution network and trajectory tracking, so as to realize real-time composite material defect diagnosis and labeling. The method is different from the defect visualization method of ultrasonic C scan, and can solve the problem that the existing composite material defect or damage detection relies on understanding of material properties of test components or extraction of predetermined signal features based on physics.

[0006] To achieve the above object, the present application provides the following technical solutions:

[0007] A composite material defect ultrasonic detection method based on convolution network and trajectory tracking, comprising the following steps,

[0008] Step 1, establishing an ultrasonic database;

[0009] Step 2, obtaining data sets from the ultrasonic database and constructing a deep learning neural network algorithm model to determine the neural network model parameters;

[0010] Step 3, training the constructed neural network deep learning algorithm model with the collected data set;

[0011] Step 4, embedding the trained neural network model parameters into the ultrasonic device to perform real-time diagnosis classification and defect visualization labeling on the ultrasonic signals collected by the current probe position.

[0012] Preferably, in step 1, the ultrasonic device is used to collect data from different regions of the sample, obtain ultrasonic signal sampling data with different sensitive waveforms and ultrasonic probe picture data, and establish an ultrasonic database containing different sensitive waveforms.

[0013] Preferably, in step 1, the establishment of the ultrasonic database comprises the following steps,

[0014] Step S11, placing the composite material sample to be measured on the detection platform;

[0015] Step S12, starting the ultrasonic device and adjusting the ultrasonic parameters;

[0016] Step S13, uniformly applying coupling agent to the composite material sample, and attaching the ultrasonic transceiver probe to the measured sample to sample data from different regions, obtaining ultrasonic signal sampling data;

[0017] Step S14, exporting the ultrasonic signal sampling data into training data set and testing data set;

[0018] Step S15, using a camera to collect pictures of the ultrasonic probe at different angles held by hand and clamped by an automatic scanning machine, obtaining ultrasonic probe picture data, and dividing the ultrasonic probe picture data into training data set and testing data set.

[0019] Preferably, in step 1, the data sets in the ultrasonic database include normal composite material data, defect composite material data and reinforced composite material data according to different sampling regions.

[0020] Preferably, in step 2, the one-dimensional convolution network model is used for feature extraction of the ultrasonic time sequence signal in the ultrasonic database, the visual feature of the ultrasonic probe is extracted by using the deep residual network model, and the extracted features are connected in parallel by using the classifier to construct a deep learning neural network algorithm model.

[0021] Further, in step 3, when training the one-dimensional convolution network model and the residual feature extraction backbone network model in the neural network deep learning algorithm model, gradient backpropagation is performed to update the parameters of the convolution network model.

[0022] Further, in step 3, the conjugate gradient method is used to predict the motion trajectory of the probe, the output of the one-dimensional convolution network module classifier is connected in parallel to the input of the prediction result, the position of the probe is recorded, and the trajectory of the probe with composite material defect information is obtained.

[0023] Preferably, in step 4, after real-time diagnosis classification and defect visualization labeling, an automatic scanning composite sample intelligent detection labeling system is built to automatically scan the composite sample and verify the visualization labeling result.

[0024] Further, the automatic scanning composite sample intelligent detection labeling system comprises a camera, a servo mechanism, a data processor, a probe and a servo mechanism controller.

[0025] The probe points to the measured sample, the probe is fixed on the servo mechanism, and the servo mechanism drives the probe to move; the camera points to the measured sample.

[0026] The data processor is used for processing the data collected by the camera and the probe; and the servo mechanism controller is used for controlling the servo mechanism.

[0027] The camera and the data processor are connected through a USB line for data transmission.

[0028] Further, the data processor is connected with a display, and the GUI window is used to display the diagnosis result, the defect information and the motion trajectory of the probe.

[0029] Compared with the prior art, the present application has the following beneficial technical effects:

[0030] The application provides a composite material defect ultrasonic detection method based on a convolution network and trajectory tracking, a one-dimensional convolution neural network model for detecting composite material damage is obtained by using a data-driven decision method and a deep learning technology, a neural network can be used to fuse multiple types of features, so that the model has the extraction and identification capability of multiple types of features, the defect state of the real-time position is recorded in combination with trajectory tracking, and the model is embedded into equipment, so that an ultrasonic equipment with real-time intelligent diagnosis function is obtained, the staff is facilitated to operate, the dependence on professionals is reduced, the cost is reduced, the efficiency is improved, and the problems of difficulty in positioning the encoder position in field work and complex curved surface work and unclear defect indication of the existing composite material damage detection method are solved. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 It is a schematic diagram of an ultrasonic intelligent recognition and labeling hardware system of the application;

[0032] Figure 2 It is a neural network model structure diagram constructed by the application;

[0033] Figure 3 It is a data flow thread diagram of an ultrasonic intelligent recognition and labeling system of the application;

[0034] In the drawings: 1 is a camera; 2 is a USB line; 3 is a servo mechanism; 4 is a data processor; 5 is a probe; 6 is a sample to be measured; 7 is a servo mechanism controller. DETAILED DESCRIPTION

[0035] The application will be further described in detail below in combination with specific embodiments, which are an explanation but not a limitation of the application.

[0036] The application provides a composite material defect ultrasonic intelligent detection and labeling method based on a convolution neural network and trajectory tracking, and the method is specifically performed according to the following steps S1-S5.

[0037] Step S1, an ultrasonic wave database containing sensitive waveforms of different features is established to provide a sample set for convolution network model training and testing;

[0038] A composite material defect sample piece with a stringer is designed and made, in this example, a custom composite laminate is used, and in two areas, an iron disc and a polytetrafluoroethylene sheet are embedded to simulate the delamination and crack defects of the composite material, and three reinforcing bars are connected below the laminate to simulate the stringer below the real aircraft skin, and the sample piece is divided into three areas of a normal composite laminate area, a defect area and a laminate with reinforcing bars.

[0039] The different areas of the sample piece are respectively subjected to data acquisition by using an ultrasonic equipment, and feature data with different sensitive waveforms are obtained, and the specific process of collecting data by using the ultrasonic equipment includes the following steps:

[0040] Step S11, placing the composite material sample on the testing platform and fixing the tested sample 6;

[0041] Step S12: Start the ultrasonic equipment and adjust relevant parameters such as material sound velocity and sampling frequency to appropriate values;

[0042] Step S13: evenly apply coupling agent on the composite material sample, attach the ultrasonic transceiver probe 5 to the sample 6 to sample ultrasonic signal data from different areas;

[0043] Step S14: The ultrasonic signal sampling data is exported and divided into a time series signal training data set and a time series signal test data set. The time series signal data set includes three types of data: normal composite material data, defective composite material data, and composite material data with reinforcement ribs according to different sampling areas.

[0044] Step S15: Use a camera to capture images of the ultrasound probe at different angles when it is handheld and clamped by an automatic scanning machine, construct a data set from the ultrasound probe images captured by the camera, and divide it into a training data set and a test data set.

[0045] Step S2: Design and construct a deep learning neural network algorithm for the data set and determine the network model parameters;

[0046] This embodiment uses one-dimensional convolution to extract features from ultrasonic time series signals, uses a deep residual network to abstract the visual features of the ultrasonic probe, and connects the abstracted features in parallel with two classifiers to construct a neural network structure.

[0047] like Figure 2As shown in the figure, the architecture of the one-dimensional convolutional neural network for extracting the timing signal feature includes five convolutional layers, five pooling layers, one fully connected layer, and one output layer. In the first convolutional layer, 16 1x5 kernel filters are used. After convolution, a relu activation function is used to generate feature signals, resulting in 16 feature signals, which are then pooled using the maximum pooling method. The kernel filters used in the subsequent convolutional layers are all 1x3, and the output channel number is twice the input channel number. The pooling is maximum pooling. After five layers of convolution and pooling operations, the feature coefficients are combined nonlinearly. Then, the outputs of these convolutions are combined in a linear connection, and finally, a probability vector with the same length as the number of signal categories is output. The visual feature abstraction network is composed of a residual network and a convolutional network in parallel. The first layer is a 7x7x64 convolution, followed by 3, 4, 6, and 3, a total of 16 building blocks, each with 3 layers. Finally, there is a fully connected layer. The resnet50 extracts the depth features and the center position of the ultrasonic probe contained in each frame of picture. The one-dimensional convolutional network extracts signal features. These two inputs are a picture and a one-dimensional signal, so different networks are used to extract features. In this example, the output layers of the one-dimensional convolutional network and the residual network are merged together and the results are input into a two-layer convolutional network. The classifier obtains the classification results of the current input picture signal and ultrasonic timing signal.

[0048] Step S3, training the constructed neural network deep learning algorithm with the collected data set;

[0049] In this example, the model parameters of the deep learning model constructed in step S2 are trained. The model mainly consists of two modules, and the main training parameters are the one-dimensional convolutional network module and the residual feature extraction backbone network. After inputting the ultrasonic timing signal data set into the one-dimensional convolutional network branch, the signal feature waveform is extracted through one-dimensional convolution and pooling operations. In the end-to-end training, the loss value is fed back through the training set data to optimize the parameters of the one-dimensional convolutional network module. After inputting the picture training data set into the target classification branch network, the ResNet50 backbone network and the additional convolution block (Classification specific Features) are used to extract the depth feature map. Then, the feature map is input into the model predictor composed of an initializer and a recurrent optimizer. The model predictor outputs the weights of the convolutional layer, which are used in the target classification operation of the feature map extracted from the test frame to obtain the probability score of the tracking target. The brighter the color, the higher the confidence that the position is the target position, and the peak of the score is the new position of the target in the test frame.

[0050] When end-to-end training is performed, test loss is calculated behind the network of the test frame, then gradient back propagation is performed with the pytorch deep learning framework, and the parameters of the convolutional network are updated, so as to optimize the network parameters such as feature extraction, online learning, etc. In the regression process, the conjugate gradient method is used instead of the steepest descent method to obtain the ideal gradient direction through iteration, predict the motion trajectory of the probe, and connect the output of the one-dimensional convolutional network module classifier to the input of the prediction result to record the position of the probe and obtain the trajectory of the probe with composite material defect information.

[0051] Step S4, embedding the trained neural network model parameters into the developed ultrasonic equipment to realize real-time diagnosis classification and defect visualization labeling of the ultrasonic signal collected at the current probe position;

[0052] In this embodiment, the trained neural network model in step S3 is used to construct the same feedforward neural network as in step S2 in the development package of the ultrasonic equipment according to convolution operation, pooling operation and full connection operation. The network parameters of the feedforward neural network are given by the model file with the highest accuracy trained in step S2, that is, a signal classification thread is created in the software control of the ultrasonic equipment. The work of this thread is to perform convolution, pooling, full connection and other operations on the input one-dimensional time series signal and the picture data collected by the camera using the feedforward neural network, obtain the confidence of the signal belonging to different regions through the final classifier, and display the result with the maximum probability to the visualization window.

[0053] Among them, the data input of the signal diagnosis network module is controlled by the data reading thread. This thread uses a pointer to access the data buffer at a fixed time interval. When there is data in the data buffer, the data is stored in the Data class, and the time series signal input of the model is completed after calling. The data input of the probe visual tracking module is controlled by the picture reading thread. This thread also uses a pointer to access the data buffer at a fixed frame rate. When there is data in the data buffer, the data is stored in the Image class, and the picture data input of the model is completed after calling.

[0054] After completing the real-time diagnosis of the ultrasonic signal, the results of the diagnosis are visualized and displayed in the GUI window, and the signal state of the current position and the defect information of the previously detected region are also displayed.

[0055] Step S5, building an automatic scanning composite material sample intelligent detection and labeling system to realize full-automatic scanning of the composite material sample and visualization labeling verification of the results;

[0056] The automatic scanning composite sample piece intelligent detection and labeling system comprises a camera 1, a servo mechanism 3, a data processor 4, a probe 5 and a servo mechanism controller 7; the probe 5 points to the sample 6 to be detected, the probe 5 is fixed on the servo mechanism 3, and the servo mechanism 3 drives the probe 5 to move; the camera 1 points to the sample 6 to be detected; the data processor 4 is used for processing the data collected by the camera 1 and the probe 5; the servo mechanism controller 7 is used for controlling the servo mechanism 3; and the camera 1 and the data processor 4 are connected through a USB line 2 for data transmission.

[0057] In the embodiment, the previous ultrasonic device embedded with the convolutional neural network model, the visual-based probe trajectory tracking system and the automatic scanning platform are connected together. The automatic scanning platform is composed of a scanning platform control system, a power supply and a three-way guide rail, and the ultrasonic probe, the composite test piece and the camera are combined and installed on the scanning platform to obtain a complete system as shown in the figure. Figure 1 The detection and scanning process of the system can be manually or automatically operated, the scanning speed and direction can be controlled by the set program, and the safety of the equipment and the experiment operation is ensured by the sensor sensing the pressure of the piezoelectric crystal and being powered off when the set pressure value is exceeded. In step S4, the visualization of the probe trajectory is completed. The signal category data file output in step S3 is input into the trajectory tracking process, and the current trajectory display color is controlled based on the current ultrasonic signal category before trajectory visualization: green corresponds to the normal skin area category, red corresponds to the defect composite material area category, and yellow corresponds to the reinforcing rib area category. Thus, the software and hardware connection of the whole system is completed, the automatic scanning of the sample piece and the real-time diagnosis of the ultrasonic signal at any position and the labeling of the composite material defect in the scanning moving area are realized.

[0058] The present application uses one-dimensional convolution to extract the sensitive waveform of the composite material, and obtains an ultrasonic signal diagnosis model with high accuracy. In view of the difference between the deep learning environment and the hardware and software control environment of the ultrasonic device, a deep learning neural network is used to train network model parameters, a feedforward neural network with the same network structure is constructed to call model parameters, the complex back propagation operation process is removed, the model is lightweight deployed in the ultrasonic detection device, and real-time intelligent diagnosis and defect visualization labeling of the ultrasonic signal are realized. The present application uses the developed intelligent ultrasonic device to realize the visualization labeling of the defects on the scanning path under the condition of single-point high-accuracy diagnosis without information loss, realizes the automatic nondestructive testing of the composite material, and verifies that the method has good research prospect.

[0059] The various embodiments in the specification are described in a related manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0060] The above only describes the preferred embodiments of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A composite material defect ultrasonic detection method based on convolutional network and trajectory tracking, characterized in that: The following steps are included: Step 1: Establish an ultrasonic database containing sensitive waveforms with different characteristics to provide a sample set for convolutional network model training and testing; Specifically, Step S11, placing the composite material sample to be tested on a testing platform; Step S12, starting the ultrasound equipment and adjusting the ultrasound parameters; Step S13: evenly apply coupling agent on the composite material sample, attach an ultrasonic transceiver probe to the sample to sample data from different areas, and obtain ultrasonic signal sampling data; Step S14, exporting the ultrasonic signal sampling data and dividing it into a time series signal training data set and a time series signal test data set; Step S15: Using a camera to capture images of the ultrasound probe at different angles, when the probe is held by hand and when the probe is held by an automatic scanning machine, to obtain ultrasound probe image data, and dividing the ultrasound probe image data into a training data set and a test data set; Step 2: Obtain a data set from the ultrasound database, build a deep learning neural network algorithm model, and determine the neural network model parameters; Specifically, A one-dimensional convolutional network model is used to extract features from ultrasonic time series signals in an ultrasound database. A deep residual network model is used to extract the visual features of the ultrasound probe. The extracted features are connected in parallel using classifiers to construct a deep learning neural network algorithm model. Step 3: Train the constructed neural network deep learning algorithm model using the collected and constructed data set; Specifically, The conjugate gradient method is used to predict the probe's trajectory. The output of the one-dimensional convolutional network module classifier is connected in parallel to the input of the prediction result. The probe's position is recorded to obtain the probe's trajectory with composite material defect information. Step 4: Embed the trained neural network model parameters into the ultrasonic equipment to perform real-time diagnosis and classification and visual defect annotation on the ultrasonic signal collected at the current probe position.

2. The method for ultrasonic detection of composite material defects based on convolutional networks and trajectory tracking according to claim 1, characterized in that: In step 1, the time series signal data sets in the ultrasonic database are divided into three types according to different sampling areas: normal composite material data, defective composite material data, and composite material data with reinforcement ribs.

3. The method for ultrasonic detection of composite material defects based on convolutional networks and trajectory tracking according to claim 1, characterized in that: In step 3, when training the one-dimensional convolutional network model and the residual feature extraction backbone network model in the neural network deep learning algorithm model, gradient backpropagation is performed to update the parameters of the convolutional network model.

4. The method for ultrasonic detection of composite material defects based on convolutional networks and trajectory tracking according to claim 1, characterized in that: In step 4, after real-time diagnosis and classification and visual annotation of defects, an automated intelligent inspection and annotation system for scanning composite material samples is built to perform fully automatic scanning of composite material samples and visual annotation verification of the results.

5. The method for ultrasonic detection of composite material defects based on convolutional networks and trajectory tracking according to claim 4, characterized in that: The automated scanning composite material sample intelligent detection and marking system comprises a camera (1), a servo mechanism (3), a data processor (4), a probe (5) and a servo mechanism controller (7); The probe (5) points to the sample to be tested (6), the probe (5) is fixed on the servo mechanism (3), and the servo mechanism (3) drives the probe (5) to move; the camera (1) points to the sample to be tested (6); The data processor (4) is used to process data collected by the camera (1) and the probe (5); the servo mechanism controller (7) is used to control the servo mechanism (3); Data is transmitted between the camera (1) and the data processor (4) via a USB cable (2).

6. The method for ultrasonic detection of composite material defects based on convolutional networks and trajectory tracking according to claim 5, characterized in that: The data processor (4) is connected to a display and uses a GUI window to display the diagnosis results and defect information as well as the motion trajectory of the probe (5).

Images