Intelligent evaluation method for infrared thermography layering defect detection capability of carbon fiber composite material

Abstract

The invention provides an intelligent evaluation method for the infrared thermography layering defect detection capability of a carbon fiber composite material, and the method specifically comprises the following steps: A, intelligently recognizing and marking the infrared thermography layering defects of the carbon fiber composite material, and automatically calculating the signal-to-noise ratio of each defect; and step B, evaluating the probability of defect detection according to the calculated signal-to-noise ratio, namely drawing a POD curve. According to the method, the POD is applied to the field of infrared thermal imaging layering defect detection of the CFRP laminates, the mode that the target detection network is combined with automatic labeling and a signal-to-noise ratio calculation algorithm is adopted, and the method has the advantages of being high in defect recognition precision, high in speed, high in defect signal-to-noise ratio calculation automation degree and the like. The method is suitable for defect detection of the carbon fiber composite material product, can improve the automation and quantification level of internal quality control of the carbon fiber composite material, and can quantitatively evaluate the defect detection capability of the detection technology.

Description

Technical Field

[0001] The present invention belongs to the field of nondestructive testing of composite materials and the field of intelligent technology application, and specifically relates to an intelligent evaluation method for the detection capability of delamination defects of carbon fiber composite materials by infrared thermal imaging based on a YOLO neural network. Background Art

[0002] Carbon Fiber Reinforced Plastics (CFRP) is a composite material with resin as the matrix and carbon fiber as the reinforcement. Compared with traditional metal materials, its structure is lighter, stronger and more corrosion-resistant. With the wide application of CFRP in aerospace, automobile, navigation and other fields, its reliability and safety have become the focus of attention. Among them, the damage defects inside CFRP are an important factor affecting its structural safety, especially the internal delamination, which will continue to expand with the action of external stress, greatly reducing the structural strength of CFRP and causing its failure. Since the damage defects are located inside CFRP, they are invisible to maintenance personnel during visual inspection and become potential safety hazards. Therefore, the use of infrared thermal imaging nondestructive testing technology to detect CFRP can realize internal visual defects and meet the qualitative requirements of defect detection, thereby realizing the detection of internal delamination defects of the structure.

[0003] Infrared thermal imaging nondestructive testing technology heats the surface of the inspected object through an external heat source, allowing the heat to propagate inside the inspected object and reflect back to the surface when encountering internal defects, resulting in differences in the surface temperature distribution of the inspected object. The operator can determine the defects by observing the thermal imaging temperature distribution on the surface of the inspected object obtained by the infrared thermal imager.

[0004] When using infrared thermal imaging nondestructive testing technology for actual testing, there are system errors, human operation errors, various accidental errors and environmental interference, which will lead to changes in the response of defects to the inspection and the uncertainty of detection. Therefore, in order to evaluate the reliability of the detection technology, the concept of probability of detection (POD) is introduced. POD is an important evaluation index in nondestructive testing technology. It conducts test tests on a large number of samples, analyzes the test results data with statistics, and makes a reasonable estimate of the detection ability of the detection technology.

[0005] At present, the research and application of POD are mainly concentrated in the field of metal fatigue cracks, and the detection probability estimation of cracks of different sizes and depths is studied respectively. However, there are few studies on the POD level of CFRP laminate infrared thermal imaging delamination defects. At present, the defects inside the CFRP structure after infrared thermal imaging mainly rely on manual qualitative identification, which is easily affected by the subjective factors, experience and environment of the inspectors. Moreover, the analysis of the detection probability after qualitative identification mostly requires manual labeling and manual calculation, which can only realize the visual identification of defects and the calculation of one-by-one labeling, which is inefficient and time-consuming. On the basis of qualitative detection, how to further realize the quantitative detection of defects, realize the automatic labeling of defects and the quantitative analysis of detection probability has important guiding significance for the development of CFRP structure maintenance work. Summary of the invention

[0006] In view of the problems existing in the prior art, the present invention provides a method for intelligent detection and detection capability evaluation of carbon fiber composite material infrared thermal imaging delamination defects based on a YOLO neural network, comprising steps A and B, as follows:

[0007] Step A: Intelligently identify and mark the delamination defects of carbon fiber composite materials by infrared thermal imaging, and automatically calculate the signal-to-noise ratio of each defect;

[0008] Step A01, obtaining an original thermal image of a carbon fiber composite material structure with known internal defects used for experiment through an infrared thermal imaging detection system, hereinafter referred to as “carbon fiber composite material structure” for short “material structure”;

[0009] Use the laser infrared thermal imaging detection system to detect defects in the material structure and obtain the original thermal image sequence data of the tested object; select the original thermal images with clearly visible defects as the source of part of the data set for the subsequent target detection neural network;

[0010] Step A02, using a thermal image data processing algorithm to process the original thermal image data to obtain a processed thermal image, which more clearly visualizes the defect features to further improve the defect detection effect; the processed thermal image provides thermal image data for the subsequent training of the target detection network;

[0011] Step 1: Use thermal imaging data processing algorithm to process the original thermal imaging data to further highlight the defect effect;

[0012] Step 2: Convert the RGB image into a grayscale image;

[0013] Step 3: Perform linear transformation on the converted grayscale image to achieve the effect of image enhancement and contrast control;

[0014] Step A03, constructing a target detection neural network;

[0015] Use YOLOv4 target detection network to build target detection neural network;

[0016] Step A04, marking defect areas of B% of the thermal image processed in step A02 to construct a defect thermal image training data set;

[0017] Image annotation software is used to mark the positions of defects that can be distinguished by the human eye in the thermal image with boxes as close to their edges as possible, and a label file containing defect annotation information is generated. The annotation information file and its corresponding annotated thermal image together constitute the training data set of the target detection network;

[0018] Step A05, training the target detection neural network using the constructed defect thermal image training data set;

[0019] The YOLOv4 network is trained with the constructed defect thermal image training data set. The trained YOLOv4 network is verified with thermal images without defects being labeled. When the training accuracy is not high, more thermal images containing defects are obtained through detection experiments or by selecting other thermal image data processing algorithms to expand the data volume of the training data set and improve the accuracy of network training; the target detection neural network is trained with the defect thermal image data set constructed in step A04; (100-B)% of the thermal images processed in step A02 and without defects being labeled are randomly selected to form a verification set, and the detection neural network with trained weights is used for training, and the recognition ability of the target detection neural network is verified with the labels formed by the above annotations;

[0020] Step A06, using the trained target detection neural network to automatically identify defects in the thermal image, marking each defect with a box, and realizing automatic counting of identified defects;

[0021] Inputting the original thermal image or processed thermal image without defect marking into the target detection neural network can complete the defect recognition. At this time, the length and width of the marked box are m and n respectively, which are recorded as box a. 1 ;After processing, the defects in the figure are marked with boxes;

[0022] Step A07, calculating the mean and variance of the pixels at the defective locations within each marked frame, and calculating the signal-to-noise ratio of each defect;

[0023] Three boxes with length and width of m, n, 2m, 2n, 3m, and 3n are respectively denoted as box a 1 、a 2 、a 3 , convert the annotated defect thermal image into a pixel value matrix, and automatically calculate the signal-to-noise ratio of each defect according to the signal-to-noise ratio calculation method. The calculation formula of the signal-to-noise ratio is shown in formulas (1)-(4);

[0024]

[0025] in, For box a 1 The mean value of pixels in the image is the mean value of pixels in the defective part. For box a 2 with a 3 The mean value of pixels in the enclosed circular frame is the mean value of pixels in the image without defects. Marked as box a 2 with a 3 The standard deviation of the pixels within the circular frame is the standard deviation of the pixel values ​​of the image without defects. The calculation formula is shown in formulas (2) to (4):

[0026]

[0027] in, Indicates a k The pixel value at x=i, y=j, The box a k The number of pixels in each row or column, k = 1, 2, 3; Indicates in a 3 -a 2 The standard deviation of the pixel values ​​in the region, i,j∈a 3 -a 2 Represents box a k The coordinate value of a 3 -a 2 Within the area, a 2 ∈a 3 Indicates area a 2 Included in area a 3 middle;

[0028] The trained neural network simultaneously outputs the labeled defect thermal image and the signal-to-noise ratio of each defect;

[0029] Step B: Based on the calculated signal-to-noise ratio, evaluate the probability of defect detection, i.e. draw a POD curve;

[0030] Step B01, calculate the diameter-to-depth ratio of each defect, denoted as r; the calculation formula is as follows:

[0031]

[0032] Where D is the diameter of the defect and d is the depth of the defect;

[0033] Step B02, performing linear regression analysis on the signal-to-noise ratio SNR and the diameter-to-depth ratio r of each defect;

[0034] The general linear model is expressed in formula (6):

[0035] Y=kX+b+ε (6)

[0036] Among them, Y represents the y-axis variable, X represents the x-axis variable, that is, the Gauss-Krüger xoy plane rectangular coordinate system in mathematics; the x-axis represents the diameter-to-depth ratio r, and the y-axis represents the signal-to-noise ratio SNR value of each defect; k, b, and ε are all linear regression parameters, k represents the slope, b represents the intercept, and ε represents the error term, and the error term ε obeys a normal distribution with a mean of 0 and a standard deviation of τ;

[0037] The linear relationship between SNR and r is analyzed. The linear relationship between SNR and r is expressed as SNR vs. r below. Four possible models are selected, namely: SNR vs. r, log(SNR) vs. r, SNR vs. log(r) and log(SNR) vs. log(r). The model showing the most linear correlation is selected and substituted into expression (6), which is rewritten as:

[0038] SNR / log(SNR)=kr / klog(r)+b+ε (7)

[0039] Step B03, establish a detection probability model to determine whether all information about the existence of the defect is contained in the signal-to-noise ratio SNR; POD(r) is used to represent the detection probability function, that is, the detection probability function can be obtained from the relationship between the signal-to-noise ratio SNR and the diameter-to-depth ratio r of the defect, and the expression is as follows:

[0040]

[0041] Among them, POD(r) represents the detection probability function, f r (SNR) is the probability density function of the signal-to-noise ratio SNR corresponding to a specific diameter-to-depth ratio r of the defect, S dec Indicates the threshold of the defect signal-to-noise ratio, P(SNR>S dec ) means SNR>S dec The probability of , d(SNR) represents the differentiation of the variable SNR;

[0042] Probability density function f r (SNR) depends on the statistical distribution characteristics of the error term ε, which follows a normal distribution with a mean of 0 and a standard deviation of τ. Therefore, equation (9) can be expressed as follows:

[0043]

[0044] Among them, φ(·) represents the standard normal distribution function / lognormal cumulative distribution function, μ and σ represent the mean and variance of the function (general variables), respectively. According to the different linear relationships between SNR vs. r, the expression of POD(r) function is different, and the expression of the mean μ of the function is also different, as shown in Table 1;

[0045] Table 1 Parameter expressions of different linear relationships

[0046]

[0047] In the table, S dec Indicates the judgment threshold of defect signal-to-noise ratio;

[0048] Step B04, using the maximum likelihood estimation method to solve the unknown parameters k, b, τ in the detection probability function;

[0049] Setting random variables That is, z obeys the standard normal distribution, and its probability density function is expressed as follows:

[0050]

[0051] Substituting z into equation (6), the expression of the random variable z will be different for different linear relationships, as shown in Table 2 below:

[0052] Table 2 Random variable z expressions for different linear relationships

[0053]

[0054] According to formulas (1)-(7), the signal-to-noise ratio obtained by the i-th detection is SNR i The probability of Indicates that z i Represents the random variable corresponding to the i-th detection, and its corresponding probability density function The expression is shown in formula (14), and the likelihood function is constructed as follows:

[0055]

[0056] Where N represents the number of defects detected;

[0057] Transform equation (16) logarithmically and Substitute z in formula (15), and the likelihood function L is as follows:

[0058]

[0059] Using the maximum likelihood estimation method, we calculate the partial derivatives of k, b, and τ respectively, set them equal to zero, solve the values ​​of the three unknown parameters k, b, and τ, and substitute them into Table 2 to calculate the mean μ and standard deviation σ of the detection probability function POD(r);

[0060] Step B05, plot the POD probability curve and the POD probability curve with a confidence level of 95%, and obtain r 90 With r 90 / 95 Two indicators for evaluating the capability of nondestructive testing technology are used to evaluate the detection capability of the testing technology; among them, r 90 It represents the defect diameter-to-depth ratio when the detection probability is 90%; r 90 / 95 It represents the defect diameter-to-depth ratio when the detection probability is 90% and the confidence level is 95%.

[0061] In one embodiment of the present invention, when the training accuracy is low, more thermal images containing defects are obtained through detection experiments or by selecting other thermal imaging data processing algorithms to expand the data volume of the training data set and improve the accuracy of network training.

[0062] In a specific embodiment of the present invention, in step A01, the infrared thermal imaging system used uses laser as a heat source to heat the inspected CFRP structure, and the infrared thermal imager in the infrared thermal imaging system records the temperature changes of the surface of the material structure during 5s heating and 5s cooling at an imaging frame rate of 50Hz, and obtains a thermal imaging video containing 500 frames of original thermal images.

[0063] In another specific embodiment of the present invention, in step A01, the last frame of thermal image in the heating stage is taken as a thermal image for defect calibration.

[0064] In another embodiment of the present invention, in Step 1 of Step A02: the thermal imaging data processing algorithm includes a thermal imaging signal reconstruction method, a principal component analysis method, and a pulse phase method.

[0065] In yet another specific embodiment of the present invention, the thermal imaging data processing algorithm adopts principal component analysis.

[0066] In another specific embodiment of the present invention, in Step 2 of Step A02, the RGB image is converted into a grayscale image by taking the weighted sum of the three-channel color components as the grayscale value, that is, Grayscale = 0.299×R+0.587×G+0.114×B.

[0067] In still another specific embodiment of the present invention, in step A04, B=70.

[0068] In yet another embodiment of the present invention, in step A06, the following steps are used to implement automatic counting of identified defects:

[0069] Step 1: Use the Roberts operator to perform edge detection on the obtained grayscale defect thermal image to generate a binary image. The purpose of edge detection is to identify the defect annotation box in the image and convert the grayscale image into a binary image. Since the image is also a two-dimensional matrix, the black area in the image represents that the element value at this position of the matrix is ​​0, and correspondingly, the white part represents that the element value is 1, thereby obtaining a black and white defect thermal image.

[0070] Step 2: Mark the connected components of the binary image obtained after edge detection. The method of connecting component marking is to assign different values ​​to each connected region in the binary image to achieve the location, counting and marking of the connected regions. That is, determine whether the element value is 1 in the order of row and column. For example, element a ij =1, judge a ij The surrounding 4 adjacent elements a i-1,j 、a i+1,j 、a i,j-1 、a i,j+1 Is the value of a also 1? If there is an element with a value of 1, then compare the element with a ij Mark them as the same type, and then determine whether the four adjacent elements around each other element of the same type are 1, until the same element as a is marked as ij When all elements of the same type are searched, that is, until there are no unconnected elements around the connected elements, the area formed by the connection of these elements is called a connected component;

[0071] Step 3: Find the next connected component until all connected components in the binary graph are marked with their ordinal numbers; mark each defect labeling box as a connected component, the value of all elements in the first connected component is 1, the value of all elements in the second connected component is 2, and so on. The value of the Nth labeling box is N.

[0072] In another embodiment of the present invention, in step A07, the specific steps of marking three boxes with lengths and widths of m, n, 2m, 2n, 3m, and 3n include:

[0073] Step 1: For the image matrix b after the connected components are marked, first find the maximum value C in b, where C is the connected components in the image, that is, the number of labeled boxes;

[0074] Step 2: Let i = 1, 2, ..., C, and find the row number set r of the element with value i in matrix b in turn i With column number set c i ;

[0075] Step 3: Find the row number set r respectively i With column number set c i The maximum value r imax and c imax With the minimum value r imin and c imin ; Therefore, the i-th annotation box a 1 The length m and width n are r imax -r imin With c imax -c imin ;

[0076] step4: box a 2 Line number set r i The maximum values ​​of and Round up, the minimum values ​​are and Round down; box a 3 Line number set r i The maximum values ​​of imax +m and c imax +n rounded up, the minimum value is r imin -m and c imin -n round down;

[0077] The three marked boxes a are obtained 1 、a 2 、a 3 .

[0078] Compared with the existing detection probability assessment method, the present invention applies POD to the field of infrared thermal imaging delamination defect detection of CFRP laminates. Compared with the visual recognition, manual labeling and calculation process of traditional POD, it innovatively adopts the method of combining target detection network with automatic labeling and signal-to-noise ratio calculation algorithm, which has the advantages of high defect recognition accuracy, fast speed, high degree of automation of defect signal-to-noise ratio calculation, etc. This method is suitable for defect detection of carbon fiber composite products, can improve the automation and quantification level of internal quality control of carbon fiber composite materials, and can quantitatively evaluate the defect detection ability of this detection technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 It is a flowchart of a specific implementation method of the carbon fiber composite material infrared thermal imaging delamination defect intelligent detection and detection capability evaluation method based on the YOLO target detection neural network of the present invention;

[0080] Figure 2 It is the original thermal image obtained by inspecting the CFRP structure through the laser infrared thermal imaging system in the example;

[0081] Figure 3 This is the processed thermal image obtained by principal component analysis;

[0082] Figure 4 is the thermal image obtained after grayscale conversion and image enhancement processing, where Figure 4 (a) shows a grayscale image, Figure 4 (b) shows the use of linear transformation to enhance the grayscale image;

[0083] Figure 5 This is a schematic diagram of using the YOLO_mark annotation tool to annotate defects in an example, where "defect" means "defect";

[0084] Figure 6 The trained YOLOv4 network identifies defects in the input thermal image and annotates the thermal image with the annotation box. In the figure, "defect" means "defect";

[0085] Figure 7 The signal-to-noise ratio value of each defect output by the neural network is automatically calculated;

[0086] Figure 8 This is the result of linear regression analysis;

[0087] Fig. 9 is the POD curve drawn. DETAILED DESCRIPTION

[0088] The specific implementation method of the present invention consists of two parts. The first part (step A) is to intelligently identify and mark the delamination defects of infrared thermal images of carbon fiber composite materials, and automatically calculate the signal-to-noise ratio of each defect. The second part (step B) is to evaluate the probability of defect detection based on the calculated signal-to-noise ratio, that is, to draw a POD curve.

[0089] Step A: Intelligently identify and annotate carbon fiber composite material infrared thermal image delamination defects, and automatically calculate the signal-to-noise ratio of each defect, as follows.

[0090] Step A01, obtaining an original thermal image of a carbon fiber composite material structure with known internal defects used for experiment through an infrared thermal imaging detection system, hereinafter referred to as “carbon fiber composite material structure” for short “material structure”;

[0091] In one example of the present invention, a laser infrared thermal imaging detection system is first used to perform defect detection on a carbon fiber composite material structure to obtain a sequence of original thermal image data of the test piece. The infrared thermal imaging system and its operation method are well known to those skilled in the art and will not be described in detail. The infrared thermal imaging system used in this example uses laser as a heat source to heat the CFRP structure being inspected. The infrared thermal imager in the infrared thermal imaging system records the temperature changes on the surface of the material structure during a 5s heating period and a 5s cooling period (stop heating) at an imaging frame rate of 50Hz. A thermal imaging video containing 500 frames of original thermal images is obtained, and the original thermal images in which defects are clearly visible are selected as a partial data set source for the subsequent target detection neural network, such as the 250th frame of thermal image (i.e., the last frame of thermal image recorded in the heating stage, at which time the temperature value in the thermal image reaches the maximum and the resolution of defects reaches the highest). Figure 2 shown.

[0092] In a specific embodiment of the present invention, the last frame of thermal image in the heating stage is taken as the thermal map for defect calibration.

[0093] Step A02, using a thermal image data processing algorithm to process the original thermal image data to obtain a processed thermal image, which more clearly visualizes the defect features to further improve the defect detection effect; in addition, the processed thermal image can also provide thermal image data for the subsequent training of the target detection network;

[0094] Step 1: Conventional thermal imaging data processing algorithms such as thermal imaging signal reconstruction, principal component analysis, pulse phase method, etc. (VPVavilov. Review of pulsed thermal NDT: Physical principles, theory and data processing. 2015). In this example, principal component analysis (N. Rajic. Principal component thermography for flaw contrast enhancement and flaw depth characterization in composite structures. 2002) is used to process the original thermal imaging data to further highlight the defect effect. In the example of the present invention, the image processing result is as follows Figure 3 As shown, compared Figure 3 and Figure 2 , the number of defects detected increases, and the clarity of defects is significantly improved;

[0095] Step 2: Convert the RGB image to a grayscale image. The conversion method is to use the weighted sum of the three-channel color components as the grayscale value, that is, Grayscale = 0.299 × R + 0.587 × G + 0.114 × B;

[0096] Step 3: Perform a linear transformation on the converted grayscale image (the linear transformation method is well known to those skilled in the art) to achieve the effect of image enhancement and contrast control. The image processing result is as follows: Figure 4 As shown in the figure, by comparison, the image contrast is significantly enhanced after linear transformation, and the contrast of defects is significantly improved.

[0097] Step A03, constructing a target detection neural network;

[0098] At present, target detection neural networks are widely used in various industries, and there are many open source target detection network models on the Internet, most of which can meet the needs of defect detection in this article. In this example, the YOLOv4 target detection network (https: / / arxiv.org / abs / 2004.10934) is used. Its open source code source can be obtained from https: / / github.com / AlexeyAB / darknet, which includes the construction, training and verification methods of the network. The specific network construction, training and verification methods involved in the subsequent steps will not be repeated.

[0099] Step A04, marking defect areas on 70% of the thermal images processed in step A02 to construct a defect thermal image training data set;

[0100] Since the target detection network used in this example is YOLOv4, the corresponding image annotation software YOLO_mark is used to mark the positions of the defects that can be distinguished by the human eye in the thermal image with boxes as close to their edges as possible, and generate a file containing defect annotation information. The annotation information file and its corresponding annotated thermal image together constitute the training data set of the target detection network. Figure 5 Annotate a schematic diagram for the defect.

[0101] Step A05, training the target detection neural network using the constructed defect thermal image training data set;

[0102] In the example, the constructed defect thermal image training data set is used to train the YOLOv4 network (the training method is well known to those skilled in the art). The trained YOLOv4 network can be verified using thermal images that have not been labeled with defects. When the training accuracy is not high, more thermal images containing defects can be obtained through detection experiments or by selecting other thermal image data processing algorithms (refer to steps A01 and A02) to expand the data volume of the training data set and improve the accuracy of network training. The target detection neural network is trained with the defect thermal image data set constructed in step A04; 30% of the thermal images processed in step A02 and without defects labeled are randomly selected to form a verification set, and the detection neural network with trained weights is used for training, and the recognition ability of the target detection neural network is verified with the labels formed by the above annotations;

[0103] Step A06, using the trained target detection neural network to automatically identify defects in the thermal image, marking each defect with a box, and realizing automatic counting of identified defects;

[0104] The defect recognition can be completed by inputting the original thermal image or the processed thermal image without defect marking into the target detection neural network. At this time, the length and width of the marked box are m and n respectively, which are recorded as box a. 1 ,Since the neural network has the ability to learn, identify and calculate autonomously, the annotation box is not limited to a square, but a margin is left to better reflect the neural network's ability to identify and detect images. After processing, the defects in the image are marked with square boxes;

[0105] Step A07, calculating the mean and variance of the pixels at the defective locations within each marked frame, and calculating the signal-to-noise ratio of each defect based on the mean and variance;

[0106] Three boxes with length and width of m, n, 2m, 2n, 3m, and 3n are respectively denoted as box a 1 、a 2 、a 3 , convert the annotated defect thermal image into a pixel value matrix (this conversion method is well known to those skilled in the art), and automatically calculate the signal-to-noise ratio of each defect according to the signal-to-noise ratio calculation method. The calculation formula of the signal-to-noise ratio is shown in formulas (1)-(4). Figure 6 This is the output labeled defect thermal image. Figure 7 is the calculated signal-to-noise ratio value output by the neural network.

[0107]

[0108] in, For box a 1 The mean value of pixels in the image is the mean value of pixels in the defective part. For box a 2 with a3 The mean value of pixels in the enclosed circular frame is the mean value of pixels in the image without defects. Marked as box a 2 with a 3 The standard deviation of the pixels within the circular frame is the standard deviation of the pixel values ​​of the image without defects. The calculation formula is shown in formulas (2) to (4):

[0109]

[0110] in, Indicates a k The pixel value at x=i, y=j, The box a k The number of pixels in each row or column, k = 1, 2, 3; Indicates in a 3 -a 2 The standard deviation of the pixel values ​​in the region (the method of calculating the standard deviation is well known to those skilled in the art), i,j∈a 3 -a 2 Represents box a k The coordinate value of a 3 -a 2 Within the area, a 2 ∈a 3 Indicates area a 2 Included in area a 3 middle.

[0111] The trained neural network will simultaneously output the labeled defect thermal image and the signal-to-noise ratio of each defect.

[0112] Step B: Based on the calculated signal-to-noise ratio, the probability of defect detection is evaluated, that is, a POD curve is drawn, as follows.

[0113] Step B01, calculate the diameter-to-depth ratio of each defect, denoted as r; the calculation formula is as follows:

[0114]

[0115] Where D is the diameter of the defect and d is the depth of the defect;

[0116] Step B02, performing linear regression analysis on the signal-to-noise ratio SNR and the diameter-to-depth ratio r of each defect;

[0117] The general linear model is expressed in formula (6):

[0118] Y=kX+b+ε (6)

[0119] Among them, Y represents the y-axis variable, and X represents the x-axis variable, which is the Gauss-Krüger xoy plane rectangular coordinate system in mathematics. The x-axis represents the diameter-to-depth ratio r, and the y-axis represents the signal-to-noise ratio SNR value of each defect. k, b, and ε are all linear regression parameters, k represents the slope, b represents the intercept, and ε represents the error term, and the error term ε obeys a normal distribution with a mean of 0 and a standard deviation of τ;

[0120] The linear relationship between SNR and r is analyzed. The linear relationship between SNR and r is expressed as SNR vs. r. Four possible models are generally selected, namely: SNR vs. r, log(SNR) vs. r, SNR vs. log(r) and log(SNR) vs. log(r). The model showing the most linear correlation is selected and substituted into expression (6), which can be rewritten as:

[0121] SNR / log(SNR)=kr / klog(r)+b+ε (7)

[0122] In one embodiment of the present invention, the linear regression analysis results are as follows Figure 8 As shown. From the regression results, it can be seen that log(SNR) vs.log(r) has the most linear correlation, that is, in this example, expression (6) can be rewritten as the following specific formula:

[0123] log(SNR)=klog(r)+b+ε (8)

[0124] Step B03, establish a detection probability model to determine whether all information about the existence of the defect is contained in the signal-to-noise ratio SNR. The detection probability function is represented by POD(r), that is, the detection probability function can be obtained from the relationship between the signal-to-noise ratio SNR and the diameter-to-depth ratio r of the defect, and the expression is as follows:

[0125]

[0126] Among them, POD(r) represents the detection probability function, f r (SNR) is the probability density function of the signal-to-noise ratio SNR corresponding to a specific diameter-to-depth ratio r of the defect, S dec Indicates the threshold of the defect signal-to-noise ratio, P(SNR>S dec ) means SNR>S dec The probability of , d(SNR) represents the differentiation of the variable SNR.

[0127] Probability density function f r (SNR) depends on the statistical distribution characteristics of the error term ε, and the error term ε obeys a normal distribution with a mean of 0 and a standard deviation of τ. Therefore, formula (9) can be expressed as follows:

[0128]

[0129] Among them, φ(·) represents the cumulative distribution function of the standard normal distribution function / lognormal distribution, μ and σ represent the mean and variance (general variables) of the function respectively. According to the different linear relationships between SNR vs. r, the expression of the POD(r) function is different, and the expression of the mean μ of the function is also different, as shown in Table 1 below.

[0130] Table 1 Parameter expressions of different linear relationships

[0131]

[0132]

[0133] In the table, S dec Indicates the judgment threshold of the defect signal-to-noise ratio.

[0134] In one embodiment of the present invention, log(SNR) vs. log(r) has the most linear correlation, so the POD(r) function can be expressed as follows:

[0135]

[0136] The calculation formulas of μ and σ are as follows:

[0137]

[0138] Step B04, using the maximum likelihood estimation method to solve the unknown parameters k, b, τ in the detection probability function;

[0139] Setting random variables That is, z obeys the standard normal distribution, and its probability density function can be expressed as follows:

[0140]

[0141] Substituting z into equation (6), the expression of the random variable z will be different for different linear relationships, as shown in Table 2 below:

[0142] Table 2 Random variable z expressions for different linear relationships

[0143]

[0144] In a specific example of the present invention, z is substituted into formula (6), and the corresponding linear relationship is as shown in formula (8). The expression of the random variable z is as follows:

[0145]

[0146] From formulas (1)-(7), it can be deduced that the signal-to-noise ratio obtained by the i-th detection is SNRi The probability can be used Indicates that z i Represents the random variable corresponding to the i-th detection, and its corresponding probability density function The expression is shown in formula (14), so the likelihood function is constructed as follows:

[0147]

[0148] Wherein, N represents the number of defects detected.

[0149] Transform equation (16) logarithmically and Substitute z in formula (15), and the likelihood function L becomes a formula containing k, b, and τ, expressed as follows:

[0150]

[0151] Using the maximum likelihood estimation method, we calculate the partial derivatives of k, b, and τ respectively, set them equal to zero, solve the values ​​of the three unknown parameters k, b, and τ, and substitute them into Table 2 to calculate the mean μ and standard deviation σ of the detection probability function POD(r);

[0152] Step B05, plot the POD probability curve and the POD probability curve with a confidence level of 95%, and obtain r 90 With r 90 / 95 Two important indicators for evaluating the capability of nondestructive testing technology, these two indicators are used to evaluate the detection capability of the testing technology. 90 It represents the defect diameter-to-depth ratio when the detection probability is 90% (POD(r) = 0.9); 90 / 95 It represents the defect diameter-to-depth ratio when the detection probability is 90% and the confidence level is 95%.

[0153] In a specific embodiment of the present invention, the POD probability curve is as follows: Fig. 9 shown.

[0154] The obtained log(r 90 )=1.852,log(r 90 / 95 )=1.998,

[0155] That is r 90 =e 1.852 =6.373,r 90 / 95 =e 1.998 =7.374,

[0156] When the detection probability of this detection technology is 90% (POD(r)=0.9), the corresponding defect diameter-to-depth ratio is 6.373; when the detection probability is 90% and the confidence level is 95%, the corresponding defect diameter-to-depth ratio is 7.374.

[0157] In another specific embodiment of the present invention, when the training accuracy is low, more thermal images containing defects are obtained through detection experiments or by selecting other thermal imaging data processing algorithms to expand the data volume of the training data set and improve the accuracy of network training.

[0158] In a specific embodiment of the present invention, in step A06, the following steps are used to realize automatic counting of identified defects:

[0159] Step 1: Use the Roberts operator to perform edge detection on the obtained grayscale defect thermal image to generate a binary image. The purpose of edge detection is to identify the defect annotation box in the image and convert the grayscale image into a binary image. Since the image is also a two-dimensional matrix, the black area in the image represents that the element value at this position of the matrix is ​​0, and correspondingly, the white part represents that the element value is 1, thereby obtaining a black and white defect thermal image.

[0160] Step 2: Mark the connected components of the binary image obtained after edge detection. The principle of connected component marking is to assign different values ​​to each connected area in the binary image to achieve the positioning, counting and marking of the connected areas. That is, determine whether the element value is 1 in the order of rows and columns. For example, element a ij =1, judge a ij The surrounding 4 adjacent elements a i-1,j 、a i+1,j 、a i,j-1 、a i,j+1 Is the value of a also 1? If there is an element with a value of 1, then compare the element with a ij Mark them as the same type, and then determine whether the four adjacent elements around each other element of the same type are 1, until the same element as a is marked as ij When all elements of the same type are searched, that is, until there are no unconnected elements around the connected elements, the area formed by the connection of these elements is called a connected component;

[0161] Step 3: Find the next connected component until all connected components in the binary graph are marked by their ordinal numbers; mark each defect labeling box as a connected component, the value of all elements in the first connected component is 1, the value of all elements in the second connected component is 2, and so on, the value of the Nth labeling box is N;

[0162] In yet another embodiment of the present invention, in step A07, the specific steps of marking three boxes with lengths and widths of m, n, 2m, 2n, 3m, and 3n include:

[0163] Step 1: For the image matrix b after the connected components are marked, first find the maximum value C in b, where C is the connected components in the image, that is, the number of labeled boxes;

[0164] Step 2: Let i = 1, 2, ..., C, and find the row number set r of the element with value i in matrix b in turn i With column number set c i ;

[0165] Step 3: Find the row number set r respectively i With column number set c i The maximum value r imax and c imax With the minimum value r imin and c imin ; Therefore, the i-th annotation box a 1 The length m and width n are r imax -r imin With c imax -c imin ;

[0166] step4: box a 2 Line number set r i The maximum values ​​of and Round up, the minimum values ​​are and Round down; box a 3 Line number set r i The maximum values ​​of imax +m and c imax +n rounded up, the minimum value is r imin -m and c imin -n round down;

[0167] The three marked boxes a are obtained 1 、a 2 、a 3 .

[0168] It should be noted that: the type of structural defects of the carbon fiber composite material described in the present invention is mainly internal delamination defects; thermal images or thermal images are visual images that record the temperature value of the surface of the inspected structure at a certain moment;

[0169] The target detection neural network is a convolutional neural network that learns image features and then identifies and locates targets in the image.

Claims

1. An intelligent evaluation method for the detection capability of infrared thermal imaging delamination defects of carbon fiber composite materials, characterized in that: The method includes step A and step B, which are as follows: Step A: Intelligently identify and mark the delamination defects of carbon fiber composite materials by infrared thermal imaging, and automatically calculate the signal-to-noise ratio of each defect; Step A01, obtaining an original thermal image of a carbon fiber composite material structure with known internal defects used for experiment through an infrared thermal imaging detection system, hereinafter the "carbon fiber composite material structure" is referred to as the "material structure" for short; Use the laser infrared thermal imaging detection system to detect defects in material structures and obtain the original thermal image sequence data of the tested parts; Select the original thermal images where the defects are clearly visible as the source of part of the data set for the subsequent target detection neural network; Step A02, using a thermal image data processing algorithm to process the original thermal image data to obtain a processed thermal image, which more clearly visualizes the defect features to further improve the defect detection effect; the processed thermal image provides thermal image data for the subsequent training of the target detection network; Step 1: Use thermal imaging data processing algorithm to process the original thermal imaging data to further highlight the defect effect; Step 2: Convert the RGB image into a grayscale image; Step 3: Perform linear transformation on the converted grayscale image to achieve the effect of image enhancement and contrast control; Step A03, constructing a target detection neural network; Use YOLOv4 target detection network to build target detection neural network; Step A04, marking defect areas of B% of the thermal image processed in step A02 to construct a defect thermal image training data set; Image annotation software is used to mark the positions of defects that can be distinguished by the human eye in the thermal image with boxes as close to their edges as possible, and a label file containing defect annotation information is generated. The annotation information file and its corresponding annotated thermal image together constitute the training data set of the target detection network; Step A05, training the target detection neural network using the constructed defect thermal image training data set; The YOLOv4 network is trained with the constructed defect thermal image training data set. The trained YOLOv4 network is verified with thermal images without defects being labeled. When the training accuracy is not high, more thermal images containing defects are obtained through detection experiments or by selecting other thermal image data processing algorithms to expand the data volume of the training data set and improve the accuracy of network training; the target detection neural network is trained with the defect thermal image data set constructed in step A04; (100-B)% of the thermal images processed in step A02 and without defects being labeled are randomly selected to form a verification set, and the detection neural network with trained weights is used for training, and the recognition ability of the target detection neural network is verified with the labels formed by the above annotations; Step A06, using the trained target detection neural network to automatically identify defects in the thermal image, marking each defect with a box, and realizing automatic counting of identified defects; Defect recognition can be completed by inputting the original thermal image or the processed thermal image without defect marking into the target detection neural network. At this time, the length and width of the marked box are m and n respectively, which is recorded as box a1. After processing, the defects in the image are marked by boxes. Step A07, calculating the mean and variance of the pixels at the defective locations within each marked frame, and calculating the signal-to-noise ratio of each defect; Mark three boxes with lengths and widths of m, n, 2m, 2n, 3m, and 3n, respectively, as boxes a1, a2, and a3. Convert the marked defect thermal image into a pixel value matrix, and automatically calculate the signal-to-noise ratio of each defect according to the signal-to-noise ratio calculation method. The calculation formula of the signal-to-noise ratio is shown in formulas (1)-(4); in, is the mean value of pixels in the box a1, that is, the mean value of pixels in the image with defects; is the mean value of pixels in the circular frame enclosed by the frames a2 and a3, i.e., the mean value of pixels in the image without defects; is the standard deviation of the pixels in the circular frame marked by boxes a2 and a3, that is, the standard deviation of the pixel values ​​of the image without defects. The calculation formula is shown in formulas (2) to (4): in, Indicates a k The pixel value at x=i, y=j, The box a k The number of pixels in each row or column, k = 1, 2, 3; represents the standard deviation of the pixel values ​​in the a3-a2 area, i,j∈a3-a2 represents the box a k The coordinate value of is within the region a3-a2, a2∈a3 means that region a2 is contained in region a3; The trained neural network simultaneously outputs the labeled defect thermal image and the signal-to-noise ratio of each defect; Step B: Based on the calculated signal-to-noise ratio, evaluate the probability of defect detection, i.e. draw a POD curve; Step B01, calculate the diameter-to-depth ratio of each defect, denoted as r; the calculation formula is as follows: Where D is the diameter of the defect and d is the depth of the defect; Step B02, performing linear regression analysis on the signal-to-noise ratio SNR and the diameter-to-depth ratio r of each defect; The general linear model is expressed in formula (6): Y=kX+b+ε (6) Among them, Y represents the y-axis variable, X represents the x-axis variable, that is, the Gauss-Krüger xoy plane rectangular coordinate system in mathematics; the x-axis represents the diameter-to-depth ratio r, and the y-axis represents the signal-to-noise ratio SNR value of each defect; k, b, and ε are all linear regression parameters, k represents the slope, b represents the intercept, and ε represents the error term, and the error term ε obeys a normal distribution with a mean of 0 and a standard deviation of τ; The linear relationship between SNR and r is analyzed. The linear relationship between SNR and r is expressed as SNR vs. r below. Four possible models are selected, namely: SNR vs. r, log(SNR) vs. r, SNR vs. log(r) and log(SNR) vs. log(r). The model showing the most linear correlation is selected and substituted into expression (6), which is rewritten as: SNR / log(SNR)=kr / klog(r)+b+ε (7) Step B03, establish a detection probability model to determine whether all information about the existence of the defect is contained in the signal-to-noise ratio SNR; POD(r) is used to represent the detection probability function, that is, the detection probability function can be obtained from the relationship between the signal-to-noise ratio SNR and the diameter-to-depth ratio r of the defect, and the expression is as follows: Among them, POD(r) represents the detection probability function, f r (SNR) is the probability density function of the signal-to-noise ratio SNR corresponding to a specific diameter-to-depth ratio r of the defect, S dec Indicates the threshold of the defect signal-to-noise ratio, P(SNR>S dec ) means SNR>S dec The probability of , d(SNR) represents the differentiation of the variable SNR; Probability density function f r (SNR) depends on the statistical distribution characteristics of the error term ε, which follows a normal distribution with a mean of 0 and a standard deviation of τ. Therefore, equation (9) can be expressed as follows: Among them, φ(·) represents the standard normal distribution function / lognormal cumulative distribution function, μ and σ represent the mean and variance of the function (general variables), respectively. According to the different linear relationships between SNR vs. r, the expression of POD(r) function is different, and the expression of the mean μ of the function is also different, as shown in Table 1; Table 1 Parameter expressions of different linear relationships In the table, S dec Indicates the judgment threshold of defect signal-to-noise ratio; Step B04, using the maximum likelihood estimation method to solve the unknown parameters k, b, τ in the detection probability function; Setting random variables That is, z obeys the standard normal distribution, and its probability density function is expressed as follows: Substituting z into equation (6), the expression of the random variable z will be different for different linear relationships, as shown in Table 2 below: Table 2 Random variable z expressions for different linear relationships According to formulas (1)-(7), the signal-to-noise ratio obtained by the i-th detection is SNR i The probability of Indicates that z i Represents the random variable corresponding to the i-th detection, and its corresponding probability density function The expression is shown in formula (14), and the likelihood function is constructed as follows: Where N represents the number of defects detected; Transform equation (16) logarithmically and Substitute z in formula (15), and the likelihood function L is as follows: Using the maximum likelihood estimation method, we calculate the partial derivatives of k, b, and τ respectively, set them equal to zero, solve the values ​​of the three unknown parameters k, b, and τ, and substitute them into Table 2 to calculate the mean μ and standard deviation σ of the detection probability function POD(r); Step B05, plot the POD probability curve and the POD probability curve with a confidence level of 95%, and obtain r 90 With r 90 / 95 Two indicators for evaluating the capability of nondestructive testing technology are used to evaluate the detection capability of the testing technology; among them, r 90 It represents the defect diameter-to-depth ratio when the detection probability is 90%; r 90 / 95 It represents the defect diameter-to-depth ratio when the detection probability is 90% and the confidence level is 95%.

2. The intelligent evaluation method for infrared thermal imaging delamination defect detection capability of carbon fiber composite materials according to claim 1, characterized in that: When the training accuracy is low, more thermal images containing defects can be obtained through detection experiments or by selecting other thermal imaging data processing algorithms to expand the data volume of the training data set and improve the accuracy of network training.

3. The intelligent evaluation method for infrared thermal imaging delamination defect detection capability of carbon fiber composite materials according to claim 1, characterized in that: In step A01, the infrared thermal imaging system uses laser as a heat source to heat the inspected CFRP structure. The infrared thermal imager in the infrared thermal imaging system records the temperature changes of the surface of the material structure during 5s heating and 5s cooling at an imaging frame rate of 50Hz, and obtains a thermal imaging video containing 500 frames of original thermal images.

4. The intelligent evaluation method for infrared thermal imaging delamination defect detection capability of carbon fiber composite materials according to claim 3, characterized in that: In step A01, the last frame of thermal image in the heating stage is taken as a thermal image for defect calibration.

5. The intelligent evaluation method for infrared thermal imaging delamination defect detection capability of carbon fiber composite materials according to claim 1, characterized in that: In Step 1 of Step A02: the thermal imaging data processing algorithm includes a thermal imaging signal reconstruction method, a principal component analysis method, and a pulse phase method.

6. The intelligent evaluation method for infrared thermal imaging delamination defect detection capability of carbon fiber composite materials according to claim 5, characterized in that In Step 1 of Step A02, the thermal image data processing algorithm adopts the principal component analysis method.

7. The intelligent evaluation method for infrared thermal imaging delamination defect detection capability of carbon fiber composite materials according to claim 1, characterized in that: In Step 2 of Step A02, the RGB image is converted into a grayscale image by taking the weighted sum of the three-channel color components as the grayscale value, that is, Grayscale = 0.299 × R + 0.587 × G + 0.114 × B.

8. The intelligent evaluation method for infrared thermal imaging delamination defect detection capability of carbon fiber composite materials according to claim 1, characterized in that: Step A04, B=70.

9. The intelligent evaluation method for infrared thermal imaging delamination defect detection capability of carbon fiber composite materials according to claim 1, characterized in that: In step A06, the following steps are used to automatically count the identified defects: Step 1: Use the Roberts operator to perform edge detection on the obtained grayscale defect thermal image to generate a binary image. The purpose of edge detection is to identify the defect annotation box in the image and convert the grayscale image into a binary image. Since the image is also a two-dimensional matrix, the black area in the image represents that the element value at this position of the matrix is ​​0, and correspondingly, the white part represents that the element value is 1, thereby obtaining a black and white defect thermal image. Step 2: Mark the connected components of the binary image obtained after edge detection. The method of connecting component marking is to assign different values ​​to each connected region in the binary image to achieve the location, counting and marking of the connected regions. That is, determine whether the element value is 1 in the order of row and column. For example, element a ij =1, judge a ij The surrounding 4 adjacent elements a i-1,j 、a i+1,j 、a i,j-1 、a i,j+1 Is the value of a also 1? If there is an element with a value of 1, then compare the element with a ij Mark them as the same type, and then determine whether the four adjacent elements around each other element of the same type are 1, until the same element as a is marked as ij When all elements of the same type are searched, that is, until there are no unconnected elements around the connected elements, the area formed by the connection of these elements is called a connected component; Step 3: Find the next connected component until all connected components in the binary graph are marked with their ordinal numbers; mark each defect labeling box as a connected component, the value of all elements in the first connected component is 1, the value of all elements in the second connected component is 2, and so on. The value of the Nth labeling box is N.

10. The intelligent evaluation method for infrared thermal imaging delamination defect detection capability of carbon fiber composite materials according to claim 1, characterized in that: In step A07, the specific steps of marking three boxes with lengths and widths of m, n, 2m, 2n, 3m, and 3n include: Step 1: For the image matrix b after the connected components are marked, first find the maximum value C in b, where C is the connected components in the image, that is, the number of labeled boxes; Step 2: Let i = 1, 2, ..., C, and find the row number set r of the element with value i in matrix b in turn i With column number set c i ; Step 3: Find the row number set r respectively i With column number set c i The maximum value r imax and c imax With the minimum value r imin and c imin ; Therefore, the length m and width n of the i-th annotation box a1 are r imax -r imin With c imax -c imin ; step4: box a2 row number set r i The maximum values ​​of and Round up, the minimum values ​​are and Round down; box a3 row number set r i The maximum values ​​of imax +m and c imax +n rounded up, the minimum value is r imin -m and c imin -n round down; This results in three marked boxes a1, a2, and a3.

Images