X-ray digital imaging deep learning deformation monitoring method based on explicit correlation layer

Abstract

The invention discloses an X-ray digital imaging deep learning deformation monitoring method based on an explicit correlation layer, and belongs to the field of image deformation measurement. The implementation method comprises the following steps: establishing a high-noise natural texture speckle feature image data set based on X-ray digital imaging and a continuous deformation theory; introducing a correlation layer of image correlation calculation into the convolutional neural network, and inputting the high-noise natural texture speckle feature image data set into the improved neural network for training to obtain a trained detection model based on a display correlation layer; and inputting to-be-detected high-noise natural texture speckle feature images before and after deformation into the trained improved neural network model to obtain a deformation field, thereby realizing X-ray digital imaging deformation monitoring. According to the method, explicit correlation calculation is introduced into the architecture of the convolutional neural network, a deep learning network based on a display correlation layer is obtained, and the deformation monitoring precision of a high-noise natural texture speckle feature image of X-ray digital imaging is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of image deformation measurement and relates to an X-ray digital imaging deep learning deformation monitoring method based on explicit correlation layers. Background Art

[0002] Braided carbon / silicon carbide composites have been used in high-temperature sealing structures on the surface of aircraft due to their good structural integrity and excellent comprehensive performance. The surface of an aircraft is not fully enclosed, and there are many gaps, such as the control and landing gear doors of the aircraft. Once the high-temperature airflow enters the interior of the aircraft from the active gaps, it will damage the internal heat-sensitive components and cause catastrophic accidents. Therefore. For hypersonic aircraft, monitoring the deformation state of the sealing structure and preventing the hot airflow from entering the interior of the aircraft are of great significance to ensure the safe service of the aircraft. In order to observe the deformation of ultra-high temperature materials such as braided carbon / silicon carbide composites at high temperatures in real time and monitor their shape changes, optical imaging and digital image correlation (DIC) are often used. Traditional DIC is a technology that uses image correlation to calculate the deformation of an object. This method calculates the displacement and strain of the object by spraying a speckle pattern on the surface of the object and comparing the images taken before and after the deformation. However, when measuring the deformation field at high temperature, optical DIC is difficult to avoid false deformation caused by thermal haze and image overexposure caused by strong thermal radiation, and faces problems such as speckle detachment and difficulty in preparing random speckle in a high temperature environment. X-ray imaging is not affected by these factors. For non-uniform materials, the natural texture inside the material can be acquired by X-ray digital imaging to create a speckle pattern for DIC. Therefore, the DIC method based on X-ray digital imaging has been developed.

[0003] The traditional DIC method is constrained by low-order shape functions when processing high-noise, low-speckle quality images, and exhibits certain inherent limitations. The undermatching of high-order deformation and low-order shape functions leads to non-negligible errors in the measurement, which becomes the main source of system error and limits the measurement accuracy. In addition, the traditional DIC method that calculates the displacement field by dividing sub-areas and establishing correlation criteria is difficult to achieve the needs of real-time monitoring.

[0004] In recent years, with the rapid development of deep learning in the field of computer vision, digital image correlation methods based on convolutional neural networks have also been widely used. At present, the computational accuracy of deep learning algorithms that are completely data-driven is greatly affected by the training data set and network structure. Existing research mainly focuses on improving measurement accuracy by enriching the data set, optimizing the network structure, or integrating multi-scale results using a pyramid strategy. However, in practical applications, it is difficult to obtain a large number of real data sets, and complex network structures require a long training time. In addition, the convolutional neural network method based on data-driven feature extraction has low computational accuracy for speckle feature images with low speckle quality and high noise, making it difficult to achieve online monitoring of the deformation process of composite materials such as woven carbon / silicon carbide. Summary of the invention

[0005] The purpose of the present invention is to provide a deep learning deformation monitoring method for X-ray digital imaging based on explicit correlation layers, introduce explicit correlation calculation into the architecture of convolutional neural networks, obtain a deep learning network based on explicit correlation layers, which is used for X-ray digital imaging deformation monitoring, and significantly improve the deformation field prediction accuracy of high-noise natural texture speckle feature images of X-ray digital imaging.

[0006] The X-ray digital imaging deep learning deformation monitoring method based on explicit correlation layer disclosed in the present invention comprises the following steps:

[0007] Step 1: Establish a high-noise natural texture speckle feature image dataset based on X-ray digital imaging and continuous deformation theory;

[0008] Data set establishment: Based on X-ray digital imaging technology, X-rays pass through the sample to reach the detector. The image formed is not affected by high temperature environmental factors. A series of speckle images are formed by using the natural texture and structural characteristics inside the object.

[0009] I=I0e μx (1)

[0010] I is the intensity of the X-ray after attenuation; I0 is the intensity of the initial X-ray; μ is the attenuation coefficient of the material, which is mainly related to the density of the material; x is the thickness of the material being measured;

[0011] The initial speckle image is divided into several random areas, the density of the deformation field is controlled, and random deformation and affine change are used to generate random deformation field data; the random deformation field data is applied to the initial speckle image using the interpolation method to generate the deformed speckle image:

[0012] (x, y) = (x0 + u, y0 + v) (2)

[0013] (x, y) is the deformation field on the generated image; (x0, y0) is the deformation field on the initial image; u is the deformation field data in the x direction; v is the deformation field data in the y direction;

[0014] The initial speckle image, deformation field data and deformed speckle image are used to establish a high-noise natural texture speckle feature image dataset.

[0015] Step 2: Input the high-noise natural texture speckle feature image dataset obtained in step 1 into the improved neural network for training, and obtain a trained detection model based on the display correlation layer.

[0016] 1) The feature extraction layer obtains the data set obtained in step 1 and extracts features using a series of convolutional layers in the feature extraction layer:

[0017] F=Conv(I) (3)

[0018] F is the image feature after convolution; I is the input image or the image feature before convolution; Conv() is the convolution operation; the initial speckle image features and the deformed speckle image features are extracted respectively.

[0019] 2) The image correlation calculation layer obtains the features in 1), selects the sub-area size and step size in the initial speckle image according to the image speckle features, and sets a maximum search range; finds the position of the corresponding sub-area on the deformed speckle image, which is called the reference sub-area; takes the center of the reference sub-area as the center, and performs correlation calculation within the maximum search range according to the sub-area size and the preset moving matching step size to obtain the correlation distribution of the initial speckle image sub-area within the maximum search range of the deformed speckle image; moves the sub-area position of the initial speckle image according to the step size to obtain the correlation distribution of each initial speckle image sub-area within the maximum search range of the deformed speckle image, and arranges the correlation distributions at different initial speckle image sub-area positions in sequence to obtain a correlation matrix.

[0020] Combine the correlation matrix with the initial speckle image features to form a new input tensor:

[0021] T input =Concate(Correlate(F1,F2),F1) (4)

[0022] T input is the input of the dynamic network structure layer; Concate() is the concatenation operation of the feature dimension; Correlate() is the calculation of the correlation coefficient; F1 is the initial speckle image feature; F2 is the initial speckle image feature.

[0023] 3) inputting the correlation coefficient matrix obtained in 2) into the subsequent dynamic network layer, and improving the network structure based on the speckle characteristics; using the improved neural network to train the data set to obtain a trained detection model based on the display correlation layer;

[0024] Step 3: Input the high-noise natural texture speckle feature image before and after the deformation to be detected into the improved neural network model trained in step 2 to obtain the deformation field, that is, to realize X-ray digital imaging deformation monitoring, and significantly improve the deformation field monitoring accuracy of the high-noise natural texture speckle feature image of X-ray digital imaging.

[0025] Beneficial effects:

[0026] 1. The X-ray digital imaging deep learning deformation monitoring method based on explicit correlation layer disclosed in the present invention introduces explicit correlation calculation into the architecture of convolutional neural network to obtain a deep learning network based on explicit correlation layer, which is used for X-ray digital imaging deformation monitoring and significantly improves the deformation field calculation accuracy of high-noise natural texture speckle feature images of X-ray digital imaging.

[0027] 2. The X-ray digital imaging deep learning deformation monitoring method based on explicit correlation layer disclosed in the present invention can capture more local features and extract more global information in one convolution step by adjusting the convolution kernel size in the traditional convolutional neural network; by using large convolution kernels and increasing downsampling, the receptive field is further increased, and the network's ability to extract information from the natural texture speckle feature is improved; at the same time, channel and spatial attention mechanisms are introduced to improve the perception of key features in the input image. The periodic natural texture speckle image dataset is trained. The improved neural network is used to train the dataset to obtain a trained detection model based on the display correlation layer; based on the detection model, the deformation field monitoring accuracy of the high-noise natural texture speckle feature image of X-ray digital imaging can be significantly improved.

[0028] 3. The X-ray digital imaging deep learning deformation monitoring method based on explicit correlation layer disclosed in the present invention introduces the correlation layer of image correlation calculation in the convolutional neural network, and can select the correlation calculation sub-area and step size according to the speckle characteristics and deformation size of the image, so as to have better deformation field calculation accuracy, effectively realize the real-time monitoring of the deformation of ultra-high temperature sealing structures such as woven carbon / silicon carbide composite materials under high temperature, better analyze their deformation mechanism, and reduce the deformation failure phenomenon of such high temperature sealing components under high temperature environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Collect natural texture speckle images based on X-ray digital imaging technology;

[0030] Figure 2 Convolutional neural network structure diagram;

[0031] Figure 3 Schematic diagram of calculation of relevant layers;

[0032] Figure 4 Calculation results of simulated high-noise natural texture speckle deformation field;

[0033] Figure 5 X-ray digital imaging technology collects the calculation results of high-noise natural texture speckle deformation field. DETAILED DESCRIPTION

[0034] In order to better illustrate the purpose and advantages of the present invention, the invention is further described below with reference to the accompanying drawings and examples.

[0035] The X-ray digital imaging deep learning deformation monitoring method based on explicit correlation layer disclosed in this embodiment is specifically implemented in the following steps:

[0036] Step 1: Dataset establishment: Based on the X-ray digital imaging technology, the natural texture speckle image of the woven carbon / silicon carbide composite material is collected. The X-ray passes through the carbon / silicon carbide composite material and reaches the detector. The image formed is not affected by the high temperature environment factors. The natural texture and structural characteristics inside the object are used to form a series of speckle images. The acquired images are as follows: Figure 1 As shown;

[0037] I=I0e μx (5)

[0038] I is the intensity of the X-ray after attenuation; I0 is the intensity of the initial X-ray; μ is the attenuation coefficient of the material, which is mainly related to the density of the material; x is the thickness of the material being measured;

[0039] The initial speckle image is divided into several random areas, the density of the deformation field is controlled, and random deformation and affine change are used to generate random deformation field data; the random deformation field data is applied to the initial speckle image using interpolation technology to generate the deformed speckle image:

[0040] (x, y) = (x0 + u, y0 + v) (6)

[0041] (x, y) is the deformation field on the generated image; (x0, y0) is the deformation field on the initial image; u is the deformation field data in the x direction; v is the deformation field data in the y direction;

[0042] The initial speckle image, deformation field data and deformed speckle image are used to establish a high-noise natural texture speckle feature image dataset.

[0043] Step 2: Input the data set obtained in step 1 into the improved neural network for training. The network structure is as follows: Figure 2As shown, a trained detection model is obtained;

[0044] 1) Feature extraction layer Figure 2 In step ①, the carbon / silicon carbide composite material dataset obtained in step 1 is obtained, and features are extracted using a series of convolutional layers of the feature extraction layer:

[0045] F=Conv(I) (7)

[0046] F is the image feature after convolution; I is the input image or the image feature before convolution; Conv() is the convolution operation; the initial speckle image features and the deformed speckle image features are extracted respectively.

[0047] 2) Image correlation calculation layer Figure 2 In ②, the features in 1) are obtained. In the initial speckle image, a sub-region size of (3*3) and a step size of 1 are selected according to the image speckle features, and a maximum search range of (9*9) is set. On the deformed speckle image, the position of the corresponding sub-region is found, which is called the reference sub-region. With the center of the reference sub-region as the center, within the maximum search range, correlation calculation is performed according to the sub-region size (3*3) and the preset moving matching step size of 1 to obtain the correlation distribution of the initial speckle image sub-region within the maximum search range of the deformed speckle image. The sub-region position of the initial speckle image is moved according to the step size of 1 to obtain the correlation distribution of each initial speckle image sub-region within the maximum search range of the deformed speckle image. The correlation distributions at different initial speckle image sub-region positions are arranged in sequence to obtain a correlation matrix of the shape [C, 3, 3, H-1, W-1], where C is the number of channels of the initial speckle image feature, H is the height of the initial speckle image feature, and W is the width of the initial speckle image feature. The calculation principle is as follows: Figure 3 shown.

[0048] Combine the correlation matrix with the initial speckle image features to form a new input tensor:

[0049] T input =Concate(Correlate(F1,F2),F1) (8)

[0050] T input is the input of the dynamic network structure layer; Concate() is the concatenation operation of the feature dimension; Correlate() is the calculation of the correlation coefficient; F1 is the initial speckle image feature; F2 is the deformed speckle image feature.

[0051] 3) Input the correlation coefficient matrix obtained in 2) into the subsequent dynamic network layer, that is, Figure 2③, and improve the network structure based on the speckle feature; adjust the size of the convolution kernel in the traditional convolutional neural network, and modify some 3*3 convolution kernels to 5*5 and 7*7, so that more local features can be captured in one convolution step, and more global information can be extracted. By using large convolution kernels and increasing downsampling, the receptive field is further increased, and the network's ability to extract information from this natural texture speckle feature is improved. At the same time, the channel and spatial attention mechanisms are introduced to improve the perception of key features in the input image. Train on a periodic natural texture speckle image dataset. Use the improved neural network to train the dataset to obtain a trained detection model;

[0052] 4) Adopt an end-to-end training method, learn directly from the original input to the final output without any intermediate steps. The design of the loss function takes into account the predictions of multiple output layers (distributed as prediction 1, prediction 2, prediction 3), where each predicted output corresponds to a weight, which allows the model to give different weights to outputs at different levels during training. Calculate the Euclidean distance (L2 norm) between the upsampled output and the true label, take the average value of the L2 norm as the loss of a single output, measure the average point-by-point distance between the network output and the label, evaluate the model performance and optimize it. The specific calculation method is

[0053]

[0054] L i is the loss value of a single output; w1, w2, w3 are the weights of the three predictions respectively, M is the dimension of loss calculation; Y′ ij1 is the deformation field predicted by prediction 1 at position (i, j); Y′ ij2 is the deformation field predicted by prediction 2 at position (i, j); Y′ ij3 is the deformation field predicted by prediction 3 at position (i, j); Y ij is the true deformation field of the label at position (i, j).

[0055] Step 3: In order to verify the effectiveness and robustness of the proposed method, the method was verified with the traditional DIC algorithm and the deep learning model in the simulated data set and the data set collected by the actual X-ray digital imaging technology; Figure 4 The calculation results of the simulated high-noise natural texture speckle deformation field show that the improved deep learning network has a lower average absolute error than the pixel displacement profile and average relative error of the displacement field. Figure 5The displacement pixel profile of the natural texture speckle deformation field collected by X-ray digital imaging technology further proves that the improved network has higher calculation accuracy for high-noise natural texture speckle images, and can better monitor the deformation of ultra-high temperature materials such as woven carbon / silicon carbide composites under high temperature conditions in real time.

[0056] The present invention introduces a correlation layer of image correlation calculation in the convolutional neural network, and can select the correlation calculation sub-area and step size according to the speckle characteristics and deformation size of the image, which has better deformation field calculation accuracy and realizes real-time and accurate monitoring of the deformation of braided carbon / silicon carbide composite materials under high temperature conditions.

[0057] The specific description above further illustrates the purpose, technical solutions and beneficial effects of the invention in detail. It should be understood that the above is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A deformation monitoring method based on X-ray digital imaging deep learning with explicit correlation layer, characterized by: The following steps are included: Step 1: Establish a high-noise natural texture speckle feature image dataset based on X-ray digital imaging and continuous deformation theory; Step 2: input the high-noise natural texture speckle feature image dataset obtained in step 1 into the improved neural network for training, and obtain a trained detection model based on the display correlation layer; 1) The feature extraction layer extracts image features from the high-noise natural texture speckle feature image data set obtained in step 1 to obtain image features; 2) The image correlation calculation layer obtains the image features in 1) and performs correlation calculation to obtain the correlation coefficient matrix of the feature map before and after deformation; 3) inputting the correlation coefficient matrix obtained in 2) into the subsequent dynamic network layer, and improving the network structure based on the speckle characteristics; using the improved neural network to train the data set to obtain a trained detection model based on the display correlation layer; Step 3: Input the high-noise natural texture speckle feature image before and after the deformation to be detected into the improved neural network model trained in step 2 to obtain the deformation field, that is, to realize X-ray digital imaging deformation monitoring, and significantly improve the deformation field monitoring accuracy of the high-noise natural texture speckle feature image of X-ray digital imaging.

2. The method according to claim 1, characterized in that: The specific implementation method of step one is: Based on X-ray digital imaging technology, X-rays pass through the sample and reach the detector. The image formed is not disturbed by high temperature environmental factors, and a series of speckle images are formed by using the natural texture and structural characteristics inside the object; I=I0e μx (1) I is the intensity of the X-ray after attenuation; I0 is the intensity of the initial X-ray; μ is the attenuation coefficient of the material, which is mainly related to the density of the material; x is the thickness of the material being measured; The initial speckle image is divided into several random areas, the density of the deformation field is controlled, and random deformation and affine change are used to generate random deformation field data; the random deformation field data is applied to the initial speckle image using the interpolation method to generate the deformed speckle image: (x, y) = (x0 + u, y0 + v) (2) (x, y) is the deformation field on the deformed speckle image; (x0, y0) is the deformation field on the initial speckle image; u is the deformation field data in the x direction; v is the deformation field data in the y direction; The initial speckle image, deformation field data and deformed speckle image are used to establish a high-noise natural texture speckle feature image dataset.

3. The method according to claim 1, characterized in that: The method for extracting image features in step 2 is: Features are extracted using a series of convolutional layers of the feature extraction layer: F=Conv(I) (3) F is the image feature after convolution; I is the input image or the image feature before convolution; Conv() is the convolution operation; the initial speckle image features and the deformed speckle image features are extracted respectively.

4. The method according to claim 1, characterized in that: The image correlation calculation layer in step 2 introduces the correlation criterion into the convolutional neural network, is located between the feature extraction layer and the dynamic network structure layer, and is used to calculate the correlation of image features before and after deformation; The features obtained in step 1 are input into the image correlation calculation layer. In the initial speckle image, the sub-area size and step size are selected according to the image speckle features, and a maximum search range is set. On the deformed speckle image, the position of the corresponding sub-area is found, which is called the reference sub-area. Taking the center of the reference sub-area as the center, within the maximum search range, correlation calculation is performed according to the sub-area size and the preset moving matching step length to obtain the correlation distribution of the initial speckle image sub-area within the maximum search range of the deformed speckle image; The sub-area position of the initial speckle image is moved according to the step length to obtain the correlation distribution of each sub-area of ​​the initial speckle image within the maximum search range of the deformed speckle image, and the correlation distributions at different sub-area positions of the initial speckle image are arranged in sequence to obtain a correlation matrix; The correlation matrix is ​​combined with the initial speckle image features to form a new input tensor T input : T input =Concate(Correlate(F1,F2),F1) (5) T input is the input of the dynamic network structure layer; Concate( ) is the concatenation operation of the feature dimension; Correlate( ) is the calculation of the correlation coefficient; F1 is the initial speckle image feature; F2 is the deformed speckle image feature.

5. The method according to claim 1, characterized in that: The dynamic network structure layer described in step 2 automatically adjusts the network structure according to the correlation matrix; In the dynamic network structure layer, the network structure is improved based on the speckle feature; the improved neural network is used to train the data set obtained in step one to obtain a trained detection model based on the display related layer.

Images