Deep learning deformation monitoring method for X-ray digital imaging based on explicit correlation layer

Abstract

The X-ray digital imaging deep learning deformation monitoring method based on an explicit correlation layer disclosed by the present invention belongs to the field of image deformation measurement. The implementation method of the present invention is as follows: Based on X-ray digital imaging and the continuity deformation theory, a high-noise natural texture speckle feature image dataset is established; a correlation layer for calculating image correlation is introduced into the convolutional neural network, and the high-noise natural texture speckle feature image dataset is input into the improved neural network for training to obtain a trained detection model based on the display correlation layer; the high-noise natural texture speckle feature images before and after deformation to be detected are input into the trained improved neural network model to obtain a deformation field, thereby realizing X-ray digital imaging deformation monitoring. The present invention introduces explicit correlation calculation into the architecture of the convolutional neural network to obtain a deep learning network based on the display correlation layer, significantly improving the deformation monitoring accuracy of high-noise natural texture speckle feature images in X-ray digital imaging.

Description

Technical Field

[0001] The present invention belongs to the field of image deformation measurement, and relates to an X-ray digital imaging deep learning deformation monitoring method based on an explicit correlation layer. Background Technique

[0002] Woven carbon / silicon carbide composites have been applied to the high-temperature sealing structures on the surfaces of aircraft due to their good structural integrity and excellent comprehensive properties. The surfaces of aircraft are not fully enclosed and there are many gaps, such as the control and landing gear hatch doors of aircraft. Once high-temperature airflows enter the interior of the aircraft through the moving gaps and damage the temperature-intolerant components inside, catastrophic accidents will occur. Therefore, for hypersonic aircraft, monitoring the deformation state of the sealing structure and preventing hot airflows from entering the interior of the aircraft are of great significance for ensuring the safe service of the aircraft. To observe the deformation of ultra-high temperature materials such as woven carbon / silicon carbide composites at high temperatures in real time and monitor the change in their shapes, optical imaging and digital image correlation method (DIC) are often used. Traditional DIC is a technique that uses image correlation to calculate the deformation of an object. This method calculates the displacement and strain of the object by spraying a speckle pattern on the surface of the object and comparing the images taken before and after deformation. However, when measuring the deformation field at high temperatures, problems such as false deformation caused by thermal haze and image overexposure caused by strong thermal radiation are inevitable for optical DIC, and it also faces problems such as speckle detachment at high temperatures and difficulty in preparing random speckles for some specimens. X-ray imaging is not affected by these factors. For non-uniform materials, the natural texture inside the material can be obtained by X-ray digital imaging to create a speckle pattern for DIC. Therefore, the DIC method based on X-ray digital imaging has been developed.

[0003] Traditional DIC methods are restricted by low-order shape functions when dealing with images with high noise and low speckle quality, showing certain inherent limitations. The mismatch between high-order deformation and low-order shape functions leads to non-negligible errors in measurement, which become the main source of systematic errors, limiting the measurement accuracy. Moreover, the traditional DIC method that calculates the displacement field by dividing sub-regions and establishing correlation criteria is difficult to meet the requirement of real-time monitoring.

[0004] In recent years, with the rapid development of deep learning in the field of computer vision, the digital image correlation method based on convolutional neural network has also been widely used. At present, the computational accuracy of the completely data-driven deep learning algorithm is greatly affected by the training dataset and network structure. Existing research mainly focuses on improving the measurement accuracy by enriching the dataset, optimizing the network structure, or adopting a pyramid strategy to integrate multi-scale results. However, in practical applications, it is difficult to obtain a large number of real datasets. A complex network structure requires a long training time, and the convolutional neural network method based on data-driven feature extraction also has low computational accuracy for speckle feature images with low speckle quality and high noise, making it difficult to realize the online monitoring of the deformation process of composites such as woven carbon / silicon carbide. Summary of the Invention

[0005] The purpose of the present invention is to provide an X-ray digital imaging deep learning deformation monitoring method based on an explicit correlation layer, introducing explicit correlation calculation into the architecture of the convolutional neural network to obtain a deep learning network based on the display correlation layer for X-ray digital imaging deformation monitoring, significantly improving the deformation field prediction accuracy of high-noise natural texture speckle feature images in X-ray digital imaging.

[0006] The X-ray digital imaging deep learning deformation monitoring method based on an explicit correlation layer disclosed by the present invention includes the following steps:

[0007] Step 1: Establish a high-noise natural texture speckle feature image dataset based on X-ray digital imaging and the theory of continuous deformation;

[0008] Dataset establishment; based on X-ray digital imaging technology, X-rays pass through the sample and reach the detector, and the formed image is not interfered by high-temperature environmental factors. A series of speckle images are formed using the natural texture and structural features inside the object;

[0009] I = I0e μx (1)

[0010] I is the intensity of the attenuated X-ray; I0 is the intensity of the initial X-ray; μ is the attenuation coefficient of the material, mainly related to the density of the material; x is the thickness of the measured material;

[0011] Divide the initial speckle image into several random regions, control the density of the deformation field, and generate random deformation field data using random deformation and affine transformation; use the interpolation method to apply the random deformation field data to the initial speckle image to generate a deformed speckle image:

[0012] (x, y) = (x0 + u, y0 + v) (2)

[0013] (x, y) is the deformation field on the generated image; (x0, y0) is the deformation field on the initial image; u is the deformation field data in the x direction; v is the deformation field data in the y direction;

[0014] Establish a high-noise natural texture speckle feature image dataset using the initial speckle image, deformation field data, and the deformed speckle image.

[0015] Step 2: Input the high-noise natural texture speckle feature image dataset obtained in Step 1 into the improved neural network for training to obtain a trained detection model based on the display-related layer.

[0016] 1), The feature extraction layer obtains the dataset obtained from Step 1 and extracts features using a series of convolutional layers in the feature extraction layer:

[0017] F = Conv(I) (3)

[0018] F is the image feature after convolution; I is the input image or the image feature before convolution; Conv() is the convolution operation; the initial speckle image feature and the deformed speckle image feature are extracted respectively.

[0019] 2), The image correlation calculation layer obtains the features in 1), selects the sub-region size and stride according to the image speckle features in the initial speckle image, and sets a maximum search range at the same time; on the deformed speckle image, find the position of the corresponding sub-region, which is called the reference sub-region; with the center of the reference sub-region as the center, within the maximum search range, perform correlation calculation according to the sub-region size and the preset moving matching stride to obtain the correlation distribution of the initial speckle image sub-region within the maximum search range of the deformed speckle image; move the sub-region position of the initial speckle image according to the stride to obtain the correlation distribution of each initial speckle image sub-region within the maximum search range of the deformed speckle image, and arrange the correlation distributions at different initial speckle image sub-region positions in sequence to obtain the correlation matrix.

[0020] Combine the correlation matrix with the initial speckle image feature to form a new input tensor:

[0021] T input = Concate(Correlate(F1, F2), F1) (4)

[0022] T input is the input of the dynamic network structure layer; Concate() is the concatenation operation of the feature dimensions; Correlate() is to calculate the correlation coefficient; F1 is the initial speckle image feature; F2 is the initial speckle image feature.

[0023] 3) Input the correlation coefficient matrix obtained in 2) into the subsequent dynamic network layer, and improve the network structure based on the speckle features; use the improved neural network to train the dataset to obtain a trained detection model based on the display correlation layer;

[0024] Step 3: Input the high-noise natural texture speckle feature images before and after deformation to be detected into the improved neural network model trained in Step 2 to obtain the deformation field, that is, realize X-ray digital imaging deformation monitoring, and significantly improve the deformation field monitoring accuracy of the high-noise natural texture speckle feature images of X-ray digital imaging.

[0025] Beneficial effects:

[0026] 1. The X-ray digital imaging deep learning deformation monitoring method based on the explicit correlation layer disclosed in the present invention introduces explicit correlation calculation into the architecture of the convolutional neural network, obtains a deep learning network based on the display correlation layer for X-ray digital imaging deformation monitoring, and significantly improves the deformation field calculation accuracy of the high-noise natural texture speckle feature images of X-ray digital imaging.

[0027] 2. The X-ray digital imaging deep learning deformation monitoring method based on the explicit correlation layer disclosed in the present invention can capture more local features and extract more global information in one convolution step by adjusting the convolution kernel size in the traditional convolutional neural network; further increase the receptive field and improve the information extraction ability of the network for this natural texture speckle feature through large convolution kernels and increased downsampling; at the same time, introduce channel and spatial attention mechanisms to improve the perception of key features in the input image. Train the periodic natural texture speckle image dataset. Use the improved neural network to train the dataset to obtain a trained detection model based on the display correlation layer; based on the detection model, the deformation field monitoring accuracy of the high-noise natural texture speckle feature images of X-ray digital imaging can be significantly improved.

[0028] 3. The X-ray digital imaging deep learning deformation monitoring method based on the explicit correlation layer disclosed in the present invention introduces a correlation layer for image correlation calculation in the convolutional neural network, can select the correlation calculation sub-region and step size according to the speckle features and deformation size of the image, has better deformation field calculation accuracy, can effectively realize the real-time monitoring of the deformation of ultra-high temperature sealing structures such as woven carbon / carbon carbide composites at high temperatures, better analyze its deformation mechanism, and reduce the deformation failure phenomenon of such high-temperature sealing components in high-temperature environments. Description of the drawings

[0029] Figure 1 Collect natural texture speckle images based on X-ray digital imaging technology;

[0030] Figure 2 Convolutional neural network structure diagram;

[0031] Figure 3 Schematic diagram of related layer calculation

[0032] Figure 4 Calculation results of simulated high-noise natural texture speckle deformation field

[0033] Figure 5 Calculation results of high-noise natural texture speckle deformation field collected by X-ray digital imaging technology Specific implementation manner

[0034] In order to better illustrate the purpose and advantages of the present invention, the following further describes the content of the invention in conjunction with the drawings and examples

[0035] The X-ray digital imaging deep learning deformation monitoring method based on an explicit related layer disclosed in this embodiment specifically includes the following implementation steps

[0036] Step 1: Dataset establishment; collect natural texture speckle images of woven carbon / silicon carbide composites based on X-ray digital imaging technology. The X-ray passes through the carbon / silicon carbide composite material and reaches the detector. The formed image is not interfered by high-temperature environmental factors. A series of speckle images are formed using the natural texture and structural features inside the object. The obtained images are as Figure 1 shown

[0037] I = I0e μx (5)

[0038] I is the intensity of the attenuated X-ray; I0 is the intensity of the initial X-ray; μ is the attenuation coefficient of the material, which is mainly related to the density of the material; x is the thickness of the material to be measured

[0039] Divide the initial speckle image into several random regions, control the density of the deformation field, and generate random deformation field data using random deformation and affine transformation; use interpolation technology to apply the random deformation field data to the initial speckle image to generate a deformed speckle image

[0040] (x, y) = (x0 + u, y0 + v) (6)

[0041] (x, y) is the deformation field on the generated image; (x0, y0) is the deformation field on the initial image; u is the deformation field data in the x direction; v is the deformation field data in the y direction

[0042] Establish a high-noise natural texture speckle feature image dataset using the initial speckle image, deformation field data, and the deformed speckle image

[0043] Step 2: Input the dataset obtained in Step 1 into the improved neural network for training. The network structure is as Figure 2As shown, a trained detection model is obtained;

[0044] 1), the feature extraction layer, i.e., Figure 2 in ①, obtain the carbon / silicon carbide composite material dataset obtained from step one, and extract features using a series of convolutional layers in the feature extraction layer:

[0045] F = Conv(I) (7)

[0046] F is the image feature after convolution; I is the input image or the image feature before convolution; Conv() is the convolution operation; the initial speckle image feature and the deformed speckle image feature are extracted respectively.

[0047] 2), the image correlation calculation layer, i.e., Figure 2 in ②, obtain the features in 1), select the sub-region size as (3*3) and the step size as 1 in the initial speckle image according to the image speckle features, and at the same time set a maximum search range as (9*9); on the deformed speckle image, find the position of the corresponding sub-region, which is called the reference sub-region; with the center of the reference sub-region as the center, within the maximum search range, perform correlation calculation according to the sub-region size (3*3) and the preset moving matching step size of 1, and obtain the correlation distribution of the initial speckle image sub-region within the maximum search range of the deformed speckle image; move the sub-region position of the initial speckle image according to the step size of 1, obtain the correlation distribution of each initial speckle image sub-region within the maximum search range of the deformed speckle image, and arrange the correlation distributions at different initial speckle image sub-region positions in sequence to obtain a correlation matrix with the shape of [C, 3, 3, H-1, W-1], where C is the number of channels of the initial speckle image feature, H is the height of the initial speckle image feature, and W is the width of the initial speckle image feature; the calculation principle is as Figure 3 shown.

[0048] Combine the correlation matrix with the initial speckle image feature to form a new input tensor:

[0049] T input = Concate(Correlate(F1, F2), F1) (8)

[0050] T input is the input of the dynamic network structure layer; Concate() is the splicing operation of feature dimensions; Correlate() is to calculate the correlation coefficient; F1 is the initial speckle image feature; F2 is the deformed speckle image feature.

[0051] 3), input the correlation coefficient matrix obtained in 2) into the subsequent dynamic network layer, i.e., Figure 2③, and improve the network structure based on speckle features; adjust the convolution kernel size in the traditional convolutional neural network, modify some 3*3 convolution kernels to 5*5 and 7*7, which can capture more local features and extract more global information in one convolution step. By using large convolution kernels and increasing downsampling, the receptive field is further increased, and the information extraction ability of the network for the natural texture speckle features is improved. At the same time, channel and spatial attention mechanisms are introduced to improve the perception of key features in the input image. Train on the periodic natural texture speckle image dataset. Use the improved neural network to train the dataset to obtain a trained detection model;

[0052] 4), Adopt an end-to-end training method to directly learn the final output from the original input without any intermediate steps. The design of the loss function considers the predictions of multiple output layers (the distributions are prediction 1, prediction 2, prediction 3), and each prediction output corresponds to a weight, which allows the model to give different emphases to different levels of output during training. Calculate the Euclidean distance (L2 norm) between the output of the upsampling and the ground truth label, and use the average value of the L2 norm as the loss of a single output to measure the average of the pointwise distances between the network output and the label, evaluate the model performance and optimize it. The specific calculation method is

[0053]

[0054] L i is the loss value of a single output; w1, w2, w3 are the weights of the three predictions respectively, M is the dimension of the loss calculation; Y′ ij1 is the deformation field predicted by prediction 1 at the position (i, j); Y′ ij2 is the deformation field predicted by prediction 2 at the position (i, j); Y′ ij3 is the deformation field predicted by prediction 3 at the position (i, j); Y ij is the true deformation field of the label at the position (i, j).

[0055] Step 3. To verify the effectiveness and robustness of the proposed method, this method is verified in the simulated dataset and the dataset collected by the actual X-ray digital imaging technology compared with the traditional DIC algorithm and deep learning models; According to Figure 4 the calculation results of the simulated high-noise natural texture speckle deformation field, it can be seen from the pixel displacement profile and the average relative error of the displacement field that the average absolute error calculated by the improved deep learning network is lower. Taking the pixel displacement profile of the deformation field calculated by the traditional DIC algorithm as a reference, from Figure 5In the displacement pixel contour of the natural texture speckle deformation field collected by X-ray digital imaging technology, it is further proved that the improved network has higher calculation accuracy for natural texture speckle images with high noise, and can better monitor the deformation of ultra-high temperature materials such as woven carbon / silicon carbide composites under high temperature conditions in real time.

[0056] The present invention introduces a correlation layer for calculating image correlation in a convolutional neural network, which can select a correlation calculation sub-region and step size according to the speckle characteristics and deformation size of the image, has better calculation accuracy of the deformation field, and realizes real-time and accurate monitoring of the deformation of woven carbon / silicon carbide composites under high temperature conditions.

[0057] The above specific description further details the purpose, technical solution and beneficial effects of the invention. It should be understood that the above is only a specific embodiment of the present invention and is not used to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An X-ray digital imaging deep learning deformation monitoring method based on an explicit correlation layer, characterized in that: It includes the following steps: Step 1: Establish a high-noise natural texture speckle feature image dataset based on X-ray digital imaging and continuous deformation theory. Step 2: Input the high-noise natural texture speckle feature image dataset obtained in Step 1 into an improved neural network for training to obtain a trained detection model based on the display correlation layer. 1). The feature extraction layer extracts image features from the high-noise natural texture speckle feature image dataset obtained in Step 1 to obtain image features. 2). The image correlation calculation layer obtains the image features in 1) and performs correlation calculation to obtain the correlation coefficient matrix of the feature maps before and after deformation. 3) Input the correlation coefficient matrix obtained in 2) into the subsequent dynamic network layer and improve the network structure based on the speckle features; use the improved neural network to train the dataset to obtain a trained detection model based on the display correlation layer. Step 3: Input the high-noise natural texture speckle feature images before and after deformation to be detected into the improved neural network model trained in Step 2 to obtain a deformation field, that is, to realize X-ray digital imaging deformation monitoring, and significantly improve the deformation field monitoring accuracy of the high-noise natural texture speckle feature images of X-ray digital imaging.

2. The method according to claim 1, characterized in that: The specific implementation method of Step 1 is as follows: Based on X-ray digital imaging technology, X-rays pass through the sample and reach the detector, and the formed image is not interfered by high-temperature environmental factors. A series of speckle images are formed by using the natural texture and structural features inside the object. I = I0e μx (1) I is the intensity of the attenuated X-ray; I0 is the intensity of the initial X-ray; μ is the attenuation coefficient of the material, which is mainly related to the density of the material; x is the thickness of the measured material. Divide the initial speckle image into several random regions, control the density of the deformation field, and generate random deformation field data by using random deformation and affine transformation; use the interpolation method to apply the random deformation field data to the initial speckle image to generate a deformed speckle image: (x, y) = (x0 + u, y0 + v) (2) (x, y) is the deformation field on the deformed speckle image; (x0, y0) is the deformation field on the initial speckle image; u is the deformation field data in the x direction; v is the deformation field data in the y direction. Establish a high-noise natural texture speckle feature image dataset by using the initial speckle image, deformation field data, and the deformed speckle image.

3. The method according to claim 1, characterized in that: The method for image feature extraction described in Step 2 is as follows: Use a series of convolutional layers of the feature extraction layer to extract features: F = Conv(I) (3) F is the image feature after convolution; I is the input image or the image feature before convolution; Conv() is the convolution operation; extract the initial speckle image feature and the deformed speckle image feature respectively.

4. The method according to claim 1, characterized in that: The image correlation calculation layer described in Step 2 introduces a correlation criterion into the convolutional neural network and is located between the feature extraction layer and the dynamic network structure layer for calculating the correlation of the image features before and after deformation. The features obtained in Step 1 are input into the image correlation calculation layer. In the initial speckle image, select the sub-region size and step size according to the image speckle features, and at the same time set a maximum search range; on the deformed speckle image, find the position of the corresponding sub-region, which is called the reference sub-region. Centered at the center of the reference sub-region, within the maximum search range, perform correlation calculation according to the sub-region size and the preset moving matching step size to obtain the correlation distribution of the initial speckle image sub-region within the maximum search range of the deformed speckle image; Move the position of the sub-region of the initial speckle image according to the step size to obtain the correlation distribution of each initial speckle image sub-region within the maximum search range of the deformed speckle image, and arrange the correlation distributions at different positions of the initial speckle image sub-region in sequence to obtain a correlation matrix; Combine the correlation matrix with the initial speckle image features to form a new input tensor T input : T input = Concate(Correlate(F1, F2), F1) (5) T input is the input of the dynamic network structure layer; Concate( ) is the splicing operation of feature dimensions; Correlate( ) is to calculate the correlation coefficient; F1 is the initial speckle image feature; F2 is the deformed speckle image feature.

5. The method according to claim 1, wherein: The dynamic network structure layer described in step two automatically adjusts the network structure according to the correlation matrix; Improve the network structure based on the speckle features in the dynamic network structure layer; use the improved neural network to train the data set obtained in step one to obtain a trained detection model based on the display correlation layer.

Images