Dic displacement field dynamic correction method and system based on lightweight convolutional network

Abstract

The present application relates to the technical field of digital image correlation (DIC) measurement, and particularly relates to a DIC displacement field dynamic correction method and system based on a lightweight convolutional network, which comprises the following steps: step 1, collecting a sequence of structural surface images and performing preprocessing; step 2, calculating an initial displacement field for the preprocessed image sequence by using a traditional DIC algorithm, and extracting key feature information as input data for dynamic correction; step 3, constructing a lightweight convolutional neural network model, transmitting the extracted input data to the lightweight convolutional neural network model for identification, designing a hybrid loss function for training, and learning the mapping relationship between the initial displacement field and the real displacement field; and step 4, correcting the initial displacement field by using the trained lightweight convolutional neural network model, and directly outputting high-precision displacement field data after correction. By using the function fitting capability of the neural network, high-precision displacement field correction is realized under limited computing resources, and the measurement accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital image correlation (DIC) measurement, and particularly relates to a DIC displacement field dynamic correction method and system based on a lightweight convolutional neural network. BACKGROUND

[0002] Digital image correlation (DIC) technology, as a non-contact and full-field measurement technology, has been widely applied in structural health monitoring, material mechanical property testing and other fields. However, in practical applications, due to the influence of noise interference, light change, complex structural surface deformation and other factors, the displacement field data obtained by DIC measurement often has certain errors and insufficient precision, especially in complex environments and monitoring scenes with high real-time requirements, how to improve the precision and reliability of displacement field measurement is a problem to be solved.

[0003] At present, the traditional DIC algorithm has low calculation efficiency and limited precision when dealing with complex deformation and noise interference. Although some deep learning-based methods improve the measurement precision to some extent, they usually require a large amount of calculation resources and time for model training and reasoning, which is difficult to meet the real-time requirements.

[0004] Lightweight convolutional neural networks (CNN) optimize the network structure and calculation process, and can greatly reduce the number of network parameters and computational complexity, and improve data processing efficiency through techniques such as depth separable convolution and network pruning, while maintaining model performance. Unlike traditional correction methods based on physical models, lightweight convolutional neural networks do not require manual design of complex feature extraction and correction algorithms, but automatically learn the optimal feature representation and correction strategy through a large amount of training data. Through end-to-end training and prediction, from input image to output displacement field, the whole process does not require human intervention, avoiding error accumulation caused by improper parameter setting or unreasonable model assumptions in traditional methods.

[0005] In the field of structural health monitoring, DIC measurement technology is increasingly widely used. In order to improve the precision and efficiency of measurement, various optimizations and improvements have been made to traditional DIC methods.

[0006] Patent CN201510100472.9 discloses a displacement field adaptive smoothing method suitable for digital image correlation, which is based on penalized least squares regression and generalized cross-validation (GCV) to automatically optimize displacement field smoothing parameters, and combines discrete cosine transform (DCT) to suppress noise and improve strain field calculation precision. However, this method involves multiple iterations and smoothing processing, has high computational complexity and poor real-time performance.

[0007] Patent CN202111160148.8 discloses a DIC-based steel structure fatigue crack propagation morphology measurement method, which distinguishes coarse / fine crack morphology through zero-mean normalized cross-correlation algorithm (ZNCC), and realizes dynamic monitoring by combining topological structure displacement field analysis, but relies on traditional DIC algorithm (such as sub-pixel interpolation) and does not consider the influence of noise, illumination change and other interference on displacement field accuracy.

[0008] Patent CN202410077820.4 proposes a non-speckle three-dimensional DIC monitoring method and system for concrete corrosion damage, which extracts concrete natural texture features through SIFT operator to replace artificial speckle, and processes speed limited based on affine mapping and iterative optimization algorithm, which cannot output correction results in real time.

[0009] Patent CN201910659769.7 discloses a strain field calculation method combining SPM and DIC technology, which uses discrete cosine transform (DCT) to adaptively smooth the displacement field and eliminate random errors, but does not involve adaptive correction mechanism in dynamic environment (such as temperature fluctuation and vibration), which is easy to cause accuracy decline due to data drift in long-term monitoring. However, there are few precision improvement methods and systems based on artificial intelligence technology.

[0010] Patent CN202310512199.5 proposes a chip warping prediction method and system based on DIC measurement and machine learning, which takes the displacement field data of the chip surface as input data, and trains a machine learning model to predict the warping of the chip. For real-time and high-precision scenarios, the existing technology has certain limitations, and there is a lack of a method and system that can simultaneously meet the requirements of calculation efficiency, strong image feature learning ability, and fast training and reasoning. SUMMARY

[0011] In view of the problems existing in the prior art, the purpose of the present application is to provide a DIC displacement field dynamic real-time correction method and system based on lightweight convolutional neural network, which can significantly improve the measurement accuracy while meeting the real-time requirement, adapt to the structure health monitoring demand in complex environment, and further promote the technical progress and development of the field of structure health monitoring.

[0012] To achieve the above purpose, the technical scheme adopted by the present application is as follows: a DIC displacement field dynamic correction method based on lightweight convolutional network, comprising the following steps:

[0013] Step 1: Collecting image sequence of structure surface, pre-processing the collected image sequence;

[0014] Step 2: Calculate the initial displacement field of the pre-processed image sequence by the traditional DIC algorithm, and extract the key feature information as the input data for dynamic correction;

[0015] Step 3: Construct a lightweight convolutional neural network model, which uses the initial displacement field containing noise interference and the corresponding true displacement field as the training data set, transmits the extracted input data to the lightweight convolutional neural network model for recognition, trains through the design of a hybrid loss function, and learns the mapping relationship between the initial displacement field and the true displacement field;

[0016] Step 4: Correct the initial displacement field through the trained lightweight convolutional neural network model, and directly output the corrected high-precision displacement field data.

[0017] The above DIC displacement field dynamic correction method based on lightweight convolutional network, the step 1 comprises:

[0018] Step 1-1: According to the actual monitoring requirements and the characteristics and deformation characteristics of the structure, select a high-speed camera with appropriate focal length and resolution, determine the best installation position and angle, use a fixing device to ensure the stability of the camera, and clearly capture the deformation of the structure surface during the loading process;

[0019] Step 1-2: According to the deformation rate of the structure and the monitoring requirements, continuously perform image acquisition according to the set frame rate, and the resolution of image acquisition satisfies: pixel number / unit length≥ , wherein d is the minimum feature size of the structure surface;

[0020] Step 1-3: Use the weighted average method to perform grayscale processing on the image, convert the color image into a grayscale image, and the calculation formula is: Y =0.299 R +0.587 G +0.114 B , wherein R , G , B are the pixel values of the red, green and blue channels of the image respectively, Y is the pixel value after grayscale processing;

[0021] Step 1-4: According to the type and intensity of the noise in the image, select a suitable noise filtering algorithm to process the grayscale image, remove the random noise interference in the image, and improve the image quality;

[0022] Step 1-5: Normalize the image, according to the input requirements of the model and the characteristics of the data distribution, unify the image pixel value to the range of [0, 1], enhance the adaptability and generalization ability of the model to different image data, the calculation is as follows: , wherein, I is the pixel value of the original image, and are the minimum and maximum pixel values of the image, is the normalized pixel value.

[0023] Step 1-6: Evaluate the quality of the preprocessed image by calculating the mean and standard deviation of the image.

[0024] The above DIC displacement field dynamic correction method based on lightweight convolutional network, in step 1-4, the noise filtering algorithm uses non-local mean filtering, which filters the image by calculating the similarity weight between pixels, and preserves the texture and details of the image, including:

[0025] Step a: Divide the image into several pixel blocks patch, each pixel block takes a pixel as the center, and contains the surrounding neighborhood pixels;

[0026] Step b: Define the pixel block, for each pixel x in the image, define a pixel block x with n × n size centered at I patch ; x

[0027] Step c: Calculate the similarity weight, for two pixels x and y in the image, calculate the similarity weight of the pixel blocks they are in, the calculation formula of the similarity weight is: , wherein, represents the Euclidean distance between two pixel blocks, h is a smoothing parameter, used to control the decay rate of the weight;

[0028] Step d: Normalization, calculate the normalization constant x of the center pixel : , wherein, is the image definition domain;

[0029] Step e: Update the pixel value, update the value of the center pixel x according to the similarity weight and the normalization constant:

[0030] ​ repeating the above steps to obtain the final filtered image .

[0031] The DIC displacement field dynamic correction method based on the lightweight convolutional network, the step 2 comprises:

[0032] Step 2-1: compare the gray correlation of pixels between adjacent images, calculate the initial displacement field of each point on the structure surface, and detect the corner point features in the image by using a corner point detection algorithm, and the calculation formula of the corner point detection is: Wherein, M is the autocorrelation matrix of the image, k is an empirical constant;

[0033] Step 2-2: use an edge detection algorithm to extract image edge information, and combine a texture feature analysis method to comprehensively extract information reflecting key deformation features of the structure surface as input data.

[0034] The DIC displacement field dynamic correction method based on the lightweight convolutional network, the step 2-2 comprises:

[0035] Step f: smoothing and denoising the image by using a Gaussian filter;

[0036] Step g: calculate the gradient amplitude and direction, and use a Sobel operator to calculate the gradient components of each pixel point in the horizontal and vertical directions:

[0037] ,

[0038] Wherein, is the gradient component of the image in the horizontal direction, is the gradient component of the image in the vertical direction, is the input image after Gaussian filtering processing;

[0039] Step h: calculate the gradient amplitude and direction according to the gradient component:

[0040] ,

[0041] Wherein, is the gradient amplitude, is the gradient direction;

[0042] Step i: non-maximum suppression, compare the gradient amplitude of each pixel point with the gradient amplitudes of two field pixel points in the same direction, if the gradient amplitude of a certain pixel point is not the maximum, suppress the pixel point and set its gradient amplitude to 0;

[0043] Step j: double threshold method is used to detect edges to obtain complete edge profile.

[0044] The DIC displacement field dynamic correction method based on lightweight convolutional network, the double threshold method for detecting edges comprises:

[0045] Setting a high threshold T high and a low threshold T low , the pixel points with gradient amplitude greater than T high are marked as strong edges E strong , indicating that the pixel points belong to obvious edges, the pixel points with gradient amplitude between T low and T high are marked as weak edges E weak , indicating that the pixel points may be part of the edge, but are uncertain, and the pixel points with gradient amplitude less than T low are marked as non-edges, indicating that the pixel points do not belong to edges.

[0046] Starting from the strong edges E strong , search the pixels in the neighborhood along the gradient direction or the opposite direction, if there is a weak edge E weak in the neighborhood, assume that the weak edge is connected with the strong edge, then mark it as a strong edge, thereby realizing the connection of the edges, repeat the above process until all possible connected weak edges are processed, and finally obtain the complete edge profile.

[0047] The DIC displacement field dynamic correction method based on lightweight convolutional network, in step 2-2, the texture feature analysis adopts a gray level co-occurrence matrix (GLCM) method, calculates contrast, correlation, energy and entropy features of the image, fuses the calculated features to form a feature vector, including:

[0048] Step k: convert the image into a gray image, construct a gray level co-occurrence matrix according to image parameters, the image parameters include a gray level number N , a spatial relationship, the spatial relationship includes determining distances d and angles between pixel pairs.

[0049] Step l: initialize the co-occurrence matrix, create a matrix N × N P , initialize all elements to 0. ​

[0050] Step m: fill the co-occurrence matrix, traverse each pixel in the image , get its gray value ; according to the set distance d and angle , find the corresponding neighbor pixel , get its gray value ; increase the count of elements in the co-occurrence matrix ;

[0051] Step n: normalize the co-occurrence matrix, divide each element in the co-occurrence matrix by the total number of pixel pairs, and get the probability matrix P ;

[0052] Step o: calculate the texture features from the normalized co-occurrence matrix, including:

[0053] Contrast reflects the uniformity of the image gray distribution:

[0054] ,

[0055] Correlation reflects the linear relationship of the gray values in the image:

[0056] ,

[0057] ,

[0058] ,

[0059] ,

[0060] ,

[0061] Energy reflects the uniformity of the image gray distribution:

[0062] ,

[0063] Entropy reflects the complexity of the image gray distribution:

[0064] , where represents the mean value of the gray value of the row index gray level in the gray co-occurrence matrix, i represents the mean value of the gray value of the column index gray level in the gray co-occurrence matrix, represents the standard deviation of the gray value of the row index gray level in the gray co-occurrence matrix, j represents the standard deviation of the gray value of the column index gray level in the gray co-occurrence matrix, represents the standard deviation of the gray value of the row index gray level in the gray co-occurrence matrix, i represents the standard deviation of the gray value of the column index gray level in the gray co-occurrence matrix, Gray value of standard deviation of gray level of column index in gray level co-occurrence matrix j .

[0065] The DIC displacement field dynamic correction method based on the lightweight convolutional network, the step 3 comprises:

[0066] Step 3-1: decompose the standard convolution into deep convolution and gradual convolution to construct a lightweight convolutional neural network model, in the model training process, design a hybrid loss function, according to the characteristics of the training data and the performance requirements of the model, set the weight between the mean square error loss , gradient loss and adversarial loss , balance the contribution of each part, the calculation formula is: , wherein, , , is the weight coefficient, respectively control the contribution of mean square error loss, gradient loss and adversarial loss;

[0067] Step 3-2: adopt learning rate adjustment strategy and optimization algorithm, carry out multi-round iteration training, until the model converges, the trained model can learn the mapping relationship between the initial displacement field and the real displacement field.

[0068] The DIC displacement field dynamic correction method based on the lightweight convolutional network, the step 4 comprises:

[0069] Step 4-1: input initial displacement field data: input the initial displacement field data calculated by traditional DIC algorithm as the input of lightweight convolutional neural network model;

[0070] Step 4-2: input the initial displacement field data into the lightweight convolutional neural network and carry out forward propagation calculation;

[0071] Step 4-3: convolution layer calculation, for each convolution kernel k , the calculation formula is:

[0072] , wherein, H and W are the height and width of the convolution kernel, is the value of the initial displacement field data at position , is the weight value of the convolution kernel at position ;

[0073] Step 4-4: activation function calculation, after the convolution layer, apply the activation function to introduce nonlinearity, make the neural network learn and fit more complex function relationship, the calculation formula is: ;

[0074] Step 4-5: Obtain correction amount After the forward propagation of the network, the network outputs the correction amount The dimension of the initial displacement field data Each element in the correction amount represents the displacement correction value of the corresponding position.

[0075] Step 4-6: Calculate the corrected displacement field : .

[0076] A DIC displacement field dynamic correction system based on a lightweight convolutional network, comprising:

[0077] An image acquisition and processing module composed of a high-speed camera, image processing software and its control circuit, used for acquiring image sequences of a structure surface and preprocessing the acquired image sequences;

[0078] An initial displacement field calculation and feature extraction module that calculates the initial displacement field from the preprocessed image sequences through a traditional DIC algorithm and extracts key feature information using a feature extraction algorithm library as input data for dynamic correction;

[0079] A neural network correction module containing a trained lightweight convolutional neural network model that receives input data from the initial displacement field calculation and feature extraction module, uses an initial displacement field containing noise interference and the corresponding true displacement field as a training data set, transmits the extracted input data to the lightweight convolutional neural network model for identification, trains through the design of a hybrid loss function, learns the mapping relationship between the initial displacement field and the true displacement field, and directly outputs the corrected high-precision displacement field data through the trained lightweight convolutional neural network model.

[0080] The DIC displacement field dynamic correction method and system based on a lightweight convolutional network has the beneficial effects of: using a lightweight convolutional neural network to dynamically correct the displacement field obtained by DIC measurement, which can fully utilize the powerful function fitting capability of the neural network to achieve high-precision displacement field correction under limited computing resources, and significantly improve the measurement accuracy.

[0081] By designing a hybrid loss function, the absolute error, gradient information and adversarial training of the displacement field are considered comprehensively, so that the model can better learn the complex mapping relationship between the initial displacement field and the true displacement field during the training process, and the reliability and stability of the correction result are improved.

[0082] From the overall process, the method can dynamically and real-time correct the DIC displacement field under the premise of meeting the accuracy requirement, is suitable for the structure health monitoring, material testing and other scenes needing high-precision real-time displacement field measurement in complex environment, and has wide application prospect and practical engineering value. BRIEF DESCRIPTION OF DRAWINGS

[0083] Figure 1 A flow chart of the DIC displacement field dynamic real-time correction method based on the lightweight convolutional network of the embodiment of the application is shown in the figure.

[0084] Figure 2 A structure schematic diagram of the DIC displacement field dynamic real-time correction system based on the lightweight convolutional neural network of the embodiment of the application is shown in the figure.

[0085] Figure 3 A process diagram of the training data set construction under the experimental conditions in the embodiment 2 of the application is shown in the figure. DETAILED DESCRIPTION

[0086] In order to make the skilled in the art better understand the technical scheme of the application, the technical scheme of the application will be described below in combination with specific embodiments and drawings.

[0087] Embodiment 1

[0088] A DIC displacement field dynamic correction method based on a lightweight convolutional network comprises the following steps.

[0089] Step 1: collect the image sequence of the structure surface, and pretreat the collected image sequence.

[0090] The image sequence of the structure surface is collected by using a high-speed camera. The high-speed camera can capture the surface images of the structure in different states at a high frame rate, thereby providing a data basis for subsequent displacement field calculation.

[0091] The pretreatment process includes gray processing of the image, reduction of the data dimension, improvement of the processing efficiency, noise filtering processing for removing random noise interference in the image and improving the image quality, and image normalization processing for unifying the image pixel value to a certain range, thereby enhancing the generalization ability and stability of the model.

[0092] Step 2: calculate the initial displacement field by using the traditional DIC algorithm on the pretreated image sequence, and extract the key feature information as the input data of the dynamic correction.

[0093] In the calculation of the initial displacement field, the traditional DIC algorithm determines the displacement of each point on the structure surface by comparing the gray level changes of the pixels between adjacent images. At the same time, in order to better capture the key deformation characteristics of the structure surface, one or more methods such as image-based corner detection, edge detection, and texture feature analysis are combined to extract key feature information that can reflect the typical deformation characteristics of the structure surface, providing more valuable input for subsequent neural network correction.

[0094] Step 3: Construct a lightweight convolutional neural network model, which uses the initial displacement field containing noise interference and the corresponding true displacement field as the training data set, transmits the extracted input data to the lightweight convolutional neural network model for recognition, and trains through the design of a hybrid loss function to learn the mapping relationship between the initial displacement field and the true displacement field.

[0095] During the training process, a hybrid loss function is designed to achieve high-precision displacement field correction under limited computing resources. The hybrid loss function includes a mean square error loss function for measuring the difference between the predicted displacement field and the true displacement field, making the model as close to the true value as possible; a gradient loss function for constraining the gradient change of the displacement field, maintaining the smoothness and continuity of the displacement field; and an adversarial loss function that improves the generalization ability and robustness to noise by introducing an adversarial training mechanism.

[0096] Step 4: Correct the initial displacement field through the trained lightweight convolutional neural network model and directly output the corrected high-precision displacement field data.

[0097] Through end-to-end training, the lightweight convolutional neural network model can learn the mapping relationship between the initial displacement field and the true displacement field, thereby achieving accurate correction of the initial displacement field.

[0098] Example 2

[0099] As shown in FIG. 1, a DIC displacement field dynamic correction method based on a lightweight convolutional network includes the following steps.

[0100] Step 1: Use a high-speed camera to capture image sequences of the structure surface. According to the actual monitoring requirements and the characteristics and deformation characteristics of the structure, select a high-speed camera with appropriate focal length and resolution, and determine the best installation position and angle to ensure that the deformation of the structure surface during the application of load and other processes can be clearly captured. Use a tripod or other fixing device to ensure the stability of the camera and avoid shaking during the acquisition process.

[0101] The image acquisition is continuous at a certain frame rate, which should be determined according to the rate of structural deformation and monitoring requirements to ensure that the deformation process of the structure can be captured completely. The resolution should satisfy the condition of pixel number / unit length ≥ ( d The minimum feature size of the structure surface) to ensure that the image can clearly distinguish the details of the structure.

[0102] Step 2, pre-processing the collected image sequence. First, the image is grayed to convert the color image to a gray image, reducing the data volume and computational complexity. The weighted average method is used for gray processing, and the calculation formula is:

[0103] Y =0.299 R +0.587 G +0.114 B，

[0104] Wherein, R 、 G 、 B The pixel values of the red, green and blue channels of the image respectively, Y is the pixel value after gray processing.

[0105] Then, according to the type and intensity of noise in the image, select the appropriate noise filtering algorithm, such as Gaussian filtering, median filtering, bilateral filtering, wavelet transform, non-local mean filtering, etc. Process the gray image to remove random noise interference in the image and improve the image quality.

[0106] Optionally, the weight distribution of Gaussian filtering is in the shape of a Gaussian function, and the farther the pixel is from the center pixel, the smaller the weight is. Therefore, the pixels near the edge will not be smoothed too much, so that the edge details of the image can be better preserved.

[0107] Optionally, median filtering is a nonlinear filtering algorithm that replaces the value of each pixel point with the median value of the pixel values in the neighborhood of the pixel point, which can effectively remove salt and pepper noise while preserving the edge and detail information of the image.

[0108] Optionally, bilateral filtering takes into account the spatial distance and gray similarity of pixels, and is suitable for images that need to be processed in detail.

[0109] Optionally, wavelet transform preserves more local features while removing noise, and is more suitable for images with complex textures and rich details by decomposing the image into different scales and frequencies.

[0110] Optionally, non-local mean filtering can better preserve the texture and details of the image by calculating the similarity weight between pixels.

[0111] The specific steps include:

[0112] Divide the image into several pixel patches, each of which is centered at a pixel and contains the surrounding neighborhood pixels.

[0113] Define a pixel patch for each pixel in the image x , define a pixel patch centered at x with size n × n I patch ( x ).

[0114] Calculate the similarity weight, for two pixels x and y in the image, calculate the similarity weight of the pixel patches they are in ω ( x , y ).

[0115] The formula for calculating the similarity weight is:

[0116] where represents the Euclidean distance between the two pixel patches, h is a smoothing parameter used to control the decay rate of the weight.

[0117] Normalization, calculate the normalization constant x of the center pixel :

[0118] where is the image definition domain.

[0119] Update the pixel value, update the value of the center pixel x according to the similarity weight and the normalization constant:

[0120] .

[0121] Repeat the above steps to get the final filtered image .

[0122] After filtering, normalize the image according to the input requirements and data distribution characteristics of the model, and unify the image pixel value to the range of [0, 1] to enhance the adaptability and generalization ability of the model to different image data, the calculation formula is:

[0123] where I is the pixel value of the original image, and​ are the minimum and maximum pixel values of the image, respectively, is the normalized pixel value.

[0124] Finally, the quality of the pre-processed image can also be evaluated by calculating its mean and standard deviation, ensuring that it meets the requirements for subsequent processing.

[0125] Step 3, initial displacement field calculation on the pre-processed image sequence by traditional DIC algorithm, and key feature information extraction. The traditional DIC algorithm calculates the initial displacement field of each point on the structure surface by comparing the gray correlation of pixels between adjacent images. According to the deformation characteristics of the structure and the monitoring requirements, appropriate DIC algorithm parameters are selected, such as search area size, matching template size, etc., to improve the calculation accuracy of the initial displacement field.

[0126] At the same time, corner detection algorithm (such as Harris corner detection) is used to detect the corner features in the image, edge detection algorithm (such as Canny edge detection) is used to extract the edge information of the image, and texture feature analysis method (such as local binary pattern) is used to comprehensively extract the information that can reflect the key deformation characteristics of the structure surface as input data.

[0127] The calculation formula of Harris corner detection is: where, M is the autocorrelation matrix of the image, k is an empirical constant, usually between 0.04-0.06.

[0128] The Canny edge detection algorithm includes the following steps.

[0129] Image smoothing and denoising, using a Gaussian filter to smooth and denoise the image.

[0130] Calculate the gradient amplitude and direction, use the Sobel operator to calculate the gradient components of each pixel point in the horizontal and vertical directions:

[0131] ,

[0132] ,

[0133] where, is the gradient component of the image in the horizontal direction, is the gradient component of the image in the vertical direction, is the input image after Gaussian filtering.

[0134] Calculate the gradient amplitude and direction according to the gradient components:

[0135] ,

[0136] ,

[0137] wherein, is the gradient magnitude, is the gradient direction.

[0138] Non-maximum suppression, compare the gradient magnitude of each pixel point with the gradient magnitude of two field pixel points in the same direction, if the gradient magnitude of a certain pixel point is not the maximum, suppress the pixel point and set its gradient magnitude to 0.

[0139] Double threshold method detects edges.

[0140] Set high threshold (T T high ) and low threshold (T T low ).

[0141] Mark the pixel point with gradient magnitude greater than T T high as strong edge (E E strong ), indicating that the pixel point belongs to obvious edge.

[0142] Mark the pixel point with gradient magnitude between T T low and T T high as weak edge (E E weak ), indicating that the pixel point may be part of the edge, but it is not certain.

[0143] Mark the pixel point with gradient magnitude less than T T low as non-edge, indicating that the pixel point does not belong to the edge.

[0144] Starting from strong edge E E strong , search for pixels in its neighborhood along the gradient direction or its opposite direction, if there is a weak edge E E weak in the neighborhood, assume that the weak edge is connected with the strong edge, then mark it as strong edge, thereby realizing the connection of edges. Repeat the above process until all possible connected weak edges are processed, and finally obtain the complete edge contour.

[0145] Texture feature analysis uses the gray level co-occurrence matrix (GLCM) method to calculate the contrast, correlation, energy and entropy of the image, and to fuse the calculated features to form a feature vector for subsequent analysis and processing.

[0146] The specific steps are as follows:

[0147] If the input image is a color image, it needs to be converted to a grayscale image first.

[0148] To construct a Gray Level Co-occurrence Matrix (GLCM), the following parameters need to be determined first.

[0149] Number of gray levels: Usually set the number of gray levels to 256 (for 8-bit images), or compress the number of gray levels as needed (such as 16 levels, 32 levels, etc.).

[0150] Spatial relationship: Determine the distance between pixel pairs d and angle . Common distances d are 1, and angles are usually 0°, 45°, 90°, 135°.

[0151] The construction steps of GLCM are as follows.

[0152] Initialize the co-occurrence matrix, create a matrix N of size N × P , where N is the number of gray levels, and initialize all elements to 0.

[0153] Fill the co-occurrence matrix, iterate through each pixel point in the image, get its gray value ; according to the set distance d and angle , find the corresponding neighbor pixel point , get its gray value ; increase the count of elements in the co-occurrence matrix .

[0154] Normalize the co-occurrence matrix, divide each element in the co-occurrence matrix by the total number of pixel pairs to get the probability matrix P .

[0155] Calculate the texture features, calculate the following texture features from the normalized co-occurrence matrix P .

[0156] Contrast (Contrast), reflects the uniformity of the image gray scale distribution:

[0157] ,

[0158] Correlation (Correlation), reflects the linear relationship of gray values in the image:

[0159] ,

[0160] ,

[0161] ,

[0162] ,

[0163] ,

[0164] Energy also reflects the uniformity of the image gray scale distribution:

[0165] ,

[0166] Entropy reflects the complexity of the image gray scale distribution:

[0167] where, denotes the mean value of the gray scale value of the row index (gray scale level i ) in the gray scale co-occurrence matrix, denotes the mean value of the gray scale value of the column index (gray scale level j ) in the gray scale co-occurrence matrix, denotes the standard deviation of the gray scale value of the row index (gray scale level i ) in the gray scale co-occurrence matrix, denotes the standard deviation of the gray scale value of the column index (gray scale level j ) in the gray scale co-occurrence matrix.

[0168] Based on the above steps, according to the specific structure and deformation situation, the parameters of the feature extraction algorithm are adjusted, such as the threshold value in the corner detection algorithm, the filter size in the edge detection algorithm, etc., and the extracted feature information is screened and fused to remove redundant and irrelevant information, so as to ensure that the feature data input to the neural network has high quality and representativeness.

[0169] Step 4, the extracted input data is transmitted to a lightweight convolutional neural network model for recognition. In the training stage, the lightweight convolutional neural network model uses the initial displacement field containing noise interference and the corresponding real displacement field as the training data set. For example, DIC measurement can be performed on the structure with known deformation in the laboratory conditions by using high-precision measurement equipment to obtain image sequences with real displacement field data. According to the noise characteristics and intensity in the actual application scene, select appropriate noise types (such as Gaussian noise, salt and pepper noise, etc.) and intensity parameters, add noise to the initial displacement field data to simulate the measurement situation in the actual complex environment, and form the training data set.

[0170] The lightweight convolutional neural network model employs depthwise separable convolution, decomposing standard convolution into depthwise convolution and progressive convolution, significantly reducing computational cost and the number of model parameters. During model training, a hybrid loss function is designed. Based on the characteristics of the training data and the model's performance requirements, the weights of the mean squared error loss, gradient loss, and adversarial loss are appropriately set to balance the contributions of each component and achieve optimal training results. The calculation formula is as follows:

[0171] ,in, , , These are weighting coefficients that control the contributions of mean squared error loss, gradient loss, and adversarial loss, respectively.

[0172] The mean squared error loss function is used to calculate the difference between the predicted displacement field and the actual displacement field:

[0173] ,in, and The first and second displacement fields are respectively the predicted displacement field and the actual displacement field. i One element, N The total number of elements.

[0174] The gradient loss function is used to constrain the gradient change of the displacement field:

[0175] ,in, and These are the gradients of the predicted displacement field and the actual displacement field, respectively.

[0176] The adversarial loss function introduces an adversarial training mechanism between the discriminant network and the generator network. Based on the discriminant network's judgment results on real and generated data, and the corresponding real labels, the adversarial loss value is calculated. The calculation formula is:

[0177] ,in, N For the data sample size, It is a real label (real data is 1, generated data is 0). It is to determine the network's accuracy of real data. The judgment result, It is a generative network for generating noisy initial displacement field data. The generated results It is to determine the network's response to the generated data. The judgment result.

[0178] With appropriate learning rate adjustment strategies and optimization algorithms, multiple rounds of iterative training are performed until the model converges. Ultimately, the trained model can better learn the mapping relationship between the initial displacement field and the true displacement field, correct the initial displacement field, and directly output the corrected high-precision displacement field data after end-to-end training.

[0179] The correction step includes:

[0180] Input initial displacement field data: the initial displacement field data calculated by the traditional DIC algorithm as the input of the lightweight convolutional neural network model. The initial displacement field data is a two-dimensional matrix representing the initial displacement values of each point on the structure surface.

[0181] Forward propagation of the network: input the initial displacement field data into the lightweight convolutional neural network for forward propagation calculation.

[0182] Convolution layer calculation: for each convolution kernel k , the calculation formula is:

[0183] wherein, H and W are the height and width of the convolution kernel, is the value of the initial displacement field data at position , is the weight value of the convolution kernel at position .

[0184] Activation function calculation: after the convolution layer, an activation function is applied to introduce nonlinearity, enabling the neural network to learn and fit more complex function relationships. The calculation formula is: .

[0185] Obtain correction amount : after forward propagation of the network, the network outputs the correction amount . which is also a two-dimensional matrix with the same dimensions as the initial displacement field data . Each element in represents the displacement correction value at the corresponding position.

[0186] Calculate the corrected displacement field :

[0187] The process of constructing the training data set under laboratory conditions mentioned in step 4 is shown in Figure 3 , and the specific steps include.

[0188] Prepare a known deformed structure specimen: Select a structure specimen with a known deformation pattern, for example, a material sample that will experience uniform stretching or bending under a specific load.

[0189] Perform DIC measurement to obtain real displacement field: Install the specimen on a high-precision measurement device, use a high-speed camera to capture image sequences of the structure surface, ensure that the camera parameters (such as frame rate, resolution) are set reasonably to capture clear deformation process; start the loading device and high-speed camera simultaneously, record the image data of the structure during the deformation process; obtain the real displacement field data of the structure surface through high-precision measurement device, these data will be used as the real label in the training data set.

[0190] Simulate the actual initial displacement field: Add noise to the obtained real displacement field data to simulate the measurement situation in the actual complex environment, you can add Gaussian noise, salt and pepper noise, etc., the noise intensity should be set according to the interference degree that may be encountered in the actual application scene.

[0191] Data pairing and organization: Pair the data after adding noise with the real displacement field data, ensure that each pair of data is aligned in time stamp and spatial position, so as to accurately correspond in model training; organize the paired data, divide the training set, validation set and test set.

[0192] Form the training data set: Store the paired and organized data in the database or file system, build a complete training data set for the training of lightweight convolutional neural network model.

[0193] Example 3

[0194] Based on example 1 and example 2, as shown in figure 2, this embodiment provides a DIC displacement field real-time correction system based on lightweight convolutional network, which includes image acquisition and processing module, initial displacement field calculation and feature extraction module and neural network correction module. Image acquisition and processing module includes image acquisition module and preprocessing module.

[0195] Image acquisition module: composed of high-speed camera and its control circuit, responsible for acquiring image sequences of the structure surface.

[0196] Preprocessing module: running on the image processing software of computer or dedicated image processing hardware, sequentially performing grayscale, noise filtering and normalization and other preprocessing operations on the acquired image sequences.

[0197] Initial displacement field calculation and feature extraction module: integrated with traditional DIC algorithm program and feature extraction algorithm library, used for calculating initial displacement field and extracting key feature information.

[0198] The neural network correction module comprises a trained lightweight convolutional neural network model, receives input data from the initial displacement field calculation and feature extraction module, and quickly outputs corrected high-precision displacement field data through model inference. The whole system works cooperatively to realize the dynamic real-time correction function of the DIC displacement field.

[0199] The lightweight convolutional neural network model uses initial displacement field data containing noise and corresponding real displacement field data as a training set, and is trained based on a designed hybrid loss function. During model training, according to the characteristics of the training data and the performance requirements of the model, the weights of each part of the hybrid loss function are reasonably set, and appropriate learning rate adjustment strategies and optimization algorithms are used for multiple rounds of iterative training until the model converges. The trained model can construct a mapping relationship function between the initial displacement field and the real displacement field, thereby correcting new initial displacement field data to make it closer to the real displacement field. In practical applications, the performance of the model is evaluated in different scenarios, such as calculating the error index (such as mean square error, mean absolute error, etc.) between the corrected displacement field data and the real displacement field data, evaluating the degree of precision improvement of the model; at the same time, the processing speed and delay of the real-time data of the measurement system are measured to evaluate the real-time performance of the system, so as to more comprehensively demonstrate the advantages and application value of the present application.

[0200] By combining the lightweight convolutional neural network with traditional DIC technology, a dynamic real-time correction method and system are innovatively proposed, effectively solving the problem of insufficient DIC displacement field measurement accuracy in complex environments, while meeting the real-time requirements, providing more accurate and reliable displacement field measurement means for structural health monitoring, material testing and other fields, and having significant technical advantages and application value. It can significantly improve the measurement accuracy while meeting the real-time requirements, and adapt to the needs of structural health monitoring in complex environments.

[0201] The above embodiments are only used to illustrate the structural concept and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the content of the present application and implement it, and cannot limit the protection scope of the present application. Any equivalent changes or modifications made according to the essence of the present application should be covered within the protection scope of the present application.

Claims

1. A DIC displacement field dynamic correction method based on a lightweight convolutional network, characterized in that, Includes the following steps: Step 1: Acquire image sequences of the structural surface and preprocess the acquired image sequences; Step 2: Calculate the initial displacement field of the preprocessed image sequence using the traditional DIC algorithm, and extract key feature information as input data for dynamic correction; Step 3: Construct a lightweight convolutional neural network model. This model uses an initial displacement field containing noise interference and the corresponding true displacement field as training datasets. The extracted input data is transmitted to the lightweight convolutional neural network model for recognition. Training is performed using a hybrid loss function to learn the mapping relationship between the initial displacement field and the true displacement field, including: Step 3-1: decompose the standard convolution into a deep convolution and a gradual convolution to build a lightweight convolutional neural network model, design a hybrid loss function during model training, set the weights between the mean square error loss , gradient loss and adversarial loss according to the characteristics of the training data and the performance requirements of the model, balance the contributions of each part, and the calculation formula is: wherein, , , is the weight coefficient, which controls the contribution of the mean square error loss, gradient loss and adversarial loss respectively; Step 3-2: Use a learning rate adjustment strategy and optimization algorithm to perform multiple rounds of iterative training until the model converges. The trained model can learn the mapping relationship between the initial displacement field and the real displacement field. Step 4: Correct the initial displacement field using the trained lightweight convolutional neural network model, and directly output the corrected high-precision displacement field data.

2. The DIC displacement field dynamic correction method based on a lightweight convolutional network according to claim 1, characterized in that, Step 1 includes: Step 1-1: Based on the actual monitoring needs and the characteristics and deformation features of the structure, select a high-speed camera with a suitable focal length and resolution, determine the optimal installation position and angle, use a fixing device to ensure the stability of the camera, and clearly capture the deformation of the structural surface during the application of load. Steps 1-2: Based on the rate of structural deformation and monitoring requirements, continuously acquire images at the set frame rate. The resolution of the acquired images must meet the following condition: number of pixels / unit length ≥ ,in d It is the minimum feature size of the structural surface; Steps 1-3: Use the weighted average method to convert the image to grayscale, transforming the color image into a grayscale image. The calculation formula is as follows: Y =0.299 R +0.587 G +0.114 B ,in, R , G , B These are the pixel values ​​of the red, green, and blue channels of the image, respectively. Y These are the pixel values ​​after grayscale conversion; Steps 1-4: Based on the type and intensity of noise in the image, select an appropriate noise filtering algorithm to process the grayscale image, remove random noise interference, and improve image quality; Steps 1-5: Normalize the image. Based on the model's input requirements and data distribution characteristics, unify the image pixel values ​​to the range of [0, 1] to enhance the model's adaptability and generalization ability to different image data. The calculation formula is as follows: Where I is the pixel value of the original image. and These are the minimum and maximum pixel values ​​of the image, respectively. These are the normalized pixel values; Steps 1-6: Evaluate the quality of the preprocessed image by calculating the mean and standard deviation of the image.

3. The method for dynamic correction of DIC displacement field based on lightweight convolutional networks according to claim 2, characterized in that, In steps 1-4, the noise filtering algorithm uses nonlocal mean filtering, which filters the image by calculating the similarity weights between pixels to preserve the image's texture and details, including: Step a: Divide the image into several pixel patches, each pixel patch being centered on a single pixel and including all surrounding neighboring pixels; Step b: Define pixel blocks for each pixel in the image. x Define a x Centered on, size is n × n pixel blocks I patch ( x ); Step c: Calculate the similarity weights. For two pixels x and y in the image, calculate the similarity weights of their respective pixel blocks. The formula for calculating similarity weight is: ,in, This represents the Euclidean distance between two pixel blocks, and h is a smoothing parameter used to control the rate of weight decay. Step d: Normalization process, calculate center pixel x normalization constant : ,in, It is the image domain; Step e: Update pixel values, updating the center pixel based on similarity weights and normalization constants. x Value: Repeat the above steps to obtain the final filtered image. .

4. The method for dynamic correction of DIC displacement field based on lightweight convolutional networks according to claim 1, characterized in that, Step 2 includes: Step 2-1: Compare the grayscale correlation of pixels between adjacent images to calculate the initial displacement field of each point on the structure surface. Use a corner detection algorithm to detect corner features in the image. The calculation formula for corner detection is: ,in, M It is the autocorrelation matrix of the image. k It is an empirical constant; Step 2-2: Use edge detection algorithms to extract image edge information, and combine them with texture feature analysis methods to comprehensively extract information that can reflect the key deformation features of the structural surface as input data.

5. The method for dynamic correction of DIC displacement field based on lightweight convolutional networks according to claim 4, characterized in that, Step 2-2 includes: Step f: Use a Gaussian filter to smooth and denoise the image; Step g: Calculate the gradient magnitude and direction, using the Sobel operator to calculate the gradient components of each pixel in the horizontal and vertical directions: ， ,in, The gradient components of the image in the horizontal direction. The gradient component of the image in the vertical direction. It is the input image after Gaussian filtering; Step h: Calculate the gradient magnitude and direction based on the gradient components: ， ,in, It is the gradient magnitude. It is the gradient direction; Step i: Non-maximum suppression. Compare the gradient magnitude of each pixel with the gradient magnitudes of two neighboring pixels in the same direction. If the gradient magnitude of a pixel is not the maximum, suppress the pixel and set its gradient magnitude to 0. Step j: Detect edges using the double threshold method to obtain the complete edge contour.

6. The method for dynamic correction of DIC displacement field based on lightweight convolutional networks according to claim 5, characterized in that, The dual-threshold method for edge detection includes: Set a high threshold T high and low threshold T low The gradient magnitude is greater than T high Pixels marked as strong edges E strong This indicates that the pixel belongs to a clear edge, and the gradient magnitude is between T low and T high Pixels between these points are marked as weak edges. E weak This indicates that the pixel may be part of an edge, but it's uncertain; the gradient magnitude is less than [value missing]. T low Pixels marked as non-edges indicate that the pixel does not belong to an edge; From the strong edge E strong Starting from the gradient direction or its opposite direction, search for pixels in its neighborhood. If a weak edge exists in the neighborhood... E weak If a weak edge is connected to a strong edge, it is marked as a strong edge, thus achieving the connection of edges. The above process is repeated until all possible weak edges that can be connected have been processed, and finally a complete edge contour is obtained.

7. The method for dynamic correction of DIC displacement field based on lightweight convolutional networks according to claim 4, characterized in that, In step 2-2, the texture feature analysis employs the Gray-Level Co-occurrence Matrix (GLCM) method to calculate the image's contrast, correlation, energy, and entropy features. The calculated features are then fused to form a feature vector, including: Step k: Convert the image to a grayscale image, and construct a gray-level co-occurrence matrix based on the image parameters, including the number of gray levels. N Spatial relationships, including determining the distance between pixel pairs. d and angle ; Step 1: Initialize the co-occurrence matrix, creating a matrix of size 1. N × N matrix P Initialize all elements to 0; Step m: Fill the co-occurrence matrix by traversing every pixel in the image. Obtain its grayscale value According to the set distance d and angle Find the corresponding neighboring pixels Obtain its grayscale value Increase the count of elements in the co-occurrence matrix. ; Step n: Normalize the co-occurrence matrix by dividing each element of the co-occurrence matrix by the total number of pixel pairs to obtain the probability matrix. P ; Step o: Calculate texture features from the normalized co-occurrence matrix, including: Contrast, which reflects the uniformity of grayscale distribution in an image: ， Correlation, which reflects the linear relationship between grayscale values ​​in an image: ， ， ， ， ， Energy, a value that reflects the uniformity of gray-level distribution in an image. ， Entropy, a value reflecting the complexity of an image's grayscale distribution: ,in, This represents the row index gray level in the gray-level co-occurrence matrix. i The mean of the gray values, This represents the column index of the gray level in the gray-level co-occurrence matrix. j The mean of the gray values, This represents the row index gray level in the gray-level co-occurrence matrix. i The standard deviation of the gray values, This represents the column index of the gray level in the gray-level co-occurrence matrix. j The standard deviation of the grayscale values.

8. The method for dynamic correction of DIC displacement field based on lightweight convolutional networks according to claim 1, characterized in that, Step 4 includes: Step 4-1: Input initial displacement field data: Input the initial displacement field data obtained by the traditional DIC algorithm. As input to a lightweight convolutional neural network model; Step 4-2: Input the initial displacement field data into the lightweight convolutional neural network and perform forward propagation calculation; Step 4-3: Convolutional layer calculation, for each convolutional kernel k The calculation formula is: ,in, H and W It refers to the height and width of the convolution kernel. The initial displacement field data is at position The value, Is the convolution kernel at position The weight value; Step 4-4: Activation function calculation. After the convolutional layer, an activation function is applied to introduce non-linearity, enabling the neural network to learn and fit more complex functional relationships. The calculation formula is as follows: ; Steps 4-5: Obtain the correction amount After forward propagation through the network, the network outputs the correction value. Compared with the initial displacement field data The same dimensions Each element in the table represents the displacement correction value at the corresponding position; Steps 4-6: Calculate the corrected displacement field : .

9. A dynamic correction system for DIC displacement field based on a lightweight convolutional network, characterized in that, include: Image acquisition and processing module: Composed of a high-speed camera, image processing software and its control circuit, used to acquire image sequences of the structural surface and preprocess the acquired image sequences; Initial displacement field calculation and feature extraction module: The initial displacement field is calculated on the preprocessed image sequence using the traditional DIC algorithm, and key feature information is extracted using a feature extraction algorithm library as input data for dynamic correction. The neural network correction module contains a pre-trained lightweight convolutional neural network model. It receives input data from the initial displacement field calculation and feature extraction module, using an initial displacement field containing noise interference and the corresponding real displacement field as the training dataset. The extracted input data is transmitted to the lightweight convolutional neural network model for recognition. Training is performed using a hybrid loss function to learn the mapping relationship between the initial and real displacement fields. This includes: decomposing standard convolution into depthwise convolution and progressive convolution to construct the lightweight convolutional neural network model; and designing a hybrid loss function during model training, setting the mean squared error loss based on the characteristics of the training data and the model's performance requirements. gradient loss and combat losses The weights between them are used to balance the contributions of each part, and the calculation formula is as follows: ,in, , , These are weight coefficients that control the contributions of mean squared error loss, gradient loss, and adversarial loss, respectively. A learning rate adjustment strategy and optimization algorithm are used to perform multiple rounds of iterative training until the model converges. The initial displacement field is corrected by the trained lightweight convolutional neural network model, and the corrected high-precision displacement field data is directly output.

Images