A strain measurement method and system based on digital speckle

Abstract

The present invention relates to a strain measurement method and system based on digital speckle. The method includes the following steps: obtaining a speckle image set for training, randomly labeling each speckle image in the speckle image set, and generating a training sample set based on the random labeling; training a neural network model based on the training sample set to obtain a speckle region recognition model; obtaining an original image and a deformed image of the surface to be measured; identifying the speckle region of the original image based on the speckle region recognition model; establishing a reference sub-region centered on each speckle in the speckle region of the original image, searching for the corresponding deformed sub-region of the reference sub-region in the deformed image through a sub-pixel search algorithm, and obtaining the displacement of the speckle based on the reference sub-region and the deformed sub-region; the displacements of all speckles constitute the displacement field of the deformed image; based on the displacement field of the deformed image, calculating the strain field of the surface to be measured through an in-plane strain calculation method.

Description

Technical Field

[0001] The present invention relates to the technical field of strain measurement, and in particular, to a strain measurement method and system based on digital speckle. Background Art

[0002] The digital speckle measurement method is a modern new optical measurement method based on image processing technology. By combining optical principles and mechanical principles, it realizes the deformation measurement of the surface of the object to be measured, and has advantages such as non-contact, full-field measurement, high automation, high precision, and high sensitivity. The digital speckle measurement method can effectively avoid the disadvantages of contact measurement, high equipment cost, and high requirements for the measurement environment in traditional measurement methods, and has now played an important role and been recognized in many fields such as biomedicine, civil engineering, and aerospace.

[0003] The digital speckle strain measurement method needs to first select the speckle position in the image as the target area, then perform relevant displacement calculations on the target area, and finally complete the strain measurement of the target area. However, in the current mainstream products on the market, the selection of the target area is manually operated by the user. For a large number of test measurement applications, it is necessary to manually select the target area of each group of images one by one, which will bring a large increase in labor costs. At the same time, due to the differences in the selection of the target area by different operators, it will also greatly increase the uncertainty of the calculation, thus resulting in inaccurate strain measurement. At the same time, the traditional method for calculating the strain field is obtained by directly differentiating the displacement field, but the displacement field distribution obtained after directly using the correlation search is uneven and fluctuates greatly. Using direct differentiation will amplify the noise, thus resulting in inaccurate strain measurement results. Summary of the Invention

[0004] In view of the above analysis, the embodiments of the present invention aim to provide a strain measurement method and system based on digital speckle to solve the problems of large workload in manually selecting the speckle area and inaccurate measurement results in the prior art.

[0005] On the one hand, an embodiment of the present invention discloses a strain measurement method based on digital speckle, including the following steps:

[0006] Obtain a set of speckle images for training, randomly mark each speckle image in the set of speckle images, and generate a training sample set based on the random marks; train a neural network model based on the training sample set to obtain a speckle area recognition model;

[0007] Obtain the original image and the deformed image of the measured surface; identify the speckle area of the original image based on the speckle area recognition model;

[0008] A reference sub-region is established with each speckle in the speckle region of the original image as the center. The deformed sub-region corresponding to the reference sub-region is searched in the deformed image through a sub-pixel search algorithm, and the displacement of the speckle is obtained based on the reference sub-region and the deformed sub-region; the displacements of all speckles constitute the displacement field of the deformed image;

[0009] Based on the displacement field of the deformed image, the strain field of the measured surface is calculated by an in-plane strain calculation method.

[0010] The beneficial effects of the above technical solution are as follows: A speckle region recognition model is obtained by training a neural network. The global speckle region is automatically recognized and located through the speckle region recognition model, replacing the operation of manually selecting the measurement target area in the traditional digital speckle measurement process. For the application scenario of batch measurement, the measurement efficiency can be greatly improved. At the same time, the uncertainty of the results caused by the differences in the selected areas is avoided, ensuring the measurement accuracy.

[0011] Furthermore, each speckle image in the speckle image set is randomly marked, and a training sample set is generated based on the random marking, including:

[0012] Random markings are respectively made on the speckle region and the background region of the speckle image with paintbrushes of different colors to generate a marked image;

[0013] All the marked points in the marked image are traversed. If the color of the marked point is the color corresponding to the speckle region, a neighborhood image of S×S is cropped with the coordinates of the marked point as the center in the speckle image as a speckle sample; if the color of the marked point is the color corresponding to the background region, a neighborhood image of S×S is cropped with the coordinates of the marked point as the center in the speckle image as a background sample, where S represents the training sample size. The beneficial effects of the above technical solution are as follows: The training sample set can be quickly generated by the method of marking first and then generating, avoiding a large amount of work of marking each sample image one by one, and greatly improving the sample generation efficiency.

[0014] Furthermore, identifying the speckle region of the original image based on the speckle region recognition model includes:

[0015] In the original image, each pixel point starting from (S / 2 + 1, S / 2 + 1) and ending at (H - S / 2, W - S / 2) is traversed. A neighborhood image of S×S is cropped with the pixel point as the center as an identification window, and the identification window is input into the speckle region recognition model to determine whether the current center point is a speckle region. If so, the pixel point is marked as a speckle; where H represents the height of the original image, W represents the width of the original image, and S represents the training sample size.

[0016] The beneficial effects of the above technical solution are as follows: By means of the recognition window and the speckle area recognition model, it can be quickly determined whether the center point is a speckle area, so as to realize the rapid recognition of the speckle area in the speckle image.

[0017] Further, a reference sub-region is established with each speckle point in the speckle area of the original image as the center, and the deformed sub-region corresponding to the reference sub-region is searched in the deformed image through the sub-pixel search algorithm. The displacement of the speckle point is obtained based on the reference sub-region and the deformed sub-region, including:

[0018] In the original image f, a reference sub-region is constructed with each speckle point (x, y) as the center and a neighborhood of (2M + 1)×(2M + 1).

[0019] Taking as the shape function and the normalized least square distance function as the objective function;

[0020] The deformed sub-region most similar to the reference sub-region is searched in the deformed image g through the inverse combination matching strategy;

[0021] The displacement of the speckle point (x, y) is expressed as Δx = x′ - x, Δy = y′ - y, where (x', y') represents the center point coordinates of the deformed sub-region, f(x, y) and g(x′, y′) are the gray values of the point (x, y) in the reference image and the deformed image respectively, M is the sub-region window radius, and u and v are the displacements of the point (x, y) in the X direction and the Y direction respectively; f m and g m are the average gray values of the reference sub-region and the deformed sub-region respectively, and the undetermined parameters

[0022] The beneficial effects of the above technical solution are as follows: Using the first-order shape function to represent the mapping shape function from the reference sub-region to the deformed sub-region is simple to calculate and easy to implement, and can well describe the actual deformation. Since the normalized least square distance function subtracts the average gray value of the sub-region and uses normalization processing, it has good anti-noise performance and insensitivity to light intensity. During the deformation process of the measured object, it is prone to problems such as overexposure and uneven illumination affected by the light source. Therefore, the normalized least square distance function is used as the objective function to measure the similarity between the reference sub-region and the deformed sub-region. Through the first-order shape function and the normalized least square distance function, the deformed sub-region corresponding to the reference sub-region can be accurately found, thus providing a basis for accurately calculating the displacement and strain.

[0023] Further, based on the displacement field of the deformed image, the strain field of the measured surface is calculated through the in-plane strain calculation method, including:

[0024] A strain sub-region is constructed with each point in the displacement field as the center;

[0025] Adopt a second-order polynomial Perform surface fitting on the displacement field within the strain sub-region; where (x, y) are the coordinates of the points within the strain sub-region in the reference image, u(x, y) and v(x, y) respectively represent the displacements of the point (x, y) in the horizontal and vertical directions, and u0, u1, u2, u3, u4, u5, v0, v1, v2, v3, v4, v5 are all fitting parameters;

[0026] Solve the fitting parameters of the second-order polynomial by the least squares method, and obtain the strain of each point in the displacement field based on the second-order polynomial;

[0027] The strains of all points in the displacement field constitute the strain field of the measured surface.

[0028] The beneficial effects of the above technical solution are: By fitting with a second-order polynomial and solving by the least squares method, the strain of each point in the displacement field can be quickly calculated, and the strain field of the measured surface can be obtained.

[0029] Further, after obtaining the displacement field of the deformed image, before calculating the strain field of the measured surface through the in-plane strain calculation method based on the displacement field of the deformed image, it further includes using a fast Gaussian low-pass filter to smooth the displacement field.

[0030] The beneficial effects of the above technical solution are: By smoothing the displacement field with a fast Gaussian filter, the problem that the displacement field distribution obtained directly after correlation search is uneven and fluctuates greatly, resulting in unstable calculation results, is solved. By using fast Gaussian filtering to smooth the displacement, the purpose of fast noise reduction can be achieved, so that the strain calculation result is more accurate.

[0031] Further, the speckle region recognition model includes:

[0032] The first convolutional layer is used to extract features from the input speckle image, and contains 20 7×7 convolutional kernels;

[0033] The first pooling layer is used to downsample the feature map output by the first local convolutional layer, and contains 20 2×2 convolutional kernels;

[0034] The second convolutional layer is used to extract features from the downsampled map output by the first pooling layer, and contains 50 7×7 convolutional kernels;

[0035] The second pooling layer is used to downsample the feature map output by the second convolutional layer, and contains 50 2×2 convolutional kernels;

[0036] The third convolutional layer is used to extract features from the downsampled map output by the second pooling layer, and contains 100 7×7 convolutional kernels;

[0037] The third pooling layer is used to downsample the feature map output by the third convolutional layer and contains 100 2×2 convolutional kernels;

[0038] The first fully connected layer is used to fully connect the feature map output by the third pooling layer into a one-dimensional feature map and contains 200 4×4 convolutional kernels;

[0039] The second fully connected layer is used to integrate the feature map output by the first fully connected layer, contains 100 neurons, and is fully connected to the first fully connected layer.

[0040] The beneficial effects of the above technical solution are: the constructed speckle area recognition model can quickly and accurately recognize the speckle area, providing a basis for realizing fully automatic strain measurement.

[0041] On the other hand, a specific embodiment of the present invention discloses a strain measurement system based on digital speckle, including the following modules:

[0042] The speckle area recognition model generation module is used to obtain a set of speckle images for training, randomly mark each speckle image in the set of speckle images, and generate a training sample set based on the random marks; train a neural network model based on the training sample set to obtain a speckle area recognition model;

[0043] The speckle area recognition module is used to obtain the original image and the deformed image of the measured surface; recognize the speckle area of the original image based on the speckle area recognition model;

[0044] The displacement field calculation module is used to establish a reference sub-region centered on each speckle in the speckle area of the original image, search for the corresponding deformed sub-region of the reference sub-region in the deformed image through a sub-pixel search algorithm, and obtain the displacement of the speckle based on the reference sub-region and the deformed sub-region; the displacements of all speckles constitute the displacement field of the deformed image.

[0045] The strain field calculation module is used to calculate the strain field of the measured surface based on the displacement field of the deformed image through an in-plane strain calculation method.

[0046] Further, the speckle area recognition model generation module includes:

[0047] The random marking module is used to randomly mark the speckle area and the background area of the speckle image with different colored brushes respectively to generate a marked image;

[0048] A sample generation module is used to traverse all marked points in an image. If the color of a marked point is the color corresponding to the speckle area, a neighborhood image of S×S is cropped with the coordinates of the marked point as the center in the speckle image as a speckle sample; if the color of the marked point is the color corresponding to the background area, a neighborhood image of S×S is cropped with the coordinates of the marked point as the center in the speckle image as a background sample, where S represents the size of the training sample.

[0049] Further, after obtaining the displacement field of the deformed image, before calculating the strain field of the measured surface by an in-plane strain calculation method based on the displacement field of the deformed image, it further includes smoothing the displacement field by using a fast Gaussian low-pass filter.

[0050] In the present invention, the above technical solutions can also be combined with each other to achieve more preferred combination schemes. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can be made obvious from the description, or understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained from the content specifically pointed out in the description and the drawings. Description of the Drawings

[0051] The drawings are only for the purpose of showing specific embodiments, and are not considered as limiting the present invention. Throughout the drawings, the same reference signs denote the same components.

[0052] Figure 1 It is a flowchart of the strain measurement method based on digital speckle according to an embodiment of the present invention;

[0053] Figure 2 It is a structural block diagram of the strain measurement system based on digital speckle according to an embodiment of the present invention;

[0054] Figure 3 It is a structural schematic diagram of the speckle area recognition model in an embodiment of the present invention. Detailed Embodiments

[0055] The following will specifically describe the preferred embodiments of the present invention in conjunction with the drawings, where the drawings form a part of this application and are used together with the embodiments of the present invention to explain the principles of the present invention, and are not used to limit the scope of the present invention.

[0056] A specific embodiment of the present invention discloses a strain measurement method based on digital speckle, as Figure 1 shown, including:

[0057] S1. Obtain a speckle image set for training, randomly mark each speckle image in the speckle image set, generate a training sample set based on the random marks; train a neural network model based on the training sample set to obtain a speckle area recognition model.

[0058] S2. Obtain the original image and the deformed image of the surface to be measured; identify the speckle area of the original image based on the speckle area recognition model.

[0059] S3. Establish a reference sub-region centered on each speckle in the speckle area of the original image, search for the corresponding deformed sub-region of the reference sub-region in the deformed image through the sub-pixel search algorithm, and obtain the displacement of the speckle based on the reference sub-region and the deformed sub-region; the displacements of all speckles constitute the displacement field of the deformed image.

[0060] S4. Calculate the strain field of the surface to be measured through the in-plane strain calculation method based on the displacement field of the deformed image.

[0061] By pre-training a neural network model, a speckle area recognition model is obtained. When strain measurement of the surface to be measured is required, only the obtained original speckle image needs to be input into the model to accurately identify the speckle area, so as to perform fully automatic strain measurement, avoiding the problems of large workload of manual speckle area calibration and easy calibration errors, resulting in inaccurate strain measurement.

[0062] During implementation, the speckle images used to construct the training samples should be speckle images collected from different-shaped measured targets under different lighting conditions, ensuring that each image contains a target area with clear speckles and the background environment.

[0063] Specifically, in step S1, each speckle image in the speckle image set is randomly marked, and a training sample set is generated based on the random marking, using the following method:

[0064] Randomly mark the speckle area and the background area of the speckle image with paintbrushes of different colors respectively to generate a marked image;

[0065] Traverse all the marked points in the marked image. If the color of the marked point is the color corresponding to the speckle area, crop an S×S neighborhood image centered on the coordinates of the marked point in the speckle image as a speckle sample; if the color of the marked point is the color corresponding to the background area, crop an S×S neighborhood image centered on the coordinates of the marked point in the speckle image as a background sample, where S represents the training sample size. During implementation, the sample size can be set according to actual needs. For example, it can be 81×81 pixel points.

[0066] During implementation, for one of the speckle images I, randomly mark the speckle area and the background area of the image I using red and blue brushes respectively to generate a marked image, and denote the marked image as image J. For example, random scribbling can be performed. Use image processing to extract the set of red pixel positions (denoted as R) and the set of blue pixel positions (denoted as B) in image J. In image I, respectively, with the positions of each pixel in set R as the center, crop images with a neighborhood size of 81×81 to form a positive training sample set. Similarly, in image I, respectively, with the positions of each pixel in set B as the center, crop images with a neighborhood size of 81×81 to form a negative training sample set.

[0067] Sequentially perform the above-mentioned process of first marking and then extracting on other speckle images, respectively generating corresponding positive and negative training sample sets, thereby constituting a training sample set.

[0068] Add different types and different parameters of noise to the extracted positive and negative sample sets, such as: salt-and-pepper noise, Gaussian noise, and Poisson noise, etc., to expand the scale of the sample set, achieving the goal of obtaining a larger-scale training sample set with fewer speckle images. At the same time, enable the trained speckle area recognition model to more accurately recognize the speckle area.

[0069] The traditional method of generating a sample set by first generating and then marking is to use a sliding window to traverse the entire image and mark each cropped regional image one by one, which will bring a large amount of work and more labor costs. For example: for a test image of five million pixels (resolution 2592*1944), if a sliding window of size 71*71 is used for traversal, nearly a thousand regional images need to be marked. When the resolution of the test image is higher or the number is larger, it will also lead to a doubling growth of the marking workload. By using the method of first marking and then extracting in the embodiments of the present invention, a training sample set can be quickly generated, greatly improving the speed of training sample generation.

[0070] Train the network with different training sample set scales, positive and negative training sample set balance ratios, network depth structures, network width structures, learning rates, and batch sizes to obtain the influence rules of the training set, network structure, and training parameters on the network training results. Based on this rule, conduct targeted optimization guidance for the network to improve the network training efficiency and recognition accuracy. The influence rules are as follows:

[0071] (1) More diverse and larger quantities of training sample data are beneficial to network training. When the balance ratio of the positive and negative sample sets is 1:1, the network recognition accuracy is the highest.

[0072] (2) Obtain a deeper network structure by using smaller convolutional kernels, which can improve the network's recognition accuracy, but an overly deep network will instead lead to a decline in recognition accuracy.

[0073] (3) Increasing the network width can extract more information on feature combinations. However, an overly wide network will increase the computational cost and reduce the network recognition efficiency.

[0074] (4) Appropriately increasing the batch size parameter can increase the memory utilization rate, reduce the training time within a single cycle, and improve the training efficiency.

[0075] (5) Appropriately reducing the learning rate can improve the network training accuracy. However, an overly small learning rate is likely to cause the network to fall into a local optimum.

[0076] During implementation, 10,000 positive and negative training sample sets are selected respectively; the ReLU form is selected as the activation function; the batch size is set to 256; the learning rate is set to 0.0002, and the training cycle is 100.

[0077] Through research on the network model and based on the above optimization principles, a network model applicable to speckle region recognition is obtained.

[0078] Specifically, the trained speckle region recognition model includes 3 convolutional layers, 3 pooling layers, and 2 fully connected layers;

[0079] The first convolutional layer is used to extract features from the input speckle image; the first convolutional layer contains 20 7×7 convolutional kernels;

[0080] The first pooling layer is used to downsample the feature map output by the first local convolutional layer; the first pooling layer contains 20 2×2 convolutional kernels;

[0081] The second convolutional layer is used to extract features from the downsampled image output by the first pooling layer; the second convolutional layer contains 50 7×7 convolutional kernels;

[0082] The second pooling layer is used to downsample the feature map output by the second convolutional layer; the second pooling layer contains 50 2×2 convolutional kernels;

[0083] The third convolutional layer is used to extract features from the downsampled image output by the second pooling layer; the third convolutional layer contains 100 7×7 convolutional kernels;

[0084] The third pooling layer is used to downsample the feature map output by the third convolutional layer; the third pooling layer contains 100 2×2 convolutional kernels;

[0085] The first fully connected layer is used to fully connect the feature map output by the third pooling layer into a one-dimensional feature map; the first fully connected layer contains 200 4×4 convolutional kernels;

[0086] The second fully connected layer is used to integrate the feature map output by the first fully connected layer, contains 100 neurons, and is fully connected to the first fully connected layer.

[0087] As shown Figure 3 in the figure, layer C1 is the first convolutional layer, which is used to extract features from the input speckle image. The first convolutional layer contains 20 convolutional kernels of 7×7. That is, through the convolution operation, layer C1 obtains 20 feature maps, and each neuron in the feature map is connected to 7×7 neighboring neurons in the input layer. During implementation, for an image with a size of 81×81, ignoring the influence of the extended boundary, the size of the feature map obtained by layer C1 is 75×75.

[0088] Layer S2 is the first pooling layer, which is used to downsample the feature map output by the first local convolutional layer. Layer S2 contains 20 convolutional kernels of 2×2. The convolutional kernels slide along the horizontal and vertical directions with a stride of 2 pixels. Therefore, the size of the feature map of layer S2 is 1 / 4 of the input image size. After the convolution operation, the first pooling layer contains the same number of feature maps as the output of layer C1, and each neuron in each feature map is connected to 2×2 neighboring neurons in layer C1. Each neuron calculates the average value of the four inputs, multiplies it by a training parameter, and then adds a training bias to input into the activation function to calculate the activation value of the current neuron. Thus, the first extraction of the features of the input image is completed.

[0089] Layer C3 is the second convolutional layer, which contains 50 convolutional kernels of 7×7. That is, it contains 50 feature maps, and each neuron is connected to the 7×7 neighborhood of the feature map of layer S2. Through the convolution operation on the feature map, descriptions of complex features under different combinations can be obtained, which helps to extract the abstraction of features.

[0090] Layer S4 is the second pooling layer, which contains 50 convolutional kernels of 2×2. That is, through the convolution operation, 50 feature maps are obtained, and the size of each feature map is 15×15. Each neuron in each feature map is connected to the 2×2 neighborhood of the corresponding feature map in layer C3.

[0091] Layer C5 is the third convolutional layer, which contains 100 convolutional kernels of 7×7. Through the convolution operation, 100 feature maps are obtained, and each neuron in the feature map is connected to 7×7 neighboring neurons in the previous layer. The size of the feature map obtained by layer C5 is 9×9.

[0092] Layer S6 is the third pooling layer, which contains 100 convolutional kernels of 2×2. Through the convolution operation, 100 feature maps are obtained, and the size is 4×4. Each neuron in each feature map is connected to the 2×2 neighborhood of the corresponding feature map in layer C5.

[0093] Layer C7 is the first fully connected layer, which contains 200 convolution kernels of 4×4. After convolution operation, 200 feature maps are obtained. The size of the feature maps in layer C7 is 1×1. Each neuron is connected to all neurons in the feature maps of layer S6 through a 4×4 convolution kernel, that is, in a fully connected manner. Through the convolution processing of the network from layer C1 to layer C7, the original 81×81 image matrix of the input is gradually reduced in size to a 1×1 vector, while increasing the depth to 200 layers to retain the feature information.

[0094] Layer F8 is the second fully connected layer, which contains 100 neurons. Each neuron is connected to all neurons in layer C7. As a feature extractor, layer F8 calculates the dot product between the input vector and the weights, adds a bias parameter, and judges the state of the current neuron through an activation function for the classification of the output layer.

[0095] The final output layer uses the Euclidean radial basis function to calculate the Euclidean distance between the output value of each neuron and the parameter vector to judge the classification. For the speckle image classification, when the output value is positive, it indicates that the image in the input layer is a speckle area, otherwise it is judged as the background area.

[0096] After obtaining the original image and the deformed image of the measured surface, the trained speckle area recognition model can be used to directly recognize the speckle area of the original image.

[0097] Preferably, step S2 further includes, in the original image, traversing each pixel point starting from (S / 2 + 1, S / 2 + 1) and ending at (H - S / 2, W - S / 2), cropping an S×S neighborhood image centered on the pixel point as the recognition window, and inputting the recognition window into the speckle area recognition model to judge whether the current center point is a speckle area. If so, mark the pixel point as a speckle point; where H represents the height of the original image, W represents the width of the original image, and S represents the training sample size.

[0098] For example, taking (41, 41) as the starting pixel and (H - 40, W - 40) as the ending pixel, creating an 81×81 recognition window centered on each pixel in this interval, inputting the recognition window into the trained network for classification and recognition, and using its classification result as the basis for judging whether the current center pixel is a speckle area. If it is recognized as a speckle area, mark the center point as red; if it is recognized as the background area, do not perform any processing. Repeat this process until the traversal of each pixel point in the above interval is completed to obtain the speckle area of the original image of the surface to be measured.

[0099] Due to the randomness of the speckle distribution, according to the relevant principles of statistics, the distribution form of any sub-region containing enough pixels in the image is unique. Therefore, an image sub-region can be created centered on any pixel point to characterize the motion and deformation of that pixel point.

[0100] For example, taking the speckle point P(x, y) in the original image f as the calculation point, and the (2M + 1)×(2M + 1) neighborhood centered on point P as the reference sub-region, denoted as Ω f ; In the deformed image g, following a certain search strategy, find the deformed sub-region corresponding to Ω f , denoted as Ω g , and its center point is P'. Calculate the coordinate relationship between the calculation point P and P', so as to determine the displacement field of this pixel in the horizontal and vertical directions.

[0101] During the movement process, the reference sub-region will simultaneously change in position and shape, and it is impossible to directly represent Ωg with a rectangular neighborhood. Therefore, the first-order shape function is used to describe the mapping shape function from the reference sub-region to the deformed sub-region.

[0102] The first-order shape function is simple to calculate and easy to implement, and can well describe the actual deformation. Therefore, a specific embodiment of the present invention adopts the first-order shape function to represent the mapping shape function from the reference sub-region to the deformed sub-region.

[0103] The correlation function is an index to measure the similarity degree between the reference sub-region and the deformed sub-region. Since the standardized least square distance function subtracts the average gray value of the sub-region and uses normalization processing, it has good anti-noise performance and insensitivity to light intensity. And during the deformation process of the measured object, it is prone to phenomena such as overexposure and uneven illumination affected by the light source. Therefore, the standardized least square distance function correlation function is used as the objective function. When the objective function is the smallest, it is considered that the reference sub-region and the deformed sub-region are the most similar.

[0104] The standardized least square distance function is expressed as

[0105]

[0106] where, (x', y') represents the center point coordinates of the deformed sub-region, f(x, y) and g(x′, y′) are the gray values of the point (x, y) in the reference image and the deformed image respectively, M is the sub-region window radius, f m and g m are the average gray values of the reference sub-region and the deformed sub-region respectively, is a parameter to be determined,

[0107] In the deformed image, the deformed sub-region that makes the correlation function reach the extreme value is obtained through the search algorithm, and this process is called correlation search.

[0108] Since the reverse combination matching strategy has a high calculation accuracy, is less affected by the sub-region size, and has good robustness and stability, preferably, in an embodiment of the present invention, the reverse combination matching strategy is adopted to search for the most similar deformed sub-region to the reference sub-region in the deformed image g. Alternatively, the Newton iteration algorithm can also be used for relevant search.

[0109] The displacements of the scatter spots (x, y) are expressed as Δx = x' - x and Δy = y' - y.

[0110] The displacements of all scatter spots are obtained to form the displacement field of the deformed image.

[0111] The strain field of the measured surface is calculated by the in-plane strain calculation method. Specifically, with each point in the displacement field as the center, a sub-region with a size of (2M + 1) × (2M + 1) is created, which is called a strain sub-region; the displacement field in the strain sub-region is subjected to surface fitting, and then the strain is obtained by calculating the difference of each point according to the fitting parameters.

[0112] The second-order polynomial is adopted to perform surface fitting on the displacement field in the strain sub-region; where (x, y) are the coordinates of the points in the strain sub-region in the reference image, u(x, y) and v(x, y) respectively represent the displacements of the point (x, y) in the horizontal and vertical directions, and u0, u1, u2, u3, u4, u5, v0, v1, v2, v3, v4, v5 are all fitting parameters.

[0113] The displacement derivatives in each direction are:

[0114]

[0115]

[0116] The least squares method is used to solve the fitting parameters of the second-order polynomial, and two types of strain components, namely Cauchy strain components and Green strain components, of each point in the displacement field are obtained based on the second-order polynomial.

[0117] Specifically, the Cauchy strain components can be further obtained as:

[0118]

[0119]

[0120] Meanwhile, the Green strain components can be calculated as:

[0121]

[0122]

[0123]

[0124] The strains of all points in the displacement field constitute the strain field of the surface to be measured.

[0125] The strain field of the deformed image can be obtained by directly differentiating the displacement field. However, during image acquisition, it is inevitable to be affected by noise, and direct differentiation will amplify the noise, resulting in unstable calculation results. Therefore, after obtaining the displacement field of the deformed image, before calculating the strain field of the surface to be measured based on the displacement field of the deformed image through the in-plane strain calculation method, it further includes smoothing the displacement field using a fast Gaussian low-pass filter.

[0126] Specifically, first, the displacement field is smoothed using a fast Gaussian low-pass filter to achieve the purpose of noise reduction. In a uniformly deformed displacement field, the displacement components of each point should be exactly the same. After research, it is found that the displacement field distribution obtained directly after correlation search is uneven and fluctuates greatly. It has been significantly improved after being smoothed by the fast Gaussian filter. At the same time, there is still a certain degree of unevenness in the displacement field distribution smoothed by a smaller convolution kernel. As the size of the convolution kernel increases, the displacement field gradually becomes smoother, indicating that increasing the size of the convolution kernel can better weaken the noise interference. However, an overly large convolution kernel size will cause over-smoothing linearity. Since it is a uniform deformation, the gradient of any two adjacent columns should be exactly the same, that is, the gradient curve should remain stable, and the displacement variance of each column should be 0. Therefore, through research, it is found that when using a 5×5 Gaussian convolution kernel, the displacement field can be kept smooth while the displacement variance of each column is minimized.

[0127] A specific embodiment of the present invention discloses a strain measurement system based on digital speckle, as Figure 2 shown, including the following modules:

[0128] A speckle region recognition model generation module, configured to obtain a speckle image set for training, randomly mark each speckle image in the speckle image set, generate a training sample set based on the random marking; train a neural network model based on the training sample set to obtain a speckle region recognition model;

[0129] A speckle region recognition module, configured to obtain the original image and the deformed image of the surface to be measured; recognize the speckle region of the original image based on the speckle region recognition model;

[0130] A displacement field calculation module, configured to establish a reference sub-region centered on each speckle in the speckle region of the original image, search for the corresponding deformed sub-region of the reference sub-region in the deformed image through a sub-pixel search algorithm, and obtain the displacement of the speckle based on the reference sub-region and the deformed sub-region; the displacements of all speckles constitute the displacement field of the deformed image;

[0131] A strain field calculation module, configured to calculate the strain field of the surface to be measured based on the displacement field of the deformed image through an in-plane strain calculation method.

[0132] Preferably, the speckle region recognition model generation module includes:

[0133] A random marking module, configured to randomly mark the speckle region and the background region of the speckle image with paintbrushes of different colors respectively to generate a marked image;

[0134] A sample generation module, configured to traverse all the marked points in the marked image. If the color of the marked point corresponds to the color of the speckle region, a neighborhood image of S×S is cropped with the coordinate of the marked point as the center in the speckle image as a speckle sample; if the color of the marked point corresponds to the color of the background region, a neighborhood image of S×S is cropped with the coordinate of the marked point as the center in the speckle image as a background sample, where S represents the size of the training sample.

[0135] Preferably, after obtaining the displacement field of the deformed image and before calculating the strain field of the surface to be measured based on the displacement field of the deformed image through an in-plane strain calculation method, it further includes smoothing the displacement field by using a fast Gaussian low-pass filter.

[0136] The above method embodiments and system embodiments are based on the same principle, and their related parts can be learned from each other and can achieve the same technical effects. For the specific implementation process, refer to the foregoing embodiments and will not be elaborated here.

[0137] Those skilled in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. Among them, the computer-readable storage medium is a disk, an optical disc, a read-only memory or a random access memory, etc.

[0138] The above is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention.

Claims

1. A strain measurement method based on digital speckle, characterized in that, It includes the following steps: Obtain a set of speckle images for training, randomly label each speckle image in the set of speckle images, and generate a training sample set based on the random labels; Train a neural network model based on the training sample set to obtain a speckle region recognition model; Obtain the original image and the deformed image of the surface to be measured; identify the speckle region of the original image based on the speckle region recognition model; Establish a reference sub-region centered on each speckle in the speckle region of the original image, search for the corresponding deformed sub-region of the reference sub-region in the deformed image through a sub-pixel search algorithm, and obtain the displacement of the speckle based on the reference sub-region and the deformed sub-region; the displacements of all speckles constitute the displacement field of the deformed image; Based on the displacement field of the deformed image, calculate the strain field of the surface to be measured through an in-plane strain calculation method; Establish a reference sub-region centered on each speckle in the speckle region of the original image, search for the corresponding deformed sub-region of the reference sub-region in the deformed image through a sub-pixel search algorithm, and obtain the displacement of the speckle based on the reference sub-region and the deformed sub-region, including: In the original image f, construct a reference sub-region with a neighborhood of (2M + 1)×(2M + 1) centered on each speckle (x, y); Taking as the shape function and using the normalized least square distance function as the objective function; Search for the deformed sub-region most similar to the reference sub-region in the deformed image g through an inverse combination matching strategy; The displacements of the scattered spots (x, y) are expressed as Δx = x' - x and Δy = y' - y, where (x', y') represents the coordinates of the center point of the deformed sub-region, f(x, y) and g(x', y') are the gray values of the point (x, y) in the reference image and the deformed image respectively, M is the radius of the sub-region window, and u and v are the displacements of the point (x, y) in the X direction and the Y direction respectively; f m and g m are the average gray values of the reference sub-region and the deformed sub-region respectively, and the parameters to be determined Based on the displacement field of the deformed image, calculate the strain field of the surface to be measured through an in-plane strain calculation method, including: Construct a strain sub-region centered on each point in the displacement field; Using a second-order polynomial Perform surface fitting on the displacement field within the strain sub-region; where (x, y) are the coordinates of the points within the strain sub-region in the reference image, u(x, y) and v(x, y) respectively represent the displacements of the point (x, y) in the horizontal and vertical directions, and u0, u1, u2, u3, u4, u5, v0, v1, v2, v3, v4, v5 are all fitting parameters; Solve the fitting parameters of the second-order polynomial by the least squares method, and obtain the strain of each point in the displacement field based on the second-order polynomial; The strains of all points in the displacement field constitute the strain field of the surface to be measured.

2. The strain measurement method based on digital speckle according to claim 1, wherein Randomly label each speckle image in the set of speckle images, and generate a training sample set based on the random labels, including: Randomly label the speckle region and the background region of the speckle image with paintbrushes of different colors respectively to generate a labeled image; Traverse all the labeled points in the labeled image. If the color of the labeled point is the color corresponding to the speckle region, crop the neighborhood image of S×S centered on the coordinate of the labeled point in the speckle image as a speckle sample; if the color of the labeled point is the color corresponding to the background region, crop the neighborhood image of S×S centered on the coordinate of the labeled point in the speckle image as a background sample, where S represents the size of the training sample.

3. A strain measurement method based on digital speckle according to claim 1, characterized in that Identify the speckle region of the original image based on the speckle region recognition model, including: In the original image, traverse each pixel point starting from (S / 2 + 1, S / 2 + 1) and ending at (H - S / 2, W - S / 2), crop the neighborhood image of S×S centered on the pixel point as an identification window, and input the identification window into the speckle region recognition model to determine whether the current center point is a speckle region. If so, label the pixel point as a speckle; where H represents the height of the original image, W represents the width of the original image, and S represents the size of the training sample.

4. The strain measurement method based on digital speckle according to claim 1, wherein After obtaining the displacement field of the deformed image, before calculating the strain field of the measured surface through an in-plane strain calculation method based on the displacement field of the deformed image, it further includes smoothing the displacement field using a fast Gaussian low-pass filter.

5. The strain measurement method based on digital speckle according to claim 1, wherein The speckle region recognition model includes: A first convolutional layer for extracting features from the input speckle image, including 20 7×7 convolutional kernels; A first pooling layer for downsampling the feature map output by the first local convolutional layer, including 20 2×2 convolutional kernels; A second convolutional layer for extracting features from the downsampled image output by the first pooling layer, including 50 7×7 convolutional kernels; A second pooling layer for downsampling the feature map output by the second convolutional layer, including 50 2×2 convolutional kernels; A third convolutional layer for extracting features from the downsampled image output by the second pooling layer, including 100 7×7 convolutional kernels; A third pooling layer for downsampling the feature map output by the third convolutional layer, including 100 2×2 convolutional kernels; A first fully connected layer for fully connecting the feature map output by the third pooling layer into a one-dimensional feature map, including 200 4×4 convolutional kernels; A second fully connected layer for integrating the feature map output by the first fully connected layer, including 100 neurons, and fully connected to the first fully connected layer.

6. A strain measurement system based on digital speckle, characterized in that, It includes the following modules: A speckle region recognition model generation module for obtaining a set of speckle images for training, randomly marking each speckle image in the set of speckle images, and generating a training sample set based on the random marking; Training a neural network model based on the training sample set to obtain a speckle region recognition model; A speckle region recognition module for obtaining the original image and the deformed image of the measured surface; identifying the speckle region of the original image based on the speckle region recognition model; A displacement field calculation module for establishing a reference sub-region centered on each speckle point in the speckle region of the original image, searching for the corresponding deformed sub-region of the reference sub-region in the deformed image through a sub-pixel search algorithm, and obtaining the displacement of the speckle point based on the reference sub-region and the deformed sub-region; the displacements of all speckle points constitute the displacement field of the deformed image; A strain field calculation module for calculating the strain field of the measured surface through an in-plane strain calculation method based on the displacement field of the deformed image; Establishing a reference sub-region centered on each speckle point in the speckle region of the original image, searching for the corresponding deformed sub-region of the reference sub-region in the deformed image through a sub-pixel search algorithm, and obtaining the displacement of the speckle point based on the reference sub-region and the deformed sub-region, including: Constructing a reference sub-region with a neighborhood of (2M + 1)×(2M + 1) centered on each speckle point (x, y) in the original image f; Using as the shape function and the normalized least square distance function as the objective function; Searching for the most similar deformed sub-region to the reference sub-region in the deformed image g through an inverse combination matching strategy; The displacements of the scattered speckles (x, y) are expressed as Δx = x′ - x and Δy = y′ - y, where (x', y') represents the center point coordinates of the deformed sub-region, f(x, y) and g(x′, y′) are the gray values of the point (x, y) in the reference image and the deformed image respectively, M is the sub-region window radius, and u and v are the displacements of the point (x, y) in the X direction and the Y direction respectively; f m and g m are the average gray values of the reference sub-region and the deformed sub-region respectively, and the parameters to be determined Calculating the strain field of the measured surface through an in-plane strain calculation method based on the displacement field of the deformed image, including: Constructing a strain sub-region centered on each point in the displacement field; Adopt a second-order polynomial Perform surface fitting on the displacement field in the strain sub-region; where (x, y) are the coordinates of the points in the strain sub-region in the reference image, u(x, y) and v(x, y) respectively represent the displacements of the point (x, y) in the horizontal and vertical directions, and u0, u1, u2, u3, u4, u5, v0, v1, v2, v3, v4, v5 are all fitting parameters; The fitting parameters of the second-order polynomial are solved by the least squares method, and the strain of each point in the displacement field is obtained based on the second-order polynomial; The strains of all points in the displacement field constitute the strain field of the measured surface.

7. The strain measurement system based on digital speckle according to claim 6, wherein, The speckle region recognition model generation module includes: A random marking module, which is used to randomly mark the speckle region and the background region of the speckle image with paintbrushes of different colors respectively to generate a marked image; A sample generation module, which is used to traverse all the marked points in the marked image. If the color of the marked point is the color corresponding to the speckle region, a neighborhood image of S×S is cropped with the coordinates of the marked point as the center in the speckle image as a speckle sample; if the color of the marked point is the color corresponding to the background region, a neighborhood image of S×S is cropped with the coordinates of the marked point as the center in the speckle image as a background sample, where S represents the size of the training sample.

8. The strain measurement system based on digital speckle according to claim 6, characterized in that, After obtaining the displacement field of the deformed image and before calculating the strain field of the measured surface by the in-plane strain calculation method based on the displacement field of the deformed image, it further includes smoothing the displacement field by using a fast Gaussian low-pass filter.

Images