A full-field displacement online recognition method based on multi-scale deep convolutional neural network

Abstract

This invention provides a method for online full-field displacement identification based on a multi-scale deep convolutional neural network. The method includes the following steps: 1. Establishing a speckle image-displacement dataset and identifying the displacement corresponding to the speckle image; 2. Based on the displacement, the speckle image is divided into datasets of different scales and the speckle pattern is cropped to the corresponding size; and 3. Constructing a multi-scale convolutional neural network based on deep learning automatically determines the scale of the input speckle pattern and outputs the corresponding displacement. The proposed multi-scale neural network achieves accurate displacement field prediction by fusing displacement information learned at multiple single scales. The effectiveness of the proposed method is verified through designed experiments, which show that the proposed method for online full-field displacement identification can provide accurate displacement prediction results.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-scale deep convolutional neural networks, and in particular to a full-field displacement online recognition method based on a multi-scale deep convolutional neural network. Background Art

[0002] The displacement field is one of the most intuitive physical quantities reflecting the current load state of a physical entity. It can be used to characterize the stiffness of materials and assess the damage state of key structural components. Existing displacement field measurement methods primarily rely on digital image correlation (DIC), which is widely used for surface displacement measurement due to its simplicity and low environmental requirements. This method involves tracking (or matching) identical points between two images. Each tracking process involves optimization calculations. Therefore, when the identified spot image area is large, the displacement field calculation is very time-consuming, making it difficult for the DIC method to achieve online, real-time prediction of the displacement field. In recent years, artificial intelligence technologies, represented by deep learning methods, have developed rapidly and have been successfully applied to fields such as face recognition, semantic segmentation, and autonomous driving. To achieve online displacement field prediction, a small number of studies have constructed a 3D deep convolutional neural network (CNN) based on deep learning to predict the displacement of spot images. However, because the CNN model is trained using only a single-scale subset of images, the prediction accuracy of this method is insufficient and is only applicable to small deformations. Summary of the Invention

[0003] In order to solve this problem, the present invention proposes a full-field displacement online recognition method based on a multi-scale neural network model that achieves accurate prediction of the displacement field by fusing displacement information learned at multiple single scales.

[0004] A full-field displacement online recognition method based on a multi-scale deep convolutional neural network is provided, characterized in that the full-field displacement online recognition method comprises the following steps:

[0005] Step 1: Establish a speckle image-displacement dataset and identify the displacement corresponding to the speckle image;

[0006] Step 2: Divide the speckle image into data sets of different scales according to its displacement, and crop the speckle pattern to the corresponding size;

[0007] Step 3: Construct a multi-scale convolutional neural network based on deep learning to automatically determine the scale of the input speckle pattern and output the corresponding displacement;

[0008] The multi-scale convolutional neural network includes a feature extraction module and a multi-scale decision fusion module;

[0009] The feature extraction module includes a small-scale CNN network, a medium-scale CNN network, a large-scale CNN network and a full-scale CNN network, which output a small-scale feature vector, a medium-scale feature vector, a large-scale feature vector and a full-scale feature vector respectively;

[0010] The multi-scale decision fusion module generates a coefficient matrix from the eigenvectors of four different scales, and generates an output vector by fusing the coefficient matrix with the small-scale eigenvector, the medium-scale eigenvector and the large-scale eigenvector.

[0011] Furthermore, the small-scale CNN network includes a first small-scale convolution module, an average pooling layer, a second small-scale convolution module, a third small-scale convolution module, a fourth small-scale convolution module, a fifth small-scale convolution module and a sixth small-scale convolution module connected in sequence;

[0012] The mesoscale CNN network includes a first mesoscale convolution module, an average pooling layer, a second mesoscale convolution module, a third mesoscale convolution module, a fourth mesoscale convolution module, a fifth mesoscale convolution module and a sixth mesoscale convolution module connected in sequence;

[0013] The large-scale CNN network includes a first large-scale convolution module, an average pooling layer, a second large-scale convolution module, a third large-scale convolution module, a fourth large-scale convolution module, a fifth large-scale convolution module, a sixth large-scale convolution module and a seventh large-scale convolution module connected in sequence;

[0014] The full-scale CNN network includes a first full-scale convolution module, an average pooling layer, a second full-scale convolution module, a third full-scale convolution module, a fourth full-scale convolution module, a fifth full-scale convolution module, a sixth full-scale convolution module and a seventh full-scale convolution module connected in sequence;

[0015] Each convolution module includes convolution kernels, normalization units and ReLU function units that are connected in sequence with different numbers and sizes.

[0016] Furthermore, in step 3, in the multi-scale fusion module, the output data of the four different scales are superimposed together and then enter the fully connected layer to generate a 6×1 vector. This vector is reshaped and then passed through a SoftMax layer to generate a 3×2 coefficient matrix.

[0017] In this coefficient matrix, the following equation is satisfied:

[0018]

[0019] Among them, c j and d jRepresents the elements of each scale coefficient matrix, and all are in the range of [0,1]. Finally, the displacement vector corresponding to the input subset image can be obtained as:

[0020]

[0021] Among them, u1 is the abscissa of the small-scale eigenvector, v1 is the ordinate of the small-scale eigenvector, u2 is the abscissa of the medium-scale eigenvector, v2 is the ordinate of the medium-scale eigenvector, u3 is the abscissa of the large-scale eigenvector, and v3 is the ordinate of the large-scale eigenvector.

[0022] Furthermore, step 3 also includes a phased training method for the displacement field multi-scale convolutional neural network:

[0023] Each single-scale network is trained separately, and the parameters after training and verification are frozen; then the multi-scale fusion module is trained, and only the weight parameters of the multi-scale fusion module are updated.

[0024] Furthermore, in step 3, the mean squared error (MSE) of the deviation between the model's predicted values and the actual values is defined as:

[0025]

[0026] Where n is the number of data used in each iteration step, is the predicted displacement of the i-th input speckle image, is the actual displacement of the i-th input speckle image, i = 1, 2, …, n;

[0027] Based on the training dataset, the optimal values of the model parameters including weights W and b are learned by minimizing the loss value:

[0028]

[0029] When the weight W and bias b are obtained, the fusion coefficient c i and d i It can be calculated through the decision layer fusion module.

[0030] The beneficial effects achieved by the present invention are:

[0031] This patent proposes a full-field displacement automatic identification method and intelligent agent based on speckle images and deep learning, which can realize the online identification of displacement fields. In the proposed new method, a multi-scale neural network model adapted to the displacement field is developed. By fusing the displacement information learned at multiple single scales, the displacement field can be accurately predicted. The effectiveness of the proposed method was verified by designing experiments. The results show that the proposed full-field displacement online identification method can provide accurate displacement prediction results. In addition, by integrating the algorithm into a mobile device, this patent has developed an intelligent agent that can automatically identify the displacement field through images.

[0032] The multi-scale convolutional neural network adaptively extracts features based on deformations of different scales, enabling the model to identify both small and large displacements. Furthermore, it achieves extremely high accuracy in displacement recognition at all scales, with a relative error of less than 2% across all data in the test set. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a flowchart of an online full-field displacement recognition method based on a multi-scale deep convolutional neural network;

[0034] Figure 2 This is a framework diagram of a multi-scale convolutional neural network model for displacement field recognition in a full-field displacement online recognition method based on a multi-scale deep convolutional neural network;

[0035] Figure 3 Schematic diagram of a multi-scale feature extraction module for displacement field speckle images in a full-field displacement online recognition method based on a multi-scale deep convolutional neural network;

[0036] Figure 4 Schematic diagram of a multi-scale decision-level fusion module of displacement field information in a full-field displacement online recognition method based on a multi-scale deep convolutional neural network;

[0037] Figure 5 Schematic diagram of the experimental scheme for multi-scale CNN training of displacement field in a full-field displacement online recognition method based on multi-scale deep convolutional neural network;

[0038] Figure 6 Schematic diagram of the displacement field recognition results based on a single-scale CNN in a full-field displacement online recognition method based on a multi-scale deep convolutional neural network;

[0039] Figure 7 A schematic diagram showing the comparison of displacement field recognition accuracy between single-scale CNN and multi-scale CNN in a full-field displacement online recognition method based on a multi-scale deep convolutional neural network.

[0040] Figure 8A schematic diagram of the displacement field recognition results based on multi-scale CNN in a full-field displacement online recognition method based on multi-scale deep convolutional neural network;

[0041] Figure 9 Schematic diagram of the assembly of the displacement field recognition intelligent agent device in a full-field displacement online recognition method based on a multi-scale deep convolutional neural network. DETAILED DESCRIPTION

[0042] The technical solutions of the present invention are described in more detail below with reference to the accompanying drawings. The present invention includes but is not limited to the following embodiments.

[0043] As attached Figure 1 As shown, the present invention proposes a full-field displacement online recognition method based on a multi-scale deep convolutional neural network, and the full-field displacement online recognition method includes the following steps:

[0044] Step 1: Establish a speckle image-displacement dataset and identify the displacement corresponding to the speckle image;

[0045] In step 1, a camera is used to capture multiple images of a sample with speckles sprayed on the surface of the axial / torsional test system, including a reference image and a deformed image. The reference image is the highlight of the sample before the test, and the deformed image is the image of the sample during the test.

[0046] A rectangular area is selected from the collected images as the analysis area, and each deformed image is superimposed with the original image to form an overlay image with speckle. These images are used to train and verify the CNN model, and as test data to verify the effectiveness of the CNN model. The ratio of training, verification and test data is 7:2:1.

[0047] Step 2: Divide the speckle image into data sets of different scales according to its displacement, and crop the speckle pattern to the corresponding size;

[0048] In step 2, a sliding sampling strategy is used to select keypoints from the stacked speckle images based on the overlay image, with a sliding window step size of 5 pixels. Then, subset images of different sizes are cropped, centered around the keypoints. Based on the size of the subset images, small-scale, medium-scale, and large-scale datasets are generated. The center points of the subset images of different scales are consistent, resulting in the same data volume for all three datasets. Furthermore, the image center and displacement labels for each dataset are identical.

[0049] Step 3 constructs a multi-scale convolutional neural network based on deep learning to automatically determine the scale of the input speckle pattern and output the corresponding displacement.

[0050] As attached Figure 2As shown in Figure 3, in step 3, the model consists of three parts: feature extraction module, single-scale output module and multi-scale fusion output module, and also includes the following steps:

[0051] Step 31, displacement field multi-scale convolutional neural network feature extraction:

[0052] Different from the traditional displacement field recognition method based on single-scale neural network, this patent divides the speckle pattern into large-scale, medium-scale and small-scale subset images with different pixels, and each scale is feature extracted through a deep convolutional layer.

[0053] During multi-scale neural network training, a 2×48×48 pixel subset of images with target values in the range [0, 8] served as input for the small-scale CNN, a 2×68×68 pixel subset of images with target values in the range [4, 22] served as input for the medium-scale CNN, and a 2×132×132 pixel subset of images with target values in the range [20, 54] served as input for the large-scale CNN. The training subsets at each scale had overlapping target values to improve the model's prediction accuracy when target values were near the cross-scale boundary. A 132×132 pixel subset of images with target values in the range [0, 54] served as input for the full-scale CNN.

[0054] As attached Figure 3 As shown in FIG, the network structure of the feature extraction module includes a small-scale CNN network, a medium-scale CNN network, a large-scale CNN network and a full-scale CNN network. Among them, the small-scale CNN network includes a first small-scale convolution module, an average pooling layer, a second small-scale convolution module, a third small-scale convolution module, a fourth small-scale convolution module, a fifth small-scale convolution module and a sixth small-scale convolution module connected in sequence; the medium-scale CNN network includes a first medium-scale convolution module, an average pooling layer, a second medium-scale convolution module, a third medium-scale convolution module, a fourth medium-scale convolution module, a fifth medium-scale convolution module and a sixth medium-scale convolution module connected in sequence; the large-scale CNN network includes a first large-scale convolution module, an average pooling layer, a second medium-scale convolution module, a third medium-scale convolution module, a fourth medium-scale convolution module, a fifth medium-scale convolution module and a sixth medium-scale convolution module connected in sequence. The full-scale CNN network includes the first full-scale convolution module, the average pooling layer, the second full-scale convolution module, the third full-scale convolution module, the fourth large-scale convolution module, the fifth large-scale convolution module, the sixth large-scale convolution module and the seventh large-scale convolution module connected in sequence; each convolution module includes convolution kernels of different numbers and sizes, normalization units and ReLU function units connected in sequence.

[0055] The small-scale CNN network outputs a small-scale feature vector, the medium-scale CNN network outputs a medium-scale feature vector, the large-scale CNN network outputs a large-scale feature vector, and the full-scale CNN network outputs a full-scale feature vector.

[0056] Step 32: Decision-level fusion of displacement field multi-scale convolutional neural network:

[0057] This decision fusion module automatically fuses displacement information from different scales using a designed neural network. Compared to traditional empirical fusion, this fusion method is more accurate and intelligent, thanks to neural network learning. After the feature extraction module, the extracted features are further passed through a fully connected layer to generate displacement results at each individual CNN scale. A multi-scale decision fusion module is then proposed to fuse these displacements learned at different scales.

[0058] like Figure 4 As shown in the figure, in the multi-scale fusion module, the output data of four different scales are superimposed together and then enter the fully connected layer to generate a 6×1 vector. This vector is reshaped and then passes through a SoftMax layer to generate a 3×2 coefficient matrix.

[0059] In this coefficient matrix, the following equation is satisfied:

[0060]

[0061] Among them, c j and d j Represents the elements of each scale coefficient matrix, and all are in the range of [0,1]. Finally, the displacement vector corresponding to the input subset image can be obtained as:

[0062]

[0063] Among them, u1 is the abscissa of the small-scale eigenvector, v1 is the ordinate of the small-scale eigenvector, u2 is the abscissa of the medium-scale eigenvector, v2 is the ordinate of the medium-scale eigenvector, u3 is the abscissa of the large-scale eigenvector, and v3 is the ordinate of the large-scale eigenvector.

[0064] The present invention uses four different scales of CNN, namely small-scale CNN, medium-scale CNN, large-scale CNN and full-scale CNN. The outputs of the small-scale network, medium-scale network and large-scale network are all accurate within their respective ranges, while the output accuracy of the full-scale network is not as high in all ranges, but it is sufficient to determine the scale to which the result belongs. Therefore, by fusing the results of these four networks, a suitable coefficient matrix can be calculated to determine the correct result. Therefore, although the result of the full-scale network does not participate in the fusion, it plays an important decision-making role.

[0065] Step 33: Phased training method of displacement field multi-scale convolutional neural network: In order to train the proposed multi-scale convolutional neural network model, this patent proposes a step-by-step training method. First, train Figure 2 The feature extraction module and single-scale output module in the model are trained separately, and the parameters after training and verification are frozen. Then the multi-scale fusion module is trained, in which only the weight parameters of the multi-scale fusion module are updated. In order to train the model, a loss function needs to be constructed to quantify the performance of the model. In this patent, the mean square error (MSE) of the deviation between the model's predicted value and the actual value is used to define:

[0066]

[0067] Where n is the number of data used in each iteration step, is the predicted displacement of the i-th input speckle image, is the actual displacement of the i-th input speckle image, i = 1, 2, ..., n. Based on the training dataset, the optimal values of the model parameters including weights W and b are learned by minimizing the loss value:

[0068]

[0069] When the weight W and bias b are obtained, the fusion coefficient c i and d i This can be calculated using the decision layer fusion module. To minimize the loss function, an algorithm that updates the model parameters is necessary to predict the true displacement. This process is considered CNN training. This patent proposes a new multi-scale CNN training strategy that uses the Adam optimization algorithm during model training. The initial learning rate is 0.01, and is divided by 10 after every 20 epochs in the optimizer.

[0070] In one embodiment, step 1, full-field displacement online recognition multi-scale convolutional neural network dataset is acquired. The dataset comes from Figure 5 The experiment shown is performed on an MTS809 axial / torsional testing system at a tensile rate of 1 mm / min. The specimen is made of 316L steel. The specimen surface is sprayed with speckle patterns and recorded using a camera (acA2440-75μm, Basler) at a frame rate of 5 Hz and a resolution of 4032 × 3024 pixels.

[0071] A total of 351 speckle images were collected, including one reference image and 350 deformed images. The specimen strain ranged from 0% to 13%. A 916×496-pixel rectangular region was selected from the original images as the analysis area. The deformed image and the reference image were superimposed to form 350 overlay images with speckle. Of these images, 315 were used to train and validate the CNN model, and 35 images were selected as test data to verify the effectiveness of the CNN model.

[0072] In step 2, based on the collected images, a single-scale CNN dataset was formed as follows: subset images were cropped from the stacked speckle images using a sliding sampling strategy. Based on the sliding window size, a small-scale dataset, a medium-scale dataset, and a large-scale dataset were generated, with sizes of 2 × 48 × 48, 2 × 68 × 68, and 2 × 132 × 132, respectively. It is important to note that the three datasets have exactly the same data volume, and the corresponding center values and target values are the same across the different datasets. Using the sliding sampling strategy, 703 sub-images were generated from each stacked image. Therefore, each of the three datasets contains 315 × 703 = 221,445 pairs of input subset images and target values, with the target values falling within the pixel range [0, 54]. These datasets were split into training and validation sets in a 7:2 ratio.

[0073] In step 3, the average pooling layer is 2*2 average pooling; the first small-scale convolution module is 15 7*7 convolution kernels with a step size of 1; the second small-scale convolution module is 15 3*3 convolution kernels with a step size of 1; the third small-scale convolution module is 20 3*3 convolution kernels with a step size of 2; the fourth small-scale convolution module is 60 3*3 convolution kernels with a step size of 1; the fifth small-scale convolution module is 60 3*3 convolution kernels with a step size of 1; the sixth small-scale convolution module is 120 3*3 convolution kernels with a step size of 2. convolution kernel; the first mesoscale convolution module is 15 7*7 convolution kernels with a step size of 1; the second mesoscale convolution module is 30 3*3 convolution kernels with a step size of 2; the third mesoscale convolution module is 30 3*3 convolution kernels with a step size of 1; the fourth mesoscale convolution module is 60 3*3 convolution kernels with a step size of 2; the fifth mesoscale convolution module is 120 3*3 convolution kernels with a step size of 2; the sixth mesoscale convolution module is 120 3*3 convolution kernels with a step size of 1; the first large-scale convolution module is 15 steps The first large-scale convolution module is composed of 7*7 convolution kernels with a step size of 1; the second large-scale convolution module is composed of 15 3*3 convolution kernels with a step size of 1; the third large-scale convolution module is composed of 30 3*3 convolution kernels with a step size of 2; the fourth large-scale convolution module is composed of 30 3*3 convolution kernels with a step size of 1; the fifth large-scale convolution module is composed of 60 3*3 convolution kernels with a step size of 2; the sixth large-scale convolution module is composed of 120 3*3 convolution kernels with a step size of 2; the seventh large-scale convolution module is composed of 240 3*3 convolution kernels with a step size of 2; the first full ... The first module consists of 15 7*7 convolution kernels with a step size of 1; the second full-scale convolution module consists of 15 3*3 convolution kernels with a step size of 1; the third full-scale convolution module consists of 30 3*3 convolution kernels with a step size of 2; the fourth full-scale convolution module consists of 30 3*3 convolution kernels with a step size of 1; the fifth full-scale convolution module consists of 60 3*3 convolution kernels with a step size of 2; the sixth full-scale convolution module consists of 120 3*3 convolution kernels with a step size of 2; and the seventh full-scale convolution module consists of 240 3*3 convolution kernels with a step size of 2.

[0074] When training the multi-scale CNN model, each single-scale CNN is first trained independently, followed by the multi-scale fusion module. To improve prediction accuracy, only a portion of the single-scale dataset mentioned above is used when training each single-scale CNN in the multi-scale model. In the multi-scale model, the small-scale sub-dataset consists of images with target values in the range [0, 8] from a single small-scale dataset; the medium-scale sub-dataset consists of images with target values in the range [4, 22] from a single medium-scale dataset. The large-scale sub-dataset consists of images with target values in the range [20, 54] from a single large-scale dataset. As a result, a total of 50,039 small-scale images, 146,236 medium-scale images, and 174,189 large-scale images are used to train the small-scale, medium-scale, and large-scale CNNs in the multi-scale model, respectively. Furthermore, the full-scale CNN is trained on the single-scale large-scale dataset mentioned above.

[0075] Multi-scale convolutional neural network training for online full-field displacement recognition. For the single-scale CNN, the model training process used a total of 172,235 images, with a mini-batch size of 8 per training iteration. During training, the training losses of the three single-scale CNNs converged to a small value after 30 epochs. A validation dataset (49,210 images) was used to verify the trained networks. The final training loss on the validation dataset was very similar to the loss obtained after 30 epochs of training. Therefore, the network trained with 30 epochs is considered a benchmark for predicting displacement in test images.

[0076] For the multi-scale CNN proposed in this patent, images from the sub-dataset are first used to train the feature extraction module in the multi-scale model. Subsequently, the parameters of the multi-scale fusion module are trained using three single-scale image sets. Finally, the training loss of the multi-scale network also converges to a small value after 30 epochs. Therefore, in subsequent tests, 30 epochs are used as a benchmark for predicting displacements in the test set.

[0077] The multi-scale convolutional neural network model for displacement field recognition was validated using a validation dataset. A test dataset was used to verify the effectiveness of the CNN trained above. In the single-scale CNN, in order to evaluate the performance of the model, the small, medium, and large single-scale CNN models predicted speckle images with 1%, 8%, and 13% strain, respectively. During the prediction process, the image was first cropped into small, medium, and large scale image subsets using a sliding sampling strategy. Then, the subset images were input into a single-scale CNN respectively, through which the displacement of the center of the subset image could be predicted. Finally, by integrating the displacements of all centers, the displacement field results predicted using CNNs of three different scales were obtained. Figure 6The results of three single-scale CNN predictions are shown. It can be found that single-scale CNN cannot effectively predict the displacement field of the specimen in the entire strain range.

[0078] The three single-scale CNNs were used to predict 15 speckle images with strains ranging from 1% to 13%. The prediction accuracy was defined as the proportion of predicted values that fell within 2% of the true value. Figure 7 The accuracy of three single-scale CNN models at different strains is shown. It can be seen that the small-scale CNN performs best at low strains, the medium-scale CNN performs best at intermediate strains, and the large-scale CNN performs best at high strains. Therefore, fusing multi-scale information is necessary.

[0079] To evaluate the performance of the proposed multi-scale CNN model, the displacement fields of speckle images at 1%, 8%, and 13% strains were predicted using the multi-scale CNN model. Figure 8 The figure shows the visualization results of the proposed multi-scale CNN model. During the multi-scale CNN prediction process, the input image is first cropped into three subset images of different sizes: 2×48×48, 2×68×68, and 2×132×132. For any point in the image, three subset images are generated, including small-scale, medium-scale, and large-scale subset images. These subset images are input into the multi-scale CNN model to predict the displacement of each point. Through sliding sampling techniques, the entire displacement field is obtained.

[0080] like Figure 8 As shown in Figure 2, it can be seen that the values and distribution of the displacement field predicted by the proposed multi-scale CNN model are exactly the same as the actual results, regardless of whether the strain is small or large. At 1%, 8%, and 13% strain, the average absolute error in the y direction is equal to 0.039, 0.121, and 0.209 pixels, respectively, and the average absolute error in the x direction is equal to 0.013, 0.031, and 0.02 pixels, respectively. The error values are very small. The prediction accuracy of the multi-scale CNN model under different strains is shown in Figure 2. Figure 7 The results show that the predicted values fall within 2% of the true values across all strain ranges. Compared with the single-scale model, the multi-scale model has significant advantages in both prediction accuracy and range.

[0081] Embed the trained model into Figure 9The multi-scale convolutional neural network displacement field online recognition agent shown in the figure consists of a Raspberry Pi 4B development board, an HQ Camera with a 35mm telephoto lens, a 7-inch capacitive touch screen, a housing, a Raspberry Pi power cord, a Type-C cable, and screws and nuts. The development board is placed inside the housing. The HQ Camera is mounted on the front of the housing. The camera is supplemented with a telephoto lens to facilitate focusing and shooting when the device is placed at different distances from the specimen. A 7-inch capacitive touch screen is installed on the rear of the housing for easy touch operation and viewing of crack test results. The Raspberry Pi power cord and Type-C cable interfaces are located on the left and right sides of the housing, respectively. The device starts up after connecting to the power supply. The device can be connected to a wired or wireless network and to a cloud server for faster calculations.

[0082] The invention is not limited to the above-mentioned specific embodiments. A person skilled in the art can implement the invention in a variety of other specific embodiments based on the embodiments and the contents disclosed in the drawings. Therefore, any design that adopts the design structure and ideas of the present invention and makes some simple transformations or changes falls within the scope of protection of the present invention.

Claims

1. A full-field displacement online recognition method based on a multi-scale deep convolutional neural network, characterized in that: The full-field displacement online identification method comprises the following steps: Step 1: Establish a speckle image-displacement dataset and identify the displacement corresponding to the speckle image; Step 2: Divide the speckle image into data sets of different scales according to its displacement, and crop the speckle pattern to the corresponding size; Step 3: Construct a multi-scale convolutional neural network based on deep learning to automatically determine the scale of the input speckle pattern and output the corresponding displacement; The multi-scale convolutional neural network includes a feature extraction module and a multi-scale decision fusion module; The feature extraction module includes a small-scale CNN network, a medium-scale CNN network, a large-scale CNN network and a full-scale CNN network, which output a small-scale feature vector, a medium-scale feature vector, a large-scale feature vector and a full-scale feature vector respectively; The multi-scale decision fusion module generates a coefficient matrix from the eigenvectors of four different scales, and generates an output vector by fusing the coefficient matrix with the small-scale eigenvector, the medium-scale eigenvector and the large-scale eigenvector.

2. The full-field displacement online identification method according to claim 1, characterized in that: The small-scale CNN network includes a first small-scale convolution module, an average pooling layer, a second small-scale convolution module, a third small-scale convolution module, a fourth small-scale convolution module, a fifth small-scale convolution module and a sixth small-scale convolution module connected in sequence; The mesoscale CNN network includes a first mesoscale convolution module, an average pooling layer, a second mesoscale convolution module, a third mesoscale convolution module, a fourth mesoscale convolution module, a fifth mesoscale convolution module and a sixth mesoscale convolution module connected in sequence; The large-scale CNN network includes a first large-scale convolution module, an average pooling layer, a second large-scale convolution module, a third large-scale convolution module, a fourth large-scale convolution module, a fifth large-scale convolution module, a sixth large-scale convolution module and a seventh large-scale convolution module connected in sequence; The full-scale CNN network includes a first full-scale convolution module, an average pooling layer, a second full-scale convolution module, a third full-scale convolution module, a fourth full-scale convolution module, a fifth full-scale convolution module, a sixth full-scale convolution module and a seventh full-scale convolution module connected in sequence; Each convolution module includes convolution kernels, normalization units and ReLU function units that are connected in sequence with different numbers and sizes.

3. The full-field displacement online identification method according to claim 1, characterized in that: In step 3, in the multi-scale fusion module, the output data of four different scales are superimposed together and then enter the fully connected layer to generate a 6×1 vector. This vector is reshaped and then passes through a SoftMax layer to generate a 3×2 coefficient matrix. In this coefficient matrix, the following equation is satisfied: Among them, c j and d j Represents the elements of each scale coefficient matrix, and all are in the range of [0,1]. Finally, the displacement vector corresponding to the input subset image can be obtained as: Among them, u1 is the abscissa of the small-scale eigenvector, v1 is the ordinate of the small-scale eigenvector, u2 is the abscissa of the medium-scale eigenvector, v2 is the ordinate of the medium-scale eigenvector, u3 is the abscissa of the large-scale eigenvector, and v3 is the ordinate of the large-scale eigenvector.

4. The full-field displacement online identification method according to claim 3, characterized in that: Step 3 also includes a phased training method for the displacement field multi-scale convolutional neural network: Each single-scale network is trained separately, and the parameters after training and verification are frozen; then the multi-scale fusion module is trained, and only the weight parameters of the multi-scale fusion module are updated.

5. The full-field displacement online identification method according to claim 4, characterized in that: In step 3, the mean squared error (MSE) of the deviation between the model's predicted values and the actual values is defined as: Where n is the number of data used in each iteration step, is the predicted displacement of the i-th input speckle image, is the actual displacement of the i-th input speckle image, i = 1, 2, …, n; Based on the training dataset, the optimal values of the model parameters including weights W and b are learned by minimizing the loss value: When the weight W and bias b are obtained, the fusion coefficient c j and d j It can be calculated through the decision layer fusion module.

Images