High-speed rotating object strain field measurement method based on deep learning DIC

Abstract

The invention relates to a high-speed rotating object strain field measurement method based on deep learning DIC, and the method comprises the steps: collecting a speckle image sequence based on a digital image method, carrying out the registration of a target image and a reference image, entering an improved DIC neural network model, and obtaining a strain field, thereby achieving the non-contact strain field measurement of a high-speed rotating object. In the improved DIC neural network model, texture features related to displacement in an initial displacement field are enhanced by using flow attention, defect spots in the displacement field can be eliminated, then global matching and displacement iteration refinement are performed to improve displacement prediction precision, and the robustness and accuracy of the improved DIC neural network model are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of non-contact measurement of digital images, and more particularly, to a method for measuring the strain field of a high-speed rotating object based on deep learning DIC. Background Art

[0002] In many engineering fields, such as aerospace and mechanical manufacturing, the measurement of the strain field of high-speed rotating objects is of crucial importance. Accurately obtaining the strain field information of high-speed rotating objects is of great significance for evaluating the structural strength, stability of the objects, and predicting their fatigue life, etc.

[0003] Currently, for the measurement of the object strain field, mainly contact measurement methods and non-contact measurement methods are used; existing technologies such as the patent document CN109883333A, "A non-contact displacement and strain measurement method based on image feature recognition technology", propose a non-contact displacement and strain measurement method based on image feature recognition and matching. The experimental equipment and process required by this method are relatively simple. It adopts a non-contact type of non-destructive measurement method, which will not damage the material properties and characteristics of the material, and the obtained data is more accurate. It can realize automatic data processing, and can obtain change values such as full-field displacement and the width of the material structure member, and has relatively low requirements for the environment. The patent document CN201852566U, "Non-contact optical strain gauge based on digital image", discloses a non-contact optical strain gauge based on digital image, which uses a common imaging lens and at least two image sensors are arranged on the image plane to achieve high-resolution strain measurement. The patent document CN113808029B, "A strain smoothing method in digital image correlation", first builds a digital image correlation measurement system to obtain digital images before and after the deformation of the specimen, and through a series of operations, the strain measurement accuracy of the digital image correlation method can be improved to a certain extent.

[0004] However, the contact measurement method has many limitations in high-speed rotating scenarios. On the one hand, the contact measurement is easily affected by the rotation speed, and it is difficult to guarantee the measurement accuracy. For example, when the blades of an aero-engine rotate at high speed, the contact measurement equipment may generate measurement errors due to high-speed vibration. On the other hand, the contact measurement may interfere with the movement of the object, changing the original force state and movement trajectory of the object. Moreover, it is usually very difficult to install contact measurement equipment on high-speed rotating objects, which requires complex installation processes and equipment, increasing the measurement cost and operation difficulty.

[0005] Non-contact measurement methods also have deficiencies when measuring the strain field of high-speed rotating objects. Some methods do not have clear enough requirements for the production of speckle patterns on the surface of high-speed rotating objects, resulting in the possible shedding or deformation of speckles during high-speed rotation, affecting the accuracy of measurement results. When facing high-speed rotating objects, the frame rate and resolution of some image acquisition systems cannot meet the shooting requirements and cannot clearly capture the state changes of the object at different times. In the link of image analysis and strain calculation, some algorithms have poor adaptability to the complex motion states of high-speed rotating objects, and the calculation accuracy and efficiency need to be improved.

[0006] In summary, the existing technologies have certain limitations in measuring the strain field of high-speed rotating objects, and there is an urgent need for a measurement method that can effectively overcome the above problems to meet the requirements of the engineering field for accurate measurement of the strain field of high-speed rotating objects. Summary of the Invention

[0007] The technical problem to be solved by the present invention is how to accurately measure the strain field of high-speed rotating objects.

[0008] The present invention provides a method for measuring the strain field of high-speed rotating objects based on deep learning DIC, including: Step 1, prepare a speckle pattern on the surface of the rotating component of the target object; Step 2, start rotating the target object, and use an industrial camera to take a series of consecutive speckle images of the selected area; randomly select two speckle images as the reference image and the target image and perform preprocessing; Step 3, register the target image and the reference image; Step 4, construct an improved DIC neural network model, and determine whether to perform segmentation processing on the registered target image and reference image. If so, enter Step 6; if not, enter Step 5; Step 5, the improved DIC neural network model performs feature extraction, feature enhancement, feature fusion and displacement iterative refinement on the registered target image and reference image to obtain a displacement field with the resolution of the original image, and calculates the strain field based on the displacement field with the resolution of the original image; Step 6, synchronously perform overlapping block processing on the registered target image and reference image to obtain a number of pairs of target image blocks and reference image blocks with overlapping regions. The improved DIC neural network model performs feature extraction, feature enhancement, feature fusion and displacement iterative refinement on the target image blocks and reference image blocks to obtain a number of displacement field blocks, splices and fuses the displacement field blocks based on the overlapping regions to obtain a displacement field with the resolution of the original image, and calculates the strain field based on the displacement field with the resolution of the original image.

[0009] Compared with the prior art, the present application has the following advantages: Based on the digital image method, the present application acquires a sequence of speckle images. After registering the target image and the reference image, it enters an improved DIC neural network model to obtain the strain field, realizing non-contact measurement of the strain field of a high-speed rotating object. First, the reference image and the target image are registered to eliminate the influence of rigid body displacement, making the prediction of non-rigid body displacement by the improved DIC neural network model more accurate, reducing noise interference caused by image misalignment, and thus more precisely iteratively optimizing the displacement field. Moreover, in the improved DIC neural network model, flow attention is used to strengthen the texture features related to displacement in the initial displacement field, eliminate the defective points in the displacement field, and then perform global matching and displacement iteration refinement to improve the displacement prediction accuracy and enhance the robustness and accuracy of the improved DIC neural network model.

[0010] In a possible implementation manner, the preprocessing of the reference image and the target image in step 2 specifically includes: Step 201, denoise the reference image and the target image by using a non-local means filtering algorithm; Step 202, improve the contrast between the speckles and the background in the reference image and the target image by using histogram equalization; Step 203, eliminate the artifacts caused by high-speed rotation in the reference image and the target image by means of digital image processing.

[0011] Compared with the prior art, adopting the above technical solution can improve the quality of the acquired speckle images and lay a foundation for the subsequent analysis of the improved DIC neural network model.

[0012] In a possible implementation manner, step 3 specifically includes: Step 301, respectively detect the matching feature points in the reference image and the target image by using the ORB detection algorithm, and respectively obtain the point set pts1 and the point set pts2; Step 302, based on the matching feature points in the point set pts1 and the point set pts2, solve the rotation center by using the least squares method ; Step 303, based on the rotation center , calculate the rotation angle by using the arctangent function ; Step 304, based on the rotation center and the rotation angle correct the rigid body displacement of the target image to register the target image and the reference image.

[0013] Compared with the prior art, the ORB detection algorithm can well handle rotation, scaling, and brightness changes, and accurately pick up the feature points in the reference image and the target image; so as to more accurately solve the rotation center and rotation angle, and thus better register the reference image and the target image.

[0014] In a possible implementation manner, the step 302 specifically includes: Step 302A, obtaining the perpendicular bisector equation through the matching feature points in the point set pts1 and the point set pts2: ; In the formula, represents the normal vector of the connection direction vector of the matching feature points, , represents the th feature point in the point set pts2, represents the th feature point in the point set pts1; , represents the y-axis coordinate of the connection direction vector , the x-axis coordinate of the connection direction vector ; represents the rotation center; represents the midpoint of the th pair of matching feature points, Step 302B, forming a linear system from the perpendicular bisector equations of all the matching feature points in the point set pts1 and the point set pts2, where represents the normal vector matrix, represents the constant term vector; the expression is: ; ; In the formula, , , represents the coordinates of the th feature point in the point set pts2, represents the coordinates of the th feature point in the point set pts1, ; Step 302C, using the least squares method to solve the rotation center , and the calculation formula is: ; In the formula, represents the normal vector matrix The transpose of

[0015] In a possible implementation, step 303 specifically includes: Step 303A, based on the rotation center Calculate the vector of each pair of matched feature points relative to the rotation center , ; Step 303B, use the arctangent function to calculate the rotation angle , and the calculation formula is: ; In the formula, and are the covariance terms between all matched feature points respectively, and the calculation formula is: ; .

[0016] In a possible implementation, the improved DIC neural network model in step 4 includes a feature extraction layer, a feature enhancement layer, a flow attention layer, a feature fusion module, a feature correlation volume, a displacement iterative refinement layer, and an output layer; step 5 specifically includes: Step 501, after the target image and the reference image are input into the feature extraction layer, first perform 1 / 2 downsampling to obtain 1 / 2-scale target features and 1 / 2-scale reference features, and then perform 1 / 4 downsampling to obtain 1 / 4-scale target features and 1 / 4-scale reference features; Step 502, perform feature enhancement operations on the 1 / 4-scale target features and 1 / 4-scale reference features in the feature enhancement layer; collect context features for the 1 / 2-scale target features and 1 / 2-scale reference features in the feature extraction layer; Step 503, perform global matching on the feature-enhanced 1 / 4-scale target features and 1 / 4-scale reference features in the flow attention layer to obtain a low-resolution initial displacement field; the initial displacement field is strengthened in the flow attention layer, which helps to eliminate the defective points in the displacement field and obtain the displacement field; Step 504, the feature-enhanced 1 / 4-scale target features and 1 / 4-scale reference features are upsampled and then fused with the 1 / 2-scale target features and 1 / 2-scale reference features in the feature fusion module, and correlation characteristics are obtained in the feature correlation volume; Step 505, the displacement field is gradually iteratively updated in the displacement iterative refinement layer in combination with the correlation features and context features, and the iteratively updated displacement field is subjected to convolutional upsampling operation in the output layer to output the displacement field at the original image resolution; Step 506: Based on the Green-Lagrange strain tensor formula, the original image resolution displacement field calculates the surface strain components of the original image resolution displacement field using the displacement gradient tensor , obtaining a strain field. represents the normal strain in the x direction. represents the normal strain in the y direction. represents the shear strain in the xy plane.

[0017] Compared with the prior art, the present application performs global matching on the 1 / 4-scale target features and 1 / 4-scale reference features to obtain a low-resolution initial displacement field; then, by upsampling the 1 / 4-scale target features and 1 / 4-scale reference features and fusing them with the 1 / 2-scale target features and 1 / 2-scale reference features, and then gradually refining and enhancing them in the displacement iteration refinement layer; performing global matching at low-resolution features and further performing iterative refinement at high-resolution features, the strategy avoids the limitations of a single scale, introduces cross-resolution feature fusion interaction, enhances the modeling ability for large displacements, solves the problem that the traditional DIC network has poor prediction effects for large displacements, and compared with the traditional DIC method, it does not require manual adjustment of the sub-region radius parameter and can directly predict the large displacement field.

[0018] In a possible implementation manner, the global matching of the 1 / 4-scale target features and 1 / 4-scale reference features after feature enhancement in step 503 to obtain a low-resolution initial displacement field specifically includes: Step 503A: Flatten the 1 / 4-scale target features and reference features into pixel sequences: , , where ; represents the width of the image. represents the height of the image. represents the number of feature channels; Step 503B: Calculate the similarity between the -th pixel feature vector of the reference image and the -th pixel feature vector of the target image. The calculation formula is: ; Step 503C: Calculate the attention weight based on the similarity . The calculation formula is: ; In the formula, represents the similarity between the -th pixel feature vector of the reference image and the -th pixel feature vector of the target image.

[0019] Step 503D, based on the attention weights calculate the coordinates in the target image that match the pixel feature vectors of the reference image, and the calculation formula is: ; In the formula, represents the coordinates of the pixel in the target image; Step 503E, based on the coordinates calculate the low-resolution initial displacement field obtained, and the calculation formula is: ; In the formula, represents the coordinates of the pixel in the reference image.

[0020] In a possible implementation manner, the initial displacement field in step 403 is strengthened in the flow attention layer, which helps to eliminate the defect points in the displacement field; the expression of the flow attention calculation is: ; In the formula, represents the feature tensor of the reference image, and respectively represent the weight matrix of the query and the weight matrix of the key; represents the feature dimension, represents the input low-resolution initial displacement field; The function represents normalizing the scaled dot product result, represents the displacement field after being strengthened by the flow attention.

[0021] In a possible implementation manner, the displacement iterative refinement layer includes T sequentially connected refinement modules, and each refinement module includes a motion encoder, an iterative refinement unit, and a displacement prediction head; the step-by-step iterative update of the displacement field in step 505 by combining the correlation feature and the context feature in the displacement iterative refinement layer specifically includes: Step 505A, the displacement field strengthened by the flow attention is used as the input of the first layer of refinement module, and the initial hidden feature state ; Step 505B, when the motion encoder in the refinement module receives the correlation feature , and the displacement field updated and output by the displacement prediction head in the previous refinement modulePerform search and convolution fusion, and output the motion feature map motionfeature. The expression is as follows: ; ; In the formula, represents performing a search in the correlation volume of all pixel pairs to extract the correlation of the displacement field; Step 505C: The iterative refinement unit in the current refinement module receives the motion feature map , the context feature, and the hidden feature state output by the iterative refinement unit in the previous refinement module to perform refinement and output the current hidden feature state ; The calculation expression is: ; In the formula, represents the concatenation operation; represents the context feature; Step 505D: The displacement prediction head in the current refinement module calculates the current displacement field residual based on the current hidden feature state , and updates the current displacement field and the displacement field to obtain the current displacement field ; The calculation expression is: ; ; Step 505E, , determine whether it satisfies . If so, return to Step 505B. If not, output the updated displacement field .

[0022] In a possible implementation manner, in Step 6, the registered target image and the reference image are synchronously subjected to overlapping block processing to obtain several pairs of target image blocks and reference image blocks with overlapping regions, which specifically includes: Step 601: Slide and divide the registered target image and the reference image according to the set size BLOCK_SIZE, and make the boundaries of each reference image block and target image block have an overlapping region according to the preset sliding step size; perform symmetric reflection filling on the reference image blocks and target image blocks with insufficient preset size at the image boundary to obtain reference image blocks and target image blocks with the same size; The expression for symmetric reflection filling is: ; ; In the formula, Denote the original reference image block, Denote the padded reference image block, Denote the image block size BLOCK_SIZE, and Denote the height and width of the original image block; Denote the original target image block, Denote the padded target image block; Step 602, both the reference image block and the target image block include a central region and an edge region, and weight masks are constructed for the central region and the edge region respectively; among them, the weight mask expression for the central region is: ; The edge region adopts a linear weight mask, and the expression is: ; In the formula, , is the starting coordinate of the current block in the original image, is the ending coordinate of the current block in the original image, and Denote the height and width of the image block, and Denote the total height and width of the image, Denote the overlapping region, Then represents the layer index of the overlapping region; Step 603, the paired reference image block and target image block are processed in the improved DIC neural network model and output the displacement field block according to Steps 501 - 506; Step 604, the displacement field block is weighted and attenuated based on the weight mask, the weighted displacement field block is accumulated to the corresponding position, and smoothed and fused with the overlapping region to obtain the displacement field at the original image resolution. The whole process expression is: ; ; ; ; ; ; ; ; In the formula, , DICNet represents the improved DIC neural network model, and Denote the displacement fields of the image blocks in the x and y directions; represents the final displacement field result, and represents the displacement field after applying the weight mask, stores the weight accumulation result.

[0023] Compared with the prior art, the present application adopts a strategy based on overlapping block and weight mask fusion, which can overcome the video memory bottleneck caused by the one-time input of high-resolution images into the DIC network. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is an experimental diagram for strain measurement of rotating components in an embodiment of the present application; Figure 2 is a schematic structural diagram of the improved DIC neural network model of the present application for processing reference targets and image targets; Figure 3 is a visual comparison diagram of the effects of applying and not applying flow attention to the initial displacement field in the present application; Figure 4 is a visual comparison diagram of the effects of the improved DIC neural network model of the present application and the prior art Ncorr model; Figure 5 is a visual comparison diagram of the effects of image overlapping segmentation processing and non-segmentation processing in the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] First of all, those skilled in the art should understand that these embodiments are only used to explain the technical principles of the embodiments of the present application and are not intended to limit the protection scope of the embodiments of the present application. Those skilled in the art can make adjustments according to needs to adapt to specific application scenarios.

[0026] In the description of the embodiments of the present application, it should be noted that unless otherwise clearly specified and limited, the terms "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific situations.

[0027] In the embodiments of the present application, unless otherwise clearly specified and limited, the first feature being "on" or "under" the second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, the first feature being "above", "over" and "on top of" the second feature may mean that the first feature is directly above or obliquely above the second feature, or simply indicates that the horizontal height of the first feature is higher than that of the second feature. The first feature being "under", "beneath" and "underneath" the second feature may mean that the first feature is directly below or obliquely below the second feature, or simply indicates that the horizontal height of the first feature is less than that of the second feature.

[0028] The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] See Figures 1 to 4 As shown, the embodiments of the present application disclose a method for measuring the strain field of a high-speed rotating object based on deep learning DIC. As Figure 1 shown, this embodiment is for non-contact strain field measurement of a helicopter blade during high-speed rotation, including: Step 1, prepare a speckle pattern on the surface of the rotating component of the target object; in this embodiment, the material for preparing the speckle pattern on the helicopter blade is selected, and a combination of white primer and black matte paint is used to ensure that the paint does not peel off or deform on the blade under high-speed rotation conditions; The production method includes: Step 101, spray the white primer: use a spray gun to evenly spray the white primer on the helicopter blade to cover the area to be measured, with the thickness controlled at 10–20 microns, and form a uniform background after drying; Step 102, randomly spray black speckles on the uniform background formed in Step 101: adjust the air pressure of the spray gun to 0.3–0.5 MPa, and randomly spray black matte paint at a distance of 20–30 cm from the surface to form speckles with a diameter of 0.1–0.5 mm and a density of 5–10 points per square millimeter; let it stand for 12 hours after spraying to ensure curing.

[0030] Step 2, start the rotation of the target object, and use an industrial camera to take a sequence of continuous multi-frame speckle images of the selected area; randomly select two of the speckle images as the reference image and the target image respectively and perform preprocessing; specifically including: The industrial camera in this embodiment is a high-speed camera with a frame rate ≥ 100,000 frames per second and a resolution ≥ 1280×1024 pixels; and adjust the frame rate according to the rotation speed of the helicopter blade; in this embodiment, the rotation speed of the helicopter blade is 1000 RPM, and the frame rate of the high-speed camera satisfies at least 50 frames captured per revolution; Illumination system setting: use an LED array light source with a wavelength of 520 nm and a power ≥ 500 W, and cooperate with a diffuser plate to achieve uniform illumination; the angle of the light source is 45° with the optical axis of the camera to avoid reflection interference; Fix the high-speed camera with a tripod so that its optical axis is perpendicular to the measured surface of the selected area, and the depth of field covers the axial movement range of the rotating object; use a laser calibrator to ensure that the camera and the lighting system are coaxial; Shooting parameter settings: Set the exposure time to 1 - 5 microseconds according to the lighting intensity to avoid motion blur; control the gain at 0 - 3 dB to ensure that the image signal-to-noise ratio ≥ 30 dB; set the frame rate of the high-speed camera to 1666 Hz; Start the rotation of the helicopter blade, and use the high-speed camera to capture multiple consecutive frames of speckle images; randomly select two of the speckle images, and preprocess the reference image and the target image, specifically including: Step 201, Use the non-local means filtering algorithm to denoise the reference image and the target image; Step 202, Use histogram equalization to improve the contrast between the speckles and the background in the reference image and the target image; Step 203, Eliminate the artifacts caused by high-speed rotation in the reference image and the target image through digital image processing; Improve the quality of the collected speckle images, and lay a foundation for the subsequent analysis of the improved DIC neural network model.

[0031] Step 3, Register the target image and the reference image; specifically including: Step 301, The ORB detection algorithm (Oriented FAST and Rotated BRIEF), based on the combination of FAST and BRIEF, introduces the concept of rotation invariance and can well handle rotation, scaling, and brightness changes; in this embodiment, the ORB detection algorithm is used to detect the matching feature points in the reference image and the target image respectively, and obtain the point sets pts1 and pts2 respectively; the ORB detection algorithm is prior art and will not be elaborated here.

[0032] Step 302, The rotation center is the invariant point of the rotation transformation between the two images. Estimate the rotation center by calculating the intersection of the perpendicular bisectors of the lines connecting each pair of matching feature points. For this purpose, based on the matching feature points in the point sets pts1 and pts2, the least squares method is used to solve the rotation center ; specifically including: Step 302A, Obtain the perpendicular bisector equation through the matching feature points in the point sets pts1 and pts2: ; In the formula, represents the normal vector of the direction vector of the line connecting the th pair of matching feature points, , represents the A feature point, represents the th feature point in the point set pts1; , represents the y-axis coordinate of the connection direction vector , The connection direction vector x-axis coordinate; represents the rotation center; represents the midpoint of the pair of matching feature points, ; Step 302B, form a linear system with the perpendicular bisector equations of all matching feature points in the point set pts1 and the point set pts2 , where represents the normal vector matrix, represents the constant term vector; the expression is: ; ; In the formula, , , represents the coordinates of the th feature point in the point set pts2, represents the coordinates of the th feature point in the point set pts1, ; Step 302C, since there may be noise in the matching points and the perpendicular bisectors may not intersect exactly at one point, the linear equations may not have an exact solution; therefore, the least squares method is used to solve for the rotation center , and the calculation formula is: ; In the formula, represents transpose of the matrix.

[0033] Step 303, based on the rotation center , use the arctangent function to calculate the rotation angle ; specifically including: Step 303A, based on the rotation center calculate the vector of each pair of matching feature points relative to the rotation center, ; Step 303B, use the arctangent function to calculate the rotation angle , and the calculation formula is: ; In the formula, and They are covariance terms between all matched feature points, and the calculation formula is: ; .

[0034] Step 304, based on the rotation center and the rotation angle perform rigid body displacement correction on the target image, and the remaining displacement is the true deformation (such as blade bending and stretching) to register the target image and the reference image.

[0035] Step 4, construct an improved DIC neural network model, and determine whether to perform segmentation processing on the registered target image and reference image. If so, go to Step 6; if not, go to Step 5; The improved DIC neural network model includes a feature extraction layer, a feature enhancement layer (Transformer), a flow attention layer, a feature fusion module, a feature correlation volume, a displacement iterative refinement layer, and an output layer.

[0036] Step 5, the improved DIC neural network model performs feature extraction, feature enhancement, feature fusion, and displacement iterative refinement on the registered target image and reference image to obtain a displacement field at the original image resolution, and calculates a strain field based on the displacement field at the original image resolution; specifically including: Step 501, after the target image and the reference image are input into the feature extraction layer, they are first downsampled by 1 / 2 to obtain 1 / 2-scale target features and 1 / 2-scale reference features, and then downsampled by 1 / 4 to obtain 1 / 4-scale target features and 1 / 4-scale reference features.

[0037] Step 502, perform feature enhancement operations on the 1 / 4-scale target features and 1 / 4-scale reference features in the Transformer feature enhancement layer.

[0038] Step 503, the 1 / 4-scale target features and 1 / 4-scale reference features after feature enhancement perform global matching in the flow attention layer to obtain an initial displacement field at low resolution; the initial displacement field is strengthened through a flow attention module in the flow attention layer to eliminate defective points in the displacement field and obtain a displacement field; Figure 3 is a deformation example obtained by applying a horizontal displacement to a real speckle picture. To test the effect of the module, an ablation experiment is performed on the module, as Figure 3 shown. It is found from the visualization diagram of applying flow attention to the initial displacement field ( Figure 3 b) that this module eliminates Figure 3 most of the defective points in the displacement field in

[0039] Obtaining the low-resolution initial displacement field specifically includes: Step 503A, flattening the target features and reference features at 1 / 4 scale into pixel sequences: , , where ; represents the width of the image, represents the height of the image, represents the number of feature channels; Step 503B, calculating the similarity between the th pixel feature vector of the reference image and the th pixel feature vector of the target image, and the calculation formula is: ; Step 503C, calculating the attention weight based on the similarity , and the calculation formula is: ; In the formula, represents the similarity between the th pixel feature vector of the reference image and the th pixel feature vector of the target image; Step 503D, calculating the coordinates in the target image where the th pixel feature vector in the reference image is matched based on the attention weight , and the calculation formula is: ; In the formula, represents the coordinates of the pixel in the target image; Step 503E, calculating the low-resolution initial displacement field based on the coordinates , and the calculation formula is: ; In the formula, represents the coordinates of the pixel in the reference image.

[0040] Strengthening the initial displacement field in the flow attention layer helps to eliminate the defective points in the displacement field; the expression for attention calculation is: ; In the formula, represents the feature tensor of the reference image, and represent the weight matrix of the query and the weight matrix of the key respectively; represents the feature dimension, represents the input low-resolution initial displacement field; The function represents the normalization of the scaled dot product result. Represents the displacement field after stream attention enhancement.

[0041] Step 504 , the enhanced 1 / 4 scale target features, 1 / 4 scale reference features, 1 / 2 scale target features, and 1 / 2 scale reference features are fused in a feature fusion module, and correlation characteristics are obtained in a feature correlation body.

[0042] Step 505: The displacement field is combined with the correlation feature and the context feature and is updated step by step in the displacement iterative refinement layer. The displacement field after iterative update is convolutionally upsampled in the output layer to output the displacement field with the original image resolution.

[0043] The displacement iterative refinement layer includes T sequentially connected refinement modules, each refinement module includes a motion encoder, an iterative refinement unit and a displacement prediction head; The displacement iterative refinement layer performs step-by-step iterative updating of the displacement field, specifically including: Step 505A, displacement field after flow attention enhancement As the input of the first layer refinement module, the initial hidden feature state ; Step 505B, when the motion encoder in the refinement module receives the correlation feature , and the displacement field updated by the displacement prediction head in the previous refinement module Perform search and convolution fusion to output the motion feature map motionfeature, which is expressed as: ; ; In the formula, It means searching in the correlation volume of all pixel pairs to extract the correlation of the displacement field; Step 505C: the iterative refinement unit in the current refinement module receives the motion feature map , context features, and the hidden feature state output by the iterative refinement unit in the previous refinement module Refine and output the current hidden feature state ; The calculation expression is: ; In the formula, Represents a splicing operation; Represents contextual features; Step 505D, the displacement prediction head in the current refinement module calculates the current displacement field residual based on the current hidden feature state and updates it based on the current displacement field residual and the displacement field to obtain the current displacement field ; its calculation expression is: ; ; ; Step 505E, , determine whether it satisfies , if so, return to Step 505B, if not, output the updated displacement field .

[0044] Step 506, the original image resolution displacement field calculates the surface strain components of the original image resolution displacement field using the Green-Lagrange strain tensor formula with the displacement gradient tensor to obtain the strain field, represents the normal strain in the x direction, describing the deformation amount along the x-axis, represents the normal strain in the y direction, describing the deformation amount along the y-axis, represents the shear strain in the xy plane, describing the angular distortion in the x-y plane.

[0045] Step 507, use finite element post-processing software (such as Abaqus or MATLAB) to draw the strain contour map.

[0046] Figure 4 is the displacement field of a pair of test samples of large displacement fields, as shown in Figure 4 a. The improved DIC neural network model of the present application uses global matching followed by iterative refinement of the initial low-resolution displacement field through the feature fusion module and the displacement iteration refinement layer Figure 4 b is the visualization diagram of the Ncorr model effect, and the estimation of large displacement further improves the accuracy in the displacement iteration refinement layer; as shown in Figure 4 a and 4b, the improved DIC neural network model of the present application achieves similar accuracy to the Ncorr software in the prediction of large displacements within 20px.

[0047] Step 6, perform overlapping block processing on the registered target image and reference image synchronously to obtain several pairs of target image blocks and reference image blocks with overlapping regions. The improved DIC neural network model performs feature extraction, feature enhancement, feature fusion, and displacement iteration refinement on the target image blocks and reference image blocks to obtain several displacement field blocks, splices and fuses the displacement field blocks based on the overlapping regions to obtain the original image resolution displacement field, and calculates the strain field based on the original image resolution displacement field, specifically including: Step 601: The registered target image and reference image are slidably segmented according to the set size BLOCK_SIZE, and the sliding segmentation step length is preset so that the boundaries of each reference image block and target image block have an overlapping area; for the reference image blocks and target image blocks with insufficient size at the image boundary, symmetric reflection filling is performed to obtain reference image blocks and target image blocks with the same size; the expression for symmetric reflection filling is: ; ; In the formula, represents the original reference image block, represents the filled reference image block, represents the image block size BLOCK_SIZE, and represent the height and width of the original image block; represents the original target image block, represents the filled target image block; Step 602: Both the reference image block and the target image block include a central region and an edge region, and weight masks are constructed for the central region and the edge region respectively; among them, the expression for the weight mask of the central region is: ; The edge region uses a linear weight mask, and the expression is: ; In the formula, , is the starting coordinate of the current block in the original image, is the ending coordinate of the current block in the original image, and represent the height and width of the image block, and represent the total height and width of the image, represents the overlapping area, then represents the layer index of the overlapping area; Step 603: The paired reference image blocks and target image blocks are processed in the improved DIC neural network model and output displacement field blocks according to Steps 501 - 506; Step 604: The displacement field blocks are weighted and attenuated based on the weight masks, the weighted displacement field blocks are accumulated to the corresponding positions, and smoothed fusion is performed with the overlapping areas to obtain the displacement field at the original image resolution. The expression for the whole process is: ; ; ; ; ; ; ; ; wherein, , DICNet represents an improved DIC neural network model, and represent the displacement fields of the image patches in the x and y directions; , represents the final displacement field result, and represent the displacement fields after applying the weight mask, , stores the weight accumulation result.

[0048] As Figure 5 shown, a represents the visualization diagram of the effect of the overlapping segmentation process, and b represents the visualization diagram of the effect of the overlapping segmentation process of the reference image and the target image in this application; it can be clearly seen that the displacement field defect points in the image are all eliminated, and the accuracy of the displacement estimation is further improved in the displacement iteration refinement layer.

[0049] In the description of the embodiments of this application, it should be noted that in the description of this application, the terms indicating the direction or positional relationship such as "inside", "outside", etc. are based on the direction or positional relationship shown in the drawings. This is only for convenience of description, rather than indicating or implying that the device or component must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be construed as a limitation of this application.

[0050] In the description of this application, the descriptions with reference to terms such as "one embodiment", "some embodiments", "in this embodiment", "specific examples", or "some examples", etc. mean that the specific features, mechanisms, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic descriptions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, mechanisms, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0051] As described above, it is only the specific implementation manner of the present application. However, the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for measuring the strain field of a high-speed rotating object based on deep learning DIC, characterized in that Including: Step 1, preparing a speckle pattern on the surface of the rotating part of the target object; Step 2, starting to rotate the target object, and using an industrial camera to capture a series of consecutive speckle images of the selected area; Randomly selecting two speckle images as the reference image and the target image and performing preprocessing; Step 3, registering the target image and the reference image; Step 4, constructing an improved DIC neural network model, and determining whether to perform segmentation processing on the registered target image and reference image. If so, go to Step 6; if not, go to Step 5; Step 5, the improved DIC neural network model performs feature extraction, feature enhancement, feature fusion, and displacement iterative refinement on the registered target image and reference image to obtain a displacement field with the original image resolution, and calculates a strain field based on the displacement field with the original image resolution; Step 6, synchronously performing overlapping block processing on the registered target image and reference image to obtain several pairs of target image blocks and reference image blocks with overlapping regions. The improved DIC neural network model performs feature extraction, feature enhancement, feature fusion, and displacement iterative refinement on the target image blocks and reference image blocks to obtain several displacement field blocks, splices and fuses the displacement field blocks based on the overlapping regions to obtain a displacement field with the original image resolution, and calculates a strain field based on the displacement field with the original image resolution.

2. The method for measuring the strain field of a high-speed rotating object based on deep learning DIC according to claim 1, wherein The preprocessing of the reference image and the target image in Step 2 specifically includes: Step 201, denoising the reference image and the target image using a non-local means filtering algorithm; Step 202, using histogram equalization to improve the contrast between the speckles and the background in the reference image and the target image; Step 203, eliminating the artifacts caused by high-speed rotation in the reference image and the target image through digital image processing.

3. The method for measuring the strain field of a high-speed rotating object based on deep learning DIC according to claim 1, wherein, Step 3 specifically includes: Step 301, using the ORB detection algorithm to detect the matching feature points in the reference image and the target image respectively, and obtaining the point sets pts1 and pts2 respectively; Step 302: Based on the matching feature points in point set pts1 and point set pts2, use the least squares method to solve for the rotation center ; Step 303, based on the rotation center , use the arctangent function to calculate the rotation angle ; Step 304, based on the rotation center and the rotation angle perform rigid body displacement correction on the target image to register the target image and the reference image.

4. The method for measuring the strain field of a high-speed rotating object based on deep learning DIC according to claim 3, characterized in that Step 302 specifically includes: Step 302A, obtaining the perpendicular bisector equation through the matching feature points in the point sets pts1 and pts2; ； In the formula, represents the normal vector of the connecting line direction vector of the matched feature points, ; , represents the th feature point in the point set pts2, represents the th feature point in the point set pts1; , represents the y-axis coordinate of the connecting line direction vector , the x-axis coordinate of the connecting line direction vector ; represents the rotation center; represents the midpoint of the th pair of matched feature points; Step 302B, form a linear system with the equations of the perpendicular bisectors of all the matching feature points in point set pts1 and point set pts2 , where represents the normal vector matrix, represents the constant term vector; the expression is: ； ； In the formula, , , represents the coordinates of the -th feature point in the point set pts2, represents the coordinates of the -th feature point in the point set pts1, ; Step 302C, using the least squares method to solve for the rotation center , and the calculation formula is: ； In the formula, represents the transpose of the normal vector matrix .

5. The method for measuring the strain field of a high-speed rotating object based on deep learning DIC according to claim 3, characterized in that, Step 303 specifically includes: Step 303A, based on the rotation center Calculate the vectors of each pair of matching feature points relative to the rotation center , ; Step 303B, calculate the rotation angle using the arctangent function , and the calculation formula is: ； In the formula, and are respectively the covariance terms between all matched feature points, and the calculation formula is: ； 。 6. The method for measuring the strain field of a high-speed rotating object based on deep learning DIC according to claim 1, wherein The improved DIC neural network model in Step 4 includes a feature extraction layer, a feature enhancement layer, a flow attention layer, a feature fusion module, a feature correlation volume, a displacement iterative refinement layer, and an output layer; Step 5 specifically includes: Step 501, after the target image and the reference image are input into the feature extraction layer, they are first downsampled by 1 / 2 to obtain 1 / 2-scale target features and 1 / 2-scale reference features, and then downsampled by 1 / 4 to obtain 1 / 4-scale target features and 1 / 4-scale reference features; Step 502, performing feature enhancement operations on the 1 / 4-scale target features and 1 / 4-scale reference features in the feature enhancement layer; collecting context features of the 1 / 2-scale target features and 1 / 2-scale reference features in the feature extraction layer; Step 503, the 1 / 4-scale target feature and the 1 / 4-scale reference feature after feature enhancement are globally matched in the flow attention layer to obtain a low-resolution initial displacement field; the initial displacement field is strengthened in the flow attention layer through a flow attention module, which helps to eliminate the defect points in the displacement field and obtain a displacement field; Step 504, the 1 / 4-scale target feature, the 1 / 4-scale reference feature, the 1 / 2-scale target feature, and the 1 / 2-scale reference feature after feature enhancement are feature fused in the feature fusion module, and correlation characteristics are obtained in the feature correlation volume; Step 505, the displacement field is gradually iteratively updated in the displacement iterative refinement layer by combining the correlation feature and the context feature, and the iteratively updated displacement field is subjected to convolutional upsampling operation in the output layer to output a displacement field with the original image resolution; Step 506: Based on the Green-Lagrange strain tensor formula, the original image resolution displacement field calculates the surface strain components of the original image resolution displacement field using the displacement gradient tensor , and obtains the strain field; represents the normal strain in the x direction, represents the normal strain in the y direction, represents the shear strain in the xy plane.

7. The method for measuring the strain field of a high-speed rotating object based on deep learning DIC according to claim 6, wherein The specific process that the 1 / 4-scale target feature and the 1 / 4-scale reference feature after feature enhancement in Step 503 are globally matched in the flow attention layer to obtain a low-resolution initial displacement field includes: Step 503A: Flatten the target feature and reference feature at 1 / 4 scale into a pixel sequence: , , where ; represents the width of the image, represents the height of the image, represents the number of feature channels; Step 503B, calculate the similarity between the th pixel feature vector of the reference image and the th pixel feature vector of the target image , and the calculation formula is: ； Step 503C, based on the similarity Calculate the attention weight , and the calculation formula is: ； In the formula, represents the similarity between the -th pixel feature vector of the reference image and the -th pixel feature vector of the target image; Step 503D, based on the attention weights Calculate the coordinates of the matching of the th pixel feature vector in the reference image in the target image , and the calculation formula is: ； In the formula, represents the coordinates of the pixel in the target image; Step 503E, based on the coordinates calculate the initial displacement field with low resolution , and the calculation formula is: ； In the formula, represents the coordinates of the pixel in the reference image.

8. The method for measuring the strain field of a high-speed rotating object based on deep learning DIC according to claim 6, wherein The initial displacement field in Step 503 is strengthened in the flow attention layer, and the calculation expression in the flow attention layer is: ； In the formula, represents the feature tensor of the reference image, and represent the weight matrix of the query and the weight matrix of the key respectively; represents the feature dimension, represents the input low-resolution initial displacement field; The function represents normalizing the scaled dot product result, represents the displacement field after being enhanced by the flow attention.

9. The method for measuring the strain field of a high-speed rotating object based on deep learning DIC according to claim 6, wherein The displacement iterative refinement layer includes T sequentially connected refinement modules, and each refinement module includes a motion encoder, an iterative refinement unit, and a displacement prediction head; the specific process that the displacement field in Step 505 is gradually iteratively updated in the displacement iterative refinement layer by combining the correlation feature and the context feature includes: Step 505A, the displacement field after stream attention enhancement As the input of the first layer refinement module, the initial hidden feature state ; Step 505B, when the motion encoder in the refinement module receives the correlation feature , and the displacement field updated and output by the displacement prediction head in the previous refinement module perform search and convolutional fusion to output the motion feature map motionfeature, and the expression is: ； ； In the formula, represents searching in the correlation body of all pixel pairs to extract the correlation of the displacement field; Step 505C, the iterative refinement unit in the current refinement module receives the motion feature map , the context feature, and the hidden feature state output by the iterative refinement unit in the previous refinement module for refinement and outputs the current hidden feature state ; the calculation expression is: ； In the formula, represents a splicing operation; represents a context feature; Step 505D, the displacement prediction head in the current refinement module is based on the current hidden feature state to calculate the current displacement field residual , and based on the current displacement field residual and the displacement field to perform an update to obtain the current displacement field ; its calculation expression is: ； ； Step 505E, , determine whether the following is satisfied . If yes, return to Step 505B; if no, output the updated displacement field .

10. The method for measuring the strain field of a high-speed rotating object based on deep learning DIC according to claim 6, wherein, The specific process that the registered target image and reference image in Step 6 are synchronously subjected to overlapping block processing to obtain several pairs of target image blocks and reference image blocks with overlapping regions includes: Step 601, the registered target image and reference image are slidably sliced according to the set size BLOCK_SIZE, and the boundary of each reference image block and target image block has an overlapping region according to the preset sliding step; symmetric reflection padding is performed on the reference image blocks and target image blocks with insufficient size at the image boundary to obtain reference image blocks and target image blocks with consistent size; the expression of symmetric reflection padding is: ； ； In the formula, represents the original reference image block, represents the padded reference image block, represents the image block size BLOCK_SIZE, and represent the height and width of the original image block; represents the original target image block, represents the padded target image block; Step 602, both the reference image block and the target image block include a central region and an edge region, and weight masks are constructed for the central region and the edge region respectively; among them, the weight mask expression of the central region is: ； The edge region adopts a linear weight mask, and the expression is: ； In the formula, , is the starting coordinate of the current block in the original image, is the ending coordinate of the current block in the original image, and represent the height and width of the image block, and represent the total height and width of the image, represents the overlapping area, represents the layer index of the overlapping area; Step 603, the paired reference image blocks and target image blocks are processed in the improved DIC neural network model according to the processing of Step 501 - Step 506 to output displacement field blocks; Step 604, the displacement field blocks are weighted and attenuated based on the weight mask, the weighted displacement field blocks are accumulated to the corresponding positions, and smoothed fusion is performed with the overlapping regions to obtain a displacement field with the original image resolution. The entire process expression is: ； ； ； ； ； ； ； ； In the formula, , DICNet represents the improved DIC neural network model, and represent the displacement fields of the image patches in the x and y directions; , represents the final displacement field result, and represent the displacement fields after applying the weight mask, , stores the weight accumulation result.

Images