Strain field measurement method for high-speed rotating objects based on deep learning DIC

Abstract

The present invention relates to a method for measuring the strain field of a high-speed rotating object based on deep learning DIC. A speckle image sequence is acquired using a digital imaging method. After registering a target image and a reference image, the image is entered into an improved DIC neural network model for strain field acquisition, thereby achieving non-contact strain field measurement of the high-speed rotating object. In the improved DIC neural network model, flow attention is used to enhance displacement-related texture features in the initial displacement field, thereby facilitating the elimination of defect points in the displacement field. Subsequently, global matching and displacement iterative refinement are performed to improve displacement prediction accuracy, thereby enhancing the robustness and accuracy of the improved DIC neural network model.

Description

Technical Field

[0001] The present invention relates to the technical field of digital image non-contact measurement, and in particular to a method for measuring the strain field of a high-speed rotating object based on deep learning DIC. Background Art

[0002] Measuring the strain field of high-speed rotating objects is crucial in many engineering fields, such as aerospace and mechanical manufacturing. Accurately acquiring this information is crucial for assessing the structural strength and stability of rotating objects and predicting their fatigue life.

[0003] Currently, contact and non-contact measurement methods are primarily used to measure strain fields in objects. For example, patent document CN109883333A, "A Non-Contact Displacement and Strain Measurement Method Based on Image Feature Recognition Technology," proposes a non-contact displacement and strain measurement method based on image feature recognition and matching. This method requires relatively simple experimental equipment and procedures, utilizes a non-contact, lossless measurement method that does not degrade the material's properties, resulting in more accurate data. Automatic data processing is also possible, enabling the determination of full-field displacement and changes in the width of structural components, while also requiring relatively low environmental requirements. Patent document CN201852566U, "Non-Contact Optical Strain Gauge Based on Digital Images," discloses a digital image-based non-contact optical strain gauge that utilizes a common imaging lens and at least two image sensors positioned on the image plane to achieve high-resolution strain measurement. Patent document CN113808029B, "A Strain Smoothing Method in Digital Image Correlation," first establishes a digital image correlation measurement system to obtain digital images of a specimen before and after deformation. Through a series of operations, the strain measurement accuracy of the digital image correlation method can be improved to a certain extent.

[0004] However, contact measurement methods have numerous limitations in high-speed rotation scenarios. First, contact measurement is easily affected by rotational speed, making it difficult to guarantee measurement accuracy. For example, when aircraft engine blades rotate at high speed, contact measurement equipment may produce measurement errors due to high-speed vibration. Second, contact measurement may interfere with the object's motion, changing the object's original force state and trajectory. Furthermore, installing contact measurement equipment on a high-speed rotating object is often difficult, requiring complex installation processes and equipment, increasing measurement costs and operational complexity.

[0005] Non-contact measurement methods also have shortcomings when measuring the strain field of high-speed rotating objects. Some methods do not have clear requirements for the production of speckle patterns on the surface of high-speed rotating objects, which may cause the speckle to fall off or deform during high-speed rotation, affecting the accuracy of the measurement results. When facing high-speed rotating objects, some image acquisition systems cannot meet the frame rate and resolution requirements and cannot clearly capture the changes in the state of the object at different times. In the image analysis and strain calculation links, some algorithms have poor adaptability to the complex motion state of high-speed rotating objects, and the calculation accuracy and efficiency need to be improved.

[0006] In summary, the existing technology has certain limitations in measuring the strain field of high-speed rotating objects. There is an urgent need for a measurement method that can effectively overcome the above problems to meet the needs of the engineering field for accurate measurement of the strain field of high-speed rotating objects. Summary of the Invention

[0007] The technical problem to be solved by the present invention is how to accurately measure the strain field of a high-speed rotating object.

[0008] The present invention provides a method for measuring the strain field of a high-speed rotating object based on deep learning DIC, comprising:

[0009] Step 1, preparing a speckle pattern on the surface of a rotating component of a target object;

[0010] Step 2: Start rotating the target object and use an industrial camera to capture multiple frames of speckle images in the selected area; randomly select two speckle images as reference images and target images and perform preprocessing;

[0011] Step 3, registering the target image and the reference image;

[0012] Step 4: construct an improved DIC neural network model to determine whether to segment the registered target image and reference image. If yes, proceed to step 6; if not, proceed to step 5.

[0013] Step 5: The improved DIC neural network model performs feature extraction, feature enhancement, feature fusion, and displacement iterative refinement on the registered target image and the reference image to obtain a displacement field at the original image resolution, and calculates the strain field based on the displacement field at the original image resolution;

[0014] In step 6, the registered target image and reference image are simultaneously subjected to overlapping block processing to obtain several pairs of target image blocks and reference image blocks with overlapping areas. The improved DIC neural network model performs feature extraction, feature enhancement, feature fusion, and displacement iterative refinement on the target image blocks and reference image blocks to obtain several displacement field blocks. The displacement field blocks are spliced ​​and fused based on the overlapping areas to obtain the original image resolution displacement field, and the strain field is calculated based on the original image resolution displacement field.

[0015] Compared with the prior art, the present application has the following advantages: the present application collects speckle image sequences based on a digital image method, and after registering the target image and the reference image, enters the improved DIC neural network model to obtain the strain field, thereby realizing non-contact strain field measurement of high-speed rotating objects; the reference image and the target image are first registered to eliminate the influence of rigid body displacement, so that the improved DIC neural network model can more accurately predict non-rigid body displacement and reduce noise interference caused by image misalignment, thereby more accurately iteratively optimizing the displacement field; and, in the improved DIC neural network model, flow attention is used to enhance displacement-related texture features in the initial displacement field, eliminate defect points in the displacement field, and then global matching and displacement iterative refinement are performed to improve the displacement prediction accuracy, thereby improving the robustness and accuracy of the improved DIC neural network model.

[0016] In a possible implementation, the preprocessing of the reference image and the target image in step 2 specifically includes:

[0017] Step 201, denoising the reference image and the target image using a non-local means filtering algorithm;

[0018] Step 202, using histogram equalization to improve the contrast between speckle and background in the reference image and the target image;

[0019] Step 203: Eliminate artifacts in the reference image and the target image caused by high-speed rotation by digital image processing.

[0020] Compared with the existing technology, the above technical solution can improve the quality of the acquired speckle images and lay a good foundation for the subsequent improved DIC neural network model analysis.

[0021] In a possible implementation, step 3 specifically includes:

[0022] Step 301: Use the ORB detection algorithm to detect matching feature points in the reference image and the target image, respectively, to obtain point sets pts1 and pts2;

[0023] Step 302: Based on the matching feature points in point set pts1 and point set pts2, the least square method is used to solve the rotation center. ;

[0024] Step 303, based on the rotation center , use the inverse tangent function to calculate the rotation angle ;

[0025] Step 304, based on the rotation center and rotation angle The target image is corrected for rigid body displacement to align the target image with the reference image.

[0026] Compared with existing technologies, the ORB detection algorithm can better cope with rotation, scaling and brightness changes, and accurately pick up feature points in the reference image and the target image; so as to more accurately solve the rotation center and rotation angle, thereby better aligning the reference image and the target image.

[0027] In a possible implementation, step 302 specifically includes:

[0028] Step 302A, obtain the perpendicular bisector equation by matching the feature points in the point set pts1 and the point set pts2:

[0029] ;

[0030] Where, Indicates the Direction vector of the line connecting the matched feature points The normal vector of , Indicates the point set pts2 feature points, Indicates the point set pts1 feature points; , Represents the direction vector of the connection The y-axis coordinate, Connection direction vector The x-axis coordinate of represents the center of rotation; Indicates the For the midpoint of the matched feature points, ;

[0031] Step 302B: compose the perpendicular bisector equations of all matching feature points in point set pts1 and point set pts2 into a linear system ,in, represents the normal vector matrix, represents the constant term vector; the expression is:

[0032] ;

[0033] ;

[0034] Where, , , Represents the point set pts2 The coordinates of the feature points, Indicates the point set pts1 The coordinates of the feature points, ;

[0035] Step 302C, using the least squares method to solve the rotation center , the calculation formula is:

[0036] ;

[0037] Where, Representation of normal vector matrix The transpose of .

[0038] In a possible implementation, step 303 specifically includes:

[0039] Step 303A, based on the rotation center Calculate the vector of each pair of matched feature points relative to the rotation center , ;

[0040] Step 303B, calculate the rotation angle using the inverse tangent function , the calculation formula is:

[0041] ;

[0042] Where, and are the covariance terms between all matched feature points, and the calculation formula is:

[0043] ;

[0044] .

[0045] In one possible implementation, the improved DIC neural network model in step 4 includes a feature extraction layer, a feature enhancement layer, a stream attention layer, a feature fusion module, a feature correlation body, a displacement iterative refinement layer, and an output layer; and step 5 specifically includes:

[0046] Step 501: After the target image and the reference image are input into the feature extraction layer, they are first down-sampled by 1 / 2 to obtain 1 / 2 scale target features and 1 / 2 scale reference features, and then down-sampled by 1 / 4 to obtain 1 / 4 scale target features and 1 / 4 scale reference features;

[0047] Step 502: The 1 / 4 scale target features and the 1 / 4 scale reference features are enhanced in a feature enhancement layer; the 1 / 2 scale target features and the 1 / 2 scale reference features are used to collect context features in a feature extraction layer.

[0048] Step 503: The enhanced 1 / 4 scale target features and the 1 / 4 scale reference features are globally matched in the stream attention layer to obtain a low-resolution initial displacement field; the initial displacement field is enhanced in the stream attention layer to help eliminate defects in the displacement field, thereby obtaining a displacement field.

[0049] Step 504: After upsampling, the enhanced 1 / 4 scale target features and 1 / 4 scale reference features are fused with the 1 / 2 scale target features and 1 / 2 scale reference features in a feature fusion module, and correlation characteristics are obtained in a feature correlation body.

[0050] Step 505: The displacement field is iteratively updated in a displacement iterative refinement layer in combination with correlation features and context features. The iteratively updated displacement field is convolutionally upsampled in an output layer to output the original image resolution displacement field.

[0051] Step 506: The original image resolution displacement field is calculated based on the Green-Lagrange strain tensor formula, and the surface strain component of the original image resolution displacement field is calculated using the displacement gradient tensor. , we get the strain field, represents the normal strain in the x-direction, represents the positive strain in the y direction, represents the shear strain in the xy plane.

[0052] Compared with the existing technology, the present application adopts global matching of 1 / 4 scale target features and 1 / 4 scale reference features to obtain a low-resolution initial displacement field; then, the 1 / 4 scale target features and 1 / 4 scale reference features are upsampled and fused with the 1 / 2 scale target features and 1 / 2 scale reference features, and then the refinement is gradually improved in the displacement iterative refinement layer; the strategy of performing global matching under low-resolution features and further iterative refinement under high-resolution features avoids the limitations of a single scale, introduces cross-resolution feature fusion interaction, enhances the modeling capability of large displacements, and solves the problem that traditional DIC networks have poor prediction effects on large displacements. Compared with traditional DIC methods, large displacement fields can be directly predicted without manual adjustment of sub-region radius parameters.

[0053] In a possible implementation, performing global matching on the stream attention layer to obtain the low-resolution initial displacement field in step 503 with the 1 / 4 scale target feature and the 1 / 4 scale reference feature after feature enhancement specifically includes:

[0054] Step 503A: Flatten the 1 / 4 scale target features and reference features into a pixel sequence: , ,in, ; Indicates the width of the image, Indicates the height of the image, Indicates the number of feature channels;

[0055] Step 503B, calculate the reference image The pixel feature vector of the target image is The similarity between pixel feature vectors , the calculation formula is:

[0056] ;

[0057] Step 503C, based on similarity Calculating attention weights , the calculation formula is:

[0058] ;

[0059] Where, The reference image The pixel feature vector of the target image is The similarity between pixel feature vectors.

[0060] Step 503D, based on attention weight Calculate the reference image The coordinates of the pixel feature vectors in the target image , the calculation formula is:

[0061] ;

[0062] Where, Represents the pixels in the target image coordinates of

[0063] Step 503E, based on coordinates Calculate the low-resolution initial displacement field , the calculation formula is:

[0064] ;

[0065] Where, Represents the pixels in the reference image 's coordinates.

[0066] In one possible implementation, the initial displacement field in step 403 is enhanced in the flow attention layer, which helps to eliminate defects in the displacement field. The expression for flow attention calculation is:

[0067] ;

[0068] Where, represents the feature tensor of the reference image, and Represent the query weight matrix and key weight matrix respectively; represents the feature dimension, Represents the input low-resolution initial displacement field; The function represents the normalization of the scaled dot product result. Represents the displacement field after stream attention enhancement.

[0069] In one possible implementation, the displacement iterative refinement layer includes T sequentially connected refinement modules, each refinement module including a motion encoder, an iterative refinement unit, and a displacement prediction head; the step 505 of iteratively updating the displacement field in the displacement iterative refinement layer in combination with the correlation feature and the context feature specifically includes:

[0070] Step 505A, displacement field after flow attention enhancement As the input of the first layer refinement module, the initial hidden feature state ;

[0071] Step 505B, when the motion encoder in the refinement module receives the correlation feature , and the displacement field updated by the displacement prediction head in the previous refinement module Perform search and convolution fusion to output the motion feature map motionfeature, which is expressed as:

[0072] ;

[0073] ;

[0074] Where, It means searching in the correlation volume of all pixel pairs to extract the correlation of the displacement field;

[0075] Step 505C: The iterative refinement unit in the current refinement module receives the motion feature map , context features and the hidden feature state output by the iterative refinement unit in the previous refinement module Refine and output the current hidden feature state ; The calculation expression is:

[0076] ;

[0077] Where, Represents a splicing operation; Represents contextual features;

[0078] Step 505D: The displacement prediction head in the current refinement module is based on the current hidden feature state. Calculate the current displacement field residual , and based on the current displacement field residual and displacement field Update to get the current displacement field ; Its calculation expression is:

[0079] ;

[0080] ;

[0081] Step 505E, , to determine whether If yes, then return to step 505B, if no, then output the updated displacement field .

[0082] In a possible implementation, the step 6 of synchronously performing overlapping block processing on the registered target image and the reference image to obtain a plurality of pairs of target image blocks and reference image blocks having overlapping areas specifically includes:

[0083] Step 601: Slidingly partition the registered target image and reference image according to a set size BLOCK_SIZE, so that the boundaries of each reference image block and target image block have overlapping areas according to a preset sliding partitioning step size; symmetrical reflection filling is performed on reference image blocks and target image blocks whose image boundaries are less than the preset size to obtain reference image blocks and target image blocks of the same size; the expression for symmetrical reflection filling is:

[0084] ;

[0085] ;

[0086] Where, represents the original reference image block, represents the reference image block after padding, Indicates the image block size BLOCK_SIZE, and Represents the height and width of the original image block; represents the original target image block, Represents the target image block after filling;

[0087] In step 602, both the reference image block and the target image block include a central region and an edge region, and weight masks are constructed for the central region and the edge region respectively. The weight mask expression for the central region is:

[0088] ;

[0089] The edge area uses a linear weight mask, the expression is:

[0090] ;

[0091] Where, , is the starting coordinate of the current block in the original image, is the end coordinate of the current block in the original image, and Represents the height and width of the image block, and Indicates the total height and width of the image, represents the overlapping area, It is represented as the layer index of the overlapping area;

[0092] Step 603 , the paired reference image block and target image block are processed and outputted in the improved DIC neural network model according to steps 501 to 506 to obtain a displacement field block;

[0093] In step 604, the displacement field blocks are weighted attenuated based on the weight mask, the weighted displacement field blocks are accumulated to the corresponding positions, and the overlapping areas are smoothly fused to obtain the original image resolution displacement field. The whole process is expressed as:

[0094] ;

[0095] ;

[0096] ;

[0097] ;

[0098] ;

[0099] ;

[0100] ;

[0101] ;

[0102] Where, , DICNet represents the improved DIC neural network model, and The displacement field of the image patch in the x and y directions; , represents the final displacement field result, and represents the displacement field after applying the weight mask, , store the weight accumulation results.

[0103] Compared with the existing technology, this application adopts a strategy based on overlapping blocking and weight mask fusion, which can overcome the video memory bottleneck caused by the one-time input of high-resolution images into the DIC network. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] Figure 1 This is an experimental diagram of the embodiment of the present application for measuring the strain of a rotating component;

[0105] Figure 2 Schematic diagram of the structure of the improved DIC neural network model for processing reference targets and image targets in this application;

[0106] Figure 3 This is a visual comparison of the effects of applying flow attention to the initial displacement field and not applying flow attention to the initial displacement field in this application;

[0107] Figure 4 A visual comparison diagram of the effects of the improved DIC neural network model of this application and the prior art Ncorr model;

[0108] Figure 5 This is a visual comparison of the effects of image overlapping segmentation processing and non-segmentation processing used in this application. DETAILED DESCRIPTION

[0109] First, those skilled in the art should understand that these embodiments are merely used to explain the technical principles of the embodiments of the present application and are not intended to limit the scope of protection of the embodiments of the present application. Those skilled in the art may adjust them as needed to suit specific application scenarios.

[0110] In the description of the embodiments of this application, it should be noted that, unless otherwise specified or limited, the terms "connected" and "connection" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium. Those skilled in the art will understand the specific meanings of the above terms in the embodiments of this application based on the specific circumstances.

[0111] In the embodiments of the present application, unless otherwise expressly specified or limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Furthermore, a first feature being "above," "above," and "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below," "below," and "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0112] The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0113] See also Figures 1 to 4 As shown, the embodiment of the present application discloses a method for measuring the strain field of a high-speed rotating object based on deep learning DIC, such as Figure 1 As shown, this embodiment is for non-contact strain field measurement of helicopter blades during high-speed rotation, including:

[0114] Step 1: Create a speckle pattern on the surface of a rotating component of a target object. In this embodiment, a material for creating a speckle pattern on a helicopter blade is selected, using a combination of white primer and black matte paint to ensure that the paint does not fall off or deform on the blade under high-speed rotation conditions.

[0115] Production methods include:

[0116] Step 101, spraying white primer: Use a spray gun to evenly spray white primer on the helicopter blades, covering the area to be tested, with a thickness of 10-20 microns, to form a uniform background after drying;

[0117] Step 102, randomly spray black speckles on the uniform background formed in step 101: adjust the air pressure of the spray gun to 0.3-0.5 MPa, and randomly spray black matte paint at a distance of 20-30 cm from the surface to form speckles with a diameter of 0.1-0.5 mm and a density of 5-10 dots per square millimeter; let it stand for 12 hours after spraying to ensure curing.

[0118] Step 2: Start the rotation of the target object and use an industrial camera to shoot the selected area to obtain a continuous multi-frame speckle image sequence; randomly select two speckle images as the reference image and the target image and perform preprocessing; specifically, the following steps are performed:

[0119] The industrial camera of this embodiment is a high-speed camera with a frame rate of ≥100,000 frames per second and a resolution of ≥1280×1024 pixels. The frame rate is adjusted according to the rotation speed of the helicopter blades. In this embodiment, the rotation speed of the helicopter blades is 1000 RPM, and the frame rate of the high-speed camera satisfies the requirement of capturing at least 50 frames per rotation.

[0120] Lighting system settings: Use an LED array light source with a wavelength of 520nm and a power of ≥500W, combined with a diffuser to achieve uniform lighting; the light source angle is 45° to the camera optical axis to avoid reflection interference;

[0121] The high-speed camera is fixed on a tripod so that its optical axis is perpendicular to the surface to be measured in the selected area, and the depth of field covers the axial movement range of the rotating object; a laser alignment device is used to ensure that the camera is coaxial with the lighting system;

[0122] Shooting parameter settings: Set the exposure time to 1-5 microseconds according to the lighting intensity to avoid motion blur; control the gain between 0-3dB to ensure the image signal-to-noise ratio ≥30dB; set the frame rate of the height camera to 1666Hz;

[0123] The helicopter blades are started to rotate, and a high-speed camera is used to capture multiple frames of speckle images. Two speckle images are randomly selected, and the reference image and target image are preprocessed, including:

[0124] Step 201, denoising the reference image and the target image using a non-local means filtering algorithm;

[0125] Step 202, using histogram equalization to improve the contrast between speckle and background in the reference image and the target image;

[0126] Step 203, eliminating artifacts caused by high-speed rotation in the reference image and the target image by digital image processing;

[0127] Improve the quality of the acquired speckle images and lay a good foundation for subsequent improved DIC neural network model analysis.

[0128] Step 3: Register the target image and the reference image; specifically, the following steps are performed:

[0129] In step 301, the ORB detection algorithm (oriented FAST and rotation BRIEF) combines FAST and BRIEF and introduces the concept of rotation invariance, which can effectively cope with rotation, scaling and brightness changes. This embodiment uses the ORB detection algorithm to detect matching feature points in the reference image and the target image, respectively, and obtains the point set pts1 and the point set pts2, respectively. The ORB detection algorithm is a prior art and will not be described in detail here.

[0130] Step 302: The rotation center is the invariant point of the rotation transformation between the two images. The rotation center is estimated by calculating the intersection of the perpendicular bisectors of each pair of matching feature points. To this end, the least squares method is used to solve the rotation center based on the matching feature points in point set pts1 and point set pts2. ; Specifically include:

[0131] Step 302A, obtain the perpendicular bisector equation by matching the feature points in the point set pts1 and the point set pts2:

[0132] ;

[0133] Where, Indicates the Direction vector of the line connecting the matched feature points The normal vector of , Indicates the point set pts2 feature points, Indicates the point set pts1 feature points; , Represents the direction vector of the connection The y-axis coordinate, Connection direction vector The x-axis coordinate of represents the center of rotation; Indicates the For the midpoint of the matched feature points, ;

[0134] Step 302B: compose the perpendicular bisector equations of all matching feature points in point set pts1 and point set pts2 into a linear system ,in, represents the normal vector matrix, represents the constant term vector; the expression is:

[0135] ;

[0136] ;

[0137] Where, , , Represents the point set pts2 The coordinates of the feature points, Indicates the point set pts1 The coordinates of the feature points, ;

[0138] Step 302C: Since there may be noise in the matching points and the perpendicular bisectors may not completely intersect at one point, the linear equations may not have an exact solution; therefore, the least squares method is used to solve the rotation center. , the calculation formula is:

[0139] ;

[0140] Where, express Transpose of a matrix.

[0141] Step 303, based on the rotation center , use the inverse tangent function to calculate the rotation angle ; Specifically include:

[0142] Step 303A, based on the rotation center Calculate the vector of each pair of matched feature points relative to the rotation center , ;

[0143] Step 303B, calculate the rotation angle using the inverse tangent function , the calculation formula is:

[0144] ;

[0145] Where, and are the covariance terms between all matched feature points, and the calculation formula is:

[0146] ;

[0147] .

[0148] Step 304, based on the rotation center and rotation angle The target image is corrected for rigid body displacement, and the remaining displacement is the real deformation (such as blade bending and expansion) to align the target image and the reference image.

[0149] Step 4: construct an improved DIC neural network model to determine whether to segment the registered target image and reference image. If yes, proceed to step 6; if not, proceed to step 5.

[0150] The improved DIC neural network model includes a feature extraction layer, a feature enhancement layer (Transformer), a stream attention layer, a feature fusion module, a feature correlation body, a displacement iterative refinement layer and an output layer.

[0151] Step 5: The improved DIC neural network model performs feature extraction, feature enhancement, feature fusion, and displacement iterative refinement on the registered target image and reference image to obtain the original image resolution displacement field, and calculates the strain field based on the original image resolution displacement field; specifically, the steps include:

[0152] Step 501: After the target image and the reference image are input into the feature extraction layer, they are first down-sampled by 1 / 2 to obtain 1 / 2 scale target features and 1 / 2 scale reference features, and then down-sampled by 1 / 4 to obtain 1 / 4 scale target features and 1 / 4 scale reference features.

[0153] In step 502 , the 1 / 4 scale target features and the 1 / 4 scale reference features are subjected to feature enhancement in a Transformer feature enhancement layer.

[0154] Step 503: The enhanced 1 / 4 scale target features and the 1 / 4 scale reference features are globally matched in the stream attention layer to obtain a low-resolution initial displacement field; the initial displacement field is enhanced in the stream attention layer by the stream attention module to eliminate defects in the displacement field and obtain a displacement field; Figure 3 This is a deformation sample obtained by adding a horizontal displacement to the real speckle image. In order to test the effect of the module, an ablation test is performed on the module, such as Figure 3 As shown, from the visualization of the flow attention applied to the initial displacement field ( Figure 3 b) found that this module eliminates Figure 3 The vast majority of defect points in the displacement field in a are clearer and more accurate than the displacement field obtained without applying the flow attention network.

[0155] Obtaining a low-resolution initial displacement field specifically includes:

[0156] Step 503A: Flatten the 1 / 4 scale target features and reference features into a pixel sequence: , ,in, ; Indicates the width of the image, Indicates the height of the image, Indicates the number of feature channels;

[0157] Step 503B, calculate the reference image The pixel feature vector of the target image is The similarity between pixel feature vectors , the calculation formula is:

[0158] ;

[0159] Step 503C, based on similarity Calculating attention weights , the calculation formula is:

[0160] ;

[0161] Where, The reference image The pixel feature vector of the target image is The similarity between pixel feature vectors;

[0162] Step 503D, based on attention weight Calculate the reference image The coordinates of the pixel feature vectors in the target image , the calculation formula is:

[0163] ;

[0164] Where, Represents the pixels in the target image coordinates of

[0165] Step 503E, based on coordinates Calculate the low-resolution initial displacement field , the calculation formula is:

[0166] ;

[0167] Where, Represents the pixels in the reference image 's coordinates.

[0168] Strengthening the initial displacement field in the flow attention layer helps to eliminate the defects in the displacement field; the expression for attention calculation is:

[0169] ;

[0170] Where, represents the feature tensor of the reference image, and Represent the query weight matrix and key weight matrix respectively; represents the feature dimension, Represents the input low-resolution initial displacement field; The function represents the normalization of the scaled dot product result. Represents the displacement field after stream attention enhancement.

[0171] In step 504 , the enhanced 1 / 4 scale target features, 1 / 4 scale reference features, 1 / 2 scale target features, and 1 / 2 scale reference features are fused in a feature fusion module, and correlation characteristics are obtained in a feature correlation body.

[0172] In step 505, the displacement field is iteratively updated step by step in the displacement iterative refinement layer in combination with the correlation feature and the context feature. The iteratively updated displacement field is convolutionally upsampled in the output layer to output the original image resolution displacement field.

[0173] The displacement iterative refinement layer includes T sequentially connected refinement modules, each refinement module including a motion encoder, an iterative refinement unit and a displacement prediction head;

[0174] The displacement iterative refinement layer performs step-by-step iterative updating of the displacement field, specifically including:

[0175] Step 505A, displacement field after flow attention enhancement As the input of the first layer refinement module, the initial hidden feature state ;

[0176] Step 505B, when the motion encoder in the refinement module receives the correlation feature , and the displacement field updated by the displacement prediction head in the previous refinement module Perform search and convolution fusion to output the motion feature map motionfeature, which is expressed as:

[0177] ;

[0178] ;

[0179] Where, It means searching in the correlation volume of all pixel pairs to extract the correlation of the displacement field;

[0180] Step 505C: The iterative refinement unit in the current refinement module receives the motion feature map , context features and the hidden feature state output by the iterative refinement unit in the previous refinement module Refine and output the current hidden feature state ; The calculation expression is:

[0181] ;

[0182] Where, Represents a splicing operation; Represents contextual features;

[0183] Step 505D: The displacement prediction head in the current refinement module is based on the current hidden feature state. Calculate the current displacement field residual , and based on the current displacement field residual and displacement field Update to get the current displacement field ; Its calculation expression is:

[0184] ;

[0185] ;

[0186] Step 505E, , to determine whether If yes, then return to step 505B, if no, then output the updated displacement field .

[0187] Step 506: The original image resolution displacement field is calculated based on the Green-Lagrange strain tensor formula, and the surface strain component of the original image resolution displacement field is calculated using the displacement gradient tensor. , we get the strain field, represents the normal strain in the x-direction, describing the deformation along the x-axis, represents the positive strain in the y direction, describing the deformation along the y axis, It represents the shear strain on the xy plane and describes the angular distortion in the xy plane.

[0188] Step 507: Use finite element post-processing software (such as Abaqus or MATLAB) to draw a strain contour diagram.

[0189] Figure 4 is the displacement field of a pair of test samples with large displacement fields, such as Figure 4 As shown in a, the improved DIC neural network model of the present application adopts the initial low-resolution displacement field after global matching and then iterative refinement through the feature fusion module and displacement iterative refinement layer. Figure 4 b is a visualization of the effect of the Ncorr model. The estimation of large displacements is further improved in the displacement iterative refinement layer; Figure 4 a and 4b The improved DIC neural network model of this application achieved similar accuracy to that of Ncorr software in predicting large displacements within 20px.

[0190] Step 6: The registered target image and reference image are simultaneously subjected to overlapping block processing to obtain several pairs of target image blocks and reference image blocks with overlapping areas. The improved DIC neural network model performs feature extraction, feature enhancement, feature fusion, and displacement iterative refinement on the target image blocks and reference image blocks to obtain several displacement field blocks. The displacement field blocks are spliced ​​and fused based on the overlapping areas to obtain the original image resolution displacement field, and the strain field is calculated based on the original image resolution displacement field. Specifically, the following steps are performed:

[0191] Step 601: Slidingly partition the registered target image and reference image according to a set size BLOCK_SIZE, so that the boundaries of each reference image block and target image block have overlapping areas according to a preset sliding partitioning step size; symmetrical reflection filling is performed on reference image blocks and target image blocks whose image boundaries are less than the preset size to obtain reference image blocks and target image blocks of the same size; the expression for symmetrical reflection filling is:

[0192] ;

[0193] ;

[0194] Where, represents the original reference image block, represents the reference image block after padding, Indicates the image block size BLOCK_SIZE, and Represents the height and width of the original image block; represents the original target image block, Represents the target image block after filling;

[0195] In step 602, both the reference image block and the target image block include a central region and an edge region, and weight masks are constructed for the central region and the edge region respectively. The weight mask expression for the central region is:

[0196] ;

[0197] The edge area uses a linear weight mask, the expression is:

[0198] ;

[0199] Where, , is the starting coordinate of the current block in the original image, is the end coordinate of the current block in the original image, and Represents the height and width of the image block, and Indicates the total height and width of the image, represents the overlapping area, It is represented as the layer index of the overlapping area;

[0200] Step 603 , the paired reference image block and target image block are processed and outputted in the improved DIC neural network model according to steps 501 to 506 to obtain a displacement field block;

[0201] In step 604, the displacement field blocks are weighted attenuated based on the weight mask, the weighted displacement field blocks are accumulated to the corresponding positions, and the overlapping areas are smoothly fused to obtain the original image resolution displacement field. The whole process is expressed as:

[0202] ;

[0203] ;

[0204] ;

[0205] ;

[0206] ;

[0207] ;

[0208] ;

[0209] ;

[0210] Where, , DICNet represents the improved DIC neural network model, and The displacement field of the image patch in the x and y directions; , represents the final displacement field result, and represents the displacement field after applying the weight mask, , store the weight accumulation results.

[0211] like Figure 5 As shown, a is a visualization diagram of the overlapping segmentation processing effect, and b is a visualization diagram of the overlapping segmentation processing effect of the reference image and the target image performed by this application; it can be clearly seen that the displacement field defect points in the image are completely eliminated, and the displacement estimation is further improved in accuracy in the displacement iterative refinement layer.

[0212] In the description of the embodiments of the present application, it should be noted that in the description of the present application, terms such as "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and does not indicate or imply that the device or component must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present application.

[0213] In the description of the present application, the description with reference to the terms "one embodiment", "some embodiments", "in the present embodiment", "specific example", or "some examples" means that the specific features, mechanisms, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, mechanisms, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0214] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for measuring the strain field of a high-speed rotating object based on deep learning DIC, characterized in that: include: Step 1, preparing a speckle pattern on the surface of a rotating component of a target object; Step 2: Start rotating the target object and use an industrial camera to capture the selected area to obtain multiple frames of speckle images; Two speckle images are randomly selected as reference images and target images and preprocessed; Step 3, registering the target image and the reference image; Step 4: construct an improved DIC neural network model to determine whether to segment the registered target image and reference image. If yes, proceed to step 6; if not, proceed to step 5. Step 5: The improved DIC neural network model performs feature extraction, feature enhancement, feature fusion, and displacement iterative refinement on the registered target image and the reference image to obtain a displacement field at the original image resolution, and calculates the strain field based on the displacement field at the original image resolution; In step 6, the registered target image and reference image are simultaneously subjected to overlapping block processing to obtain several pairs of target image blocks and reference image blocks with overlapping areas. The improved DIC neural network model performs feature extraction, feature enhancement, feature fusion, and displacement iterative refinement on the target image blocks and reference image blocks to obtain several displacement field blocks. The displacement field blocks are spliced ​​and fused based on the overlapping areas to obtain the original image resolution displacement field, and the strain field is calculated based on the original image resolution displacement field.

2. The method for measuring strain fields of high-speed rotating objects based on deep learning DIC according to claim 1, characterized in that: The preprocessing of the reference image and the target image in step 2 specifically includes: Step 201, denoising the reference image and the target image using a non-local means filtering algorithm; Step 202, using histogram equalization to improve the contrast between speckle and background in the reference image and the target image; Step 203: Eliminate artifacts in the reference image and the target image caused by high-speed rotation by digital image processing.

3. The method for measuring strain fields of high-speed rotating objects based on deep learning DIC according to claim 1, characterized in that: The step 3 specifically includes: Step 301: Use the ORB detection algorithm to detect matching feature points in the reference image and the target image, respectively, to obtain point sets pts1 and pts2; Step 302: Based on the matching feature points in point set pts1 and point set pts2, the least square method is used to solve the rotation center. ; Step 303, based on the rotation center , use the inverse tangent function to calculate the rotation angle ; Step 304, based on the rotation center and rotation angle The target image is corrected for rigid body displacement to align the target image with the reference image.

4. The method for measuring strain fields of high-speed rotating objects based on deep learning DIC according to claim 3, characterized in that: The step 302 specifically includes: Step 302A, obtain the perpendicular bisector equation by matching the feature points in the point set pts1 and the point set pts2: ； Where, Indicates the Direction vector of the line connecting the matched feature points The normal vector of , Indicates the point set pts2 feature points, Indicates the point set pts1 feature points; , Represents the direction vector of the connection The y-axis coordinate of Connection direction vector The x-axis coordinate of represents the center of rotation; Indicates the For the midpoint of the matched feature points, ; Step 302B: compose the perpendicular bisector equations of all matching feature points in point set pts1 and point set pts2 into a linear system ,in, represents the normal vector matrix, represents the constant term vector; the expression is: ； ； Where, , , Represents the point set pts2 The coordinates of the feature points, Indicates the point set pts1 The coordinates of the feature points, ; Step 302C, using the least squares method to solve the rotation center , the calculation formula is: ； Where, Representation of normal vector matrix The transpose of .

5. The method for measuring strain fields of high-speed rotating objects based on deep learning DIC according to claim 3, characterized in that: The step 303 specifically includes: Step 303A, based on the rotation center Calculate the vector of each pair of matched feature points relative to the rotation center , ; Step 303B, calculate the rotation angle using the inverse tangent function , the calculation formula is: ； Where, and are the covariance terms between all matched feature points, and the calculation formula is: ； 。 6. The method for measuring strain fields of high-speed rotating objects based on deep learning DIC according to claim 1, characterized in that: The improved DIC neural network model in step 4 includes a feature extraction layer, a feature enhancement layer, a stream attention layer, a feature fusion module, a feature correlation body, a displacement iterative refinement layer and an output layer; step 5 specifically includes: Step 501: After the target image and the reference image are input into the feature extraction layer, they are first down-sampled by 1 / 2 to obtain 1 / 2 scale target features and 1 / 2 scale reference features, and then down-sampled by 1 / 4 to obtain 1 / 4 scale target features and 1 / 4 scale reference features; Step 502: The 1 / 4 scale target features and the 1 / 4 scale reference features are enhanced in a feature enhancement layer; the 1 / 2 scale target features and the 1 / 2 scale reference features are used to collect context features in a feature extraction layer. In step 503, the enhanced 1 / 4 scale target features and the 1 / 4 scale reference features are globally matched in the stream attention layer to obtain a low-resolution initial displacement field; the initial displacement field is enhanced in the stream attention layer by the stream attention module to help eliminate defects in the displacement field and obtain a displacement field; Step 504: The enhanced 1 / 4 scale target features, 1 / 4 scale reference features, 1 / 2 scale target features, and 1 / 2 scale reference features are fused in a feature fusion module, and correlation characteristics are obtained in a feature correlation body. Step 505: The displacement field is iteratively updated in a displacement iterative refinement layer in combination with correlation features and context features. The iteratively updated displacement field is convolutionally upsampled in an output layer to output the original image resolution displacement field. Step 506: The original image resolution displacement field is calculated based on the Green-Lagrange strain tensor formula, and the surface strain component of the original image resolution displacement field is calculated using the displacement gradient tensor. , and obtain the strain field; represents the normal strain in the x-direction, represents the positive strain in the y direction, represents the shear strain in the xy plane.

7. The method for measuring strain fields of high-speed rotating objects based on deep learning DIC according to claim 6, characterized in that: In step 503, the 1 / 4 scale target features and the 1 / 4 scale reference features after feature enhancement are globally matched in the stream attention layer to obtain the low-resolution initial displacement field. Specifically, the following steps are performed: Step 503A: Flatten the 1 / 4 scale target features and reference features into a pixel sequence: , ,in, ; Indicates the width of the image, Indicates the height of the image, Indicates the number of feature channels; Step 503B, calculate the reference image The pixel feature vector of the target image is The similarity between pixel feature vectors , the calculation formula is: ； Step 503C, based on similarity Calculating attention weights , the calculation formula is: ； Where, The reference image The pixel feature vector of the target image is The similarity between pixel feature vectors; Step 503D, based on attention weight Calculate the reference image The coordinates of the pixel feature vectors in the target image , the calculation formula is: ； Where, Represents the pixels in the target image coordinates of Step 503E, based on coordinates Calculate the low-resolution initial displacement field , the calculation formula is: ； Where, Represents the pixels in the reference image 's coordinates.

8. The method for measuring strain fields of high-speed rotating objects based on deep learning DIC according to claim 6, characterized in that: In step 503, the initial displacement field is enhanced in the flow attention layer. The expression calculated in the flow attention layer is: ； Where, represents the feature tensor of the reference image, and Respectively represent queries The weight matrix of and the weight matrix of the key; represents the feature dimension, Represents the input low-resolution initial displacement field; The function represents the normalization of the scaled dot product result. Represents the displacement field after stream attention enhancement.

9. The method for measuring strain fields of high-speed rotating objects based on deep learning DIC according to claim 6, characterized in that: The displacement iterative refinement layer includes T sequentially connected refinement modules, each refinement module including a motion encoder, an iterative refinement unit, and a displacement prediction head; the displacement field is combined with the correlation feature and the context feature to be gradually iteratively updated in the displacement iterative refinement layer in step 505, specifically including: Step 505A, displacement field after flow attention enhancement As the input of the first layer refinement module, the initial hidden feature state ; Step 505B, when the motion encoder in the refinement module receives the correlation feature , and the displacement field updated by the displacement prediction head in the previous refinement module Perform search and convolution fusion to output the motion feature map motionfeature, which is expressed as: ； ； Where, It means searching in the correlation volume of all pixel pairs to extract the correlation of the displacement field; Step 505C: The iterative refinement unit in the current refinement module receives the motion feature map , context features and the hidden feature state output by the iterative refinement unit in the previous refinement module Refine and output the current hidden feature state ; The calculation expression is: ； Where, Represents a splicing operation; Represents contextual features; Step 505D: The displacement prediction head in the current refinement module is based on the current hidden feature state. Calculate the current displacement field residual , and based on the current displacement field residual and displacement field Update to get the current displacement field ; Its calculation expression is: ； ； Step 505E, , to determine whether If yes, then return to step 505B, if no, then output the updated displacement field .

10. The method for measuring strain fields of high-speed rotating objects based on deep learning DIC according to claim 6, characterized in that: In step 6, the registered target image and the reference image are simultaneously subjected to overlapping block processing to obtain a plurality of pairs of target image blocks and reference image blocks having overlapping areas. Specifically, the following steps are performed: Step 601: Slidingly partition the registered target image and reference image according to a set size BLOCK_SIZE, so that the boundaries of each reference image block and target image block have overlapping areas according to a preset sliding partitioning step size; symmetrical reflection filling is performed on reference image blocks and target image blocks whose image boundaries are less than the preset size to obtain reference image blocks and target image blocks of the same size; the expression for symmetrical reflection filling is: ； ； Where, represents the original reference image block, represents the reference image block after padding, Indicates the image block size BLOCK_SIZE, and Represents the height and width of the original image block; represents the original target image block, Represents the target image block after filling; In step 602, both the reference image block and the target image block include a central region and an edge region, and weight masks are constructed for the central region and the edge region respectively. The weight mask expression for the central region is: ； The edge area uses a linear weight mask, the expression is: ； Where, , is the starting coordinate of the current block in the original image, is the end coordinate of the current block in the original image, and Represents the height and width of the image block, and Indicates the total height and width of the image, represents the overlapping area, It is represented as the layer index of the overlapping area; Step 603 , the paired reference image block and target image block are processed and outputted in the improved DIC neural network model according to steps 501 to 506 to obtain a displacement field block; In step 604, the displacement field blocks are weighted attenuated based on the weight mask, the weighted displacement field blocks are accumulated to the corresponding positions, and the overlapping areas are smoothly fused to obtain the original image resolution displacement field. The whole process is expressed as: ； ； ； ； ； ； ； ； Where, , DICNet represents the improved DIC neural network model, and The displacement field of the image patch in the x and y directions; , represents the final displacement field result, and represents the displacement field after applying the weight mask, , store the weight accumulation results.

Images