Cross-attention mechanism driven high-fidelity optical deformation measurement technology and image generation method and system

Abstract

This invention provides a high-fidelity optical deformation measurement technology and image generation method and system driven by a cross-attention mechanism, including: a high-precision deformation measurement step, which uses a feature extraction network to map a reference image and a deformed image to a high-dimensional feature space, constructs key-value vectors and query vectors respectively through learnable transformation matrices, constructs a similarity loss function based on a cross-attention mechanism, and calculates the displacement field and strain field by combining a preset parameterized representation mode; and a high-fidelity image generation step, which defines the deformed image as a pixel-level optimizable tensor, calculates the gradient and iteratively corrects the pixel gray-level distribution of the tensor under the constraint of a preset displacement field, and derives a high-fidelity deformed image that conforms to physical laws. This invention transforms image generation into an optimization problem under physical constraints, solving the defect of traditional interpolation methods that do not conform to physical laws, and providing a high-quality training dataset for deep learning-driven algorithms.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, measurement, and image generation technology, and specifically to a high-fidelity optical distortion measurement technology and image generation method and system driven by a cross-attention mechanism. Background Technology

[0002] Digital image correlation (DIR) measurement is an optical measurement method based on image matching. It calculates the displacement and strain fields of the object's surface by comparing image information from the reference state and the deformed state, based on a preset correlation criterion. It is widely used in materials mechanics, structural inspection, and other fields. Current mainstream DIR methods largely rely on grayscale consistency criteria (such as the Zero Mean Normalized Minimum Squared Distance Criterion, ZNSSD), directly using the original grayscale values ​​of image pixels as the matching basis. Matching and positioning are achieved by calculating the grayscale difference between corresponding regions in the reference image and the deformed image. However, in actual measurement scenarios, deformed images are often affected by various factors: for example, changes in illumination intensity in the measurement environment can cause overall grayscale shifts or local distortions; uneven material surface textures and poor speckle pattern quality can cause blurred grayscale features; stretching and overlapping of surface textures during large deformations can disrupt the continuity of grayscale distribution; and noise interference from image acquisition equipment can also significantly affect the matching reliability of the grayscale consistency criterion, leading to incorrect matching of corresponding points. Ultimately, this results in a significant decrease in the measurement accuracy of the displacement and strain fields, and may even lead to measurement failure.

[0003] Meanwhile, with the deep integration of deep learning technology and digital image correlation measurement technology, deep learning-based deformation measurement algorithms have shown broad application prospects due to their powerful feature learning and fitting capabilities. High-quality, large-scale training datasets are crucial for improving algorithm performance. Currently, most deformation image datasets used to train these deep learning algorithms are generated by randomly interpolating reference images. This generation method only performs simple numerical transformations at the pixel level, without considering the underlying physical laws such as geometric constraints and texture continuity of the measured object during physical deformation, resulting in deviations between the generated deformation images and the actual physical deformation. Training models based on such synthetic image data that does not conform to physical laws will cause the neural network to learn false deformation patterns generated by interpolation, rather than the true underlying deformation laws, thus limiting the generalization ability, measurement accuracy, and reliability of deep learning-driven digital image correlation measurement algorithms in complex real-world scenarios.

[0004] Patent CN116934820A discloses a method and system for fabric image registration based on a multi-size window Transformer network with cross-attention. By continuously matching and fusing image features on a cross-attention Transformer architecture, two input features are fused into one attention information, and shared parameters are used for feature matching to better address the image registration problem, thereby completing the task of identifying defective fabrics. The patent does not disclose that this invention uses a feature extraction network to map the original grayscale image to a high-dimensional feature space, and combines a cross-attention mechanism for correlation measurement. The attention loss is used as a global constraint term to adjust the neural network parameters and guide the adaptive optimization of the grid node coordinates, thereby achieving a technical solution for sub-pixel level high-precision deformation measurement.

[0005] In summary, given the problems of the existing technologies, researching a high-fidelity optical deformation measurement technology and image generation method and system driven by a cross-attention mechanism has become a critical task that urgently needs to be addressed. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to provide a high-fidelity optical deformation measurement technology and image generation method and system driven by a cross-attention mechanism.

[0007] The present invention provides a high-fidelity optical distortion measurement technology and image generation method driven by a cross-attention mechanism, comprising the following steps:

[0008] Step S1, high-precision deformation measurement step: the reference image and the deformed image are mapped to a high-dimensional feature space using a feature extraction network. The key vector K and the query vector Q are constructed respectively through a learnable transformation matrix. A similarity loss function based on the cross-attention mechanism is constructed. Combined with the preset parameterized representation mode, the displacement field and strain field are obtained by iteratively solving using an optimization algorithm. Step S2, the high-fidelity image generation step, defines the generated deformed image as a pixel-level optimizable tensor. Under the constraint of a preset displacement field, a joint objective function containing a similarity term and a regularization term is constructed with physical consistency as the guide. The gradient is calculated through an automatic differentiation framework and the pixel grayscale distribution of the tensor is iteratively corrected to derive a high-fidelity deformed image that conforms to physical laws.

[0009] Preferably, the high-precision deformation measurement steps include: Step S1.1, recording image information of the sample in the reference state and the deformed state, the image information including the reference image F and the deformed image G, and extracting the region of interest (ROI); Step S1.2, using a feature extraction network to map the image to a high-dimensional feature space, and constructing the key vector K of the reference image and the query vector Q of the deformed image through a learnable transformation matrix; Step S1.3, measuring image similarity by calculating the inner product of the query vector Q and the key vector K, and constructing a correlation loss function based on a cross-attention mechanism; Step S1.4, selecting a local sub-region mode, a global finite element mode, or a white-box neural network representation mode according to the measurement requirements, and representing the displacement field... Perform parameterized mapping; in step S1.5, using the correlation loss function as the objective function, use an optimization algorithm to iteratively correct the parameters and calculate the high-precision displacement field and strain field.

[0010] Preferably, in step S1.2, constructing the key vector K of the reference image and the query vector Q of the deformed image using a learnable transformation matrix includes: for the reference image F, using a learnable transformation matrix... The pixel data is encoded to obtain the key vector K, expressed as:

[0011] For a deformable image G, a learnable transformation matrix can be used. The pixel data is encoded to obtain the query vector Q, expressed as:

[0012] Where K corresponds to the pixel distribution or local sub-region within the region of interest in the reference image F, Q corresponds to the search region or matching sub-region in the deformed image G, and Net represents the feature extraction network.

[0013] Preferably, in step S1.3, constructing a relevance loss function based on a cross-attention mechanism includes: calculating the attention score function between the query vector Q and the key-value vector K, expressed as:

[0014] in, The dimension of the feature vector is used to scale the dot product result; Constructing a loss function based on cross-attention mechanism This is used to replace or enhance the traditional zero-mean normalized minimum squared distance standard, and its expression is:

[0015] The dot product is maximized and the attention score is highest when the physical points in the reference image and the matching points in the deformed image are aligned in the feature space. The loss function achieves the optimal match between the reference image sub-region and the deformed image sub-region in the physical space. It approaches 0.

[0016] Preferably, in step S1.4, a local sub-region mode, a global finite element mode, or a white-box neural network representation mode are selected according to the measurement requirements to represent the displacement field. Performing parameterized mapping includes: in the local sub-region pattern, dividing the ROI into a series of independent sub-regions, for the reference sub-region center point any point inside it Corresponding points in the deformed image Determined by the shape function W:

[0017] Where P represents the shape function parameters to be optimized. For displacement changes; In the global finite element model, the ROI is divided into interconnected finite elements, and element shape functions are used. Combined with nodal displacement Interpolate the displacement of any point within the element:

[0018] In the white-box neural network representation mode, the ROI is globally meshed to generate shape functions, all node displacements are determined by simultaneously matching all elements, and a coordinate-driven neural network is used to calculate a direct and continuous displacement field mapping.

[0019] in, For displacement field, The number of nodes in the grid cell. For displacement field coordinates, For grid nodes coordinates For grid node displacement; for The shape function network structure, the weights of the intermediate layers of the neural network. and deviation Grid node coordinates Functions: .

[0020] Preferably, the high-fidelity image generation step includes: step S2.1, defining the deformable image G to be generated as an optimizable tensor. Where h and w represent the height and width of the image, respectively, and the gray value at each pixel coordinate point is an independent optimizable variable; Step S2.2, given the reference image F and the preset displacement field u, the pixel generation optimization range in the deformed image G is pre-defined according to the measurement mode. Step S2.3: Within the defined area, using the grayscale values ​​of pixels in the deformed image G as the optimization objective, construct a joint objective function based on physical consistency, which includes a similarity loss term and a regularization constraint term. Step S2.4: Calculate the gradient of the objective function with respect to the gray value of each pixel to be generated using an automatic differentiation framework, and iteratively correct the gray distribution through an optimization algorithm. Step S2.5: After the loss function of the local region converges, stitch the generated gray blocks together to export the complete high-fidelity deformed image G.

[0021] Preferably, in step S2.2, the pixel generation optimization range in the deformed image G is pre-defined according to the measurement mode. This includes: the methods for defining the pixel generation optimization range include local sub-region mode, global finite element mode, and white-box neural network representation mode; for the local sub-region mode, the reference image F is divided into independent sub-regions of a predetermined size, the shape function W and its corresponding shape function parameter P of each sub-region are calculated according to a preset displacement field, the displacement of each point in the sub-region and the mapped sub-region coordinates are determined, and the gray value of the i-th pixel in the reference sub-region f is... The gray value of the i-th pixel in the deformable sub-region g They are represented as follows:

[0022]

[0023] in, This indicates the coordinates of the center pixel of the sub-region. For this sub-region The coordinates of each pixel; The coordinate set boundary is defined based on the mapped deformed sub-region g, and the pixel optimization range of this sub-region is defined within the deformed image G. and in Integer pixels are selected for pixel grayscale value optimization; for the global finite element mode, the reference image F is divided into finite element meshes, and shape functions are used. Interpolating for each grid cell, the mapped coordinates of the pixels within the grid cell in the deformed image G are calculated as follows:

[0024] in, The displacement of the grid nodes is obtained through preset displacement calculation; Based on the deformed twisted mesh boundary, the pixel optimization range of the corresponding unit is defined in the deformed image G. , and in Integer pixels are selected for pixel grayscale value optimization; for the white-box neural network representation mode, its shape function... From neural network parameters The composition and mapping coordinates of pixels within a grid cell in the deformed image G are calculated as follows:

[0025] Among them, the weights of the intermediate layers of the neural network and deviation The coordinates of the unit node The function, i.e. This is used to define the adaptive optimization region based on the white-box neural network architecture.

[0026] Preferably, in step S2.3, the objective function for:

[0027] in, For similarity loss terms, As a reference sub-region, For preset sub-regions, For the preset displacement field, This is a regularization constraint term used to ensure the smoothness of the generated image and suppress numerical noise. These are the weighting coefficients.

[0028] Preferably, step S2.4 includes: for any pixel in the image Its gradient update direction is represented as:

[0029] in, Let the objective function be... For similarity loss terms, As a reference sub-region, For preset sub-regions, For the preset displacement field, For regularization constraints, These are the weighting coefficients; An optimization algorithm is used to continuously correct the gray-level distribution at various points in the deformed image G based on gradient information:

[0030] in, For learning rate, For the number of iterations, For gradient operators, The gradient of the loss function with respect to gray level is used in this process to iteratively correct the gray level within a defined range, so that the generated image is visually aligned with the reference image.

[0031] This invention also provides a high-fidelity optical distortion measurement technology and image generation system driven by a cross-attention mechanism, comprising the following modules: Data generation module M1 is used to execute the high-fidelity image generation step, and iteratively generate a high-fidelity deformable image dataset that conforms to physical laws based on a variety of preset physical deformation fields u and a reference image F. Model training module M2 uses the high-fidelity deformed image dataset as training samples to perform supervised or self-supervised training on the feature extraction neural network, and optimizes the learnable transformation matrix through backpropagation. , and network weight parameters; The measurement application module M3 executes the high-precision deformation measurement method on the trained feature extraction neural network to calculate the actual displacement field and strain field of the test sample.

[0032] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention proposes a digital image correlation measurement method based on cross-attention mechanism enhancement. It uses a feature extraction network to map the original grayscale image to a high-dimensional feature space and combines the cross-attention mechanism to measure the correlation. Compared with the traditional ZNSSD standard that directly depends on grayscale values, the high-dimensional features contain rich texture and semantic consistency information, which enables the system to maintain accurate matching and positioning even when facing complex environments such as drastic changes in illumination, image noise and poor speckle pattern quality. (2) This invention proposes a physically constrained image generation method, which transforms image generation into an optimization problem under physical constraints. Through pixel-level parameterization and gradient descent, the generated deformed image not only conforms to the preset displacement at the pixel level, but also aligns with the reference image at the low-level features. This solves the limitation of traditional interpolation generation methods that do not conform to the laws of physical deformation, and provides a higher-quality training dataset that conforms to real physical characteristics for deep learning-based digital image-related algorithms, significantly improving the generalization ability and reliability of such algorithms; (3) This invention utilizes an automatic differentiation framework and can switch between different representation modes. Furthermore, due to the independence of each sub-region or unit during the reverse generation process, an efficient parallel computing strategy can be adopted, which satisfies the computational efficiency requirements of complex scenarios while ensuring high fidelity. Attached Figure Description

[0033] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart of a high-fidelity optical deformation measurement technology and image generation method driven by a cross-attention mechanism according to the present invention.

[0034] Figure 2 This is a block diagram of a high-fidelity optical deformation measurement technology and image generation system driven by a cross-attention mechanism according to the present invention.

[0035] Figure 3 The diagram shows the full-field displacement results of the method of the present invention under simulated uniaxial tensile conditions.

[0036] Figure 4 This is a diagram showing the full-field displacement results under simulated uniaxial tensile conditions using the traditional grayscale consistency method.

[0037] Figure 5 This is a high-fidelity deformed image and measured displacement map of a physically constrained generative framework. Detailed Implementation

[0038] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0039] like Figure 1 As shown, this embodiment provides a high-fidelity optical deformation measurement technology and image generation method driven by a cross-attention mechanism. This method encompasses two main modules: high-precision deformation measurement and high-fidelity image generation. In the measurement stage, a feature extraction network maps the reference image and the deformed image to a high-dimensional feature space. A learnable transformation matrix is ​​used to construct a key vector K and a query vector Q. A similarity loss function is constructed based on the cross-attention mechanism. Combining local sub-region, global finite element, or white-box neural network representation modes, an optimization algorithm iteratively solves the displacement and strain fields. In the generation stage, the deformed image to be generated is defined as a pixel-level optimizable tensor. Under a preset displacement field constraint, a joint objective function containing a similarity term and a regularization term is constructed with physical consistency as the guiding principle. An automatic differentiation framework iteratively corrects the pixel grayscale distribution, deriving a high-fidelity deformed image that conforms to physical laws.

[0040] In the measurement phase, this embodiment provides a digital image correlation measurement method based on multi-channel convolutional feature enhancement, specifically including the following steps: Step 1.1: Record the image information of the sample in the reference state and the deformed state, namely the reference image F and the deformed image G, and extract the region of interest (ROI).

[0041] Step 1.2: Use a feature extraction network to map the image to a high-dimensional feature space, and construct the key vector K of the reference image and the query vector Q of the deformed image through a learnable transformation matrix.

[0042] Specifically, for the reference image F, the pixel distribution or local sub-region within its region of interest is defined as the key vector K; for the deformed image G, its corresponding search region or matching sub-region is defined as the query vector Q. Unlike traditional methods that directly compare raw grayscale values, this approach utilizes a learnable transformation matrix... and The pixel data is encoded to obtain the key vector K and the query vector Q, as expressed by:

[0043] in, The feature transformation matrix is ​​a learnable feature. Feature extraction network.

[0044] This step extracts high-dimensional features that are a mixture of textures, edges, and geometric structures with semantic consistency, thereby enhancing the system's ability to resist interference from drastic changes in lighting, large deformations, and image noise.

[0045] Step 1.3: Measure image similarity by calculating the inner product of query vector Q and key vector K, and construct a relevance loss function based on cross-attention mechanism.

[0046] Specifically, the attention score function expression is:

[0047] in, The dimension of the feature vector is used to scale the dot product result to prevent gradient vanishing. The dot product result is maximized and the attention score is highest when the physical point in the reference image and the matching point in the deformed image are aligned in the same direction in the feature space.

[0048] An attention-based loss function is used to replace or enhance the traditional zero-mean normalized minimum squared distance criterion (ZNSSD). This means that the loss function tends to 0 when the reference image sub-region and the deformed image sub-region achieve optimal matching in physical space. The loss function is defined as follows:

[0049] Step 1.4: Select the local sub-region mode, global finite element mode, or white box neural network representation mode according to the measurement requirements, and perform parameterized mapping of the displacement field u.

[0050] Specifically, for the local sub-region pattern, the ROI is divided into a series of independent sub-regions. For the reference sub-region center point... any point inside it Corresponding points in the deformed image Determined by the shape function W:

[0051] Where P is the shape function parameter to be optimized, the cross-attention mechanism applies to each independent sub-region, and the loss function Loss is independently differentiated with respect to the parameter P of each sub-region.

[0052] For the global finite element model, the ROI is divided into interconnected finite elements to ensure the continuity of the displacement field at the element boundaries, utilizing element shape functions. Combined with nodal displacement Interpolate the displacement of any point within the element:

[0053] For the shape function of a bilinear mesh element, the formula is:

[0054] in, Indicates the first The first four-node unit Each node coordinate; Indicates the first The first four-node unit Each node coordinate.

[0055] This method treats the attention loss as a global constraint and optimizes the displacement vectors of all grid nodes. By using an automatic differentiation method, the gradient propagates throughout the entire grid, effectively suppressing local noise and obtaining a physically consistent continuous displacement field.

[0056] For the white-box neural network representation model, the shape function is generated by global meshing of the ROI. The displacements of all nodes are determined by simultaneously matching all elements, and the direct and continuous displacement field mapping is calculated using a coordinate-driven neural network.

[0057] This method uses the attention loss as a global constraint term, adjusting the neural network parameters... Guide grid node coordinates The adaptive optimization enables high-precision deformation measurement at the sub-pixel level while ensuring the global continuity of the displacement field.

[0058] Step 1.5: Using the attention loss function as the objective function, iteratively correct the parameters using Adam or other optimization algorithms to obtain the high-precision displacement field and strain field.

[0059] Specifically, in step 1.5, using the attention loss function established in step 1.3 as the optimization objective, Adam or other optimization algorithms are used to iteratively correct the shape function parameter P in the local sub-region mode and the nodal displacements in the global finite element mode and the white-box neural network representation mode. or neural network parameters Once the calculation converges, a high-precision global displacement field and strain field are derived based on the optimization results.

[0060] In the generation phase, this embodiment provides a physically constrained reverse generation framework, which specifically includes the following steps: Step 2.1: Define the deformed image G to be generated as an optimizable tensor, where the gray value at each pixel coordinate point is an independent optimizable variable.

[0061] Specifically, the deformed image G to be generated is defined as an optimizable tensor. , where H and W represent the height and width of the image, respectively; given a reference image F and a preset displacement field u, the gray value at each pixel coordinate point of the deformed image G is an independent optimizable variable, thus allowing more complex physical features and lighting changes to be captured through gradient flow during the generation process.

[0062] Step 2.2: Given a reference image F and a preset displacement field u, pre-define the pixel generation optimization range in the deformed image G according to the measurement mode. .

[0063] Among them, the methods for defining the pixel generation optimization range include local sub-region mode, global finite element mode, and white box neural network representation mode: For the local sub-region mode, the reference image F needs to be divided into sub-regions according to a predetermined sub-region size. Then, in each sub-region, the shape function and its corresponding shape function parameters are calculated based on a preset displacement field, thereby determining the displacement of each point in the sub-region and the mapped sub-region coordinates. Therefore, the reference sub-region f and the deformed sub-region g can be written as:

[0064]

[0065] in, This indicates the coordinates of the center pixel of the sub-region. The coordinates of other pixels in this sub-region For shape functions, It is the parameter of its corresponding shape function.

[0066] The coordinate set boundary is defined based on the mapped deformed sub-region g, and the pixel optimization range of this sub-region is defined within the deformed image G. And in Optimize pixel grayscale values ​​by selecting integer pixels.

[0067] For the global finite element model, first, a finite element mesh is generated for the reference image F, and then shape functions are used. Interpolating for each grid cell, the mapped coordinates of the pixels within the grid cell in the deformed image G are calculated as follows:

[0068] in The displacement of the grid nodes is obtained by calculating the preset displacement.

[0069] Based on the deformed twisted mesh boundary, the pixel optimization range of the corresponding unit is defined in the deformed image G. , and in Optimize pixel grayscale values ​​by selecting integer pixels.

[0070] For the white-box neural network representation mode, its shape function From neural network parameters The composition and mapping coordinates of pixels within a grid cell in the deformed image G are calculated as follows:

[0071] The weights of the intermediate layers of the neural network and deviation The coordinates of the unit node The function, i.e. This is used to define the adaptive optimization region based on the white-box neural network architecture.

[0072] Step 2.3: Within the defined area, using the grayscale values ​​of pixels in the deformed image G as the optimization objective, construct a joint objective function based on physical consistency, which includes a similarity loss term and a regularization constraint term. .

[0073] Specifically, the location range defined in step 2.2 ( or Within the image G, identify all pixels located at integer coordinates, and use the grayscale values ​​of the pixels in the deformed image G as the optimization target to construct a target optimization function based on physical consistency.

[0074] Define the objective function for inverse generation using the cross-attention loss function or ZNSSD. for:

[0075] in, For similarity loss terms, This is a regularization constraint term used to ensure the smoothness of the generated image and suppress numerical noise. These are the weighting coefficients.

[0076] in, For regularization, total variation regularization, Tikhonov regularization (L2 regularization), and gradient consistency constraints can be used.

[0077] Total variational regularization (TVGF) suppresses numerical noise generated by pixel-level independent optimization and enhances the physical smoothness of the generated image by penalizing the magnitude of gray-level gradients between adjacent pixels in a deformed image. Its expression is:

[0078] Tikhonov regularization, by constraining the L2 norm of the gray-level difference between the generated deformed image and the initial state image, limits the evolution range of pixel values ​​during the optimization process and ensures numerical stability. Its expression is:

[0079] in, This is a preset initial pixel tensor. This item anchors the initial guessed grayscale value to prevent numerical divergence or abnormal grayscale jumps in complex lighting or extreme deformation areas.

[0080] The gradient consistency constraint ensures that the structural texture features of the generated image remain highly consistent with those of the reference image by minimizing the difference between the gradient field of the generated deformed image and the gradient field of the reference image under a given preset displacement field mapping. Its expression is:

[0081] Step 2.4: Calculate the gradient of the objective function with respect to the grayscale value of each pixel to be generated using an automatic differentiation framework, and iteratively correct the grayscale distribution through an optimization algorithm.

[0082] Specifically, the loss function is backpropagated using an automatic differentiation framework to calculate the gradient of the objective function with respect to the grayscale value of each pixel to be generated. For any pixel in the image... Its gradient update direction is represented as:

[0083] This gradient field precisely describes how the grayscale value at each location in the deformed image should be adjusted to achieve optimal matching with the reference image F under a given displacement field u. Since the mappings between subsets or units are independent, a parallel computing strategy can be adopted.

[0084] Secondly, optimization algorithms such as Adam are used to continuously correct the gray-level distribution at various points in the deformed image G based on gradient information:

[0085] in For learning rate, For the number of iterations, For gradient operators, This represents the gradient of the loss function with respect to gray levels. The process iteratively refines the gray values ​​within a defined area, aligning the generated image with the reference image in terms of visual features.

[0086] Step 2.5: After the local region loss function converges, the generated grayscale blocks are stitched together to export the complete high-fidelity deformed image G.

[0087] like Figure 2 As shown, this embodiment also provides a high-fidelity optical deformation measurement technology and image generation system driven by a cross-attention mechanism, including the following modules: Data generation module M1: This module runs a physically constrained inverse generation framework, defining the deformed image to be generated as a pixel-level optimizable tensor. It selects generation modes such as local sub-region, global finite element method, and white-box neural network representation, and under preset displacement field constraints, constructs a joint objective function including similarity and regularization terms, guided by physical consistency. It then uses an automatic differentiation framework to iteratively correct the pixel grayscale distribution, deriving a high-fidelity deformed image dataset that conforms to physical laws. Model training module M2: Used to train the feature extraction network using the generated high-fidelity deformed image dataset as training samples. It optimizes the learnable transformation matrix in the feature extraction network by utilizing the physical law information in the dataset, thereby enhancing the network's ability to extract high-dimensional features with semantic consistency. Measurement Application Module M3: Used to run an enhanced measurement program based on the cross-attention mechanism. It uses a trained and optimized feature extraction network to map the reference image and the deformed image to a high-dimensional feature space to construct the key value vector K and the query vector Q. It constructs the cross-attention correlation loss function by calculating the inner product of Q and the key value vector K, and selects displacement field measurement modes such as local sub-region, global finite element and white box neural network representation to solve the displacement field and strain field.

[0088] To verify the effectiveness of the present invention, this example sets up a set of comparative experiments to compare the performance of the image correlation measurement method based on the cross-attention mechanism of the present invention with the traditional measurement method based on the zero-mean normalized minimum distance squared standard.

[0089] The experiment used a set of sizes A speckle image of pixels. The reference image F is the original speckle image, the deformed image G is a linearly stretched deformation in the X direction, and the deformation value in the Y direction should be strictly 0. The ROI is set to the center. Pixel area.

[0090] This invention utilizes a pre-trained residual neural network and a Transformer neural network as feature extraction networks to map images to a high-dimensional feature space. The reference image F, after being mapped by the feature network, outputs a high-dimensional feature map with a feature dimension of 16, defined as the key-value vector K. Then, a global finite element method is employed to divide the Region of Interest (ROI) into... The cell grid, total There are 100 mesh element nodes. In each iteration, a sampled mesh is generated based on the current node displacement parameters, and a bilinear interpolation shape function is used. Constructing continuous displacement field and pseudo-deformation image region This ensures the continuity of displacement at the element boundaries. The input feature network extracts the query vector Q. The attention score is obtained by calculating the dot product and inner product of Q and the pre-stored key vector K, using a similarity function based on cross-attention. Let the objective function be $\mathbf{a}$. The Adam optimizer is used iteratively to update all nodal displacement parameters. Once the loss function converges, the final nodal displacements are derived.

[0091] Experimental results are as follows Figure 3 and Figure 4 As shown, Figure 3 This is a diagram showing the full-field displacement results of the method of the present invention under simulated uniaxial tensile conditions. Figure 4 The figure shows the full-field displacement results of the traditional grayscale consistency method under simulated uniaxial tensile conditions. As shown, in the X direction, both methods can accurately identify the linear tensile trend in this direction; however, in the Y direction, the traditional grayscale consistency method, affected by image noise and interpolation errors, exhibits significant numerical fluctuations in the Y direction where the theoretical displacement is 0, with the maximum deviation reaching [value missing]. The method of this invention benefits from the noise suppression capability of high-dimensional features, resulting in an extremely smooth Y-direction displacement field with a maximum deviation of only [value missing]. The comparative results show that, under the same grid configuration, the measurement method based on the cross-attention mechanism proposed in this invention significantly outperforms the traditional gray-scale consistency method in displacement recognition accuracy. In particular, when dealing with artifact noise in non-deformation directions, this method reduces the measurement error by about 70%, demonstrating its powerful ability to extract high-fidelity physical deformation features in complex environments.

[0092] To verify the effectiveness of the present invention, this embodiment also uses a uniaxial tensile deformation scenario to conduct an image generation experiment based on physical constraints.

[0093] The experiment first loaded an image of size [size missing]. A reference speckle image F for each pixel is used, and its grayscale values ​​are normalized to the range [0,1]. To address image boundary shift caused by deformation, a [presumably a specific algorithm or method] is established. Optimizable background tensor of pixels The reference image F is placed in the center region as an initial guess, and the rest of the background is filled appropriately.

[0094] In terms of displacement field construction, a... A preset deformation displacement field u exists with a linear stretch of 0.05 in the direction. Based on the global finite element model, the area is divided within the ROI. The finite element mesh. Using shape functions. Calculate the mapped coordinates of the mesh nodes after deformation. ,in The center alignment offset is used. The optimized range of unit pixels is defined within the deformed image G using the calculated twisted mesh boundary. And generate the corresponding mask.

[0095] During the unit-parallel optimization process, the Adam optimization algorithm is used to optimize the image tensor. To improve computational efficiency in each optimization iteration, the following parallel strategies are employed: First of all The mesh is decomposed into a series of independent computational units, and a mask region is generated independently for each unit. A padding buffer is set to handle feature connections between units; then, a pseudo-deformation map is obtained by bilinear interpolation sampling. Since the mapping relationships between each unit are independent in physical space, and the ZNSSD or cross-attention loss calculation with the reference graph F is performed in parallel within all grid units; using automatic differentiation techniques, the local gradients of all units are calculated. Directly acting on Each individual pixel grayscale value in the dataset; after N iterations, the total loss value decreases significantly and tends to stabilize.

[0096] Depend on Figure 5Image generation experiments show that the optimized deformed image G visually preserves the texture features of the original speckle pattern and precisely follows the physical constraint of 0.05 stretching in pixel distribution. This is achieved by cropping the center... The deformation was measured and verified in the region, demonstrating that the deformation value could be accurately measured. This proves that the reverse generation framework described in this invention can capture complex physical features through pixel-level optimization, providing the algorithm with a high-quality training dataset that conforms to physical laws.

[0097] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0098] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A high-fidelity optical distortion measurement technique and image generation method driven by a cross-attention mechanism, characterized in that, include: Step S1, high-precision deformation measurement step: the reference image and the deformed image are mapped to a high-dimensional feature space using a feature extraction network. The key vector K and the query vector Q are constructed respectively through a learnable transformation matrix. A similarity loss function based on the cross-attention mechanism is constructed. Combined with the preset parameterized representation mode, the displacement field and strain field are obtained by iteratively solving using an optimization algorithm. Step S2, the high-fidelity image generation step, defines the generated deformed image as a pixel-level optimizable tensor. Under the constraint of a preset displacement field, a joint objective function containing a similarity term and a regularization term is constructed with physical consistency as the guide. The gradient is calculated through an automatic differentiation framework and the pixel grayscale distribution of the tensor is iteratively corrected to derive a high-fidelity deformed image that conforms to physical laws.

2. The high-fidelity optical distortion measurement technology and image generation method driven by a cross-attention mechanism according to claim 1, characterized in that, The high-precision deformation measurement steps include: Step S1.1: Record image information of the sample in the reference state and the deformed state. The image information includes the reference image F and the deformed image G, and extract the region of interest (ROI). Step S1.2: The image is mapped to a high-dimensional feature space using a feature extraction network, and the key value vector K of the reference image and the query vector Q of the deformed image are constructed through a learnable transformation matrix. Step S1.3: Measure image similarity by calculating the inner product of query vector Q and key vector K, and construct a relevance loss function based on cross-attention mechanism; Step S1.4: Select the local sub-region mode, global finite element mode, or white-box neural network representation mode according to the measurement requirements to represent the displacement field. Perform parameterized mapping; Step S1.5: Using the correlation loss function as the objective function, an optimization algorithm is used to iteratively correct the parameters, and the high-precision displacement field and strain field are obtained.

3. The high-fidelity optical distortion measurement technology and image generation method driven by a cross-attention mechanism according to claim 2, characterized in that, In step S1.2, constructing the key vector K of the reference image and the query vector Q of the deformed image using a learnable transformation matrix includes: For a reference image F, through a learnable transformation matrix The pixel data is encoded to obtain the key vector K, expressed as: For a deformable image G, a learnable transformation matrix can be used. The pixel data is encoded to obtain the query vector Q, expressed as: Where K corresponds to the pixel distribution or local sub-region within the region of interest in the reference image F, Q corresponds to the search region or matching sub-region in the deformed image G, and Net represents the feature extraction network.

4. The high-fidelity optical distortion measurement technology and image generation method driven by a cross-attention mechanism according to claim 2, characterized in that, In step S1.3, the relevance loss function based on the cross-attention mechanism is constructed, including: The attention score function for the query vector Q and the key vector K is calculated as follows: in, The dimension of the feature vector is used to scale the dot product result; Constructing a loss function based on cross-attention mechanism This is used to replace or enhance the traditional zero-mean normalized minimum squared distance standard, and its expression is: The dot product is maximized and the attention score is highest when the physical points in the reference image and the matching points in the deformed image are aligned in the feature space. The loss function achieves the optimal match between the reference image sub-region and the deformed image sub-region in the physical space. It approaches 0.

5. The high-fidelity optical distortion measurement technology and image generation method driven by a cross-attention mechanism according to claim 2, characterized in that, In step S1.4, a local sub-region mode, a global finite element mode, or a white-box neural network representation mode are selected according to the measurement requirements to represent the displacement field. Perform parameterized mapping, including: In the local sub-region mode, the ROI is divided into a series of independent sub-regions, with a reference sub-region center point. any point inside it Corresponding points in the deformed image Determined by the shape function W: Where P represents the shape function parameters to be optimized. For displacement changes; In the global finite element model, the ROI is divided into interconnected finite elements, and element shape functions are used. Combined with nodal displacement Interpolate the displacement of any point within the element: In the white-box neural network representation mode, the ROI is globally meshed to generate shape functions, all node displacements are determined by simultaneously matching all elements, and a coordinate-driven neural network is used to calculate a direct and continuous displacement field mapping. in, For displacement field, The number of nodes in the grid cell. For displacement field coordinates, For grid nodes coordinates For grid node displacement; for The shape function network structure, the weights of the intermediate layers of the neural network. and deviation Grid node coordinates Functions: .

6. The high-fidelity optical distortion measurement technology and image generation method driven by a cross-attention mechanism according to claim 1, characterized in that, The high-fidelity image generation step includes: Step S2.1: Define the deformed image G to be generated as an optimizable tensor. , where h and w represent the height and width of the image, respectively, and the gray value at each pixel coordinate point is an independent optimizable variable; Step S2.2: Given a reference image F and a preset displacement field u, pre-define the pixel generation optimization range in the deformed image G according to the measurement mode. ; Step S2.3: Within the defined area, using the grayscale values ​​of pixels in the deformed image G as the optimization objective, construct a joint objective function based on physical consistency, which includes a similarity loss term and a regularization constraint term. ; Step S2.4: Calculate the gradient of the objective function with respect to the gray value of each pixel to be generated using an automatic differentiation framework, and iteratively correct the gray distribution through an optimization algorithm. Step S2.5: After the local region loss function converges, the generated grayscale blocks are stitched together to export the complete high-fidelity deformed image G.

7. The high-fidelity optical distortion measurement technology and image generation method driven by a cross-attention mechanism according to claim 6, characterized in that, In step S2.2, the pixel generation optimization range in the deformed image G is pre-defined according to the measurement mode. ,include: The methods for defining the optimization range of pixel generation include local sub-region mode, global finite element mode, and white box neural network representation mode; For the local sub-region pattern, the reference image F is divided into independent sub-regions of a predetermined size. The shape function W and its corresponding shape function parameter P of each sub-region are calculated based on a preset displacement field. The displacement of each point in the sub-region and the mapped sub-region coordinates are determined. The grayscale value of the i-th pixel in the reference sub-region f is then... The gray value of the i-th pixel in the deformable sub-region g They are represented as follows: in, This indicates the coordinates of the center pixel of the sub-region. For this sub-region The coordinates of each pixel; The coordinate set boundary is defined based on the mapped deformed sub-region g, and the pixel optimization range of this sub-region is defined within the deformed image G. and in Optimize pixel grayscale values ​​by selecting integer pixels; For the global finite element model, the reference image F is divided into finite element meshes, and shape functions are used. Interpolating for each grid cell, the mapped coordinates of the pixels within the grid cell in the deformed image G are calculated as follows: in, The displacement of the grid nodes is obtained through preset displacement calculation; Based on the deformed twisted mesh boundary, the pixel optimization range of the corresponding unit is defined in the deformed image G. , and in Integer pixels are selected for pixel grayscale value optimization. For the white-box neural network representation mode, its shape function From neural network parameters The composition and mapping coordinates of pixels within a grid cell in the deformed image G are calculated as follows: Among them, the weights of the intermediate layers of the neural network and deviation The coordinates of the unit node The function, i.e. This is used to define the adaptive optimization region based on the white-box neural network architecture.

8. The high-fidelity optical distortion measurement technology and image generation method driven by a cross-attention mechanism according to claim 6, characterized in that, In step S2.3, the objective function for: in, For similarity loss terms, As a reference sub-region, For preset sub-regions, For the preset displacement field, This is a regularization constraint term used to ensure the smoothness of the generated image and suppress numerical noise. These are the weighting coefficients.

9. The high-fidelity optical distortion measurement technology and image generation method driven by a cross-attention mechanism according to claim 6, characterized in that, Step S2.4 includes: For any pixel in the image Its gradient update direction is represented as: in, Let the objective function be... For similarity loss terms, As a reference sub-region, For preset sub-regions, For the preset displacement field, For regularization constraints, These are the weighting coefficients; An optimization algorithm is used to continuously correct the gray-level distribution at various points in the deformed image G based on gradient information: in, For learning rate, For the number of iterations, For gradient operators, The gradient of the loss function with respect to gray level is used in this process to iteratively correct the gray level within a defined range, so that the generated image is visually aligned with the reference image.

10. A high-fidelity optical distortion measurement technology and image generation system driven by a cross-attention mechanism, characterized in that, include: Data generation module M1 is used to execute the high-fidelity image generation step, and iteratively generate a high-fidelity deformable image dataset that conforms to physical laws based on a variety of preset physical deformation fields u and a reference image F. Model training module M2 uses the high-fidelity deformed image dataset as training samples to perform supervised or self-supervised training on the feature extraction neural network, and optimizes the learnable transformation matrix through backpropagation. , and network weight parameters; The measurement application module M3 executes the high-precision deformation measurement method on the trained feature extraction neural network to calculate the actual displacement field and strain field of the test sample.

Images