Interference micro-displacement measurement method and system based on multi-modal fusion

Abstract

The invention relates to an interference micro-displacement measurement method and system based on multi-modal fusion. Comprising the steps of S1, collecting images of interference fringes in real time; s2, three types of input modes are generated in real time for each frame of interference image, three types of mode images are formed, and each type of mode carries different physical information; s3, constructing a seven-dimensional time sequence numerical value feature vector; s4, constructing a local branch for physically sensing local feature extraction, a global branch for geometrically constraining global feature extraction and a time sequence branch for time sequence physical feature extraction; s5, calculating the weight of each branch in real time based on the seven-dimensional time sequence numerical value feature vector by constructing a lightweight evaluation sub-network, and then fusing each vector according to the weight; s6, constructing a physical coupling double output head; and S7, constructing a differentiable physical decoding layer, combining the differentiable physical decoding layer with a Michelson interference equation, constructing an end-to-end differentiable displacement calculation layer, and calculating to obtain interference micro-displacement. According to the invention, the measurement efficiency and precision are effectively improved.

Description

Interferometric Micro-displacement Measurement Method and System Based on Multimodal Fusion Technical Field

[0001] This application relates to the field of precision measurement and automatic control technology, and more specifically, to an interferometric micro-displacement measurement method and system based on multimodal fusion. Background Technology

[0002] Since its invention in the late 19th century, the Michelson interferometer has become the standard instrument for precision displacement measurement. Its core principle is the formation of fringes through the interference of two laser beams, using the formula: Phase information is extracted, and displacement is calculated using Δd = Δm·λ / 2. However, traditional systems suffer from a fundamental problem of low automation: each hardware unit (voltage drive, image acquisition, displacement calculation) operates independently, lacking a unified timing synchronization and closed-loop feedback mechanism, making the entire measurement process reliant on extensive manual operation. Current technology primarily employs an open-loop manual control scheme, where researchers manually adjust the voltage via knobs or software interfaces, visually observe stripe changes, and process image data offline. This architecture leads to: 1. Poor data correlation: Voltage output, image acquisition, and displacement calculation are performed asynchronously, resulting in timestamp alignment errors of hundreds of milliseconds, severely impacting dynamic measurement accuracy; 2. Lack of real-time feedback capability: The system cannot dynamically adjust the drive strategy based on the current displacement results, and measurement must be interrupted in the event of noise interference or signal degradation; 3. Low efficiency: A single experiment requires repeated manual parameter adjustments, data export, and algorithm execution, with the entire process taking several hours.

[0003] To expand the measurement range, existing technologies mainly employ: 1. Dual-wavelength interferometry: requiring an additional laser source and wavelength synthesis device, resulting in high hardware complexity; 2. Wavelength scanning interferometry: requiring a precision tuning mechanism and a broadband light source, leading to high costs; 3. Phase fitting algorithms: using optimization methods such as differential evolution to fit fringes, but computation takes hundreds of seconds and is performed offline. In recent years, deep learning-based interferometric analysis techniques have emerged, such as using a three-branch network to process the original image, difference image, and frequency domain image separately, and then regressing the displacement through feature stitching. However, such solutions still do not solve the problem of automated control: the network is only used as a post-processing tool, disconnected from hardware control, and cannot achieve closed-loop automation of "drive-acquisition-inference-feedback". Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies based on Michelson interferometers for measuring micro-displacement, such as low computational efficiency and lack of closed-loop automated control. This invention provides an interferometric micro-displacement measurement method and system based on multimodal fusion, which realizes closed-loop control and effectively improves measurement efficiency and accuracy.

[0005] To solve the above technical problems, the technical solution adopted by the present invention is as follows: A multimodal fusion-based interferometric micro-displacement measurement method is provided, comprising the following steps: S1. Data acquisition: Real-time acquisition of images of interference fringes and voltage data of piezoelectric ceramics; S2. Multimodal input construction layer: Real-time generation of three types of input modes for each frame of interference image, forming three types of modal images, each carrying different physical information; S3. Construction of a seven-dimensional time-series numerical feature vector: Statistical features are extracted from continuous frame sequences to construct the numerical features of the time series; S4. Construction of a three-branch feature extraction architecture, including: Construction of a local branch: Used for physical perception local feature extraction, employing three parallel single-channel convolutional neural networks to process the three types of modal images in step S2 respectively, and flattening and stitching the feature maps extracted by the three networks to form a local feature vector. Constructing a global branch: This branch is used for geometrically constrained global feature extraction. It takes three modal images as pseudo-RGB three-channel inputs, introduces a radial stripe attention mechanism into the deep learning model for image processing, and outputs a global context feature vector. Temporal branch construction: used for temporal physical feature extraction, employing an improved gated recurrent unit to process the seven-dimensional temporal numerical feature vector, outputting a temporal evolution feature vector. S5. Construct a dynamic modality-gated fusion layer. By building a lightweight evaluation subnetwork, the weights of each branch are calculated in real time based on the seven-dimensional temporal numerical feature vector, including global branch weights, local branch weights, and temporal branch weights. Then, the local feature vector, global context feature vector, and temporal evolution feature vector are fused according to their weights to obtain the fused feature vector. S6. Construct a physically coupled dual-output head, including a phase regression head and an order confidence head. The phase regression head is used to receive the fused feature vector and output the predicted phase after processing by a fully connected network. The order confidence head is used to receive the concatenated features of the fused feature vector and the temporal evolution feature vector and output the predicted interferometric order after processing by a fully connected network. S7. Construct a differentiable physical decoding layer, combining the information output from step S6 with the Michelson interferometry equation to construct an end-to-end differentiable displacement calculation layer to calculate the interferometric micro-displacement.

[0006] Furthermore, in step S2, the three types of input modes include: the original interference mode. It preserves the complete spatial distribution information of the interference fringes and directly records the light intensity information of the image at time t. After normalization, we get: Where x and y represent image coordinates; frame difference mode Capture dynamic changes in adjacent frames: Frequency domain representation of modes Extracting global frequency distribution and periodic features: Where F represents performing a Fast Fourier Transform, and u and v represent the transformed coordinates. It is a constant.

[0007] Furthermore, in step S3, constructing the seven-dimensional time-series numerical feature vector includes: normalized index: T represents the total sequence length; average intensity: H and W represent the height and width of the image, respectively; intensity variance: Number of horizontal interference fringe peaks: , () represents the peak value calculation function; Low-frequency energy ratio: M represents the spectral amplitude value; spectral centroid distance: , = ; , Represents the horizontal and vertical coordinates of the spectrum center. , The horizontal and vertical coordinates representing the centroid position; image entropy: δ is a constant. This represents the probability of the i-th gray level appearing in the image.

[0008] Furthermore, in the construction local branch of step S4, the first convolutional layer is initialized using a Gabor filter bank, and the parameters are dynamically generated based on the characteristics of the interference fringes:

[0009] In the formula, stripe period Adaptive calculation is performed based on the estimated fringe density; directions θ∈[0,π), with uniform sampling in 8 directions. When constructing the global branch, MobileViT is used as the backbone network, and three types of modal images are used as pseudo-RGB three-channel inputs. A radial stripe attention mechanism is introduced in the Vision Transformer. The image pixel coordinates are converted to polar coordinates (r, θ), and geometric priors are injected into the self-attention calculation.

[0010] In the formula, The radial and angular bias matrices are learnable, with α and β initialized to 0.1. The query vector Q represents the question of the current pixel, obtained by linear transformation of the input features, and is used to find relevance for other pixels. The key vector K represents the answer label of other pixels, obtained by linear transformation of the input features, and is used to match Q to calculate similarity. The value vector V represents the actual information content carried by other pixels. When Q matches a certain K, the corresponding V will be extracted and weighted and fused. The dimension of the key vector K is represented (i.e., the length of the feature vector); in the self-attention mechanism, Q and K are dot-producted QK. T It will produce a numerical value, the size of which is related to the vector dimension. Directly proportional. If The value is very large, so the dot product result will be very large, leading to... When the input to the function enters the gradient saturation region (near the maximum), the gradient vanishes, making training difficult. Therefore, dividing by... Scaling is applied to keep the dot product result moderate and ensure gradient stability. () represents an activation function that transforms any real vector into a probability distribution: .

[0011] In the build timeline branch, in the GRU update gate Explicitly introduce the displacement increment of the previous frame :

[0012] In the formula, γ is the learnable inertia weight parameter; σ represents the sigmoid nonlinear activation function, and its mathematical expression is: b z This represents a learnable bias vector.

[0013] Furthermore, in step S5, modal trust weights are first generated by constructing a lightweight evaluation subnetwork, which extracts the strength variance from the seven-dimensional time-series numerical feature vector. and image entropy Real-time calculation of the weights of each branch: w=Softmax(FC3([f3, f7]))∈ 4 In the formula, FC3 represents a three-layer fully connected network, outputting global branch weights. Local branch weights Time-series branch weights Then, dynamic weighted fusion is performed; the three feature vectors are fused according to their weights to obtain a fused feature vector. : .

[0014] Furthermore, in step S6, the phase regression head receives the fused feature vector. Processed via a fully connected network: φ pred =FC phase ( And use an activation function to constrain the output range: =π·tanh( The subwavelength shift corresponding to ) + π ∈ [0, 2π) is: d sub = λ / (4π)· The level confidence header receives and concatenates the feature vectors. and temporal evolution feature vector After processing via a fully connected network: p(m) = Softmax(FC) order ([ Output the probability distribution of 9 interference orders {-4, -3, -2, -1, 0, 1, 2, 3, 4}, and output the predicted interference order: =argmax m ∈{-4,...,4}p(m).

[0015] Further, in step S7, the interference micro-displacement is calculated using the following formula: d total =d sub + ·λ / 2 = λ / (4π)· + ·λ / 2.

[0016] This invention also provides an interferometric micro-displacement measurement system based on multimodal fusion, comprising: a data acquisition module for real-time acquisition of images of interference fringes and voltage data of piezoelectric ceramics; a multimodal input construction module for real-time generation of three types of input modes from each frame of the interferometric image, forming three types of modal images, each carrying different physical information; a seven-dimensional time-series numerical feature vector module for statistically analyzing features from a continuous frame sequence to construct the numerical features of the time series; and a three-branch feature extraction module, comprising: a local branch unit for physical perception local feature extraction, employing three parallel single-channel convolutional neural networks to process the three types of modal images in the multimodal input construction module respectively, and flattening and stitching the feature maps extracted by the three networks to form a local feature vector. Global Branch Unit: Used for geometrically constrained global feature extraction, it takes three modal images as pseudo-RGB three-channel input, introduces a radial stripe attention mechanism into the image processing deep learning model, and outputs a global context feature vector. Temporal branch unit: used for temporal physical feature extraction, employing an improved gated recurrent unit to process the seven-dimensional temporal numerical feature vector, outputting a temporal evolution feature vector. The dynamic modal gating fusion module is used to construct a lightweight evaluation subnetwork and calculate the weights of each branch in real time based on a seven-dimensional temporal numerical feature vector, including global branch weights, local branch weights, and temporal branch weights. Then, the local feature vector, global context feature vector, and temporal evolution feature vector are fused according to their weights to obtain a fused feature vector. The physically coupled dual-output head module includes a phase regression head unit and a level confidence head unit. The phase regression head unit receives the fused feature vector and outputs the predicted phase after processing by a fully connected network. The level confidence head unit receives the concatenated features of the fused feature vector and the temporal evolution feature vector and outputs the predicted interferometric level after processing by a fully connected network. The differentiable physical decoding module combines the information output by the physically coupled dual-output head module with the Michelson interferometry equation to construct an end-to-end differentiable displacement calculation layer and calculate the interferometric micro-displacement.

[0017] This invention also provides an interferometric micro-displacement measurement device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above; further comprising a data acquisition card for receiving digital instructions from the processor and converting them into analog voltage signals; a piezoelectric amplifier for amplifying the voltage signal output from the data acquisition card to the range required to drive the piezoelectric ceramic; a piezoelectric ceramic for receiving the amplified voltage signal and generating displacements at the nanometer to micrometer level; a Michelson interferometer for converting the minute displacements of the piezoelectric ceramic into observable changes in interference fringes; an image acquisition unit for capturing the interference fringes image generated by the Michelson interferometer in real time; and a voltage measurement unit for measuring the voltage actually applied across the piezoelectric ceramic in real time.

[0018] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described above.

[0019] Compared with the prior art, the beneficial effects of the present invention are: the interferometric micro-displacement measurement method and system based on multimodal fusion of the present invention effectively improves the measurement efficiency and accuracy. Attached Figure Description

[0020] Figure 1 is a flowchart of an interferometric micro-displacement measurement method based on multimodal fusion in one embodiment; Figure 2 is a schematic diagram of a neural network algorithm model in one embodiment; Figure 3 is a schematic diagram of a control system in one embodiment. Detailed Implementation

[0021] The present invention will be further described below with reference to specific embodiments. The accompanying drawings are for illustrative purposes only, representing schematic diagrams rather than actual physical objects, and should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some components in the drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions of the product. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0022] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0023] This embodiment is a first embodiment of an interferometric micro-displacement measurement method based on multimodal fusion. The method provided in this embodiment is used for the automated control, data acquisition, and real-time displacement calculation of a Michelson interferometer, and can be widely applied in cutting-edge scientific research fields such as precision metrology, material property testing, and microelectromechanical system calibration. As shown in Figures 1 and 2, the specific steps include: Step S1. Data acquisition: Real-time acquisition of interference fringe images and piezoelectric ceramic voltage data; the piezoelectric ceramic receives the amplified voltage, generating precise displacements at the nanometer to micrometer level; the Michelson interferometer converts the minute displacements of the piezoelectric ceramic into observable changes in interference fringes; the interference fringe images generated by the Michelson interferometer can be captured by a camera.

[0024] Step S2. Multimodal input construction layer: Generate three types of input modes in real time for each frame of interferometric image to form three types of modal images, each carrying different physical information.

[0025] The three types of input modes include: (1) primitive interference modes Preserve complete spatial distribution information of interference fringes and directly record the light intensity information of the image at time t. After normalization, we get:

[0026] Where x and y represent image coordinates; (2) Frame difference mode Capture dynamic changes between adjacent frames to highlight moving areas:

[0027] (3) Frequency domain representation of modes Extracting global frequency distribution and periodic features:

[0028] Where F represents performing a Fast Fourier Transform, and u and v represent the transformed coordinates. It is a constant.

[0029] This embodiment innovatively encodes interference physical quantities at the input layer: the original interferogram, the difference graph, and the frequency domain graph. This physical prior embedding enables the network to shift from a purely data-driven approach to a joint physics-data-driven approach.

[0030] Step S3. Construct a seven-dimensional time-series numerical feature vector: Statistical features are obtained from continuous frame sequences to form the numerical features of the time series.

[0031] Constructing a seven-dimensional time-series numerical feature vector includes: normalized index: T represents the total sequence length; average intensity: H and W represent the height and width of the image, respectively; intensity variance: Number of horizontal interference fringe peaks: , () represents the peak value calculation function; Low-frequency energy ratio: M represents the spectral amplitude value; spectral centroid distance: , = ; , Represents the horizontal and vertical coordinates of the spectrum center. , The horizontal and vertical coordinates representing the centroid position; image entropy: δ is a constant. This represents the probability of the i-th gray level appearing in the image.

[0032] Step S4. Construct a three-branch feature extraction architecture, including: Branch 1: Local branch, used for physical perception local feature extraction, employing three parallel single-channel convolutional neural networks to process the three modalities of images in Step S2 respectively. The key innovation lies in the physical constraint initialization of the convolutional kernels: the first convolutional layer is initialized using a Gabor filter bank, and the parameters are dynamically generated based on the characteristics of the interference fringes.

[0033] In the formula, stripe period Adaptive calculation is performed based on the estimated fringe density; directions θ∈[0,π), with uniform sampling in 8 directions. The network structure is a four-level convolution with a kernel size of 3×3, a stride of 2, and output channels of 32, 64, 128, and 256 respectively. Finally, the feature maps extracted by the three networks are flattened and concatenated to form a 128-dimensional local feature vector. .

[0034] To address the issues of slow convergence and poor performance with small samples caused by random initialization of convolutional kernels in existing technologies, this embodiment proposes dynamically generating Gabor filter banks to initialize the first-layer convolutional kernel. The filter frequency λ... f Adaptively matching the predicted fringe period, the direction θ is uniformly sampled across 8 directions, giving the network phase sensitivity from the very first layer. This design forces the extraction of fringe structure features rather than generic edges.

[0035] Branch 2: Global branch, used for global feature extraction with geometric constraints; MobileViT is used as the backbone network, and three types of modal images are used as pseudo-RGB three-channel inputs. A radial stripe attention mechanism is introduced in the Vision Transformer; the image pixel coordinates are converted to polar coordinates (r, θ), and geometric structure priors are injected into the self-attention calculation.

[0036] In the formula, This represents the dimension of the key vector K (i.e., the length of the feature vector). () represents an activation function that transforms any real vector into a probability distribution: .

[0037] The radial and angular bias matrices are learnable, with α and β initialized to 0.1. The query vector Q represents the information that the current pixel wants to focus on, which can be understood as the "question" of that pixel. It is obtained by linearly transforming the input features and is used to "find" relevance in other pixels. The key vector K represents the information that other pixels can provide, which can be understood as the "answer label" of other pixels. It is also obtained by linearly transforming the input features and is used to match and calculate similarity with Q. The value vector V represents the actual information content carried by other pixels, i.e., the real "answer". When Q matches a certain K, the corresponding V is extracted and weighted and fused; the final output is a 256-dimensional global context feature vector. .

[0038] In this embodiment, a polar coordinate geometric bias matrix is ​​introduced into the Vision Transformer module of MobileViT during the global feature extraction branch. and This mechanism utilizes the inherent centrosymmetric geometric prior of Michelson interference fringes to establish radial and angular distance relationships between pixels. This design significantly differs from the general-purpose visual backbone networks of existing technologies and represents a customized architectural innovation for interferometry tasks.

[0039] Branch 3: Temporal branch, used for temporal physical feature extraction. It employs an improved gated recurrent unit (GRU) to process the seven-dimensional temporal numerical feature vector. The core innovation lies in the embedding of motion continuity constraints. This is done in the GRU update gate... Explicitly introduce the displacement increment of the previous frame :

[0040] In the formula, γ is the learnable inertia weight parameter with an initial value of 0.1; σ represents the sigmoid nonlinear activation function, and its mathematical expression is: Its function is to compress the input value to the (0,1) interval and output a "gate value"; b z This represents a learnable bias vector with 128 dimensions, and is related to a linear transformation. The values ​​are summed to provide an independent offset for each hidden unit, preventing the network from overly relying on the mean of the input data. The hidden layer has a dimension of 128, and the input sequence length is... = 5 frames. The final output is a 128-dimensional temporal evolution feature vector. .

[0041] Step S5. Construct a dynamic modality-gated fusion layer. By constructing a lightweight evaluation subnetwork, the weights of each branch are calculated in real time based on the seven-dimensional temporal numerical feature vector, including global branch weights, local branch weights, and temporal branch weights. Then, the local feature vector, global context feature vector, and temporal evolution feature vector are fused according to their weights to obtain the fused feature vector. Specifically: First, modal trust weights are generated. Then, a lightweight evaluation subnetwork is constructed to extract the strength variance from the seven-dimensional temporal numerical feature vector. and image entropy Real-time calculation of the weights of each branch: w=Softmax(FC3([f3, f7]))∈ 4 In the formula, FC3 represents a three-layer fully connected network (2→16→4), outputting global branch weights. Local branch weights Time-series branch weights Then, dynamic weighted fusion is performed; the three feature vectors are fused according to their weights to obtain a fused feature vector. : .

[0042] This also includes sparsification regularization, which applies an L1 sparsity constraint λ to the weights w. sparse ·‖w‖1, where λ sparse =0.01, which encourages the network to prioritize sequential branching when there are high-quality stripes.

[0043] Existing technologies employ static feature stitching, which cannot adapt to dynamic changes in noise. This embodiment designs an adaptive weighted fusion based on stripe quality: a lightweight evaluation subnetwork is used to evaluate intensity variance. and image entropy The trust weight w of each branch is calculated in real time, and local branches, global branches, and time-series branches are dynamically weighted. When Q=(f4·f5) / (f3·f7) is lower than the threshold, the weight of the time-series branch is automatically increased to achieve adaptive mode switching in noisy environments.

[0044] Step S6. Construct a physically coupled dual-output head, including a phase regression head and a level confidence head. The phase regression head is used to receive the fused feature vector and output the predicted phase after processing by a fully connected network. The level confidence head is used to receive the spliced ​​features of the fused feature vector and the temporal evolution feature vector and output the predicted interferometric level after processing by a fully connected network.

[0045] Among them, the phase regression head receives the fused feature vector. After processing through a three-layer fully connected network (256→128→64): φ pred =FC phase ( And use the tanh activation function to constrain the output range: =π·tanh( The subwavelength shift corresponding to ) + π ∈ [0, 2π) is: d sub = λ / (4π)· The level confidence header receives and concatenates the feature vectors. and temporal evolution feature vector ([ (640-dimensional), processed by a three-layer fully connected network (256→128→5): p(m) = Softmax(FC) order ([ Output the probability distribution of 9 interference orders {-4, -3, -2, -1, 0, 1, 2, 3, 4}, and output the predicted interference order: =argmax m ∈{-4,...,4}p(m).

[0046] Step S7. Construct a differentiable physical decoding layer, combining the information output from step S6 with the Michelson interferometry equations to construct an end-to-end differentiable displacement calculation layer: d total =dsub + ·λ / 2 = λ / (4π)· + The λ / 2 layer provides explicit physical constraints during backpropagation, and gradients can be propagated to other layers of the network.

[0047] Unlike the black-box regression of existing technologies, this embodiment constructs an explicit interferometric equation decoding layer at the network end: d total =d sub + ·λ / 2 = λ / (4π)· + ·λ / 2. This layer normalizes the phase output of the phase regression head. The probability distribution p(m) of the confidence head output in the range [0, 2π) is combined with differentiable physical formulas to transform the neural network into a "gray box" model. The gradient can be backpropagated to each layer of the network to ensure that the prediction results strictly satisfy the interference principle.

[0048] In addition, in this embodiment, a multi-constraint joint loss function is constructed: the training objective function consists of four terms: L total =α·L disp +β·L order +γ·L orth +δ·L sparse Displacement loss: L disp = (1 / N)Σ(d i - i )², mean square error; d i Indicates the actual displacement; i Indicates the predicted displacement; level loss: L order = -(1 / N)ΣΣy i , k ·logp i,k Cross-entropy; y i , k This represents the actual label, with a value of 0 or 1; p i,k This represents the model's predicted probability, with values ​​in the interval (0,1). It also represents the network's confidence in predicting that the i-th sample belongs to the k-th level. The () function is obtained from the output of the level confidence header.

[0049] Orthogonal constraint: L orth = ‖h s h n ‖², Forced phase head hidden layer h s With the hidden layer h of the second level n Feature decoupling; sparse constraint: L sparse= ‖w‖1, to promote modality selection; preferred weighting coefficients: α=1.0, β=0.5, γ=0.05, δ=0.01.

[0050] In this embodiment, an incremental noise intensity curriculum and adversarial example generation are employed: Stage 1: For the first 30 rounds, weak Gaussian noise η=5 is added; Stage 2: From rounds 30 to 80, the noise is linearly increased to η=20; Stage 3: After round 80, the noise η=20-40 is randomly varied, and the adversarial generator is enabled to generate adversarial examples: I adv =I clean +ε·sign( I L disp ), ε=0.01. It is the gradient operator, and the subscript I indicates that the gradient is calculated over the input image. Therefore I L disp Represents the loss function L disp The gradient of each pixel in the input image is calculated, i.e., "how much does a tiny change in the value of each pixel in the image affect the displacement prediction error?" clean The term "clean" interferometric image refers to the original interferogram actually acquired by the system, without any adversarial perturbation; in adversarial training, it serves as the benchmark sample to be attacked, while it is the input to the model under normal conditions. adv This refers to "adversarial" samples, which are images generated by adding carefully designed, tiny perturbations to a clean image.

[0051] In this embodiment, a training strategy combining incrementally increasing noise intensity course scheduling (η: 5→20→40) with adversarial example generation is proposed. The adversarial generator adds worst-case perturbations along the gradient direction, enabling the model to proactively adapt to harsh environments.

[0052] This embodiment provides an interferometric micro-displacement measurement method based on multimodal fusion, achieving fully automated closed-loop control and significantly reducing hardware costs. Traditional systems rely on dual-wavelength light sources or wavelength scanning devices to resolve phase ambiguity, resulting in high hardware costs. This embodiment achieves a breakthrough through a neural network physical information embedding design: the input layer explicitly encodes the interferometric visibility map and phase gradient map, enabling the network to directly learn fringe quality and phase direction information; the physical intermediate quantities output by the network are decoded through differentiable interferometric equations, directly regressing the total displacement from the single-wavelength interferogram, eliminating the need for multi-wavelength hardware. This software-defined intelligent algorithm requires only a standard He-Ne laser, a common CCD camera, and a DAQ card for the system hardware, reducing hardware costs by more than 60%. Simultaneously, the automated closed-loop architecture eliminates manual operation, significantly reducing system complexity.

[0053] This embodiment achieves real-time inference through targeted optimization of the neural network architecture: Gabor filter bank physical initialization: the parameters of the first-layer convolutional kernel are dynamically matched with the period of the interference fringes to directly extract phase-sensitive features: Radial fringe attention mechanism, introducing polar coordinate geometric bias in MobileViT: Dynamic modal gating fusion: The quality assessment network calculates weights in real time based on image entropy and intensity variance, automatically skips redundant branch calculations, and optimizes the average inference time to <5ms; Through hardware-triggered synchronization and GPU acceleration, it can achieve real-time displacement calculation and closed-loop feedback control at 100 frames per second, which is four orders of magnitude more efficient than traditional algorithms.

[0054] The method provided in this embodiment has noise resistance and adaptive adjustment capabilities, expanding its applicability by more than three times. The integrated physical information dynamic fusion network significantly improves robustness through a multi-level adaptive mechanism: noise-aware dynamic gating: when image entropy increases (noise enhancement), the temporal branch weights automatically increase, and the system learns the noise evolution law from temporal statistical features; when the intensity variance is abnormal, the global branch weights decrease, reducing dependence on erroneous features. This is an online quality index Q-driven adaptive switching.

[0055] The directional selectivity of Gabor convolution kernels makes them insensitive to speckle noise, and the phase gradient map channel can still retain phase jump information when out of focus and blurred. The multimodal redundancy design ensures that at least one branch works stably in harsh environments.

[0056] Online adversarial fine-tuning: When the system accumulates 1000 frames of high-quality data (Q>0.5), adversarial training is automatically triggered. The generator adds worst-case perturbations along the gradient direction, allowing the model to continuously adapt to the aging of optical components. This mechanism can update the model without system downtime, expanding its applicability from clean laboratories to ordinary workshops, increasing environmental adaptability by more than 3 times.

[0057] The method provided in this embodiment is highly customizable and scalable, supporting multi-mode measurements. The invention's design stems from the decoupling characteristics of neural network architectures: an independent three-path design with clearly defined local branches (Gabor-CNN), global branches (MobileViT), and temporal branches (improved GRU), supporting hot switching and replacement. Users can enable / disable specific branches via configuration files to adapt to different speed-accuracy requirements. For example, during static measurements, the temporal branch can be disabled to save 40% of computational resources.

[0058] Programmable loss function: The loss weights α, β, γ, and δ can be freely adjusted, allowing users to enhance displacement accuracy or fuzziness elimination capabilities according to the scenario. For example, for large dynamic range measurements, β can be increased to 1.0 to ensure stability is prioritized in large-range displacement measurements.

[0059] Course countermeasures are adjustable: Noise upper limit η max The incremental step size of the course and the anti-disturbance strength ε are both openly configurable, supporting rapid adaptation to scenarios such as new material testing and different light source wavelengths. Compared with traditional fixed-function systems, this invention can be flexibly configured from scientific research experiments to industrial testing.

[0060] Example 2 This example is an embodiment of an interferometric micro-displacement measurement device. In this embodiment, it includes: a control unit: which can be a computer, having a memory, a processor, and a computer program and a control program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the method described in Example 1; a data acquisition card (DAQ) for receiving digital instructions from the processor and converting them into analog voltage signals; a piezoelectric amplifier for amplifying the voltage signal output from the data acquisition card to the range required to drive the piezoelectric ceramic; a piezoelectric ceramic for receiving the amplified voltage signal and generating displacements at the nanometer to micrometer level; a Michelson interferometer for converting the minute displacements of the piezoelectric ceramic into observable changes in interference fringes; an image acquisition unit, a CCD camera, for capturing the interference fringes image generated by the Michelson interferometer in real time; and a voltage measurement unit, a high-precision voltmeter, for measuring the voltage actually applied across the piezoelectric ceramic in real time.

[0061] The control program in the control unit generates digital control signals based on preset parameters (such as voltage scanning mode, step size, frequency, etc.) and sends them to the data acquisition card (DAQ). Simultaneously, the control program sends a synchronous trigger signal to the CCD camera and voltmeter, instructing them to begin acquisition. The DAQ receives the digital signal and converts it into a corresponding analog voltage. This analog voltage is amplified by a voltage amplifier and applied to the piezoelectric ceramic. Synchronously with the applied voltage, the voltmeter accurately measures the actual voltage across the piezoelectric ceramic and transmits this data back to the control unit. Under voltage drive, the piezoelectric ceramic produces a minute linear displacement, which is transmitted to the movable mirror of the Michelson interferometer, causing a change in the optical path difference between the two coherent beams, ultimately resulting in a change in the shape or position of the interference fringes. Upon receiving the trigger signal, the CCD camera captures dynamic images of the interference fringes at a set frame rate and exposure time, and transmits the image data back to the control unit in real time. The control unit receives and synchronously stores the interference image sequence from the CCD and the voltage data from the voltmeter. The built-in neural network algorithm (i.e., the method steps described in Example 1) takes the interferometric image sequence as the main input, combines it with synchronized voltage data, and calculates the displacement value corresponding to each moment directly and with high precision through the learned nonlinear mapping relationship.

[0062] The control program allows users to customize voltage control methods (such as linear scanning, stepped wave, sine wave, etc.) and image acquisition parameters (such as acquisition frequency, exposure time, image resolution, etc.), giving the system a high degree of flexibility and adaptability.

[0063] Example 3: This example is an embodiment of an interferometric micro-displacement measurement system based on multimodal fusion. Similar to Example 1, this example includes: a data acquisition module for real-time acquisition of images of interference fringes and voltage data of piezoelectric ceramics; a multimodal input construction module for real-time generation of three types of input modes from each frame of the interferometric image, forming three types of modal images, each carrying different physical information; a seven-dimensional time-series numerical feature vector module for statistically analyzing features from a continuous frame sequence to form the numerical features of the time series; and a three-branch feature extraction module, including: a local branch unit for physical perception local feature extraction, employing three parallel single-channel convolutional neural networks to process the three types of modal images in the multimodal input construction module, and flattening and stitching the feature maps extracted by the three networks to form a local feature vector. Global Branch Unit: Used for geometrically constrained global feature extraction, it takes three modal images as pseudo-RGB three-channel input, introduces a radial stripe attention mechanism into the image processing deep learning model, and outputs a global context feature vector. Temporal branch unit: used for temporal physical feature extraction, employing an improved gated recurrent unit to process the seven-dimensional temporal numerical feature vector, outputting a temporal evolution feature vector. The dynamic modal gating fusion module is used to construct a lightweight evaluation subnetwork and calculate the weights of each branch in real time based on a seven-dimensional temporal numerical feature vector, including global branch weights, local branch weights, and temporal branch weights. Then, the local feature vector, global context feature vector, and temporal evolution feature vector are fused according to their weights to obtain a fused feature vector. The physically coupled dual-output head module includes a phase regression head unit and a level confidence head unit. The phase regression head unit receives the fused feature vector and outputs the predicted phase after processing by a fully connected network. The level confidence head unit receives the concatenated features of the fused feature vector and the temporal evolution feature vector and outputs the predicted interferometric level after processing by a fully connected network. The differentiable physical decoding module combines the information output by the physically coupled dual-output head module with the Michelson interferometry equation to construct an end-to-end differentiable displacement calculation layer and calculate the interferometric micro-displacement.

[0064] Example 4 This example is a computer-readable storage medium storing a computer program thereon, the computer program being executed by a processor using the steps of the method described in Example 1.

[0065] In the specific implementation of the above embodiments, the technical features can be combined in any non-contradictory way. For the sake of brevity, not all possible combinations of the above technical features are described. However, as long as the combination of these technical features is not contradictory, it should be considered to be within the scope of this specification.

[0066] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for measuring interferometric micro-displacement based on multimodal fusion, characterized in that, Includes the following steps: S1. Data Acquisition: Real-time acquisition of images of interference fringes and voltage data of piezoelectric ceramics; S2. Multimodal input construction layer: Generate three types of input modes in real time for each frame of interferometric image, forming three types of modal images, each carrying different physical information; S3. Construct a seven-dimensional time-series numerical feature vector: Statistically analyze features from continuous frame sequences to construct the numerical features of the time series; S4. Construct a three-branch feature extraction architecture, including: constructing a local branch: for physical perception local feature extraction, using three parallel single-channel convolutional neural networks to process the three modalities of the images in step S2 respectively, and flattening and concatenating the feature maps extracted by the three networks to form a local feature vector. Constructing a global branch: This branch is used for geometrically constrained global feature extraction. It takes three modal images as pseudo-RGB three-channel inputs, introduces a radial stripe attention mechanism into the deep learning model for image processing, and outputs a global context feature vector. Temporal branch construction: used for temporal physical feature extraction, employing an improved gated recurrent unit to process the seven-dimensional temporal numerical feature vector, outputting a temporal evolution feature vector. S5. Construct a dynamic modality-gated fusion layer. By building a lightweight evaluation subnetwork, the weights of each branch are calculated in real time based on the seven-dimensional temporal numerical feature vector, including global branch weights, local branch weights, and temporal branch weights. Then, the local feature vector, global context feature vector, and temporal evolution feature vector are fused according to their weights to obtain the fused feature vector. S6. Construct a physically coupled dual-output head, including a phase regression head and an order confidence head. The phase regression head is used to receive the fused feature vector and output the predicted phase after processing by a fully connected network. The order confidence head is used to receive the concatenated features of the fused feature vector and the temporal evolution feature vector and output the predicted interferometric order after processing by a fully connected network. S7. Construct a differentiable physical decoding layer, combining the information output from step S6 with the Michelson interferometry equation to construct an end-to-end differentiable displacement calculation layer to calculate the interferometric micro-displacement.

2. The interferometric micro-displacement measurement method based on multimodal fusion according to claim 1, characterized in that, In step S2, the three types of input modes include: the original interference mode. It preserves the complete spatial distribution information of the interference fringes and directly records the light intensity information of the image at time t. After normalization, we get: Where x and y represent image coordinates; frame difference mode Capture dynamic changes in adjacent frames: Frequency domain representation of modes Extracting global frequency distribution and periodic features: Where F represents performing a Fast Fourier Transform, and u and v represent the transformed coordinates. It is a constant.

3. The interferometric micro-displacement measurement method based on multimodal fusion according to claim 2, characterized in that, In step S3, constructing the seven-dimensional time-series numerical feature vector includes: normalized index: T represents the total sequence length; average intensity: H and W represent the height and width of the image, respectively; intensity variance: Number of horizontal interference fringe peaks: ， () represents the peak value calculation function; Low-frequency energy ratio: M represents the spectral amplitude value; spectral centroid distance: ， = ； 、 Represents the horizontal and vertical coordinates of the spectrum center. 、 The horizontal and vertical coordinates representing the centroid position; image entropy: δ is a constant. This represents the probability of the i-th gray level appearing in the image.

4. The interferometric micro-displacement measurement method based on multimodal fusion according to claim 1, characterized in that, In the construction of the local branch in step S4, the first convolutional layer is initialized using a Gabor filter bank, and the parameters are dynamically generated based on the characteristics of the interference fringes. In the formula, stripe period Adaptive calculation is performed based on the estimated fringe density; directions θ∈[0,π), with uniform sampling in 8 directions. ； When constructing the global branch, MobileViT is used as the backbone network, and three types of modal images are used as pseudo-RGB three-channel inputs. A radial stripe attention mechanism is introduced in the Vision Transformer. The image pixel coordinates are converted to polar coordinates (r, θ), and geometric priors are injected into the self-attention calculation. In the formula, The radial and angular bias matrices are learnable, with α and β initialized to 0.

1. The query vector Q represents the problem of the current pixel, which is obtained by linearly transforming the input features and is used to find correlations for other pixels. The key vector K represents the answer label of other pixels, which is obtained by linear transformation of the input features and is used to calculate the similarity with Q; the value vector V represents the actual information content carried by other pixels. When Q is successfully matched with a certain K, the corresponding V will be extracted and weighted and fused. This represents the dimension of the key vector K. () denotes the activation function; in the construction timeline branch, in the GRU update gate Explicitly introduce the displacement increment of the previous frame ： In the formula, γ is the learnable inertia weight parameter; σ represents the sigmoid nonlinear activation function, and its mathematical expression is: b z This represents a learnable bias vector.

5. The interferometric micro-displacement measurement method based on multimodal fusion according to claim 3, characterized in that, In step S5, modal confidence weights are first generated by constructing a lightweight evaluation subnetwork, which extracts the strength variance from the seven-dimensional time-series numerical feature vector. and image entropy Real-time calculation of the weights of each branch: w=Softmax(FC3([f3, f7]))∈ 4 In the formula, FC3 represents a three-layer fully connected network, outputting global branch weights. Local branch weights Time-series branch weights Then, dynamic weighted fusion is performed; the three feature vectors are fused according to their weights to obtain a fused feature vector. ： 。 6. The interferometric micro-displacement measurement method based on multimodal fusion according to claim 5, characterized in that, In step S6, the phase regression head receives the fused feature vector. Processed via a fully connected network: φ pred =FC phase ( And use an activation function to constrain the output range: =π·tanh( The subwavelength shift corresponding to ) + π ∈ [0, 2π) is: d sub = λ / (4π)· The level confidence header receives and splices the feature vectors. and temporal evolution feature vector After processing via a fully connected network: p(m) = Softmax(FC) order ([ Output the probability distribution of 9 interference orders {-4, -3, -2, -1, 0, 1, 2, 3, 4}, and output the predicted interference order: =argmax m ∈{-4,...,4}p(m)。 7. The interferometric micro-displacement measurement method based on multimodal fusion according to claim 6, characterized in that, In step S7, the interference micro-displacement is calculated using the following formula: d total =d sub + ·λ / 2 = λ / (4π)· + ·λ / 2。 8. An interferometric micro-displacement measurement system based on multimodal fusion, characterized in that, include: Data acquisition module: used to acquire images of interference fringes and voltage data of piezoelectric ceramics in real time; Multimodal input construction module: used to generate three types of input modes in real time for each frame of interferometric image, forming three types of modal images, each carrying different physical information; Seven-dimensional time series numerical feature vector module: used to statistically analyze features from continuous frame sequences to form the numerical features of the time series; The three-branch feature extraction module includes: a local branch unit for extracting local features from physical perception. This unit employs three parallel single-channel convolutional neural networks to process the three modalities of the multimodal input construction module, respectively. The feature maps extracted by the three networks are then flattened and concatenated to form a local feature vector. Global Branch Unit: Used for geometrically constrained global feature extraction, it takes three modal images as pseudo-RGB three-channel input, introduces a radial stripe attention mechanism into the image processing deep learning model, and outputs a global context feature vector. Temporal branch unit: used for temporal physical feature extraction, employing an improved gated recurrent unit to process the seven-dimensional temporal numerical feature vector, outputting a temporal evolution feature vector. The dynamic modal gating fusion module is used to construct a lightweight evaluation subnetwork and calculate the weights of each branch in real time based on a seven-dimensional temporal numerical feature vector, including global branch weights, local branch weights, and temporal branch weights. Then, the local feature vector, global context feature vector, and temporal evolution feature vector are fused according to their weights to obtain a fused feature vector. The physically coupled dual-output head module includes a phase regression head unit and a level confidence head unit. The phase regression head unit receives the fused feature vector and outputs the predicted phase after processing by a fully connected network. The level confidence head unit receives the concatenated features of the fused feature vector and the temporal evolution feature vector and outputs the predicted interferometric level after processing by a fully connected network. The differentiable physical decoding module combines the information output by the physically coupled dual-output head module with the Michelson interferometry equation to construct an end-to-end differentiable displacement calculation layer and calculate the interferometric micro-displacement.

9. An interferometric micro-displacement measuring device, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the method according to any one of claims 1 to 7; it also includes a data acquisition card for receiving digital instructions from the processor and converting them into analog voltage signals; a piezoelectric amplifier for amplifying the voltage signal output from the data acquisition card to the range required to drive the piezoelectric ceramic; a piezoelectric ceramic for receiving the amplified voltage signal and generating displacements at the nanometer to micrometer level; a Michelson interferometer for converting the minute displacements of the piezoelectric ceramic into observable changes in interference fringes; an image acquisition unit for capturing interference fringes images generated by the Michelson interferometer in real time; and a voltage measurement unit for measuring the voltage actually applied across the piezoelectric ceramic in real time.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images