Liquid crystal elastomer deformation state real-time recognition method based on multi-modal fusion

Abstract

The application belongs to the technical field of liquid crystal elastomer deformation measurement, and relates to a liquid crystal elastomer deformation state real-time identification method based on multi-modal fusion.The application synchronously collects visual, electrical and optical three-modal signals, introduces a physical prior model in the feature processing stage, converts the micro molecular orientation features of the optical mode into macro deformation correction parameters by using the mapping relationship between the experimentally calibrated liquid crystal elastomer molecular orientation order parameter and the macro tensile rate, optimizes the overall surface deformation features extracted by the visual mode, and makes up for the deficiency that the visual mode only reflects surface information.Finally, the application integrates the multi-modal features through an LSTM fusion network, outputs comprehensive deformation state parameters, covers the overall space range by means of the visual mode, introduces internal micro and stress information through the electrical and optical modes and the physical prior, breaks through the limitation of local strain information, and realizes real-time and comprehensive identification of the overall macro deformation and internal stress distribution of the liquid crystal elastomer.

Description

Technical Field

[0001] This application belongs to the field of liquid crystal elastomer deformation measurement technology, and more specifically, relates to a method for real-time identification of liquid crystal elastomer deformation state based on multimodal fusion. Background Technology

[0002] Liquid crystal elastomers (LCEs) are a class of smart soft materials that combine the anisotropy of liquid crystal molecules with the macroscopic flexibility of elastomers. Their molecular chains form a three-dimensional network through cross-linking. Under external stimuli (such as temperature, light, electricity, magnetic field, etc.), the liquid crystal units can undergo orientation rearrangement, thereby driving the material to produce reversible and programmable macroscopic deformations (such as bending, stretching, torsion, folding, etc.). With their excellent deformation response sensitivity, large strain output (up to 300% or more) and biocompatibility, LCEs have shown great application potential in fields such as flexible actuators, soft robots, wearable devices, and biomedical implants.

[0003] To achieve precise control, performance optimization, and safety monitoring of LCE devices, real-time and comprehensive deformation state identification has become a core requirement. Specifically, the deformation state of LCE needs to include three key types of information: (1) macroscopic deformation parameters (such as bending angle, elongation, torsion angle, and other overall geometric shapes) to determine whether the device's actions meet expectations; (2) internal stress state (such as stress distribution and maximum stress location) to assess the material's mechanical load and fatigue life; and (3) molecular orientation changes (such as the orientation degree and orientation direction of liquid crystal units) to reveal the microscopic mechanism of deformation (such as whether it is driven by liquid crystal phase transition or orientation rearrangement) and to correlate macroscopic performance with microscopic structure. Therefore, deformation identification needs to cover macroscopic-mesoscopic-microscopic multi-scale information to provide complete feedback for the design and application of LCE devices.

[0004] Existing LCE deformation identification methods mainly rely on a single sensing mode, such as attaching strain gauges, fiber optic gratings, or other sensors to the LCE surface or embedding them inside, and inferring deformation from local strain signals. These methods have high accuracy, but are limited by the spatial distribution of sensors and can only obtain local strain information at the contact point. They cannot reflect the overall macroscopic deformation field or internal stress distribution, and contact measurement may interfere with the natural deformation behavior of the LCE. Summary of the Invention

[0005] This invention provides a real-time identification method for the deformation state of liquid crystal elastomers based on multimodal fusion, which aims to solve the technical problem that existing technologies are limited by the spatial distribution of sensors, can only obtain local strain information at contact points, and cannot reflect the overall macroscopic deformation field or internal stress distribution.

[0006] A real-time identification method for the deformation state of liquid crystal elastomers based on multimodal fusion includes the following steps:

[0007] S1. Synchronously acquire the original visual, electrical, and optical three-modal signals during the deformation process of the liquid crystal elastomer;

[0008] S2. Denoise and time-align the acquired visual, electrical, and optical three-modal raw signals respectively;

[0009] S3. Extract visual, electrical, and optical features from the denoised and time-aligned visual, optical, and electrical tri-modal signals;

[0010] S4. Calculate the confidence weight for each mode, then introduce the physical prior model, and through the mapping relationship between the molecular orientation order parameter of the liquid crystal elastomer and the macroscopic stretching rate calibrated by the experiment, transform the molecular orientation characteristics of the optical mode into macroscopic deformation correction parameters to optimize the features extracted from the visual mode.

[0011] S5. Input the corrected visual modal features, extracted electrical features, and optical features into the LSTM fusion network to obtain the comprehensive deformation state parameters.

[0012] This invention overcomes the spatial distribution limitations of a single contact sensor by simultaneously acquiring visual, electrical, and optical signals, achieving collaborative acquisition of non-contact overall information and contact internal state information. In the feature processing stage, a physical prior model is introduced. Utilizing the experimentally calibrated mapping relationship between the molecular orientation order parameter of the liquid crystal elastomer and the macroscopic stretching rate, the microscopic molecular orientation features of the optical mode are transformed into macroscopic deformation correction parameters. This optimizes the overall surface deformation features extracted by the visual mode, compensating for the deficiency of the visual mode in only reflecting surface information, and realizing the correlation between the microscopic internal state and the macroscopic overall deformation. Finally, an LSTM fusion network integrates multimodal features to output comprehensive deformation state parameters. This approach leverages the visual mode to cover the overall spatial range and introduces internal microscopic and stress information through electrical, optical, and physical priors, overcoming the limitations of local strain information and achieving real-time and comprehensive identification of the overall macroscopic deformation and internal stress distribution of the liquid crystal elastomer.

[0013] Preferably, the noise reduction includes the following steps:

[0014] Visual signal denoising: Guided by the original image itself, the brightness relationship of each region is calculated within a predetermined pixel window; by balancing pixel differences and smoothing requirements, a linear fitting result is output for each window, and the average value of all overlapping windows is taken to obtain the denoised visual signal.

[0015] Electrical signal denoising: The true resistance value and the temperature drift are combined to form a state vector. The current value is predicted based on the state of the previous moment. By comparing the deviation between the predicted value and the actual measured value, the noise tracking sensitivity and observation weight are dynamically adjusted to obtain a stable resistance sequence after drift compensation.

[0016] Optical signal denoising: Generate sine and cosine reference signals that perfectly match the laser frequency. Multiply the original optical signal with the sine and cosine reference signals respectively. After low-pass filtering to remove high-frequency noise, extract the DC components of the sine and cosine reference signals. Calculate the light intensity change and polarization angle shift based on the orthogonal component relationship to obtain pure light intensity change and polarization angle change signals.

[0017] Preferably, the extraction of the visual features includes the following steps:

[0018] The U-Net network is used to segment the LCE region in the image to generate a mask, and the bending angle, stretching ratio and twist angle are calculated based on the mask contour.

[0019] Meanwhile, deep texture features of the LCE region are extracted using the ResNet-18 network to generate multidimensional visual feature vectors.

[0020] Preferably, the extraction of the electrical features includes the following steps:

[0021] The rate of change of resistance is obtained based on the considerable difference between the current resistance and the initial resistance;

[0022] The main peak frequency in the signal spectrum is extracted by Fourier transform and used as the peak frequency of the dynamic resistance.

[0023] The Daubechies 4 mother wavelet was used to decompose the 1kHz resistance signal into 5 layers with 16 points per window. The energy of the wavelet coefficients and the energy of the approximation coefficients of each decomposition layer were calculated. The obtained energy values ​​were spliced ​​into a 128-dimensional vector, which, together with the resistance change rate and the peak frequency of the dynamic resistance, constituted the electrical feature vector.

[0024] Preferably, the extraction of the optical features includes the following steps:

[0025] The degree of molecular orientation order is calculated based on the change in polarization angle, and the average molecular orientation angle is calculated by combining the light intensity difference in orthogonal directions.

[0026] The dynamic statistical characteristics of the phase orientation order and molecular orientation angle are calculated within a window of 16 sampling points to obtain an optical feature vector containing molecular orientation parameters and dynamic statistical characteristics.

[0027] Preferably, the confidence weight calculation includes the following steps:

[0028] Visual confidence weights: The Sobel operator is applied to each frame of visual image to obtain the gradient magnitude, the gradient magnitude variance of the deformed region is calculated, and Canny edge detection is performed on the deformed region. The edge integrity score is obtained by the proportion of complete edges. Then, the gradient magnitude variance and the edge integrity score are weighted and summed using experimentally calibrated coefficients to obtain the visual base weights. The visual base weights are mapped to a predetermined range to obtain the visual weights.

[0029] Electrical confidence weight: Perform a fast Fourier transform on the resistance signal, calculate the ratio of fundamental frequency power to out-of-band noise and convert it into signal-to-noise ratio. If the signal-to-noise ratio is greater than or equal to a predetermined threshold, the electrical confidence weight is set to 0.7; if it is less than the predetermined threshold, it is set to 0.3.

[0030] Optical confidence weight: Within a sliding window containing 16 sampling points, the sample standard deviation of the polarization angle is calculated. If the standard deviation is less than a predetermined threshold, the optical weight is set to 0.6; if it is greater than or equal to the predetermined threshold, it is set to 0.3.

[0031] Normalized weights: Based on the visual confidence weight, electrical confidence weight, and optical confidence weight, normalized visual weight, normalized electrical weight, and normalized optical weight are obtained by normalization.

[0032] Preferably, the features for optimizing visual modality extraction include the following steps:

[0033] Based on the molecular-macroscopic correlation model, molecular orientation parameters are converted into equivalent stretching rates. Then, according to the normalized weight ratio of vision and optics, the stretching rates extracted from the original vision are weighted and fused. The fusion weight is dynamically adjusted based on the normalized visual weight and the normalized optical weight, so that more original visual features are retained when the normalized visual weight increases, and molecular-level physical correction is enhanced when the normalized visual weight decreases. Based on this, the corrected visual feature vector is obtained.

[0034] Preferably, the comprehensive deformation state parameters are based on the following steps:

[0035] The modal features are linearly superimposed with normalized weights to obtain a fused feature sequence. The fused feature sequence is then input into a two-layer LSTM network. The two-layer LSTM uses the historical features from the previous n milliseconds as the temporal context to process the weighted feature sequence frame by frame. The processed result is input into a fully connected layer to output the comprehensive deformation state parameters.

[0036] The beneficial effects of this invention include:

[0037] This invention overcomes the spatial distribution limitations of a single contact sensor by simultaneously acquiring visual, electrical, and optical signals, achieving collaborative acquisition of non-contact overall information and contact internal state information. In the feature processing stage, a physical prior model is introduced. Utilizing the experimentally calibrated mapping relationship between the molecular orientation order parameter of the liquid crystal elastomer and the macroscopic stretching rate, the microscopic molecular orientation features of the optical mode are transformed into macroscopic deformation correction parameters. This optimizes the overall surface deformation features extracted by the visual mode, compensating for the deficiency of the visual mode in only reflecting surface information, and realizing the correlation between the microscopic internal state and the macroscopic overall deformation. Finally, an LSTM fusion network integrates multimodal features to output comprehensive deformation state parameters. This approach leverages the visual mode to cover the overall spatial range and introduces internal microscopic and stress information through electrical, optical, and physical priors, overcoming the limitations of local strain information and achieving real-time and comprehensive identification of the overall macroscopic deformation and internal stress distribution of the liquid crystal elastomer. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 An overall flowchart is provided for embodiments of the present invention.

[0040] Figure 2 A feature processing flowchart provided for embodiments of the present invention.

[0041] Figure 3 This is a schematic diagram of the rangefinder assembly position provided for an embodiment of the present invention. Detailed Implementation

[0042] To make the technical problems, technical solutions, and beneficial effects to be solved by this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0043] See Figure 1 As shown, the real-time identification method for the deformation state of liquid crystal elastomers based on multimodal fusion includes the following steps:

[0044] S1. Synchronously acquire the original visual, electrical, and optical three-modal signals during the deformation process of the liquid crystal elastomer;

[0045] In this embodiment, a miniature flexible camera is used, which is fixed to the edge of the LCE (liquid crystal elastomer) deformation area by a bracket. The angle between the lens optical axis and the LCE surface normal is less than 15 degrees, ensuring that a clear two-dimensional image sequence of the deformation area is captured.

[0046] A flexible silver nanowire electrode array (e.g., 10×10 arrays with a spacing of 0.5 mm) was formed in an LCE precursor solution using inkjet printing technology. Subsequently, UV curing was used to crosslink the LCE, ensuring no interfacial delamination between the electrode and the LCE matrix. Resistance changes were collected based on the flexible silver nanowire electrode. The resistance change is caused by variations in the internal molecular chains and the spacing of the silver nanowire mesh during LCE deformation, and the relationship is expressed as follows:

[0047] ;

[0048] In the formula: This represents the change in resistance. Indicates the initial resistance. Indicates the current resistance; Indicates the stress sensitivity coefficient; Indicates internal stress; The strain sensitivity coefficient is dimensionless and depends on the orientation of the AgNW electrodes; b≈5 for axial tension. Indicates strain;

[0049] Based on the above formula, it can be seen that the relative change in resistance is affected by both stress and strain. Therefore, the electrical signal can comprehensively reflect the internal stress state of the LCE.

[0050] The optical signal is acquired through a non-contact polarization optical system (including a laser source, polarizer, LCE analyzer, and detector) to collect the polarization state and intensity changes of the LCE reflected light, reflecting the changes in the orientation of liquid crystal molecules. The relationship between the polarization state change and the molecular orientation is as follows:

[0051] The initial orientation angle of the liquid crystal molecules in LCE is (When there is no change, it is arranged along the surface.) =0°), during deformation, molecules orient themselves along the stress direction, and the orientation angle becomes 0°. (such as when stretching) (Increase to align with the stretching direction); changes in molecular orientation alter the birefringence properties of the LCE, affecting the polarization angle of the reflected light. With molecular orientation angle satisfy:

[0052] ;

[0053] In the formula: This indicates the change in the polarization angle of the reflected light; This represents the change in molecular orientation angle;

[0054] Simultaneously, the increased molecular orientation order leads to a change in reflectivity, with the change in light intensity being:

[0055] ;

[0056] In the formula: This indicates the change in the intensity of reflected light; Indicates strong deformation over time; Indicates the initial light intensity; Indicates the light intensity coefficient; The parameter representing the degree of molecular orientation order (0≤S≤1, , (This represents the average value).

[0057] In this embodiment, the optical signals are the changes in polarization angle and light intensity, which are obtained by rotating the analyzer and synchronously acquiring the light intensity. The sampling rate is 1kHz and the response time is less than 1μs.

[0058] Secondly, in this embodiment, in order to ensure subsequent timing alignment, the hardware synchronization of trimodal data acquisition is achieved through trigger signal control. That is, based on the frame synchronization signal of the visual camera (60fps, period ~16.7ms), a synchronization pulse (rising edge trigger) is generated by the FPGA to control the sampling start time of the electrical ADC and the optical detector, ensuring the trigger synchronization of the trimodal raw signals on the time axis, and providing a hardware foundation for subsequent timing alignment.

[0059] S2. Denoise and time-align the acquired visual, electrical, and optical three-modal raw signals respectively;

[0060] Visual signal denoising: Based on the local linear relationship between the guide image and the output image, noise is suppressed while preserving edge features, solving the edge problems of traditional Gaussian filtering, as detailed below:

[0061] Let the input noisy image be I, and the guiding image be... (Here, G=I is chosen, i.e., self-guided), the filter window radius is r=5 (5×5 local window, balancing denoising effect and real-time performance), and the regularization parameter... Local window for each pixel k Assuming the output image With guide image Satisfies a linear relationship:

[0062] ;

[0063] In the formula: This represents the grayscale value of the output image at pixel i; This represents the grayscale value of the guiding image at pixel i; and For window The linear coefficients within the range are obtained by minimizing the cost function:

[0064] ;

[0065] Solving the above minimized cost function yields the coefficients:

[0066] ;

[0067] In the formula: Display window Internal input image The average pixel value; Display window Internal guidance image The mean; Display window Inside and covariance; Display window Inside The variance;

[0068] The final filtered output is the average value of the linear model within the window:

[0069] ;

[0070] In the formula: Indicates the index of the pixel currently being processed; Indicates the pixel index within a local window;

[0071] Based on the above steps, a smooth image with edge preservation is obtained.

[0072] Electrical signal denoising: Adaptive Kalman filtering is used for denoising. This involves constructing a state vector from the true resistance value and the temperature drift, predicting the current value based on the previous state, and dynamically adjusting the noise tracking sensitivity and observation weights by comparing the deviation between the predicted and actual measured values. This yields a drift-compensated stable resistance sequence. The expressions for dynamically adjusting the noise tracking sensitivity and observation weights are as follows:

[0073] ;

[0074] ;

[0075] In the formula: Let k represent the residual at time k; Indicates the measured value; This represents the predicted value from the previous state. This represents the residual covariance at time k; Represents the observation matrix; Let represent the prediction error covariance matrix at time k based on the previous state; This represents the variance of the observation noise at time k; Represents the process covariance matrix at time k; Represents the initial process noise covariance matrix; Indicates the residual weighting coefficient; superscript Indicates transpose;

[0076] Based on the above formula, it can be seen that the greater the drift, the better. The larger the value, the faster it allows for tracking changes;

[0077] ;

[0078] In the formula: This represents the initial observation noise variance; This represents the residual weighting coefficient;

[0079] Based on the above formula, it can be seen that the greater the noise, the... The larger the value, the lower the observation weight.

[0080] Optical signal denoising: Extracting the change in polarization angle of the 650nm laser through coherent detection of the reference signal and the input signal. and changes in light intensity To suppress weak signals and ambient light interference beyond 10kHz, the following measures are taken:

[0081] Mixed signal output by the detector ,in Indicates the signal amplitude; Indicates phase; Indicates the reference angular frequency; Indicates bandwidth noise; This indicates the corresponding time index;

[0082] In generating orthogonal references with the same frequency: and , This represents the cosine component in a quadrature reference signal with the same frequency. This represents the sinusoidal component in a quadrature reference signal with the same frequency.

[0083] mixing: After being filtered out by a low-pass filter Components, output DC signal Similarly and Mixing, output ;

[0084] Calculate the change in light intensity based on the two DC components. and the change in polarization angle :

[0085] ;

[0086] .

[0087] In this embodiment, taking the visual signal acquisition time as the reference, let the acquisition time of the i-th frame of visual image be . ,in At the initial moment, (Frame rate of 60fps) The index represents the visual frame; the time of the j-th sampling point of the electrical and optical signals is... , Indices representing sampling point indices for electrical and optical signals. This represents the timestamp of the j-th sampling point of the electrical signal. =1ms is the sampling period for electrical and optical signals.

[0088] For each visual frame Determine its time window And find the range of sampling point indices for electrical and optical signals within that window: Number of sampling points within the window: Rounded down to 16;

[0089] Then, the average of the N sampling points within the window is taken as the electrical and optical signal values ​​corresponding to the aligned visual signal of the i-th frame:

[0090] ;

[0091] In the formula: Represents the sampled value after denoising of an optical or electrical signal; This represents the aligned high-frequency signal value corresponding to the i-th frame visual image. By averaging the N original sampling points within the window, the temporal resolution of the high-frequency signal (1kHz) is reduced to be consistent with that of the visual signal (60fps), thus achieving temporal axis unification.

[0092] S3. Extract visual, electrical, and optical features from the denoised and time-aligned visual, optical, and electrical tri-modal signals;

[0093] The visual feature extraction includes the following steps:

[0094] The U-Net network is used to segment each frame of the image, locate the LCE deformation region, and output a binary mask of the LCE region;

[0095] The bending angle, stretching ratio, and torsion angle are calculated based on the output binary mask:

[0096] ;

[0097] ;

[0098] ;

[0099] In the formula: Indicates the bending angle of the LCE; Represents the tangent vectors at both ends of the LCE; Indicates the elongation of LCE; Indicates the skeleton length of the LCE in the current frame; Indicates the initial length of the LCE; Indicates the torsion angle of the LCE; This represents the rotational offset angle of adjacent local regions along the LCE skeleton direction; This represents the total number of local regions along the skeleton direction (divided evenly according to the skeleton length, such as taking one local region for every 10 pixels).

[0100] Then, the spatial boundary of the LCE region is located based on the output binary mask to obtain the cropped LCE region image. The cropped region image is used as the input of the ResNet-18 model, and a 256-dimensional feature vector is output.

[0101] The electrical feature extraction includes the resistance change rate, the peak frequency of the dynamic resistance spectrum, and the wavelet time-frequency feature vector; the resistance change rate is obtained by dividing the difference between the current resistance value and the initial resistance value by the initial resistance value; the peak frequency of the dynamic resistance spectrum is obtained by extracting the peak frequency of the spectrum after performing a fast Fourier transform on the resistance signal; the wavelet time-frequency feature vector is obtained by performing a 5-level decomposition based on the Daubechies wavelet and calculating the energy distribution of each frequency band.

[0102] The optical feature extraction includes the extraction of molecular orientation order parameters, molecular orientation angles, and optical feature vectors. The orientation order parameters... The calculation formula is as follows:

[0103] ;

[0104] In the formula: This indicates the change in the polarization angle of the emitted light; This represents the threshold for the maximum polarization angle change;

[0105] The molecular orientation order parameter The calculation formula is as follows:

[0106] ;

[0107] In the formula: This represents the change in light intensity along the parallel polarization direction; This represents the change in light intensity in the direction perpendicular to the polarization direction.

[0108] The optical feature vector is extracted as a dynamic statistical orientation orderliness parameter. The mean and standard deviation, and the molecular orientation order parameter. The mean and peak frequencies.

[0109] S4. Calculate the confidence weight for each mode, then introduce the physical prior model, and through the mapping relationship between the molecular orientation order parameter of the liquid crystal elastomer and the macroscopic stretching rate calibrated by the experiment, transform the molecular orientation characteristics of the optical mode into macroscopic deformation correction parameters to optimize the features extracted from the visual mode.

[0110] See appendix Figure 2 As shown, in this embodiment, the reliability of each mode is evaluated in real time based on signal quality indicators, and dynamic weights are assigned to adaptively adjust the mode contribution under environmental interference to ensure fusion robustness. The specific steps are as follows:

[0111] Visual confidence weight calculation: This is based on the quality of the visual signal, which includes image sharpness and occlusion level. Image sharpness is evaluated using gradient magnitude variance, and occlusion level is evaluated using edge integrity. The weights are dynamically adjusted within the range of 0.2-0.8. The specific expression is as follows:

[0112] , ;

[0113] In the formula: Indicates the variance of gradient magnitude; Indicates the edge integrity score; and Indicates the weighting coefficient; This represents the baseline value for visual confidence. Indicates the final visual confidence weight;

[0114] The calculation of visual confidence weights is performed on each frame of visual data points (60fps), using the image sequence processed in step S2 to obtain the final visual confidence weights.

[0115] Electrical confidence weight calculation: based on the signal-to-noise ratio of electrical signals This reflects temperature drift and noise interference.

[0116] ;

[0117] In the formula: The electrical confidence weight is represented; the signal-to-noise ratio is calculated based on the fast Fourier transform, taking the ratio of the signal power spectrum to the noise power spectrum, and converting it to a decibel value.

[0118] The above electrical confidence weights are calculated at the electrical signal sampling point (1kHz), but aligned with the visual frame rate as a reference, and the corresponding electrical confidence weights are output.

[0119] Optical confidence weight calculation: based on the stability of polarization angle change, reflecting ambient light interference, as detailed below:

[0120] ;

[0121] In the formula: The standard deviation of the change in polarization angle is represented; it is the sample standard deviation of the change in polarization angle calculated within a sliding window (window size 16 sampling points, corresponding to one frame of visual time).

[0122] The above optical confidence calculation is performed at the optical signal sampling point (1kHz), synchronized with the electrical signal to the visual frame rate, and the corresponding optical confidence weight is output.

[0123] Based on the weights calculated above, the three confidence weights are normalized so that the sum of the visual confidence weight, the electrical confidence weight, and the optical confidence weight is 1.

[0124] In this embodiment, the physical prior model uses molecular orientation information provided by the optical modality to correct the macroscopic deformation parameters extracted by the visual modality, enhancing the physical consistency between features and reducing purely data-driven errors. The correction is based on a molecular-macroscopic correlation model, which correlates the molecular orientation orderliness parameter with the macroscopic stretching rate. Specifically, based on the liquid crystal elastic constitutive equation and experimental calibration data, a linear relationship between the molecular orientation orderliness parameter and the macroscopic stretching rate is established, as shown in the following expression:

[0125] ;

[0126] In the formula: A parameter representing the degree of order in molecular orientation; Indicates the macroscopic stretching rate;

[0127] Macroscopic stretching rate extracted using S-correction for visual processing To compensate for errors caused by visual occlusion or rapid deformation, the specific expression is as follows:

[0128] ;

[0129] ;

[0130] In the formula: This represents the stretching ratio estimated based on optical characteristics; This represents the original stretching ratio of the extracted visual modality; This indicates the corrected elongation. The fusion weights are dynamically designed based on confidence levels.

[0131] ;

[0132] In the formula: This represents the normalized visual confidence weights; Represents the normalized optical confidence weights;

[0133] By dynamically adjusting the fusion weights, more visual information is preserved when visual reliability is required, and corrections are added when optical reliability is required.

[0134] In this embodiment, the correction only applies to the stretching ratio parameter. The original stretching ratio parameter is replaced with the corrected stretching ratio parameter to obtain the corrected visual modal features.

[0135] S5. Input the corrected visual modal features, extracted electrical features, and optical features into the LSTM fusion network to obtain the comprehensive deformation state parameters.

[0136] In this embodiment, the extracted electrical features, optical features, and corrected visual modal features are weighted by their respective normalized weights and then summed to generate a weighted feature vector as the input to the LSTM. Since the feature dimensions of each feature are: 256 dimensions for visual feature vectors, 128 dimensions for electrical feature vectors, and 64 dimensions for optical feature vectors, the dimensions need to be adjusted before the weighted summation to obtain the weighted feature vector. That is, the dimensions of each feature vector are adjusted to 128 dimensions before the weighted summation.

[0137] The LSTM network consists of two LSTM layers and one fully connected layer. The two LSTM layers are stacked, and each layer has 128 hidden units. The input is the weighted sum of the fused features, and the output is a comprehensive deformation state parameter, which is a four-dimensional vector, namely the bending angle, elongation, torsion angle and internal stress.

[0138] See appendix Figure 3 As shown, in this embodiment, a miniature laser triangulation rangefinder (model: KeyenceLK-G80, accuracy ±0.1mm, sampling rate 1kHz, measurement distance 50-300mm) is used as the core ground truth sensor. It is deployed in a fixed reference coordinate system (such as the end effector of a robotic arm or the base of an experimental platform) within the LCE deformation region to obtain the deformation geometry parameters of the LCE through non-contact measurement. To cover multi-dimensional deformation (bending / tension / torsion), three rangefinders are deployed:

[0139] Rangefinder 1 (bending / tension measurement): Installed along the length (axial direction) of the LCE, it measures the distance change between the fixed points at both ends of the LCE to calculate the bending angle and elongation rate;

[0140] Rangefinder 2 (torsion measurement): Installed along the radial direction (perpendicular axis) of the LCE, it calculates the torsion angle by measuring the circumferential displacement of a marked point on the LCE surface and combining it with the LCE radius;

[0141] Rangefinder 3 (Stress-assisted calibration): Mounted at a 45° angle to the LCE surface, it indirectly assists in verifying the internal stress trend by measuring the fluctuation of reflected light intensity.

[0142] The rangefinder lens uses an anti-glare coating to avoid interference from reflections on the LCE surface; the measuring spot diameter is ≤0.05mm to ensure that the focus is on the marker point in the LCE deformation area (the surface is printed with sub-millimeter positioning dots, which do not affect molecular orientation), achieving sub-pixel positioning accuracy.

[0143] Among them, the appendix Figure 3 The text only provides the location of the laser emitter, but not the location of the optical receiving system. Given the above technical solution and the location of the laser emitter, how to set up the optical receiving system is a conventional technical approach in this field.

[0144] This embodiment is based on a precision deformation control platform. By controlling the LCE, a preset standard deformation state is generated. The specific parameter range is set according to the actual application scenario of the LCE (such as flexible robot joints: bending 0°-90°, elongation 0%-20%, torsion angle 0°-30°, and step size 5°, 1%, and 3° respectively). A total of 100 sets of discrete deformation states are generated (covering static / dynamic deformation, with dynamic states achieved by uniform platform movement at a speed of 0.1° / ms-1° / ms).

[0145] Under the set standard deformation state, visual images, electrical resistance signals, and optical polarization angle and light intensity signals (200 sampling points, average value) are simultaneously acquired to generate multimodal feature vectors. ;

[0146] Three rangefinders simultaneously measure and calculate the true deformation parameters under this condition. ;

[0147] Bending angle: In the formula This indicates the deformed length measured by rangefinder 1; The distance between the two endpoints measured by the rangefinder;

[0148] elongation In the formula The deformed length measured by rangefinder 1;

[0149] Twist angle: In the formula The circumferential displacement measured by rangefinder 2 Let LCE be the radius;

[0150] Internal stress: In the formula The Young's modulus of LCE (pre-calibrated through material testing) ).

[0151] The 100 data pairs mentioned above were stored in a local dataset library, with 80 pairs used as the training set and 20 pairs as the validation set. The Adam optimizer was used to optimize the training set data pairs. Minimize loss function :

[0152] ;

[0153] In the formula: Represented as network to the first The first sample Predicted values ​​of each deformation parameter; It is 80; Indicates the first The first sample The training continues until the training error meets the preset threshold, providing the true value of each deformation parameter.

[0154] This invention overcomes the spatial distribution limitations of a single contact sensor by simultaneously acquiring visual, electrical, and optical signals, achieving collaborative acquisition of non-contact overall information and contact internal state information. In the feature processing stage, a physical prior model is introduced. Utilizing the experimentally calibrated mapping relationship between the molecular orientation order parameter of the liquid crystal elastomer and the macroscopic stretching rate, the microscopic molecular orientation features of the optical mode are transformed into macroscopic deformation correction parameters. This optimizes the overall surface deformation features extracted by the visual mode, compensating for the deficiency of the visual mode in only reflecting surface information, and realizing the correlation between the microscopic internal state and the macroscopic overall deformation. Finally, an LSTM fusion network integrates multimodal features to output comprehensive deformation state parameters. This approach leverages the visual mode to cover the overall spatial range and introduces internal microscopic and stress information through electrical, optical, and physical priors, overcoming the limitations of local strain information and achieving real-time and comprehensive identification of the overall macroscopic deformation and internal stress distribution of the liquid crystal elastomer.

[0155] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for real-time identification of deformation state of liquid crystal elastomers based on multimodal fusion, characterized in that, Includes the following steps: S1. Synchronously acquire the original visual, electrical, and optical three-modal signals during the deformation process of the liquid crystal elastomer; S2. Denoise and time-align the acquired visual, electrical, and optical three-modal raw signals respectively; S3. Extract visual, electrical, and optical features from the denoised and time-aligned visual, optical, and electrical tri-modal signals; S4. Calculate the confidence weight for each mode, then introduce the physical prior model, and through the mapping relationship between the molecular orientation order parameter of the liquid crystal elastomer and the macroscopic stretching rate calibrated by the experiment, transform the molecular orientation characteristics of the optical mode into macroscopic deformation correction parameters to optimize the features extracted from the visual mode. The confidence weight calculation includes the following steps: Visual confidence weights: The Sobel operator is applied to each frame of visual image to obtain the gradient magnitude, the gradient magnitude variance of the deformed region is calculated, and Canny edge detection is performed on the deformed region. The edge integrity score is obtained by the proportion of complete edges. Then, the gradient magnitude variance and the edge integrity score are weighted and summed using experimentally calibrated coefficients to obtain the visual base weights. The visual base weights are mapped to a predetermined range to obtain the visual weights. Electrical confidence weight: Perform a fast Fourier transform on the resistance signal, calculate the ratio of fundamental frequency power to out-of-band noise and convert it into signal-to-noise ratio. If the signal-to-noise ratio is greater than or equal to a predetermined threshold, the electrical confidence weight is set to 0.7; if it is less than the predetermined threshold, it is set to 0.

3. Optical confidence weight: Within a sliding window containing 16 sampling points, the sample standard deviation of the polarization angle is calculated. If the standard deviation is less than a predetermined threshold, the optical weight is set to 0.6; if it is greater than or equal to the predetermined threshold, it is set to 0.

3. Normalized weights: Based on the visual confidence weight, electrical confidence weight, and optical confidence weight, normalized visual weight, normalized electrical weight, and normalized optical weight are obtained by normalization. The features extracted for optimized visual modality include the following steps: Based on the molecular-macro correlation model, molecular orientation parameters are converted into equivalent stretching ratios. Then, according to the normalized weight ratio of vision and optics, the stretching ratios extracted from the original vision are weighted and fused. The fusion weights are dynamically adjusted based on the normalized visual weights and normalized optical weights, so that more original visual features are retained when the normalized visual weights increase, and molecular-level physical corrections are enhanced when the normalized visual weights decrease. Based on this, the corrected visual feature vector is obtained. S5. Input the corrected visual modal features, extracted electrical features, and optical features into the LSTM fusion network to obtain the comprehensive deformation state parameters.

2. The real-time identification method for deformation state of liquid crystal elastomer based on multimodal fusion according to claim 1, characterized in that, The noise reduction includes the following steps: Visual signal denoising: Guided by the original image itself, the brightness relationship of each region is calculated within a predetermined pixel window; by balancing pixel differences and smoothing requirements, a linear fitting result is output for each window, and the average value of all overlapping windows is taken to obtain the denoised visual signal. Electrical signal denoising: The true resistance value and the temperature drift are combined to form a state vector. The current value is predicted based on the state of the previous moment. By comparing the deviation between the predicted value and the actual measured value, the noise tracking sensitivity and observation weight are dynamically adjusted to obtain a stable resistance sequence after drift compensation. Optical signal denoising: Generate sine and cosine reference signals that perfectly match the laser frequency. Multiply the original optical signal with the sine and cosine reference signals respectively. After low-pass filtering to remove high-frequency noise, extract the DC components of the sine and cosine reference signals. Calculate the light intensity change and polarization angle shift based on the orthogonal component relationship to obtain pure light intensity change and polarization angle change signals.

3. The real-time identification method for deformation state of liquid crystal elastomer based on multimodal fusion according to claim 1, characterized in that, The extraction of the visual features includes the following steps: The U-Net network is used to segment the LCE region in the image to generate a mask, and the bending angle, stretching ratio and twist angle are calculated based on the mask contour. Meanwhile, deep texture features of the LCE region are extracted using the ResNet-18 network to generate multidimensional visual feature vectors.

4. The real-time identification method for deformation state of liquid crystal elastomer based on multimodal fusion according to claim 1, characterized in that, The extraction of the electrical features includes the following steps: The rate of change of resistance is obtained based on the considerable difference between the current resistance and the initial resistance; The main peak frequency in the signal spectrum is extracted by Fourier transform and used as the peak frequency of the dynamic resistance. The Daubechies 4 mother wavelet was used to decompose the 1kHz resistance signal into 5 layers with 16 points per window. The energy of the wavelet coefficients and the energy of the approximation coefficients of each decomposition layer were calculated. The obtained energy values ​​were spliced ​​into a 128-dimensional vector, which, together with the resistance change rate and the peak frequency of the dynamic resistance, constituted the electrical feature vector.

5. The real-time identification method for deformation state of liquid crystal elastomer based on multimodal fusion according to claim 1, characterized in that, The extraction of the optical features includes the following steps: The degree of molecular orientation order is calculated based on the change in polarization angle, and the average molecular orientation angle is calculated by combining the light intensity difference in orthogonal directions. The dynamic statistical characteristics of the phase orientation order and molecular orientation angle are calculated within a window of 16 sampling points to obtain an optical feature vector containing molecular orientation parameters and dynamic statistical characteristics.

6. The real-time identification method for deformation state of liquid crystal elastomer based on multimodal fusion according to claim 1, characterized in that, The comprehensive deformation state parameters are based on the following steps: The modal features are linearly superimposed with normalized weights to obtain a fused feature sequence. The fused feature sequence is then input into a two-layer LSTM network. The two-layer LSTM uses the historical features from the previous n milliseconds as the temporal context to process the weighted feature sequence frame by frame. The processed result is input into a fully connected layer to output the comprehensive deformation state parameters.

Images