A structural dynamic displacement full-field reconstruction method, system and medium

Abstract

The application discloses a kind of structure dynamic displacement full-field reconstruction method, system and medium, it is related to structural health monitoring technical field, including steps: obtaining low-frequency displacement registration time series data and high-frequency acceleration registration time series data of target monitoring structure;Initial physical information neural network model is constructed;Composite loss function is constructed, composite loss function includes displacement data loss term, physical residual loss term and structure dynamics regularization term;Initial physical information neural network model is trained until convergence by back propagation, and network model parameter is updated;Each space-time coordinate point of the structure to be reconstructed is input into the trained physical information neural network model, and the estimated displacement value corresponding to each space-time coordinate point is output.The method of the application effectively fuses the absolute benchmark advantage of low-frequency visual displacement and the dynamic detail advantage of high-frequency acceleration, realizes structure dynamic displacement high-frequency interpolation and full-field high-frequency displacement time series reconstruction.

Description

Technical Field

[0001] This invention relates to the field of structural health monitoring technology, and in particular, to a method, system, and medium for full-field reconstruction of structural dynamic displacement. Background Technology

[0002] The displacement response of a structural dynamic system is a core physical quantity characterizing the overall mechanical behavior of the structure. It directly reflects the deformation form, vibration characteristics, and energy transfer mechanism of the structure under external loads. In the health monitoring of large civil engineering structures (such as bridges, high-rise buildings, and wind turbine blades), displacement is one of the most intuitive and critical indicators reflecting the structural safety status. Currently, structural health monitoring methods are mainly divided into two categories: one is non-contact visual measurement (such as digital image correlation), which has the advantages of full-field measurement and no wiring required, and can provide rich spatial information. However, this method is limited by camera transmission bandwidth and real-time processing capabilities, and its sampling rate is usually low (tens of hertz), making it difficult to capture the high-frequency vibration characteristics of the structure. It is prone to high-frequency aliasing and is easily affected by ambient light, resulting in high-frequency noise. The other is contact inertial measurement (such as accelerometers), which has an extremely high sampling rate (kilohertz) and signal-to-noise ratio, and can accurately capture weak high-frequency vibrations. However, the acceleration data itself lacks an absolute reference frame. When calculating displacement through quadratic numerical integration, the initial error and low-frequency noise are amplified sharply, resulting in severe baseline drift in the calculated displacement, making it unsuitable for long-term monitoring.

[0003] In summary, given the technical problems of high-frequency aliasing in non-contact visual measurement and low displacement monitoring accuracy due to baseline drift in contact inertial measurement in existing structural displacement monitoring technologies, it is necessary to provide a method, system, and medium for full-field reconstruction of structural dynamic displacement to solve or at least partially solve the above-mentioned technical problems. Summary of the Invention

[0004] The method, system, and medium for full-field reconstruction of structural dynamic displacement solve the technical problem of low displacement monitoring accuracy in existing structural displacement monitoring.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for full-field reconstruction of structural dynamic displacement includes the following steps: S10, acquire heterogeneous sensor data of the target monitoring structure. The heterogeneous sensor data includes a visual low-frequency full-field displacement time series and a single-point high-frequency acceleration sequence of at least one acceleration spatial measurement point. The sampling frequency of the single-point high-frequency acceleration sequence is higher than the sampling frequency of the visual low-frequency full-field displacement time series. S20, based on the physical coordinate mapping relationship, performs physical spatial registration on heterogeneous sensor data to obtain low-frequency displacement registration time series data and high-frequency acceleration registration time series data; S30, Construct the initial physical information neural network model. The physical information neural network model uses the physical location coordinates of discrete points in the low-frequency displacement registration time series data as spatial input variables, continuous time variables as time input variables, and displacement prediction values ​​as output. S40, Construct a composite loss function, which includes a displacement data loss term, a physical residual loss term, and a structural dynamics regularization term. The displacement data loss term is used to constrain the displacement difference between the predicted and measured displacement values ​​at spatiotemporal points in the low-frequency displacement registration time series data of the physical information neural network model. The physical residual loss term is used to constrain the acceleration difference between the measured and predicted acceleration values ​​in the high-frequency acceleration registration time series data of the physical information neural network model. The acceleration prediction value is the second time derivative of the displacement prediction value. The structural dynamics regularization term is used to constrain the displacement field corresponding to the displacement prediction value to satisfy the structural dynamics equation. S50 uses a composite loss function to perform backpropagation to train the initial physical information neural network model until it converges, and then updates the network model parameters. S60 inputs each spatiotemporal coordinate point of the structure to be reconstructed into the trained physical information neural network model, outputs the estimated displacement value corresponding to each spatiotemporal coordinate point, and reconstructs to obtain the full-field high-frequency displacement time series.

[0006] Furthermore, in step S30, before inputting the time variable into the fully connected layer of the physical information neural network model, the step further includes: using multi-scale Fourier feature encoding to map the time variable and extract multi-frequency information of the time variable to obtain the encoded time feature vector, and the fully connected layer of the physical information neural network model outputs the predicted displacement value based on the time feature vector and the physical location coordinates.

[0007] Furthermore, the mapping formula for multi-scale Fourier feature encoding is as follows: ,in, , This is the time feature vector after multi-scale Fourier encoding. Represents a continuous time variable. For multi-frequency scale vectors, For the number of frequency scales. To ensure the physical meaning of frequency, the angular frequency coefficient.

[0008] Furthermore, the physical information neural network model includes a learnable time bias parameter; When calculating the physical residual loss term, the time deviation parameter is used to perform time-domain registration of low-frequency displacement registration time series data and high-frequency acceleration registration time series data. Step S50 also includes: using a composite loss function to perform backpropagation to train the initial physical information neural network model until convergence, and updating the time bias parameter.

[0009] Furthermore, in step S50, a multi-stage training strategy is adopted when training the initial physical information neural network model, specifically including: In the first stage, the weight of the physical residual loss term in the composite loss function is set to zero, and the loss is calculated using the displacement data loss term and the structural dynamics regularization term to train the physical information neural network model. In the second stage, the weight of the physical residual loss term in the composite loss function is increased to a value greater than zero, and the loss is calculated using the composite loss function to train the physical information neural network model trained in the first stage.

[0010] Furthermore, using the formula Calculate and obtain the composite loss function. For composite loss function, For displacement data loss, This is the physical residual loss term. For structural dynamics regularization, , , These are the weights corresponding to the displacement data loss term, the physical residual loss term, and the structural dynamics regularization term, respectively.

[0011] Furthermore, in step S10, The visual low-frequency full-field displacement time series is obtained by acquiring and solving image sequences by a visual acquisition device at a first sampling frequency. The visual low-frequency full-field displacement time series includes the measured displacement values ​​of multiple discrete spatial measurement points in a preset region of interest at multiple low-frequency sampling times. The single-point high-frequency acceleration sequence is acquired by an accelerometer installed at an acceleration measurement point in the acceleration space at a second sampling frequency. The single-point high-frequency acceleration sequence includes the measured acceleration values ​​at multiple high-frequency sampling times. The second sampling frequency is higher than the first sampling frequency.

[0012] Furthermore, the visual acquisition device acquires images of a preset region of interest and images from an accelerometer; In step S20, the camera of the visual acquisition device is calibrated to establish a physical coordinate mapping relationship between the pixel coordinate system and the physical world coordinate system. By utilizing the physical coordinate mapping relationship, the pixel coordinates corresponding to each discrete spatial measurement point in the visual low-frequency full-field displacement time series are transformed to the displacement sampling measured values ​​in the physical world coordinate system, thus obtaining the spatially registered low-frequency displacement registration time series data. By utilizing the physical coordinate mapping relationship, the physical coordinates of the accelerometer sensor under the acceleration space measurement point are determined to be transformed into the physical world coordinate system. Based on the physical coordinates of the accelerometer, high-frequency acceleration registration time series data are obtained.

[0013] This invention also provides a full-field reconstruction system for structural dynamic displacement. It includes a visual acquisition module, which is used to acquire low-frequency image sequences of the target monitoring structure and calculate the visual low-frequency full-field displacement time series; The acceleration acquisition module is used to acquire single-point high-frequency acceleration sequences of the target monitoring structure at acceleration space measurement points; The data processing module includes a processor and a memory. The memory stores a computer program, and when the processor executes the computer program, it implements the above-mentioned method for full-field reconstruction of structural dynamic displacement.

[0014] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described method for full-field reconstruction of structural dynamic displacement.

[0015] The present invention has the following beneficial effects: The present invention relates to a method, system, and medium for full-field reconstruction of structural dynamic displacement. Based on physical information neural network constraints, it achieves full-field reconstruction of structural dynamic displacement. Addressing the dual challenges of low sampling rate in non-contact visual measurements leading to high-frequency aliasing and baseline drift caused by inertial measurement integration in contact methods, a physical information neural network fusion reconstruction framework is constructed. The absolute reference of the full-field low-frequency displacement in the low-frequency displacement registration time series data is obtained through a displacement data loss term, eliminating drift caused by acceleration integration. Furthermore, the second-order time derivative at the accelerometer position (accelerometer spatial measurement point) is constrained by a physical residual loss term to match the measured high-frequency acceleration (accelerometer physical residual measurement point). (Measured values), recovering high-frequency aliasing caused by visual undersampling. Based on a trained physical information neural network model, it can predict the full-field high-frequency displacement time series with acceleration sampling rate level time resolution based on any spatiotemporal coordinate point, realizing accurate extrapolation from single-point high-frequency information to full-field high-frequency displacement, significantly improving the accuracy and frequency response range of structural displacement monitoring, effectively integrating the absolute benchmark advantage of low-frequency visual displacement with the dynamic detail advantage of high-frequency acceleration, and realizing high-frequency interpolation of structural dynamic displacement and high-precision, high-frequency response, full-field coverage reconstruction of full-field high-frequency displacement time series without the need for expensive hardware triggering equipment.

[0016] Other beneficial effects of this application will be described in detail in the following detailed description section. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the full-field reconstruction method for structural dynamic displacement in one embodiment of the present invention. Figure 2 This is a schematic diagram illustrating an application scenario of the full-field reconstruction method for structural dynamic displacement in one embodiment of the present invention. Figure 3 This is a schematic diagram of the physical information neural network model in the structural dynamic displacement full-field reconstruction method of one embodiment of the present invention. Detailed Implementation

[0019] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0020] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0021] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0022] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0023] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0024] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0025] Research has revealed that non-contact visual measurement (such as digital image correlation) offers advantages such as full-field measurement and no wiring required, providing rich spatial information. However, limited by camera transmission bandwidth and real-time processing capabilities, its sampling rate is typically low (tens of hertz), making it difficult to capture high-frequency vibration characteristics of structures and susceptible to high-frequency noise from ambient light. Contact inertial measurement (such as accelerometers), on the other hand, boasts extremely high sampling rates (kilohertz) and signal-to-noise ratios, accurately capturing weak high-frequency vibrations. However, the acceleration data itself lacks an absolute reference frame. When calculating displacement through quadratic numerical integration, initial errors and low-frequency noise are drastically amplified, leading to severe baseline drift in the calculated displacement, making it unsuitable for long-term monitoring. To combine the advantages of both methods, existing data fusion methods (such as Kalman filtering and complementary filtering) typically require strict time synchronization between sensors and processing at a uniform sampling rate (often involving complex resampling and anti-aliasing filtering). This not only increases the integration cost and complexity of the hardware system but also makes it difficult for traditional filtering algorithms to describe the evolution of full-field displacement using simple linear assumptions when dealing with nonlinear structural responses or complex boundary conditions. Physical Information Neural Networks (PINNs) offer a new approach to solving the above problems. PINNs can be used to embed partial differential equations (PDEs) into the loss function of neural networks, enabling the solution of complex physical fields using sparse data. However, when PINNs are directly applied to high-frequency vibration monitoring, challenges such as spectral bias (difficulty in learning high-frequency components) and asynchronous multi-source data are still faced.

[0026] like Figure 1 , Figure 2 and Figure 3 As shown, this invention provides a method for full-field reconstruction of structural dynamic displacement, including the following steps: S10, acquire heterogeneous sensor data of the target monitoring structure. The heterogeneous sensor data includes visual low-frequency full-field displacement time series (referring to the data series of displacement changing with time) and single-point high-frequency acceleration sequence of at least one acceleration spatial measurement point. The sampling frequency of the single-point high-frequency acceleration sequence is higher than the sampling frequency of the visual low-frequency full-field displacement time series. S20, based on the physical coordinate mapping relationship, performs physical spatial registration on heterogeneous sensor data to obtain low-frequency displacement registration time series data and high-frequency acceleration registration time series data; S30, Construct the initial physical information neural network model. The physical information neural network model uses the physical location coordinates of discrete points in the low-frequency displacement registration time series data as spatial input variables, continuous time variables as time input variables, and displacement prediction values ​​as output. S40, Construct a composite loss function, which includes a displacement data loss term, a physical residual loss term, and a structural dynamics regularization term. The displacement data loss term is used to constrain the displacement difference between the predicted and measured displacement values ​​at spatiotemporal points in the low-frequency displacement registration time series data of the physical information neural network model. The physical residual loss term is used to constrain the acceleration difference between the measured and predicted acceleration values ​​in the high-frequency acceleration registration time series data of the physical information neural network model. The acceleration prediction value is the second time derivative of the displacement prediction value. The structural dynamics regularization term is used to constrain the displacement field corresponding to the displacement prediction value to satisfy the structural dynamics equation. S50 uses a composite loss function to perform backpropagation to train the initial physical information neural network model until it converges, and then updates the network model parameters. S60 inputs each spatiotemporal coordinate point of the structure to be reconstructed into the trained physical information neural network model, outputs the estimated displacement value corresponding to each spatiotemporal coordinate point, and reconstructs to obtain the full-field high-frequency displacement time series.

[0027] The present invention relates to a method, system, and medium for full-field reconstruction of structural dynamic displacement. Addressing the dual challenges of low sampling rates in non-contact visual measurements leading to high-frequency aliasing and baseline drift caused by inertial measurement integration in contact measurements, a physical information neural network fusion reconstruction framework is constructed. By obtaining the absolute benchmark of full-field low-frequency displacement in the low-frequency displacement registration time series data through a displacement data loss term, the drift caused by acceleration integration is eliminated. By constraining the second-order time derivative at the accelerometer position (acceleration spatial measurement point) with a physical residual loss term to match the measured high-frequency acceleration (measured acceleration value), the high-frequency aliasing caused by visual undersampling is restored. Based on the trained physical information neural network model, a full-field high-frequency displacement time series with acceleration sampling rate-level time resolution can be predicted for any spatiotemporal coordinate point. This achieves accurate extrapolation from single-point high-frequency information to full-field high-frequency displacement, significantly improving the accuracy and frequency response range of structural displacement monitoring. It effectively integrates the absolute benchmark advantage of low-frequency visual displacement with the dynamic detail advantage of high-frequency acceleration, enabling high-frequency interpolation of structural dynamic displacement and high-precision, high-frequency response, and full-field coverage reconstruction of the full-field high-frequency displacement time series without the need for expensive hardware triggering equipment.

[0028] In an optional embodiment of the present invention, a structural dynamic displacement full-field reconstruction system is provided to implement the steps of the above-described structural dynamic displacement full-field reconstruction method. The system includes a visual acquisition module, an acceleration acquisition module, and a processing device. In a specific embodiment, the visual acquisition module uses a visual displacement sensor, such as an industrial camera; the acceleration acquisition module uses an acceleration sensor, such as an accelerometer; and the processing device uses an edge data processing module. Specifically, the visual acquisition module acquires the accelerometer readings within the field of view and the visual low-frequency full-field displacement time series corresponding to each discrete point in the image. The actual displacement and actual acceleration are obtained after transformation based on the physical coordinate mapping relationship. During image acquisition, the image sampling frequency is set. The vision acquisition module captures speckle image sequences from the surface of the structure under test (such as cantilever beams, bridge models, etc.) in a non-contact manner. Although the 60Hz image sampling frequency poses a risk of undersampling for high-frequency vibrations of the structure (such as frequencies above 100Hz), the high-precision, drift-free low-frequency displacement registration time series data it provides will serve as low-frequency anchor points for the PINN network. The acceleration acquisition module is a high-frequency dynamic detail source. During acceleration acquisition, a triaxial piezoelectric accelerometer with a frequency response range of 0.2Hz to 4000Hz is used. In specific implementation, an acceleration sampling frequency of not less than 1000Hz should be used. Sampling is performed by attaching accelerometers to key stress points or mid-span locations of the structure under test to capture minute transient vibrations and high-frequency acceleration signals. The high-frequency acceleration signals are then converted into measured acceleration signals based on physical coordinate mapping. The edge data processing module enables localized data processing and real-time fusion, used for image processing, spatiotemporal registration, PINN model training, and inference tasks using Digital Image Correlation (DIC) algorithms. The structural dynamic displacement full-field reconstruction system of this invention constructs a complementary sensing environment of low-frequency vision and high-frequency inertial navigation, providing a physical basis for resolving drift and aliasing contradictions in structural monitoring.

[0029] Understandably, in step S10, the visual low-frequency full-field displacement time series is obtained by acquiring and processing image sequences using a visual acquisition device at a first sampling frequency. Each frame of the visual low-frequency full-field displacement time series includes the low-frequency sampling time. Each image spatial measurement point (discrete point) within the preset region of interest. The image displacement values ​​are transformed into measured displacement values ​​based on the physical coordinate mapping relationship. The single-point high-frequency acceleration sequence is acquired by the accelerometer at the second sampling frequency. The single-point high-frequency acceleration sequence includes the measured acceleration values ​​at multiple high-frequency sampling times. The second sampling frequency is higher than the first sampling frequency. Since the visual low-frequency full-field displacement time series and the accelerometer installation point are obtained in the image coordinate system in step S10, it is necessary to calibrate and transform the data in the image coordinate system to the physical world coordinate system to obtain low-frequency displacement registration time series data and high-frequency acceleration registration time series data. The low-frequency displacement registration time series data is a dataset composed of the physical position coordinates, time nodes, and physical displacement values ​​of discrete points. The high-frequency acceleration registration time series data is a dataset composed of the physical position coordinates, time nodes, and acceleration values ​​of the accelerometer. In step S30, the input of the physical information neural network model includes spatial position. The coordinates and time coordinates are used to output a displacement prediction field. In step S40, the physical residual loss term calculated based on the laws of physical kinematics is obtained by performing a second-order automatic differentiation of the displacement prediction field output by the neural network with respect to time and calculating its error with the high-frequency single-point acceleration sequence. In step S40, the structural dynamics regularization term is used to constrain the displacement field in the non-sensor coverage area to satisfy the boundary conditions, geometric continuity conditions, or structural mechanical control equations of the structure. In step S50, the composite loss function is minimized when training the physical information neural network model, and the network parameters are updated. In step S60, each spatiotemporal coordinate point of the structure to be reconstructed is input into the trained physical information neural network model, and the estimated displacement value corresponding to each spatiotemporal coordinate point is output to reconstruct and obtain a full-field high-frequency displacement time series with a sampling frequency of not less than the second sampling frequency.

[0030] Understandably, when constructing a Physical Information Neural Network (PINN) model, the input to the PINN model is spatiotemporal coordinates, and the output is the displacement prediction value. To solve the high-frequency learning problem, multi-scale Fourier feature encoding is introduced at the input to map the time coordinates to a high-dimensional frequency feature space. Furthermore, before inputting the time variable into the fully connected layer of the PINN model, the following steps are included: using multi-scale Fourier feature encoding to map the time variable and extract the multi-frequency information of the time variable to obtain the encoded time feature vector. The fully connected layer of the PINN model outputs the predicted displacement value based on the time feature vector and the physical location coordinates.

[0031] Furthermore, in step S10, The visual low-frequency full-field displacement time series is obtained by acquiring and solving image sequences by a visual acquisition device at a first sampling frequency. The visual low-frequency full-field displacement time series includes the measured displacement values ​​of multiple discrete spatial measurement points in a preset region of interest at multiple low-frequency sampling times. The single-point high-frequency acceleration sequence is acquired by an accelerometer installed at an acceleration measurement point in the acceleration space at a second sampling frequency. The single-point high-frequency acceleration sequence includes the measured acceleration values ​​at multiple high-frequency sampling times. The second sampling frequency is higher than the first sampling frequency.

[0032] Furthermore, the visual acquisition device acquires images of a preset region of interest and images from an accelerometer; In step S20, the camera of the visual acquisition device is calibrated to establish a physical coordinate mapping relationship between the pixel coordinate system and the physical world coordinate system. By utilizing the physical coordinate mapping relationship, the pixel coordinates corresponding to each discrete spatial measurement point in the visual low-frequency full-field displacement time series are transformed to the displacement sampling measured values ​​in the physical world coordinate system, thus obtaining the spatially registered low-frequency displacement registration time series data. By utilizing the physical coordinate mapping relationship, the physical coordinates of the accelerometer sensor under the acceleration space measurement point are determined to be transformed into the physical world coordinate system. Based on the physical coordinates of the accelerometer, high-frequency acceleration registration time series data are obtained.

[0033] Understandably, in step S20, the mapping relationship between the pixel coordinate system of the visual acquisition device and the physical world coordinate system is established through camera calibration; the installation position of the accelerometer in the physical world coordinate system is determined so that when calculating the physical residual loss term, the spatial constraint is limited to the installation position point of the accelerometer, while the displacement constraint is applied to the entire field area covered by the visual acquisition device, realizing the spatial extrapolation from single-point high-frequency information to full-field high-frequency information. In a specific embodiment, after acquiring the original heterogeneous sensor data, the image pixel coordinate system of the visual acquisition module and the physical space coordinate system of the accelerometer acquisition module must be unified to ensure that the PINN model can understand the spatial relationship of different sensor data based on the physical space coordinate system; firstly, the visual coordinate system calibration (image pixel coordinate system) and displacement calculation are performed. First, the intrinsic parameters and distortion parameters of the shooting camera are calibrated using the Zhang Zhengyou calibration method to eliminate the distortion caused by the wide-angle lens; then, the acquired speckle image sequence is processed by the digital image correlation (DIC) algorithm. Optionally, a region of interest (ROI) on the structural surface is selected, and a layout is arranged within the ROI. One virtual measuring point (discrete measuring point) DIC algorithm for tracking discrete measurement points Motion in the image pixel coordinate system, combined with camera calibration parameters (homography matrix) or projection matrix This data is converted into low-frequency displacement registration time series data in the physical world coordinate system, and the measured displacement values ​​are virtual measurement points calculated by DIC. Displacement at low-frequency sampling time, and time interval between two adjacent low-frequency sampling times. Secondly, regarding the spatial positioning and mapping of the accelerometer, to apply single-point acceleration constraints to the correct location, it is necessary to determine the precise position of the accelerometer in the physical coordinate system. A visual positioning method is used, where prominent artificial markers (such as QR codes or crosshairs) are affixed to the accelerometer's casing. During DIC measurements, these artificial markers are simultaneously captured by the camera. By identifying the pixel center of the artificial marker in the image, its physical coordinates are deduced using a calibration matrix. This allows the system to automatically obtain the corresponding spatial index regardless of where the accelerometer is installed, thus accurately representing the measured acceleration value in the PINN loss function calculation. The physical constraints are applied to specific spatial points in the network output. Based on the above concept, Full-field displacement data and The single-point acceleration data were unified into the same spacetime reference frame.

[0034] Furthermore, traditional neural networks tend to prioritize learning low-frequency components, resulting in poor fitting of high-frequency vibration details (such as impact response and higher-order modes). Therefore, this embodiment incorporates time coordinates... Before being input to the fully connected layer, it is first processed through a fixed multi-scale Fourier feature encoding layer, with the mapping formula being: ,in, It is a collection Frequency matrices (or vectors) at different frequency scales. Represents a continuous time variable. For multi-frequency scale vectors, For the number of frequency scales. To ensure the angular frequency coefficient has a physical meaning; in an optional embodiment, Set to 10, For an m×1 vector, the frequency values ​​follow a Gaussian distribution. Initialize with random sampling, or set it to a geometric sequence to cover the sequence from... arrive Wide bandwidth; after encoding, time scalar input Expanded to dimension The high-dimensional feature vectors enable neural networks to perceive input changes at different frequency scales, thereby significantly enhancing their ability to express the high-frequency details captured by the accelerometer.

[0035] The present invention projects a scalar time coordinate onto a high-dimensional feature space and encodes the input signal using a set of sine and cosine functions of different frequencies to enhance the network’s sensitivity to high-frequency vibration features.

[0036] Due to the time delay between different sensors, time-domain alignment is crucial for ensuring physical plausibility when calculating the composite loss function. This alignment can be achieved by matching phase spectra or time-frequency coherence, thus aligning the time in the time domain. Furthermore, the physical information neural network model includes a learnable time bias parameter. When calculating the physical residual loss term, the time bias parameter is used to perform time-domain registration of low-frequency displacement registration time series data and high-frequency acceleration registration time series data. Step S50 further includes training the initial physical information neural network model using backpropagation with the composite loss function until convergence, and updating the time bias parameter. In this invention, when calculating the physical residual loss term, the network uses the time bias parameter to correct the input time coordinates and automatically searches for the optimal time alignment phase between the visual acquisition device and the acceleration sensor, thereby synchronously achieving time-domain registration of heterogeneous data during training. The present invention addresses the issue of heterogeneous data temporal registration during training by using a time offset parameter to correct the input time coordinates when calculating the physical residual loss term. This automatic search for the optimal time alignment phase between the visual acquisition device and the accelerometer allows for simultaneous temporal registration of heterogeneous data during training. Specifically, to address the problem of asynchronous drift caused by different sampling frequencies of heterogeneous sensors, a dynamic correction logic and a phase adaptive strategy are employed. When calculating the physical residual loss term, a lightweight sub-network controlled by the CPU is used to calculate the time offset parameter in real time. This is used to compensate for the dynamic time drift caused by crystal frequency fluctuations between heterogeneous sensors. The time-domain registration optimization strategy ensures that the high-frequency acceleration sequence and the low-frequency displacement sequence achieve nonlinear alignment on the order of milliseconds, which significantly improves the phase accuracy of the full-field high-frequency displacement time sequence output in step S60.

[0037] Please refer to this again. Figure 3 Understandably, in an optional embodiment of the present invention, the Physical Information Neural Network (PINN) model is designed with a fully connected deep neural network that integrates multi-scale Fourier feature embedding to address the spectral bias problem of traditional MLP networks when learning high-frequency functions. The PINN model uses the physical location coordinates of discrete points in low-frequency displacement registration time series data as spatial input variables. Using continuous time variables as time input variables Time input variable With time nodes Using the displacement prediction value as For output, time nodes The corresponding predicted displacement value is Specifically, firstly, by embedding a multi-scale Fourier feature encoding layer, traditional neural networks tend to prioritize learning low-frequency components, resulting in poor fitting of high-frequency vibration details (such as impact response and higher-order modes). Therefore, this embodiment incorporates a multi-scale Fourier feature encoding layer. Before being input to the fully connected layer, the input is first processed through a fixed Fourier feature mapping layer. After encoding and mapping, the scalar input... Expanded to dimension high-dimensional feature vectors This enables neural networks to perceive input changes at different frequency scales, thus significantly enhancing their ability to represent high-frequency details captured by accelerometers. Secondly, in the main design of the deep fully connected network (MLP), the encoded temporal feature vector and spatial coordinates... After concatenation, the data is input into a multilayer perceptron (MLP). The network structure uses 5 fully connected layers, each containing 50 neurons. When designing the activation function, in order to support second-order automatic differentiation, a smooth and infinitely differentiable function is selected to avoid using ReLU (whose second derivative is 0 and cannot transmit acceleration physical information). When designing the output layer, the output layer contains 1 neuron, corresponding to the predicted scalar displacement value.

[0038] Furthermore, using the formula Calculate and obtain the composite loss function. For composite loss function, For displacement data loss, This is the physical residual loss term. For structural dynamics regularization, , , These are the weights corresponding to the displacement data loss term, the physical residual loss term, and the structural dynamics regularization term, respectively.

[0039] Understandably, in an optional embodiment of the present invention, a composite loss function is constructed to closely integrate data-driven approaches with physical laws, and learnable parameters are introduced to solve asynchronous problems; wherein, the meaning of the parameters is defined, This represents the total number of discrete visual measurement points across the entire field. This indicates the total number of frames in the low-frequency image. This represents the total number of low-frequency spatiotemporal points. , The physical spatial coordinates of the visual discrete measurement point i represent a fixed spatial location that does not change over time. Indicates the first Low-frequency sampling time of the frame This indicates that the PINN model is at a spatiotemporal point ( , The predicted displacement value at () Indicates at a point in spacetime ( , The measured displacement value at the location; This represents the total number of high-frequency acceleration samples. This represents the total number of high-frequency moments. , The physical spatial coordinates representing the installation location of the accelerometer sensor are fixed spatial coordinates that do not change over time. Indicates the first A high-frequency sampling time, This represents a learnable time bias parameter used to achieve automatic temporal registration between vision and accelerometer sensors. Represents the physical space coordinates corresponding to the accelerometer position. At the time of correction The predicted acceleration (second derivative of displacement). , This indicates the accelerometer at high-frequency sampling times. The measured acceleration at that time; Displacement data loss term Used to anchor low-frequency references and prevent drift, the displacement data loss term is the mean square error between the predicted and measured displacement values ​​at the sampling time: ,in, This represents the total number of visual measurement points, i.e., the total number of discrete spatial measurement points (virtual measurement points / visual measurement points) i in the entire field; The physical residual loss term utilizes acceleration data to inject high-frequency dynamic features, and directly calculates the displacement prediction value output by the network through automatic differentiation technology. Regarding time The second derivative of the given value is used to force it to match the measured acceleration. : Physical residual loss term Only the physical space coordinates corresponding to the accelerometer installation location The calculations are performed only at high frequencies, but due to the continuity of the neural network, these high-frequency constraints propagate throughout the entire spatial domain through the network parameters. Considering the potential time skew between the DIC camera and the acceleration acquisition (due to the lack of hardware synchronization), this invention designs a time skew parameter. As a trainable scalar parameter, the network is actually evaluating the physical residual when computing it. During training, the optimizer adjusts the network weights and time deviation parameters simultaneously, automatically finding the optimal phase that aligns the displacement waveform with the acceleration waveform. The software synchronization design greatly reduces the reliance on expensive hardware synchronization devices. Generating structural dynamics regularization terms based on structural dynamics equations Structural dynamics regularization term The calculation is based on existing methods that consider mass matrix, damping matrix, stiffness matrix, external load, velocity, and acceleration. In a specific embodiment of the present invention, the structural dynamics regularization term... The equations of motion for the mass-damped-stiffness system are used to constrain the displacement field corresponding to the predicted displacement value to satisfy the structural dynamics equations.

[0040] Furthermore, when training the initial physical information neural network model, the weights of the equilibrium displacement data loss term, the physical residual loss term, and the structural dynamics regularization term are adjusted based on the gradient statistics or neural tangent kernel features obtained during the training process.

[0041] Furthermore, in step S50, a multi-stage training strategy is adopted when training the initial physical information neural network model, specifically including: In the first stage, the weight of the physical residual loss term in the composite loss function is set to zero, and the loss is calculated using the displacement data loss term and the structural dynamics regularization term to train the physical information neural network model. In the second stage, the weight of the physical residual loss term in the composite loss function is increased to a value greater than zero, and the loss is calculated using the composite loss function to train the physical information neural network model trained in the first stage.

[0042] In the scheme of this invention, the first stage only enables the displacement data loss term and the structural dynamics regularization term, and uses the low-frequency full-field displacement sequence to train the network and establish the low-frequency trend and absolute benchmark of the structural displacement; the second stage enables or increases the weight of the physical residual loss term, and uses the high-frequency single-point acceleration sequence to perform high-frequency detail correction and waveform completion on the network output.

[0043] In an optional embodiment of the present invention, to prevent the network from failing to converge due to high-frequency noise interference in the early stages of training, a multi-stage adaptive training strategy from coarse to fine is adopted; wherein, the first stage is used for low-frequency trend locking, and the weight of the physical loss term is set to 0 or a minimum value (e.g., The first stage uses only image-to-displacement (DIC) data for training, allowing the neural network to quickly learn the overall motion trend and equilibrium position of the structure, establishing a correct low-frequency baseline. At this stage, the network output resembles a smooth fit to the DIC data. The second stage injects high-frequency details by gradually increasing the weight values ​​and activating a learnable time bias parameter in the network. Acceleration constraints are introduced, forcing the network to match the high-frequency readings of the accelerometer with its second derivative while fitting the DIC trend. This results in rich high-frequency textures in the previously smooth displacement curve, aligned with the acceleration signal. To balance the order-of-magnitude difference between the gradients of the displacement data loss term and the physical residual loss term, the Neural Tangent Kernel (NTK) algorithm or gradient statistics method is used to dynamically adjust the ratio of the two weights, achieving adaptive weight matching. This ensures that the optimization process considers not only the numerical magnitude but also the contribution of the gradient.

[0044] This invention provides a specific method for full-field reconstruction of structural dynamic displacement, including: Step 1: Acquire heterogeneous sensor data of the structure to be monitored. The heterogeneous sensor data includes a low-frequency full-field displacement sequence acquired by a visual acquisition device at a first sampling frequency, and a high-frequency single-point acceleration sequence acquired by an acceleration sensor installed at an acceleration space measurement point.

[0045] Step 2: Perform physical space registration to obtain low-frequency displacement registration time series data and high-frequency acceleration registration time series data; Step 3: Construct a Physical Information Neural Network (PINN) model. The model input is spatiotemporal coordinates, and the output is displacement prediction. To solve the high-frequency learning problem, multi-scale Fourier feature encoding is introduced at the input to map the time coordinates to a high-dimensional frequency feature space. Step four: Construct a composite loss function. The composite loss function includes a displacement data loss term, a physical residual loss term, a structural dynamics regularization term, and a time offset parameter. It not only includes the displacement data fitting error, but also the physical residual term (i.e., the error between the network's predicted acceleration and the measured acceleration) calculated based on second-order automatic differentiation. The introduction of a learnable time offset parameter enables the network to automatically search for and correct the time difference between visual and acceleration data during training.

[0046] Step 5: A multi-stage adaptive training strategy is adopted. In the first stage, only low-frequency displacement data is used to lock the low-frequency trend of structural motion. In the second stage, acceleration physical constraints are introduced to refine the details of high-frequency vibration and dynamically adjust the weights of each loss term to balance gradient differences.

[0047] Step 6: Using the trained network, input any time and position coordinates to output the corresponding high-resolution displacement field data.

[0048] Compared with the prior art, the present invention has the following beneficial effects: Eliminating Drift and Aliasing: By anchoring low-frequency references using image data, the drift problem of acceleration integrals is completely eliminated; by filling the sampling interval of image data with acceleration data, the aliased high-frequency vibration details are perfectly restored; the time synchronization problem is innovatively transformed into an optimization problem, achieving high-precision alignment of heterogeneous sensors without the need for hardware trigger lines or GPS timing, thus reducing system costs; full-field virtual sensing breaks through the limitation of the number of physical sensors, broadcasting single-point acceleration information to the entire field using physical laws, enabling high-frequency displacement extrapolation for areas where no sensors are installed; strong noise resistance and robustness: the PINN model itself has a smoothing effect, which, combined with physical constraints, can effectively suppress measurement noise and output a smooth curve that conforms to the laws of mechanics.

[0049] This invention also provides a full-field reconstruction system for structural dynamic displacement. It includes a visual acquisition module, which is used to acquire low-frequency image sequences of the target monitoring structure and calculate the visual low-frequency full-field displacement time series; The acceleration acquisition module is used to acquire single-point high-frequency acceleration sequences of the target monitoring structure at acceleration space measurement points; The data processing module includes a processor and a memory. The memory stores a computer program, and when the processor executes the computer program, it implements the above-mentioned method for full-field reconstruction of structural dynamic displacement.

[0050] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described method for full-field reconstruction of structural dynamic displacement.

[0051] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles described in this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for full-field reconstruction of structural dynamic displacement, characterized in that, Including the following steps: S10, acquire heterogeneous sensor data of the target monitoring structure. The heterogeneous sensor data includes a visual low-frequency full-field displacement time series and a single-point high-frequency acceleration sequence of at least one acceleration spatial measurement point. The sampling frequency of the single-point high-frequency acceleration sequence is higher than the sampling frequency of the visual low-frequency full-field displacement time series. S20, perform physical spatial registration on the heterogeneous sensor data based on the physical coordinate mapping relationship to obtain low-frequency displacement registration time series data and high-frequency acceleration registration time series data; S30, Construct an initial physical information neural network model. The physical information neural network model uses the physical location coordinates of discrete points in the low-frequency displacement registration time series data as spatial input variables, continuous time variables as time input variables, and displacement prediction values ​​as output. S40, Construct a composite loss function, which includes a displacement data loss term, a physical residual loss term, and a structural dynamics regularization term. The displacement data loss term is used to constrain the displacement difference between the predicted and measured displacement values ​​at spatiotemporal points in the low-frequency displacement registration time series data by the physical information neural network model. The physical residual loss term is used to constrain the acceleration difference between the measured and predicted acceleration values ​​in the high-frequency acceleration registration time series data by the physical information neural network model. The predicted acceleration value is the second time derivative of the predicted displacement value. The structural dynamics regularization term is used to constrain the displacement field corresponding to the predicted displacement value to satisfy the structural dynamics equation. S50, the initial physical information neural network model is trained by backpropagation using the composite loss function until convergence, and the network model parameters are updated. S60, input each spatiotemporal coordinate point of the structure to be reconstructed into the trained physical information neural network model, output the estimated displacement value corresponding to each spatiotemporal coordinate point, and reconstruct to obtain the full-field high-frequency displacement time series.

2. The structural dynamic displacement full-field reconstruction method according to claim 1, characterized in that, In step S30, before inputting the time variable into the fully connected layer of the physical information neural network model, the step further includes: using multi-scale Fourier feature encoding to map the time variable and extract multi-frequency information of the time variable to obtain an encoded time feature vector, and the fully connected layer of the physical information neural network model outputs a predicted displacement value based on the time feature vector and the physical location coordinates.

3. The structural dynamic displacement full-field reconstruction method according to claim 2, characterized in that, The mapping formula for multi-scale Fourier feature encoding is: ,in, , This is the time feature vector after multi-scale Fourier encoding. Represents a continuous time variable. It is a multi-frequency scale vector. For the number of frequency scales, To ensure the physical meaning of frequency, the angular frequency coefficient.

4. The structural dynamic displacement full-field reconstruction method according to claim 2, characterized in that, The physical information neural network model includes a learnable time deviation parameter; When calculating the physical residual loss term, the time deviation parameter is used to perform time-domain registration of the low-frequency displacement registration time series data and the high-frequency acceleration registration time series data. Step S50 further includes: using the composite loss function to perform backpropagation to train the initial physical information neural network model until convergence, and updating the time deviation parameter.

5. The structural dynamic displacement full-field reconstruction method according to claim 4, characterized in that, In step S50, a multi-stage training strategy is adopted when training the initial physical information neural network model, specifically including: In the first stage, the weight of the physical residual loss term in the composite loss function is set to zero, and the loss is calculated using the displacement data loss term and the structural dynamics regularization term to train the physical information neural network model. In the second stage, the weight of the physical residual loss term in the composite loss function is increased to a value greater than zero, and the loss is calculated using the composite loss function to train the physical information neural network model trained in the first stage.

6. The structural dynamic displacement full-field reconstruction method according to claim 5, characterized in that, Using formula Calculate and obtain the composite loss function. The composite loss function is... For the displacement data loss term, For the physical residual loss term, For the structural dynamics regularization term, , , These are the weights corresponding to the displacement data loss term, the physical residual loss term, and the structural dynamics regularization term, respectively.

7. The structural dynamic displacement full-field reconstruction method according to any one of claims 1 to 6, characterized in that, In step S10, The visual low-frequency full-field displacement time series is obtained by acquiring and solving image sequences by a visual acquisition device at a first sampling frequency. The visual low-frequency full-field displacement time series includes the displacement sampling measured values ​​of multiple discrete spatial measurement points in a preset region of interest at multiple low-frequency sampling times. The single-point high-frequency acceleration sequence is acquired by an acceleration sensor installed at an acceleration space measurement point at a second sampling frequency. The single-point high-frequency acceleration sequence includes measured acceleration values ​​at multiple high-frequency sampling times, and the second sampling frequency is higher than the first sampling frequency.

8. The structural dynamic displacement full-field reconstruction method according to claim 7, characterized in that, The visual acquisition device acquires images of the preset region of interest and images from the accelerometer sensor; In step S20, the camera of the visual acquisition device is calibrated to establish the physical coordinate mapping relationship between the pixel coordinate system and the physical world coordinate system; Using the physical coordinate mapping relationship, the pixel coordinates corresponding to each discrete spatial measurement point in the visual low-frequency full-field displacement time series are transformed to the displacement sampling measured values ​​in the physical world coordinate system, so as to obtain the low-frequency displacement registration time series data after spatial registration. Using the physical coordinate mapping relationship, the physical coordinates of the accelerometer at the acceleration space measurement point are determined to be transformed into the physical world coordinate system. Based on the physical coordinates of the accelerometer, the high-frequency acceleration registration time series data is obtained.

9. A full-field reconstruction system for structural dynamic displacement, characterized in that, It includes a visual acquisition module, which is used to acquire low-frequency image sequences of the target monitoring structure and calculate the visual low-frequency full-field displacement time series; The acceleration acquisition module is used to acquire single-point high-frequency acceleration sequences of the target monitoring structure at acceleration space measurement points; The data processing module includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the full-field reconstruction method for structural dynamic displacement as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the full-field reconstruction method for dynamic structural displacement as described in any one of claims 1 to 8.

Images