A method for reconstructing structural response based on visual aided adaptive period

Abstract

The application relates to the technical field of structural health monitoring, and discloses a structural response reconstruction method based on visual auxiliary adaptive cycles, which comprises the following steps: acquiring multi-modal sensor data, calculating the statistics of known observation nodes, normalizing the statistics, saving a mapping dictionary, combining autocorrelation analysis and frequency domain amplitude spectrum to extract low-frequency quasi-periodic and high-frequency periodic characteristics to construct a candidate cycle set, establishing a parallel processing branch to perform two-dimensional rearrangement and feature extraction, acquiring independent reconstruction features, weighting and fusing the independent reconstruction features according to global statistical characteristics, calling the mapping dictionary and utilizing a preset spatial distribution coefficient matrix to deduce the statistical value of a missing node and perform inverse normalization. The application eliminates the dependence on a pure computer large model, combines structural spatial distribution constraints and multi-modal characteristics, eliminates dimensional differences, solves the pain point that missing nodes cannot be inversely normalized, and improves reconstruction accuracy.

Description

Technical Field

[0001] This invention relates to the field of structural health monitoring technology, specifically to a structural response reconstruction method based on visual-assisted adaptive cycles. Background Technology

[0002] In structural health monitoring, acquiring dynamic response data of a structure by deploying sensors is fundamental to assessing its safe service status. However, due to practical engineering limitations such as sensor hardware failures, data transmission interruptions, or restricted installation locations, response data for target nodes are often missing or cannot be directly measured. Therefore, it is necessary to use limited known observation node data to extrapolate the response sequence of unknown nodes; this process is known as structural response reconstruction.

[0003] Currently, conventional structural response reconstruction methods mainly rely on single-type sensor data. When multimodal sensors (such as accelerometers and visual measurement devices) are introduced for joint monitoring, existing data processing methods often struggle to achieve precise alignment in both the time and physical dimensions due to differences in sampling rates and physical dimensions among different hardware devices. This can easily lead to the loss of original physical magnitude information during data fusion. Furthermore, the dynamic response of structures in real-world complex service environments often contains mixed periodic components caused by multiple excitation sources, including both low-frequency quasi-periodic patterns and high-frequency periodic vibrations.

[0004] Traditional time-series prediction models typically treat response data directly as a one-dimensional sequence or use a fixed single period extraction method. This lacks the ability to deeply extract and adaptively capture the multiple mixed periodic features in non-stationary signals, resulting in insufficient analysis of complex dynamic response patterns. Furthermore, to handle long-term monitoring data, existing technologies generally employ piecewise sliding windows for truncation calculations. However, in the sequence splicing stage after calculation, there is often a lack of tracking of local statistical features and effective smoothing mechanisms for overlapping regions. This not only leads to numerical shifts when reconstructing data back to the true physical magnitude but also easily generates significant boundary truncation errors at the splicing points of adjacent time windows, making it difficult to obtain continuous, stable, and accurately amplitude-accurate structural response reconstruction sequences. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a structure response reconstruction method based on visual-assisted adaptive periodicity. This method solves the problems of existing structure response reconstruction methods, which are unable to effectively integrate multimodal sensor data with different sampling rates and dimensions, and cannot accurately capture and process the multiple mixed periodic features in complex non-stationary structural responses. These problems result in distortion of the physical magnitude of the reconstructed data, insufficient accuracy, and boundary truncation errors in time sequence splicing.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a structural response reconstruction method based on visual-assisted adaptive periodicity, comprising the following steps: Acquire multimodal sensor sample data, calculate the amplitude mean and standard deviation of known observation node channels, perform normalization to obtain input feature sequences, and save the amplitude mean and standard deviation in the mapping dictionary; The input feature sequence is input into the time domain and frequency domain analysis branches. Low-frequency quasi-periodic features and high-frequency periodic features are extracted by autocorrelation analysis and frequency domain amplitude spectrum, respectively. The candidate period set is then deduplicated and constructed. Multiple parallel processing branches are established based on the candidate period set. The input feature sequence is rearranged in two dimensions and cross-period dimension feature is extracted. The independent reconstructed feature sequence of each branch is obtained by reverse reconstruction. Calculate the global statistical features of the input feature sequence and obtain the weight coefficients of each branch. After weighted fusion of the independent reconstructed feature sequences, map them to the mapped feature sequence of the target node to be reconstructed. The mean amplitude and standard deviation stored in the mapping dictionary are called, and the statistical estimates of the target node channel to be reconstructed are derived using the preset spatial distribution coefficient matrix. Based on this, the mapping feature sequence is inversely normalized and spliced ​​to obtain the structural response reconstruction sequence.

[0007] Furthermore, acquiring multimodal sensor sample data includes: acquiring multi-channel acceleration sequences and visual displacement sequences; using external timestamps to establish a global time reference for interpolation and resampling to align the time dimension with the sampling rate; stitching the time-aligned sequences along the channel dimension to form a multimodal aligned sequence; and using a sliding window to slice the multimodal aligned sequence to obtain sample data of a fixed dimension, which serves as multimodal sensor sample data.

[0008] Furthermore, extracting low-frequency quasi-periodic features through autocorrelation analysis includes: calculating the autocorrelation sequence of the input feature sequence along the time dimension in the time-domain analysis branch; identifying local maxima points in the autocorrelation sequence where the correlation coefficient is greater than a set threshold; excluding the main peak with zero hysteresis; and extracting the hysteresis of discrete sampling points corresponding to local maxima points that meet the threshold condition, or calculating the sampling point spacing between two adjacent local maxima points as low-frequency quasi-periodic features.

[0009] Furthermore, the process of deduplicating and constructing the candidate period set includes: summarizing low-frequency quasi-periodic features and high-frequency periodic features to construct an initial period set; if the absolute value of the difference between any two period values ​​in the initial period set is less than a set period tolerance threshold, then the larger period value is removed; calculate the quotient of the larger and smaller period values ​​in the initial period set, and if the absolute value of the difference between the quotient and the nearest integer is less than a harmonic determination threshold, then the larger period value is removed.

[0010] Furthermore, the two-dimensional rearrangement of the input feature sequence includes: within the parallel processing branch, appending zero-value data to the end of the time axis of the input feature sequence for the assigned period values ​​to pad the time dimension; decomposing the length-padded feature sequence into cross-period dimension and intra-period phase dimension, and rearranging it along the channel dimension into a two-dimensional feature matrix with the number of rows equal to the number of complete period segments and the number of columns equal to the assigned period values. Cross-period dimension feature extraction and inverse reconstruction to obtain the independent reconstructed feature sequences of each branch includes: using a multilayer perceptron composed of linear mapping layers and nonlinear activation functions to extract features along the cross-period dimension direction of the two-dimensional feature matrix to obtain a two-dimensional hidden state matrix; concatenating and flattening the two-dimensional hidden state matrices of each channel row by row along the row direction to restore them to one-dimensional time-series data, and pruning the excess parts at the end to obtain the independent reconstructed feature sequences.

[0011] Furthermore, calculating the global statistical features of the input feature sequence and obtaining the weight coefficients of each branch includes: performing one-dimensional adaptive average pooling on the input feature sequence to obtain the dimensionality-reduced global statistical features; projecting the input fully connected neural network module onto the complete periodic set space to obtain the global evaluation vector; using the candidate periodic set as an index mask to extract the element scores at corresponding positions in the global evaluation vector; and processing the data using a normalized exponential function to obtain the weight coefficients.

[0012] Furthermore, the weighted fusion mapping to the target node to be reconstructed includes: while keeping the total length of the time step unchanged, using a fully connected neural network layer without nonlinear activation functions to perform matrix multiplication on the feature vectors of the channel dimension of the observed node at independent time sampling steps; and performing dimensional transformation by using a weight matrix with a shape and size equal to the total number of nodes to be reconstructed multiplied by the total number of observed nodes, to obtain a mapping feature sequence with the number of channels equal to the number of channels of the target node to be reconstructed.

[0013] Furthermore, the process of performing inverse normalization and splicing to obtain the structural response reconstruction sequence includes: using the statistical estimates of the target node channel to be reconstructed obtained through deduction, performing inverse linear scaling calculation on the mapping feature sequence to obtain time segment reconstruction data; arranging the time segment reconstruction data in chronological order, extracting multiple predicted response values ​​at the same time sampling step within the overlapping time period and calculating the arithmetic mean, and splicing them to obtain the structural response reconstruction sequence.

[0014] Furthermore, before being applied to structural response reconstruction, there is a model training stage. The model training stage includes: using historical monitoring records containing real data of all nodes in the structure as a training set to obtain mapping feature sequences, calculating the error with the real standardized observation sequence as a loss function to back-update the weight parameters; in the training stage, extracting the statistics of known observation nodes and target nodes to be reconstructed in all historical time windows in parallel, using the least squares method to perform multiple linear regression fitting, and solidifying the spatial distribution coefficient matrix.

[0015] This invention provides a structural response reconstruction method based on visual-assisted adaptive periodicity. It has the following beneficial effects: 1. This invention overcomes the problems of sampling rate differences and dimension inconsistencies between different sensor hardware by interpolating, resampling, and normalizing multi-channel acceleration sequences and visual displacement sequences. At the same time, it saves the mean amplitude and standard deviation of known observation node channels in the mapping dictionary, retains the physical scale information of the original data, and provides a standard-aligned multimodal input basis for subsequent feature extraction.

[0016] 2. This invention utilizes a dual-branch approach combining time-domain and frequency-domain analysis. Low-frequency quasi-periodic and high-frequency periodic features are extracted through autocorrelation analysis and frequency-domain amplitude spectrum analysis, respectively. Multiple parallel processing branches are established based on a candidate period set, rearranging the one-dimensional time series into a two-dimensional matrix for feature learning. Combined with an adaptive period selector that dynamically allocates weight coefficients to each branch based on the global statistical characteristics of the data, this approach accurately captures multiple mixed periodic patterns in structural responses, improving the method's ability to analyze complex and non-stationary structural response signals.

[0017] 3. This invention utilizes a fully connected neural network layer for spatial mapping of the channel dimension and calls upon the mean and standard deviation of the amplitude stored in the mapping dictionary. Combined with a pre-set spatial distribution coefficient matrix, it derives the statistical estimates of the target node's channels to be reconstructed, and performs inverse normalization accordingly. This mechanism overcomes the limitation of missing nodes being unable to be directly reverse-mapped due to the lack of real data, restoring the true physical magnitude of the target node's reconstructed data. Furthermore, the predicted values ​​within the overlapping time period of the sliding window are processed by arithmetic averaging and sequentially concatenated, eliminating boundary truncation errors caused by segmented calculations and ensuring the continuity and stability of the final output response reconstruction sequence. Attached Figure Description

[0018] Figure 1 This is a flowchart of the structure response reconstruction method based on visual assistance adaptive periodicity according to an embodiment of the present invention; Figure 2 This is a comparison diagram of the reconstructed waveforms of the target node displacement response in an embodiment of the present invention; Figure 3 This is a comparison chart of the absolute reconstruction errors of different methods in embodiments of the present invention. Detailed Implementation

[0019] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] See attached document Figure 1 This invention provides a structural response reconstruction method based on visual-assisted adaptive periodicity, comprising: Step S100: Obtain the multi-channel acceleration sequence and visual displacement sequence during structural operation. Establish a global time reference using an external timestamp, and perform interpolation and resampling on the multi-channel acceleration sequence and visual displacement sequence to achieve temporal alignment. Use a sliding window to slice the time-aligned sequence to obtain sample data of a preset length. Perform reversible instance normalization on the sample data to obtain a standardized input feature sequence.

[0021] Step S200: Input the input feature sequence into the time-domain analysis branch and the frequency-domain analysis branch, respectively. Low-frequency quasi-periodic features are obtained through autocorrelation analysis in the time-domain analysis branch. The dominant frequency of the signal is extracted through the frequency-domain analysis branch and converted into high-frequency periodic features. The low-frequency quasi-periodic features and the high-frequency periodic features are merged to construct a candidate period set.

[0022] Step S300: Establish multiple parallel processing branches based on the candidate period set. Within each parallel processing branch, rearrange the one-dimensional input feature sequence into a two-dimensional feature matrix containing both period and channel dimensions according to the period length corresponding to that branch. Use a prediction network to learn features from the two-dimensional feature matrix, and then reverse-arrange the learned features back to the original time dimension to obtain the independent reconstructed feature sequences output by each parallel processing branch.

[0023] Step S400: Obtain the global statistical features of the input feature sequence. An adaptive period selector is used to calculate the global statistical features, obtaining the weight coefficients of each parallel processing branch in the current input state. The weight coefficients are then used to perform a weighted summation of the independently reconstructed feature sequences output by all parallel processing branches to obtain the fused feature sequence.

[0024] In step S500, the fused feature sequence is input into the linear output layer. While keeping the time step constant, the fused feature sequence is mapped from the observation node channel to the target node channel to be reconstructed. Using the statistics recorded during the reversible instance normalization process in step S100, an inverse normalization operation is performed on the mapped sequence to obtain the response reconstruction sequence of the target node to be reconstructed.

[0025] It should be noted that the computational modules involving internal weight parameters, such as the lightweight prediction network, adaptive period selector, and linear output layer, in the embodiments of the present invention need to be optimized and solidified through an offline training phase before being actually applied to structural response reconstruction.

[0026] Specifically, during the model training phase, the system first establishes a complete set of periods containing all possible discrete period values ​​based on the frequency range and sampling rate predicted by the structure. For each period value in the complete set of periods, a corresponding lightweight prediction network is initialized and trained to construct a pre-trained period branch library; simultaneously, the output dimension of the adaptive period selector is fixed to the total number of elements in the complete set of periods. Historical monitoring records containing complete real data of all observation nodes and target nodes under baseline conditions are used as the training set.

[0027] Following the same forward data processing flow as steps S100 to S501 above, the mapping feature sequence of the target node channel to be reconstructed is calculated and obtained. Based on this mapping feature sequence and the true standardized observation sequence of the target node to be reconstructed within the corresponding time period, the mean squared error or mean absolute error between the two is calculated and used as the loss function for network optimization. Subsequently, the backpropagation algorithm and gradient descent optimizer (such as the Adam optimizer) are used to perform end-to-end joint updates of all weight parameters in the prediction network, adaptive periodic selector, and linear output layer.

[0028] Simultaneously, during the training phase, the mean and standard deviation statistics of all observed nodes and target nodes to be reconstructed within all historical time windows are extracted in parallel. Least squares are used for multiple linear regression fitting, and the spatial distribution coefficient matrix projected from the observed node statistics to the target node statistics is calculated and fixed. After multiple iterations until the loss function converges, the weight parameters of each network module and the spatial distribution coefficient matrix are saved, allowing the model to be deployed for actual online response reconstruction tasks.

[0029] Step S100, obtaining the standardized input feature sequence, includes the following sub-steps: Step S101: Obtain the multi-channel acceleration sequence and visual displacement sequence during structural operation, and use an external timestamp to establish a global time base to interpolate and resample the multi-channel acceleration sequence and visual displacement sequence.

[0030] High-frequency vibration signals from localized areas of the structure are acquired by deploying accelerometer nodes on the surface of the tested structure to obtain multi-channel acceleration sequences. Simultaneously, a visual measurement camera deployed at a specific external observation location captures images of key target areas of the structure, obtaining video stream data. For the specific algorithm used to extract the visual displacement sequence from the video stream, those skilled in the art can employ optical flow methods or digital image correlation techniques; feature point displacement tracking is a well-known technique in the field and will not be elaborated upon here.

[0031] Since the accelerometer and visual measurement camera are independent in hardware architecture and have different signal sampling rates, a hardware-triggered synchronization mechanism is adopted. An external trigger simultaneously sends pulse signals to both the accelerometer and the visual measurement camera as the starting point for the global time reference. Using the time axis of the high-sampling-rate multi-channel acceleration sequence as the reference axis, a cubic spline interpolation algorithm is used to perform numerical calculations and node encryption on the low-sampling-rate visual displacement sequence. After resampling, each data point in the visual displacement sequence corresponds to the acceleration sequence in the time dimension. Subsequently, the time-aligned multi-channel acceleration sequence and the visual displacement sequence are concatenated along the channel dimension at the feature level, thus merging to form a multimodal aligned sequence.

[0032] Step S102: Use a sliding window to slice the time-aligned multimodal alignment sequence to obtain sample data of a preset length.

[0033] After acquiring continuous multimodal aligned sequences, a fixed time window length and sliding step size are set. The time window length is determined based on the estimated lowest-order natural frequency of the structure, ensuring that each time window contains at least one complete low-frequency vibration cycle. The sliding step size is determined based on a preset overlap rate, typically one-quarter to one-half of the time window length. At each sliding position, the system extracts all channel time series data within the current window and combines them into a sample matrix with a fixed dimension, i.e., sample data. Through the sliding window slicing operation, the temporal dependencies of the monitored signals can be preserved, while continuously long sequences are divided into easily processed short sequence segments to accommodate subsequent fixed-dimensional matrix calculations.

[0034] Step S103: Perform reversible instance normalization on the sample data to obtain a standardized input feature sequence.

[0035] The distribution of response data under different operating conditions often shifts, and directly inputting raw data with large absolute value ranges into the computational model can easily lead to numerical instability. To eliminate the interference of different physical dimensions and absolute amplitude ranges on subsequent periodic feature extraction, for the single sample data output in step S102, the statistics of each physical channel sequence are independently calculated along the time dimension, and the sequence is scaled accordingly. The normalization calculation logic is shown in the following formula: ; in, Represents the first in the sample data One physical measurement channel; Indicates the first time within the current time window One time sampling step; Indicates the first digit before normalization. The physical measurement channel in the first The original measurement amplitude of each time sampling step; Indicates the first The average amplitude of each physical measurement channel within the current time window; Indicates the first The standard deviation of the amplitude of each physical measurement channel within the current time window; This represents a very small constant to prevent the denominator from being zero, and its value is usually in the range of 10. 8 Up to 10 5 ; This represents the dimensionless standardized eigenvalues ​​output after scaling.

[0036] During the channel-independent scaling calculations described above, the system simultaneously builds a mapping dictionary in memory, recording and saving the mean amplitude of each channel in each sample data. and amplitude standard deviation The retained statistical parameters serve as the basis for subsequent inverse mapping, ensuring accurate recovery of the signal's true physical amplitude scale during the feature reconstruction stage. The scaled data from each channel are combined to form a standardized input feature sequence.

[0037] Step S200 involves inputting the input feature sequence into the time-domain analysis branch and the frequency-domain analysis branch, respectively, to construct the candidate period set. The specific process includes the following sub-steps: Step S201: Calculate the autocorrelation sequence of the input feature sequence in the time domain analysis branch and extract low-frequency quasi-periodic features.

[0038] Visual displacement signals in structural health monitoring primarily reflect the macroscopic low-frequency motion of the structure, which exhibits a clear low-frequency quasi-periodic pattern in the time domain. Autocorrelation sequences can quantify the similarity between a time series and itself at different time lags. For signals containing periodic components, their autocorrelation sequences will show significant peaks at integer multiples of the period's lag. Therefore, this physical property can be used to effectively identify the potential periodicity of a signal.

[0039] For the standardized input feature sequence, its autocorrelation sequence is calculated along the time dimension. For the calculation of the autocorrelation function of discrete time series, those skilled in the art can use a fast Fourier transform algorithm based on Wiener-Khinchin's theorem to accelerate the process, which is a well-known technique in the field and will not be elaborated here.

[0040] After obtaining the autocorrelation sequence, a peak-finding algorithm is used to identify local maxima. To eliminate spurious correlation peaks caused by noise, a correlation amplitude threshold is set, and only local maxima with correlation coefficients greater than this threshold are retained. The correlation amplitude threshold is typically set between 0.3 and 0.5. Among the selected local maxima, the main peak with zero hysteresis is excluded, and the hysteresis of the discrete sampling point corresponding to the first local maximum that meets the threshold condition is extracted, or the sampling point spacing between two adjacent local maxima is calculated and used as a low-frequency quasi-periodic feature.

[0041] Step S202: Perform spectral analysis on the input feature sequence in the frequency domain analysis branch, extract the dominant frequency, and convert it into high-frequency periodic features.

[0042] The acceleration signal of the structure mainly reflects local high-frequency vibration characteristics, which manifest as significant energy concentration in the frequency domain. A Fast Fourier Transform (FFT) is performed on the input feature sequence to convert the time-domain signal into a frequency-domain amplitude spectrum. In the acquired frequency-domain amplitude spectrum, the amplitudes corresponding to each discrete frequency component are arranged in descending order of magnitude, and several frequency points with the highest amplitude values ​​are selected as dominant frequencies. The number of dominant frequencies selected is typically set to 3 to 5 based on the estimated modal order of the structure.

[0043] After obtaining the dominant frequency, the physical quantities in the frequency domain need to be converted into the discrete time domain dimension required for subsequent model calculations, that is, converted into high-frequency periodic features represented by the number of discrete sampling points. The calculation logic for this periodic conversion is shown in the following formula: ; in, Indicates by the first The high-frequency periodic features obtained from the dominant frequency conversion are of positive integer data type; This represents the rounding function for numerical values; The unified global sampling rate of the multimodal alignment sequence in step S100 is expressed in Hz. This represents the extraction of the first value from the frequency domain amplitude spectrum. The dominant frequency is expressed in Hz.

[0044] Step S203: Merge low-frequency quasi-periodic features with high-frequency periodic features, and construct a candidate period set through deduplication.

[0045] The low-frequency quasi-periodic characteristics output from the time-domain analysis branch and the high-frequency periodic characteristics output from the frequency-domain analysis branch are combined to construct a one-dimensional initial period set. Since joint analysis of the time and frequency branches can produce similar numerical evaluation results when processing complex structural multi-source response signals, or where the high-frequency period happens to be a harmonic component of the low-frequency period, directly using the initial period set for subsequent calculations would lead to redundancy in the processing branches.

[0046] Therefore, distance evaluation and deduplication are performed on the initial period set. This deduplication includes removing similar periods and harmonic periods. For similar period removal, a period tolerance threshold is set, and the absolute difference between any two period values ​​in the initial period set is calculated. If the absolute difference is less than the preset period tolerance threshold, the two periods are determined to represent similar dynamic evolution patterns in the time domain, and the system retains only the one with the smaller value. Considering the resolution of discrete sampling points, this period tolerance threshold is determined based on the uniform global sampling rate, and its value is typically set to the range of 2 to 5 discrete sampling points.

[0047] For harmonic period elimination, the quotient of the larger and smaller period values ​​in the initial period set is calculated. If the absolute difference between this quotient and the nearest integer is less than a preset harmonic determination threshold, the larger period is determined to be an approximately integer multiple of the smaller period, indicating a harmonic relationship. In this case, the larger period value is eliminated, and the smaller period value representing the fundamental frequency is retained. This harmonic determination threshold is typically set to 0.05 to 0.1. After the above screening and elimination operations, the remaining, non-redundant discrete integer sequences constitute the final multi-scale candidate period set.

[0048] Step S300, which establishes multiple parallel processing branches based on the candidate period set, includes the following sub-steps: Step S301: Based on the period values ​​in the candidate period set, dynamically call and instantiate the corresponding number of parallel processing branches from the pre-trained period branch library, and perform a length padding operation on the time dimension of the input feature sequence for each parallel processing branch.

[0049] After obtaining the multi-scale candidate period set, the system instantiates the same number of data processing streams (i.e., parallel processing branches) within the computational model based on the number of period elements contained in the set. Each parallel processing branch is uniquely assigned a specific period value from the candidate period set.

[0050] Since the total length of the input feature sequence extracted by the sliding window in step S100 may not be divisible by the period values ​​allocated to each branch, direct partitioning would result in incomplete data at the end. Therefore, within each parallel processing branch, the remainder of the original total length of the time steps divided by the allocated period value is calculated. If the remainder is not zero, the difference between the allocated period value and the remainder is calculated as the number of steps to be padded. Zero-value data corresponding to the number of steps is appended to the end of the time axis of the input feature sequence, so that the length of the time dimension of the padded feature sequence is divisible by the allocated period value. Since the input feature sequence has been normalized in the preprocessing stage and its overall mean is zero, performing zero-value padding at the end of the sequence is essentially equivalent to extending it based on the average response state of the channel, avoiding the artificial introduction of high-frequency abrupt interference. After the length padding operation, it is ensured that subsequent processing can completely divide it into integer number of equal-length period segments.

[0051] Step S302: The padded feature sequences in each parallel processing branch are rearranged according to the corresponding period length to construct a two-dimensional feature matrix containing cross-period dimension and intra-period phase dimension.

[0052] In a one-dimensional long-time-series structural response signal, different data points separated by an integer number of periods often exhibit similar phase states. To extract this phase consistency pattern, a dimensionality transformation is performed on the padded feature sequence. Specifically, the time dimension, whose length is an integer multiple of the period, is decomposed into two independent dimensions: the cross-period dimension and the intra-period phase dimension.

[0053] At the tensor operation level, the one-dimensional time series of each channel is rearranged into a multi-row, multi-column two-dimensional feature matrix. The number of rows in the matrix equals the number of complete period segments it contains, corresponding to the cross-period dimension; the number of columns equals the period values ​​assigned to that branch, corresponding to the phase dimension within the period. Through this dimensional rearrangement mechanism, in-phase data points that are far apart in the one-dimensional sequence are aligned to the same column of the two-dimensional feature matrix. This operation transforms the long-range dependencies in traditional long-sequence modeling into local mappings between adjacent elements in the same column, reducing computational complexity and feature extraction difficulty.

[0054] Step S303: Use a lightweight prediction network to perform feature learning on the two-dimensional feature matrix to obtain the reconstructed two-dimensional hidden state matrix.

[0055] After dimensional alignment, a lightweight prediction network is used to independently process the two-dimensional feature matrices of each channel. Those skilled in the art can use a multilayer perceptron composed of linear mapping layers and nonlinear activation functions as the lightweight prediction network. The prediction network extracts features along the cross-period dimension of the two-dimensional feature matrix, enabling information interaction between data at the same phase point in different periods. This allows the network to learn the nonlinear evolution of the structural response across multiple period spans and filter out non-periodic random noise. The core nonlinear mapping calculation logic is shown in the following formula: ; in, Represents the first in the sample data One physical measurement channel; This represents the corresponding two-dimensional feature matrix, whose shape and size are... ,in The total number of periodic segments across the periodic dimension. A specific periodic value to be assigned; This represents the weight matrix parameters used for cross-period feature extraction in a lightweight prediction network. These parameters are used to ensure that the matrix size remains unchanged after feature learning, facilitating subsequent reconstruction operations. The shape size is set to Array; This represents the bias term parameter, whose size is... Column vectors; This represents a nonlinear activation function; here, a modified linear unit activation function is used. This represents the corresponding two-dimensional hidden state matrix output after feature learning, whose shape and size are maintained at 1. constant.

[0056] Step S304: Perform reverse dimension rearrangement and data truncation on the two-dimensional hidden state matrix to obtain the independent reconstructed feature sequences output by each parallel processing branch.

[0057] After completing the feature learning across the cycle dimension, perform the dimension reshaping operation, which is the opposite of step S302. Concatenate the two-dimensional hidden state matrices of each channel in chronological order, merge the cross-cycle dimension and the intra-cycle phase dimension, and restore them to a one-dimensional time-series data form.

[0058] Since zero-value padding was introduced in step S301 to satisfy the divisibility condition, the restored one-dimensional time series data is longer than the original input feature sequence. Therefore, based on the total length of the time steps of the original input feature sequence, the excess portion at the end of the restored one-dimensional time series data is pruned and deleted to restore its time dimension length to its original state. The time series data of each channel after pruning and alignment are combined to form the independent reconstructed feature sequence output by the current parallel processing branch. Each independent reconstructed feature sequence output by the parallel processing branch represents the dynamic response reconstruction hypothesis of the structure at the corresponding single candidate period scale.

[0059] Step S400, which involves obtaining the fused feature sequence using weight coefficients, includes the following sub-steps: Step S401: Perform pooling operation on the input feature sequence along the time dimension to extract global statistical features that reflect the current input state.

[0060] The input feature sequence contains measurement information across multiple channel dimensions and a dynamic evolution process over time. To assess the current macroscopic operating condition of the structure, it is necessary to reduce the impact of high-frequency fluctuations in the time dimension on the overall state assessment. A one-dimensional adaptive average pooling calculation is performed along the time dimension on the standardized preprocessed input feature sequence. This calculation process compresses the time series data within each measurement channel from a multi-dimensional time step length into a single scalar value, which quantifies the average response energy state of the corresponding physical measurement channel within the current time window. Combining the compressed scalar values ​​from all channels forms a one-dimensional vector with channel-dimensional length, which is the dimensionality-reduced global statistical feature. Extracting the global statistical feature enables the model to perceive the current overall observation state input distribution pattern.

[0061] Step S402: An adaptive period selector is used to perform nonlinear mapping and normalization calculation on the global statistical features to obtain the weight coefficients of each parallel processing branch in the current input state.

[0062] Under different external stimuli or changes in operating conditions, the dominant dynamic response period of a structure will drift. An adaptive period selector is used to determine and allocate the contribution ratio of different period scales based on the currently extracted global statistical features. The lower-level feature implementation of the adaptive period selector is a fully connected neural network module containing two layers of linear mapping networks. The extracted global statistical features are input into this module. The first layer of the linear mapping network performs dimensionality reduction on the input one-dimensional vector, and the number of output channels is determined according to a preset dimensionality reduction ratio, which is typically one-quarter to one-eighth of the total number of channels in the input feature sequence. The dimensionality-reduced features undergo nonlinear processing through a modified linear unit activation function, and are then input into the second layer of the linear mapping network. The second layer of the linear mapping network further projects the dimensionality-reduced features onto a complete periodic set space, outputting a global evaluation vector with a length equal to the total number of elements in the complete periodic set.

[0063] After obtaining the global evaluation vector, the candidate period set constructed in step S200 is used as an index mask to extract the element scores corresponding to the candidate period positions in the global evaluation vector, forming a local evaluation vector with a length equal to the total number of current parallel processing branches. Subsequently, the values ​​of the local evaluation vector need to be converted into proportional coefficients that can be used for allocation. The local evaluation vector is processed using a normalized exponential function. For the calculation logic of the normalized exponential function, those skilled in the art can use the Softmax function algorithm, whose multi-class output probability distribution calculation is a well-known technique in the field and will not be elaborated here. After normalization, each real-valued element in the local evaluation vector is mapped to a probability value between 0 and 1, ensuring that the sum of all elements equals 1. These mapped probability values ​​are the weight coefficients allocated to the corresponding parallel processing branches. The magnitude of the weight coefficients quantifies the relative importance of each period hypothesis branch under the current data slice features.

[0064] Step S403: Use weight coefficients to perform weighted summation and integration of the independent reconstructed feature sequences output by all parallel processing branches to generate a fused feature sequence.

[0065] After obtaining the dynamic weight coefficients of all branches, a scalar multiplication operation is performed on the independent reconstructed feature sequences generated independently in step S300 by each parallel processing branch and their assigned weight coefficients. After the reverse dimensional rearrangement in the above steps, the time dimension and channel dimension of each independent reconstructed feature sequence are restored to be consistent. Subsequently, all weighted independent reconstructed feature sequences are added element-wise at the corresponding channel and time step positions. The calculation logic of this dynamic feature integration is shown in the following formula: ; in, This represents the total number of cycles contained in the multi-scale candidate cycle set. Its value is equal to the total number of parallel processing branches and is a positive integer greater than or equal to 1. Indicates the number of currently participating calculations. The index number of each parallel processing branch; Indicates by the first The independent reconstructed feature sequences corresponding to the single-cycle assumption output by each parallel processing branch; This indicates that the adaptive periodic selector is the first... Each parallel processing branch calculates the output weight coefficients, and satisfies ; This represents the fused feature sequence output after the multi-scale periodic information superposition and integration is completed.

[0066] By introducing weighting coefficients for weighted summation, the data processing model can dynamically increase the characteristic response weight of the dominant periodic branch that best matches the current working condition based on the real-time observed vibration state distribution, while reducing the weight of redundant periodic features that are not obvious or have deviations under the current working condition, thus completing the weighted integration of multi-scale periodic features.

[0067] Step S500, obtaining the response reconstruction sequence of the target node to be reconstructed, includes the following sub-steps: Step S501: Input the fused feature sequence into the linear output layer, and map the fused feature sequence from the observation node channel to the target node channel to be reconstructed while keeping the time step unchanged.

[0068] After weighted integration of multi-period features, the fused feature sequence remains within the channel dimension space of the known observation nodes. Under operational conditions, the dynamic response of each physical measurement point within the same structural system is constrained by the overall structural stiffness and mass distribution, exhibiting a fixed spatial correlation. Utilizing this physical characteristic, the fused feature sequence is input into a linear output layer for spatial projection. The lower-level features of the linear output layer are implemented as fully connected neural network layers without nonlinear activation functions.

[0069] During the computation, the linear output layer operates independently along the time dimension of the fused feature sequence, maintaining a constant total length of time steps. At each independent time sampling step, matrix multiplication is performed on the feature vectors of the observation node's channel dimension. Specifically, the number of channels in the input fused feature sequence is set to the total number of observation nodes, and the number of target mapping channels is set to the total number of nodes to be reconstructed. The linear output layer internally constructs a weight matrix with a shape and size equal to the product of the total number of nodes to be reconstructed and the total number of observation nodes. Through this matrix multiplication, the channel dimension of the input features is transformed from the number of channels in the observation nodes to the number of channels in the target nodes to be reconstructed, thus obtaining the mapped feature sequence. This operation completes the data transformation from the observation sensor location space to the target missing sensor location space.

[0070] Step S502: Perform an inverse normalization operation on the mapped feature sequence using the statistics recorded during the reversible instance normalization process to restore the true physical dimensions of the target node to be reconstructed.

[0071] The output mapping feature sequence after spatial mapping is still in a dimensionless, standardized numerical state. To recover its true physical dimensions and amplitude scale, inverse normalization calculation is required. The mean and standard deviation statistical parameters stored in the mapping dictionary in step S100 are called. Since there is no measured data for the target node to be reconstructed in the current time period, the system uses the mean and standard deviation recorded by known observation channels, and performs a weighted summation through a pre-calculated spatial distribution coefficient matrix to deduce the corresponding mean and standard deviation estimates for the target node to be reconstructed within the current time window. This spatial distribution coefficient matrix was pre-trained and solidified during the training phase before model deployment using complete historical monitoring data containing all node channels under the structural baseline state; it records the linear mapping weights between the observed node statistics and the target node statistics.

[0072] After obtaining the estimated statistical parameters of the target node to be reconstructed, an inverse linear scaling calculation is performed on the mapping feature sequence. The logic of this inverse normalization calculation is shown in the following formula: ; in, Indicates the index number of the target node channel to be reconstructed; Indicates the first time within the current time window One time sampling step; Represents the first element in the mapping feature sequence. The target node channel to be reconfigured is in the first... Standardized dimensionless eigenvalues ​​for each time sampling step; This indicates the number of times the simulation is performed within the current time window. The mean amplitude estimate of the channel of each target node to be reconstructed; This indicates the number of times the simulation is performed within the current time window. Estimated amplitude standard deviation of the target node channel to be reconstructed; This represents the response value with true physical dimensions output after inverse normalization. Through the above calculations, the time segment reconstruction data corresponding to a single sliding time window is obtained.

[0073] Step S503: Overlap and stitch the time segment reconstruction data corresponding to each sliding window along the time axis to obtain the response reconstruction sequence of the target node to be reconstructed.

[0074] Since the original continuous long sequence was sliced ​​and preprocessed using a sliding window with a fixed step size in step S100, the current inverse normalization output data only represents the response state within a local time segment. Based on the original timestamp information, the system rearranges all independently processed time segment reconstruction data in chronological order and places them on the global time axis.

[0075] During continuous stitching, due to the set ratio of the sliding step size to the time window length, there will be overlap in time range between the reconstructed data of adjacent time segments. Directly truncating the stitching will cause numerical jumps at the window boundaries. Therefore, for the same time sampling step within the overlapping time period, the system will obtain multiple different predicted response values. The system extracts multiple predicted response values ​​from the same time sampling step and calculates their arithmetic mean, using this arithmetic mean as the final reconstruction output for that time sampling step. Through overlapping stitching and averaging operations, numerical fluctuations at the boundaries of adjacent time windows can be smoothed, eliminating stitching gap errors caused by data truncation. After complete stitching of all time segments, a continuous and complete response reconstruction sequence of the target node to be reconstructed is output.

[0076] Specific application example: Response reconstruction of a five-story shear-type building structure Experimental platform and data acquisition: A five-story steel-frame shear-type building structure model was used as the test object. The following monitoring system was deployed when the structure was subjected to random excitation forces from the base: Observation nodes (known data): Accelerometers are installed on the 1st, 3rd and 5th layers to collect high-frequency vibration signals of the local structure and obtain multi-channel acceleration sequences with a sampling rate of 100Hz; simultaneously, a visual measurement camera is set up outside the structure to capture key target areas of the structure and extract the visual displacement sequence of the target on the 5th layer with a video sampling rate of 30Hz.

[0077] Target node to be reconstructed (missing data): Assuming that no sensors are installed in layers 2 and 4, this embodiment will use the response reconstruction sequence of the target node to be reconstructed in layer 4 as an example for illustration.

[0078] Implementation details of the refactoring process: Perform online inference according to the steps of the method of the present invention: Multi-source data preprocessing (step S100): Using the time axis of a multi-channel acceleration sequence with a high sampling rate as the reference axis, a cubic spline interpolation algorithm is used to perform numerical calculations and node densification on the 30Hz visual displacement sequence, increasing its sampling rate to 100Hz to achieve time dimension alignment. The time window length of the sliding window is set to 1000 sampling points (i.e., 10 seconds), and the sliding step size is 250 points. Reversible instance normalization is performed on the sample data obtained from the slices. Obtain the dimensionless standardized eigenvalues ​​output after scaling. A mapping dictionary is built in memory to record and save the mean amplitude of each channel in each sample data. and amplitude standard deviation .

[0079] Candidate period extraction (step S200): Identify and extract low-frequency quasi-periodic features in the time-domain analysis branch. =85 (number of discrete sampling points). Extract the first two dominant frequencies from the frequency domain analysis branch. and and using the formula Convert it to the first High-frequency periodic characteristics obtained from the dominant frequency conversion =28 and =14. By merging low-frequency quasi-periodic features and high-frequency periodic features and performing deduplication, a multi-scale candidate period set {85,28,14} is constructed.

[0080] Parallel sparse modeling (step S300): Instantiate three parallel processing branches based on the candidate period set. Taking the branch with a period value of 85 as an example, perform a length padding operation on the time dimension of the input feature sequence so that the length of the time dimension of the padded feature sequence is divisible by 85. Then, rearrange the one-dimensional sequence into a two-dimensional feature matrix containing cross-period and channel dimensions. Use a lightweight prediction network to perform feature learning on the two-dimensional feature matrix to obtain the corresponding two-dimensional hidden state matrix output after feature learning. Subsequently, reverse dimension rearrangement and data truncation are performed on the two-dimensional hidden state matrix to obtain the independent reconstructed feature sequences output by each parallel processing branch.

[0081] Adaptive multi-cycle fusion (step S400): An adaptive cycle selector is used to calculate the global statistical features and obtain the weight coefficients of each parallel processing branch in the current input state. For example, in the current evaluation state, the adaptive cycle selector is the first... The weighting coefficients of the output are calculated by each parallel processing branch. The values ​​are [0.72, 0.20, 0.08]. Using the formula... For the first The outputs of each parallel processing branch correspond to the independently reconstructed feature sequences under the single-period assumption. Weighted summation integration is performed to generate a fused feature sequence output after the multi-scale periodic information superposition integration is completed. .

[0082] Spatial Mapping and Reconstruction (Step S500): The fused feature sequence is input to the linear output layer and matrix multiplication is performed. The channel dimension is transformed from the number of channels in the observation nodes (3) to the number of channels in the target node to be reconstructed (1), obtaining the mapped feature sequence. The pre-trained spatial distribution coefficient matrix and known measurement point statistics are used to calculate the th element within the current time window obtained through deduction. The mean amplitude estimate of the target node channel to be reconstructed Within the current time window obtained from the simulation, the first Estimated amplitude standard deviation of the target node channel to be reconstructed Perform inverse normalization calculation Obtain the response value with real physical dimensions after inverse normalization. Finally, multiple predicted response values ​​from the same time sampling step are extracted and their arithmetic mean is calculated. These values ​​are then overlapped and stitched together to obtain a continuous response reconstruction sequence for the target node to be reconstructed.

[0083] Experimental verification and effect comparison: The method of this invention is compared with the traditional one-dimensional CNN method that directly performs end-to-end convolution mapping on time-series signals without periodic dimension rearrangement.

[0084] Regarding the fitting comparison of the reconstructed waveforms, refer to the appendix. Figure 2 The diagram shows a comparison of the reconstructed waveforms of the target node displacement response. The horizontal axis represents time (in seconds), with a 2-5 second segment used to illustrate local waveform details. The vertical axis represents the displacement amplitude (in millimeters) of the target node to be reconstructed (layer 4). The light gray solid line represents the theoretical standard displacement data actually collected by sensors under baseline conditions, serving as a benchmark for evaluating reconstruction accuracy. The dark gray dotted line represents the displacement sequence reconstructed using the traditional one-dimensional CNN method. This line shows misalignment with the light gray solid line at peaks and troughs, exhibiting phase delay and localized jagged high-frequency spikes. The black dashed line represents the displacement sequence reconstructed using the method of this invention; this line has a high degree of overlap with the light gray solid line.

[0085] For the quantitative analysis of reconstruction error, refer to the appendix. Figure 3The graph showing the absolute error analysis of different reconstruction methods uses time (in seconds) on the horizontal axis, capturing data from the 2nd to the 5th second; and absolute error (in millimeters) on the vertical axis, representing the absolute difference between the predicted displacement and the actual observed displacement. A value closer to 0 indicates higher reconstruction accuracy. The dark gray solid line represents the error distribution of the traditional one-dimensional CNN method at each time step; this line shows high values ​​and significant fluctuations, with peak errors exceeding 1.0 mm multiple times. The black solid line represents the error distribution of the proposed solution at each time step; this line is close to the bottom 0 axis, with the absolute error generally remaining below 0.1 mm.

[0086] Statistical results from the test set data show that the root mean square error (RMSE) of the reconstructed fourth-layer displacement sequence using the method of this invention is 0.85 mm, while the RMSE of the traditional one-dimensional CNN method is 2.14 mm. The scheme provided by this invention reduces the phase deviation and error magnitude of local feature reconstruction.

Claims

1. A structural response reconstruction method based on visual-assisted adaptive periodicity, characterized in that, include: Acquire multimodal sensor sample data, calculate the mean amplitude and standard deviation of known observation node channels, perform normalization to obtain input feature sequences, and save the mean amplitude and standard deviation in the mapping dictionary; The input feature sequence is input into the time domain and frequency domain analysis branches, and low-frequency quasi-periodic features and high-frequency periodic features are extracted by autocorrelation analysis and frequency domain amplitude spectrum, respectively, and duplicates are removed to construct a candidate period set. Multiple parallel processing branches are established based on the candidate period set, and the input feature sequence is rearranged in two dimensions and extracted across the period dimension. The independent reconstructed feature sequence of each branch is obtained by reverse reconstruction. Calculate the global statistical features of the input feature sequence and obtain the weight coefficients of each branch. After weighted fusion of the independent reconstructed feature sequences, map them into the mapping feature sequence of the target node to be reconstructed. The mean amplitude and standard deviation stored in the mapping dictionary are called, and the statistical estimates of the target node channel to be reconstructed are derived using a preset spatial distribution coefficient matrix. Based on this, the mapping feature sequence is inversely normalized and spliced ​​to obtain the structural response reconstruction sequence.

2. The structural response reconstruction method based on visual-assisted adaptive periodicity according to claim 1, characterized in that, The acquisition of multimodal sensor sample data includes: Acquire multi-channel acceleration and visual displacement sequences, and use external timestamps to establish a global time base for interpolation and resampling to achieve alignment of time dimension and sampling rate. The time-aligned sequences are spliced ​​along the channel dimension to form a multimodal aligned sequence. A sliding window is used to slice the multimodal aligned sequence to obtain sample data of a fixed dimension, which is used as the multimodal sensor sample data.

3. The structural response reconstruction method based on visual-assisted adaptive periodicity according to claim 1, characterized in that, The extraction of low-frequency quasi-periodic features through autocorrelation analysis includes: In the time-domain analysis branch, the autocorrelation sequence of the input feature sequence is calculated along the time dimension, and local maxima points with correlation coefficients greater than a set threshold are identified in the autocorrelation sequence. Exclude the main peak with zero hysteresis, extract the hysteresis of discrete sampling points corresponding to local maxima that meet the threshold condition, or calculate the sampling point spacing between two adjacent local maxima as the low-frequency quasi-periodic feature.

4. The structural response reconstruction method based on visual-assisted adaptive periodicity according to claim 1, characterized in that, The deduplication candidate period set includes: By combining the low-frequency quasi-periodic features with the high-frequency periodic features, an initial period set is constructed. If the absolute value of the difference between any two period values ​​in the initial period set is less than the set period tolerance threshold, then the larger period value is removed. Calculate the quotient of the larger and smaller period values ​​in the initial period set. If the absolute value of the difference between the quotient and the nearest integer is less than the harmonic determination threshold, then the larger period value is removed.

5. The structural response reconstruction method based on visual-assisted adaptive periodicity according to claim 1, characterized in that, The two-dimensional rearrangement of the input feature sequence includes: Within the parallel processing branch, zero-value data is appended to the end of the time axis of the input feature sequence to fill the time dimension length for the assigned period values; The length-padded feature sequence is decomposed into a cross-period dimension and an intra-period phase dimension, and rearranged along the channel dimension into a two-dimensional feature matrix with the number of rows equal to the number of complete period segments and the number of columns equal to the assigned period values.

6. The structural response reconstruction method based on visual-assisted adaptive periodicity according to claim 5, characterized in that, The cross-period dimension feature extraction and inverse reconstruction to obtain the independent reconstructed feature sequences of each branch include: A two-dimensional hidden state matrix is ​​obtained by extracting features along the cross-period dimension of the two-dimensional feature matrix using a multilayer perceptron composed of a linear mapping layer and a nonlinear activation function. The two-dimensional hidden state matrices of each channel are stitched together and flattened row by row along the row direction to restore one-dimensional time series data, and the redundant part at the end is trimmed to obtain the independent reconstructed feature sequence.

7. The structural response reconstruction method based on visual-assisted adaptive periodicity according to claim 1, characterized in that, The calculation of the global statistical features of the input feature sequence and the acquisition of the weight coefficients of each branch include: One-dimensional adaptive average pooling is performed on the input feature sequence to obtain the dimensionality-reduced global statistical features. The input is then projected into a fully connected neural network module onto a complete periodic set space to obtain a global evaluation vector. The candidate period set is used as an index mask to extract the element scores at corresponding positions in the global evaluation vector, and the weight coefficients are obtained by processing them using a normalized exponential function.

8. The structural response reconstruction method based on visual-assisted adaptive periodicity according to claim 1, characterized in that, The weighted fusion mapping to the target node to be reconstructed includes the following mapping feature sequence: While keeping the total length of the time step constant, a fully connected neural network layer without nonlinear activation functions is used to perform matrix multiplication on the feature vectors of the channel dimension of the observation node at independent time sampling steps; By performing dimensional transformation on a weight matrix whose shape and size are the product of the total number of nodes to be reconstructed and the total number of observed nodes, a mapping feature sequence in which the number of channels is equal to the number of channels of the target node to be reconstructed is obtained.

9. The structural response reconstruction method based on visual-assisted adaptive periodicity according to claim 1, characterized in that, The sequence of performing inverse normalization and splicing to obtain structural response reconstruction includes: Using the statistical estimates of the target node channel to be reconstructed obtained through deduction, the mapping feature sequence is subjected to inverse linear scaling calculation to obtain time segment reconstruction data; The reconstructed data of the time segments are arranged chronologically, and multiple predicted response values ​​at the same time sampling step within the overlapping time period are extracted and their arithmetic mean is calculated. The reconstructed sequence of the structural response is then obtained by splicing them together.

10. The structural response reconstruction method based on visual-assisted adaptive periodicity according to claim 1, characterized in that, Before practical application to structural response reconstruction, there is a model training phase, which includes: The mapping feature sequence is obtained by using historical monitoring records containing real data of all nodes as the training set, and the error with the real standardized observation sequence is calculated as the loss function to update the weight parameters in reverse. During the training phase, statistics of known observation nodes and target nodes to be reconstructed within all historical time windows are extracted in parallel. The least squares method is used to perform multiple linear regression fitting, and the spatial distribution coefficient matrix is ​​solidified.

Images