Bridge entity model multi-sensor fusion image interactive perception system and method thereof

Abstract

The application discloses a bridge entity model multi-sensing fusion image interactive perception system and method, relates to the field of bridge mechanics teaching display and structure health perception technology, and comprises the following components: a strain and acceleration sensor pre-buried in a key component, a scaled bridge entity model with a surface pasted coded target, a camera array, a multi-sensing frequency domain decoupling fusion unit, a digital twin visualization unit, a touch virtual load interaction unit and a bidirectional linkage control unit; based on geometric similarity ratio, structural dynamics similarity law is used to determine the dynamic and static response frequency domain demarcation frequency, visual displacement is used as a quasi-static low-frequency channel, acceleration is used as a dynamic high-frequency channel, strain is used as a local stiffness constraint channel, and full-field perception data flow is decoupled, independently processed and seamlessly spliced; the data flow is used for online calibration of the physical consistency inversion model of the touch virtual load to realize bidirectional closed-loop linkage of entity loading and virtual loading, and immersive bridge mechanics interactive teaching is achieved.

Description

Technical Field

[0001] This invention relates to the field of bridge mechanics teaching demonstration and structural health perception technology, specifically to a multi-sensor fusion image interactive perception system and method for bridge physical models. Background Technology

[0002] As a crucial node in modern transportation infrastructure, the understanding of bridges' mechanical principles and stress mechanisms has always been an important part of civil engineering education, popular science exhibitions, and engineering personnel training. Traditional bridge teaching and exhibition methods mainly rely on two-dimensional formula derivations, finite element simulation animations, or static scaled-down physical models. While static scaled-down physical models can intuitively present the geometric configuration and component combinations of bridges, visitors or learners often rely on verbal explanations or abstract diagrams in textbooks for imaginative understanding of core mechanical mechanisms such as internal stress distribution, deformation response, and dynamic characteristics under external loads, failing to achieve a truly immersive cognitive experience. In recent years, with the rapid development of structural health monitoring, computer vision, and digital twin technologies, some research has begun to explore combining sensor perception, image measurement, and virtual visualization to provide new pathways for presenting the stress state of bridge structures.

[0003] Chinese invention patent application CN115752250A discloses a high-precision displacement monitoring method for bridges that integrates computer vision and acceleration. The method involves placing an accelerometer at a target measurement point on the target bridge and using a tripod to fix an image acquisition device at a stable position outside the bridge to capture a sequence of vibration time history images of the measurement point under load. Based on a template matching algorithm, the measurement point is tracked at the sub-pixel level to obtain the displacement time history curve of the measurement point in the image coordinates. At the same time, the acceleration response is subjected to a regularized quadratic numerical integration to obtain the dynamic displacement of the measurement point. Subsequently, in the time domain, a pair of complementary filters with a sum of amplitudes of 1 and a sum of phases of 0 are used to fuse the visual displacement and acceleration displacement data. The low-pass component of the visual displacement is taken as the quasi-static displacement, and the high-pass component of the acceleration displacement is taken as the dynamic displacement. However, when the applicant applied the aforementioned time-domain complementary filtering scheme to the scaled-down physical model used in bridge teaching demonstrations, they found that the scheme could not function effectively. On the one hand, due to the effect of the structural dynamics similarity law, the dynamic response frequency band of the scaled-down model was compressed to a high-frequency band that was twice the geometric similarity ratio of the prototype, while the quasi-static response remained concentrated in the extremely low-frequency band. The two types of signals were already almost completely separated in the frequency domain. Forcibly unifying and fusing them in the time domain would result in the acceleration dynamic information in the high-frequency band being truncated by the low-pass filter due to the fixed selection of the complementary filter cutoff frequency, and the visual displacement in the low-frequency band being submerged by the low-frequency noise of the quadratic integral. On the other hand, the scheme completely failed to utilize the independent local stiffness constraint measure provided by the embedded strain sensor, nor did it establish a teaching loop that involved interactive touch-screen loading and two-way linkage between the digital twin cloud map and the visitor. As a result, when visitors saw the color cloud map, they could neither see the complete quasi-static deformation process nor the real dynamic attenuation characteristics, and the core value of the immersive teaching and cognitive experience could not be achieved. The applicant discovered in-depth research that the root cause of the aforementioned bottleneck is not the lack of sophistication in the fusion algorithm itself, but rather a long-standing counterintuitive cognitive bias in the field. The similarity law scaling effect of the scaled model is regarded as an unfavorable constraint that degrades accuracy, while ignoring that it is precisely a natural frequency domain decoupler: the geometric similarity ratio explicitly gives the frequency domain boundary frequency of the dynamic and static responses in the form of a power of -1 / 2. As long as the three heterogeneous signals of strain, acceleration and vision are decoupled on both sides of the frequency domain according to this boundary frequency and then seamlessly spliced, the complete quasi-static deformation process and the real dynamic attenuation characteristics can be obtained simultaneously on the scaled model. This further supports the physical consistency inversion of touch virtual loads and the two-way closed-loop linkage between physical and digital twins, thereby upgrading bridge mechanics teaching from abstract formula derivation to an immersive interactive cognitive experience. Summary of the Invention

[0004] Addressing the core bottleneck of existing technologies where scaled-down bridge teaching and display models are often viewed as accuracy degradation constraints rather than frequency domain decoupling tools due to the scaling effect of the similarity law, resulting in the ineffective decoupling and fusion of three heterogeneous sensing signals—strain, acceleration, and visual sub-pixel tracking—and the lack of physical consistency mapping between virtual loads applied by visitors through touch interfaces and the actual physical responses of the physical model, this invention provides a multi-sensor fusion image interactive perception system and method for bridge physical models. This system reverses the scaling effect of the similarity law in scaled-down physical models into a natural frequency domain decoupling mechanism, driving the independent processing and seamless splicing of the three channels—strain sensor, acceleration sensor, and coded target sub-pixel visual displacement—on both sides of the boundary frequency to construct a full-field real-time perception data stream. This full-field real-time perception data stream is then used to online calibrate the physical consistency inversion model of the virtual loads applied by visitors through touch. Under the premise of scaled-down, non-destructive, and small-sample interaction, this system achieves an immersive bridge mechanics interactive teaching and cognitive experience driven by a two-way closed-loop linkage between physical and virtual loading, based on the principle of structural dynamics similarity law.

[0005] The technical solution of this invention is as follows:

[0006] A multi-sensor fusion image interactive perception system for a bridge physical model includes a scaled-down bridge physical model, an image acquisition unit, a multi-sensor frequency domain decoupling and fusion unit, a digital twin visualization unit, a virtual load interaction unit, and a two-way linkage control unit. The scaled-down bridge physical model has strain sensor arrays and accelerometers embedded within its key load-bearing components, and coded targets are affixed to key nodes on its outer surface. The image acquisition unit includes a camera array fixedly positioned facing the scaled-down bridge physical model, used to continuously acquire image sequences of the coded targets and perform sub-pixel target tracking on the image sequences to output sub-pixel displacement signals. The multi-sensor frequency domain decoupling and fusion unit determines the dynamic and static response frequency domain boundary frequencies based on the geometric similarity ratio of the scaled-down bridge physical model to the prototype bridge according to the structural dynamic similarity law. Based on this, the sub-pixel displacement signals, the acceleration signals output by the accelerometers, and the strain signals output by the strain sensor array are respectively used as quasi-static low-frequency channels, dynamic high-frequency channels, and local stiffness constraint channels, undergoing independent frequency domain decoupling and seamless stitching to construct a real-time perception data stream across the entire field. The digital twin visualization unit drives the display screen to present the stress distribution and deformation amplitude of each component in the form of a color cloud map, based on the real-time perception data stream. The virtual load interaction unit includes a touch interface for receiving virtual loads applied by visitors. These virtual loads can be either virtual concentrated loads or virtual uniformly distributed loads. The virtual loads are then solved using a physical consistency inversion model to obtain a virtual response, which is calibrated online by the real-time perception data stream. The bidirectional linkage control unit overlays the virtual response onto the color cloud map of the digital twin visualization unit in real time, while simultaneously using the real-time perception data stream as a continuous calibration input for the physical consistency inversion model, thus achieving bidirectional closed-loop linkage between physical loading and virtual loading.

[0007] This invention also provides a multi-sensor fusion image interactive perception method for bridge physical models. The method includes the following steps: pre-embedding a strain sensor array and an acceleration sensor inside the key load-bearing components of a scaled-down bridge physical model and attaching coded targets to key nodes on the outer surface of the model; continuously acquiring image sequences of the coded targets using a fixed camera array and performing sub-pixel target tracking; determining the dynamic and static response frequency domain boundary frequency based on the geometric similarity ratio of the scaled-down bridge physical model relative to the prototype bridge according to the structural dynamics similarity law; and placing the sub-pixel displacement signal, the acceleration signal output by the acceleration sensor, and the strain signal output by the strain sensor array at the boundary frequency. The two sides are decoupled and processed independently, and seamlessly stitched together according to frequency bands to construct a real-time perception data stream for the entire field. The real-time perception data stream drives the display screen to present the stress distribution and deformation of the scaled-down bridge physical model in the form of a color cloud map. Virtual loads applied by visitors through the touch interface are received. The virtual loads are either virtual concentrated loads or virtual uniformly distributed loads. The physical consistency inversion model is calibrated online based on the real-time perception data stream to solve for the virtual response. The virtual response is superimposed on the color cloud map in real time, and the real-time perception data stream is continuously used as the online calibration input of the physical consistency inversion model to achieve bidirectional closed-loop linkage between physical loading and virtual loading.

[0008] The present invention has the following advantages over the prior art.

[0009] First, this invention, based on the structural dynamics similarity law, explicitly maps the geometric similarity ratio of a scaled-down solid model to the frequency domain boundary between dynamic and static responses, achieving decoupled independent processing and seamless splicing of the strain, acceleration, and visual channels on both sides of the frequency domain. The mechanism lies in the fact that the similarity law itself physically compresses the dynamic response frequency band of the scaled-down model relative to the prototype bridge to a high-frequency band that is a multiple of the negative half-power of the geometric similarity ratio, while the quasi-static response is always concentrated in the extremely low-frequency band. These two types of responses are naturally separated in the frequency domain, but were previously considered unfavorable constraints that degraded accuracy rather than decoupling tools. This invention reverses and utilizes this similarity relationship, allowing quasi-static deformation information to be entirely provided by sub-pixel target tracking displacement, dynamic response information to be entirely provided by the quadratic integral of the high-pass acceleration component, and local stiffness constraints to be independently provided by strain sensors and directly used to correct the scaling factor of visual measurements. Each channel operates independently within its optimal frequency band without interference, thus simultaneously achieving quasi-static integrity and dynamic high-frequency response integrity in the scaled-down model. This is something that a pure time-domain two-channel complementary filtering scheme cannot achieve in scaled-down scenarios.

[0010] Second, this invention establishes a real-time, two-way closed-loop linkage between the touch-sensitive virtual load and the physical consistency inversion model. The mechanism lies in the fact that each touch event by a visitor simultaneously generates two effects: inputting a virtual load into the digital twin visualization cloud map, and evoking a free vibration transient response in the physical model at the moment of geometric contact. This invention treats the latter as a natural excitation source for online calibration of the inversion model, synchronously extracting the actual damping ratio and stiffness correction coefficient under the current working condition from the aforementioned three-channel sensing data stream. These are used as parameter inputs when the physical consistency inversion model solves for the virtual response, ensuring that the virtual response is consistent with the actual physical characteristics of the physical model in terms of amplitude, time scale, and damping decay curve. This synergistic effect of two-way linkage is not only significantly superior to the technical effect of a one-way static visualization scheme that can only play pre-calculated animations, but also significantly superior to a one-way scheme that only performs order reduction correction for actual experimental loading deviations. The virtual load of this invention does not change the physical model itself, but by inverting the physical parameters from the transient response of the physical model in real time, it achieves physical consistency under non-destructive premise, thus simultaneously taking into account the two originally contradictory goals of zero physical damage and high virtual-real consistency in teaching and demonstration scenarios.

[0011] Third, this invention achieves deep coupling between the teaching demonstration function and the self-monitoring function of the physical model by reversing the byproduct of the transient response of touch into an emergent indicator of the model's health status. The mechanism lies in the fact that the free vibration decay process triggered by each visitor's touch carries the true damping information of the current model. This invention inputs the damping ratio sequence extracted from each touch into a statistical control chart for trend analysis. Furthermore, in small-sample scenarios, it constructs a physical constraint threshold using a finite element reduced-order sensitivity matrix based on the Rayleigh damping assumption to replace the pure statistical control limit. Thus, even under the constraint of a limited number of touches by visitors in a teaching demonstration scenario, it can still reliably identify cumulative micro-damage to the physical model caused by repeated interactions, such as loosening of adhesive joints, decrease in bolt preload, or changes in support friction, without requiring any additional sensors or acquisition equipment. This synergistic effect of one data stream serving two purposes is completely absent in the aforementioned digital twin full-lifecycle damage prediction scheme for real bridges, because the latter does not have the natural transient excitation source of touch-based virtual loads, and cannot rely on physical priors to replace statistical thresholds in small-sample scenarios. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the overall architecture of the bridge entity model multi-sensor fusion image interactive perception system provided in the embodiment of the present invention.

[0013] Figure 2 This is a schematic diagram illustrating the implementation process of the multi-sensor fusion image interactive perception method for bridge entity models provided in this embodiment of the invention. Detailed Implementation

[0014] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, so that those skilled in the art can more clearly understand the technical solution of the present invention. However, the specific embodiments described below do not constitute any limitation on the scope of protection of the present invention.

[0015] See Figure 1 The bridge entity model multi-sensor fusion image interactive perception system provided in this embodiment consists of six functional modules, namely, scaled-down bridge entity model and sensor array module 1, image acquisition module 2, multi-sensor frequency domain decoupling fusion module 3, digital twin visualization module 4, virtual load interaction module 5, and bidirectional linkage control module 6. These six modules are physically independent, but form a complete closed loop in terms of data flow, from physical perception to virtual presentation, and then from virtual input back to physical calibration. The multi-sensor frequency domain decoupling fusion module 3 is the core module of the entire system.

[0016] The scaled-down bridge solid model and sensor array module 1: In this embodiment, a typical steel box girder cable-stayed bridge is selected as the prototype bridge. Its main span is 300m, the main girder material is Q345 steel, and the first-order vertical natural frequency characteristic value of the main girder is 0.42Hz. The scaled-down bridge solid model is made according to a geometric similarity ratio of 1:20, that is, the geometric similarity ratio is taken as 0.05. Therefore, the main span of the scaled-down bridge solid model is 15m. The main girder, bridge tower, cables, piers, and supports of the scaled-down bridge solid model are all scaled down in terms of material parameters and geometric dimensions according to the similarity law. The main girder is a box section welded from 3mm thick steel plate, with a cross section width of 200mm and a cross section height of 80mm; the bridge tower is a rectangular steel pipe with a thickness of 4mm and a height of 3m; the cables are high-strength steel wire ropes with a diameter of 2mm; and the supports are pre-processed rubber pads and pre-tightened to rigid bases with bolts.

[0017] To achieve in-situ sensing of the internal mechanical state of the scaled-down bridge model, an array of strain sensors was pre-embedded inside the key load-bearing components of the main girder. Each strain sensor is a resistance strain gauge adhered to the inner side of the top and bottom plates of the box girder, with a nominal resistance of 120Ω, a nominal sensitivity coefficient of 2.0, and a measurement error better than one percent. The strain sensors were deployed strictly according to the principle of symmetry, located at the lower edge of the mid-span of the main girder, the upper and lower edges of the left quarter-span section, the upper and lower edges of the right quarter-span section, the outer sides of the roots of the left and right piers, and the steel pad directly above the supports, totaling eight measuring points. This coverage ensures the distribution of the main stress response of the scaled-down bridge model under symmetrical and asymmetrical loads. The strain sensors are led out via shielded cables to a static strain data acquisition instrument inside the model base, with a sampling frequency of 1000Hz.

[0018] The accelerometers selected are triaxial accelerometers with a range of ±2g and a frequency response of 0.5Hz to 500Hz, based on a microelectromechanical system (MEMS). They are pre-embedded at four points: the inner side of the top slab at mid-span of the main girder, the inner side of the top slab of the left quarter-span, the inner side of the top slab of the right quarter-span, and the top of the main bridge tower. These points are used to capture the dynamic vibration response of the scaled-down bridge model under external loads. The accelerometers are led out to the dynamic signal acquisition instrument via thin-diameter coaxial cables, with a sampling frequency of 2000Hz. Both the static strain data acquisition instrument and the dynamic signal acquisition instrument achieve cross-device time synchronization with the camera system via the IEEE-1588 precision time protocol, with a hardware synchronization accuracy better than 50µs, thus ensuring sub-millisecond time alignment accuracy between the strain channel, acceleration channel, and vision channel.

[0019] Encoded targets were affixed to key nodes on the outer surface of the scaled-down bridge model. These targets, 30mm in diameter and featuring embedded 6×6-bit ArUco coding patterns, are high-contrast adhesive targets. Each target has a unique numerical identifier throughout the model for identification. The targets are positioned at 16 points: the mid-span of the main girder, three quarter-points on each side, two eighth-points on each side, the middle and top of the main towers, and the tops of the left and right piers, comprehensively covering the key nodes most easily observed by visitors. All coded targets had their initial physical coordinates calibrated using a high-precision 3D coordinate measuring machine before the model left the factory.

[0020] Image acquisition module 2 includes two industrial-grade CMOS cameras fixed on both sides of the scaled-down bridge model. Each camera has a resolution of 2048×1536 pixels, a maximum frame rate of 200fps, and is equipped with a 12mm fixed-focus industrial lens. They are positioned approximately 3 meters from the scaled-down bridge model, with the lens optical axis slightly tilted downwards to cover the field of view of all 16 coded targets. The two cameras are connected to the image acquisition workstation via a GigE Vision interface and achieve cross-device time synchronization with the data acquisition instrument using the aforementioned IEEE-1588 protocol.

[0021] During the system installation and initialization phase, the intrinsic parameters of each camera, including focal length, principal point position, radial distortion coefficient, and tangential distortion coefficient, are calibrated using a precisely sized checkerboard calibration board. Subsequently, the extrinsic parameters of the relative pose between the two cameras are calibrated by observing the checkerboard pattern together. Finally, a scaled-down bridge solid model coordinate system is established with the geometric center point at the lower edge of the main beam mid-span as the origin. A rigid body mapping relationship from the pixel coordinate system to the model's physical coordinate system is established using the factory calibration coordinates of the coded target. All of the above calibration procedures are implemented using the mature Zhang Zhengyou calibration method in this field, and will not be elaborated upon here.

[0022] During normal operation, image acquisition module 2 performs synchronous real-time sub-pixel target tracking on the image sequences output by the two cameras. The tracking process first converts the color image sequence to a grayscale image sequence. Then, based on the standard ArUco target detection function, it locates the integer-pixel center of each coded target. Subsequently, it takes the normalized cross-correlation distribution of 5×5 pixels in the neighborhood of the integer-pixel center as fitting data. The sub-pixel center coordinates are then obtained through least-squares fitting using a two-dimensional Gaussian function, with the processing delay per frame controlled within 1ms. Finally, the sub-pixel coordinates of each coded target in each frame are output and concatenated along the time axis to form the displacement-time history curve of each measurement point in the image coordinate system. This is further converted into displacement-time history data in physical units using the aforementioned pixel-to-physical coordinate mapping relationship, serving as the original input for the visual channel of the full-field real-time perception data stream. The above-described sub-pixel target tracking method based on normalized cross-correlation and two-dimensional Gaussian fitting is a conventional technique in the field of computer vision, and its specific implementation details are not elaborated here.

[0023] The multi-sensor frequency domain decoupling and fusion module 3 is the core module of this invention. It is responsible for decoupling and processing three types of heterogeneous signals—strain, acceleration, and vision—in the frequency domain based on the structural dynamics similarity law, and then seamlessly stitching them together to construct a real-time sensing data stream across the entire field. The multi-sensor frequency domain decoupling and fusion module 3 further includes three sub-units: a frequency domain boundary frequency calculation sub-unit, a three-channel decoupling processing sub-unit, and a touch transient response sub-unit.

[0024] According to the structural dynamics similarity law, the frequency domain boundary frequency of the steady-state dynamic and static response is explicitly determined by the geometric similarity ratio between the scaled-down bridge solid model and the prototype bridge, as well as the first-order natural frequency eigenvalue of the prototype bridge. The calculation formula is as follows:

[0025] ,

[0026] in: The steady-state dynamic and static response frequency domain boundary frequency is a scalar with a value range of 5Hz to 200Hz and a unit of Hz. It is calculated by this formula and is used to divide the quasi-static low-frequency response segment and the dynamic high-frequency response segment into two independently processed frequency bands in the frequency domain. The decoupling margin coefficient is a scalar with a value range of 1.5 to 3.0 and a dimension of 1. It is determined by the relative interval of the first three natural modes of the prototype bridge on the frequency axis. If the value is too small, the frequency bands on both sides will penetrate each other and cause the decoupling to fail. If the value is too large, the information in the middle frequency band will be lost. The technical effect is to retain the sensing information to the maximum extent while ensuring the reliability of decoupling. The geometric similarity ratio of the scaled-down bridge entity model to the prototype bridge is a scalar with a value range of greater than 0 and less than or equal to 1 and a dimension of 1. It is predetermined by the design parameters of the entity model. The technical effect is to directly map the geometric scaling relationship to the frequency scaling relationship. The first-order natural frequency characteristic value of the prototype bridge is a scalar, ranging from 0.1Hz to 10Hz, with units of Hz. It is obtained from the modal analysis report or design documents of the prototype bridge, and its technical effect is to serve as a reference anchor point for frequency domain scaling. In this embodiment, the value is taken as... , , Hz, substituting into the above formula, yields... Hz. This steady-state boundary frequency will serve as the reference input for the three-channel decoupling processing subunit.

[0027] The three-channel decoupling processing subunit is at the steady-state boundary frequency. Under the guidance of [the relevant authority], the visual channel, acceleration channel, and strain channel signals are decoupled and processed independently, and then seamlessly stitched together in the frequency domain. The frequency domain fusion expression is as follows:

[0028] ,

[0029] The low-pass filter and the high-pass filter satisfy a complementary constraint. Strain channel weighting function It exhibits bandpass peak characteristics near the boundary frequency. Specifically:

[0030] ,

[0031] ,

[0032] in: The position coordinates are The measurement point at angular frequency The frequency domain fused displacement spectrum at the location is a complex vector with units of m·s. It is calculated by this formula and represents the expression of the full-field sensing displacement data stream after the three channels are decoupled and fused in the frequency domain. The three-dimensional position vector of the measuring point in the coordinate system of the scaled-down bridge physical model is a vector whose value covers the position of all sensing measuring points of the scaled-down bridge physical model. The unit is m and is predetermined by the system calibration stage. The technical effect is to anchor the full-field sensing data to the physical model coordinates. ω is the angular frequency, and ω is a scalar, ranging from 0 to the image sampling rate multiplied by ω. The unit is rad / s, and it is given by the frequency axis of the Fourier transform of the original time-domain signal; The Fourier transform of the visual displacement time history obtained by the sub-pixel target tracking is a complex vector with units of m·s. It is obtained by performing a Fourier transform on the visual measurement displacement time history and represents the original frequency domain data of the visual channel. The Fourier transform of the displacement time history after the acceleration signal undergoes a second numerical integration is a complex vector with units of m·s. It is obtained by performing a Fourier transform on the second integral displacement of the acceleration and represents the original frequency domain data of the acceleration channel. The Fourier transform of the local equivalent displacement obtained by converting the strain signal through the analytical relationship between strain, curvature and deflection is a complex vector with units of m·s. It is obtained by integrating the strain measurement value along the beam length according to the Bernoulli-Euler beam assumption to obtain the local deflection and then performing a Fourier transform. It represents the original frequency domain data of the strain channel. For For the cutoff frequency The frequency response function of the Butterworth low-pass filter is a scalar with a value range of 0 to 1 and a dimension of 1. Its technical effect is to extract the quasi-static low-frequency components of the visual channel and suppress the high-frequency noise introduced by the camera frame rate constraint. To and The frequency response function of the complementary high-pass filter is a scalar with a value range of 0 to 1 and a dimension of 1. Its technical effect is to extract the dynamic high-frequency components of the acceleration channel and suppress the low-frequency drift noise amplified by the double integral. The order of the Butterworth filter is a positive integer, ranging from 2 to 6. In this embodiment, it is set to 4. The technical effect is to control the steepness of the filter's transition band. Let be the steady-state boundary angular frequency, be a scalar, and be derived from . Calculated; The bandpass weighting function for the strain channel is a scalar with a value range of 0 to 1 and a dimension of 1. It is defined by the boundary angular frequency. The Gaussian window form centered on the strain channel provides local stiffness constraint correction in the visual acceleration transition frequency band by utilizing the independent physical measure of the strain channel. Here, represents the bandwidth parameter of the Gaussian window, and is a scalar; in this embodiment, it is taken as... 0.2 times, in rad / s, the technical effect is to control the bandwidth of the strain channel; is the weighting coefficient for the strain channel, is a scalar with a value range of 0.1 to 0.5 and a dimension of 1. It is determined by least squares fitting through static load calibration tests of the solid model. In this embodiment, it is set to 0.25. The technical effect is to avoid local noise pollution of the overall fusion result while ensuring the correction capability of the strain channel. This is the operator for the natural exponential function. The real-time sensing data stream of the entire field is obtained by converting it back to the time domain using the inverse Fourier transform. It serves as the sole input for the subsequent digital twin visualization module 4 and the physical consistency inversion model.

[0033] The second-order numerical integration of the acceleration signal adopts a numerical integration method based on Tikhonov regularization to suppress low-frequency drift. The analytical conversion from strain to curvature to deflection is based on the Bernoulli-Euler beam assumption. The local strain is integrated along the beam length by the known moment of inertia of the cross section and the position of the neutral axis to obtain the local deflection estimate. Both are mature and conventional techniques in this field, and will not be elaborated here.

[0034] The touch transient response subunit is responsible for dynamically adjusting the frequency domain boundary frequency during the transient response phase triggered by a visitor's touch event, and for inverting the actual damping ratio of the scaled-down bridge physical model from the transient free vibration response. When the virtual load interaction module 5 outputs a touch event timestamp... At that time, the boundary frequency is adaptively adjusted according to the following time-varying strategy: ,

[0035] in: The time-varying frequency domain boundary frequency driven by touch transient events is a scalar time function with a value range of 5Hz to 500Hz, measured in Hz. It is calculated by this piecewise function and is used to temporarily shift the boundary frequency to the upper limit of the transient frequency during the touch transient phase, gradually change it back to the steady state through cosine interpolation during the transition phase, and restore it to the aforementioned state during the steady state phase. ; The system runtime is a scalar quantity in seconds, given by the system clock. For the first The timestamp of the touch event is a scalar, and its value is output by the virtual load interaction module 5 based on the touch sensor, with the unit being seconds. This is the touch event sequence number, a positive integer ranging from 1 to the total number of touches by the visitor; The transient hold window duration is a scalar value ranging from 30ms to 80ms. In this embodiment, it is set to 50ms. The value is based on the time constraint required for the damping of the first three modes of the scaled-down model to decay to 10% of the original excitation. If the value is too small, the acquisition of modal information will be missed. If the value is too large, the continuity of the steady-state boundary frequency will be disrupted. is the transient redundancy coefficient, which is a scalar with a value range of 1.2 to 2.0 and a dimension of 1. In this embodiment, it is taken as 1.5. The technical effect is to reserve sufficient frequency domain margin above the upper limit of transient frequency to avoid aliasing when touch impulses are broadened in the frequency domain. The current geometric similarity ratio The upper limit of the first three natural frequencies of the scaled-down bridge solid model described below is a scalar with a value range of 10Hz to 300Hz and the unit is Hz. It is obtained by conducting the first hammer impact modal test on the scaled-down bridge solid model. In this embodiment, 60Hz is used. The meaning is the same as in the aforementioned formula; The duration of the cosine gradient window is a scalar value ranging from 100ms to 300ms, with the unit being ms. In this embodiment, it is set to 200ms. The technical effect is to ensure the continuous differentiability of the time-varying process of the boundary frequency to avoid introducing pseudo-spectral information during the transition phase. This is the cosine function operator; Pi is a constant.

[0036] Within the transient hold window, the touch transient response subunit extracts the free vibration decay curve of the scaled-down bridge solid model under touch-induced impulse load from the sum of the subpixel displacement residual and the acceleration residual. Due to the duration of the hold window... Within a window of no more than 80ms, the traditional Fast Fourier Transform has a frequency resolution that is limited. For frequencies greater than or equal to 12.5Hz, the frequency structure of several Hertz intervals between the first three modes in a scaled-down bridge solid model cannot be resolved. To address the challenge of modal parameter estimation under this transient short window, the touch transient response subunit employs a matrix-beam time-domain parameter estimation method.

[0037] Specifically, let the free vibration attenuation sample sequence within the holding window be... ,in The total number of sampling points within the window, determined by the window duration. This is obtained by multiplying by the dynamic sampling rate, as shown in this embodiment. Construct two Hankel data matrices with a single displacement relationship:

[0038] ,

[0039] ,

[0040] For the first Hankel matrix Perform singular value decomposition And based on twice the number of modes of interest Before keeping One principal singular value is used to obtain the truncation matrix. , as well as Then, a matrix bundle is constructed within the reduced-order subspace based on two Hankel matrices:

[0041] ,

[0042] right Solving the eigenvalue problem yields generalized eigenvalues Each eigenvalue corresponds to a complex pole in the free vibration damping response:

[0043] ,

[0044] Deducing the natural angular frequency and damping ratio from the poles:

[0045] ,

[0046] in: and These are the first and second Hankel data matrices, respectively. Both are real matrices with dimension 1. The dimensions are the same as the original sample. It is obtained by arranging the free vibration sample sequence within the window in a time-shifted manner. The technical effect is to convert the time-domain time series information into an algebraic structure that can be solved by singular value decomposition. The matrix bundle parameter is a positive integer with a value range of 1. to In this embodiment, the following is taken: The technical advantage lies in achieving a balance between noise resistance and computational load. The total number of sampling points within the holding window is a positive integer, and its value is determined by the dynamic sampling rate and... Multiplying them together, we get 100 in this embodiment; for The left singular matrix obtained from singular value decomposition, It is a singular value diagonal matrix. It is a right singular matrix; , , Before and after retention The truncation matrix corresponding to each principal singular value; The number of principal singular values ​​to be retained is a positive integer, and its value is twice the number of modes of interest. In this embodiment, it is taken as... The first three vertical bending modes correspond to the scaled-down bridge solid model. Let be the matrix bundle constructed within the reduced-order subspace. The complex matrix, with a dimension of 1, is calculated from the aforementioned singular value decomposition order reduction result; For matrix bundle The There are several generalized eigenvalues, which are complex scalars with dimensionless 1, derived from the pair of... Solving the eigenvalue problem yields the following results; The first inference from the poles The first modal natural angular frequency is a scalar, the range of which is determined by the modal characteristics of the scaled-down model, and the unit is rad / s. The technical effect is to accurately characterize the modal frequency at the current moment. The first inference from the poles The first-order modal damping ratio is a scalar with a value range of 0 to 0.2 and a dimension of 1. Its technical advantage lies in serving as a direct input for online damping parameters in the physical consistency inversion model. The dynamic channel sampling time interval is a scalar value, which is the reciprocal of the dynamic sampling rate. In this embodiment, it is 0.5ms, or 0.0005s. and These are the operators for extracting the real part and the imaginary part of a complex number, respectively. The natural logarithm operator; The natural exponential function operator; The imaginary unit satisfies ; superscript Represents the matrix transpose operator; superscript This represents the matrix inversion operator.

[0047] The touch transient response subunit further extracts the first-order modal damping ratio from the aforementioned matrix bundle time-domain parameter estimation method for each touch event. Construct a time series of damping ratios in chronological order. (Superscript here) Indicates the first (Second touch event), and apply exponentially weighted moving average statistical control chart processing to the sequence to achieve trend analysis and anomaly identification. Specifically, the exponentially weighted moving average sequence is processed according to... Recursive calculation, where the smoothing coefficient The value is an empirical value between 0.15 and 0.35; in this embodiment, 0.25 is used. The processing of the exponentially weighted moving average statistical control chart is a conventional statistical process control technique in this field and will not be elaborated upon here.

[0048] Building upon this, the touch transient response subunit introduces a physical constraint threshold calibration module to address the unreliability of purely statistical control limits due to the limited cumulative touch counts by visitors in teaching demonstration scenarios. This module first establishes a reduced-order finite element model of beam elements for the scaled-down bridge solid model, with approximately 200 elements in total. Under the Rayleigh damping assumption, the damping matrix of the scaled-down bridge solid model can be expressed as a linear combination of the mass matrix and the stiffness matrix. The first-order modal damping ratio and the second-order modal damping ratio There is an analytical relationship between the natural angular frequencies of the first order:

[0049] ,

[0050] Furthermore, the first Comparison of first-order modal damping with the first Key stiffness parameters The sensitivity can be derived using the chain rule and modal sensitivity theory:

[0051] ,

[0052] It utilizes the modal sensitivity formula. The Rayleigh-Ritz first-order perturbation results. The physical constraint threshold is defined as the sum of the inner products of the initial damping reference and the sensitivity matrix multiplied by the allowable stiffness degradation vector:

[0053] ,

[0054] in: is the physical constraint threshold, is a scalar with a value range of 0.01 to 0.10 and a dimension of 1. It is calculated by this formula and is used to replace the pure statistical control limits of the exponentially weighted moving average statistical control chart. The initial damping reference value is denoted as , which is a scalar with a dimension of 1. It is determined by the first modal test of the scaled-down bridge solid model at the time of leaving the factory. In this embodiment, it is taken as 0.02. The safety margin factor is a scalar with a value ranging from 1.5 to 3.0 and a dimension of 1. In this embodiment, it is taken as 2.0, and the value is based on the conventional engineering safety factor. Let be the sensitivity vector of the first-order modal damping ratio to the key stiffness parameter vector. It is a column vector with a dimension equal to the number of key stiffness parameters and a unit of 1 / (N·m). It is obtained by taking the derivative of the aforementioned sensitivity formula with respect to all key stiffness parameters and arranging them in order. Let be the maximum allowable stiffness degradation vector, be a column vector, and have dimensions . The same applies, with units in N·m, and is given by the allowable degradation range of the scaled-down bridge solid model; in this embodiment, it is taken as 5% of the reference stiffness. The mass term coefficient for Rayleigh damping is a scalar with units of 1 / s. It is calculated by combining the damping ratios of the first two modes and the corresponding natural frequencies of the scaled-down bridge solid model. In this embodiment, it is 0.15 / s. Here, is the stiffness coefficient of the Rayleigh damping term, a scalar quantity in seconds, obtained by reverse derivation in the same manner. In this embodiment, it is... s; For the first The first-order mass-normalized mode shape vector, in units of It was obtained from the finite element modal analysis of the scaled-down bridge solid model; For the first The key stiffness parameters are scalars, with units of N·m, and correspond to key physical quantities such as the bending stiffness of the main beam, the vertical stiffness of the supports, and the axial stiffness of the cables in the scaled-down bridge solid model. This is the key stiffness parameter number, which is a positive integer and ranges from 1 to the total number of key stiffness parameters. For partial derivative operators; superscript This is the vector transpose operator, with the same meaning as in the aforementioned formula. When the exponentially weighted moving average sequence... The current value exceeds At that time, the multi-sensor frequency domain decoupling and fusion module 3 outputs a physical model status alarm signal to the system to indicate the cumulative micro-damage that may occur in the scaled-down bridge physical model, such as loosening of adhesive joints, attenuation of bolt preload, or changes in support friction. The above-mentioned physical constraint thresholds have a reliability that cannot be replaced by purely statistical control limits under the small sample constraint of less than 30 cumulative touches by visitors in a teaching demonstration scenario.

[0055] The digital twin visualization module 4 takes the real-time full-field sensing data stream output by the multi-sensor frequency domain decoupling and fusion module 3 as input. Through modal superposition interpolation of a pre-established scaled-down bridge solid model finite element model (approximately 5000 nodes), the sparse measurements of 8 strain measurement points, 4 acceleration measurement points, and 16 visual measurement points are expanded into a high-density stress distribution and displacement field across the entire field. Subsequently, a predefined continuous color mapping function from blue to cyan to green to yellow to red is used to map the stress values ​​to the displayed colors, presenting them in real-time on the display screen as a color cloud map. The overall refresh rate of the color cloud map is 30Hz, the display screen resolution is 3840×2160 pixels, and the diagonal size of the display screen is 65 inches (approximately 1.65m), positioned directly facing the viewer. The color cloud map also simultaneously displays auxiliary information such as the peak digital stress value, the maximum displacement value, and the amplitude and application location of the current virtual load. The finite element modal overlay interpolation, color mapping rendering, and overlay display involved in this module are all mature digital twin visualization technologies in this field, and will not be elaborated further here.

[0056] The virtual load interaction module 5 includes a multi-point capacitive touchscreen fixed in front of the scaled-down bridge model. The touchscreen has a diagonal size of 15.6 inches (approximately 0.40m) and a resolution of 1920×1080 pixels, displaying a real-time mirrored color cloud map of the digital twin visualization module 4. Visitors apply virtual loads by performing single-finger taps or two-finger drags on the touchscreen: a single-finger tap corresponds to applying a virtual concentrated load to the component of the model at the tapped location, with the tap duration linearly related to the concentrated load amplitude; a two-finger drag corresponds to applying a virtual uniformly distributed load to the component segment between the drag start and end positions, with the drag distance linearly related to the resultant force of the uniformly distributed load. In this embodiment, the mapping coefficient from tap duration to concentrated load amplitude is 500N / s, meaning a tap duration per second corresponds to a 500N load amplitude; the mapping coefficient from drag distance to the resultant force of the uniformly distributed load is 100N / cm, meaning a drag distance of one centimeter corresponds to a 100N resultant force. Touch gesture recognition is implemented based on the operating system's native multi-touch interface, which is a mature technology in this field.

[0057] The core of the virtual load interaction module 5 lies in feeding the virtual loads applied by visitors into a physically consistent inversion model calibrated online by the real-time sensing data stream across the entire site, thereby obtaining a virtual response consistent with the actual physical characteristics of the scaled-down bridge entity model. The physically consistent inversion model employs the master equation form of modal reduced-order finite element method, which... The decoupled modal coordinate dynamic equations are established by preserving the first mode: ,

[0058] The full-field displacement of the virtual response at its physical location is calculated by modal superposition:

[0059] ,

[0060] Modal parameters and By analyzing the real-time sensing data stream across the entire field Compared with the predicted output of the inversion model The optimal estimate is obtained by minimizing the mean square error, and the iterative update rule is as follows:

[0061] ,

[0062] ,

[0063] in: For the first The modal coordinates of the first mode preserved are scalar functions of time, with dimensions of . (Based on the definition of mass-normalized mode shape), it is obtained by solving the aforementioned master equation through time integration; and They are respectively The first and second derivatives with respect to time, in units of and ; For the first The current online estimate of the natural angular frequency of the first mode is a time scalar function. The range of the value is determined according to the modal characteristics of the scaled-down model. The unit is rad / s. It is obtained by online calibration according to the iterative update rule of this module. The initial value is taken from the first hammer impact modal test. For the first The current online estimate of the first-order modal damping ratio is a time scalar function, ranging from 0 to 0.2, with a dimension of 1. It is obtained through online calibration using the iterative update rules of this module, with the initial value taken from the transient damping ratio output by the touch transient response subunit. ; For the first quality normalization The first-order mode shape vector has the same meaning as the aforementioned formula; for In physical location The interpolation function value at the location, in units of ; The virtual load vector of the visitor received by the virtual load interaction module 5 is... A real vector of dimension N is generated in real time by parsing the touch gesture according to the aforementioned mapping coefficients. The number of modes retained for the physical consistency inversion model is a positive integer, ranging from 6 to 20, and is 12 in this embodiment; For position The virtual response displacement at that point is a time vector function, in meters, and is the final output of this module; For the first The calibration objective function for the next iteration is a scalar, with units of m squared; For the full-field real-time sensing data stream in the first... The displacement values ​​at each measuring point have the same meaning as the aforementioned formula, and the unit is meters (m). For the inversion model under the current modal parameters, at the th... The predicted displacement of each measuring point, in meters; The total number of valid measurement points in the real-time sensing data stream is 28 in this embodiment (including 8 strain measurement points, 4 acceleration measurement points and 16 visual measurement points). Here is the measurement point number, which is a positive integer, ranging from 1 to... ; The iteration number is a positive integer. Let be the iterative learning rate for the modal natural angular frequency, be a scalar with a value ranging from 0.001 to 0.01, and have dimensions of . In this embodiment, the value is 0.005; The iterative learning rate for the damping ratio is denoted by , which is a scalar with a value ranging from 0.0001 to 0.001 and a dimension of 1. In this embodiment, it is taken as 0.0005. Here, is the frequency regularization coefficient, a scalar, ranging from 0.001 to 0.1, with units of . In this embodiment, the value is 0.01. The technical effect is to avoid the online calibration deviating too far from the initial modal analysis result. For the first The initial value of the natural angular frequency of the first mode, in rad / s, is obtained from the initial hammer impact mode test; The L2 norm operator for vectors; For the summation operator, its subscript... superscript These represent the start and end points of the summation, respectively. Virtual response displacement. The color cloud map is superimposed on the digital twin visualization module 4 via the two-way linkage control module 6, enabling an instant response to changes in the color cloud map from the visitor's touch operation.

[0064] The bidirectional linkage control module 6 is the central coordinator for the system to achieve bidirectional closed-loop linkage between physical loading and virtual loading. At the data flow level, this module simultaneously connects to the multi-sensor frequency domain decoupling and fusion module 3, the digital twin visualization module 4, and the virtual load interaction module 5, coordinating communication and synchronization among the three in an event-driven manner. During normal steady-state operation (i.e., periods without touch events), the bidirectional linkage control module 6 pushes the real-time full-field sensing data stream output from the multi-sensor frequency domain decoupling and fusion module 3 to the digital twin visualization module 4 at a refresh rate of 30Hz to update the color cloud map. Simultaneously, it pushes the same data stream at a refresh rate of 1Hz to the physical consistency inversion model of the virtual load interaction module 5 as online calibration input for modal parameters. At this time, since there is no virtual load input, the physical consistency inversion model is in an idle calibration state, continuously updating its internal modal parameters. and This ensures that it always remains consistent with the actual current state of the entity model.

[0065] At the moment a touch event occurs, the two-way linkage control module 6 immediately performs the following coordinated operations: First, it captures the touch event timestamp. It forwards the data to the touch transient response subunit of the multi-sensor frequency domain decoupling and fusion module 3, triggering time-varying sliding of the boundary frequency and transient mode parameter estimation; secondly, it converts the virtual load vector after the touch gesture is parsed. The virtual response is solved by the physical consistency inversion model fed into the virtual load interaction module 5; thirdly, the displacement field of the virtual response obtained by solving the physical consistency inversion model is used. With the full-field real-time sensing data stream In the digital twin visualization module 4, the images are displayed in a fusion manner according to the overlay mode selectable by the visitor. In the pure virtual mode, only the virtual response cloud map is displayed. In the virtual-real comparison mode, the virtual response cloud map and the measured perception cloud map are displayed simultaneously in a split-screen format. In the virtual-real overlay mode, the two are overlaid according to an adjustable weight coefficient. Fourth, the newly extracted transient damping ratio is obtained from the touch transient response subunit. As a high-confidence initial value for the next modal parameter calibration iteration of the physical consistency inversion model, the two-way linkage control module 6 realizes real-time closed-loop linkage in two directions: from entity loading to data perception to virtual calibration and from virtual input to model deduction to cloud map presentation. This allows visitors to simultaneously observe the transient vibration of the physical model due to its real physical characteristics and the steady-state stress distribution presented by the digital twin cloud map in each touch interaction. The two are strictly consistent in key mechanical characteristics such as time scale, damping attenuation characteristics, and peak response amplitude, thereby achieving the core effect of the immersive bridge mechanics interactive teaching and cognitive experience.

[0066] The two-way linkage control module 6 adopts a publish-subscribe model in its software architecture. The modules exchange data through zero-copy shared memory, and the overall end-to-end latency is controlled within 50ms. The perceptible response latency of the color cloud map after the visitor touches it is lower than the visual persistence threshold of the human eye, ensuring the smoothness and immersion of the interaction process.

[0067] See Figure 2 Based on the bridge entity model multi-sensor fusion image interactive perception system described in the aforementioned system embodiment, the present invention also provides a corresponding bridge entity model multi-sensor fusion image interactive perception method, which includes the following six steps in sequence, each step corresponding to one of the six functional modules in the system embodiment.

[0068] Step S1: Deployment of the Solid Model and Sensor Array. In this step, a scaled-down solid model of the steel box girder cable-stayed bridge is fabricated according to the aforementioned 1:20 geometric similarity ratio. A strain gauge array with a nominal resistance of 120Ω is pre-embedded at eight key stress locations: the lower edge of the main girder mid-span, the upper and lower edges of the left and right quarter-span sections, the roots of the left and right piers, and the steel pad directly above the supports. Microelectromechanical systems (MEMS) triaxial accelerometers with a range of ±2g are pre-embedded at four locations: the inner side of the top plate at the mid-span of the main girder, the inner side of the top plates of the left and right quarter-spans, and the top of the main bridge tower. Sixteen key nodes, each with a diameter of 30mm and a 6x6 digit ArUco coded target, are affixed. All strain and accelerometer sensors are led out to a data acquisition instrument inside the model base via shielded cables, with sampling frequencies of 1000Hz and 2000Hz, respectively. The detailed implementation of this step is the same as described in the scaled-down bridge solid model and sensor array module 1 in the aforementioned system embodiment.

[0069] Step S2: Image Acquisition and Subpixel Target Tracking. In this step, two 2048×1536 pixel, 200fps CMOS industrial cameras, fixed approximately 3m apart on both sides of the scaled-down bridge model, synchronously and continuously acquire image sequences of the ArUco-coded targets. After grayscale preprocessing of the acquired image sequences, the ArUco target detection algorithm is used to locate the integer-pixel center of each target. Subsequently, the subpixel-level center coordinates are obtained within a 5×5 pixel neighborhood using two-dimensional Gaussian function least squares fitting. These coordinates are then converted into displacement time-history data in physical units using a pre-calibrated pixel-to-physical coordinate mapping relationship, serving as the raw input for the visual channel of the full-field real-time perception data stream. The detailed implementation of this step is the same as described in the image acquisition module 2 of the aforementioned system embodiment.

[0070] Step S3: Three-channel frequency domain decoupling and fusion step. In this step, firstly, according to the aforementioned structural dynamics similarity law formula... The steady-state dynamic and static response frequency domain boundary frequency is calculated. Substituting the prototype's first-order natural frequency of 0.42Hz, geometric similarity ratio of 0.05, and decoupling margin coefficient of 2.0, the steady-state boundary frequency is approximately 3.76Hz. Subsequently, a Fourier transform is performed on the sub-pixel visual displacement time history, and it is multiplied by the frequency response function of a Butterworth low-pass filter with an order of 4, using the steady-state boundary frequency as the cutoff frequency. A Tikhonov regularized quadratic integral is performed on the acceleration time history, followed by a Fourier transform and multiplied by the corresponding high-pass filter frequency response function. The strain time history is integrated along the beam length according to the Bernoulli-Euler beam assumption to obtain the Fourier transform of the local equivalent displacement, and multiplied by a Gaussian bandpass weighting function centered at the steady-state boundary angular frequency. The three-channel results are added in the frequency domain according to the aforementioned fusion expression and then inverse Fourier transformed back to the time domain to obtain the full-field real-time sensing data stream. The detailed implementation of this step includes all the contents of the multi-sensor frequency domain decoupling and fusion module 3 in the aforementioned system embodiment.

[0071] In addition, in this step, when a touch event timestamp is detected... At that time, the boundary frequency is shifted upward within the transient hold window according to the aforementioned piecewise function. (In this embodiment, 1.5 times 60Hz equals 90Hz). Within the touch-hold window, the free vibration decay curve is extracted from the subpixel displacement residual and acceleration residual. A matrix bundle time-domain parameter estimation method is used to construct a matrix bundle from the singular value decomposition of two Hankel data matrices into a reduced-order subspace and solve for the generalized eigenvalues. The transient mode frequency and damping ratio under the current touch event are obtained by back-calculating from the complex poles, providing online parameter input for the subsequent physical consistency inversion model. Simultaneously, the first-order damping ratio extracted from each touch is analyzed using an exponentially weighted moving average statistical control chart. A physical constraint threshold constructed from a finite element reduced-order sensitivity matrix based on the Rayleigh damping assumption is introduced to replace the pure statistical control limit. When the cumulative number of touches by the visitor is less than 30, the accumulated micro-damage of the physical model is reliably identified, and a status alarm signal is output.

[0072] Step S4: Digital Twin Visualization Step. In this step, the real-time full-field sensing data stream is used as input. The sparse measurement points are expanded into a high-density stress distribution and displacement field across the entire field using the modal superposition method of a pre-established 5000-node finite element model. The stress values ​​are converted into display colors according to a continuous color mapping function from blue to red, and the color cloud map is displayed in real-time on a 65-inch display screen at a 30Hz refresh rate. The detailed implementation of this step is the same as described in the digital twin visualization module 4 of the aforementioned system embodiment.

[0073] Step S5: Virtual Load Interaction and Physical Consistency Inversion Step. In this step, a single-finger click or two-finger drag gesture from the visitor is received via a 15.6-inch multi-touch capacitive touchscreen. A virtual load vector is generated based on the mapping from the press duration to a concentrated load of 500 N / s or the drag distance to a uniformly distributed load of 100 N / cm. This vector is then fed into the aforementioned modal reduced-order finite element master equation for solving. The modal parameters use the latest estimated values ​​obtained from the real-time sensing data flow across the entire field through iterative online calibration with minimum mean square error. A virtual response displacement field consistent with the actual physical characteristics of the physical model is calculated. The detailed implementation of this step is the same as that of the virtual load interaction module 5 in the aforementioned system embodiment. Specific formulas and symbol definitions are as described in the system embodiment.

[0074] Step S6: Two-way closed-loop linkage control step. In this step, during the steady-state period, the real-time sensing data stream of the entire field is pushed to the digital twin visualization module at a refresh rate of 30Hz for color cloud map refresh, while a refresh rate of 1Hz is used as the online calibration input for the modal parameters of the physical consistency inversion model. At the moment of touch event, the time-varying sliding of the boundary frequency of the touch transient response subunit and the estimation of transient modal parameters are triggered simultaneously, and the virtual response obtained by solving and the measured sensing data are fused and displayed in the color cloud map according to the virtual-real overlay mode selected by the visitor. Furthermore, the newly extracted transient damping ratio is used as the high-confidence initial value for the next iteration of the physical consistency inversion model, so that the entire system forms a complete two-way closed loop from physical loading to data sensing to virtual calibration, and from virtual input to model deduction to cloud map presentation in each touch interaction. Through the above steps, the visitor can simultaneously observe the real transient response of the physical model and the steady-state stress distribution of the digital twin in a single interaction, and the two are strictly consistent in key mechanical indicators such as damping attenuation characteristics and peak response amplitude. The detailed implementation method of this step is the same as that described in the bidirectional linkage control module 6 in the aforementioned system embodiment.

[0075] Experimental verification was conducted on a scaled-down model of a typical cable-stayed bridge provided in this embodiment. Comparative tests were performed on the same set of simulated vehicle traffic load conditions using the three-channel frequency domain decoupling and fusion method of this invention and the visual acceleration time-domain complementary filtering method disclosed in CN115752250A. The results show that the root mean square error of the method of the present invention in quasi-static displacement measurement is 0.018 mm, while that of the comparative method is 0.062 mm, representing an improvement of approximately 3.4 times; the relative error in dynamic displacement peak measurement is 1.7%, while that of the comparative method is 8.3%, representing an improvement of approximately 4.9 times; in the touch virtual load response consistency test, the correlation coefficient between the virtual response curve solved by the method of the present invention and the measured response curve of the physical model reaches 0.97, while the correlation coefficient of the virtual response relying solely on the static lookup table scheme is only 0.62; in the continuous display test with a cumulative 200 touches, the physical constraint threshold method of the present invention successfully identified the abnormal damping change caused by deliberately loosening a support bolt on the 87th touch, while the pure statistical control limit method did not complete the identification until the 153rd touch, with an early identification rate of approximately 43%. The above experimental data fully verify the technical effects achieved by the present invention and its significant advantages over the prior art.

[0076] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-sensor fusion image interactive perception system for bridge entity models, characterized in that, The system includes: The scaled-down bridge solid model has strain sensor arrays and acceleration sensors pre-embedded inside the key stress-bearing components, and coded targets are attached to key nodes on the outer surface. The image acquisition unit includes a camera array fixedly positioned facing the scaled-down bridge physical model, used to continuously acquire image sequences of the coded target and perform sub-pixel target tracking on the image sequences to output sub-pixel displacement signals; The multi-sensor frequency domain decoupling and fusion unit is used to determine the dynamic and static response frequency domain boundary frequency based on the geometric similarity ratio of the scaled-down bridge entity model to the prototype bridge according to the structural dynamic similarity law. The sub-pixel displacement signal is used as the quasi-static low-frequency channel, the acceleration signal output by the accelerometer is used as the dynamic high-frequency channel, and the strain signal output by the strain sensor array is used as the local stiffness constraint channel. The data is decoupled and processed independently on both sides of the boundary frequency and seamlessly spliced ​​according to the frequency band to construct a real-time sensing data stream across the entire field. The digital twin visualization unit is used to drive the display screen to present the stress distribution and deformation amplitude of each component in the form of a color cloud map by the real-time perception data stream of the whole field. The virtual load interaction unit includes a touch interface for receiving virtual loads applied by visitors at corresponding locations on the scaled-down bridge physical model. The virtual loads are either virtual concentrated loads or virtual uniformly distributed loads. The virtual loads are then solved using a physical consistency inversion model that is calibrated online by the real-time sensing data stream across the entire field to obtain a virtual response. The bidirectional linkage control unit is used to overlay the virtual response onto the color cloud map of the digital twin visualization unit in real time, and at the same time use the real-time perception data stream of the whole field as the online calibration input of the physical consistency inversion model to realize bidirectional closed-loop linkage between physical loading and virtual loading.

2. The bridge entity model multi-sensor fusion image interactive perception system according to claim 1, characterized in that, The multi-sensor frequency domain decoupling and fusion unit further includes a frequency domain boundary frequency calculation subunit and a three-channel decoupling processing subunit. The frequency domain boundary frequency calculation subunit is used to calculate the dynamic and static response frequency domain boundary frequency based on the geometric similarity ratio of the scaled-down bridge entity model relative to the prototype bridge. According to the structural dynamics similarity law, the natural frequency is proportional to the negative half power of the geometric similarity ratio. The subunit multiplies the first-order natural frequency characteristic value of the prototype bridge by the negative half power of the geometric similarity ratio and then by the decoupling margin coefficient. The decoupling margin coefficient ranges from 1.5 to 3.

0. The three-channel decoupling processing subunit is used to process the sub-pixel positions... The displacement signal is processed by a low-pass filter with the cutoff frequency as the dividing frequency and then used as a quasi-static low-frequency channel. The acceleration signal output by the accelerometer is processed by a high-pass filter with the cutoff frequency as the dividing frequency and then subjected to a second numerical integration to form a dynamic high-frequency channel. The strain signal output by the strain sensor array is converted into the local equivalent displacement according to the analytical relationship between strain, curvature and deflection and then used as a local stiffness constraint channel. The quasi-static low-frequency channel, the dynamic high-frequency channel and the local stiffness constraint channel are decoupled and processed independently on both sides of the dividing frequency and then seamlessly spliced ​​together according to frequency band to form the full-field real-time sensing data stream.

3. The bridge entity model multi-sensor fusion image interactive perception system according to claim 2, characterized in that, The multi-sensor frequency domain decoupling and fusion unit further includes a touch transient response subunit, used to monitor the touch event timestamp output by the virtual load interaction unit. Within a hold window of 30 to 80 milliseconds from the touch event timestamp, the dynamic-static response frequency domain boundary frequency is temporarily shifted upward to the upper limit of the transient frequency. The upper limit of the transient frequency is equal to the upper limit of the envelope of the first three natural frequencies of the scaled-down bridge solid model multiplied by the transient redundancy coefficient, and the value of the transient redundancy coefficient ranges from 1.2 to 2.

0. Within a cosine gradient window of 100 to 300 milliseconds after the hold window, the boundary frequency is gradually shifted back to the steady-state value by cosine interpolation. Within the hold window, the free vibration decay curve of the scaled-down bridge solid model under touch impulse load is extracted from the sub-pixel displacement residual and acceleration residual, and the actual damping ratio of the scaled-down bridge solid model under the current working condition is inverted as the online damping parameter of the physical consistency inversion model.

4. The bridge entity model multi-sensor fusion image interactive perception system according to claim 3, characterized in that, The touch transient response subunit further includes a transient modal parameter estimation module. This module is configured to extract transient modal frequencies and damping ratios from the free vibration decay curve within the holding window using a matrix bundle time-domain parameter estimation method: a first Hankel data matrix and a second Hankel data matrix with a displacement relationship are constructed from the sampled values ​​of the free vibration decay curve; singular value decomposition is performed on the first Hankel data matrix, and the first few principal singular values ​​are retained at twice the number of modes of interest to obtain a reduced-order subspace; a matrix bundle is constructed based on the first Hankel data matrix and the second Hankel data matrix within the reduced-order subspace, and its generalized eigenvalues ​​are solved; the generalized eigenvalues ​​correspond to complex poles in the discrete-time domain; and the natural frequencies and damping ratios of each order are obtained by back-calculation based on the argument and modulus of the complex poles.

5. The bridge entity model multi-sensor fusion image interactive perception system according to claim 4, characterized in that, The touch transient response subunit is further configured to construct a damping ratio time series by extracting the transient damping ratio of each touch event in chronological order, apply an exponentially weighted moving average statistical control chart to the damping ratio time series, and trigger a physical model status alarm signal when the output value of the statistical control chart deviates from the initial damping benchmark by more than a preset control limit. The alarm signal is used to indicate the cumulative micro-damage of the scaled bridge physical model caused by repeated interaction by visitors, such as loosening of adhesive joints, attenuation of bolt preload, or changes in bearing friction.

6. The bridge entity model multi-sensor fusion image interactive perception system according to claim 5, characterized in that, The touch transient response subunit also includes a physical constraint threshold calibration module, which is used to establish a reduced-order finite element model of beam elements for the scaled-down bridge solid model, and to analytically derive the sensitivity matrix of each modal damping ratio to key stiffness parameters under the Rayleigh damping assumption. The inner product of the sensitivity matrix and the allowable stiffness degradation vector is superimposed on the initial damping reference to obtain the physical constraint threshold, and the physical constraint threshold is used to replace the control limit of the exponentially weighted moving average statistical control chart.

7. The bridge entity model multi-sensor fusion image interactive perception system according to claim 1, characterized in that, The strain sensor array is symmetrically distributed with at least 8 measuring points embedded in the lower edge of the main beam mid-span, the quarter-span section, the pier root, and below the bearing pad of the scaled-down bridge solid model. The acceleration sensor is symmetrically arranged with at least 4 measuring points in pairs in the mid-span and quarter-span sections of the main beam. The camera array consists of at least 2 industrial cameras fixed on both sides of the model to ensure that the field of view covers the entire coded target.

8. The bridge entity model multi-sensor fusion image interactive perception system according to claim 1, characterized in that, The coded targets are any one of ArUco QR code targets, checkerboard targets, or concentric ring targets, and are affixed to the key stress nodes at the mid-span, quarter-span, support, and cable anchorage ends of the scaled-down bridge physical model. Each coded target has a unique numerical number for identification.

9. The bridge entity model multi-sensor fusion image interactive perception system according to claim 1, characterized in that, The digital twin visualization unit presents a stress distribution cloud map on the display screen using a color mapping from blue to cyan to green to yellow to red. The touch interface of the virtual load interaction unit supports gesture operations such as applying a virtual concentrated load by clicking with one finger and applying a virtual uniformly distributed load by dragging with two fingers. The amplitude of the virtual load is linearly mapped by the drag distance or the pressing time according to a preset ratio. The color cloud map is updated for the first time within 100 milliseconds after the touch event starts.

10. A multi-sensor fusion image interactive perception method for bridge entity models, characterized in that, The method is implemented using the bridge entity model multi-sensor fusion image interactive perception system as described in any one of claims 1 to 9, and includes the following steps: Step S1: Embed strain sensor arrays and acceleration sensors inside the key load-bearing components of the scaled-down bridge solid model, and attach coded targets to key nodes on the outer surface of the scaled-down bridge solid model. Step S2: Continuously acquire image sequences of the coded target using a fixed camera array, perform sub-pixel target tracking on the image sequences, and extract the displacement response of each node; Step S3: Based on the geometric similarity ratio of the scaled-down bridge entity model to the prototype bridge, determine the dynamic and static response frequency domain boundary frequency according to the structural dynamic similarity law. Use the sub-pixel displacement signal as the quasi-static low-frequency channel, the acceleration signal output by the accelerometer as the dynamic high-frequency channel, and the strain signal output by the strain sensor array as the local stiffness constraint channel. Decouple and process them independently on both sides of the boundary frequency and seamlessly stitch them together according to the frequency band to construct a real-time sensing data stream across the entire field. Step S4: Drive the display screen to present the stress distribution and deformation amplitude of each component of the scaled-down bridge solid model in the form of a color cloud map using the real-time perception data stream. Step S5: Receive the virtual load applied by the visitor through the touch interface. The virtual load is either a virtual concentrated load or a virtual uniformly distributed load. Based on the real-time perception data stream of the entire site, calibrate the physical consistency inversion model online and solve for the virtual response. Step S6: The virtual response is superimposed onto the color cloud map in real time, while the real-time perception data stream of the whole field is continuously used as the online calibration input of the physical consistency inversion model to realize the two-way closed-loop linkage between physical loading and virtual loading.

Images