A structural visual vibration measurement and modal recognition method and system based on improved YOLOv11 multi-target tracking

By improving the YOLOv11 multi-target tracking and ERA algorithm, and combining it with traditional modal testing, the problems of target drift and mode identification difficulties in video vibration measurement have been solved, achieving high-precision structural modal parameter identification and mode shape reconstruction, which is suitable for non-contact monitoring of large structures.

CN122760950BActive Publication Date: 2026-11-10NANJING UNIV OF AERONAUTICS & ASTRONAUTICS +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202611225584.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-08-13
Publication Date
2026-11-10
Estimated Expiration
2046-08-13

AI Technical Summary

Technical Problem

Existing video vibration measurement solutions suffer from target drift and tracking loss in complex backgrounds, making it impossible to fully identify the complete set of modal parameters and vibration modes. Furthermore, traditional contact measurement equipment is difficult to deploy and interferes with the structural dynamics.

Method used

An improved YOLOv11 multi-target tracking method was adopted, which combined Deep SORT cross-frame continuous tracking and ERA algorithm. The YOLOv11-iEMA network model was used to identify measurement points, extract pixel displacements and calculate modal parameters, and the results were verified by traditional modal experiments.

Benefits of technology

It achieves stable tracking of measurement points under complex backgrounds, outputs natural frequency, damping ratio and mode shape, with reliable accuracy, meets engineering testing specifications, and reduces hardware and maintenance costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122760950B_ABST
    Figure CN122760950B_ABST
Patent Text Reader

Abstract

This invention discloses a structural visual vibration measurement and modal recognition method and system based on improved YOLOv11 multi-target tracking. It embeds an efficient iEMA inverted attention module into the YOLOv11 feature fusion layer to enhance the detection accuracy of small measurement points in complex scenes. It combines Deep SORT to fuse motion and appearance costs to construct a matching matrix, and uses Kalman filtering to predict measurement point trajectories, achieving long-term drift-free continuous tracking of multiple measurement points. Through scale calibration, pixel displacements are converted into actual vibration quantities, and signal noise reduction preprocessing is performed. Then, the ERA algorithm is used to construct the Hankel matrix, and singular value decomposition is used to solve for the structure's natural frequencies, damping ratios, and reconstruct mode shapes. This invention constructs a complete link from video acquisition to measurement point detection, trajectory tracking, and modal recognition, eliminating the need for contact sensors and enabling simultaneous acquisition of dynamic data from multiple measurement points across the entire field. It is suitable for long-term health monitoring of large-scale projects such as cranes, bridges, and pipe corridors.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of structural dynamics testing and numerical simulation, specifically relating to a structural visual vibration measurement and modal recognition method and system based on improved YOLOv11 multi-target tracking. Background Technology

[0002] Structural vibration measurement and modal parameter identification are core technologies for structural health monitoring and safety assessment. Large engineering structures such as cranes, bridges, and pipe corridors are subjected to long-term operating loads, environmental disturbances, loose connections, and fatigue damage, which causes dynamic parameters such as natural frequencies, damping ratios, and mode shapes to change. Accurately extracting vibration responses and identifying modal parameters is crucial for structural condition assessment, fault diagnosis, and operation and maintenance management.

[0003] The current mainstream testing methods in engineering are contact measurement devices such as accelerometers, strain gauges, and force hammers. Although these technologies are mature and reliable, they have significant shortcomings in field applications on large structures: the number of measurement points is limited, making it impossible to obtain full-field vibration data; wiring and installation at high-altitude and dangerous measurement points are difficult; and the added mass of the sensors and cable constraints can interfere with the local dynamic characteristics of the structure.

[0004] With the development of computer vision and image processing technologies, video-based non-contact vibration measurement methods have gradually become an important direction in structural dynamics testing. These methods acquire image sequences of the vibration process of the structure under test using a camera, and extract the structural displacement response using algorithms such as image matching, optical flow estimation, phase analysis, digital image correlation, or target tracking. They offer advantages such as being non-contact, easy to deploy, having a large measurement range, and low cost. Compared with traditional contact sensors, video vibration measurement methods can achieve long-distance, multi-regional, or even full-field response acquisition without altering the structure's own dynamic characteristics, making them particularly suitable for dynamic monitoring of large structures and complex field environments.

[0005] However, existing video vibration measurement solutions have obvious drawbacks: traditional optical flow methods, template matching methods, and feature point tracking rely on manual point selection, which can easily lead to target drift and tracking loss in complex backgrounds, lighting fluctuations, occlusion, and small vibration scenarios; digital image correlation requires pre-spraying speckle patterns, resulting in a large workload on-site and high real-time computing overhead; most video vibration measurement algorithms only extract a single frequency and fail to fully utilize the spatial correlation information of multiple measurement points, making it impossible to completely identify the full set of modal parameters and vibration modes.

[0006] In recent years, deep learning object detection algorithms, especially the YOLO series networks, have been increasingly adopted in engineering object recognition and visual monitoring tasks due to their advantages such as fast detection speed, strong feature extraction capabilities, and good adaptability to complex scenes. Automatically identifying multiple feature points on a structural surface using deep learning models can reduce manual point selection and template construction, improving the robustness of object localization in complex backgrounds. However, relying solely on object detection networks can only obtain the target position within a video frame and cannot directly generate modal parameter identification results in the sense of structural dynamics. How to combine deep learning multi-object detection with cross-frame tracking, displacement response extraction, modal parameter identification, and traditional modal testing to form a complete method for structural visual vibration measurement and mode shape identification remains a problem that needs to be solved in current engineering visual vibration measurement technology. Summary of the Invention

[0007] The purpose of this invention is to provide a structural visual vibration measurement and modal recognition method and system based on improved YOLOv11 multi-target tracking, thereby solving the above-mentioned technical problems.

[0008] To solve the above-mentioned technical problems, the present invention adopts the following solution:

[0009] A structural visual vibration measurement and modal identification method based on improved YOLOv11 multi-target tracking includes the following steps:

[0010] Step S1: Collect vibration video of the structure under test under excitation, perform scale calibration, and establish the conversion relationship between image pixels and actual structure size.

[0011] Step S2: Build and train a YOLOv11-iEMA network model that embeds the iEMA efficient inverted attention module;

[0012] Step S3: Use the trained YOLOv11-iEMA network model to identify measurement points frame by frame, and combine Deep SORT to complete multi-measurement point cross-frame continuous tracking.

[0013] Step S4: Extract pixel displacement based on the continuous trajectory of the measurement point, and complete displacement conversion and signal preprocessing;

[0014] Step S5: Use the ERA algorithm to calculate the natural frequencies and damping ratios of the structure and reconstruct the mode shapes of the preprocessed displacement sequence;

[0015] Step S6: Compare and verify the results of visual vibration identification through traditional contact modal testing.

[0016] Further optimization, step S1, video acquisition and scale calibration, includes the following sub-steps:

[0017] Step S1.1: Position the high-speed industrial camera facing the structure to be measured, adjust the lens optical axis to be perpendicular to the plane to be measured, and adjust the supplementary light and aperture to ensure clear imaging of the surface measurement points; use hammer excitation, environmental excitation or equipment operation excitation to excite structural vibration, and the camera continuously acquires and stores the vibration image sequence at a preset high frame rate;

[0018] Step S1.2: Select two sets of calibration points on the structural surface that are unobstructed and whose actual spacing is known. The actual physical distance between the two points is measured as L. Read the still reference frame image from the acquired video, extract the pixel coordinates of the two calibration points, and calculate the pixel distance P between the two points. Calculate the pixel-physical size scaling factor S using the following formula, which will be used to convert the measured pixel displacement into the actual displacement response in the future.

[0019] S=L / P.

[0020] Further optimization involves the following steps in step S2: Building and training the YOLOv11-iEMA measurement point detection model.

[0021] Step S2.1, Dataset Creation: Extract image frames from the collected vibration video, use the DarkLabel annotation tool to semi-automatically interpolate and annotate all visual measurement points of the structure, and divide the data into training set, validation set, and test set according to a preset ratio;

[0022] Step S2.2, Improved Network Construction: Based on the original YOLOv11 network, a structured target detection model YOLOv11-iEMA is constructed, including a Backbone network, a Neck feature fusion network, and a Head network. In the Neck feature fusion network, based on the original YOLOv11 PAN-FPN upsampling and downsampling fusion structure, iEMA efficient inverted attention modules are embedded at the end of each of the three output branches to specifically enhance features at large, medium, and small scales, strengthening the features of small target points and suppressing complex background noise. The output features of the three iEMA layers are then fed into the Head network. Specifically:

[0023] 1) Top-level large-scale fusion branch: The deep features of the bottom layer of the backbone are upsampled and enlarged in size, and then concatenated and fused with the shallow features of the middle layer of the backbone through the Concat channel; the concatenated features are fed into the C3K2 module to complete nonlinear transformation, and the output is connected to the iEMA high-efficiency inverted attention module; the iEMA output is split into two paths, one path is fed to the right into Conv convolution and output to the first layer Detect head, and the other path flows down into the middle layer fusion branch to participate in the secondary concatenation.

[0024] 2) Mid-level mesoscale fusion branch: The features passed down from the top iEMA are adjusted in channel dimension by Conv and concatenated with the upsampled features from the lower layer by Concat. After being fused by C3K2, they are connected to the second group of iEMA high-efficiency inverted attention modules. One iEMA output is sent to the right into the second layer Detect head, and the other is sent down into the bottom fusion branch.

[0025] 3) Low-level small-scale fusion branch: The mid-level iEMA features are passed down and adjusted by the Conv channel. They are then concatenated with the lowest-level global features of the backbone and processed by C3K2 before being connected to the third group of iEMA high-efficiency inverted attention modules. The iEMA output is directly fed to the right into the third-level Detect small target detection head.

[0026] Neck internal module descriptions: Upsample: interpolation upsampling, achieving fusion of deep semantic features and shallow high-resolution features; Concat: channel-dimensional feature concatenation, preserving multi-scale original image information; C3K2: lightweight residual convolution block, compressing channels and improving feature expression capabilities; Conv: standard single-layer convolution, unifying feature channels and sizes; iEMA: this efficient multi-scale attention module, with three independently configured branches, performing bidirectional one-dimensional pooling feature enhancement for measurement points of different sizes, suppressing complex background noise.

[0027] Step S2.3, Training Environment Configuration and Model Training: Build a training environment based on the PyTorch framework, load YOLOv11 pre-trained weights to initialize the network, and configure training hyperparameters such as optimizer, learning rate, batch size, and number of iterations; input the measurement point dataset to carry out end-to-end training, synchronously calculate classification loss and bounding box regression loss, and use the validation set to evaluate precision, recall, and mAP metrics after each training round.

[0028] Step S2.4, Optimal Model Saving: After training, use the test set to complete the comprehensive evaluation of the model and save the optimal weight file of the mAP index as the inference model.

[0029] Further optimization is achieved by the iEMA high-efficiency inverted attention module, which executes the following steps:

[0030] Step A1: Input features output by YOLOv11 Neck Figure X Input the iEMA module and perform normalization processing first to stabilize the numerical distribution of features at different scales;

[0031] Step A2: Perform inverted residual channel expansion on the normalized features through 1×1 convolution to obtain intermediate features Xm, so that the channel expression dimension is expanded from C to Cm, Cm=C×e, where e is the channel expansion factor, and the recommended value for engineering is e=4.

[0032] Step A3: In the attention path, the intermediate feature Xm is divided into G groups according to the channel. One-dimensional global pooling is performed on each group of features along the height and width directions to obtain the horizontal and vertical context encodings. Directional attention weights are generated by 1×1 convolution mapping and Sigmoid activation, and dynamic multi-scale attention response is generated by combining 3×3 convolution branches and cross space weighting.

[0033] Step A4: In the local perception path, perform depthwise separable convolution on the intermediate feature Xm to extract spatial detail features such as measurement point edges, reflective areas and local texture changes, and enhance the effective response through channel recalibration;

[0034] Step A5: Fuse the attention path output with the local perception path output to form an enhanced feature that combines directional context, multi-scale dynamic weights, and local spatial details;

[0035] Step A6: Map the enhanced features back to the original channel dimension through 1×1 projective convolution, and establish a residual connection with the module input when the input and output dimensions are consistent, to obtain the iEMA module output feature Oi, which is used by the YOLOv11 detector head for measurement point localization.

[0036] Further optimization includes the following steps in step S3: multi-point detection and cross-frame continuous tracking:

[0037] Step S3.1, Frame-by-Frame Measurement Point Inference: Input the complete vibration video frame by frame into the trained YOLOv11-iEMA network model, and output the detection boxes, confidence scores, and center pixel coordinates of all measurement points in a single frame, forming a single-frame measurement point set D. t ;

[0038] ;

[0039] in, Indicates the first The first frame The test results at each measuring point Indicates the first The number of measurement points detected in the frame; each detection result includes the detection box position, confidence level, and center coordinates:

[0040] ;

[0041] in, Indicates the detection box. Indicates the detection confidence level. This indicates the coordinates of the center of the detection frame.

[0042] Step S3.2, Kalman trajectory prediction: For each measurement point trajectory that already exists in the previous frame, use Kalman filtering combined with the state transition matrix to predict the estimated state of the current frame. Specifically:

[0043] 1) Let the first The trajectory of the measuring point is in the first The frame's state is:

[0044] ;

[0045] in, Indicates the coordinates of the center of the detection box. Indicates the aspect ratio of the detection frame. The height of the detection box is represented by the dotted variable, which represents the first-order rate of change of the corresponding state.

[0046] 2) Based on the Kalman filter prediction model, the first... Frame prediction state:

[0047] ;

[0048] in, Represents the state transition matrix. Indicates the first The trajectory of the measuring point is in the first The predicted state of the frame. The state transition matrix F is constructed based on the discrete model of uniform motion, and an 8th order square matrix is ​​constructed with a frame interval Δt=1. The state vector includes the center coordinates of the measurement point frame, aspect ratio, frame height and corresponding motion velocity. The first 4 rows of the matrix realize the position and size by superimposing the velocity term across frames, and the last 4 rows represent that the velocity component remains unchanged under the assumption of uniform motion, so as to realize the unobserved prediction of the motion state of the measurement point trajectory in the next frame.

[0049] Step S3.3, Dual Matching Cost Calculation: Calculate the motion Mahalanobis distance between the predicted trajectory and the current frame detection, and the normalized appearance feature matching cost, respectively. Then, weightedly fuse them to obtain the comprehensive matching cost. :

[0050] ;

[0051] Among them, the moving Mahalanobis distance The calculation formula is: This is used to measure the consistency between the current detection result and the predicted trajectory in terms of motion state; appearance matching cost , and This is the normalized appearance feature vector. The smaller the value, the higher the similarity between the detected target and the historical trajectory. These are weighting coefficients used to adjust the contributions of motion information and appearance information in target association; Represents the observation matrix. Indicates the first The covariance matrix corresponding to each predicted trajectory.

[0052] Step S3.4, Hungarian Optimal Matching: Construct a cost matrix based on the comprehensive matching cost, and use the Hungarian algorithm to complete the association matching between historical trajectories and current measurement points, distinguishing between successfully matched trajectories, unmatched old trajectories, and newly detected measurement points. The cost matrix C is an M-row N-column two-dimensional matrix; row index k: corresponds to the k-th historical trajectory; column index j: corresponds to the j-th detection box in the current frame; matrix element C[k,j]=c(k,j): the comprehensive matching cost between the k-th trajectory and the j-th detection box.

[0053] Step S3.5, Trajectory Update and Creation: For successfully matched trajectories, update the Kalman state with the current detection coordinates; for unmatched new measurement points, assign a unique ID to generate a new trajectory; if an old trajectory with no match for multiple consecutive frames reaches the loss threshold, delete it.

[0054] Step S3.6, Trajectory Output: Output the complete pixel coordinate time sequence of each measuring point with the ID remaining unchanged throughout the entire process, forming a continuous motion trajectory set which is then sent into the displacement extraction process.

[0055] Further optimization includes the following steps in step S4: displacement extraction and signal preprocessing:

[0056] Step S4.1 Coordinate Timing Reading: Read the complete trajectory of each measurement point output by Deep SORT, and extract the center pixel coordinates of each measurement point in the whole frame. Select the coordinates of the stationary frame before vibration. p k (0) serves as the baseline;

[0057] Step S4.2 Pixel displacement calculation: Calculate the relative pixel displacement of the measurement point frame by frame, with the reference coordinates as the zero point. Step S4.3 Actual Displacement Conversion: Using the scaling factor S obtained in step S1, the pixel displacement is converted into the actual vibration displacement time history. ;

[0058] Step S4.4 Basic signal correction: For each displacement time history, perform mean removal to eliminate static offset and trend removal to eliminate slow camera drift components in sequence;

[0059] Step S4.5 Anomaly and Filtering: Set an amplitude threshold to identify and track abrupt anomalies and replace them using neighborhood interpolation; use low-pass filtering to remove high-frequency, minute noise from the image and detection boxes;

[0060] Step S4.6 Sequence Normalization: Normalize the uniform amplitude of the multi-point displacement after correction filtering, and output a synchronous low-noise displacement sequence to supply the ERA algorithm.

[0061] Further optimization, step S5 includes the following steps:

[0062] Step S5.1, Hankel matrix construction: Using the preprocessed multi-point displacement sequence as input, set the number of rows n and columns q, and construct the Hankel matrix by piecing together the blocks. :

[0063] ;

[0064] Among them, Y( ) represents the first Displacement response matrix of all measuring points at the sampling time.

[0065] Step S5.2, Singular Value Decomposition and Order Selection: Perform singular value decomposition on the Hankel matrix: Based on the singular value decay curve inflection point determination method, small singular values ​​corresponding to noise are eliminated, and the effective system order m is selected to obtain the truncation matrix. , and ;in, and These are the left and right singular vector matrices, respectively. It is a singular value matrix.

[0066] Step S5.3, Solving the Discrete State Matrix: Construct the time-shifted Hankel matrix H1, and calculate the state matrix of the discrete system. : .

[0067] in, For relative to The Hankel matrix is ​​constructed by shifting it by one sampling time step. , and These are the left and right singular vector matrices and singular value matrices, respectively, truncated according to the system order m.

[0068] Step S5.4, Modal parameter solution: Perform eigenvalue decomposition on the state matrix A to obtain discrete eigenvalues. Combined with sampling interval Solving for poles of a continuous system : From the pole The real and imaginary parts yield the natural frequencies of the m-th mode of the structure. Damping ratio :

[0069] ;

[0070] in, and These represent the real and imaginary parts of a complex number, respectively.

[0071] Step S5.5, Modal response extraction and mode shape normalization reconstruction: Combining the eigenvectors and output matrix, calculate the relative vibration amplitude and phase of each measuring point under each mode; arrange and normalize the modal responses of different measuring points according to their spatial positions on the structure, and the complete mode shapes of the measured structure under each mode can be reconstructed.

[0072] Further optimization includes the following steps in step S6, modal result comparison and verification:

[0073] Step S6.1, synchronous setup of contact test: Install accelerometers at the structural positions that correspond one-to-one with the visual measurement points, and connect the sensors to the dynamic signal acquisition instrument; use force hammer pulse excitation at the same position to ensure that the excitation conditions are completely consistent with the visual test.

[0074] Step S6.2, Reference Mode Acquisition: Acquire the time-domain acceleration signals of each measuring point, import them into commercial modal analysis software, and identify the reference natural frequencies, reference damping ratios, and standard mode shapes of each order;

[0075] Step S6.3, Data Alignment and Matching: Match the modal order and measurement point sorting obtained from visual recognition with the contact test reference data one by one to eliminate coordinate and order misalignment;

[0076] Step S6.4, Calculation of parameter quantitative error: Calculate the absolute error and relative error of the visual recognition result relative to the contact reference step by step, and record the error values ​​of each step;

[0077] Step S6.5, Qualitative comparison of mode shapes: Compare the reconstructed mode shapes obtained in step S5 with the reference mode shapes on the same graph, and analyze the amplitude changes and the degree of matching of vibration phases at each measuring point;

[0078] Step S6.6, Comprehensive Accuracy Judgment: The measurement accuracy of this method is comprehensively judged based on the frequency relative error threshold and the mode shape matching effect. If it exceeds the allowable error range of the engineering, the sampling frame rate, filtering parameters, and model weights are adjusted and the test is repeated.

[0079] Further optimization is needed, with the tested engineering structure being any one of the following: crane main beam, bridge beam, or underground integrated pipe gallery. The excitation method can be any one of the following: hammer pulse excitation, environmental natural excitation, or equipment operation excitation.

[0080] In step S1.1, the high-speed industrial camera captures frames at a rate of no less than 3000fps, and the lens focal length is set to 24mm when shooting the overall structural area.

[0081] A structural visual vibration measurement and modal recognition system based on improved YOLOv11 multi-target tracking, used to perform the above-mentioned method, includes: a video acquisition and scale calibration module, a YOLOv11-iEMA measurement point detection module, a Deep SORT multi-target tracking module, a displacement extraction and signal preprocessing module, an ERA modal recognition and mode shape reconstruction module, and an experimental comparison and verification module;

[0082] The video acquisition and scale calibration module is used to acquire vibration videos and calculate the pixel-to-structure size ratio factor.

[0083] The YOLOv11-iEMA measurement point detection module has a built-in iEMA high-efficiency inverted attention module for frame-by-frame identification and positioning of structural visual measurement points;

[0084] The Deep SORT multi-target tracking module is used to realize cross-frame association of measurement points and output uninterrupted continuous motion trajectory of measurement points.

[0085] The displacement extraction and signal preprocessing module is used for pixel displacement conversion and displacement signal noise reduction and correction.

[0086] The ERA mode identification and mode shape reconstruction module is used to construct the Hankel matrix, solve for the natural frequencies and damping ratios, and normalize and reconstruct the mode shapes.

[0087] The test comparison and verification module is used to import contact modal test benchmark data and complete the accuracy verification of visual vibration measurement results.

[0088] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described structural visual vibration measurement and modal recognition method based on improved YOLOv11 multi-target tracking.

[0089] Compared with the prior art, the present invention has the following beneficial effects:

[0090] 1. The solution described in this invention does not require the deployment of sensors or the application of speckle spray over a large area. It is suitable for long-term monitoring of large, high-altitude, and high-risk structures such as cranes, bridges, and pipe corridors. The hardware only requires a high-speed camera and a mid-range GPU, which significantly reduces deployment and maintenance costs.

[0091] 2. In the solution described in this invention, the three branches of YOLOv11-Neck are respectively embedded with iEMA high-efficiency inverted attention modules, which enhance the features of large, medium and small measurement points in layers. The mAP@0.5 of measurement points can reach 0.995 under complex lighting and background noise, suppressing missed detections and detection frame jitter, and reducing the extraction error of small displacements from the source.

[0092] 3. In the solution described in this invention, Deep SORT integrates the weighted matching of Marvone motion cost and appearance features, maintains a unique ID for the same measurement point across frames, solves the problems of drift, trajectory breakage, and numbering disorder of traditional visual vibration measurement points, and establishes a stable "pixel trajectory - structural physical measurement point" mapping.

[0093] 4. The solution described in this invention breaks through the limitation of traditional visual vibration measurement that can only extract a single frequency. It integrates the ERA time-domain modal identification algorithm and simultaneously outputs the natural frequency, damping ratio, and normalized mode shape. The data can directly support structural damage identification and safety status assessment.

[0094] 5. Strong engineering feasibility and reliable accuracy: The entire process is standardized and equipped with the traditional hammer impact modal test verification process. The relative error of the measured modal frequency is only 3.06% at most. The vibration mode is highly consistent with the test results of the contact sensor, which meets the accuracy requirements of the engineering testing specifications. Attached Figure Description

[0095] Figure 1 This is a flowchart illustrating the overall process of visual vibration measurement and modal recognition of the structure according to the present invention.

[0096] Figure 2 This is a schematic diagram of the test object and measuring point arrangement of the present invention;

[0097] Figure 3 This is a schematic diagram showing the target annotation of the measuring points in the structure of this invention;

[0098] Figure 4 This is a schematic diagram of the YOLOv11-iEMA target detection network structure of the present invention;

[0099] Figure 5 This is a graph showing the training results of the target detection model of this invention.

[0100] Figure 6 This is a precision-recall curve of the target detection model of this invention;

[0101] Figure 7 This is a confusion matrix diagram of the target detection model of the present invention;

[0102] Figure 8 This is a label distribution diagram of the target detection dataset of this invention;

[0103] Figure 9 This is a schematic diagram of the Deep SORT multi-target tracking process of the present invention;

[0104] Figure 10 This is a schematic diagram of the ERA modal recognition process of the present invention;

[0105] Figure 11 This is a time history diagram of the pixel coordinate displacement of multiple measuring points in this invention;

[0106] Figure 12 These are the mode shape diagrams of each order identified by the ERA algorithm in this invention;

[0107] Figure 13 This is a comparison diagram of the vibration modes of the method of the present invention and those of traditional modal testing. Detailed Implementation

[0108] To enable those skilled in the art to more clearly understand the technical solution of the present invention, the present invention will be further described in detail below with reference to embodiments and accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the scope of protection of the present invention.

[0109] Example 1:

[0110] like Figure 1 As shown, the structural visual vibration measurement and modal recognition method based on improved YOLOv11 multi-target tracking includes the following steps:

[0111] Step S1: Vibration video acquisition and image scale calibration, specifically:

[0112] Step S1.1: Prepare a gantry crane model as the structure under test. In this embodiment, the width of the structure under test is approximately 3m. Nine sets of speckle patterns are created on the surface of the main beam, and black and white concentric reflective markers are arranged as visual tracking targets. The physical size of the speckle pattern is approximately 4mm, and the diameter of the white inner circle of the reflective marker is 10mm, while the diameter of the black outer circle is 20mm. An external pulse excitation is applied using a modal force hammer to trigger structural vibration, such as... Figure 2 As shown,

[0113] Step S1.2: Use a monocular high-speed camera to take a frontal picture of the structure under test and collect the vibration video of the structure under test under the excitation of the hammer. In this embodiment, the camera is a G2110M_pro high-speed camera, the acquisition resolution is set to 5120×384, the acquisition frame rate is set to 3000fps, the lens is a 24-70mm zoom lens, and the focal length is set to 24mm when shooting the entire area.

[0114] Step S1.3: Before acquisition, adjust the camera position and field of view so that the camera is directly in front of the structure under test, the lens optical axis is basically perpendicular to the beam area to be measured, and the main beam under test and each reflective mark point are within the image field of view; adjust the aperture and supplementary light source so that the target of the marked point is clearly imaged and the brightness meets the requirements for subsequent recognition.

[0115] Step S1.4: Scale calibration is performed using the scaling factor method. In this embodiment, two calibration points with an actual distance of 2484 mm on the measured structure are selected, and their pixel distance in the image is calculated to be 3580.999 px. The scale scaling factor is thus obtained. 0 is used to convert subsequent pixel displacements into actual displacement responses.

[0116] Step S2: Extract continuous image frames from the vibration video, and use Dark Label annotation software to perform semi-automatic interpolation annotation on the nine four-quadrant black and white marker points on the surface of the tested structure. The annotation results are as follows: Figure 3 As shown, a structural measurement point target detection dataset is formed. The dataset label distribution is as follows. Figure 8 As shown, Figure 8 This provides a statistical analysis of the sample size of the measurement point dataset and a distribution map showing the spatial and dimensional distribution of the measurement points. Figure 8 The middle (a) bar chart shows the number of sample instances for the nine categories of test points (box1 to box9). The number of samples for each category is basically equal, and the total number of samples is sufficient. The dataset has a balanced distribution of samples across categories, and there will be no model training bias caused by category imbalance. Figure 8 The scatter plot in (b) shows the distribution of the center coordinates of all measurement point bounding boxes in the image coordinate system. The measurement point coordinates are evenly distributed, which can cover the feature learning of different areas of the image. Figure 8 The scatter plot in (c) shows the distribution of the width and height of the measurement point labeling box. The dimensions of the reflective measurement points in this experiment are concentrated in a narrow range, and the measurement point scale is uniform.

[0117] In this embodiment, the target detection model is randomly divided into a training set, a validation set, and a test set according to an 8:1:1 random stratified sampling ratio for training and performance evaluation.

[0118] Step S3: Construction and training of the improved YOLOv11-iEMA detection network, specifically including:

[0119] Step 3.1: Network Structure Setup: such as Figure 4 As shown, YOLOv11-iEMA consists of a three-layer architecture: a backbone network layer, a neck feature fusion network layer with embedded iEMA modules, and a head network layer for multi-scale detection heads. The backbone uses Conv, C3K2, SPPF, and C2PSA stacking to extract multi-scale shallow features. The neck layer upsamples and embeds iEMA's efficient inverted attention module at the feature splicing node to enhance the features of small, medium, and large measurement points in multiple dimensions. The three-layer detection heads synchronously output the detection boxes, confidence scores, and center pixel coordinates of measurement points of different sizes.

[0120] Step S3.2: Internal Implementation Process of iEMA's High-Efficiency Inverted Attention Module:

[0121] (1) Input the features after YOLOv11 Neck multi-scale fusion Figure X Normalize it, and complete the inverted residual channel expansion through 1×1 convolution to obtain the intermediate feature Xm;

[0122] (2) The attention path groups Xm and performs one-dimensional global pooling in the height and width directions to generate directional context encoding. The directional weights are obtained by 1×1 convolution and Sigmoid, and then combined with 3×3 convolution features and cross space weighting to generate dynamic multi-scale attention features.

[0123] (3) The local path performs a depth convolution on Xm to capture the edge of the measurement point, local texture and subtle spatial changes caused by small displacements. If necessary, the background interference is suppressed by channel recalibration. Then the attention path and local path outputs are added and fused together, and the original channel dimension is restored by Dropout and 1×1 projection convolution. The output is then connected with the input features by residual connection, and the output enhanced features are sent to the subsequent Detect head.

[0124] Step S3.3 Model Training: The network is initialized by loading general pre-trained weights using transfer learning, and hyperparameters such as optimizer, fixed learning rate, batch size, and number of iterations are set for iterative training. An end-to-end optimization method based on transfer learning is adopted, using the pre-trained weights as the initial parameters of the YOLOv11 network, and training is performed on the structural vibration measurement point image dataset. The hardware and software environment used is shown in Table 1, and the specific training parameter settings are shown in Table 2.

[0125] Table 1 Experimental Environment Configuration

[0126]

[0127] Table 2 Model Training Parameters

[0128]

[0129] Training process loss, accuracy metrics such as Figure 5 As shown, Figure 5 The curves show the changes in the loss function and accuracy index with training epochs during the training of the YOLOv11-iEMA measurement point detection model; among them, Figure 5 The curves show the changes in loss function and accuracy index with training epochs during the training of the YOLOv11-iEMA measurement point detection model. Figure 5 In the middle (a), (b), and (c), the curves for training set bounding box loss, classification loss, and distribution focus loss are respectively. Figure 5 In the middle, (d), (e), and (f) are the validation set bounding box loss, classification loss, and distribution focus loss curves, respectively; Figure 5 In the figure, (g), (h), (i), and (j) are the curves showing the changes in the evaluation metrics precision, recall, mAP@0.5, and mAP@0.5:0.95, respectively.

[0130] In the initial stage of model training, the box loss, cls loss, and dfl loss values ​​of both the training and validation sets dropped rapidly and precipitously. As the number of iterations increased, the training loss gradually and steadily converged to a lower range. The validation set loss, with slight oscillations, continued to decline overall without a significant rebound, proving that the model did not overfit. Simultaneously, the model precision and recall rapidly approached 1 and remained stable, with mAP@0.5 eventually stabilizing at a high level of 0.995, and mAP50-95 continuously increasing and then stabilizing. These trends indicate that the improved network with the embedded iEMA efficient inverted attention module has fast convergence speed, excellent fitting effect, and superior detection and localization performance for measurement points. It can stably output high-precision measurement point coordinate data, meeting the detection accuracy requirements for structural micro-vibration measurements.

[0131] The PR curve of the trained model is as follows Figure 6 As shown, Figure 6 The figure shows the precision-recall (PR) curves of the YOLOv11-iEMA measurement point detection model of this invention. The horizontal axis represents recall, which characterizes the completeness of successful detection of each reflective measurement point of the tested structure. The vertical axis represents precision, which characterizes the accuracy of the model's detection box corresponding to the real measurement point. The area enclosed by the PR curve of a single class is the AP value of the corresponding measurement point class, and the average AP of all classes is the mAP@0.5 index. In this experiment, the nine measurement points deployed on the structure were sequentially labeled as nine detection categories, box1 to box9. As can be seen from the figure, the AP values ​​corresponding to the nine measurement point categories are all 0.995, and the overall mAP@0.5 of all categories also reaches 0.995. The PR curves of each category are closely distributed along the upper boundary of the coordinate system, and both precision and recall are close to the value of 1. The test results confirm that the improved network with embedded iEMA high-efficiency inverted attention module can effectively distinguish between small measurement points and complex background information, with extremely low probability of missed or false detection of measurement points and strong stability of measurement point center coordinate output, providing high-precision raw positioning data support for subsequent multi-target trajectory tracking and vibration displacement data calculation.

[0132] Figure 7This is a confusion matrix diagram of the YOLOv11-iEMA detection model of the present invention for the nine groups of measurement point categories (boxes 1-9) and the background region. The horizontal axis of the matrix represents the true label of the sample, and the vertical axis represents the model's predicted label. The darker the color of the block, the higher the matching probability of the corresponding category. As shown in the figure, the predicted values ​​and true values ​​of all nine measurement points correspond one-to-one along the main diagonal, and the matching value of each diagonal square is 1.00. There are no misclassifications between different measurement point categories, and the background region is not misidentified by the model as any measurement point target. This result intuitively confirms that the improved detection model can accurately distinguish the nine measurement point targets from the experimental background, without problems such as measurement point misdetection, category confusion, or background misjudgment. The measurement point classification and recognition accuracy reaches 100%, and it can stably and accurately output the independent detection information corresponding to each measurement point, providing a reliable classification and recognition foundation for subsequent multi-measurement point independent tracking and accurate vibration displacement calculation.

[0133] Step S3.4 Model Inference: After training converges, load weights to detect vibration video frame by frame, and output the detection box position, confidence score, and center coordinates (x, y) of 9 measurement points in each frame.

[0134] Step S3.5: After training, the YOLOv11-iEMA target detection network is used for frame-by-frame detection of vibration video, and the detection box, class confidence and center coordinates of each target point are output.

[0135] Step S4: Introduce the Deep SORT target tracking method to perform cross-frame correlation and trajectory preservation on the detection results output by YOLOv11-iEMA, obtaining the continuous motion trajectory of each target at each marker point; the Deep SORT multi-target tracking process is as follows: Figure 9 As shown.

[0136] Step 5: Extract the center coordinates of the detection box and calculate the pixel displacement in the Y direction based on the initial frame. Following the displacement extraction module operation, obtain the displacement time history of multiple measurement points. The results are as follows: Figure 11 As shown, the Y-axis pixel displacement time history curves of nine measuring points are presented. The horizontal axis represents the relative time after hammer excitation, and the vertical axis represents the pixel offset of the measuring point. The red vertical line in the figure corresponds to the starting time of the hammer excitation. As can be seen from each sub-figure, under the action of pulse excitation, all measuring points synchronously generate regular decaying vibration waveforms. The vibration amplitudes of measuring points 4, 5, and 6 in the middle area of ​​the main beam are significantly higher than those of measuring points 1 and 9 at both ends, which is consistent with the actual vibration distribution law of a simply supported main beam after being subjected to hammer excitation. The overall oscillation pattern of the vibration waveform is continuous and complete, without trajectory breaks or data jump distortion. The vibration amplitude shows a natural decay trend over time. This curve intuitively proves that the detection and tracking link of this invention can output continuous and reliable multi-measuring point synchronous pixel vibration data. After scale calibration and signal preprocessing, it can be converted into the actual physical displacement of the structure, providing high-quality time-domain response raw data for subsequent ERA modal parameter identification.

[0137] Step 6: Input the displacement response sequences from multiple measurement points into the ERA algorithm, construct the Hankel matrix, and perform modal parameter identification; the ERA modal identification process is as follows: Figure 10 As shown, in this embodiment, the frequencies of the first three modes of the tested structure were 28.42Hz, 87.33Hz and 165.95Hz, respectively. The frequency comparison results are shown in Table 3.

[0138] Table 3 Comparison of Crane Modal Test and Video Vibration Measurement Natural Frequency Identification Results

[0139]

[0140] Step 7: Reconstruct the corresponding mode shapes based on the modal responses of each measurement point identified by the ERA algorithm. The mode shape identification results are as follows: Figure 12 As shown in the figure, the horizontal axis represents measuring points 1 to 9 along the main beam, and the vertical axis represents the normalized mode amplitude corresponding to each measuring point. The three curves correspond to the first natural frequency of 28.42Hz, the second natural frequency of 87.33Hz, and the third natural frequency of 165.95Hz, respectively. The first mode exhibits a single-peak symmetrical bending deformation, with the amplitude reaching its peak at measuring point 5 at the mid-span of the main beam. The second mode exhibits double-bending node deformation characteristics, with two regions of opposite amplitude extremes. The third mode has three deformation peaks, consistent with the theoretical bending mode variation law of a simply supported beam with uniform cross-section. The mode amplitude fluctuates smoothly with the position of the measuring points, without distortion or abrupt changes, verifying the accuracy of the modal mode reconstruction based on visual displacement data in this invention. It can completely restore the actual vibration deformation morphology of the structure at different orders, providing intuitive data support for structural dynamics analysis and structural damage identification.

[0141] The modal recognition results obtained by the method of this invention are compared and verified with the results of traditional modal testing. The comparison results are as follows: Figure 13 As shown. Figure 13 The image shows a comparison between the identification results of this invention and the first four modal shapes of traditional contact hammer impact modal tests. The left side of the image shows the normalized mode shape curves and corresponding identification frequencies calculated by this method, while the right side shows a magnified view of the measured mode shapes from the sensor and the standard test frequencies. The comparison reveals that, for the first to fourth modes, the node positions, amplitude distribution, and overall bending deformation trends of the reconstructed mode shapes from this invention highly match the measured mode shapes. Regarding frequency values, the maximum relative error for identifying the fourth natural frequency is only 3.06%, and the frequency deviations are within the allowable error range for engineering testing. This comparison fully verifies the measurement accuracy and engineering reliability of the entire visual vibration measurement system of this invention in modal frequency solving and mode shape reconstruction. It achieves the detection level of traditional contact modal tests using a non-contact visual method, meeting the requirements for modal testing and health status assessment of large-scale engineering structures.

[0142] Example 2:

[0143] A structural visual vibration measurement and modal recognition system based on improved YOLOv11 multi-target tracking, used to perform the above-mentioned method, includes: a video acquisition and scale calibration module, a YOLOv11-iEMA measurement point detection module, a Deep SORT multi-target tracking module, a displacement extraction and signal preprocessing module, an ERA modal recognition and mode shape reconstruction module, and an experimental comparison and verification module;

[0144] The video acquisition and scale calibration module is used to acquire vibration videos and calculate the pixel-to-structure size ratio factor.

[0145] The YOLOv11-iEMA measurement point detection module has a built-in iEMA high-efficiency inverted attention module for frame-by-frame identification and positioning of structural visual measurement points;

[0146] The Deep SORT multi-target tracking module is used to realize cross-frame association of measurement points and output uninterrupted continuous motion trajectory of measurement points.

[0147] The displacement extraction and signal preprocessing module is used for pixel displacement conversion and displacement signal noise reduction and correction.

[0148] The ERA mode identification and mode shape reconstruction module is used to construct the Hankel matrix, solve for the natural frequencies and damping ratios, and normalize and reconstruct the mode shapes.

[0149] The test comparison and verification module is used to import contact modal test benchmark data and complete the accuracy verification of visual vibration measurement results.

[0150] Example 3:

[0151] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described structural visual vibration measurement and modal recognition method based on improved YOLOv11 multi-target tracking.

[0152] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make several improvements and adjustments without departing from the principle of the present invention, and these improvements and adjustments should also be considered within the scope of protection of the present invention.

Claims

1. A structural visual vibration measurement and modal recognition method based on improved YOLOv11 multi-target tracking, characterized in that, Includes the following steps: Step S1: Collect vibration video of the structure under test under excitation, perform scale calibration, and establish the conversion relationship between image pixels and actual structure size; Step S2: Build and train a YOLOv11-iEMA network model that embeds the iEMA efficient inverted attention module; Building and training the YOLOv11-iEMA measurement point detection model includes the following steps: Step S2.1, Dataset Creation: Extract image frames from the collected vibration video, use the Dark Label annotation tool to semi-automatically interpolate and annotate all visual measurement points of the structure, and divide the data into training set, validation set, and test set according to a preset ratio; Step S2.2, Improved Network Construction: Based on the original YOLOv11 network, a structured target detection model YOLOv11-iEMA is constructed, including a Backbone network, a Neck feature fusion network, and a Head network. In the Neck feature fusion network, based on the original YOLOv11 PAN-FPN upsampling and downsampling fusion structure, iEMA efficient inverted attention modules are embedded at the end of each of the three output branches to specifically enhance features at large, medium, and small scales, strengthening the features of small target points and suppressing complex background noise. The output features of the three iEMA layers are then fed into the Head network. Step S2.3, Training environment configuration and model training: Build a training environment based on the PyTorch framework, load YOLOv11 pre-trained weights to initialize the network, and configure the optimizer, learning rate, batch size, number of iterations, and training hyperparameters. End-to-end training is carried out using the input measurement point dataset, and classification loss and bounding box regression loss are calculated synchronously. After each round of training, the precision, recall and mAP metrics are evaluated using the validation set. Step S2.4, Optimal Model Saving: After training, use the test set to complete the comprehensive evaluation of the model and save the optimal weight file of the mAP index as the inference model; Step S3: Use the trained YOLOv11-iEMA network model to identify measurement points frame by frame, and combine Deep SORT to complete multi-measurement point cross-frame continuous tracking. Step S4: Extract pixel displacement based on the continuous trajectory of the measurement point, and complete displacement conversion and signal preprocessing; Step S5: Use the ERA algorithm to calculate the natural frequencies and damping ratios of the structure and reconstruct the mode shapes of the preprocessed displacement sequence; Step S6: Compare and verify the results of visual vibration identification through traditional contact modal testing.

2. The structural visual vibration measurement and modal recognition method based on improved YOLOv11 multi-target tracking according to claim 1, characterized in that, Step S1, video acquisition and scale calibration, includes the following steps: Step S1.1: Position a high-speed industrial camera facing the structure under test, and use hammer excitation, environmental excitation or equipment operation excitation to excite structural vibration. The camera continuously acquires and stores vibration image sequences at a preset high frame rate. Step S1.2: Select two sets of calibration points on the structural surface that are unobstructed and whose actual spacing is known. The actual physical distance between the two points is measured as L. Read the still reference frame image from the acquired video, extract the pixel coordinates of the two calibration points, and calculate the pixel distance P between the two points. Calculate the pixel-physical size scaling factor S using the following formula, which will be used to convert the measured pixel displacement into the actual displacement response in the future. S=L / P.

3. The structural visual vibration measurement and modal recognition method based on improved YOLOv11 multi-target tracking according to claim 2, characterized in that, The iEMA high-efficiency inverted attention module constructs a dual-path collaborative enhancement structure based on inverted residual channel adjustment, deep convolutional local perception, and pluggable EMA attention branches. Its execution steps are as follows: Step A1: Input the input feature map X output by YOLOv11 Neck into the iEMA module and perform normalization processing to stabilize the numerical distribution of features at different scales. Step A2: Perform inverted residual channel expansion on the normalized features through 1×1 convolution to obtain intermediate features Xm, so that the channel expression dimension is expanded from C to Cm, Cm=C×e, where e is the channel expansion factor; Step A3: In the attention path, the intermediate feature Xm is divided into G groups according to the channel. One-dimensional global pooling is performed on each group of features along the height and width directions to obtain the horizontal and vertical context encodings. Directional attention weights are generated by 1×1 convolution mapping and Sigmoid activation, and dynamic multi-scale attention responses are generated by combining 3×3 convolution branches and cross-space weighting. Step A4: In the local perception path, perform depthwise separable convolution on the intermediate feature Xm to extract spatial detail features such as measurement point edges, reflective areas and local texture changes, and enhance the effective response through channel recalibration; Step A5: Fuse the attention path output with the local perception path output to form an enhanced feature that combines directional context, multi-scale dynamic weights, and local spatial details; Step A6: Map the enhanced features back to the original channel dimension through 1×1 projective convolution, and establish a residual connection with the module input when the input and output dimensions are consistent, to obtain the iEMA module output feature Oi, which is used by the YOLOv11 detector head for measurement point localization.

4. The structural visual vibration measurement and modal recognition method based on improved YOLOv11 multi-target tracking according to claim 3, characterized in that, Step S3, multi-point detection and cross-frame continuous tracking, includes the following steps: Step S3.1, Frame-by-Frame Measurement Point Inference: Input the complete vibration video frame by frame into the trained YOLOv11-iEMA network model, and output the detection boxes, confidence scores, and center pixel coordinates of all measurement points in a single frame, forming a single-frame measurement point set D. t ; ; in, Indicates the first The first frame Detection results at each measuring point Indicates the first The number of measurement points detected in the frame; each detection result includes the detection box position, confidence score, and center coordinates: ; in, Indicates the detection box. Indicates the detection confidence level. Indicates the coordinates of the center of the detection frame; Step S3.2, Kalman trajectory prediction: For each measurement point trajectory that already exists in the previous frame, use Kalman filtering combined with the state transition matrix to predict the estimated state of the current frame. Specifically: 1) Let the first The trajectory of the measuring point is in the first The frame's state is: ; in, Indicates the coordinates of the center of the detection box. Indicates the aspect ratio of the detection frame. The height of the detection box is represented by the dotted variable, which represents the first-order rate of change of the corresponding state. 2) Based on the Kalman filter prediction model, the first... Frame prediction state: ; in, Represents the state transition matrix. Indicates the first The trajectory of the measuring point is in the first The predicted state of the frame; Step S3.3, Dual Matching Cost Calculation: Calculate the motion Mahalanobis distance between the predicted trajectory and the current frame detection, and the normalized appearance feature matching cost, respectively. Then, weightedly fuse them to obtain the comprehensive matching cost. : ; Among them, the moving Mahalanobis distance The calculation formula is: This is used to measure the consistency between the current detection result and the predicted trajectory in terms of motion state; appearance matching cost , and This is the normalized appearance feature vector. The smaller the value, the higher the similarity between the detected target and the historical trajectory. , which are weighting coefficients used to adjust the contributions of motion information and appearance information in target association; Represents the observation matrix. Indicates the first The covariance matrix corresponding to each predicted trajectory; Step S3.4, Hungarian optimal matching: Construct a cost matrix based on the comprehensive matching cost; Use the Hungarian algorithm to complete the association matching between historical trajectories and current measuring points, distinguishing between successfully matched trajectories, unmatched old trajectories, and newly detected measuring points; Step S3.5, Trajectory Update and Creation: For successfully matched trajectories, update the Kalman state with the current detection coordinates; for unmatched new measurement points, assign a unique ID to generate a new trajectory; if an old trajectory with no match for multiple consecutive frames reaches the loss threshold, delete it. Step S3.6, Trajectory Output: Output the complete pixel coordinate time sequence of each measuring point with the ID remaining unchanged throughout the entire process, forming a continuous motion trajectory set which is then sent into the displacement extraction process.

5. The structural visual vibration measurement and modal recognition method based on improved YOLOv11 multi-target tracking according to claim 4, characterized in that, Step S4, displacement extraction and signal preprocessing, includes the following steps: Step S4.1 Coordinate Timing Reading: Read the complete trajectory of each measurement point output by Deep SORT, and extract the center pixel coordinates of each measurement point in the whole frame. Select the coordinates of the stationary frame before vibration. p k (0) serves as the baseline; Step S4.2 Pixel displacement calculation: Calculate the relative pixel displacement of the measurement point frame by frame, with the reference coordinates as the zero point. ; Step S4.3 Actual Displacement Conversion: Using the scaling factor S obtained in step S1, the pixel displacement is converted into the actual vibration displacement time history. ; Step S4.4 Basic signal correction: For each displacement time history, perform mean removal to eliminate static offset and trend removal to eliminate slow camera drift components in sequence; Step S4.5 Anomaly and Filtering: Set an amplitude threshold to identify and track abrupt anomalies and replace them using neighborhood interpolation; use low-pass filtering to remove high-frequency, minute noise from the image and detection boxes; Step S4.6 Sequence Normalization: Normalize the uniform amplitude of the multi-point displacement after correction filtering, and output a synchronous low-noise displacement sequence to supply the ERA algorithm.

6. The structural visual vibration measurement and modal recognition method based on improved YOLOv11 multi-target tracking according to claim 5, characterized in that, Step S5 includes the following steps: Step S5.1, Hankel matrix construction: Using the preprocessed multi-point displacement sequence as input, set the number of rows n and columns q, and construct the Hankel matrix by piecing together the blocks. : ; Among them, Y( ) represents the first Displacement response matrix of all measuring points at the sampling time; Step S5.2, Singular Value Decomposition and Order Selection: Perform singular value decomposition on the Hankel matrix: Based on the singular value decay curve, small singular values ​​corresponding to noise are removed, and the effective system order m is selected to obtain the truncation matrix. , and ;in, and These are the left and right singular vector matrices, respectively. It is a singular value matrix; Step S5.3, Solving the Discrete State Matrix: Construct the time-shifted Hankel matrix H1 and calculate the state matrix of the discrete system. : ; in, For relative to The Hankel matrix is ​​constructed by shifting it by one sampling time step. , and These are the left and right singular vector matrices and singular value matrices truncated according to the system order m, respectively; Step S5.4, Modal parameter solution: Perform eigenvalue decomposition on the state matrix A to obtain discrete eigenvalues. Combined with sampling interval Solving for poles of a continuous system : From the pole The real and imaginary parts yield the natural frequencies of the m-th mode of the structure. Damping ratio : ; in, and These represent the real and imaginary parts of a complex number, respectively. Step S5.5, Modal response extraction and mode shape normalization reconstruction: Combining the eigenvectors and output matrix, calculate the relative vibration amplitude and phase of each measuring point under each mode; arrange and normalize the modal responses of different measuring points according to their spatial positions on the structure, and the complete mode shapes of the measured structure under each mode can be reconstructed.

7. The structural visual vibration measurement and modal recognition method based on improved YOLOv11 multi-target tracking according to claim 6, characterized in that, The modal result comparison and verification step S6 includes the following steps: Step S6.1, synchronous setup of contact test: Install accelerometers at the structural positions that correspond one-to-one with the visual measurement points, and connect the sensors to the dynamic signal acquisition instrument; use force hammer pulse excitation at the same position to ensure that the excitation conditions are completely consistent with the visual test. Step S6.2, Reference Mode Acquisition: Acquire the time-domain acceleration signals of each measuring point, import them into commercial modal analysis software, and identify the reference natural frequencies, reference damping ratios, and standard mode shapes of each order; Step S6.3, Data Alignment and Matching: Match the modal order and measurement point sorting obtained from visual recognition with the contact test reference data one by one to eliminate coordinate and order misalignment; Step S6.4, Calculation of parameter quantitative error: Calculate the absolute error and relative error of the visual recognition result relative to the contact reference step by step, and record the error values ​​of each step; Step S6.5, Qualitative comparison of mode shapes: Compare the reconstructed mode shapes obtained in step S5 with the standard mode shapes on the same graph, and analyze the amplitude changes and the degree of matching of vibration phase at each measuring point; Step S6.6, Comprehensive Accuracy Judgment: The measurement accuracy of this method is comprehensively judged based on the frequency relative error threshold and the mode shape matching effect. If it exceeds the allowable error range of the engineering, the sampling frame rate, filtering parameters, and model weights are adjusted and the test is repeated.

8. A structural visual vibration measurement and modal recognition system based on improved YOLOv11 multi-target tracking, characterized in that, The method for performing any one of claims 1 to 7 includes: a video acquisition and scale calibration module, a YOLOv11-iEMA measurement point detection module, a Deep SORT multi-target tracking module, a displacement extraction and signal preprocessing module, an ERA modal recognition and mode shape reconstruction module, and an experimental comparison and verification module. The video acquisition and scale calibration module is used to acquire vibration videos and calculate the pixel-to-structure size ratio factor. The YOLOv11-iEMA measurement point detection module has a built-in iEMA high-efficiency inverted attention module for frame-by-frame identification and positioning of structural visual measurement points; The Deep SORT multi-target tracking module is used to realize cross-frame association of measurement points and output uninterrupted continuous motion trajectory of measurement points. The displacement extraction and signal preprocessing module is used for pixel displacement conversion and displacement signal noise reduction and correction. The ERA mode identification and mode shape reconstruction module is used to construct the Hankel matrix, solve for the natural frequencies and damping ratios, and normalize and reconstruct the mode shapes. The test comparison and verification module is used to import contact modal test benchmark data and complete the accuracy verification of visual vibration measurement results.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the structural visual vibration measurement and modal recognition method based on improved YOLOv11 multi-target tracking as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Steel surface defect detection method based on EMSD-YOLO algorithm

    CN120510121A

  • Crop pest detection method and system based on YOLO11 lightweight improvement

    CN122176517A