A cable vibration frequency identification method based on visual collaborative segmentation and computer storage medium

Abstract

A cable vibration frequency identification method based on visual cooperative segmentation, comprising the following steps: S1, key frame extraction is performed on a cable vibration video to obtain a key frame image; S2, the key frame image is input into a target detection model to perform coarse-grained target detection and obtain target detection information; S3, the target detection information is input into an image segmentation model to perform mask segmentation on the key frame image, obtain a mask image, and obtain a mask region; S4, a corner point detection algorithm is used in the mask region to extract feature points; S5, the corner points are tracked between frames based on an optical flow method to obtain a displacement sequence of the corner points; and S6, frequency domain analysis is performed on the displacement sequence to extract a vibration frequency. The application also relates to a computer storage medium storing a program for executing the above method. The application does not need a physical target, does not need manual ROI initialization, and can realize multi-target cable vibration frequency in a complex environment, and belongs to the field of structural health monitoring.

Description

Technical Field

[0001] This invention relates to the field of structural health monitoring, specifically to a method for identifying cable vibration frequencies based on visual collaborative segmentation and a computer storage medium. Background Technology

[0002] As a critical load-bearing component of bridge structures, the health of bridge cables is crucial to the overall safety of the bridge. The vibration characteristics of cables, especially their natural frequencies and vibration modes, are important indicators for assessing their health. The vibration frequency of a cable reflects changes in its axial force, and this force, as a key parameter for determining the structural stress state, directly relates to the mechanical performance and service life of the bridge structure. Therefore, accurately monitoring the vibration characteristics of cables is of great significance for stability assessment, early damage identification, and overall health management during bridge operation.

[0003] Traditional cable vibration detection technology mainly relies on contact sensor systems, including accelerometers, strain gauges, electromagnetic exciters, and data acquisition systems. While these methods offer high measurement accuracy and mature theoretical support, they have several limitations in practical engineering applications. Firstly, installation costs are high. Contact sensors must be directly installed on the cable itself or its nearby supporting structure, which is particularly difficult in high-altitude, long-span, or multi-cable structures, often requiring scaffolding, lifting platforms, and other auxiliary equipment, leading to a significant increase in manpower and material resources. Secondly, maintenance is cumbersome. Long-term service sensors face performance degradation and increased failure rates, requiring regular calibration, replacement, and troubleshooting. Furthermore, maintenance work at height or in enclosed structures presents significant operational risks and safety hazards. Thirdly, environmental adaptability is poor. In high-humidity, highly corrosive, high-temperature, or heavy-load environments, the measurement accuracy and stability of sensors are easily affected. In addition, contact methods typically only deploy a limited number of measuring points, making it difficult to achieve comprehensive monitoring of the entire bridge deck or multiple cables, resulting in limited information coverage.

[0004] To overcome the aforementioned shortcomings, non-contact monitoring technologies based on computer vision have gradually emerged in the field of structural health monitoring in recent years. These methods utilize remote camera equipment to acquire images or videos of the structure, and combine image processing and motion analysis algorithms to extract motion information of the structure or components, thereby enabling the analysis of vibration frequency, dynamic displacement, and other state parameters.

[0005] Although existing visual monitoring technologies have made significant progress in identifying vibrations in bridge cables, some problems and technical bottlenecks still exist in practical engineering applications, mainly in the following aspects:

[0006] 1. Physical target-based methods rely on manual deployment, resulting in insufficient flexibility. These methods require manual installation of targets or markers on the monitored object. For high-altitude, high-cable-density, or inaccessible bridge structures, the installation process is difficult, carries high safety risks, and large-scale deployment is costly. The long-term stability of the targets is also difficult to guarantee, as they are susceptible to damage or detachment due to wind, rain, and dirt, leading to monitoring interruptions. Furthermore, the presence of the targets may interfere with the structure's dynamic response or aesthetic integrity, making them unsuitable for long-term online monitoring scenarios.

[0007] 2. Label-free methods based on manually defined Regions of Interest (ROIs) lack automation and stability. While these methods eliminate dependence on physical targets to some extent, their analysis process still relies on manually defining the Region of Interest (ROI) where the cable is located. This operation not only affects efficiency but may also introduce region bias due to operator subjectivity. More importantly, under conditions of low texture, complex backgrounds, or varying lighting, corner matching within the ROI is prone to failure, causing tracking interruptions and frequency extraction errors. Furthermore, if the manually selected ROI contains non-rigid backgrounds or interfering regions, it will also affect the accuracy of motion estimation, leading to misjudgment or drift of vibration frequencies.

[0008] 3. Deep learning-based automated methods suffer from generalization and engineering stability issues. While emerging deep learning methods have demonstrated superior performance in automatic detection and target tracking, current methods largely rely on task-specific datasets for supervised training, limiting model generalization capabilities. In practical bridge applications, complex background interference, varying target scales, and limited image resolution can easily cause model performance fluctuations. For example, detection accuracy decreases when cable textures are weak, contrast is low, or occlusion occurs; tracking stability is also affected under long-distance shooting or drastic lighting changes. Furthermore, existing deep learning models have high computational overhead and demand high performance from on-site computing platforms. A mature and deployable engineering toolchain has not yet been developed, limiting their widespread adoption in bridge engineering.

[0009] 4. Challenges remain in multi-target simultaneous identification and long-term continuous monitoring. Bridge engineering often involves scenarios where multiple cables vibrate simultaneously, but existing methods are mostly designed for single-target analysis and lack the ability to simultaneously detect, segment, and track multiple targets, especially in areas where cables intersect, overlap, or are densely distributed, making target boundaries difficult to separate. Furthermore, in long-term continuous monitoring scenarios, models are susceptible to cumulative environmental changes such as lighting, camera shake, and occlusion, lacking adaptive mechanisms for correction or relocation, thus affecting the integrity and consistency of monitoring data.

[0010] In summary, while existing visual monitoring methods each have their own characteristics, they generally face problems such as difficult deployment, reliance on manual labor, high environmental sensitivity, insufficient automation, and limited multi-target processing capabilities. These problems are key obstacles restricting the widespread application of visual technology in cable vibration monitoring, necessitating the development of a comprehensive identification method that combines automation, robustness, and engineering adaptability to meet the long-term stable operation requirements of practical bridge structural health monitoring systems. Summary of the Invention

[0011] To address the technical problems existing in the prior art, the purpose of this invention is to provide a method and computer storage medium for identifying cable vibration frequencies based on visual collaborative segmentation, which can achieve automatic identification of multi-target cable vibration frequencies in complex environments without the need for physical targets or manual ROI initialization.

[0012] To achieve the above objectives, the present invention adopts the following technical solution: a method for identifying cable vibration frequency based on visual collaborative segmentation, comprising the following steps,

[0013] S1, extract keyframes from the cable vibration video to obtain keyframe images;

[0014] S2, input the keyframe image into the target detection model to perform coarse-grained target detection and obtain target detection information, which includes location information, category and confidence level;

[0015] S3, Input the target detection information into the image segmentation model, perform mask segmentation on the keyframe image to obtain the mask map and the mask region;

[0016] S4, feature point extraction is performed in the masked area using a corner detection algorithm;

[0017] S5, based on optical flow method, performs inter-frame tracking of corner points to obtain the displacement sequence of corner points;

[0018] S6. Perform frequency domain analysis on the displacement sequence to extract the vibration frequency.

[0019] As a preferred option, the keyframe extraction method is adjusted based on the vibration frequency range, shooting frame rate, and target motion change characteristics;

[0020] For the same cable vibration video, only one keyframe image is extracted for coarse-grained target detection and mask segmentation. Other frames are processed using the spatial location information obtained from the keyframe image, and steps S4 to S6 are performed.

[0021] As a preferred option, the object detection model adopts the YOLO object detection algorithm and introduces the Cable-IoU loss function as the bounding box regression loss term. The Cable-IoU loss function is as follows:

[0022]

[0023] L1=α·β·sin 2 (∣θ-θ gt ∣)

[0024] L2=η w (ww gt ) 2 +η h (hh gt ) 2

[0025] Among them, L CableIoU For joint angle-distance decoupling optimization and decoupling aspect ratio optimization, the loss function is 1-IoU as the basic IoU loss. ω1·L1 is the center point distance penalty, ω2·L2 is the angle penalty, ω2·L2 is the axial decoupling penalty, IoU is the intersection-union ratio of the predicted box and the ground truth box, and ρ is the center point distance penalty. 2 Let c be the Euclidean distance from the center point, c be the diagonal length of the closure region, and α be the aspect ratio sensitivity factor. The aspect ratio sensitivity factor adaptively adjusts the angle penalty weight based on the target shape, and the formula is defined as follows: w represents the width of the predicted bounding box, and h represents its height. β is the IoU dynamic decay mechanism, which dynamically adjusts the angle to optimize the intensity based on the overlap between the detected bounding box and the ground truth bounding box. Its formula is defined as β = exp(-IoU / 0.3), η w and η h As the axial importance weight, the penalty weights for width and height errors are dynamically allocated based on the target's true aspect ratio. The formulas are defined as follows: and θ gt For the direction-adaptive coordinate system to transform the angle, the formula is defined as follows: This angle represents the angle between the major axis of the true frame and the y-axis. gt h gt Let θ = arctan(w / h) represent the actual width and height of the target, and let θ = arctan(w / h) represent the angle between the major axis of the detection frame and the vertical direction.

[0026] The weighting coefficient ω1 represents the angle penalty weight, with a value of 0.2, and the weighting coefficient ω2 represents the axial decoupling penalty weight, with a value of 0.5.

[0027] As a preferred method, an image dilation operation is performed on the mask image to expand the boundary of the mask region corresponding to the cable. This boundary is used to define the spatial search area of ​​the corner detection algorithm.

[0028] Image dilation is achieved through morphological processing functions, using a fixed structuring element to dilate the mask once or multiple times.

[0029] As a preferred approach, in the masked region, the main axis of the structure is calculated by extracting the boundary contour point set and using the least squares fitting method, and a local orthogonal coordinate system (u,v) is constructed, where u is the direction of the main axis of the cable and v is the direction perpendicular to the main axis of the cable.

[0030] During inter-frame tracking, only the displacement component of each corner point in the v direction is retained, while the motion in the u direction is ignored;

[0031] The inter-frame tracking result is the displacement sequence of each corner point, represented as: in, This represents the displacement of the i-th corner point in the v direction in frame t, where T is the total number of frames.

[0032] As a preferred approach, before inter-frame tracking, Harris corner detection is performed on the mask region generated by the image segmentation model, and the number of valid corners is counted. If the number is lower than a preset threshold, it is determined that the mask image may have segmentation anomalies, and a mask inversion operation is performed. Then, Harris corner detection is re-performed. If the number of re-detected corners is not lower than the threshold, it indicates that the original mask segmentation direction is incorrect, and subsequent processing is performed on the mask region after the mask inversion operation. If the number of re-detected corners is still lower than the threshold, it indicates that the region may have severe occlusion, and the system automatically discards the mask of the cable region.

[0033] The mask inversion operation is to swap the pixel values ​​of the current mask region with those of the background region.

[0034] As a preferred approach, after inter-frame tracking is completed, if the displacement of a corner point between any two frames exceeds the set upper limit threshold, the point is considered an abnormal trajectory and is removed.

[0035] As a preferred approach, the displacement sequence of each corner point is analyzed in the frequency domain using Fast Fourier Transform to extract the vibration frequency.

[0036] As a preferred method, a fixed camera is used to acquire cable image data and train the target detection model.

[0037] The cable image data used covers a variety of bridge types, cable structure features, shooting angles and environmental conditions. After the cable images are acquired, the cable targets in the images are labeled frame by frame and cable by cable to build the manually labeled dataset required for training.

[0038] A computer storage medium storing a program that, when run on a computer, causes the computer to perform the methods described above.

[0039] In summary, the present invention has the following advantages:

[0040] (1) This invention achieves fully automatic cable target detection and pixel-level segmentation without physical targets and manual ROI setting by adopting an improved Cable-YOLO model and Segment Anything Model (SAM), which reduces deployment costs and manual intervention, and improves real-time performance and flexibility.

[0041] (2) The present invention uses an optical flow tracking strategy constrained by the main axis direction to retain only the vertical displacement, thereby avoiding the influence of background motion and camera shake, and enhancing the stability and accuracy of vibration recognition.

[0042] (3) The present invention designs a mask reversal mechanism and a displacement amplitude verification mechanism, which can effectively eliminate segmentation errors and abnormal displacement points, ensuring high reliability and accuracy of the results.

[0043] In summary, the above design makes the present invention more adaptable and robust in complex environments, and can provide more accurate and reliable bridge health monitoring data than existing methods. Attached Figure Description

[0044] Figure 1 This is the sample set of bridge cable images collected in Example 1.

[0045] Figure 2 This is a schematic diagram of the bridge cable image annotation interface in Example 1.

[0046] Figure 3 This is a graph showing the performance metrics changes during the training process of the Cable-YOLO model in Example 1.

[0047] Figure 4 This is a visual inspection result of the Cable-YOLO model in the bridge cable inspection task in Example 1.

[0048] Figure 5 This is a schematic diagram of image mask segmentation for bridge cables in Example 1.

[0049] Figure 6 This is a schematic diagram of the effective corner point screening module in Example 1.

[0050] Figure 7 This is a schematic diagram of the multi-cable synchronous vibration frequency identification system in Example 1.

[0051] Figure 8 This is a schematic diagram of the core process of Example 2.

[0052] Figure 9 This is a data comparison chart for Test 1 in Example 2.

[0053] Figure 10 This is a data comparison chart for Test 2 in Example 2.

[0054] Figure 11 This is a comparison chart of the data from test 3 in Example 2. Detailed Implementation

[0055] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0056] Example 1

[0057] A method for identifying cable vibration frequencies based on visual collaborative segmentation includes the following steps:

[0058] 1. Step S0: Establish the target detection model

[0059] To achieve automatic identification of bridge cable targets, this embodiment employs a deep learning-based target detection method and establishes a "Cable-YOLO" cable target detection model. This model is an improvement upon YOLOv8 (You Only Look Once, the 8th generation fast target detection network), possessing excellent real-time performance and high accuracy, and is suitable for structural identification tasks such as small targets and slender targets.

[0060] YOLO is an end-to-end object detection algorithm. The input is the original image, and the output is the location information (bounding boxes) of all objects in the image, along with their categories and confidence scores. YOLOv8 introduces a lighter network structure and a multi-scale feature fusion mechanism, improving the recognition ability of small objects without sacrificing speed. This invention introduces a self-developed model optimization mechanism based on YOLOv8, allowing the object detection model to specialize for cable-stayed objects, enhancing the model's ability to locate "slender targets."

[0061] Step S0.1: Acquisition of cable image data (optional)

[0062] To construct a target detection model with good detection capabilities and environmental adaptability, the acquisition of cable-stayed bridge image data should cover as many bridge types, structural features, shooting angles, and environmental conditions as possible. This embodiment, based on actual engineering needs, collected a total of 836 high-resolution cable-stayed bridge images from ten typical cable-stayed bridges in Guangdong Province to construct a diverse and highly generalizable training dataset. All images were captured using high-definition industrial cameras with a resolution of no less than 1920×1080, ensuring complete and clear cable edge information.

[0063] The data collection work follows the following principles: (1) Multi-view coverage: including front view, side view, and tilted distant view of the cable body, covering shooting distances of approximately 10 to 200 meters; (2) Multiple environmental scenes: shooting images of the bridge under different times and lighting conditions, covering scenes under sunny, cloudy, and nighttime lighting; (3) Multiple background elements: including background elements such as tall buildings, water surfaces, traffic lights, vehicles, billboards, cranes, cables, and other cable bodies, to improve the robustness of the model in real-world scenarios; (4) Diverse bridge structures: the selected bridges include typical structural forms such as municipal main road bridges, cross-river passages, and urban landscape bridges, which are widely representative.

[0064] After image acquisition, all data are categorized and organized according to structural features, shooting direction, and lighting conditions for subsequent training and validation of the target detection model.

[0065] Figure 1 This paper showcases cable-stayed bridge images collected by this invention from ten typical cable-stayed bridges in Guangdong Province, including various structural types such as urban expressway bridges, cross-river bridges, and bridges for nighttime scenic views. The images were collected from various angles, including frontal, oblique, and distant views, with complex and varied background elements such as urban buildings, bridge towers, traffic flow, lights, and water surfaces, demonstrating the dataset's ability to cover both environmental diversity and target complexity. A total of 836 images were collected and used to construct a target detection training set with real-world engineering adaptability.

[0066] Step S0.2: Data labeling (optional)

[0067] After acquiring the cable image data, to ensure the target detection model has clear supervisory information, it is necessary to annotate the cables in the images frame by frame and cable by cable. This embodiment uses the open-source image annotation tool LabelImg to select bounding boxes, thereby constructing the manually annotated dataset required for training.

[0068] During the annotation process, the operator needs to manually mark each clearly visible cable in the image with a rectangle and assign it a uniform category number. All annotation information is saved in XML (Extensible Markup Language, PascalVOC format) or TXT (text file format, YOLO format). The file records the category number of each cable target and its corresponding boundary coordinates, i.e., the pixel positions of the upper left and lower right corners.

[0069] To ensure consistency and learnability of the annotation results, standardized annotation specifications were implemented in practice. All cables should be labeled as whole sections whenever possible, with the bounding box covering the entire length of the cable to avoid training bias caused by labeling only localized areas. In actual engineering images, due to shooting angles or obstructions, some cables may appear interrupted or partially occluded in the image. In such cases, the cable can be treated as multiple visible segments and labeled separately to ensure that each visible area provides boundary information. Cable targets that cannot be accurately identified in the image due to blurring, overexposure, or low contrast are not recommended for annotation to avoid interference from noisy data in model training.

[0070] During the annotation process, the operator should flexibly adjust the range and number of annotation boxes based on image quality and cable features. The final annotated dataset should cover various cable arrangements, background complexity, and viewpoint conditions to improve the model's adaptability to different scenes. In this embodiment, all label files were uniformly named, validated, and formatted after annotation to ensure seamless use in the training process of the YOLOv8 object detection model.

[0071] Figure 2 This illustration demonstrates the interface for cable annotation using the LabelImg tool in this embodiment. The image shows four parallel cables. The operator selects each clearly visible cable with a rectangular bounding box, creating a gray bounding box. The blue area represents the region enclosed by the selected bounding box. Each bounding box requires corresponding annotation information, including the target number, boundary coordinates, and label file path. This example illustrates the annotation strategy for multiple parallel cables and demonstrates the strict control requirements of this invention regarding image annotation accuracy and consistency.

[0072] Step S0.3: Model Optimization and Training (Optional)

[0073] To improve the accuracy and stability of cable target detection models in the identification of slender structures, this invention constructs a dedicated detection model "Cable-YOLO" based on the YOLOv8 target detection network and designs an improved loss function, Cable-IoU, suitable for bridge cable detection tasks. This step aims to optimize the training process to enable the model to have strong robustness and generalization ability in engineering scenarios such as multi-target, complex backgrounds, and weak textures.

[0074] Although YOLOv8 has strong detection performance, its traditional bounding box regression mechanism is prone to fitting bias when dealing with "slender targets", especially when the cables are distributed diagonally or partially occluded, the error accumulation is more obvious.

[0075] To address the aforementioned issues, this embodiment introduces a custom loss function, Cable-IoU, to replace the bounding box regression loss term in the standard YOLOv8. This loss function is based on the IoU (Intersection over Union) principle and, by fusing multiple structural optimization terms, applies more refined geometric constraints to the detection boundaries of slender targets. Cable-IoU integrates two self-developed IoU improvements: angle-distance decoupled optimization Cable-SIoU and decoupled aspect ratio optimization Cable-EIoU. Extensive algorithmic experiments have verified that fusing these two methods yields better performance. The fused loss function is defined as Cable-IoU, as shown in the following formula:

[0076]

[0077] L1=α·β·sin 2 (∣θ-θ gt ∣)

[0078] L2=η w (ww gt ) 2 +η h (hh gt ) 2

[0079] Among them, L CableIoU For joint angle-distance decoupling optimization and decoupling aspect ratio optimization, the loss function is 1-IoU as the basic IoU loss. ω1·L1 is the center point distance penalty, ω2·L2 is the angle penalty, ω2·L2 is the axial decoupling penalty, IoU is the intersection-union ratio of the predicted box and the ground truth box, and ρ is the center point distance penalty. 2 Let c be the Euclidean distance from the center point, c be the diagonal length of the closure region, and α be the aspect ratio sensitivity factor. The aspect ratio sensitivity factor adaptively adjusts the angle penalty weight based on the target shape, and the formula is defined as follows: w represents the width of the predicted bounding box, and h represents its height. β is the IoU dynamic decay mechanism, which dynamically adjusts the angle to optimize the intensity based on the overlap between the detected bounding box and the ground truth bounding box. Its formula is defined as β = exp(-IoU / 0.3), η w and η h As the axial importance weight, the penalty weights for width and height errors are dynamically allocated based on the target's true aspect ratio. The formulas are defined as follows: and θ gt For the direction-adaptive coordinate system to transform the angle, the formula is defined as follows: This angle represents the angle between the major axis of the true frame and the y-axis. gt h gtLet θ = arctan(w / h) represent the actual width and height of the target, and let θ = arctan(w / h) represent the angle between the major axis of the detection frame and the vertical direction.

[0080] In the Cable-IoU loss function design, in addition to the basic IoU loss and center distance penalty term, the Cable-SIoU angle penalty and the Cable-EIoU axial decoupling penalty are also activated simultaneously. The weight coefficient ω1 represents the angle penalty weight, adjusting the sensitivity of directional alignment optimization to avoid over-optimization in high IoU scenarios. The weight coefficient ω2 represents the axial decoupling penalty weight, controlling the impact of the axial decoupling penalty term on CableIoU, allowing the model to adapt to the extreme aspect ratio characteristics of the cable shape.

[0081] The default values ​​for the two weighting coefficients ω1 and ω2 are designed to be 0.2 and 0.5, respectively. These default values ​​represent the optimal parameter combination determined by the parametric grid search experiment. While maintaining the main IoU structure, this function strengthens the explicit penalty for error factors such as center point offset, boundary angle, and aspect ratio mismatch, significantly improving the boundary alignment capability for slender targets (such as cables) and reducing the occurrence of misaligned, missing, and off-center frames.

[0082] The training phase utilizes the bridge cable image dataset collected and labeled using this invention, and is performed on a high-performance GPU server based on the PyTorch deep learning framework. Specific parameter configurations are as follows: Hardware platform: NVIDIA A800 data center-grade GPU, 80GB of memory per card; Network architecture: YOLOv8-s (lightweight version); Optimizer: Adam optimizer with weight decay, adaptive weight adjustment; Initial learning rate: 0.001, learning rate decay method: cosine annealing; Batch size: 128; Training epochs: 674 epochs; Loss function configuration: classification loss (cross-entropy) + target confidence loss + regression loss (Cable-IoU); Data augmentation methods: random image cropping, brightness perturbation, rotation, mirror flipping, Gaussian blur, etc.

[0083] During training, the performance metrics of the model in this invention gradually converged on the validation set, and finally achieved the following performance results on the test set: Precision 98.462%, Recall 95.238%, Average Precision (mAP@0.5) 98.583%, and Overall Precision (mAP@0.5:0.95) 92.988%. These metrics demonstrate that the Cable-YOLO model proposed in this invention possesses extremely high detection accuracy and boundary fitting ability in bridge cable inspection tasks, and is particularly suitable for engineering requirements involving high-precision detection of slender targets.

[0084] In terms of operational efficiency, the Cable-YOLO model, under 1080P image input conditions, has a measured single-frame inference time of less than 20 milliseconds, and a theoretical processing speed of more than 50 frames per second, which can meet the comprehensive requirements of bridge monitoring systems for real-time performance, stability and processing throughput in actual deployment.

[0085] Figure 3 This paper demonstrates the performance metrics of the Cable-YOLO model during the training and validation phases, including the trends of bounding box regression loss (boxloss), classification loss (clsloss), distribution focal loss (DFLloss), precision, recall, mean accuracy (mAP@0.5), and mAP@0.5:0.95 as a function of training epochs. The results show that the model's metrics stabilize after 500 training epochs, exhibiting good convergence and high detection performance, laying the foundation for subsequent engineering deployment.

[0086] Figure 4 The results of the model of this invention on typical bridge cable images are shown. Blue bounding boxes mark each cable identified by the model, and the numbers inside the boxes represent the category "cable" and its confidence score. The figures show that the model can simultaneously identify multiple cables and adapt to different viewing angles, occlusions, lighting conditions, and background conditions, demonstrating its good robustness and reliability in complex real-world environments.

[0087] After the training and testing are completed, the weight file is packaged into a standard deployment format, allowing users to directly call the model for image detection tasks without repeating the training process. If performance optimization is required under specific bridge structures, special climates, or lighting conditions, the model can also be transferred or fine-tuned according to the above training process to enhance its scene adaptability.

[0088] Once trained, the Cable-YOLO model can be directly deployed in a bridge monitoring system to achieve automatic detection and region extraction of bridge cables in images or video streams.

[0089] 2. Step S1, Video processing based on keyframe selection

[0090] In this embodiment, target detection and segmentation operations are performed only on preset keyframes in the video sequence to reduce redundant calculations and improve the overall processing speed.

[0091] Video input can be real-time monitoring streams acquired by structural health monitoring systems, video recorded by drone inspections, or offline structural inspection images. The selection of keyframes can be flexibly adjusted based on the vibration frequency range, frame rate, and target motion characteristics to ensure a balance between the completeness and efficiency of spatial information acquisition. By default, the system extracts only the first frame of the video as a keyframe for processing, completing automatic cable detection and mask extraction. The system also supports user-defined keyframe extraction strategies, such as selecting one frame every N frames from the video as a keyframe (N can be set to 5, 10, 15, etc.). Non-keyframes reuse the spatial location information analyzed from the previous keyframe and are then used in the corner tracking and vibration analysis modules. This design significantly reduces redundant image recognition overhead and is suitable for typical bridge monitoring scenarios with small cable movement amplitudes and stable camera shooting. In applications requiring higher spatiotemporal resolution, users can also configure the keyframe interval or enable a dynamic keyframe detection strategy as needed.

[0092] 3. Step S2: Perform coarse-grained detection of the cable region using the target detection model.

[0093] When extracting multiple keyframes, at each keyframe, the input image is first processed by the Cable-YOLO model, which automatically identifies all "cable" category targets and returns the bounding box location and confidence score. Each target is defined by [x...]. min ,y min ,x max ,y max The quintuple representation is called [confidence]. Where x... min and y min These represent the horizontal and vertical coordinates of the top-left corner of the bounding box, in pixels, and are used to determine the starting position of the cable in the image; x max and y max ...

[0094] For the output format, the system packages and stores the keyframe image number, detection bounding box coordinates, target category, and confidence score together as a structured file (such as JavaScript object representation, JSON; comma-separated value format, CSV), and provides an interface for subsequent modules to call (such as corner detection and optical flow tracing). This strategy is particularly suitable for real-world deployment scenarios with limited computing resources, many static backgrounds, and slow inter-frame motion changes.

[0095] The output detection results will be passed as prompts to the subsequent image segmentation module, which will then perform mask segmentation in step S3 below. It is worth noting that during non-keyframe periods, the system does not repeat object detection and image segmentation, thus significantly reducing processing resource consumption and exhibiting good real-time performance and computational efficiency.

[0096] 4. Step S3, fine segmentation of the cable region, that is, mask segmentation of the keyframe image.

[0097] After completing the coarse-grained target detection of the cables in step S2, this embodiment further employs image segmentation technology to refine the boundary of each cable at the pixel level. This process is achieved by introducing the Segment Anything Model (SAM), proposed by Meta AI, which possesses powerful zero-shot instance segmentation capabilities and supports the automatic generation of fine masks of targets based on prompt boxes without relying on manual annotation. This embodiment introduces this technology for the first time into the visual monitoring task of bridge cables to improve the spatial description accuracy of the cable area, reduce background interference, and enhance the accuracy and robustness of subsequent vibration analysis. Specifically, it includes the following steps:

[0098] Step S3.1: Generate bounding box hints

[0099] In this embodiment, each cable bounding box [x] output by the Cable-YOLO detection in step S2 will be... min ,y min ,x max ,y max [confidence] is used alone as a spatial cue input for the SAM model. This cue tells the SAM model: the target to be segmented is located within this rectangle, please generate a mask image within this area to accurately delineate the target outline.

[0100] The system iterates through all bounding boxes on each keyframe and calls the SAM inference interface to perform mask prediction tasks one by one.

[0101] Step S3.2: Mask Output and Format

[0102] The output of the SAM model is a binary mask image, which is a binary image of the same size as the input image. In the masked area, the pixel value is 1 (or white), indicating that the area belongs to the cable; the pixel value of the non-masked area is 0 (or black), indicating the background.

[0103] Each cable corresponds to a separate mask image. The system supports exporting and saving all masks in Portable Network Graphics (PNG) or Numerical Python (NumPy) format, and recording them together with the original keyframe numbers and bounding box coordinates for easy subsequent calling and result traceability.

[0104] Step S3.3: Mask expansion treatment

[0105] In this invention, after the cable mask map is generated, an image dilation operation is performed on the initial mask region of each cable to more fully capture the subtle vibration changes of the cable in the video. This step aims to appropriately expand the mask boundary, thereby ensuring that the subsequent corner extraction and optical flow tracking stages cover all areas that may deform, especially edge offsets caused by slight lateral swaying or cable surface fluctuations.

[0106] The dilation operation is implemented through morphological processing functions, using fixed structuring elements (such as 3×3 or 5×5 cross-shaped kernels) to dilate the mask once or multiple times. The result is an "expanded mask map" for each cable, with the mask area slightly larger than the original SAM output, providing stronger spatial coverage.

[0107] This process has the following characteristics: it maintains the overall shape of the mask and expands uniformly only in the edge direction; it does not perform other complex edge repair or smoothing operations to avoid introducing boundary deviations; and it is directly used to limit the spatial search area of ​​the next corner extraction algorithm.

[0108] This simple yet crucial processing step effectively enhances the system's response to minute cable vibrations and improves the robustness of vibration feature extraction. When multiple cables are present in an image, the system's final output is a set of high-precision pixel-level mask images of individual cables, which are used in subsequent corner extraction and optical flow tracking modules.

[0109] Figure 5The image illustrates typical results of the cable mask generation process in this embodiment. The left column shows the cable boundary box region (red box) detected by Cable-YOLO, corresponding to coarse-grained target localization; the right column shows the refined segmentation mask image generated by Segment AnythingModel (SAM) based on the boundary box prompts. The white area represents the range of cable pixels identified by the model, and the black area represents the background. It can be seen that SAM can accurately extract cable edge information, significantly outperforming the traditional rectangular box representation, providing a high-quality input foundation for subsequent vibration analysis. Simultaneously, the image demonstrates the system's ability to parallel identify and synchronously segment multiple cable targets, indicating that this invention possesses efficient processing and automated adaptability capabilities for complex multi-cable environments in actual bridge structures.

[0110] 5. Step S4: Corner Extraction

[0111] This step aims to extract corner points from the cable image as representative feature points and track their inter-frame motion using optical flow to obtain the cable's vibration response time series. The entire process revolves around the extracted cable mask region to avoid background interference and improve tracking accuracy. The specific operations are as follows:

[0112] Within the cable mask region, this invention employs the classic Harris Corner Detection algorithm for feature point extraction, used for initialization of subsequent optical flow tracking. Based on the autocorrelation of image grayscale changes, the Harris algorithm can generate strong responses at locations with obvious local structural features (such as corners and edge intersections), and is therefore widely used for image feature extraction in stable scenes.

[0113] Considering the slender shape and weak texture of the cable target, this invention limits corner detection to the area extracted in the previous stage using the cable mask. The mask can be expanded appropriately through dilation to avoid boundary truncation. In this way, the system avoids interference from background textures, other components, or noise on corner detection, ensuring that all extracted points are located on the structural target.

[0114] 6. Step S5: Optical flow tracing with spindle constraints

[0115] After corner point extraction, this invention performs inter-frame tracking of each corner point based on sparse optical flow to obtain the motion trajectory of cable vibration in the video time series. While traditional optical flow methods possess high-precision displacement estimation capabilities, they are easily affected by background motion, image jitter, or unstructured interference in engineering practice, leading to corner point drift or distortion of vibration features. Therefore, this invention proposes an optical flow tracking strategy that incorporates structural principal axis direction constraints, effectively improving the physical rationality and robustness of the tracking results.

[0116] Step S5.1: The corner points are tracked frame by frame using the Pyramidal Lucas-Kanade Optical Flow algorithm.

[0117] This algorithm, based on the assumption of local brightness consistency, estimates the displacement of corner points across consecutive frames by minimizing the gray-level difference in the image. Considering the characteristics of cable targets, such as weak texture and small displacement, the system constructs a multi-layer pyramid in the image and performs scale recursive calculation, supporting sub-pixel level displacement resolution and is suitable for scenarios involving subtle vibration analysis.

[0118] To further improve the physical interpretability of the tracking results, this invention introduces a cable main axis direction constraint mechanism. Specifically, in the masked region of each cable, the system extracts its boundary contour point set, calculates the main axis of the structure using a least squares fitting method, and constructs a local orthogonal coordinate system (u,v), where: u is along the cable main axis direction (i.e., the axis of the structure body); v is perpendicular to the cable main axis direction, i.e., the main vibration direction.

[0119] During optical flow tracing, this invention retains only the displacement component of each corner point in the v direction, ignoring the u-motion. This projection operation eliminates parallel drift caused by changes in imaging viewpoint, slight camera movement, or background disturbances, ensuring that the extracted motion trajectory represents only the true structural vibration response.

[0120] The tracking results will create a time series for each corner point: in, This represents the vertical displacement of the i-th corner point in frame t, where T is the total number of frames. This time series will serve as the input signal for subsequent spectral analysis. The system supports parallel processing of multiple cables, associating each set of corner point data with its corresponding structure number through an indexing mechanism to achieve multi-cable synchronous tracking capability.

[0121] The spindle-constrained optical flow strategy of this invention not only significantly improves the stability of corner tracking in low-texture, background interference and low-light environments, but also enhances the physical consistency of motion data through structural prior information, which is one of the key steps to achieve high-precision, low-noise cable vibration recognition.

[0122] In addition, this embodiment also includes a robustness enhancement mechanism, comprising the following steps S5.0 and S5.2:

[0123] To improve the stability of optical flow tracking and the reliability of structure recognition, this invention designs a robustness enhancement mechanism that includes both feedforward and feedback stages. This mechanism operates before corner point extraction (pre-judgment verification) and after vibration trajectory generation (result verification), ensuring that the feature point data ultimately used for frequency analysis has high quality, high reliability, and strong physical consistency.

[0124] Step S5.0, Valid Corner Filtering (Feedforward Verification)

[0125] This module dynamically verifies the cable segmentation mask before optical flow tracing, primarily addressing segmentation failures such as incorrect mask orientation recognition and occlusion. Its core mechanism is as follows:

[0126] like Figure 6 As shown, within the Cable-YOLO detection bounding box, the system first performs Harris corner detection on the masked area generated by SAM (Segment Anything Model) and counts the number of valid corners. When this number is lower than a preset threshold N... min (default N) min If the value is 5), then the mask may have a segmentation anomaly, and the "mask inversion mechanism" will be entered. Figure 6 (a), (b), and (c) in the figure respectively illustrate the process of Cable-YOLO selecting the region, the error in the segmentation direction of SAM segmentation, and the triggering of the inversion mask mechanism.

[0127] The mask inversion mechanism operates by swapping the foreground and background pixel values ​​of the current mask and re-performing corner detection. The specific judgment process is as follows: If the number of corners after inversion is still below the threshold, it indicates severe occlusion in the area (such as objects hanging on the bridge, advertising banners, dense fog, or dense vegetation cover), and the system automatically discards the mask for that cable area to avoid invalid tracking; if the number of corners after inversion meets the threshold, it indicates an error in the original mask segmentation direction (such as mistakenly setting the background as the foreground), and the inverted mask is used to continue the subsequent corner extraction and optical flow tracking process.

[0128] This fault-tolerance mechanism can effectively improve segmentation reliability, fix the mask failure problem caused by insufficient zero-sample generalization ability of the SAM model, and ensure that the image region input to the optical flow method truly covers the structure itself.

[0129] Step S5.2, Effective Displacement Judgment (Feedback Verification)

[0130] After completing feature point tracking using the optical flow method, this invention further introduces a dynamic motion analysis mechanism to perform "displacement amplitude" verification on the tracking trajectory, thereby filtering out unstructured disturbances caused by changes in illumination, background interference, etc.

[0131] The system monitors the pixel displacement values ​​of all corner points in each frame. If the displacement of a corner point between any two frames exceeds a set upper limit threshold D... max (Default D) max If the value is less than 4 pixels, then the point is considered an abnormal trajectory and is immediately removed from the tracking queue.

[0132] The threshold setting is based on the prior knowledge of the physical amplitude of bridge cables: under normal operating conditions, the inter-frame displacement of the cable body is minimal, while strong light sources in the background (such as car headlights at night, birds flying across, construction machinery, etc.) will cause strong grayscale changes, misleading the optical flow method to produce large drift. This mechanism can significantly eliminate the error points caused by the above-mentioned non-structural motion.

[0133] Through the above dual mechanisms, this invention establishes physical and motion consistency checks at both ends of the "segmentation-tracking" process, effectively avoiding invalid mask propagation and abnormal trajectory contamination. This method does not rely on additional model training.

[0134] 7. Step S6: Vibration Frequency Extraction and Analysis Module

[0135] After completing optical flow tracing and obtaining time-series data of feature points on each cable, this invention further designs a frequency extraction and modal analysis module to identify the dominant frequency and structural response characteristics of the cables. This module, based on classical signal processing methods and combined with the actual needs of the engineering environment, provides stable, fast, and computationally inefficient frequency domain analysis capabilities.

[0136] Step S6.1: Organizing and Standardizing the Feature Point Displacement Sequence

[0137] After completing optical flow tracking, the system uniformly organizes the coordinate changes of all effective corner points in the keyframe sequence, generating multiple sets of time displacement sequences as the raw input data for frequency extraction.

[0138] For each corner point i, the system records its position coordinates in each frame and calculates the displacement increment along the vertical principal axis between adjacent frames, forming a discrete-time series. To facilitate unified analysis and processing, this invention converts all sequences into NumPy array format and categorizes and stores them according to the cable number to which the corner points belong.

[0139] If some corner points are lost in tracking in some frames, the system will automatically remove the corner points or treat the missing frames as "default values" and only retain complete sequences with good continuity.

[0140] The core objective of this step is to transform image-level tracking data into a structured time-displacement sequence data format, providing standardized input for subsequent spectrum calculation and modal fusion modules.

[0141] Step S6.2: Spectrum Analysis and Main Frequency Identification

[0142] After completing the corner displacement sequence, this invention uses Fast Fourier Transform (FFT) to perform frequency domain analysis on the vertical displacement sequence of each corner point in order to extract its main vibration frequencies.

[0143] The system first performs a Discrete Fourier Transform on the time series of each valid corner point, transforming it from the time domain to the frequency domain. By analyzing the amplitude distribution of the spectrum, the dominant vibration frequency of that point throughout the entire video segment can be accurately identified. The frequency unit is Hertz (Hz), which is typically concentrated in the range of 0.5Hz to 20Hz in bridge cable scenes.

[0144] To ensure the stability and reliability of the main frequency peak, the system sets the following identification principles: the selected main frequency peak must have a significant energy advantage, distinguishing it from background noise; the main frequency should be within a reasonable physical range of structural vibration; if there are multiple similar frequency components, the one with the largest amplitude is selected.

[0145] In summary, this invention constructs a method for identifying the vibration frequency of bridge cables based on visual collaborative segmentation, enabling automatic detection and dynamic characteristic extraction of bridge cables in a non-contact manner. This method integrates deep learning object detection, zero-shot segmentation, corner tracking, and structural constraint optical flow algorithms, forming a bridge structural health monitoring solution with high precision, high automation, and strong environmental adaptability.

[0146] The above method can be implemented using the following multi-cable synchronous vibration frequency identification system based on visual segmentation and optical flow tracing, such as... Figure 7 As shown, the system includes the following four modules:

[0147] (1) Feature extraction module: Execute the above steps S1, S2 and S3, input video key frames, complete the coarse-grained detection of the cable target through the Cable-YOLO model, and output bounding box prompts; combine the SAM (Segment Anything Model) model to perform fine-grained segmentation and generate pixel-level masks for each cable.

[0148] (2) Robustness Enhancement Mechanism: This includes an effective corner selection module and an effective displacement threshold module to execute steps S5.0 and S5.2 above. Based on the segmentation results, corner number screening and abnormal direction verification are performed. If the number of corners is insufficient (less than the threshold N),... min If the segmentation direction is incorrect, the mask inversion mechanism is triggered. Furthermore, abnormal amplitude feature points (displacement greater than the threshold D) are removed. max This is to improve the reliability of downstream tracking.

[0149] (3) Vibration analysis module: Execute the above steps S4 and S5.1, extract Harris corner points in each mask area, and perform frame-by-frame tracking using the pyramid-level Lucas-Kanade optical flow algorithm; using the unidirectional optical flow method, the system only retains the displacement of the corner points in the direction perpendicular to the main axis of the cable (the main direction of vibration) to form a high-quality time series.

[0150] (4) Frequency Extraction Module: Execute step S6 above to perform a Fast Fourier Transform (FFT) on the corner time series obtained from the tracking, extract the dominant frequency response results, and use them to evaluate the structural state. Ultimately, synchronous vibration analysis and frequency statistics of multiple cables can be achieved.

[0151] Example 2

[0152] This embodiment aims to verify the effectiveness of the cable vibration frequency identification method based on visual collaborative segmentation proposed in this invention. By comparing with an accelerometer, the stability and accuracy of the method in an indoor control environment are verified. In conjunction with Embodiment 1, the experimental steps of this embodiment's cable vibration frequency identification method based on visual collaborative segmentation mainly include cable image acquisition, target detection and segmentation, optical flow tracking, and frequency extraction, as detailed below.

[0153] Step 1: Image Acquisition

[0154] In a laboratory environment, images of bridge cables are acquired using a fixed camera.

[0155] Step 2: Object Detection and Segmentation

[0156] like Figure 8 As shown in (a), the acquired image is input into the Cable-YOLO model, which automatically detects the cable positions in the image and generates bounding boxes. Then, as... Figure 8 As shown in (b), the detected region is finely segmented using the SAM model to generate an accurate mask for each cable.

[0157] Step 3: Optical Flow Tracing

[0158] like Figure 8 As shown in (c), within each segmented region of the cable, Harris corner detection is used to extract feature points, and a principal axis constrained optical flow strategy is used for frame-by-frame displacement tracking. By tracking the movement of corner points between consecutive frames, dynamic displacement data of the cable is obtained.

[0159] Step 4: Vibration frequency extraction

[0160] Spectral analysis was performed on the time-series data generated from optical flow tracing, and the dominant frequency was extracted using Fast Fourier Transform (FFT). The system can identify the main vibration modes of the cable and accurately extract their dominant frequencies.

[0161] Step 5: Data Analysis and Evaluation

[0162] The reliability of the frequency extraction results obtained by the method of the present invention is evaluated by comparing them with actual vibration data measured by an accelerometer. An accelerometer is used as a comparison standard to measure the vibration frequency of the cable under the same vibration conditions, and the results are compared with those obtained through image analysis.

[0163] Figure 8 The diagram illustrates the main steps of the above process. In summary, in this process, firstly, the input image undergoes coarse-grained object detection using the Cable-YOLO model, marking the approximate location of each cable in the image, such as... Figure 8 As shown in (a). Next, based on the bounding box cueing of the Cable-YOLO model, the SAM model performs fine segmentation of the detection region and outputs a precise mask of the cable, as shown in (a). Figure 8 As shown in (b). In this process, the pixel-level segmentation effect of the cable is significantly better than that of traditional rectangular box detection. Finally, after obtaining the refined segmentation results, Harris corner detection and principal axis constrained optical flow are applied for corner extraction and tracking to ensure that only the displacement of the cable in the principal axis direction is tracked, such as... Figure 8 As shown in (c). Through this series of processing steps, the present invention can accurately identify and analyze the vibration characteristics of cables, providing reliable data support for structural health monitoring.

[0164] This embodiment acquires vibration response data through three independent repeated tests, simultaneously obtaining vibration signals from two sets of cables (cable A and cable B) in each test. The testing system integrates a visual measurement device and a contact accelerometer, simultaneously recording the optical flow displacement time history curve and the accelerometer integrated displacement data.

[0165] Tables 1 and 2 present the frequency detection results for cable A and cable B. Table 1 lists the frequency detection results for cable A, including the measured values ​​of the primary and secondary frequencies, the accelerometer's baseline data, and the absolute and relative errors; Table 2 lists the frequency detection results for cable B, containing similar data items.

[0166] Table 1. Results of Cable A Frequency Detection

[0167]

[0168] Table 2. Results of Cable B Frequency Detection

[0169]

[0170] Figures 9 to 11The data comparison between the method of this invention and that of a traditional accelerometer is presented in three tests. The green curve represents the measurement results of the method of this invention, and the red curve represents the baseline data obtained by the accelerometer. To highlight the performance differences between the two types of sensing technologies in the dynamic response stage, the displacement comparison chart focuses on the dynamic response data during the artificial excitation stage for comparative analysis. Local scaling is used to enhance the visual recognition of key time-domain features.

[0171] Figure 9 The data comparison of Test 1 is shown, including the displacement and spectrum comparison of cable A and cable B; Figure 10 For the data comparison of Test 2, Figure 11 The data from Test 3 is compared. These test results demonstrate that the method of this invention exhibits high tracking accuracy during the excitation process, accurately reflecting the real-time dynamics of cable vibration. Particularly in the displacement curve after excitation, it maintains consistency and effectively eliminates unnecessary baseline shifts, ensuring data stability and reliability. Spectral analysis further proves that the method of this invention can accurately capture the response of the main frequency bands in vibration frequency identification, especially demonstrating excellent noise suppression in higher frequency bands. By introducing a pixel-level motion decoupling mechanism, the method of this invention effectively reduces the influence of background interference and unstructured noise, improving the accuracy of spectral analysis. This mechanism ensures clear and stable vibration characteristic data in both high-frequency and low-frequency noise environments, thereby enhancing the adaptability of the monitoring system in complex environments.

[0172] Example 3

[0173] A computer storage medium storing a program that, when run on a computer, causes the computer to execute a cable vibration frequency identification method based on visual collaborative segmentation as described in Embodiment 1.

[0174] The above embodiments are preferred embodiments of the invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for identifying the vibration frequency of cables based on visual collaborative segmentation, characterized in that: Includes the following steps, S1, extract keyframes from the cable vibration video to obtain keyframe images; S2, input the keyframe image into the target detection model to perform coarse-grained target detection and obtain target detection information, which includes location information, category and confidence level; S3, Input the target detection information into the image segmentation model, perform mask segmentation on the keyframe image to obtain the mask map and the mask region; S4, feature point extraction is performed in the masked area using a corner detection algorithm; S5, based on optical flow method, performs inter-frame tracking of corner points to obtain the displacement sequence of corner points; S6. Perform frequency domain analysis on the displacement sequence to extract the vibration frequency; The object detection model uses the YOLO object detection algorithm and introduces the Cable-IoU loss function as the bounding box regression loss term. The Cable-IoU loss function is as follows: ， ， ， Among them, L CableIoU For joint angle-distance decoupling optimization and decoupling aspect ratio optimization, the loss function is 1-IoU as the basic IoU loss. The center point distance penalty term, For angle penalty items, Here, IoU is the intersection-union ratio of the predicted bounding box and the ground truth bounding box, representing the axial decoupling penalty. The Euclidean distance is the center point. The length of the diagonal of the closure region. The aspect ratio sensitivity factor is used to adaptively adjust the angle penalty weight based on the target shape. The formula is defined as follows: w is the width of the prediction box and h is the height of the prediction box; This is an IoU dynamic attenuation mechanism that dynamically adjusts the angle to optimize the intensity based on the overlap between the detection box and the ground truth box. Its formula is defined as follows: ; and As the axial importance weight, the penalty weights for width and height errors are dynamically allocated based on the target's true aspect ratio. The formulas are defined as follows: and , For the direction-adaptive coordinate system to transform the angle, the formula is defined as follows: This angle represents the angle between the major axis and the y-axis of the true bounding box. , For the target's actual width and height, , representing the angle between the major axis of the detection frame and the vertical direction; Weighting coefficient The representative angle penalty weight has a value of 0.2, and the weight coefficient is... This represents the axial decoupling penalty weight, with a value of 0.

5. Within the masked region, the main axis of the structure is calculated by extracting the boundary contour point set and using the least squares fitting method. A local orthogonal coordinate system (u,v) is then constructed, where u is the direction of the main axis of the cable and v is the direction perpendicular to the main axis of the cable. During inter-frame tracking, only the displacement component of each corner point in the v direction is retained, while the motion in the u direction is ignored; The inter-frame tracking result is the displacement sequence of each corner point, represented as: ,in, This represents the displacement of the i-th corner point in the v direction in frame t, where T is the total number of frames.

2. The cable vibration frequency identification method based on visual collaborative segmentation according to claim 1, characterized in that: The keyframe extraction method is adjusted based on the vibration frequency range, shooting frame rate, and target motion change characteristics; For the same cable vibration video, only one keyframe image is extracted for coarse-grained target detection and mask segmentation. Other frames are processed using the spatial location information obtained from the keyframe image, and steps S4 to S6 are performed.

3. The cable vibration frequency identification method based on visual collaborative segmentation according to claim 1, characterized in that: The mask image is dilated to expand the boundary of the mask region corresponding to the cable. This boundary is used to limit the spatial search area of ​​the corner detection algorithm. Image dilation is achieved through morphological processing functions, using a fixed structuring element to dilate the mask once or multiple times.

4. The cable vibration frequency identification method based on visual collaborative segmentation according to claim 1, characterized in that: Before inter-frame tracking, Harris corner detection is performed on the mask region generated by the image segmentation model to count the number of valid corners. If the number is lower than a preset threshold, it is determined that the mask image may have segmentation abnormalities. A mask inversion operation is performed, and then Harris corner detection is re-executed. If the number of re-detected corners is not lower than the threshold, it indicates that the original mask segmentation direction is incorrect. Subsequent processing is performed on the mask region after the mask inversion operation. If the number of re-detected corners is still lower than the threshold, it indicates that the region may have severe occlusion, and the system automatically discards the mask of the cable region. The mask inversion operation is to swap the pixel values ​​of the current mask region with those of the background region.

5. A method for identifying cable vibration frequency based on visual collaborative segmentation according to claim 1, characterized in that: After inter-frame tracking is completed, if the displacement of a corner point between any two frames exceeds the set upper limit threshold, the point is considered an abnormal trajectory and is removed.

6. The cable vibration frequency identification method based on visual collaborative segmentation according to claim 1, characterized in that: The displacement sequence of each corner point was analyzed in the frequency domain using Fast Fourier Transform to extract the vibration frequency.

7. A method for identifying cable vibration frequency based on visual collaborative segmentation according to claim 1, characterized in that: A fixed camera was used to collect cable image data, which was then used to train the target detection model. The cable image data used covers a variety of bridge types, cable structure features, shooting angles and environmental conditions. After the cable images are acquired, the cable targets in the images are labeled frame by frame and cable by cable to build the manually labeled dataset required for training.

8. A computer storage medium, characterized in that: The computer stores a program that, when run on the computer, causes the computer to perform the method as described in any one of claims 1 to 7.

Images