A video recognition algorithm based on a vortex video recognition model

Abstract

The application provides a video recognition algorithm based on a vortex video recognition model, relates to the technical field of video analysis, is deployed in the form of a back-end service on a platform layer of a dike safety monitoring system, and a construction process comprises video data acquisition and processing, data labeling, model construction, model training and optimization, and model deployment and application.The application can effectively overcome the interference of light changes, turbid water quality and complex backgrounds in natural water bodies by performing feature-level fusion of video image features and hydrological, GNSS, osmotic pressure and other multi-source sensing data, and by incorporating a composite loss function and targeted data enhancement in model training, so as to realize high-precision and high-robustness recognition of dike vortices, especially small-scale and weak-intensity vortices that are prone to be missed, and the hazard level output by the model fuses physical signals, so that the recognition result has preliminary risk assessment capability, and provides a reliable technical basis for accurate early warning.

Description

Technical Field

[0001] This invention relates to the field of video analysis technology, and in particular to a video recognition algorithm based on a vortex video recognition model. Background Technology

[0002] As a core infrastructure of the flood control and disaster reduction system, the real-time monitoring of the seepage stability of the dikes in the middle and lower reaches of the Han River is crucial for their safe operation, while the eddies on the water surface around the dikes are the core hydrological indicators of concentrated seepage channels.

[0003] In existing levee engineering safety management, real-time monitoring of eddies is virtually nonexistent. Current management relies mainly on manual inspections, paper records, and telephone notifications, which are inefficient and severely lack informatization. Furthermore, the existing extensive video surveillance system along the levees suffers from extremely low video data utilization due to a lack of deep intelligent analysis, resulting in a huge consumption of storage resources. In addition, while the "Deep Learning-Based Method for Identifying Eddies in Front of Dams" (publication number CN113255448A) is designed for reservoir environments and has reference value, its computational environment is relatively simple compared to the complex levee environment. Therefore, the effectiveness and adaptability of its algorithm and model in practical applications of watershed levee management remain flawed, failing to solve the problem of accurate and robust eddy identification in complex open water environments. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies. By fusing video image features with multi-source sensor data such as hydrology, GNSS, and seepage pressure at the feature level, and incorporating composite loss functions and targeted data augmentation into the model training, the algorithm can effectively overcome interference from changes in lighting, water turbidity, and complex backgrounds in natural water bodies. This enables high-precision and robust identification of eddy eddies, especially small-scale, weak-intensity eddies that are easily missed. At the same time, the hazard level output by the model incorporates physical signals, giving the identification results a preliminary risk assessment capability and providing a reliable technical basis for accurate early warning.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a video recognition algorithm based on a vortex video recognition model, deployed as a backend service on the platform layer of a dike safety monitoring system, comprising the following steps:

[0006] S1. Video Data Acquisition and Processing: Acquire video monitoring data of the water body in front of the embankment, extract video frames at a fixed frame rate of 30 frames / second, and use the Canny edge detection algorithm and pre-trained model to filter and obtain a valid image dataset;

[0007] S2. Data annotation: A seven-level annotation system covering the location, type, scale, intensity, development stage, and environmental information of the vortex is used to annotate the effective image dataset in step S1.

[0008] S3. Model Construction: Based on the performance comparison results, YOLOv11x was selected as the base model, and the CBAM attention module and the Swing Transformer attention module were integrated into its C2f module, SPPF module and detection head to construct the initial recognition model.

[0009] S4. Model Training and Optimization: The initial recognition model is trained using the labeled dataset. During training, multi-scale image input, AdamW optimizer, and composite loss function are used. After training, the model is further optimized using a model ensemble strategy.

[0010] S5. Model Deployment and Application: Deploy the optimized model on the computing platform to perform inference and recognition on real-time video streams, and input the recognition results into the business management system.

[0011] In a preferred embodiment, in the seven-level labeling system of step S2, the vortex position is labeled with the smallest bounding rectangle with a coordinate accuracy of 1 pixel; the vortex type includes surface vortex, underwater vortex, strong vortex and weak vortex, wherein the rotation speed of strong vortex is not less than 1 m / s; the vortex scale is divided into small scale, medium scale and large scale according to the diameter.

[0012] In a preferred embodiment, in step S3, a multi-scale feature fusion network combining FPN-PAN and NAS-FPN structures is introduced into the feature fusion layer of the YOLOv11x model.

[0013] In a preferred embodiment, the composite loss function in step S4 is:

[0014]

[0015] in: , , The cross-entropy losses are calculated for the RGB, LAB, and LCH color spaces, respectively, with weighting coefficients of α=0.2; β=0.3; and γ=0.2.

[0016] CIoU loss is used to optimize the accuracy of bounding box regression, with a weighting coefficient δ=0.2;

[0017] To address multimodal loss, multimodal information such as hydrological data, GNSS data, and seepage pressure data is integrated, with weighting coefficients... =0.2;

[0018] L is the weighted sum, and the expression includes: cross-entropy loss components in RGB, LAB, and LCH color spaces, CIoU bounding box regression loss components, and multimodal loss components that integrate hydrological, GNSS, and seepage pressure data.

[0019] As a preferred implementation, step S4 employs water surface scene data enhancement, specifically including dynamic ripple overlay, highlight and shadow simulation, siltation simulation, random cropping, and extreme condition simulation.

[0020] In a preferred embodiment, the model integration strategy in step S4 is Bagging integration and Stacking integration, which are executed synchronously. Bagging integration is achieved by constructing multiple heterogeneous initial models and voting on them, while Stacking integration is achieved by cascading three different types of base models with a meta-model.

[0021] In a preferred embodiment, the dike safety monitoring system is functionally divided into a perception layer, a platform layer, and an application layer, wherein:

[0022] The sensing layer is used to collect video, hydrological, GNSS, and seepage pressure data;

[0023] The platform-layer integration algorithm performs data fusion and intelligent analysis;

[0024] The application layer is based on the output of the platform layer and is used for vortex identification, early warning, and operation and maintenance management functions.

[0025] In a preferred embodiment, the platform layer receives video and multi-source sensor data from the perception layer via a data transmission network, and the algorithm provides a unified AI analysis service to push the results to the application layer.

[0026] In a preferred embodiment, the business management system in step S5 is deployed at the application layer of the dike safety monitoring system. The business management system includes an early warning management module and a closed-loop response module. Specifically:

[0027] The early warning management module is used to configure and execute identification tasks and archive the results;

[0028] The closed-loop processing module is used to generate processing work orders based on the identification results and to track and record the entire process from dispatch to feedback.

[0029] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0030] 1. In this invention, by fusing video image features with multi-source sensor data such as hydrology, GNSS, and seepage pressure at the feature level, and incorporating composite loss functions and targeted data augmentation into the model training, the algorithm can effectively overcome the interference of light changes, water turbidity, and complex backgrounds in natural water bodies. This enables high-precision and robust identification of eddy eddies, especially small-scale and weak-intensity eddies that are easily missed. At the same time, the hazard level output by the model incorporates physical signals, giving the identification results a preliminary risk assessment capability and providing a reliable technical basis for accurate early warning.

[0031] 2. In this invention, through the seamless connection between the platform layer and the application layer, the structured early warning information identified by AI is automatically converted into standardized handling work orders in the business system, and the entire process of digital work order dispatch, on-site location check-in of handling personnel and multimedia feedback to final review and archiving is enforced online. This transforms the inefficient traditional model that relies on manual inspection and telephone notification into a fully automatic and traceable digital management model, thereby shortening the response time for safety hazards from hours to minutes, and greatly improving the timeliness, standardization and overall management efficiency of dike safety operation and maintenance. Attached Figure Description

[0032] Figure 1 The present invention proposes a framework diagram for a video recognition algorithm based on a vortex video recognition model;

[0033] Figure 2 This invention proposes a flowchart for constructing a video recognition algorithm based on a vortex video recognition model. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Example

[0036] like Figure 1 and Figure 2 As shown, this invention provides a technical solution: a video recognition algorithm based on a vortex video recognition model, deployed as a backend service on the platform layer of a dike safety monitoring system. The dike safety monitoring system is functionally divided into a perception layer, a platform layer, and an application layer. The platform layer receives video and multi-source sensor data from the perception layer via a data transmission network, and the algorithm provides a unified AI analysis service, pushing the results to the application layer. Specifically:

[0037] The sensing layer is used to collect video and hydrological, GNSS, and seepage pressure data. Specifically, it includes flow velocity and water depth measured by hydrological monitoring equipment, levee deformation data obtained by GNSS displacement monitoring stations, and seepage pressure data measured by piezometers.

[0038] The platform layer integrates algorithms for data fusion and intelligent analysis. Specifically, the data transmission network of the platform layer includes 5G and optical fiber. The platform layer performs feature-level fusion of video frame image features with multimodal sensor data such as hydrology, GNSS, and seepage pressure, thereby achieving accurate identification and risk assessment of eddies.

[0039] The application layer, based on the output of the platform layer, is used for vortex identification, early warning, and operation and maintenance management business functions. Specifically, the application layer drives the "vortex identification, early warning, and operation and maintenance management business functions" based on the identification results with multi-dimensional labels output by the platform layer. The application layer automatically connects the structured early warning information of AI to the business process engine, which is specifically implemented as the early warning management module automatically creates handling work orders, and the handling closed-loop module tracks the entire digital closed loop from dispatch, on-site check-in, situation reporting to administrator review and archiving.

[0040] In the above, the perception layer can simultaneously collect water body video and physical state data through deployed surveillance cameras, hydrometers, GNSS displacement monitoring stations, and piezometers, providing a multi-dimensional factual basis for analysis; the platform layer receives raw data of water body video and physical state data through 5G, fiber optic networks, etc., and achieves accurate detection of eddies and collaborative assessment of potential leakage risks through feature-level fusion technology; the application layer, based on the structured recognition results with multi-dimensional labels such as location, scale, intensity, and risk level output by the platform layer, drives its internal business processes. Through the early warning management module, AI events are automatically converted into executable disposal work orders, and the disposal closed-loop module digitally dispatches the work orders, mandates on-site geographical location check-in and multimedia feedback, and connects every link from reporting to administrator review and archiving, thereby solidifying the traditional loose management process that relies on manual transmission into a traceable and assessable online automated workflow;

[0041] This design significantly improves the accuracy of vortex identification and the rationality of risk assessment in complex natural environments through multimodal data fusion, effectively reducing false alarms and missed alarms from single visual analysis. At the same time, by connecting the entire chain of "perception-analysis-decision-action", it upgrades the original time-consuming and potentially delaying manual inspection, telephone notification, and paper record mode into a highly efficient management mode that can automatically identify vortex hazards in dikes in real time, conduct intelligent risk assessment, and immediately trigger standardized handling procedures. Thus, it achieves a comprehensive improvement in the pre-warning capability, in-event response speed, and post-event management traceability of dike safety monitoring.

[0042] The video recognition algorithm specifically includes the following steps:

[0043] S1. Video Data Acquisition and Processing: Video monitoring data of the water body in front of the embankment is acquired. The real-time data collected by the existing video monitoring system deployed in the scene system can be directly used to avoid the investment in building a new monitoring network. Video frames are extracted at a fixed frame rate of 30 frames / second. Based on a full consideration of the frequency of water vortex movement, the frame rate of 30 frames / second can effectively capture the dynamic formation and change process of the vortex, avoiding motion blur or loss of key frames due to too low a frame rate. The Canny edge detection algorithm and pre-trained model are used for screening. In the screening process, the Canny edge detection algorithm is first applied to calculate the sharpness value of each video frame, and blurry frames with a sharpness lower than 0.8 are removed. Then, the YOLO model pre-trained on a general large-scale dataset is used to perform a preliminary scan of the remaining frames, quickly filtering out a large number of invalid frames that do not contain any suspected vortex water body features, and obtaining an effective image dataset.

[0044] The above content avoids the capital investment of building new infrastructure by reusing the existing video surveillance system as the data source. Based on the physical characteristics of whirlpool water movement, video frames are extracted at a fixed frame rate of 30 frames / second, which can meet the basic requirements of the Nyquist sampling theorem and effectively cover the typical frequency components in the process of whirlpool rotation, generation and dissipation. This avoids motion blur or loss of key dynamic information due to undersampling in the time dimension. The screening process first uses the Canny edge detection algorithm to calculate the gradient magnitude of each video frame to quantify its sharpness. Frames with a sharpness value lower than the preset threshold of 0.8 are judged as blurry frames and removed. The integrity of image edge information is used as a proxy indicator of sharpness to quickly filter low-quality data caused by lens damage, bad weather or transmission errors. Then, a YOLO model pre-trained on a general large-scale dataset is used to perform forward inference on the remaining frames. The output of the pre-trained YOLO model can quickly filter out invalid frames that do not contain any moving water texture or whirlpool morphology features. The generalization feature extraction capability of the pre-trained model is used to perform an efficient negative sample coarse screening.

[0045] This design significantly improves the signal-to-noise ratio and quality of the raw data at the very beginning of the data input process, providing a clean, relevant and highly accurate "effective image dataset" for subsequent annotation and model training. This directly reduces the labor costs and computational resource waste in annotation work, and fundamentally lays a reliable data foundation for training a high-precision dedicated vortex recognition model. At the same time, the entire preprocessing process is highly automated and can be seamlessly integrated into existing video streaming pipelines, maximizing the value of data at low cost.

[0046] S2. Data Labeling: A seven-level labeling system covering the location, type, scale, intensity, development stage, and environmental information of the vortex is used to label the effective image dataset in step S1; wherein:

[0047] In the seven-level annotation system, the location of a vortex is marked with the smallest bounding rectangle, with a coordinate accuracy of 1 pixel; the vortex types include surface vortices, underwater vortices, strong vortices, and weak vortices, among which the rotation speed of strong vortices is not less than 1 m / s; the vortex scale is divided into small scale, medium scale, and large scale according to the diameter.

[0048] Specifically:

[0049] Vortex intensity can be quantified into 1 to 5 levels based on rotation speed and range of influence.

[0050] The development of a vortex can be divided into four stages: generation, development, maturity, and dissipation.

[0051] Environmental information can include water flow speed, water depth, light intensity, and background type;

[0052] The annotation process follows a quality control procedure of "three-person annotation + cross-review + expert review", and is assisted by Labelme semi-automation tools to ensure that the annotation accuracy is no less than 98% and the consistency is no less than 95%.

[0053] In this step, a refined annotation system that deeply integrates professional knowledge and far surpasses conventional target detection is constructed to provide the model with rich and structured supervision signals; the physical properties, dynamic processes and occurrence scenarios of vortices are defined, thereby forcing the model to establish a mapping relationship between the visual representation of vortices and their hydraulic meaning and engineering risks during the learning process;

[0054] The above content firstly describes the spatial positioning, using the minimum bounding rectangle and defining the vortex location with 1-pixel coordinate precision, providing the model with a high-precision regression learning target. In terms of physical attribute definition, vortex types are classified into surface vortices, underwater vortices, strong vortices, and weak vortices based on their visual visibility and hydraulic characteristics; scales are divided into small, medium, and large scales based on diameter; intensity is quantified into 1 to 5 levels based on rotation speed and influence range; and the dynamic process is divided into four stages: generation, development, maturation, and dissipation. Simultaneously, environmental information such as water flow velocity, water depth, light intensity, and background type are associated with the annotations, providing the model with multi-dimensional, structured truth labels from geometric morphology to physical attributes and evolutionary processes. The annotation process follows a "three-person annotation + cross-verification + expert review" workflow, supplemented by Labelme design. Individual errors are reduced through independent work and cross-validation by multiple people, and final arbitration and correction are achieved through expert knowledge. This systematically controls subjectivity and inconsistency in the annotation process, ensuring high accuracy and consistency in the output annotations.

[0055] This design generates a dedicated dataset for subsequent model training that is of controlled quality and has a much higher information dimension than conventional target detection tasks. This dataset not only indicates the location of vortices, but also clarifies the nature, stage, and environment of the vortices. This forces the model to establish a complex mapping relationship between visual features and rich hydraulic and engineering semantics during the training process, directly driving the model to learn more discriminative and generalizable feature representations. This provides a crucial foundation of supervisory signals for achieving high-precision identification and preliminary risk assessment in real-world scenarios.

[0056] S3. Model Construction: Based on the performance comparison results of a systematic evaluation of Faster R-CNN, Mask R-CNN, Cascade R-CNN, EfficientDet-D4, EfficientDet-D5, YOLOv12x, DETR, ViTDet, and Swin Transformer Detection with metrics of mAP50≥0.98, small target detection mAP50≥0.95, complex background mAP50≥0.97, and inference speed≥10 FPS, YOLOv11x, which achieved the best balance between comprehensive accuracy and real-time performance, was selected as the base model. The CBAM attention module and the Swin Transformer attention module were then integrated into its C2f module, SPPF module, and detection head to construct the initial recognition model.

[0057] Among them, the CBAM module enables the model to focus on key channels and image regions related to vortex features through serial channel attention and spatial attention sub-modules, while the Swin Transformer module establishes long-range dependencies between vortex features and global context in a computationally efficient framework through window partitioning and shifting window multi-head self-attention mechanism. The parallel fusion of serial channel attention and spatial attention sub-modules and Swin Transformer module can achieve synergy between local focus and global understanding.

[0058] Furthermore, a multi-scale feature fusion network combining FPN-PAN and NAS-FPN structures is introduced into the feature fusion layer of the YOLOv11x model. The FPN-PAN structure can fully fuse deep semantic features and shallow detail features through bidirectional paths from top to bottom and bottom to top, while NAS-FPN can automatically learn and optimize the fusion path and connection mode between feature maps of different scales by using neural architecture search technology.

[0059] In the above content, the channel attention and spatial attention sub-modules of the CBAM module are used to enable the network to adaptively calibrate the channel weights and focus on the key spatial regions of the vortex. At the same time, the window partitioning and shift window multi-head self-attention mechanism of the Swing Transformer module is used to establish long-range dependencies between the local features of the vortex and the global context of the image under controllable computational burden. The collaborative design realizes multi-level feature understanding from local details to the global scene. Furthermore, FPN-PAN integrates deep features containing rich semantics and shallow features containing precise localization information through bidirectional paths from top to bottom and bottom to top. NAS-FPN automatically learns and optimizes the cross-scale connection and fusion method between feature maps of different scales through neural architecture search, thereby constructing an enhanced multi-scale feature architecture that can efficiently represent vortices with diameters ranging from tens of centimeters to several meters.

[0060] S4. Model Training and Optimization: The initial recognition model is trained using the labeled dataset. During training, multi-scale image input, AdamW optimizer, and composite loss function are used. After training, the model is further optimized using a model ensemble strategy.

[0061] Specifically:

[0062] During training, the size of the input image is randomly switched between four preset scales: 640×640, 800×800, 1024×1024, and 1280×1280. By forcing the model to learn the representation of the same vortex target at different resolutions, the scale invariance and generalization ability of the model to vortices of different distances and sizes are improved.

[0063] The AdamW optimizer is specifically designed with an initial learning rate of 0.001, dynamically adjusted using a cosine annealing strategy, a total of 1000 training epochs, and an early stopping strategy that stops the model if the validation set loss does not decrease for 30 consecutive epochs. Combined with adaptive moment estimation and decoupled weight decay, it can accelerate convergence while effectively controlling model complexity to prevent overfitting.

[0064] The composite loss function constrains the discrimination of vortex categories under different color representations, enabling the model to learn more essential vortex texture and morphological features that are independent of color.

[0065] The composite loss function is:

[0066]

[0067] in: , , The cross-entropy losses are calculated for the RGB, LAB, and LCH color spaces, respectively, with weighting coefficients of α=0.2; β=0.3; and γ=0.2.

[0068] CIoU loss is used to optimize the accuracy of bounding box regression. It directly optimizes the overlap area, center point distance, and aspect ratio between the predicted bounding box and the labeled box, with a weighting coefficient δ=0.2.

[0069] For multimodal loss, multimodal information is embedded at the loss function level to guide the model in establishing a correlation mapping between visual vortex features and physical sensor signals. This method integrates multimodal information such as hydrological data, GNSS data, and seepage pressure data. A sub-network aligns and compares these synchronously collected hydrological, GNSS, and seepage pressure data with corresponding image features, and assigns weight coefficients accordingly. =0.2;

[0070] L is the weighted sum, and the expression includes: cross-entropy loss components in RGB, LAB, and LCH color spaces, CIoU bounding box regression loss components, and multimodal loss components that integrate hydrological, GNSS, and seepage pressure data.

[0071] Furthermore, water surface scene data augmentation is employed, specifically including dynamic ripple overlay based on fluid dynamics models to generate dynamic ripples and superimpose them onto the image; specular shadow simulation by randomly adding 1 to 3 highlight areas and 1 to 2 shadow areas to the image to change the lighting conditions; turbidity simulation of water bodies with high sediment content by adjusting image saturation and contrast; random cropping of images at a ratio of 0.1 to 0.5 to construct small target samples at a distance; and extreme working condition simulation of synthetic data simulating extreme flow velocity, extreme water depth, and extreme lighting. By expanding the diversity of training data through algorithms, the model covers weather, water quality, lighting, and observation conditions that are difficult to exhaust with a single real dataset, thereby directly improving the robustness of the model in complex real-world environments.

[0072] Furthermore, the model ensemble strategy employs both Bagging and Stacking ensembles, which are executed simultaneously. By constructing and combining multiple models with inherent differences, the variance or bias that may exist in a single model is reduced, thereby obtaining a more stable and accurate overall prediction. Specifically:

[0073] The Bagging ensemble is achieved by training 10 YOLOv11x models with different random initial weights and summing the detection results of all YOLOv11x models with different random initial weights through majority voting.

[0074] The Stacking integration is achieved by cascading three different types of base models with a meta-model. YOLOv11x, EfficientDet-D5 and ViTDet are three heterogeneous models as base models. The prediction outputs of the three heterogeneous models on the training set samples are used as new features. Then, a logistic regression model is input as a meta-model for training and final decision-making.

[0075] In the above content, at the training data level, multi-scale image input is used to adapt the model to the appearance of targets at different resolutions, enhancing scale invariance. Water surface scene data augmentation, including dynamic ripple overlay, specular highlight and shadow simulation, siltation simulation, random cropping, and extreme condition simulation, is applied to expand the diversity of training samples through algorithmic means. This covers complex situations in the real world caused by changes in weather, lighting, water quality, and viewing angle, directly improving the model's generalization ability and robustness in practical applications. Furthermore, the AdamW optimizer is used with an initial learning rate of 0.001, dynamically adjusted using a cosine annealing strategy, supplemented by an early stopping mechanism. Adaptive moment estimation optimizes the convergence path, and decoupled weight decay controls model complexity, effectively suppressing overfitting while achieving efficient training. The composite loss function L is a weighted sum of multiple sub-losses, where L... RGB L LAB L LCH The cross-entropy loss of the three components is calculated in the RGB, LAB, and LCH color spaces respectively. By constraining the classification under different color representations, the model is driven to remove irrelevant color interference such as lighting and water quality, and learn the more essential texture and morphological features of the vortex; L CIoU The loss function directly optimizes the regression accuracy of the predicted bounding box by simultaneously considering the overlap area, center point distance, and aspect ratio; L MultiModal Multimodal loss uses a sub-network to align and compare synchronously acquired hydrological, GNSS, and seepage pressure data with image features. At the loss function level, it forces the model to establish a correlation mapping between visual features and physical sensor signals, thereby embedding hydrodynamic and engineering mechanics knowledge into the model. After training, an integration strategy of simultaneous Bagging and Stacking is further adopted to integrate the advantages of each base model through meta-learning and reduce prediction bias.

[0076] The resulting vortex recognition model can achieve high-precision results. It not only has strong adaptability to changes in target scale and appearance, but also integrates multimodal information for decision-making. This significantly improves the accuracy of recognition and the rationality of risk assessment in complex natural environments. Furthermore, the integration strategy further ensures the stability and reliability of the prediction results, providing a high-performance AI model for subsequent real-time deployment and business applications.

[0077] S5. Model Deployment and Application: Deploy the optimized model on the computing platform to perform inference and recognition on real-time video streams, and input the recognition results into the business management system.

[0078] Specifically:

[0079] The trained and integrated optimized vortex recognition model is converted into ONNX format, and layer fusion, kernel automatic tuning and low-precision quantization optimization are performed using the TensorRT 8.6.0 inference engine. It is then deployed on a server cluster. The deployment strategy can adopt a combination of data parallelism, model parallelism and tensor parallelism.

[0080] The server cluster accesses the real-time bitstream of the existing video surveillance network of the dike through standard protocols, and calls the optimized model to perform frame-by-frame or frame-by-frame analysis, outputting structured data containing vortex bounding boxes, category labels, confidence levels and associated multimodal hazard levels;

[0081] The AI ​​service at the platform layer pushes structured recognition events to the corresponding business system database or event bus at the application layer through application programming interfaces / message queues;

[0082] in:

[0083] The business management system is deployed at the application layer of the dike safety monitoring system. The business management system includes an early warning management module and a closed-loop response module, specifically:

[0084] The early warning management module is used to configure and execute recognition tasks and archive the results. The early warning management module provides a graphical interface that allows managers to formulate periodic / manually triggered AI recognition tasks. The task configuration content may include selecting specific video surveillance sites, setting the start and stop time and cycle of video analysis, and associating the corresponding AI model version. After the task is executed, the system automatically schedules computing resources and stores the recognition results together with the corresponding video segments and metadata in the database to form a queryable archive record.

[0085] The closed-loop disposal module is used to generate disposal work orders based on the identification results and track and record the entire process from dispatch to feedback. Specifically, when the early warning management module generates an identification result at the early warning level, the closed-loop disposal module will automatically create a disposal work order. The work order contains early warning information and is dispatched to the designated in-house disposal personnel or third-party teams according to preset rules. After receiving the task, the disposal personnel will check in at the designated site and upload pictures, videos, or text descriptions of the site situation via mobile terminal. Then, the system records the disposal measures and conclusions reported by the disposal personnel and transfers them to the administrator for review and closed-loop archiving, thereby completely recording the status and operation logs of each link in "dispatch-check-in-on-site disposal-reporting-review-archiving".

[0086] In this embodiment, the existing video surveillance system is directly used to collect water body videos at 30 frames per second. After the videos are filtered by clarity screening and pre-trained model, high-quality image data is obtained. Then, a refined seven-level annotation system that integrates water conservancy professional knowledge is used to annotate the data, providing structured supervision signals for model training. The model is built based on YOLOv11x with performance comparison and optimization, and is enhanced by integrating a dual attention mechanism and a combined multi-scale feature network.

[0087] During training, multi-scale input, composite loss function, targeted data augmentation and model ensemble strategies are used to drive the model to learn the essential characteristics of vortices and their correlation with multimodal physical signals. The optimized model is deployed on the computing platform through a high-performance inference engine to analyze real-time video streams and output structured results with risk assessment.

[0088] This invention receives and integrates multi-source data from the perception layer, automatically triggering a closed loop of early warning management and digital handling in the application layer business system. It achieves high-precision automatic identification of vortex hazards, intelligent risk assessment, and rapid handling through standardized processes, transforming the traditional manual mode into an efficient and traceable intelligent management system, and comprehensively improving the level of dike safety operation and maintenance.

[0089] Working principle:

[0090] like Figure 1 and Figure 2 As shown, during operation, the monitoring cameras, hydrometers, GNSS displacement monitoring stations, and piezometers deployed in the perception layer simultaneously collect video, flow velocity, water depth, dike deformation, and seepage pressure data of the water body in front of the dam. The platform layer receives this multi-source data through the data transmission network and initiates the video data acquisition and processing steps. Frames are extracted from the video stream at 30 frames per second. After screening by Canny edge detection and a pre-trained model, a valid image dataset is obtained. Then, a seven-level annotation system is used to provide an interpretation benchmark for the real-time data. Finally, the vortex recognition model performs feature analysis on the converged video and multimodal sensor data at the platform layer. Through multi-level fusion and real-time reasoning analysis, accurate identification and risk assessment are achieved. Then, the platform layer pushes the structured identification results with labels of location, scale, intensity and risk level to the application layer. The early warning management module of the application layer then automatically creates and archives early warning tasks based on the results. At the same time, the closed-loop handling module automatically generates handling work orders based on the early warning level, digitally dispatches them to designated personnel, and strictly tracks and records the entire process of "dispatch-sign-handling-reporting-review-archiving" from dispatch, on-site check-in, situation reporting to administrator review and archiving, thereby realizing complete automated closed-loop management from intelligent perception to business handling.

[0091] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments for application in other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A video recognition algorithm based on a vortex video recognition model, deployed as a backend service on the platform layer of a dike safety monitoring system, characterized in that: Includes the following steps: S1. Video Data Acquisition and Processing: Acquire video monitoring data of the water body in front of the embankment, extract video frames at a fixed frame rate of 30 frames / second, and use the Canny edge detection algorithm and pre-trained model to filter and obtain a valid image dataset; S2. Data annotation: A seven-level annotation system covering the location, type, scale, intensity, development stage, and environmental information of the vortex is used to annotate the effective image dataset in step S1. S3. Model Construction: Based on the performance comparison results, YOLOv11x was selected as the base model, and the CBAM attention module and the Swing Transformer attention module were integrated into its C2f module, SPPF module and detection head to construct the initial recognition model. S4. Model Training and Optimization: The initial recognition model is trained using the labeled dataset. During training, multi-scale image input, AdamW optimizer, and composite loss function are used. After training, the model is further optimized using a model ensemble strategy. S5. Model Deployment and Application: Deploy the optimized model on the computing platform to perform inference and recognition on real-time video streams, and input the recognition results into the business management system.

2. The video recognition algorithm based on a vortex video recognition model according to claim 1, characterized in that: In the seven-level annotation system of step S2, the vortex position is marked with the smallest bounding rectangle and the coordinate accuracy is 1 pixel; the vortex type includes surface vortex, underwater vortex, strong vortex and weak vortex, among which the rotation speed of strong vortex is not less than 1 m / s; the vortex scale is divided into small scale, medium scale and large scale according to the diameter.

3. The video recognition algorithm based on a vortex video recognition model according to claim 1, characterized in that: In step S3, a multi-scale feature fusion network combining FPN-PAN and NAS-FPN structures is introduced into the feature fusion layer of the YOLOv11x model.

4. The video recognition algorithm based on a vortex video recognition model according to claim 1, characterized in that: The composite loss function in step S4 is: in: , , The cross-entropy losses are calculated for the RGB, LAB, and LCH color spaces, respectively, with weighting coefficients of α=0.2; β=0.3; and γ=0.

2. CIoU loss is used to optimize the accuracy of bounding box regression, with a weighting coefficient δ=0.2; To address multimodal loss, multimodal information such as hydrological data, GNSS data, and seepage pressure data is integrated, with weighting coefficients... =0.2; L is the weighted sum, and the expression includes: cross-entropy loss components in RGB, LAB, and LCH color spaces, CIoU bounding box regression loss components, and multimodal loss components that integrate hydrological, GNSS, and seepage pressure data.

5. The video recognition algorithm based on a vortex video recognition model according to claim 1, characterized in that: In step S4, water surface scene data enhancement is employed, specifically including dynamic ripple overlay, specular shadow simulation, siltation simulation, random cropping, and extreme working condition simulation.

6. The video recognition algorithm based on a vortex video recognition model according to claim 1, characterized in that: The model integration strategy in step S4 is Bagging integration and Stacking integration. Bagging integration and Stacking integration are executed synchronously. Bagging integration is achieved by constructing multiple heterogeneous initial models and voting on them. Stacking integration is achieved by cascading three different types of base models with a meta-model.

7. The video recognition algorithm based on a vortex video recognition model according to claim 1, characterized in that: The dike safety monitoring system is functionally divided into a perception layer, a platform layer, and an application layer, wherein: The sensing layer is used to collect video, hydrological, GNSS, and seepage pressure data; The platform-layer integration algorithm performs data fusion and intelligent analysis; The application layer is based on the output of the platform layer and is used for vortex identification, early warning, and operation and maintenance management functions.

8. The video recognition algorithm based on a vortex video recognition model according to claim 7, characterized in that: The platform layer receives video and multi-source sensor data from the perception layer through a data transmission network, and the algorithm provides a unified AI analysis service to push the results to the application layer.

9. A video recognition algorithm based on a vortex video recognition model according to claim 7, characterized in that: The business management system in step S5 is deployed at the application layer of the dike safety monitoring system. The business management system includes an early warning management module and a closed-loop response module, specifically: The early warning management module is used to configure and execute identification tasks and archive the results; The closed-loop processing module is used to generate processing work orders based on the identification results and to track and record the entire process from dispatch to feedback.

Images