Water area ship intelligent monitoring system and method based on video analysis

Abstract

The invention discloses a water area ship intelligent monitoring system and method based on video analysis, the output end of a sensing layer is in communication connection with the input end of an analysis layer, the analysis layer is in bidirectional communication connection with a fusion layer, and the output end of the fusion layer is in communication connection with the input end of a decision-making layer and the input end of a storage module; the decision-making layer is in two-way communication connection with the interaction layer, and the analysis layer and the decision-making layer are respectively in communication connection with the storage module; the sensing layer is used for collecting water area video data and ship and environment related auxiliary data; and the analysis layer is used for performing target detection, feature extraction and dynamic tracking on the video data, and outputting a ship identification result and trajectory data. Through a multi-source data acquisition and fusion mechanism, multiple types of data such as videos, ship states, environments and identities are integrated, data islands are broken, ship full-dimension monitoring is achieved, and the multi-scene anomaly judgment requirement which is not involved in the prior art is met.

Description

Technical Field

[0001] This invention belongs to the field of waterway traffic monitoring and intelligent ship management technology, specifically relating to a smart waterway ship monitoring system and method based on video analysis. Background Technology

[0002] With the rapid development of the shipping industry, the density of vessels navigating waterways is continuously increasing, and the types of vessels are becoming increasingly complex. This necessitates ever-increasing demands for intelligent, precise, and efficient waterway traffic monitoring. Traditional waterway vessel monitoring mainly relies on manual screen monitoring, single-device data collection, or simple video analysis, which is no longer sufficient to meet the management needs of modern smart shipping. Existing technologies have many shortcomings that urgently need to be addressed: At the monitoring and perception level, existing systems mostly employ single-type video acquisition equipment (such as fixed-focus bullet cameras and zoom PTZ cameras), which can only cover a limited monitoring range and have poor adaptability to long-distance and complex aquatic environments. Although some systems have introduced video recognition technology, they mostly use traditional Gaussian mixture models, which are prone to low recognition accuracy and target loss when faced with scenarios such as water surface fluctuations, changes in lighting, and ship obstruction. Furthermore, the filtering effect of non-ship pixels is poor, consuming a large amount of computing resources. At the same time, existing technologies generally suffer from data silos. Video data lacks an effective integration mechanism with AIS (Automatic Identification System) data, meteorological and hydrological data, and ship equipment status data, resulting in a single monitoring dimension and an inability to comprehensively reflect the ship's operational status.

[0003] At the intelligent analysis level, existing multi-target ship tracking methods suffer from problems such as insufficient adaptation to differences in imaging area and trajectory breakage caused by occlusion, making it difficult to achieve stable tracking in complex navigation scenarios. Anomaly detection is mostly limited to single dimensions such as collision risk and channel deviation, failing to cover multi-dimensional anomaly scenarios such as ship equipment failure, crew violations, and environmental adaptability. Furthermore, warning thresholds are fixed and cannot be dynamically adjusted according to water type and ship type. Fault diagnosis relies heavily on single parameter threshold judgments, lacking correlation analysis based on video features and equipment data, resulting in delayed fault identification and a high false alarm rate.

[0004] At the collaborative response level, existing systems primarily rely on single audible and visual alarms for early warning, lacking tiered early warning mechanisms and ship-shore collaborative response channels. This results in untimely transmission of early warning information, slow response times, and an inability to provide regulatory authorities with complete evidence collection data and remote guidance support. Furthermore, the systems suffer from poor compatibility and scalability, making them difficult to adapt to equipment from different manufacturers and various water environments, leading to high upgrade and modification costs. In addition, existing technologies have shortcomings in energy consumption management and environmental adaptability. The video acquisition and analysis modules operate under continuous high loads, resulting in excessive energy consumption; and under severe weather conditions such as strong winds, heavy rain, and dense fog, monitoring accuracy drops significantly, making it impossible to guarantee stable operation around the clock.

[0005] To address the shortcomings of the existing technologies, this invention proposes a video analytics-based intelligent monitoring system and method for waterway vessels. Through multi-source perception fusion, deep learning optimization, multi-dimensional anomaly detection, and hierarchical collaborative handling, it achieves precise, intelligent, and full-process monitoring of waterway vessels, providing technical support for smart shipping. Summary of the Invention

[0006] In response to the problems raised in the background technology above, the purpose of this invention is to provide a smart monitoring system and method for waterway vessels based on video analysis. Through multi-source perception fusion, deep learning optimization, multi-dimensional anomaly judgment and hierarchical collaborative handling, it can achieve precise, intelligent and full-process monitoring of waterway vessels, and provide technical support for smart shipping.

[0007] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows: A smart monitoring system for vessels in waterways based on video analytics includes a perception layer, an analysis layer, a fusion layer, a decision-making layer, an interaction layer, and a storage module. The output of the perception layer is communicatively connected to the input of the analysis layer; the analysis layer is bidirectionally communicatively connected to the fusion layer; the output of the fusion layer is communicatively connected to the input of the decision layer and the input of the storage module, respectively; the decision layer is bidirectionally communicatively connected to the interaction layer; and the analysis layer and the decision layer are communicatively connected to the storage module, respectively. The perception layer is used to collect video data of the water area and auxiliary data related to ships and the environment; the analysis layer is used to perform target detection, feature extraction and dynamic tracking on the video data, and output ship identification results and trajectory data; the fusion layer is used to integrate the data output by the analysis layer and the auxiliary data collected by the perception layer to construct a multi-dimensional monitoring dataset; the decision layer is used to determine the abnormal state of ships based on the monitoring dataset, generate graded early warning signals and trigger disposal instructions; the interaction layer is used to realize data interaction, collaborative disposal and information feedback between the ship and the shore; the storage module is used to store the raw monitoring data collected by the perception layer, the analysis results output by the analysis layer, the monitoring dataset constructed by the fusion layer, and the early warning records and disposal instruction logs generated by the decision layer.

[0008] Further specifying, the perception layer includes a video acquisition unit and an auxiliary data acquisition unit; The video acquisition unit employs distributed, multi-type video acquisition devices to cover video data acquisition under different water areas and environmental conditions. The auxiliary data acquisition unit is used to collect ship operation status data, water environment data, and ship identification data. The ship operation status data includes ship power system parameters and hull attitude parameters. The water environment data includes meteorological and hydrological parameters. The ship identification data is used to associate basic ship information.

[0009] Further specifying, the analysis layer includes a video preprocessing unit, a target recognition unit, a trajectory tracking unit, and a feature extraction unit; The video preprocessing unit is used to perform noise filtering, background modeling, and foreground segmentation on the acquired video data, and output the enhanced video data; the target recognition unit uses a deep learning model to achieve accurate identification and classification of ship targets, and outputs ship type and shape feature information; the trajectory tracking unit is used to continuously track the identified ship targets and generate time-series trajectory data; the feature extraction unit is used to extract ship navigation status features and abnormal behavior features from the recognition results and trajectory data.

[0010] Further specifying, the fusion layer includes a data standardization unit and an association integration unit; The data standardization unit is used to convert heterogeneous data output from the perception layer and the analysis layer into a unified format, eliminating data dimensional differences; the association and integration unit establishes mapping relationships between different types of data through time alignment, spatial matching and feature association, eliminates redundant and conflicting data, and constructs a structured multi-dimensional monitoring dataset.

[0011] Furthermore, the decision-making layer includes an anomaly detection unit and an early warning and handling unit; The anomaly determination unit has a built-in multi-dimensional anomaly determination rule library, which makes a comprehensive determination based on the monitoring dataset from four dimensions: navigation status, equipment status, personnel behavior, and environmental adaptation. The early warning and handling unit generates graded early warning signals according to the severity of the anomaly and triggers corresponding handling measures, including ship-side warnings, shore-side alarms, and collaborative handling instructions.

[0012] Further specifying, the interaction layer includes a communication unit and a coordination unit; The communication unit supports multi-mode data transmission, and the coordination unit is used to realize two-way information interaction, remote guidance and progress feedback between the ship and the shore.

[0013] A smart monitoring method for vessels in waterways based on video analytics includes the following steps: S1: Collect water area video data, ship operation status data, water area environment data, and ship identification data through the perception layer; S2: Perform noise filtering, background modeling, and foreground segmentation on the acquired video data, and output the enhanced video data; S3: Based on a deep learning model, ship target recognition and classification are performed on the preprocessed video data. Continuous ship trajectory data is generated through a multi-target tracking algorithm, and ship feature information is extracted simultaneously. S4: Standardize and integrate the ship identification results, trajectory data, ship operation status data, water environment data, and ship identity data to construct a multi-dimensional monitoring dataset; S5: Based on a pre-set anomaly judgment rule base, it comprehensively analyzes the monitoring dataset from four dimensions: navigation status, equipment status, personnel behavior, and environmental adaptation, and determines the anomaly status and severity. S6: Generate graded early warning signals based on the anomaly determination results, trigger corresponding handling measures at the ship and shore ends, and achieve coordinated handling and progress feedback through ship-shore interaction; S7: The storage module stores the raw monitoring data of the perception layer, the analysis results of the analysis layer, the monitoring dataset of the fusion layer, and the early warning records and handling logs of the decision layer, supporting historical data backtracking and statistical analysis.

[0014] Further specifying, in S3, the ship identification process includes multi-scale feature extraction, pixel-level target segmentation and type matching; the trajectory tracking process uses a target association algorithm to solve the trajectory loss problem caused by ship occlusion and changes in imaging area, and generates complete ship dynamic trajectory data.

[0015] Furthermore, in S5, the anomaly judgment rule base supports dynamic updates, and the judgment threshold and associated conditions can be adjusted according to the water type, vessel type, and management needs; navigation status anomalies are judged based on the comparison results of the vessel's trajectory, speed, and preset waterways and speed limit standards; equipment status anomalies are judged based on the deviation of the vessel's operating parameters from the normal operating range; personnel behavior anomalies are judged based on the crew's operation and wearing standards identified by video analysis; and environmental adaptation anomalies are judged based on the matching degree between the current environmental parameters and the vessel's seaworthiness range.

[0016] Further defining the tiered early warning system, it includes three levels: emergency warning, important warning, and general warning. Different levels correspond to different warning methods and handling priorities. The collaborative handling includes automatically locking abnormal targets, saving evidence data, generating handling suggestions, and ship-to-shore two-way communication guidance measures to ensure that anomalies are handled quickly and effectively.

[0017] The beneficial effects of this invention are: This invention integrates multiple types of data, such as video, ship status, environment, and identification, through a multi-source data acquisition and fusion mechanism, breaking down data silos and realizing full-dimensional ship monitoring, covering multiple scenarios of anomaly detection needs not covered by existing technologies.

[0018] By adopting generalized deep learning algorithms and video preprocessing techniques, and adapting to complex scenarios such as water surface fluctuations, lighting changes, and ship occlusion, the stability of ship identification and trajectory tracking is significantly improved, making it suitable for different water areas and environmental conditions.

[0019] This invention shortens anomaly response and handling time by establishing a tiered early warning mechanism and a ship-shore collaborative response channel, significantly improving response efficiency compared to existing technologies and saving valuable time for accident prevention. By replacing manual monitoring with automated monitoring and intelligent analysis, manual intervention is greatly reduced. Simultaneously, ship-shore collaboration and multi-department data sharing mechanisms lower coordination costs, significantly improving the intelligence level and efficiency of waterway traffic management. The standardized design of the storage module enables centralized storage of all-process data, and clear data interaction relationships ensure the integrity and timeliness of stored data, providing solid data support for subsequent law enforcement evidence collection, algorithm optimization, and management decision-making. Attached Figure Description

[0020] The present invention can be further illustrated by the non-limiting embodiments given in the accompanying drawings; Figure 1 This is a system module connection diagram of an embodiment of a video analysis-based intelligent monitoring system and method for vessels in waterways according to the present invention. Figure 2 This is a flowchart illustrating the steps of an embodiment of a video analytics-based intelligent monitoring system and method for vessels in waterways according to the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be further described below with reference to the accompanying drawings and embodiments. The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. The technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention. It should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0022] like Figure 1 As shown, the present invention provides a video analytics-based intelligent monitoring system for vessels in waterways, comprising a perception layer, an analysis layer, a fusion layer, a decision-making layer, an interaction layer, and a storage module. The output of the perception layer is communicatively connected to the input of the analysis layer; the analysis layer is bidirectionally communicatively connected to the fusion layer; the output of the fusion layer is communicatively connected to the input of the decision layer and the input of the storage module, respectively; the decision layer is bidirectionally communicatively connected to the interaction layer; and the analysis layer and the decision layer are communicatively connected to the storage module, respectively. The perception layer is used to collect video data of the water area and auxiliary data related to ships and the environment; the analysis layer is used to perform target detection, feature extraction and dynamic tracking on the video data, and output ship identification results and trajectory data; the fusion layer is used to integrate the data output by the analysis layer and the auxiliary data collected by the perception layer to construct a multi-dimensional monitoring dataset; the decision layer is used to determine the abnormal state of ships based on the monitoring dataset, generate graded early warning signals and trigger disposal instructions; the interaction layer is used to realize data interaction, collaborative disposal and information feedback between the ship and the shore; the storage module is used to store the raw monitoring data collected by the perception layer, the analysis results output by the analysis layer, the monitoring dataset constructed by the fusion layer, and the early warning records and disposal instruction logs generated by the decision layer.

[0023] The core working principle of this invention is as follows: The perception layer continuously collects various types of data, such as waterway video, ship status, environment, and ship identity, through various standardized acquisition devices, and transmits them in real time to the analysis layer and local storage; the analysis layer preprocesses the video data, completes ship identification and feature extraction through a deep learning model, generates continuous trajectory data using a multi-target tracking algorithm, and transmits the analysis results synchronously to the fusion layer and storage module; the fusion layer standardizes and integrates the video analysis results with other multi-source data to construct a multi-dimensional monitoring dataset, which is transmitted to the decision layer and stored in cloud storage; the decision layer performs multi-dimensional anomaly judgment based on the monitoring dataset and a preset rule base, generates graded early warning signals according to the severity of the anomalies, triggers corresponding response measures, and stores early warning records and response instructions synchronously; the interaction layer realizes ship-to-shore transmission of early warning information and response instructions through dual-mode communication, supports collaborative response and progress feedback between the shore and the ship, and stores feedback information in the storage module; the storage module centrally stores the entire process data, supports subsequent backtracking and analysis, and provides data support for management decisions and algorithm optimization.

[0024] In the practical application of this embodiment, the perception layer includes a video acquisition unit and an auxiliary data acquisition unit; The video acquisition unit employs distributed, multi-type video acquisition devices to cover video data acquisition under different water areas and environmental conditions. The auxiliary data acquisition unit is used to collect ship operation status data, water environment data, and ship identification data. The ship operation status data includes ship power system parameters and hull attitude parameters. The water environment data includes meteorological and hydrological parameters. The ship identification data is used to associate basic ship information.

[0025] Specifically, the perception layer forms the foundation of the system's data acquisition. Through diverse and highly reliable acquisition equipment, it achieves comprehensive coverage of monitoring data, providing raw data support for subsequent analysis and decision-making. The video acquisition unit employs a distributed deployment of visible light / infrared dual-spectrum integrated cameras (model: Hikvision DS-2TD6215-635), featuring electric zoom, gimbal rotation, and defogging wipers. It is adaptable to complex environments such as day and night, rain, snow, and fog, ensuring clear video data acquisition across different water areas. The ship operation status sensing subunit collects real-time parameters of the ship's core equipment through high-precision sensors, directly reflecting the ship's operational health. The speed sensor utilizes the Hall effect principle, converting the number of gear rotations into a speed signal, while the oil pressure sensor uses a diffused silicon piezoresistive principle, converting pressure signals into voltage signals. The water environment sensing subunit is deployed at monitoring stations on both banks of the waterway, comprehensively collecting meteorological and hydrological parameters affecting ship navigation, providing a basis for environmental adaptability assessment. The ship identification receiving subunit connects to the AIS shore-based network to achieve accurate ship identification, avoiding misidentification. All devices in the sensing layer have an IP67 or higher protection rating, enabling them to work stably in harsh aquatic environments. The accuracy and frequency of data acquisition meet the needs of subsequent analysis.

[0026] In practical applications of this embodiment, the analysis layer includes a video preprocessing unit, a target recognition unit, a trajectory tracking unit, and a feature extraction unit; The video preprocessing unit is used to perform noise filtering, background modeling, and foreground segmentation on the acquired video data, and output the enhanced video data; the target recognition unit uses a deep learning model to achieve accurate identification and classification of ship targets, and outputs ship type and shape feature information; the trajectory tracking unit is used to continuously track the identified ship targets and generate time-series trajectory data; the feature extraction unit is used to extract ship navigation status features and abnormal behavior features from the recognition results and trajectory data.

[0027] Specifically, the analysis layer is the core processing unit of the system. Based on the edge computing module and optimized deep learning algorithms, it realizes intelligent analysis and feature extraction of video data. The video preprocessing unit, addressing the unique characteristics of waterborne videos, employs an improved GMM algorithm to adapt to the background of water surface ripples, combined with wavelet denoising algorithms to effectively filter various types of noise, significantly improving video frame quality. The target recognition unit integrates FPN and Transformer attention mechanisms to solve the problems of multi-scale imaging of ships and background interference, combining U-Net semantic segmentation to achieve pixel-level accurate segmentation of ships, and using feature library matching to complete the identification of ship type and key parameters. The trajectory tracking unit uses an algorithm combining FairMOT and BYTE data association to solve the problem of trajectory loss caused by ship occlusion and scaling in complex navigation scenarios, ensuring trajectory continuity and accuracy. The feature extraction unit integrates statistical features and deep learning features to accurately extract features related to abnormal ship behavior, providing strong support for subsequent anomaly detection. The edge computing module used in the analysis layer has powerful computing capabilities, enabling real-time data processing and meeting the system's low-latency requirements.

[0028] In practical applications of this embodiment, the fusion layer includes a data standardization unit and an association integration unit; The data standardization unit is used to convert heterogeneous data output from the perception layer and the analysis layer into a unified format, eliminating data dimensional differences; the association and integration unit establishes mapping relationships between different types of data through time alignment, spatial matching and feature association, eliminates redundant and conflicting data, and constructs a structured multi-dimensional monitoring dataset.

[0029] Specifically, the fusion layer integrates and optimizes multi-source heterogeneous data, providing a unified and reliable data foundation for decision-making. The data standardization unit employs the Z-Score standardization method to convert monitoring data of different magnitudes and units into a unified format, eliminating data dimensional differences and ensuring data comparability and the applicability of subsequent algorithms. The association and integration unit establishes precise mapping relationships between different types of data through a four-step process: time alignment, spatial matching, feature association, and conflict removal. Time alignment addresses the issue of asynchronous data acquisition, spatial matching converts video pixel coordinates to actual geodetic coordinates, feature association ensures accurate binding of ship identity information to monitoring data, and conflict removal guarantees the reliability of the dataset. The fused multi-dimensional monitoring dataset covers all aspects of ship operation information, providing comprehensive data support for multi-dimensional anomaly detection by decision-makers.

[0030] In practical applications of this embodiment, the decision-making layer includes an anomaly determination unit and an early warning handling unit; The anomaly determination unit has a built-in multi-dimensional anomaly determination rule library, which makes a comprehensive determination based on the monitoring dataset from four dimensions: navigation status, equipment status, personnel behavior, and environmental adaptation. The early warning and handling unit generates graded early warning signals according to the severity of the anomaly and triggers corresponding handling measures, including ship-side warnings, shore-side alarms, and collaborative handling instructions.

[0031] Specifically, the decision-making layer is the core decision-making unit of the system. Based on a microcontroller and a pre-set rule base, it achieves accurate anomaly detection and tiered response. The anomaly detection unit's built-in multi-dimensional rule base covers four core dimensions: navigation, equipment, personnel, and environment. It adopts a production rule representation, which is logically clear, easy to understand and update, and significantly reduces the false alarm rate through a multi-data source cross-validation mechanism. The early warning and handling unit divides the anomaly into three warning levels according to its severity, with corresponding differentiated warning methods and handling measures. This ensures that emergency anomalies are prioritized and responded to quickly, while general anomalies receive reasonable prompts, balancing response efficiency and resource optimization. Level 1 warnings (such as collision risk, main engine failure, and personnel falling overboard) utilize shipboard audible and visual alarms, bridge voice broadcasts, emergency push notifications via crew mobile apps, red alerts on the shore monitoring platform, and real-time video transmission, while simultaneously triggering the shore-based remote guidance channel. Level 2 warnings (such as channel deviation, speeding, and general equipment malfunctions) utilize shipboard audible and visual alarms, crew mobile app push notifications, and yellow alerts on the shore monitoring platform. Level 3 warnings (such as minor operational violations and slight exceedances of environmental parameters) utilize shipboard voice prompts and blue alerts on the shore monitoring platform. Response measures also include automatically adjusting the focus of the ship's monitoring equipment, locking onto and tracking abnormal targets, saving evidence videos, and generating suggested response plans. The decision-making layer utilizes a high-speed, low-power microcontroller, enabling rapid response in anomaly detection and command generation, meeting the system's real-time requirements.

[0032] In practical applications of this embodiment, the interaction layer includes a communication unit and a coordination unit; The communication unit supports multi-mode data transmission, and the coordination unit is used to realize two-way information interaction, remote guidance and progress feedback between the ship and the shore.

[0033] Specifically, the interaction layer constructs an efficient collaborative channel between the ship and shore to ensure the effective execution of response measures. The communication unit adopts a 5G+satellite dual-mode transmission mechanism to solve the communication coverage problem in complex waters (such as remote inland rivers and coastal areas), ensuring the stability and real-time performance of data transmission. The shore-based receiving unit interfaces with the existing maritime management system through standardized interfaces, enabling multi-department data sharing and breaking down "information silos." The collaborative interaction unit supports two-way voice communication and video transmission between ship and shore, allowing shore-based staff to provide remote response guidance to the ship based on real-time data, and the ship to provide timely feedback on response progress, forming a closed-loop collaborative mechanism that significantly improves the efficiency and accuracy of anomaly response.

[0034] Furthermore, the storage module, serving as the system's end-to-end data support unit, adopts a distributed storage architecture of "local SSD + cloud OSS," balancing the real-time nature and security of data storage. The local SSD stores recent raw data, ensuring rapid data access and local backtracking needs; the cloud OSS service features elastic capacity expansion, high reliability, and high availability, used for long-term storage of end-to-end data. The module supports rapid multi-condition retrieval, accurately locating historical data based on vessel identity, time range, and anomaly type, providing reliable data support for maritime law enforcement, accident investigation, statistical analysis, and algorithm optimization.

[0035] A smart monitoring method for vessels in waterways based on video analytics includes the following steps: S1: Collect water area video data, ship operation status data, water area environment data, and ship identification data through the perception layer; S2: Perform noise filtering, background modeling, and foreground segmentation on the acquired video data, and output the enhanced video data; S3: Based on a deep learning model, ship target recognition and classification are performed on the preprocessed video data. Continuous ship trajectory data is generated through a multi-target tracking algorithm, and ship feature information is extracted simultaneously. S4: Standardize and integrate the ship identification results, trajectory data, ship operation status data, water environment data, and ship identity data to construct a multi-dimensional monitoring dataset; S5: Based on a pre-set anomaly judgment rule base, it comprehensively analyzes the monitoring dataset from four dimensions: navigation status, equipment status, personnel behavior, and environmental adaptation, and determines the anomaly status and severity. S6: Generate graded early warning signals based on the anomaly determination results, trigger corresponding handling measures at the ship and shore ends, and achieve coordinated handling and progress feedback through ship-shore interaction; S7: The storage module stores the raw monitoring data of the perception layer, the analysis results of the analysis layer, the monitoring dataset of the fusion layer, and the early warning records and handling logs of the decision layer, supporting historical data backtracking and statistical analysis.

[0036] In the practical application of this embodiment, in S3, the ship identification process includes multi-scale feature extraction, pixel-level target segmentation and type matching; the trajectory tracking process solves the problem of trajectory loss caused by ship occlusion and changes in imaging area through target association algorithm, and generates complete ship dynamic trajectory data.

[0037] In the practical application of this embodiment, in S5, the anomaly judgment rule base supports dynamic updates and can adjust the judgment threshold and associated conditions according to the water type, ship type and management needs; the judgment of navigation status anomalies is based on the comparison results of ship trajectory, speed and preset waterway and speed limit standards; the judgment of equipment status anomalies is based on the deviation of ship operating parameters from the normal operating range; the judgment of personnel behavior anomalies is based on the crew operation and wearing standards identified by video analysis; and the judgment of environmental adaptation anomalies is based on the matching degree between the current environmental parameters and the ship's seaworthiness range.

[0038] In the practical application of this embodiment, the graded early warning includes three levels: emergency early warning, important early warning, and general early warning. Different levels correspond to different warning methods and handling priorities. The collaborative handling includes automatically locking abnormal targets, saving evidence data, generating handling suggestions, and ship-shore two-way communication guidance measures to ensure that abnormalities are handled quickly and effectively.

[0039] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A smart monitoring system for vessels in waterways based on video analytics, characterized in that: It includes a perception layer, analysis layer, fusion layer, decision-making layer, interaction layer, and storage module; The output of the perception layer is communicatively connected to the input of the analysis layer; the analysis layer is bidirectionally communicatively connected to the fusion layer; the output of the fusion layer is communicatively connected to the input of the decision layer and the input of the storage module, respectively; the decision layer is bidirectionally communicatively connected to the interaction layer; and the analysis layer and the decision layer are communicatively connected to the storage module, respectively. The perception layer is used to collect video data of the water area and auxiliary data related to ships and the environment; the analysis layer is used to perform target detection, feature extraction and dynamic tracking on the video data, and output ship identification results and trajectory data. The fusion layer is used to integrate the output data of the analysis layer and the auxiliary data collected by the perception layer to construct a multi-dimensional monitoring dataset; The decision-making layer is used to determine the abnormal state of the ship based on the monitoring dataset, generate graded early warning signals and trigger disposal instructions; the interaction layer is used to realize data interaction, collaborative disposal and information feedback between the ship and the shore; the storage module is used to store the raw monitoring data collected by the perception layer, the analysis results output by the analysis layer, the monitoring dataset constructed by the fusion layer, and the early warning records and disposal instruction logs generated by the decision-making layer.

2. The intelligent monitoring system for vessels in waterways based on video analysis according to claim 1, characterized in that: The perception layer includes a video acquisition unit and an auxiliary data acquisition unit; The video acquisition unit employs distributed, multi-type video acquisition devices to cover video data acquisition under different water areas and environmental conditions. The auxiliary data acquisition unit is used to collect ship operation status data, water environment data, and ship identification data. The ship operation status data includes ship power system parameters and hull attitude parameters. The water environment data includes meteorological and hydrological parameters. The ship identification data is used to associate basic ship information.

3. The intelligent monitoring system for vessels in waterways based on video analysis according to claim 1, characterized in that: The analysis layer includes a video preprocessing unit, a target recognition unit, a trajectory tracking unit, and a feature extraction unit; The video preprocessing unit is used to perform noise filtering, background modeling and foreground segmentation on the acquired video data, and output the enhanced video data. The target recognition unit uses a deep learning model to accurately identify and classify ship targets, and outputs information on ship type and shape characteristics. The trajectory tracking unit is used to continuously track the identified ship target and generate time-series trajectory data; the feature extraction unit is used to extract ship navigation status features and abnormal behavior features from the identification results and trajectory data.

4. The intelligent monitoring system for vessels in waterways based on video analysis according to claim 1, characterized in that: The fusion layer includes a data standardization unit and an association integration unit; The data standardization unit is used to convert heterogeneous data output from the perception layer and the analysis layer into a unified format, eliminating data dimensional differences; the association and integration unit establishes mapping relationships between different types of data through time alignment, spatial matching and feature association, eliminates redundant and conflicting data, and constructs a structured multi-dimensional monitoring dataset.

5. The intelligent monitoring system for vessels in waterways based on video analytics according to claim 1, characterized in that: The decision-making layer includes an anomaly detection unit and an early warning and handling unit; The anomaly determination unit has a built-in multi-dimensional anomaly determination rule library, which makes a comprehensive determination based on the monitoring dataset from four dimensions: navigation status, equipment status, personnel behavior, and environmental adaptation. The early warning and handling unit generates graded early warning signals according to the severity of the anomaly and triggers corresponding handling measures, including ship-side warnings, shore-side alarms, and collaborative handling instructions.

6. The intelligent monitoring system for vessels in waterways based on video analytics according to claim 1, characterized in that: The interaction layer includes a communication unit and a coordination unit; The communication unit supports multi-mode data transmission, and the coordination unit is used to realize two-way information interaction, remote guidance and progress feedback between the ship and the shore.

7. A smart monitoring method for vessels in waterways based on video analytics, characterized in that, Includes the following steps: S1: Collect water area video data, ship operation status data, water area environment data, and ship identification data through the perception layer; S2: Perform noise filtering, background modeling, and foreground segmentation on the acquired video data, and output the enhanced video data; S3: Based on a deep learning model, ship target recognition and classification are performed on the preprocessed video data. Continuous ship trajectory data is generated through a multi-target tracking algorithm, and ship feature information is extracted simultaneously. S4: Standardize and integrate the ship identification results, trajectory data, ship operation status data, water environment data, and ship identity data to construct a multi-dimensional monitoring dataset; S5: Based on a pre-set anomaly judgment rule base, it comprehensively analyzes the monitoring dataset from four dimensions: navigation status, equipment status, personnel behavior, and environmental adaptation, and determines the anomaly status and severity. S6: Generate graded early warning signals based on the anomaly determination results, trigger corresponding handling measures at the ship and shore ends, and achieve coordinated handling and progress feedback through ship-shore interaction; S7: The storage module stores the raw monitoring data of the perception layer, the analysis results of the analysis layer, the monitoring dataset of the fusion layer, and the early warning records and handling logs of the decision layer, supporting historical data backtracking and statistical analysis.

8. The intelligent monitoring system and method for vessels in waterways based on video analysis according to claim 7, characterized in that: In S3, the ship identification process includes multi-scale feature extraction, pixel-level target segmentation and type matching; the trajectory tracking process uses a target association algorithm to solve the trajectory loss problem caused by ship occlusion and changes in imaging area, and generates complete ship dynamic trajectory data.

9. The intelligent monitoring system and method for vessels in waterways based on video analysis according to claim 7, characterized in that: In S5, the anomaly judgment rule base supports dynamic updates and can adjust the judgment threshold and associated conditions according to the water type, ship type and management needs; the judgment of navigation status anomalies is based on the comparison results of ship trajectory, speed and preset waterway and speed limit standards. Abnormal equipment status is determined based on the deviation of ship operating parameters from the normal operating range; abnormal personnel behavior is determined based on video analysis and identification of crew operation and attire standards. The assessment of environmental compatibility anomalies is based on the degree of matching between current environmental parameters and the ship's seaworthiness range.

10. A video analytics-based intelligent monitoring system and method for vessels in waterways, as described in claim 7, characterized in that: In S6, the graded early warning includes three levels: emergency early warning, important early warning, and general early warning. Different levels correspond to different warning methods and handling priorities. The collaborative handling includes automatically locking abnormal targets, saving evidence data, generating handling suggestions, and ship-shore two-way communication guidance measures to ensure that abnormalities are handled quickly and effectively.

Images