Intelligent waste dumping ship monitoring method and system

Abstract

The invention discloses an intelligent waste dumping ship monitoring method and system. The method comprises the following steps: S1, multi-source data real-time acquisition: acquiring real-time multi-source data from an AIS receiver, a radar, an optical and infrared camera, an acoustic sensor and an environment monitoring sensor; according to the invention, by constructing a dumping monitoring system integrating multi-source sensing, edge calculation and cloud intelligent analysis, real-time identification, dynamic prediction and traceable evidence obtaining of ship illegal dumping behaviors are realized, and compared with the prior art, early warning can be carried out through a track prediction model before the dumping behaviors occur, and the accuracy and the reliability of the system are improved. In addition, fine-grained actions such as cabin opening, dumping and cabin closing can be identified based on video spatial-temporal characteristics, acoustic events and micro-motion information, and meanwhile, a non-tampering evidence chain is formed by utilizing a block chain technology, so that a monitoring result has law enforcement-level credibility, and the discovery rate, accuracy and traceability of marine dumping behaviors are integrally and remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of monitoring technology for dumping vessels, and in particular to an intelligent method and system for monitoring dumping vessels. Background Technology

[0002] With the increase in marine resource development and maritime transport activities, illegal dumping of solid waste, sludge, and engineering waste has become more covert and fragmented. Existing intelligent dumping vessel monitoring systems mainly rely on AIS tracks, radar, or manual patrols, which are ineffective in monitoring illegal dumping at night, in remote sea areas, or under adverse weather conditions. Existing technologies have the following shortcomings: 1. Insufficient real-time performance and lack of dynamic prediction capabilities: Most solutions use offline analysis, which cannot predict the future trajectory of vessels in real time, nor can they issue early warnings in the initial stages of dumping; 2. Lack of fine-grained action recognition capabilities: Existing monitoring can only identify "abnormal stay" or "abnormal trajectory," but cannot identify key actions of dumping, such as hatch opening, material dumping, and hatch closing; 3. Lack of evidence chains for law enforcement: Evidence is scattered and lacks time-series labeling, making it difficult to form a traceable and verifiable behavioral chain. In light of the above, this invention proposes an intelligent dumping vessel monitoring method and system. Summary of the Invention

[0003] Based on the technical problems existing in the background technology, the present invention proposes an intelligent monitoring method and system for dumping waste ships.

[0004] The present invention proposes an intelligent monitoring method for dumping waste vessels, comprising the following steps: S1: Real-time acquisition of multi-source data: Acquire real-time multi-source data from AIS receivers, radar, optical and infrared cameras, acoustic sensors, and environmental monitoring sensors, and add a unified timestamp to all types of data; S2: Edge Preprocessing and Primary Anomaly Detection: Edge nodes perform denoising, keyframe extraction, optical flow analysis, acoustic feature extraction, and trajectory interpolation on the acquired data, and calculate the primary anomaly score A based on a lightweight model. edge ; S3: Multi-source stream fusion: The cloud platform receives data summaries and video clips uploaded by edge nodes, performs time synchronization and cross-sensor target association, and generates a unified behavior situation stream; S4: Online Trajectory Prediction and Risk Scoring: A Transformer / GRU trajectory prediction model supporting incremental learning is used to predict the trajectory of the target vessel for the next 1-30 minutes, and the trajectory deviation D is calculated. t And prediction confidence level C t The comprehensive risk score R is calculated by combining multimodal features. t ; S5: Intelligent Evidence Collection Trigger: If Rt If the value exceeds the preset threshold, the system automatically sends an evidence collection instruction to the edge node, increases the video resolution, encrypts and stores key segments, and generates evidence slices with timestamps. S6: Multimodal fine-grained action recognition: Based on the TSM model, video spatiotemporal features are extracted and combined with acoustic event recognition, attitude estimation and radar micro-motion analysis to identify waste dumping actions, which include continuous actions such as opening the cabin, dumping and closing the cabin, and output action labels and time boundaries. S7: Semanticization of Action Events: Semantically describe the identified action sequences, and generate structured event records by combining trajectory prediction and environmental changes. The structured event records include event time, location, action type, confidence level, and evidence fragment index. S8: Evidence chain generation and tamper-proof storage: Generate evidence packages from keyframes, video clips, acoustic clips and model judgment results, and write them to the blockchain using a hash algorithm to achieve evidence chain storage; S9: Alarm and Coordination: After receiving an alarm, the law enforcement unit will intercept the target vessel on site.

[0005] Preferably, the specific logical steps of S2 are as follows: S201: Edge nodes perform filtering and denoising on the collected multi-source data to reduce the interference of environmental noise on subsequent recognition. Specifically, this includes: (1) AIS trajectory denoising: Kalman filtering is used to smooth the ship speed, heading, and position sequences to obtain a continuous trajectory sequence: ; (2) Video frame denoising: bilateral filtering is used to eliminate sea surface wave noise and low-light noise from the camera; (3) Acoustic signal denoising: an adaptive filter is used to suppress underwater noise and highlight the low-frequency impact sound caused by dumping waste; S202: Extract keyframes containing "possible motion changes" from a continuous video stream based on the difference between adjacent frames. The formula is as follows: ,when At that time, I t Mark these as keyframes, and use histogram similarity to determine whether there is ship tilting or deck change within the camera's field of view, and output the keyframe sequence: ; S203: Optical flow calculation is performed on keyframe segments to obtain micro-motion information of the hull surface and hatch areas, which is used to identify waste dumping characteristics. The formula used to calculate the optical flow vector is: The following indicators were extracted: rate of change of inclination angle, hatch opening speed, and consistency coefficient of deck material flow direction, to form a micro-motion feature vector. The formula used is as follows: ; S204: Frame and window the audio signal acquired by the underwater acoustic sensor, calculate short-time energy and short-time zero-crossing rate, extract MFCC vectors, and calculate low-frequency impulse characteristics to obtain acoustic feature vectors. ; The formula used to calculate short-time energy is: E m Let m be the short-time energy of the m-th frame, where m is the frame index, representing the m-th frame. represents the windowed signal sample of the m-th frame, and N is the number of samples contained in each frame; The formula used to calculate the short-time zero crossing rate is: ZCR m Let m be the zero-crossing rate of the m-th frame. This is a sign function used to determine the sign of a sign object. The formula used to calculate the characteristics of low-frequency impacts is: ,in The low-frequency energy of the m-th frame. For the corresponding low frequency range Frequency index range The power spectrum of the m-th frame; S205: Cubic spline interpolation is used to interpolate missing points in the AIS / BeiDou trajectory to obtain a smooth trajectory sequence on a uniform time scale, ensuring the continuity of subsequent behavior recognition. The formula used for cubic spline interpolation is: The smooth trajectory sequence is And calculate the behavioral dynamics characteristics: rate of change of speed: Rate of change of heading: Abnormal dwell time: And constitute the behavioral feature vector: ; S206: A lightweight deep learning model is used to fuse the above features to obtain a preliminary anomaly score. The formula used is: Fused Feature Vector: ; The lightweight model outputs a basic anomaly score: The value ranges from 0 to 1, representing the probability of suspected dumping behavior being determined by the edge side; S207: When At that time, edge nodes send primary anomaly scores, high-frequency feature summaries, keyframe snippets, optical flow vector summaries, and acoustic feature profiles to the cloud.

[0006] Preferably, the specific logical steps of S3 are as follows: S301: Converts the acquisition timestamps of various data streams from multiple sources, including cameras, AIS, IMU, and acoustic sensors, to a standard timeline. And a sliding time window is used to align observations of different modes: , where t i For the original timestamps of each device, This is the calibration offset between the device and standard time. The allowed synchronization error threshold; S302: Matching ship targets from different sensors to form a unique target ID, which requires calculating multi-feature similarity during target matching: And the optimal correlation across sensors is obtained using the maximum matching method: S p For positional similarity, S v For speed similarity, S h For heading similarity, S a For visual / shape feature similarity, For feature weights, This represents the optimal target matching result. S303: Merge the time-synchronized data and the associated unique target into a behavioral situation sequence in chronological order. When merging the behavioral situation sequence, all modal features need to be merged according to time windows. And calculate the behavior state vector: Where F(t) is the concatenated vector of multimodal features under a unified timestamp. For the behavior feature encoding model, B(t) is the final behavior state flow single-frame vector; Final output: .

[0007] Preferably, in step S4, the trajectory deviation D is calculated. t The formula used is: ,in The actual observed target state, H represents the predicted target state, and H represents the prediction steps. Euclidean distance or weighted Euclidean distance; Calculate the prediction confidence level C t The formula used is: C t The confidence level for trajectory prediction, with a value between 0 and 1. For the uncertainty of the prediction at step k, For actual observation, For model prediction, H is the number of prediction steps; the higher the confidence level, the smaller the bias. Calculate the comprehensive risk score R by combining multimodal features t The formula used is: , where R t For comprehensive risk scoring, the range is 0-1, D tC represents the trajectory deviation. t To predict confidence levels, F i For the i-th multimodal feature, Assigning weights to each indicator. As a normalization function, different dimensions are mapped to 0-1. The larger the trajectory deviation, the lower the confidence level, and the higher the multimodal anomaly characteristics, the higher the risk score.

[0008] Preferably, the specific logical steps of S6 are as follows: S601: Video Frame Sampling and Preprocessing: Preprocessing a continuous video stream at a frame rate f v Frame segmentation, constructing a time window clip length L and a sliding step size S; S602: TSM Spatiotemporal Feature Extraction: For each clip, input the TSM backbone to obtain the spatiotemporal feature tensor: The visual feature vector of each frame can be taken. E t Given a sequence of frames of length L ending at time t. For a spatiotemporal convolutional network containing a Temporal Shift Module, d v For visual feature dimensions; S603: Acoustic Event Detection: Performs STFT / Mel processing on audio segments synchronized with the clip to obtain MFCC or log-Mel spectra, which are then input into the lightweight acoustic classifier g. ac The formula used is: ,in For the Mel-spectrogram of the corresponding time period, d a For acoustic feature dimensions; S604: Attitude Estimation: Run the attitude estimator on the keyframes to obtain the set of key points: Where k is the number of key points, c k To determine the confidence level, geometric features related to "hatch door" and "deck inclination" can be extracted, including hatch door angle. Local tilt angle ; S605: Radar Micro-motion Feature Extraction: Perform short-time Fourier transform analysis on the radar echo to obtain a time-frequency representation. And extract the statistics: And calculate the micro-motion energy mutation index: ,like A significant increase in a short period of time serves as evidence of dumping / object entering the water, where d r For radar feature dimensions, F low For the frequency band of interest; S606: Multimodal Feature Alignment and Normalization: Aligning Visual Vt Acoustics A t Key points / geometry Radar R t Align to a uniform time latt and normalize; S607: Multimodal Fusion and Per-Frame Action Probability Estimation: Constructing Stitching Features: Using a fusion classifier Output the frame-level probability vector p for each action class t , ,in To converge networks, The length of the historical window used for fusion; S608: Temporal Segmentation and Event Boundary Detection: From Frame-Level Probability Sequences Get the action event range ; S609: Output Action Labels and Time Boundaries: For candidate events Calculate event-level confidence: And integrate acoustic / radar event evidence enhancement factors: ,in To determine the normalized acoustic / radar evidence strength within this interval, perform NMS on overlapping or nearest-neighbor events. If two events... and Time intersection and union ratio: Exceeding the threshold If the confidence level is higher, then the result with higher confidence level will be retained, and the final output will be: ; in For time intersection and union comparison, To enhance the weights of acoustic / radar imaging, Let be the probability of category c at time t.

[0009] Preferably, the specific logical steps of S7 are as follows: S701: Input: Action recognition output, fused trajectory situation flow S, environmental sensor sequence, evidence fragment index set; S702: Maps actions to a unified temporal reference for trajectory prediction, specifying the start time of each action segment. Predicting time axis from trajectory Select the moment with the smallest error; S703: For the location where the action occurs, the system uses the position sequence output by the trajectory prediction module. Compared with the actual observed trajectory By performing fusion estimation, the spatial location corresponding to the action is obtained: , where the parameters The trajectory fusion weights are used to balance sensor observation noise and prediction error; S704: Perform semantic association analysis on the sorted action sequence to identify the continuous logical relationships between actions and define events; S705: To quantify the reliability of an event, the system performs weighted fusion based on the feature confidence levels of visual TSM, acoustic, attitude estimation, and radar micro-motion analysis. ,in The weighting coefficients are adaptively adjustable and satisfy the following conditions: This ensures that the fusion results are stable and have physical meaning; S706: Generate a uniform evidence index for each action segment: ,in These are the corresponding video segment numbers, acoustic signal segment numbers, and radar micro-motion segment numbers, respectively, and standardized event logs are generated for regulatory agencies to access. .

[0010] Preferably, the specific logical steps of S8 are as follows: S801: The system automatically extracts keyframes, corresponding video clips, acoustic signal clips, radar micro-motion clips, as well as action recognition, trajectory prediction, and risk scoring results from the identified waste dumping events to form a set of original evidence information to be stored. S802: The above multi-source data is structured and packaged according to a unified evidence format to form an evidence package: This includes data content, timestamps, source device IDs, and event index metadata; S803: Perform a hash operation on the evidence packet to generate an irreversible evidence digest. ; S804: Write the evidence summary, timestamp, and evidence index into the blockchain transaction record: The blockchain nodes reach a consensus on the transaction and upload it to the chain, thereby achieving tamper-proof evidence storage; S805: The complete evidence package is stored in a low-cost off-chain storage system and has a one-to-one mapping relationship with the hash value recorded on the chain.

[0011] This invention also proposes an intelligent monitoring method and system for dumping waste vessels, including a sensor layer, edge computing nodes, a communication module, a cloud analysis platform, and an enforcement interface; The sensor layer is deployed in the monitored sea area or on law enforcement vessels, and includes AIS receivers, radar, optical and infrared cameras, acoustic sensors, and environmental monitoring sensors. The edge computing node includes a preprocessing module, a lightweight anomaly detection module, and a basic inference engine. The edge computing node is deployed in shore base stations, shipborne terminals, or buoy devices. It processes near-source data in real time through the local preprocessing module, lightweight anomaly detection module, and basic inference engine. It can perform noise suppression, keyframe extraction, optical flow analysis, acoustic feature extraction, and simple target recognition operations, and generate a basic anomaly score and data summary. This allows for rapid filtering of non-critical data and reduces network transmission pressure at locations close to the data source. The communication module is responsible for establishing a reliable data link between the sensor layer, edge nodes and cloud platform. It supports multiple communication methods such as 4G / 5G, Beidou short message, and microwave link. It can adaptively select the encoding compression rate and transmission strategy according to the network status to ensure that video clips, trajectory summaries and anomaly tags are uploaded to the cloud analysis platform in a timely and stable manner. The cloud-based analysis platform includes a multi-source fusion module, a trajectory prediction module, an action recognition module, and an evidence chain management module. It is used to receive data summaries and key fragments from the edge, and to achieve in-depth analysis of dumping behavior through cross-sensor target re-identification, spatiotemporal alignment, behavior sequence modeling, future trajectory prediction, and action recognition. It also generates structured records and risk scores by combining environmental changes and event sequences for platform-level intelligent monitoring and auxiliary law enforcement. The enforcement interface is open to environmental protection, maritime or regulatory departments. It provides risk alerts, structured event records, evidence chain verification results and related evidence fragments in real time through API, platform terminals or command systems, enabling law enforcement personnel to grasp the risk of dumping and retrieve the complete evidence chain in the first instance, supporting precise supervision and subsequent enforcement processes.

[0012] Compared with existing technologies, the beneficial effects of this invention are: 1. Edge computing preprocessing and a lightweight anomaly detection model are adopted to achieve near-source real-time analysis; a trajectory prediction network supporting incremental learning is introduced in the cloud, which can predict the movement trend of ships in the next 1 to 30 minutes and calculate the trajectory deviation and risk score, so as to provide early warning before the dumping behavior has fully occurred, solving the problem that traditional solutions are mainly based on offline analysis and lack dynamic prediction capabilities. 2. By integrating video TSM spatiotemporal features, acoustic event recognition, attitude estimation and radar micro-motion analysis, the system automatically identifies the continuous actions of "opening the cabin - dumping - closing the cabin" during the waste dumping process, and outputs action labels and time boundaries. This breaks through the limitation of existing technologies that can only identify abnormal stay or trajectory, and realizes direct detection of the waste dumping action itself, greatly improving the accuracy and targeting of the identification. 3. A structured evidence package is generated from keyframes, video clips, acoustic clips, and model judgment results. This package is then written into the blockchain ledger using a hash algorithm to ensure the immutability of the evidence. Simultaneously, a structured behavioral record with time sequence and event semantics is formed, providing a complete chain of evidence that is verifiable and traceable for environmental law enforcement. This addresses the shortcomings of the existing monitoring system, which suffers from fragmented evidence collection and weak evidence. This invention constructs a waste dumping monitoring system that integrates multi-source sensing, edge computing, and cloud-based intelligent analysis. This system enables real-time identification, dynamic prediction, and traceable evidence collection of illegal waste dumping by ships. Compared with existing technologies, it can not only provide early warnings through trajectory prediction models before waste dumping occurs, but also identify fine-grained actions such as opening, dumping, and closing of cabins based on video spatiotemporal features, acoustic events, and micro-motion information. At the same time, it uses blockchain technology to form an immutable chain of evidence, giving the monitoring results law enforcement-level credibility. Overall, it significantly improves the detection rate, accuracy, and traceability of waste dumping at sea. Attached Figure Description

[0013] Figure 1 This is a flowchart of an intelligent waste dumping vessel monitoring method proposed in this invention; Figure 2 This is a block diagram of an intelligent waste dumping vessel monitoring system proposed in this invention. Detailed Implementation

[0014] The present invention will be further explained below with reference to specific embodiments. Example 1

[0015] Reference Figure 1 This embodiment proposes an intelligent monitoring method for dumping waste vessels, including the following steps: S1: Real-time acquisition of multi-source data: Acquire real-time multi-source data from AIS receivers, radar, optical and infrared cameras, acoustic sensors, and environmental monitoring sensors, and add a unified timestamp to all types of data; S2: Edge Preprocessing and Primary Anomaly Detection: Edge nodes perform denoising, keyframe extraction, optical flow analysis, acoustic feature extraction, and trajectory interpolation on the acquired data, and calculate the primary anomaly score A based on a lightweight model. edge ; The specific logical steps are as follows: S201: Edge nodes perform filtering and denoising on the collected multi-source data to reduce the interference of environmental noise on subsequent recognition. Specifically, this includes: (1) AIS trajectory denoising: Kalman filtering is used to smooth the ship speed, heading, and position sequences to obtain a continuous trajectory sequence: ; (2) Video frame denoising: bilateral filtering is used to eliminate sea surface wave noise and low-light noise from the camera; (3) Acoustic signal denoising: an adaptive filter is used to suppress underwater noise and highlight the low-frequency impact sound caused by dumping waste; S202: Extract keyframes containing "possible motion changes" from a continuous video stream based on the difference between adjacent frames. The formula is as follows: ,when At that time, I t Mark these as keyframes, and use histogram similarity to determine whether there is ship tilting or deck change within the camera's field of view, and output the keyframe sequence: ; S203: Optical flow calculation is performed on keyframe segments to obtain micro-motion information of the hull surface and hatch areas, which is used to identify waste dumping characteristics. The formula used to calculate the optical flow vector is: The following indicators were extracted: rate of change of inclination angle, hatch opening speed, and consistency coefficient of deck material flow direction, to form a micro-motion feature vector. The formula used is as follows: ; S204: Frame and window the audio signal acquired by the underwater acoustic sensor, calculate short-time energy and short-time zero-crossing rate, extract MFCC vectors, and calculate low-frequency impulse characteristics to obtain acoustic feature vectors. ; The formula used to calculate short-time energy is: E m Let m be the short-time energy of the m-th frame, where m is the frame index, representing the m-th frame. represents the windowed signal sample of the m-th frame, and N is the number of samples contained in each frame; The formula used to calculate the short-time zero crossing rate is: ZCR m Let m be the zero-crossing rate of the m-th frame. This is a sign function used to determine the sign of a sign object. The formula used to calculate the characteristics of low-frequency impacts is: ,in The low-frequency energy of the m-th frame. For the corresponding low frequency range Frequency index range The power spectrum of the m-th frame; S205: Cubic spline interpolation is used to interpolate missing points in the AIS / BeiDou trajectory to obtain a smooth trajectory sequence on a uniform time scale, ensuring the continuity of subsequent behavior recognition. The formula used for cubic spline interpolation is: The smooth trajectory sequence is And calculate the behavioral dynamics characteristics: rate of change of speed: Rate of change of heading: Abnormal dwell time: And constitute the behavioral feature vector: ; S206: A lightweight deep learning model is used to fuse the above features to obtain a preliminary anomaly score. The formula used is: Fused Feature Vector: ; The lightweight model outputs a basic anomaly score: The value ranges from 0 to 1, representing the probability of suspected dumping behavior being determined by the edge side; S207: When At the same time, edge nodes send primary anomaly scores, high-frequency feature summaries, keyframe snippets, optical flow vector summaries, and acoustic feature summaries to the cloud; S3: Multi-source stream fusion: The cloud platform receives data summaries and video clips uploaded by edge nodes, performs time synchronization and cross-sensor target association, and generates a unified behavior situation stream; The specific logical steps are as follows: S301: Converts the acquisition timestamps of various data streams from multiple sources, including cameras, AIS, IMU, and acoustic sensors, to a standard timeline. And a sliding time window is used to align observations of different modes: , where t i For the original timestamps of each device, This is the calibration offset between the device and standard time. The allowed synchronization error threshold; S302: Matching ship targets from different sensors to form a unique target ID, which requires calculating multi-feature similarity during target matching: And the optimal correlation across sensors is obtained using the maximum matching method: S p For positional similarity, S v For speed similarity, S h For heading similarity, S a For visual / shape feature similarity, For feature weights, This represents the optimal target matching result. S303: Merge the time-synchronized data and the associated unique target into a behavioral situation sequence in chronological order. When merging the behavioral situation sequence, all modal features need to be merged according to time windows. And calculate the behavior state vector: Where F(t) is the concatenated vector of multimodal features under a unified timestamp. For the behavior feature encoding model, B(t) is the final behavior state flow single-frame vector; Final output: ; S4: Online Trajectory Prediction and Risk Scoring: A Transformer / GRU trajectory prediction model supporting incremental learning is used to predict the trajectory of the target vessel for the next 1-30 minutes, and the trajectory deviation D is calculated. t And prediction confidence level C t The comprehensive risk score R is calculated by combining multimodal features. t ; The trajectory deviation D is calculated. t The formula used is: ,in The actual observed target state, H represents the predicted target state, and H represents the prediction steps. Euclidean distance or weighted Euclidean distance; Calculate the prediction confidence level C t The formula used is: C t The confidence level for trajectory prediction, with a value between 0 and 1. For the uncertainty of the prediction at step k, For actual observation, For model prediction, H is the number of prediction steps; the higher the confidence level, the smaller the bias. Calculate the comprehensive risk score R by combining multimodal features t The formula used is: , where R t For comprehensive risk scoring, the range is 0-1, D t C represents the trajectory deviation. t To predict confidence levels, F i For the i-th multimodal feature, Assigning weights to each indicator. As a normalization function, different dimensions are mapped to 0-1. The larger the trajectory deviation, the lower the confidence level, and the higher the multimodal anomaly characteristics, the higher the risk score. S5: Intelligent Evidence Collection Trigger: If R t If the value exceeds the preset threshold, the system automatically sends an evidence collection instruction to the edge node, increases the video resolution, encrypts and stores key segments, and generates evidence slices with timestamps. S6: Multimodal fine-grained action recognition: Based on the TSM model, video spatiotemporal features are extracted and combined with acoustic event recognition, attitude estimation and radar micro-motion analysis to identify waste dumping actions, which include continuous actions such as opening the cabin, dumping and closing the cabin, and output action labels and time boundaries. The specific logical steps are as follows: S601: Video Frame Sampling and Preprocessing: Preprocessing a continuous video stream at a frame rate f v Frame segmentation, constructing a time window clip length L and a sliding step size S; S602: TSM Spatiotemporal Feature Extraction: For each clip, input the TSM backbone to obtain the spatiotemporal feature tensor: The visual feature vector of each frame can be taken. E t Given a sequence of frames of length L ending at time t. For a spatiotemporal convolutional network containing a Temporal Shift Module, d v For visual feature dimensions; S603: Acoustic Event Detection: Performs STFT / Mel processing on audio segments synchronized with the clip to obtain MFCC or log-Mel spectra, which are then input into the lightweight acoustic classifier g. ac The formula used is: ,in For the Mel-spectrogram of the corresponding time period, d a For acoustic feature dimensions; S604: Attitude Estimation: Run the attitude estimator on the keyframes to obtain the set of key points: Where k is the number of key points, c k To determine the confidence level, geometric features related to "hatch door" and "deck inclination" can be extracted, including hatch door angle. Local tilt angle ; S605: Radar Micro-motion Feature Extraction: Perform short-time Fourier transform analysis on the radar echo to obtain a time-frequency representation. And extract the statistics: And calculate the micro-motion energy mutation index: ,like A significant increase in a short period of time serves as evidence of dumping / object entering the water, where d r For radar feature dimensions, F low For the frequency band of interest; S606: Multimodal Feature Alignment and Normalization: Aligning Visual V t Acoustics A t Key points / geometry Radar R t Align to a uniform time latt and normalize; S607: Multimodal Fusion and Per-Frame Action Probability Estimation: Constructing Stitching Features: Using a fusion classifier Output the frame-level probability vector p for each action class t , ,in To converge networks, The length of the historical window used for fusion; S608: Temporal Segmentation and Event Boundary Detection: From Frame-Level Probability Sequences Get the action event range ; S609: Output Action Labels and Time Boundaries: For candidate events Calculate event-level confidence: And integrate acoustic / radar event evidence enhancement factors: ,in To determine the normalized acoustic / radar evidence strength within this interval, perform NMS on overlapping or nearest-neighbor events. If two events... and Time intersection and union ratio: Exceeding the threshold If the confidence level is higher, then the result with higher confidence level will be retained, and the final output will be: ; in For time intersection and union comparison, To enhance the weights of acoustic / radar imaging, Let c be the probability of category c at time t; S7: Semanticization of Action Events: Semantically describe the identified action sequences, and generate structured event records by combining trajectory prediction and environmental changes. The structured event records include event time, location, action type, confidence level, and evidence fragment index. The specific logical steps are as follows: S701: Input: Action recognition output, fused trajectory situation flow S, environmental sensor sequence, evidence fragment index set; S702: Maps actions to a unified temporal reference for trajectory prediction, specifying the start time of each action segment. Predicting time axis from trajectory Select the moment with the smallest error; S703: For the location where the action occurs, the system uses the position sequence output by the trajectory prediction module. Compared with the actual observed trajectory By performing fusion estimation, the spatial location corresponding to the action is obtained: , where the parameters The trajectory fusion weights are used to balance sensor observation noise and prediction error; S704: Perform semantic association analysis on the sorted action sequence to identify the continuous logical relationships between actions and define events; S705: To quantify the reliability of an event, the system performs weighted fusion based on the feature confidence levels of visual TSM, acoustic, attitude estimation, and radar micro-motion analysis. ,in The weighting coefficients are adaptively adjustable and satisfy the following conditions: This ensures that the fusion results are stable and have physical meaning; S706: Generate a uniform evidence index for each action segment: ,in These are the corresponding video segment numbers, acoustic signal segment numbers, and radar micro-motion segment numbers, respectively, and standardized event logs are generated for regulatory agencies to access. ; S8: Evidence chain generation and tamper-proof storage: Generate evidence packages from keyframes, video clips, acoustic clips and model judgment results, and write them to the blockchain using a hash algorithm to achieve evidence chain storage; The specific logical steps are as follows: S801: The system automatically extracts keyframes, corresponding video clips, acoustic signal clips, radar micro-motion clips, as well as action recognition, trajectory prediction, and risk scoring results from the identified waste dumping events to form a set of original evidence information to be stored. S802: The above multi-source data is structured and packaged according to a unified evidence format to form an evidence package: This includes data content, timestamps, source device IDs, and event index metadata; S803: Perform a hash operation on the evidence packet to generate an irreversible evidence digest. ; S804: Write the evidence summary, timestamp, and evidence index into the blockchain transaction record: The blockchain nodes reach a consensus on the transaction and upload it to the chain, thereby achieving tamper-proof evidence storage; S805: The complete evidence package is stored in a low-cost off-chain storage system and a one-to-one mapping relationship is established with the hash value recorded on the chain; S9: Alarm and Coordination: After receiving an alarm, the law enforcement terminal will intercept the target vessel on site. This embodiment constructs a dumping monitoring system that integrates multi-source sensing, edge computing, and cloud-based intelligent analysis. It achieves real-time identification, dynamic prediction, and traceable evidence collection of illegal dumping behavior by ships. Compared with existing technologies, it can not only provide early warnings through trajectory prediction models before dumping occurs, but also identify fine-grained actions such as opening, dumping, and closing of cabins based on video spatiotemporal features, acoustic events, and micro-motion information. At the same time, it uses blockchain technology to form an immutable chain of evidence, giving the monitoring results law enforcement-level credibility. Overall, it significantly improves the detection rate, accuracy, and traceability of marine dumping behavior. Example 2

[0016] Reference Figure 2 This embodiment also proposes an intelligent monitoring method system for dumping waste vessels, including a sensor layer, edge computing nodes, a communication module, a cloud analysis platform, and an enforcement interface; The sensor layer is deployed in the monitored sea area or on law enforcement vessels, including AIS receivers, radar, optical and infrared cameras, acoustic sensors, and environmental monitoring sensors. Edge computing nodes include a preprocessing module, a lightweight anomaly detection module, and a basic inference engine. Deployed in shore base stations, shipborne terminals, or buoy devices, edge computing nodes process near-source data in real time through local preprocessing modules, lightweight anomaly detection modules, and basic inference engines. They can perform noise suppression, keyframe extraction, optical flow analysis, acoustic feature extraction, and simple target recognition operations, and generate basic anomaly scores and data summaries. This allows for rapid filtering of non-critical data and reduces network transmission pressure at locations close to the data source. The communication module is responsible for establishing a reliable data link between the sensor layer, edge nodes and the cloud platform. It supports multiple communication methods such as 4G / 5G, Beidou short message, and microwave link. It can adaptively select the encoding compression rate and transmission strategy according to the network status to ensure that video clips, trajectory summaries and anomaly tags are uploaded to the cloud analysis platform in a timely and stable manner. The cloud-based analytics platform includes a multi-source fusion module, a trajectory prediction module, an action recognition module, and an evidence chain management module. It receives data summaries and key fragments from the edge and performs in-depth analysis of dumping behavior through cross-sensor target re-identification, spatiotemporal alignment, behavior sequence modeling, future trajectory prediction, and action recognition. It also generates structured records and risk scores by combining environmental changes and event sequences for platform-level intelligent monitoring and auxiliary law enforcement. The enforcement interface is open to environmental protection, maritime or regulatory departments. It provides risk alerts, structured event records, evidence chain verification results and related evidence fragments in real time through API, platform terminals or command systems, enabling law enforcement personnel to grasp the risk of dumping and retrieve the complete evidence chain in the first time, supporting precise supervision and subsequent enforcement processes.

[0017] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for monitoring intelligent waste dumping vessels, characterized in that, Includes the following steps: S1: Real-time acquisition of multi-source data: Acquire real-time multi-source data from AIS receivers, radar, optical and infrared cameras, acoustic sensors, and environmental monitoring sensors, and add a unified timestamp to all types of data; S2: Edge Preprocessing and Primary Anomaly Detection: Edge nodes perform denoising, keyframe extraction, optical flow analysis, acoustic feature extraction, and trajectory interpolation on the acquired data, and calculate the primary anomaly score A based on a lightweight model. edge ; S3: Multi-source stream fusion: The cloud platform receives data summaries and video clips uploaded by edge nodes, performs time synchronization and cross-sensor target association, and generates a unified behavior situation stream; S4: Online Trajectory Prediction and Risk Scoring: A Transformer / GRU trajectory prediction model supporting incremental learning is used to predict the trajectory of the target vessel for the next 1-30 minutes, and the trajectory deviation D is calculated. t And prediction confidence level C t The comprehensive risk score R is calculated by combining multimodal features. t ; S5: Intelligent Evidence Collection Trigger: If R t If the value exceeds the preset threshold, the system automatically sends an evidence collection instruction to the edge node, increases the video resolution, encrypts and stores key segments, and generates evidence slices with timestamps. S6: Multimodal fine-grained action recognition: Based on the TSM model, video spatiotemporal features are extracted and combined with acoustic event recognition, attitude estimation and radar micro-motion analysis to identify waste dumping actions, including continuous actions of opening the cabin, dumping and closing the cabin, and output action labels and time boundaries. S7: Semanticization of Action Events: Semantically describe the identified action sequences, and generate structured event records by combining trajectory prediction and environmental changes. The structured event records include event time, location, action type, confidence level, and evidence fragment index. S8: Evidence chain generation and tamper-proof storage: Generate evidence packages from keyframes, video clips, acoustic clips and model judgment results, and write them to the blockchain using a hash algorithm to achieve evidence chain storage; S9: Alarm and Coordination: After receiving an alarm, the law enforcement unit will intercept the target vessel on site.

2. The intelligent waste dumping vessel monitoring method according to claim 1, characterized in that, The specific logical steps of S2 are as follows: S201: Edge nodes perform filtering and denoising on the collected multi-source data to reduce the interference of environmental noise on subsequent recognition. Specifically, this includes: (1) AIS trajectory denoising: Kalman filtering is used to smooth the ship speed, heading, and position sequences to obtain a continuous trajectory sequence: ; (2) Video frame denoising: bilateral filtering is used to eliminate sea surface wave noise and low-light noise from the camera; (3) Acoustic signal denoising: an adaptive filter is used to suppress underwater noise and highlight the low-frequency impact sound caused by dumping waste; S202: Extract keyframes containing "possible motion changes" from a continuous video stream based on the difference between adjacent frames. The formula is as follows: ,when At that time, I t Mark these as keyframes, and use histogram similarity to determine whether there is ship tilting or deck change within the camera's field of view, and output the keyframe sequence: ; S203: Optical flow calculation is performed on keyframe segments to obtain micro-motion information of the hull surface and hatch areas, which is used to identify waste dumping characteristics. The formula used to calculate the optical flow vector is: The following indicators were extracted: rate of change of inclination angle, hatch opening speed, and consistency coefficient of deck material flow direction, to form a micro-motion feature vector. The formula used is as follows: ; S204: Perform framing and windowing on the audio signal acquired by the underwater acoustic sensor, calculate short-time energy and short-time zero-crossing rate, extract MFCC vectors, and calculate low-frequency impulse characteristics to obtain acoustic feature vectors. ; The formula used to calculate short-time energy is: E m Let m be the short-time energy of the m-th frame, where m is the frame index, representing the m-th frame. represents the windowed signal sample of the m-th frame, and N is the number of samples contained in each frame; The formula used to calculate the short-time zero crossing rate is: ZCR m Let m be the zero-crossing rate of the m-th frame. This is a sign function used to determine the sign of a sign object. The formula used to calculate the characteristics of low-frequency impacts is: ,in The low-frequency energy of the m-th frame. For the corresponding low frequency range Frequency index range The power spectrum of the m-th frame; S205: Cubic spline interpolation is used to interpolate missing points in the AIS / BeiDou trajectory to obtain a smooth trajectory sequence on a uniform time scale, ensuring the continuity of subsequent behavior recognition. The formula used for cubic spline interpolation is: The smooth trajectory sequence is And calculate the behavioral dynamics characteristics: rate of change of speed: Rate of change of heading: Abnormal dwell time: And constitute the behavioral feature vector: ; S206: A lightweight deep learning model is used to fuse the above features to obtain a preliminary anomaly score. The formula used is: Fused Feature Vector: ; The lightweight model outputs a basic anomaly score: The value ranges from 0 to 1, representing the probability of suspected dumping behavior being determined by the edge side; S207: When At that time, edge nodes send primary anomaly scores, high-frequency feature summaries, keyframe snippets, optical flow vector summaries, and acoustic feature profiles to the cloud.

3. The intelligent waste dumping vessel monitoring method according to claim 1, characterized in that, The specific logical steps of S3 are as follows: S301: Converts the acquisition timestamps of various data streams from multiple sources, including cameras, AIS, IMU, and acoustic sensors, to a standard timeline. And a sliding time window is used to align observations of different modes: , where t i For the original timestamps of each device, This is the calibration offset between the device and standard time. The allowed synchronization error threshold; S302: Matching ship targets from different sensors to form a unique target ID, which requires calculating multi-feature similarity during target matching: And the optimal correlation across sensors is obtained using the maximum matching method: S p For positional similarity, S v For speed similarity, S h For heading similarity, S a For visual / shape feature similarity, For feature weights, This is the optimal target matching result; S303: Merge the time-synchronized data and the associated unique target into a behavioral situation sequence in chronological order. When merging the behavioral situation sequence, all modal features need to be merged according to time windows. And calculate the behavior state vector: Where F(t) is the concatenated vector of multimodal features under a unified timestamp. For the behavior feature encoding model, B(t) is the final behavior state flow single-frame vector; Final output: .

4. The intelligent waste dumping vessel monitoring method according to claim 1, characterized in that, In step S4, the trajectory deviation D is calculated. t The formula used is: ,in The actual observed target state, H represents the predicted target state, and H represents the prediction steps. Euclidean distance or weighted Euclidean distance; Calculate the prediction confidence level C t The formula used is: C t The confidence level for trajectory prediction, with a value between 0 and 1. For the uncertainty of the prediction at step k, For actual observation, For model prediction, H is the number of prediction steps; the higher the confidence level, the smaller the bias. Calculate the comprehensive risk score R by combining multimodal features t The formula used is: , where R t For comprehensive risk scoring, the range is 0-1, D t C represents the trajectory deviation. t To predict confidence levels, F i For the i-th multimodal feature, Assigning weights to each indicator. As a normalization function, different dimensions are mapped to 0-1. The larger the trajectory deviation, the lower the confidence level, and the higher the multimodal anomaly characteristics, the higher the risk score.

5. The intelligent waste dumping vessel monitoring method according to claim 1, characterized in that, The specific logical steps of S6 are as follows: S601: Video Frame Sampling and Preprocessing: Preprocessing a continuous video stream at a frame rate f v Frame segmentation, constructing a time window clip length L and a sliding step size S; S602: TSM Spatiotemporal Feature Extraction: For each clip, input the TSM backbone to obtain the spatiotemporal feature tensor: The visual feature vector of each frame can be taken. E t Given a sequence of frames of length L ending at time t. For a spatiotemporal convolutional network containing a Temporal Shift Module, d v For visual feature dimensions; S603: Acoustic Event Detection: Performs STFT / Mel processing on audio segments synchronized with the clip to obtain MFCC or log-Mel spectra, which are then input into the lightweight acoustic classifier g. ac The formula used is: ,in For the Mel-spectrogram of the corresponding time period, d a For acoustic feature dimensions; S604: Attitude Estimation: Run the attitude estimator on the keyframes to obtain the set of key points: Where k is the number of key points, c k To determine the confidence level, geometric features related to "hatch door" and "deck inclination" can be extracted, including hatch door angle. Local tilt angle ; S605: Radar Micro-motion Feature Extraction: Perform short-time Fourier transform analysis on the radar echo to obtain a time-frequency representation. And extract the statistics: And calculate the micro-motion energy mutation index: ,like A significant increase in a short period of time serves as evidence of dumping / object entering the water, where d r For radar feature dimensions, F low For the frequency band of interest; S606: Multimodal Feature Alignment and Normalization: Aligning Visual V t Acoustics A t Key points / geometry Radar R t Align to a uniform time latt and normalize; S607: Multimodal Fusion and Per-Frame Action Probability Estimation: Constructing Stitching Features: Using a fusion classifier Output the frame-level probability vector p for each action class t , ,in To converge networks, The length of the historical window used for fusion; S608: Temporal Segmentation and Event Boundary Detection: From Frame-Level Probability Sequences Get the action event range ; S609: Output Action Labels and Time Boundaries: For candidate events Calculate event-level confidence: And integrate acoustic / radar event evidence enhancement factors: ,in To determine the normalized acoustic / radar evidence strength within this interval, perform NMS on overlapping or nearest-neighbor events. If two events... and Time intersection and union ratio: Exceeding the threshold If the confidence level is higher, then the result with higher confidence level will be retained, and the final output will be: ; in For time intersection and union comparison, To enhance the weights of acoustic / radar imaging, Let be the probability of category c at time t.

6. The intelligent waste dumping vessel monitoring method according to claim 1, characterized in that, The specific logical steps of S7 are as follows: S701: Input: Action recognition output, fused trajectory situation flow S, environmental sensor sequence, evidence fragment index set; S702: Maps actions to a unified temporal reference for trajectory prediction, specifying the start time of each action segment. Predicting time axis from trajectory Select the moment with the smallest error; S703: For the location where the action occurs, the system uses the position sequence output by the trajectory prediction module. Compared with the actual observed trajectory By performing fusion estimation, the spatial location corresponding to the action is obtained: , where the parameters The trajectory fusion weights are used to balance sensor observation noise and prediction error; S704: Perform semantic association analysis on the sorted action sequence to identify the continuous logical relationships between actions and define events; S705: To quantify the reliability of an event, the system performs weighted fusion based on the feature confidence levels of visual TSM, acoustic, attitude estimation, and radar micro-motion analysis. ,in The weighting coefficients are adaptively adjustable and satisfy the following conditions: This ensures that the fusion results are stable and have physical meaning; S706: Generate a uniform evidence index for each action segment: ,in These are the corresponding video segment numbers, acoustic signal segment numbers, and radar micro-motion segment numbers, respectively, and standardized event logs are generated for regulatory agencies to access. .

7. The intelligent waste dumping vessel monitoring method according to claim 1, characterized in that, The specific logical steps of S8 are as follows: S801: The system automatically extracts keyframes, corresponding video clips, acoustic signal clips, radar micro-motion clips, as well as action recognition, trajectory prediction, and risk scoring results from the identified waste dumping events to form a set of original evidence information to be stored. S802: The above multi-source data is structured and packaged according to a unified evidence format to form an evidence package: This includes data content, timestamps, source device IDs, and event index metadata; S803: Perform a hash operation on the evidence packet to generate an irreversible evidence digest. ; S804: Write the evidence summary, timestamp, and evidence index into the blockchain transaction record: The blockchain nodes reach a consensus on the transaction and upload it to the chain, thereby achieving tamper-proof evidence storage; S805: The complete evidence package is stored in a low-cost off-chain storage system and has a one-to-one mapping relationship with the hash value recorded on the chain.

8. A smart monitoring system for dumping waste vessels, used to implement the method described in any one of claims 1-7, characterized in that, This includes a sensor layer, edge computing nodes, communication modules, a cloud analytics platform, and law enforcement interfaces; The sensor layer is deployed in the monitored sea area or on law enforcement vessels, and includes AIS receivers, radar, optical and infrared cameras, acoustic sensors, and environmental monitoring sensors. The edge computing node includes a preprocessing module, a lightweight anomaly detection module, and a basic inference engine. The edge computing node is deployed in shore base stations, shipborne terminals, or buoy devices. It processes near-source data in real time through the local preprocessing module, lightweight anomaly detection module, and basic inference engine. It can perform noise suppression, keyframe extraction, optical flow analysis, acoustic feature extraction, and simple target recognition operations, and generate a basic anomaly score and data summary. This allows for rapid filtering of non-critical data and reduces network transmission pressure at locations close to the data source. The communication module is responsible for establishing a reliable data link between the sensor layer, edge nodes and cloud platform. It supports multiple communication methods such as 4G / 5G, Beidou short message, and microwave link. It can adaptively select the encoding compression rate and transmission strategy according to the network status to ensure that video clips, trajectory summaries and anomaly tags are uploaded to the cloud analysis platform in a timely and stable manner. The cloud-based analysis platform includes a multi-source fusion module, a trajectory prediction module, an action recognition module, and an evidence chain management module. It is used to receive data summaries and key fragments from the edge, and to achieve in-depth analysis of dumping behavior through cross-sensor target re-identification, spatiotemporal alignment, behavior sequence modeling, future trajectory prediction, and action recognition. It also generates structured records and risk scores by combining environmental changes and event sequences for platform-level intelligent monitoring and auxiliary law enforcement. The enforcement interface is open to environmental protection, maritime or regulatory departments. It provides risk alerts, structured event records, evidence chain verification results and related evidence fragments in real time through API, platform terminals or command systems, enabling law enforcement personnel to grasp the risk of dumping and retrieve the complete evidence chain in the first instance, supporting precise supervision and subsequent enforcement processes.

Images