Building protection method and device for channel improvement

Abstract

The invention relates to a building protection method and device for channel improvement, and belongs to the technical field of water traffic safety and channel management, and the method comprises the steps: collecting the dynamic information of a ship in real time through multiple types of sensors, carrying out the fusion of the dynamic information of the ship through hierarchical adaptive filtering, and obtaining the fused ship behavior data; and carrying out real-time monitoring and analysis on ship behaviors based on the fused ship behavior data, predicting a motion track of the ship based on the constructed track prediction model, judging whether the ship has a risk of crossing a channel to renovate a building based on the predicted motion track of the ship, and if the risk exists, carrying out real-time monitoring and analysis on the ship behaviors. And early warning is performed on the ship based on the preset early warning level, so that threat of the ship to the channel improvement building is effectively prevented in time, and the safety and stability of the channel are guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of water traffic safety and waterway management, and in particular to a waterway regulation building protection method and device. Background Art

[0002] Channel regulation structures, such as spur dams and spur dikes, play an important role in waterways like the Yangtze River. They alter the direction and velocity of water flow, ensuring channel stability and the safe passage of ships. However, these structures face multiple safety threats, including ship collisions.

[0003] Existing technical means still have some shortcomings in terms of ship behavior monitoring, data fusion and early warning response in complex water scenarios. For example, the traditional AIS system has strong dependence, low positioning accuracy, information delay and other problems, making it difficult to comprehensively monitor ship behavior; multi-sensor systems face timeliness and consistency issues in data fusion, affecting monitoring accuracy and system real-time performance; traditional monitoring systems lack effective ship behavior analysis strategies and are prone to misjudging the behavior of ships undergoing normal channel inspections and maintenance, resulting in unnecessary early warnings, and at the same time, the potential risk assessment of other ships is not accurate enough.

[0004] Therefore, in complex water scenarios, existing technologies have problems such as incomplete ship behavior monitoring, insufficient timeliness and consistency of data fusion, which affects ship behavior analysis and risk assessment, and leads to low accuracy of early warnings. Summary of the Invention

[0005] In view of this, it is necessary to provide a method, system and electronic equipment for protecting channel regulation structures to solve the technical problems of incomplete monitoring of ship behavior in complex water scenarios, insufficient timeliness and consistency of data fusion, and low accuracy of early warning.

[0006] In order to solve the above problems, in a first aspect, the present invention provides a method for protecting a waterway regulation structure, comprising: The ship's dynamic information is collected in real time by multiple types of sensors, and the ship's dynamic information is fused using a layered adaptive filter to obtain fused ship behavior data, wherein the layered adaptive filter includes a primary filter layer and a collaborative filter layer; Based on the fused ship behavior data, the ship behavior is monitored and analyzed in real time, wherein the movement trajectory of the ship is predicted based on the constructed trajectory prediction model, and based on the predicted movement trajectory of the ship, it is determined whether the ship has a risk of crossing a channel regulation structure. When there is a risk, the ship is warned based on a preset warning level.

[0007] In one possible implementation, the multiple types of sensors include millimeter wave radar, visual sensor, and AIS; and the real-time collection of dynamic information of the ship by the multiple types of sensors includes: Collect distance and speed information of ships through millimeter-wave radar; Using visual sensors to capture the appearance and behavior of ships in the monitored waters, and to collect environmental conditions, including image information of water surface conditions, weather conditions, and lighting changes; Collect vessel identity and location information through AIS.

[0008] In a possible implementation, the layered adaptive filtering is used to fuse the dynamic information of the ship and the environmental parameters to obtain fused ship behavior data, wherein the layered adaptive filtering includes a primary filtering layer and a collaborative filtering layer, including: The primary filtering layer filters the dynamic information of the ship collected by each sensor through an improved strong tracking Kalman filter to obtain preliminary fusion data of each sensor; The collaborative filtering layer constructs a spatiotemporal consistency evaluation function based on a multi-sensor dynamic weight allocation mechanism, calculates the weight of each sensor based on the spatiotemporal consistency evaluation function, and applies the weight of each sensor to the preliminary fusion data of the corresponding sensor to obtain the fused ship behavior data.

[0009] In a possible implementation, the adopting layered adaptive filtering to fuse the dynamic information of the ship to obtain fused ship behavior data further includes: The missing values ​​of the fused ship behavior data are determined, and linear interpolation and spline interpolation are used to supplement the missing values.

[0010] In one possible implementation, the improved strong tracking Kalman filter is: , , in, is the state vector in the state prediction equation, is the state transition matrix, is the control input matrix, for The control input at the moment, is the forecast error covariance matrix, is the fading factor, is the process noise covariance matrix, is the matrix transpose, For the current moment, For the previous moment; The spatiotemporal consistency evaluation function is: , in, For the The weight of each sensor, is the variance of the sensor data, is the time attenuation coefficient, is the timestamp of the sensor data, The current system time.

[0011] In a possible implementation, the real-time monitoring of ship behavior based on the fused ship behavior data includes: Detecting ships based on the fused ship behavior data and the improved YOLOv7 channel detection model to identify ship targets in the channel; The SORT++ multimodal tracking algorithm is used to track ship targets in the channel, and the MTL-ShipNet multi-task feature extraction model is used to classify ships, detect key points, and recognize colors.

[0012] In one possible implementation, the improved YOLOv7 channel detection model includes a backbone network, a neck network, and a head network, the backbone network includes an ECA-Net attention module, and the neck network includes a multi-scale feature pyramid fusion module; detecting ships based on the fused ship behavior data and the improved YOLOv7 channel detection model to identify ship targets in the channel includes: The fused ship behavior data is input into the improved YOLOv7 channel detection model, and multi-scale feature extraction is performed on the fused ship behavior data through the convolutional layer of the backbone network and the ECA-Net attention module; Inputting the extracted multi-scale features into the neck network, and performing feature fusion on the extracted multi-scale features through the multi-scale feature pyramid fusion module of the neck network; The fused features are input into the head network, and the fused features are detected by the head network to identify the ship targets in the channel.

[0013] In one possible implementation, the SORT++ multimodal tracking algorithm includes a SORT algorithm and a CNN; the MTL-ShipNet multitask feature extraction model includes a backbone network layer and a shared feature layer, wherein the backbone network layer includes ConvNeXt-Base; the SORT++ multimodal tracking algorithm is used to track ship targets in a waterway, and the MTL-ShipNet multitask feature extraction model is used to classify ships, detect key points, and recognize colors, including: A pre-trained CNN is used to extract features of the ship target to obtain the ship's appearance features. The appearance feature score and motion feature score of the ship are calculated using the SORT algorithm. A comprehensive matching score is determined based on the appearance feature score and motion feature score. The ship target in the channel is tracked based on the comprehensive matching score. The fused ship behavior data is input into the MTL-ShipNet multi-task feature extraction model. After feature extraction of the fused ship behavior data is performed through ConvNeXt-Base extraction, the shared feature layer optimizes the extracted features through a multi-task collaborative mechanism to obtain common features of multiple tasks. Ship classification, key point detection and color recognition are completed based on the common features of the multiple tasks.

[0014] In a possible implementation, predicting the motion trajectory of the ship based on the constructed trajectory prediction model includes: The current position, speed, heading, and historical trajectory of the ship are obtained based on the fused ship behavior data, and a trajectory prediction model for the ship in the area of ​​the renovated buildings is constructed based on the current position, speed, heading, and historical trajectory of the ship; Predicting the motion trajectory of the ship based on the trajectory prediction model to obtain a predicted trajectory of the ship; Taking into account the influence of various environmental factors on the ship's motion, the predicted trajectory is corrected using a constructed multivariate regression model, wherein the various environmental factors include wind speed and water speed.

[0015] In a second aspect, the present invention further provides a waterway regulation structure protection device, comprising: A data fusion module is used to collect dynamic information of the ship in real time through multiple sensors, fuse the dynamic information of the ship using layered adaptive filtering, and obtain fused ship behavior data; The regulation building protection module is used to monitor and analyze the ship behavior in real time based on the fused ship behavior data, wherein the movement trajectory of the ship is predicted based on the constructed trajectory prediction model, and based on the predicted movement trajectory of the ship, it is judged whether the ship has the risk of crossing the channel regulation building. When there is a risk, the ship is warned based on a preset warning level.

[0016] The beneficial effects of the present invention are as follows: dynamic information of ships is collected in real time by multiple types of sensors, and the dynamic information of ships is fused by hierarchical adaptive filtering to obtain fused ship behavior data. The multi-source sensor data fusion can integrate the advantages of each sensor to provide more accurate ship behavior data, so that the monitoring accuracy reaches the centimeter level, and the subtle dynamic changes of ships can be accurately captured. The problems of incomplete ship behavior monitoring and insufficient timeliness and consistency of data fusion are solved. The fused ship behavior data is monitored and analyzed in real time, wherein the motion trajectory of the ship is predicted based on the constructed trajectory prediction model, and the risk of the ship crossing the channel regulation structure is determined based on the predicted motion trajectory of the ship. When the risk exists, the ship is warned based on the preset warning level. Through comprehensive ship behavior monitoring and intelligent warning, the collision of ships with channel regulation structures is effectively prevented. Through real-time monitoring and data analysis of ship behavior, the behavior pattern of the ship is identified, abnormal behavior and potential threats of the ship can be discovered in a timely manner, the operation status of the channel can be better understood, and reasonable management strategies and emergency plans can be formulated, thereby improving the accuracy and effectiveness of the warning and ensuring the safety of the channel regulation structure and ships. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 A flow chart of an embodiment of the waterway regulation structure protection method provided by the present invention; Figure 2 A schematic diagram of the device perception effect of the waterway regulation building protection method provided by the present invention; Figure 3 A schematic diagram of a gap in a waterway regulation building provided by the method for protecting a waterway regulation building provided by the present invention; Figure 4 A schematic diagram of a ship traveling in a warning area but parallel to the direction of the channel regulation structure according to the method for protecting the channel regulation structure provided by the present invention; Figure 5A schematic diagram of a ship traveling in a warning area and facing the direction of the channel regulation structure according to the method for protecting the channel regulation structure provided by the present invention; Figure 6 A schematic diagram of AIS data on June 1 for the waterway regulation structure protection method provided by the present invention; Figure 7 A schematic diagram of the AIS data on June 25th for the waterway regulation structure protection method provided by the present invention; Figure 8 A schematic diagram of the AIS data on August 13 for the waterway regulation structure protection method provided by the present invention; Figure 9 A schematic diagram of AIS data on September 17th for the waterway regulation structure protection method provided by the present invention; Figure 10 A schematic diagram of AIS data on September 23rd for the waterway regulation structure protection method provided by the present invention; Figure 11 This is a structural schematic diagram of an embodiment of the waterway regulation structure protection device provided by the present invention. DETAILED DESCRIPTION

[0019] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, and are not used to limit the scope of the present invention.

[0020] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0021] Before presenting the embodiments, the following terms are explained.

[0022] The SORT++ algorithm is an enhanced multi-target tracking algorithm based on SORT (Simple Online and Realtime Tracking). It aims to improve tracking performance in complex scenarios. The SORT algorithm predicts target motion trajectories through Kalman filtering and uses the Hungarian algorithm to correlate detection boxes with predicted boxes. Its advantage is strong real-time performance, but it relies on detection quality and is prone to ID switching issues when there is high occlusion or when the targets look similar.

[0023] MTL-ShipNet is a deep learning model based on Multi-Task Learning (MTL), designed specifically for ship-related tasks. Its core idea is to simultaneously learn multiple related tasks (such as ship detection, classification, trajectory prediction, etc.) by sharing the underlying network representation, in order to improve the model's overall understanding of ship scenarios and prediction efficiency.

[0024] The present invention discloses a method and device for protecting waterway regulation structures, such as Figure 1 As shown in the figure, the protection methods of waterway regulation structures include: S101. Using multiple sensors to collect dynamic information of the ship in real time, and using layered adaptive filtering to fuse the dynamic information of the ship to obtain fused ship behavior data, wherein the layered adaptive filtering includes a primary filtering layer and a collaborative filtering layer; It should be noted that multiple types of sensors, including millimeter-wave radar, visual sensors, and AIS, work together in the data fusion process through the high-precision detection of millimeter-wave radar, the image recognition capabilities of visual sensors, and the dynamic information supplementation of the AIS system, so that the monitoring accuracy reaches the centimeter level, and can accurately capture the tiny dynamic changes of the ship.

[0025] S102: Real-time monitoring and analysis of ship behavior based on the integrated ship behavior data. The ship's trajectory is predicted based on the constructed trajectory prediction model. Based on the predicted trajectory, it is determined whether the ship is at risk of crossing a channel regulation structure. If a risk exists, an early warning is issued to the ship based on a preset early warning level. It should be noted that the improved YOLOv7 channel detection model improves the detection accuracy of small targets in channel scenarios. The SORT++ multimodal tracking algorithm solves the ID switching problem in dense ship scenarios. The MTL-ShipNet multi-task feature extraction model completes ship classification, key point detection (mast, bow) and color recognition. By simultaneously completing ship type classification, key point positioning and color recognition, it provides multimodal data support for ship behavior analysis, breaking through the performance bottleneck of traditional image processing algorithms in complex water scenarios.

[0026] In some embodiments, in step S101, dynamic information of the ship is collected in real time through multiple types of sensors, including millimeter wave radar, visual sensor and AIS. Millimeter wave radar and visual sensor are installed in key areas of the channel regulation building, such as the ends and middle parts of the regulation building. The millimeter wave radar NSR1500W adopts a highly complex continuous frequency modulation (FMCW) solution. The radar adopts a lightweight design, its power consumption is 18W, and its size is (mm), 1.5m distance resolution, effective ranging range of 1500m, which can meet the needs, and collects the distance and speed information of the ship through millimeter wave radar. Millimeter wave radar is used to detect the distance, speed and direction of the ship. The appearance and behavior information of the ship in the monitored waters are captured by the visual sensor, and the environmental conditions are collected. Among them, the environmental conditions include water surface conditions, weather conditions and image information of light changes, which are used to capture the appearance characteristics and behavior patterns of the ship. The visual sensor can provide rich image information to help identify the type, size, color and other characteristics of the ship, as well as the movement trajectory and behavioral abnormalities of the ship. The visual sensor is an IP67-level protective camera to ensure the clarity of the image. A panoramic camera with 4K resolution and a wide viewing angle of 120 degrees is selected, which can provide sufficient details in a wide area to maximize the coverage of the monitoring area and reduce blind spots. The camera uses 8K resolution. It also supports adjustable focal length to meet monitoring needs at different distances. To cope with various climatic conditions, the visual sensor uses an IP67-rated protective camera to ensure its stable operation in harsh environments. AIS collects the identity and location information of the ship, and receives dynamic data such as the ship's identity, position, heading and speed through AIS. AIS can realize automatic identification and information exchange between ships, providing an important reference basis for ship behavior analysis and risk assessment. The AIS system is installed on the ship to receive and send dynamic data such as the ship's identity, position, heading and speed. At the same time, AIS base stations are deployed along the waterway to receive AIS signals sent by the ship and transmit the data to the central control system. The selection criteria for AIS receivers include frequency band, receiving sensitivity and data processing capability. To ensure that all relevant AIS signals can be received, receivers with standard AIS frequency bands (161.975 MHz and 162.025 MHz) are selected. For a schematic diagram of the device perception effect, please refer to Figure 2 .

[0027] Data collected by various sensors is transmitted from various hardware components to the control center and cloud platform. This includes wireless transmission equipment, network connection equipment, and data processing modules. The wireless transmission equipment is responsible for remote data transmission, while the network connection equipment ensures stable data transmission and system interoperability. The data processing module processes, forwards, and stores data. The wireless transmission equipment collaborates with the fiber optic network via Wi-Fi 6 / 5G wireless networks to achieve high-speed data transmission (latency <50ms), supporting wide-area coverage in complex waters. The network connection equipment, which is responsible for data transmission and network interoperability, includes routers, switches, and network interface cards. High-bandwidth routers and switches are selected for network connection equipment to support high-speed data flow and reduce network bottlenecks. Devices with high processing power are selected to cope with the rapid processing requirements of large amounts of data. For redundancy design, devices supporting dual-link redundancy and automatic failover are selected to ensure stable network connections and data transmission in the event of device failure. In addition, network equipment supporting advanced encryption protocols is selected to protect information security during data transmission. Key tasks include data decoding, data format conversion, data storage, and data forwarding. Through these processes, data can be effectively transmitted to the control center and cloud platform for further analysis and processing. The data processing module plays an important role in ensuring the system's data integrity and real-time performance. The data processing module decodes, converts the format, and caches the received data to ensure data integrity and real-time performance. Data from wireless transmission equipment, network connection equipment, and data processing modules are integrated and coordinated through a unified network architecture, ensuring smooth data flow and system stability, and achieving efficient data transmission and processing.

[0028] A layered adaptive filter is used to fuse the ship's dynamic information to obtain fused ship behavior data. The layered adaptive filter consists of a primary filtering layer and a collaborative filtering layer. In the management of waterways near waterway regulation structures, actual data often contains noise such as sensor errors and environmental interference. Sensor errors may arise from hardware limitations or calibration issues, while environmental interference may include changes in water volume, rain, snow, and the influence of surrounding terrain. This noise typically manifests as random fluctuations in the data, which may be high-frequency instantaneous changes or low-frequency long-term trends, threatening the accuracy of trajectory data. A layered adaptive filter is used to fuse the ship's dynamic information to address the timeliness and consistency issues of multi-sensor data fusion in complex environments. In the primary filtering layer, an improved strong tracking Kalman filter is used to filter the ship's dynamic information collected by each sensor to obtain preliminary fused data from each sensor. For single sensor data, an improved strong tracking Kalman filter (STKF) is used. By introducing a fading factor to adjust the state covariance matrix in real time, it enhances the tracking capability of sudden trajectory changes (tracking error is reduced by 62% at a mutation rate of 3 m / s²). The improved strong tracking Kalman filter is: , , in, is the state vector in the state prediction equation, is the state transition matrix, is the control input matrix, for The control input at the moment, is the forecast error covariance matrix, is the fading factor, is the process noise covariance matrix, is the matrix transpose, For the current moment, is the previous moment; the collaborative filtering layer constructs a spatiotemporal consistency evaluation function based on the multi-sensor dynamic weight allocation mechanism, and its spatiotemporal consistency evaluation function is: , in, For the The weight of each sensor, is the variance of the sensor data, is the time attenuation coefficient, is the timestamp of the sensor data, The current time of the system; The weight of each sensor is calculated based on the spatiotemporal consistency evaluation function, and the weight of each sensor is applied to the preliminary fusion data of the corresponding sensor to obtain the fused ship behavior data; By adopting time synchronization and spatial registration technology, the data collected by millimeter-wave radar, visual sensors and AIS systems are aligned and fused. Through feature-level fusion, characteristic information such as the ship's position, speed, and heading are extracted. Combined with the ship's appearance characteristics and behavior patterns, comprehensive ship behavior data is generated.

[0029] The data processed by the Kalman filter can be used to generate a smoother trajectory map. In order to reduce data fluctuations, it can more accurately reflect the actual motion path of the ship, reduce data errors caused by noise, make the trajectory data more stable and reliable, and enhance the monitoring ability of ship behavior. The missing values ​​of the fused ship behavior data are determined, and linear interpolation and spline interpolation are used to supplement the missing values. Missing values ​​may be caused by sensor failure, data transmission problems or environmental factors. The existence of missing values ​​will affect the continuity of the trajectory and the accuracy of the analysis results. Linear interpolation and spline interpolation are used to supplement the missing values ​​to ensure the integrity and continuity of the trajectory data; linear interpolation fills the gap by performing linear estimation between two known data points before and after the missing value. Linear interpolation uses the linear relationship between known data points to estimate the missing value. If a data point is missing between time t1 and t2, the data point can be supplemented by linear interpolation. The linear interpolation is: , in, For time The interpolation result of and is the value of a known data point, and is the time of the known data point; Spline interpolation estimates missing values ​​by constructing a smooth curve to maintain data continuity. It fits a cubic polynomial function between each pair of adjacent data points to ensure the smoothness of the entire data set. The cubic spline interpolation is: , in, It is A cubic polynomial on an interval, and It is The endpoints of the interval, 、 、 、 The coefficients are determined by satisfying the conditions of continuity of the interpolation function at the data points, continuity of the first-order derivative, and continuity of the second-order derivative. The advantage of spline interpolation is that it can smoothly transition between each data point, providing smoother and more accurate results than linear interpolation.

[0030] In some embodiments, in step S102, the ship behavior is monitored and analyzed in real time based on the fused ship behavior data. In the real-time monitoring of the ship behavior, the ship is detected based on the fused ship behavior data and the improved YOLOv7 channel detection model to identify the ship target in the channel. The improved YOLOv7 channel detection model includes a backbone network, a neck network and a head network. The backbone network includes an ECA-Net attention module, and the neck network includes a multi-scale feature pyramid fusion module. The improved YOLOv7 channel detection model detects the ship, specifically as follows: the fused ship behavior data is input into the improved YOLOv7 channel detection model (YOLOv7-ECA), and the fused ship is detected through the convolution layer of the backbone network and the ECA-Net attention module. Multi-scale feature extraction is performed on behavioral data. The extracted multi-scale features are fed into the neck network, where they are fused using the multi-scale feature pyramid fusion module. The fused features are then fed into the head network, where they are detected to identify ship targets in the waterway. To address the challenge of detecting small ship targets in waterway scenarios, a waterway-specific variant of YOLOv7, YOLOv7-ECA (with integrated lightweight attention module), is proposed. This embeds a lightweight ECA-Net attention module into the backbone network, reducing computational complexity by 30% while improving small target detection accuracy. Multi-scale feature pyramid fusion (MFPN) is also used to support simultaneous detection of ships ranging from 1×1 to 200×200 pixels in 4K images. The SORT++ multimodal tracking algorithm is used to track ship targets in the waterway. The SORT++ multimodal tracking algorithm includes the SORT algorithm and CNN. A pre-trained CNN is used to extract features of the ship target to obtain the appearance features of the ship. The SORT algorithm calculates the appearance feature score and the motion feature score of the ship. Based on the appearance feature score and the motion feature score, a comprehensive matching score is determined. The ship target in the waterway is tracked based on the comprehensive matching score. To solve the ID switching problem in dense ship scenes, the SORT++ algorithm is designed. The SORT++ algorithm upgrades the traditional SORT IOU matching to appearance-motion bimodal matching. Appearance-motion bimodal matching: Combine the ship appearance features (128-dimensional vector) extracted by CNN with the motion trajectory similarity (IoU) to calculate the comprehensive matching score. The comprehensive matching score is: , in, is the comprehensive matching score, is the appearance feature score, Score the motion feature, Weight parameters, where , ID switching rate <2.5%; a trajectory segment compensation mechanism is introduced in the SORT++ algorithm. When the target is temporarily lost (<5 frames), the missing segment is filled based on the LSTM predicted trajectory to ensure tracking continuity; The MTL-ShipNet multi-task feature extraction model is used to classify ships, detect key points and recognize colors. The MTL-ShipNet multi-task feature extraction model includes a backbone network layer and a shared feature layer. The backbone network layer includes ConvNeXt-Base. The fused ship behavior data is input into the MTL-ShipNet multi-task feature extraction model. After ConvNeXt-Base extracts features from the fused ship behavior data, the shared feature layer optimizes the extracted features through a multi-task collaborative mechanism to obtain the common features of multiple tasks. Based on the common features of multiple tasks, the ship is classified. Classification, key point detection and color recognition; by building a multi-task deep learning model (MTL-ShipNet) to identify ship types and extract ship appearance features. The backbone network of the multi-task deep learning model adopts the ConvNeXt-Base pre-trained model, and simultaneously completes ship classification, key point detection (mast, bow, etc.) and color recognition through a shared feature layer. The shared feature layer reduces computational redundancy and designs a channel scenario adversarial training strategy: adversarial samples such as simulated waves and haze are added in the data enhancement stage to improve the model's feature extraction robustness in harsh environments (the classification accuracy of heavy rain scenes remains at 92.3%, while the untrained model is only 76%).

[0031] The core purpose of analyzing ship behavior is to determine the ship's movement trend and, in turn, whether the ship intends to cross the remediation buildings. Ship behavior analysis includes multimodal behavior analysis and specific target behavior analysis. Multimodal behavior analysis includes motion trajectory analysis, speed change analysis, comprehensive judgment, and abnormal behavior detection. Motion trajectory analysis extracts the ship's motion trajectory in the waterway and uses geometric analysis to determine the ship's status, such as straight or turning. The analysis of straight and curved trajectories helps determine whether the ship has deviated from the planned navigation path, which may indicate an intention to cross the boundary. Speed ​​change analysis calculates the ship's speed by position changes between consecutive frames and analyzes the speed change pattern (acceleration, deceleration, stability, etc.) to ensure real-time identification of the ship's status. Comprehensive judgment combines trajectory and speed analysis to improve identification accuracy. For example, the relationship between the curved trajectory and speed changes of a ship when turning can provide a more comprehensive basis for judgment. Abnormal behavior detection identifies potential abnormal behaviors by monitoring drastic changes in trajectory or sudden fluctuations in speed. Abnormal behaviors include sudden changes in direction, sudden acceleration or deceleration, and abnormal docking. In multimodal behavior analysis, the ship's trajectory data is first processed to extract the characteristics of the ship's behavior, such as the ship's speed, acceleration, course change, and distance from the warning area of ​​the regulation building. During the extraction process, millimeter-wave radar data, image data, and AIS data each provide different information dimensions. By comprehensively utilizing different information dimensions, the accuracy of ship behavior pattern recognition can be improved. By analyzing AIS data, the ship's navigation status (such as straight, turning, acceleration, deceleration, etc.) can be effectively identified, and abnormal behavior can be detected and intervened in advance. Through time series analysis of AIS data, the ship's navigation status such as straight, turning, acceleration, and deceleration can be identified; AIS data is used to identify abnormal behaviors of ships that may cross regulation buildings to ensure the safety of waterways and buildings. Identification of abnormal behaviors includes sudden changes in direction: when a ship approaches the regulation building area, a sudden change in direction is a high-risk signal of potential crossing the regulation building. By monitoring the sharp increase in the course change rate in the AIS data, when the ship makes a sharp turn and deviates near the regulation building When a vessel deviates from its planned navigation path, it can identify attempts to circumvent or cross the boundary, thereby issuing an early warning before the crossing. Sudden acceleration or deceleration: If a vessel suddenly accelerates or decelerates near a remediation structure, it may intend to quickly cross or approach a sensitive area, increasing the risk of crossing the remediation structure. Speed ​​mutation monitoring in AIS data can help identify such behavior, especially in the event of a sudden increase in speed. The early warning system will be on high alert and alert management personnel to possible cross-border attempts. Abnormal docking: When a vessel docks in an unauthorized area near a remediation structure, it may attempt to prepare to cross the remediation structure or observe the surrounding environment. By comparing the positioning of the AIS data with the preset mooring point, abnormal docking behavior can be identified and the vessel can be prevented from approaching the sensitive area further. AIS data can provide early identification and prevention of behavior crossing the remediation structure, ensuring that the monitoring system can quickly take countermeasures when the ship actually approaches or attempts to cross the boundary, thereby effectively protecting the safety of the remediation structure. Millimeter-wave radar data can provide information such as the ship's distance, speed, and direction, which is crucial for identifying behavioral patterns.

[0032] Specific target behavior analysis includes determining whether the detected ship is a channel inspection and maintenance ship, and determining whether other ships except channel boats have the risk of crossing the regulation buildings. It determines whether the detected ship is a channel inspection and maintenance ship to avoid treating channel boats that are normally patrolling the channel as risks and triggering early warnings. When determining whether the detected ship is a channel inspection and maintenance ship, if the work boat is equipped with an AIS transceiver, the corresponding ship number will be added to the electronic fence "white list", that is, the AIS receiving device of the electronic fence will not trigger the sound and light alarm after receiving the AIS signal from the white list.

[0033] Determine whether other ships besides channel boats are at risk of crossing channel regulation buildings, predict the ship's motion trajectory based on the constructed trajectory prediction model, obtain the ship's current position, speed, heading and historical trajectory based on the fused ship behavior data, and construct a trajectory prediction model for ships in the regulation building area based on the ship's current position, speed, heading and historical trajectory; predict the ship's motion trajectory based on the trajectory prediction model to obtain the predicted trajectory of the ship; consider the impact of various environmental factors on the ship's motion, and use the constructed multivariate regression model to correct the predicted trajectory, wherein the various environmental factors include wind speed and water flow speed; determine whether the ship is at risk of crossing channel regulation buildings based on the predicted ship's motion trajectory, and when there is a risk, issue an early warning to the ship based on a preset early warning level; The trajectory prediction model adopts the PC-LSTM model, which embeds the ship kinematic equation in the LSTM hidden layer and introduces the ship kinematic constraint term in the LSTM hidden layer: , , in, For the time step The LSTM hidden layer state, is the input feature of the current time step, is the hidden layer state at the previous time step, is the weight matrix of the output layer, is the bias vector, is the constraint weight, For speed, is the heading angle, The trajectory prediction model uses the current state (including speed, direction, position, etc.) and historical state of the ship to estimate the displacement trajectory of the ship in the future, thereby determining whether there is a risk of the ship crossing the renovated building. Different warning levels are set based on the prediction results of the trajectory prediction model. If the risk of a ship entering the remediation building area within the next 10 seconds exceeds 80%, a level one warning will be triggered; if it is predicted that the ship will approach within 120 meters of the remediation building within 5 seconds, a level two warning will be triggered.

[0034] In addition, considering the impact of environmental factors on ship motion, such as wind speed, water flow, etc., a multiple regression model is used for correction. The multiple regression model comprehensively considers the impact of various environmental factors (such as wind speed, water flow, etc.) on ship motion to improve the accuracy of prediction. The multiple regression model takes wind speed into account. , water flow velocity , ship speed and heading These input variables are regressed with the actual displacement of the ship, and the regression equation is: , , in, and For ships in and Displacement in direction, 、 、 、 is the weight coefficient, is the comprehensive impact factor, and is the error term; based on the analysis of ship motion and multi-source sensor data, a comprehensive influencing factor is introduced, which combines the influence of wind, wave, current and meteorological factors on ship motion. The comprehensive influencing factor is obtained through in-depth analysis of historical data; the least squares method is used to fit the weights of each parameter to ensure that the model can accurately describe the motion state of the ship under different conditions, and the weights of environmental factors are dynamically adjusted according to the specific channel conditions. For example, when the wind speed is high, the influence of wind speed on the lateral deviation of the ship increases significantly. At this time, the wind speed weight in the comprehensive influencing factor is will be assigned a higher value; in rapid waters, the dominant effect of water velocity on ship movement is more prominent, and the water flow weight It will rise accordingly.

[0035] By introducing comprehensive influencing factors, the regression model can not only more accurately describe the movement characteristics of ships in complex environments, but also make dynamic adjustments based on real-time environmental data, thereby improving the accuracy and applicability of the prediction. Combined with the trajectory prediction model and the multivariate regression model, it can achieve accurate prediction and real-time adjustment of ship movement. When the risk of a ship crossing a regulated building increases, the risk value calculated based on the comprehensive influencing factors and the trajectory model triggers an early warning and dynamically adjusts the warning level, significantly improving the safety of waterways and buildings, while enhancing the system's adaptability to complex shipping environments.

[0036] Take the renovated buildings in a certain area as an example, the schematic diagram of the renovated buildings can be found in Figure 3 , a diagram of a ship traveling in the warning area but parallel to the direction of the rectification building, please refer to Figure 4 , a diagram of a ship in the warning area and heading towards the remediation building, please refer to Figure 5 Before the integration of millimeter-wave radar and visual sensors on the renovated buildings, the AIS data of ships, as shown in the diagram of AIS data on June 1, can be found in the following figure: Figure 6 For the AIS data diagram on June 25, please refer to Figure 7 ,like Figure 6 、 Figure 7 As shown in the figure, the AIS data shows that there are obvious ships illegally passing through the regulation buildings. After integrating millimeter wave radar and visual sensors on the regulation buildings, please refer to the AIS data diagram on August 13. Figure 8 For the AIS data diagram on September 17, please refer to Figure 9 For the AIS data diagram on September 23, please refer to Figure 10 ,like Figure 8 、 Figure 9 、 Figure 10 According to the AIS data, no crossing of the remediation buildings was observed. Statistics on the ship's navigation tracks from July 18 to November 21, combined with the alarm records, showed that all ships along the route did not enter the dangerous waters and no collision with the remediation buildings occurred, which effectively reduced the risk of ships mistakenly entering the dangerous waters and played a protective role in the channel buildings in the remediation building area.

[0037] Through real-time monitoring and data analysis of ship behavior, we can better understand the operation status of the waterway and formulate reasonable management strategies and emergency plans. At the same time, the intelligent early warning and response mechanism has also transformed waterway management from passive response to active prevention, promoting the rapid development of waterway management towards intelligence and informatization.

[0038] In summary, the waterway regulation building protection method provided by the present invention collects the dynamic information of the ship in real time through multiple sensors, adopts layered adaptive filtering to fuse the dynamic information of the ship, and obtains the fused ship behavior data; based on the fused ship behavior data, the ship behavior is monitored and analyzed in real time, wherein the motion trajectory of the ship is predicted based on the constructed trajectory prediction model, and whether the ship has the risk of crossing the waterway regulation building is judged based on the predicted motion trajectory of the ship. When there is a risk, the ship is warned based on the preset warning level, which improves the accuracy of the warning, timely and effectively prevents the threat of ships to the waterway regulation buildings, and ensures the safety and stability of the waterway.

[0039] In order to better implement the waterway regulation building protection method in the embodiment of the present invention, based on the waterway regulation building protection method, correspondingly, Figure 11 As shown, an embodiment of the present invention further provides a waterway regulation building protection device, and the waterway regulation building protection device 1100 includes: The data fusion module 1101 is used to collect the dynamic information of the ship in real time through multiple sensors, fuse the dynamic information of the ship using layered adaptive filtering, and obtain fused ship behavior data. The layered adaptive filtering includes a primary filtering layer and a collaborative filtering layer. The regulation building protection module 1102 is used to monitor and analyze the ship behavior in real time based on the integrated ship behavior data. The ship's motion trajectory is predicted based on the constructed trajectory prediction model. Based on the predicted ship's motion trajectory, it is determined whether the ship is at risk of crossing the channel regulation building. When there is a risk, the ship is warned based on a preset warning level.

[0040] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. A method for protecting waterway regulation structures, characterized in that: include: The ship's dynamic information is collected in real time by multiple types of sensors, and the ship's dynamic information is fused using a layered adaptive filter to obtain fused ship behavior data, wherein the layered adaptive filter includes a primary filter layer and a collaborative filter layer; Based on the fused ship behavior data, the ship behavior is monitored and analyzed in real time, wherein the movement trajectory of the ship is predicted based on the constructed trajectory prediction model, and based on the predicted movement trajectory of the ship, it is determined whether the ship has a risk of crossing a channel regulation structure. When there is a risk, the ship is warned based on a preset warning level.

2. The waterway regulation structure protection method according to claim 1, characterized in that: The multiple types of sensors include millimeter wave radar, visual sensor and AIS; the real-time collection of dynamic information of the ship by the multiple types of sensors includes: Collect distance and speed information of ships through millimeter-wave radar; Using visual sensors to capture the appearance and behavior of ships in the monitored waters, and to collect environmental conditions, including image information of water surface conditions, weather conditions, and lighting changes; Collect vessel identity and location information through AIS.

3. The waterway regulation structure protection method according to claim 2, characterized in that: The layered adaptive filtering is used to fuse the dynamic information of the ship to obtain fused ship behavior data, wherein the layered adaptive filtering includes a primary filtering layer and a collaborative filtering layer, including: The primary filtering layer filters the dynamic information of the ship collected by each sensor through an improved strong tracking Kalman filter to obtain preliminary fusion data of each sensor; The collaborative filtering layer constructs a spatiotemporal consistency evaluation function based on a multi-sensor dynamic weight allocation mechanism, calculates the weight of each sensor based on the spatiotemporal consistency evaluation function, and applies the weight of each sensor to the preliminary fusion data of the corresponding sensor to obtain the fused ship behavior data.

4. The waterway regulation structure protection method according to claim 3, characterized in that: The method of fusing the dynamic information of the ship using layered adaptive filtering to obtain fused ship behavior data further includes: The missing values ​​of the fused ship behavior data are determined, and linear interpolation and spline interpolation are used to supplement the missing values.

5. The waterway regulation structure protection method according to claim 3, characterized in that: The improved strong tracking Kalman filter is: ， ， in, is the state vector in the state prediction equation, is the state transition matrix, is the control input matrix, for The control input at the moment, is the forecast error covariance matrix, is the fading factor, is the process noise covariance matrix, is the matrix transpose, For the current moment, For the previous moment; The spatiotemporal consistency evaluation function is: ， in, For the The weight of each sensor, is the variance of the sensor data, is the time attenuation coefficient, is the timestamp of the sensor data, The current system time.

6. The waterway regulation structure protection method according to claim 3, characterized in that: The real-time monitoring of ship behavior based on the fused ship behavior data includes: Detecting ships based on the fused ship behavior data and the improved YOLOv7 channel detection model to identify ship targets in the channel; The SORT++ multimodal tracking algorithm is used to track ship targets in the channel, and the MTL-ShipNet multi-task feature extraction model is used to classify ships, detect key points, and recognize colors.

7. The waterway regulation structure protection method according to claim 6, characterized in that: The improved YOLOv7 channel detection model includes a backbone network, a neck network, and a head network. The backbone network includes an ECA-Net attention module, and the neck network includes a multi-scale feature pyramid fusion module. The ship is detected based on the fused ship behavior data and the improved YOLOv7 channel detection model to identify ship targets in the channel, including: The fused ship behavior data is input into the improved YOLOv7 channel detection model, and multi-scale feature extraction is performed on the fused ship behavior data through the convolutional layer of the backbone network and the ECA-Net attention module; Inputting the extracted multi-scale features into the neck network, and performing feature fusion on the extracted multi-scale features through the multi-scale feature pyramid fusion module of the neck network; The fused features are input into the head network, and the fused features are detected by the head network to identify the ship targets in the channel.

8. The waterway regulation structure protection method according to claim 6, characterized in that: The SORT++ multimodal tracking algorithm includes the SORT algorithm and CNN; the MTL-ShipNet multitask feature extraction model includes a backbone network layer and a shared feature layer, and the backbone network layer includes ConvNeXt-Base; the SORT++ multimodal tracking algorithm is used to track ship targets in the channel, and the MTL-ShipNet multitask feature extraction model is used to classify ships, detect key points, and recognize colors, including: A pre-trained CNN is used to extract features of the ship target to obtain the ship's appearance features. The appearance feature score and motion feature score of the ship are calculated using the SORT algorithm. A comprehensive matching score is determined based on the appearance feature score and motion feature score. The ship target in the channel is tracked based on the comprehensive matching score. The fused ship behavior data is input into the MTL-ShipNet multi-task feature extraction model. After feature extraction is performed on the fused ship behavior data through ConvNeXt-Base, the shared feature layer optimizes the extracted features through a multi-task collaborative mechanism to obtain common features of multiple tasks. Ship classification, key point detection and color recognition are completed based on the common features of the multiple tasks.

9. The waterway regulation structure protection method according to claim 6, characterized in that: The motion trajectory of the ship is predicted based on the constructed trajectory prediction model, including: The current position, speed, heading, and historical trajectory of the ship are obtained based on the fused ship behavior data, and a trajectory prediction model for the ship in the area of ​​the renovated buildings is constructed based on the current position, speed, heading, and historical trajectory of the ship; Predicting the motion trajectory of the ship based on the trajectory prediction model to obtain a predicted trajectory of the ship; Taking into account the influence of various environmental factors on the ship's motion, the predicted trajectory is corrected using a constructed multivariate regression model, wherein the various environmental factors include wind speed and water speed.

10. A waterway regulation building protection device, characterized in that: include: A data fusion module is used to collect dynamic information of the ship in real time through multiple sensors, fuse the dynamic information of the ship using layered adaptive filtering, and obtain fused ship behavior data, wherein the layered adaptive filtering includes a primary filtering layer and a collaborative filtering layer; The regulation building protection module is used to monitor and analyze the ship behavior in real time based on the fused ship behavior data, wherein the movement trajectory of the ship is predicted based on the constructed trajectory prediction model, and based on the predicted movement trajectory of the ship, it is judged whether the ship has the risk of crossing the channel regulation building. When there is a risk, the ship is warned based on a preset warning level.

Images