Anti-collision early warning system for ship navigation bridge

Abstract

The invention discloses an anti-collision early warning system for a ship navigation bridge, and belongs to the technical field of bridge collision prevention. Comprising a monitoring module used for collecting ship information in an early warning area; the information processing module is used for obtaining ship poses, mast recognition and the height difference between the highest point of the mast and the water surface based on the ship information; the dangerous water area dividing module is used for calculating the longitudinal minimum safety distance and dividing the grade of the dangerous water area based on the longitudinal minimum safety distance; the early warning module is used for obtaining a comprehensive risk value based on the ship pose, mast identification and the height difference between the highest point of the mast and the water surface; early warning grades are divided based on the comprehensive risk values and the grades of the dangerous water areas, and anti-collision early warning is carried out based on the early warning grades. The method is superior to an existing bridge anti-collision technology in the aspects of monitoring precision, environmental adaptability, early warning timeliness, warning diversity and deployment economy.

Description

Technical Field

[0001] The present invention belongs to the technical field of bridge anti-collision, and in particular relates to an anti-collision warning system for a ship navigation bridge. Background Art

[0002] Currently, mainstream bridge collision avoidance technologies are categorized into two main types: passive protection and active warning. Passive protection primarily relies on physical devices (such as fenders) to absorb collision energy and reduce bridge damage. However, passive protection cannot prevent accidents; it can only mitigate the consequences. Active warning technology currently relies primarily on a single technology approach: video surveillance and AIS (Automatic Identification System). Traditional video surveillance is susceptible to interference from environmental factors such as heavy fog, heavy rain, and low light at night, making it difficult to accurately capture ship movements around the clock. While the AIS system can provide ship identity and trajectory data, it relies heavily on active reporting by ships. Therefore, developing a ship bridge collision avoidance warning system with high monitoring accuracy and timely and accurate warnings to reduce the incidence of ship-bridge collisions is of great theoretical and practical significance.

[0003] Regarding collision avoidance warning systems for ship navigation bridges, the paper "Research on Ship-Bridge Collision Avoidance Monitoring and Warning Systems" uses intelligent video processing technology, combined with manipulation theory and motion mathematical models, to predict collision risks, but monitoring accuracy needs to be improved. The paper "Design and Experimental Research on an Active Warning System for Bridge-Ship Collision Prevention" uses video surveillance technology, utilizing infrared, visible light, and laser composite detection and multi-source information fusion to design an active warning system for bridge-ship collision avoidance. However, the warning method is limited and lacks a hierarchical response mechanism. Summary of the Invention

[0004] The present invention aims to solve the deficiencies of the prior art and proposes a ship navigation bridge anti-collision warning system, comprising:

[0005] A monitoring module is used to collect ship information within the warning area; the ship information includes visible light and infrared images, lidar point cloud data, and AIS information;

[0006] An information processing module, configured to obtain a ship posture and a height difference between a highest point of the mast and a water surface based on the ship information;

[0007] A dangerous water area classification module is used to calculate the longitudinal minimum safety distance and classify the dangerous water area based on the longitudinal minimum safety distance;

[0008] An early warning module is used to calculate a comprehensive risk value based on the ship's posture and the height difference from the highest point of the mast to the water surface, and to divide the early warning level based on the comprehensive risk value and the level of dangerous waters in the area where the ship is located, and to perform anti-collision early warning based on the early warning level.

[0009] Further preferably, the information processing module includes:

[0010] An image processing unit for identifying the position of a ship based on visible light and infrared images;

[0011] A mast recognition unit, used to recognize masts based on visible light and infrared images and based on lidar point cloud data;

[0012] LiDAR point cloud processing unit, used to estimate the ship's position and posture based on LiDAR point cloud data;

[0013] The AIS processing unit is used to extract the ship's position and track data based on the AIS information.

[0014] Further preferably, the method for identifying the position of a ship based on visible light and infrared images includes: identifying the position of the ship using a single-stage detector YOLO-G model;

[0015] The single-stage detector YOLO-G model includes a feature extractor and a YOLO interaction layer;

[0016] The feature extractor is improved based on the Darknet-53 network. The improvement method includes:

[0017] For visible light and infrared images with an input resolution of 416×416 and 3 channels, the deep network architecture design constructs a structure consisting of 12 convolutional layers and 6 residual layers, obtaining output feature maps of 4 different scales. All convolutional layers perform batch normalization and activation function operations in sequence. The first convolutional layer is configured with 32 3×3 convolution kernels to extract features from visible light and infrared images. The output is used as the input of the second layer, which uses 64 3×3 convolution kernels with a step size of 2 to complete the downsampling operation. Based on the residual connection mechanism of YOLOv7, a residual convolution group is formed by alternating stacking of 3×3 and 1×1 convolution layers, and the final output feature map of size 208×208 is obtained. In the residual convolution group:

[0018] ;

[0019] Where, Feature map for network input; For the The network output of the layer; Represents the splicing between feature maps; ( ) is a combination function of batch normalization, activation function and convolution operation, which is used to implement the first Nonlinear transformation of layers;

[0020] in, The operation of ( ) is: Conv(1,1)-BN-Relu-Conv(3,3)-BN-Relu, further constructing 4 groups of residual convolution groups, and downsampling at 4 times, 8 times, 16 times, and 32 times, respectively, corresponding to the output size of 104×104, 52×52, 26×26, and 13×13 feature maps, and the feature maps are fused through the upsampling layer to construct a feature pyramid structure;

[0021] The YOLO interaction layer is divided into four independent detection branches, each of which contains six convolutional layers and performs a 2x upsampling operation. Each branch is connected to the upsampling layer through tensor splicing to complete the multi-scale feature fusion of the shallow position information and deep semantic information of the input image. The multi-scale feature fusion formula is:

[0022] ;

[0023] ;

[0024] Where, Indicates the feature map that needs to be fused; Represents an upsampling operation; Represents the concatenation of feature map tensors at the same scale; Represents the fused feature map; Indicates the reconstruction and recognition of the obtained fused feature map; Represents the feature map after reconstruction and recognition;

[0025] After multi-scale feature fusion, the YOLO interaction layer finally outputs four feature maps with scales of 13×13, 26×26, 52×52, and 104×104.

[0026] Further preferably, the method for estimating the ship's pose based on the laser radar point cloud data comprises: segmenting the laser radar point cloud data using a region growing algorithm based on an indexed eight-neighborhood to obtain a segmented point cloud; and estimating the ship's pose based on the segmented laser radar point cloud data;

[0027] The process of cutting the laser radar point cloud data using the region growing algorithm based on the indexed eight-neighborhood includes:

[0028] S1. Traverse the lidar point cloud data in scanning order and find the first unattributed data point, which is set as the seed point ;

[0029] S2, seed point Centered, computing The eight neighborhood points of the index matrix ,if Satisfy the point cloud growth criteria, and into one category and Save in the stack;

[0030] S3. Take the top point of the stack and use it as a new seed Repeat S2 and S3;

[0031] S4: When the stack is empty, it means that the region has grown. Return to S1 and look for a new seed point for the new region.

[0032] S5. Repeat S1 to S4 until every data point in the index has an attribute.

[0033] Further preferably, the method for calculating the longitudinal minimum safety distance by the dangerous water area division module includes: calculating the first longitudinal minimum safety distance by a collision avoidance method based on a dynamic trajectory and calculating the second longitudinal minimum safety distance by a collision avoidance method based on a static rule;

[0034] Calculate the first longitudinal minimum safe distance based on the collision avoidance method of dynamic trajectory The calculation methods include:

[0035] ;

[0036] Where, Indicates the current speed of the ship; represents the reaction time window; Indicates the human-machine reaction time, Indicates the effective deceleration that can be achieved by the ship when emergency deceleration is adopted. Indicates the captain;

[0037] Calculation of the second longitudinal minimum safety distance based on the collision avoidance method of static rules The calculation methods include:

[0038] ;

[0039] Where, 、 Respectively represent the safety factors required for uplink and downlink;

[0040] The first longitudinal minimum safety distance and the second longitudinal minimum safety distance are weightedly integrated to obtain the longitudinal minimum safety distance:

[0041] ;

[0042] Where, Represents the weighting coefficient.

[0043] Further preferably, the method for calculating the comprehensive risk value includes:

[0044] ;

[0045] Where, is the risk zone coefficient; is the heading deviation risk factor; is the heel angle risk factor; is the pitch angle risk factor; is the headroom risk factor; is the speed risk factor; is the weight coefficient of the risk factor, which adds up to 1.

[0046] Further preferably, the method for calculating the risk factor includes:

[0047] ;

[0048] ;

[0049] Where, is the ship’s course deviation angle; is the maximum allowed heading deviation angle; is the ship's heel angle, is the maximum permissible heel angle; is the ship's trim angle, is the maximum permissible trim angle; Indicates the current speed of the ship. is the set maximum safe speed; is the height difference between the highest point of the mast and the water surface. is the bridge clearance height.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] The present invention achieves all-weather high-precision monitoring in fog and haze, heavy rain and nighttime conditions through the deep integration of visible light and infrared dual-spectrum cameras, lidar and AIS, overcoming the reliance of traditional single-video or single-AIS solutions on visibility and active reporting. The system constructs a quantitative risk value based on multiple factors such as ship-bridge distance, ship speed, posture and the height difference from the highest point of the mast to the water surface, and can accurately predict the collision trend a few minutes before the ship enters the bridge area. According to the risk value, the warning level is divided into three levels: low, medium and high, and the handling strategy is matched to avoid the interference caused by "one-size-fits-all" alarms. VHF voice and bridge area sound and light signals are linked to the alarm, so that the information is directly transmitted to the ship's duty station, shortening the response time. In summary, the present invention is superior to existing bridge collision avoidance technologies in terms of monitoring accuracy, environmental adaptability, warning timeliness, alarm diversity and deployment economy. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 This is a schematic diagram of the distribution of cameras and lidar according to an embodiment of the present invention;

[0054] Figure 2 A schematic diagram of a detection range expansion algorithm according to an embodiment of the present invention;

[0055] Figure 3 This is a schematic diagram of radar angle accuracy and detection range according to an embodiment of the present invention;

[0056] Figure 4 This is a flow chart of estimating ship pose based on laser point cloud data according to an embodiment of the present invention;

[0057] Figure 5 This is a schematic diagram of the dangerous area division according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0059] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0060] Example 1:

[0061] This embodiment provides a collision avoidance warning system for a ship navigation bridge, including: a monitoring module for collecting ship information within a warning area; the ship information includes visible light and infrared images, lidar point cloud data, and AIS information; an information processing module for obtaining the ship's posture and the height difference between the highest point of the mast and the water surface based on the ship information; a dangerous water area classification module for calculating the longitudinal minimum safety distance and classifying the dangerous water area based on the longitudinal minimum safety distance; an early warning module for calculating a comprehensive risk value based on the ship's posture and the height difference between the highest point of the mast and the water surface, and classifying the early warning level based on the comprehensive risk value and the dangerous water area level of the ship's area, and performing collision avoidance warning based on the early warning level.

[0062] Further implementation is that the monitoring module includes a beam / pier monitoring unit and an AIS information collection unit; wherein the beam / pier monitoring unit is mainly composed of a laser radar and a camera, etc., which monitors the trajectory of super-high ships in the warning area in real time. The AIS information collection unit obtains the longitude and latitude position, speed, heading, maritime service identification code (MMSI), ship name and other information of the navigable ship based on AIS. The AIS installation position is arbitrary, and the camera and laser radar installation positions are as follows: Figure 1 As shown in Figure 1, the positions and numbers of cameras and lidars are set according to the length of the bridge, the lateral viewing angle of the camera, and the lateral coverage angle of the lidar. The principles are as follows: (1) All camera images can be stitched together to form a complete visible light and infrared image of the water surface; (2) All lidar images can be stitched together to form a complete laser point cloud image of the water surface.

[0063] Further implementation is that the information processing module includes: an image processing unit, used to identify the ship's position based on visible light and infrared images; a mast identification unit, used to identify the mast based on visible light and infrared images and based on lidar point cloud data; a lidar point cloud processing unit, used to estimate the ship's pose based on the lidar point cloud data; an AIS processing unit, used to extract the ship's position based on AIS information and obtain ship trajectory data based on the AIS information.

[0064] The single-stage detector YOLO-G model is used to identify ship positions in visible light and infrared images captured by cameras. To achieve real-time ship recognition and analysis and high recognition accuracy for small targets, the single-stage detector YOLO-G model was developed by improving the YOLOv7 network structure and recognition algorithm. This model consists of two main components: the Darknet-65 feature extractor and the YOLO interaction layer.

[0065] (1) Feature Extractor: The Darknet-53 network outputs three feature maps of different sizes, with the largest feature map size being 52×52. This network can complete the task of recognizing regular-sized ship targets, but it cannot complete the task of detecting ship targets when the resolution of the recognition sample is low or the size of the ship target is extremely small. The network structure of the original Darknet-53 network is improved to obtain Darknet-65. For visible and infrared images with an input resolution of 416×416 and a number of channels of 3, a structure consisting of 12 convolutional layers and 6 residual (res) layers is constructed in the deep network architecture design to achieve the output of four feature maps of different scales. All convolutional layers perform batch normalization and activation function operations in sequence to improve the model stability and nonlinear expression ability. Specifically, the first convolutional layer is configured with 32 3×3 convolution kernels to extract features from the original image data. Its output is used as the input of the second layer, which uses 64 3×3 convolution kernels and sets the stride to 2 to complete the downsampling operation. The subsequent structure draws on the residual connection mechanism of YOLOv7, forming a residual convolution group by alternately stacking 3×3 and 1×1 convolution layers, and finally outputting a feature map of size 208×208. In the residual convolution group:

[0066] ; (1)

[0067] Where, Feature map for network input; For the The network output of the layer; Represents the splicing between feature maps; ( ) A combination of batch normalization, activation function, and convolution operation to achieve the first The nonlinear transformation of the layer. The operation of ( ) is: Conv(1,1)-BN-Relu-Conv(3,3)-BN-Relu, and four groups of residual convolution groups are further constructed. The downsampling positions are 4 times, 8 times, 16 times, and 32 times, respectively, corresponding to the output feature maps of size 104×104, 52×52, 26×26, and 13×13. The feature maps are fused through the upsampling layer to construct a feature pyramid structure to achieve extremely small ship target recognition.

[0068] (2) YOLO interaction layer: In deep convolutional neural networks, the target coordinate information of shallow features is accurate but lacks semantic information, while deep features are semantically rich but have the problem of rough coordinate positioning. To this end, YOLO-G adopts a multi-scale prediction strategy, dividing the YOLO interaction layer into 4 independent detection branches, each branch contains 6 convolutional layers, and performs a 2x upsampling operation to enhance the feature pyramid learning ability. Subsequently, each branch is connected to the upsampling layer through tensor splicing to achieve interaction between feature maps at different levels, thereby highly fusing the shallow position information of the input image with the deep semantic information, helping the network learn fine-grained features, obtain more valuable semantic content, and efficiently regress the ship bounding box and category label. The multi-scale feature fusion formula is:

[0069] ; (2)

[0070] ; (3)

[0071] Where, Indicates the feature map that needs to be fused; Represents an upsampling operation; Represents the concatenation of feature map tensors at different scales; Represents the fused feature map; Indicates the reconstruction and recognition of the obtained fused feature map; Represents the reconstructed recognized feature map.

[0072] After multi-scale feature fusion processing, the YOLO interaction layer ultimately generates feature maps of four different scales: 13×13, 26×26, 52×52, and 104×104. Since feature map size is negatively correlated with receptive field size, these four scales correspond to detecting large, medium, small, and very small objects, respectively. During actual detection, taking an S×S feature map as an example, the network divides the ship image into S×S grid cells. When the center coordinates of an object fall within a grid cell, that grid cell is responsible for detecting the object. Each grid cell outputs 4D coordinate information and the corresponding category label.

[0073] In the task of identifying small ship targets, it's difficult to distinguish between foreground and background in the input data. Furthermore, the target's proportion in the image is much smaller than the background, resulting in a dataset dominated by negative samples. This sample imbalance makes it difficult for the network to learn effective information, leading to degradation in model performance. To address this, a modulation loss function is used:

[0074] ; (4)

[0075] Where, is the loss value after modulation, is a coefficient ranging from 0 to 1, is a parameter greater than zero, is the probability of the correct category.

[0076] By constructing a feature extractor and YOLO interaction layer to complete target feature extraction, obtain the target's coordinate position information and category label, introduce an improved loss function, and improve the recognition accuracy and recognition precision of the network model, thus forming a new single-stage detector YOLO-G model.

[0077] For the detection of small targets such as masts, this embodiment uses an improved algorithm based on YOLOv7 to identify masts in visible light and infrared images. YOLOv7 is a detection model based on anchor frames. It has a small number of parameters and occupies little memory space, making it suitable for deployment on edge devices. The network model is mainly divided into three parts: 1) The backbone network is used to extract features at different output scales; 2) The feature fusion network is mainly responsible for feature fusion; 3) The detection head is responsible for the prediction output of the results.

[0078] The algorithm structure is adjusted by combining the prior frame and model scale. Through the idea of ​​multi-scale fusion detection, the ship data set is clustered using the K-means clustering algorithm. In the K-means algorithm, a fixed number of clusters is Under this circumstance, the optimal centroid is solved by minimizing the overall sum of squares, and anchor box clustering is achieved based on the optimal centroid. The K-Means clustering calculation formula using the CIoU distance metric is:

[0079] ; (5)

[0080] Where, is the average of the center of mass and all target boxes Score; is the true target box; is the centroid of the cluster; is the similarity factor that measures the aspect ratio, is the weight coefficient, is the Euclidean distance between the center of mass and the center point of the target box; is the minimum enclosing rectangle diagonal distance between the anchor box and the target box; Represent and compare.

[0081] The attention mechanism helps neural networks accurately focus on important local feature areas and has become a research direction for improving the performance of deep neural networks. Commonly used attention mechanisms include spatial attention and channel attention, which aim to capture pixel-level pairwise relationships and channel dependencies, respectively. Typically, a fusion of the two achieves better feature extraction performance, but this can increase the computational burden. This paper introduces a more efficient SA mechanism that does not incur additional computational overhead. The idea of ​​group convolution is used to enable channel and spatial attention to work synergistically. The specific process is as follows:

[0082] 1) Group operation. Get The purpose of grouping different feature maps is to reduce the amount of computation of the model and speed up the operation of the network. After the grouping operation, the width and height of the feature layer remain unchanged, and the number of channels becomes ,in, Indicates the number of channels in the input feature map. Each group is processed using the SA unit. This unit contains both a channel attention mechanism and a spatial attention mechanism. The former is implemented similarly to a squeeze-and-excitation (SE) operation, while the latter uses a group normalization (GN) operation to achieve feature extraction.

[0083] 2) Concatenation: The feature maps processed by the SA unit are concatenated using the Concat method to achieve information fusion and interaction within the group.

[0084] 3) Shuffle operation: Use shuffle to rearrange the groups to allow information to flow between different groups and output a feature layer with an attention mechanism.

[0085] To address the problem of insufficient learning features for small objects, existing research has mostly used conventional image enhancement techniques. This approach provides limited small object features and cannot meet the needs of this scenario. To this end, this embodiment uses a negative data enhancement method that takes advantage of the prominence of image edge information, combines the negative effect with image sharpening mechanisms, and expands more effective samples. The specific process is as follows:

[0086] Input: Inland waterway shipping dense small target ship data set .

[0087] Output: original dataset pixel information set ,in: is the width and height of the sample; is the number of samples. The data set after grayscale image conversion is , the negative data set is , the data set after Gaussian blur processing is , the target data set after USM sharpening is .

[0088] Step 1: When the collection When not empty, repeat the following steps.

[0089] Step 2: Get Pixel information of the sample in , and then convert the sample to be processed into grayscale image .

[0090] Step 3: By Traverse the pixels in the grayscale image of the dataset one by one, and then use the following formula:

[0091] ; (6)

[0092] Where: For collection Zhongzai Gray value of the pixel before inversion; For collection Zhongzai The grayscale value of the pixel after inversion.

[0093] Pair Collection Invert the grayscale value of each pixel in the set .

[0094] Step 4: In the collection Gaussian blur processing is used to obtain the set .

[0095] Step 5: In the collection The USM sharpening enhancement algorithm is used in the image to obtain the set , the specific formula is:

[0096] ; (7)

[0097] Where: is the sample key value corresponding to the three sets; and The scaling factor for the image enhancement effect.

[0098] In this embodiment, the VoxelNet algorithm is used to detect the ship mast based on the lidar point cloud data. VoxelNet is suitable for processing sparse three-dimensional point cloud data, and can provide high detection accuracy while ensuring high real-time performance. The network structure of VoxelNet is divided into three parts: feature learning network, convolutional intermediate layer, and RPN layer. The feature learning network includes steps such as voxel segmentation, point cloud grouping, random sampling, multi-layer voxel feature encoding, and sparse tensor representation. First, a fixed voxel grid size is set and the point cloud area is divided into voxel grids. Subsequently, the point cloud is grouped according to the voxel grid, and a fixed number of points are randomly sampled within each voxel. Each convolutional intermediate layer includes a 3D convolution, batch normalization, and ReLU nonlinear activation, with Conv3D( ) describes a convolutional middle layer, Conv3D represents a three-dimensional convolution, are the number of input and output channels respectively, and the convolution kernel size is , the step size is , the filling size is The RPN network contains three fully convolutional layer blocks. The processing logic of each layer block is as follows: first, a convolution layer with a stride of 2 is used to compress the size of the input feature map to 50% of the original size; then three convolution layers with a stride of 1 are connected, and each convolution layer is followed by a batch normalization (BN) layer and a ReLU activation function to enhance the network's feature extraction efficiency and nonlinear expression capabilities. The output of each layer block is concatenated after an upsampling operation to construct a high-resolution feature map. The RPN layer eventually forms two branches, one for outputting the category probability distribution, and the other for outputting the transformation process from the anchor to the true box.

[0099] To verify the accuracy of the laser point cloud mast recognition results, this embodiment compares and fuses the image recognition results with the point cloud results. Specifically, the extracted mast point cloud is projected onto the image plane using the sensor's internal and external parameters to obtain the image pixel positions corresponding to the mast point cloud. This result is then compared with the mast pixel positions obtained based on visible light and infrared image recognition, and the degree of overlap between the two is calculated at the pixel level. When the overlap between the mast pixels in the image and point cloud data exceeds 90% (i.e., more than 90% of the image mast pixels match the mast pixel positions obtained by point cloud projection), the detection results of the two sensors are considered highly consistent. In this case, the laser point cloud data is reliable and can be used to obtain the highest point of the mast.

[0100] Under the premise that the point cloud data is verified to be reliable, the distance from the mast to the water surface is estimated by the 3D coordinates of the highest point in the mast point cloud. Specifically, when the height reference of the water surface is known in the LiDAR coordinate system (for example, the water surface is set to =0 plane), the highest point of the mast The value directly reflects its vertical height relative to the water surface; if the water surface height reference is unknown, the height of the water surface in the coordinate system can be determined by detecting the position of the horizontal plane in the point cloud or using information such as the sensor installation height, and then calculating the height difference between the highest point of the mast and the water surface. .

[0101] A further implementation is to estimate the ship's pose based on the lidar point cloud data, and the method for obtaining the point cloud pose estimation data includes:

[0102] (1) Laser point cloud segmentation algorithm design: This embodiment uses a region growing algorithm to perform laser point cloud segmentation, which is generally divided into three steps: 1. Determine the seed point for growth; 2. Establish the conditions for seed growth; 3. Set the conditions for stopping growth. Now the idea of ​​this region growing algorithm is applied to the field of point cloud segmentation. The main difference from image segmentation is the seed growth conditions. The core issue of the region growing algorithm is to establish the judgment criteria for similar regions. The judgment criteria in the field of image segmentation are mainly the grayscale difference of pixel points, while in the field of point cloud segmentation, it is more complicated. Generally, it is necessary to calculate the normal vector, curvature and other geometric parameters of each point, and to judge whether to grow by comparing the geometric parameters of two points (such as distance, curvature, normal vector, etc.). This region growing method can segment each surface in the point cloud, and the segmentation effect is very good. However, when performing region growth using this method, a large amount of calculation processing is required for the point cloud. The point cloud segmentation purpose of this embodiment is to obtain each independent ship as a whole, so the above-mentioned growth rule is obviously not suitable for the segmentation requirements. Therefore, this embodiment uses distance information to obtain the desired result. Since the LiDAR scans point cloud data sequentially, the point cloud obtained is an ordered point cloud. Therefore, we can imitate the image segmentation method to calculate the distance between two points with adjacent indexes and use the distance as the growth criterion for region growing. The steps of the region growing algorithm based on the index eight neighborhood are as follows:

[0103] S1. Traverse the point cloud data in the scanning order and find the first unattributed data point, setting this point as the seed point .

[0104] S2, seed point Centered, computing The eight neighborhood points of the index matrix ,if Satisfy the point cloud growth criteria, and into one category and Stored on the stack.

[0105] S3. Take the top point of the stack and use it as a new seed Repeat S2 and S3.

[0106] S4. When the stack is empty, it means that the growth of this area is completed, and return to S1 to find a new seed point for the new area.

[0107] S5. Repeat S1 to S4 until every data point in the index has an attribute.

[0108] The growth condition of the point cloud is the Euler distance between the candidate point and the seed point, which is calculated as follows:

[0109] ; (8)

[0110] Where, represents the Euler distance; Represents the three-dimensional coordinates of the candidate point; Represents the 3D coordinates of the seed point.

[0111] Since the minimum size of a ship at sea is more than ten meters, the distance for judging whether it is a class is set to 5 meters. When the distance is less than 5 meters, the two points are classified into one category. The region growing algorithm based on the eight-neighborhood has a fast segmentation speed, but there are over-segmentation phenomena in many places, resulting in unsatisfactory segmentation results. In order to solve the above problem, the detection range of the region growing can be expanded from the eight-neighborhood, such as Figure 2 As shown in the figure, when the seed is in block A, if method 1, that is, the eight-neighborhood method is used to grow, part A and part B will be divided into two categories. If method 2, that is, the detection range diffusion algorithm, is used to expand the search range outward, part A and part B will be connected, thereby avoiding over-segmentation.

[0112] For targets at close range, the point cloud density is very high. If the number of expansion circles is small, it may not be able to achieve good results. For targets at far distances, if the number of expansion circles is large, two adjacent ships may be separated together. In order to take into account both close and far distance ship targets, the number of expansion circles must be set to dynamic. The number of neighborhood expansion circles that should be set at different distances is calculated by fixing the chord length. Figure 3 As shown, the accuracy of the radar horizontal angle is defined as , the accuracy of the vertical angle is , the horizontal detection distance is , the vertical detection distance is , the distance between the obstacle and the radar is .

[0113] Since the accuracy requirement here is not high, for the convenience of calculation, the red arc length is used instead of the blue chord length, and the number of horizontal expansions is and vertical expansion number The calculation formula is as follows:

[0114] ; (9)

[0115] The +1 in the above formula ensures that the minimum number of expansion circles is 1, thus ensuring that at least eight neighborhoods are grown. This basically achieves the dynamic change of neighborhood range according to distance.

[0116] (2) The method of estimating the position and attitude of the ship based on the segmented laser point cloud and obtaining the point cloud pose estimation data includes: when performing pose estimation based on point cloud registration, the point cloud registration process needs to detect the feature points of the point cloud and describe the features of the feature points. The extracted feature points are generally required to have the characteristics of stability, rotation invariance, and resistance to density interference. The feature points are extracted by using the Harris3D extraction method. Harris3D is an extension of the two-dimensional feature Harris on the image in the three-dimensional model. The detection principle is: use a window to move in the point cloud, and judge whether there are feature points in the window by the change in the number of point clouds before and after the window moves. The expected formula for the number of point clouds in the window is shown below:

[0117] ; (10)

[0118] Where, The window function represents the weight of the point cloud at different positions in the window, and generally takes a Gaussian weighting function; ( ) represents a given spatial coordinate in a point cloud (or voxel grid) The attribute value at the location (such as grayscale, reflectivity, or scalar information such as distance); Indicates Displacement vectors in three directions.

[0119] Formula (10) can be expressed as the following matrix:

[0120] ; (11)

[0121] . (12)

[0122] In the above formula, Represents the pixels at Directional gradient; It contains the changes in the grayscale value when the window moves in various directions. In order to directly use numerical values ​​to represent the corner points in the window, the following formula is designed:

[0123] ; (13)

[0124] Where, is a constant; Represents the feature intensity value; det represents the matrix The determinant of Representation matrix The trace of , that is, the sum of the main diagonal elements; when When it is greater than the specified threshold and is a local maximum, the point is judged as a key point.

[0125] When performing point cloud matching through key points, it is necessary to determine which key points in the two models (the template point cloud library and the point cloud library to be registered) are one-to-one corresponding. At this time, it is necessary to use a certain form to describe the characteristics of each key point, and then achieve matching between points based on the characteristics of the key points.

[0126] The Fast Point Feature Histogram (FPFH) descriptor digitizes the geometric features of the neighborhood, such as position and curvature, and encodes them into a multidimensional histogram. The specific calculation steps are as follows:

[0127] 1) For each feature point , use kd_tree to find the distance less than Neighborhood points , and then calculate the triples between the feature points and the neighborhood points The calculation formula for triples is as follows:

[0128] ; (14)

[0129] In the above formula, is the unit vector in the three directions of the established local three-dimensional coordinates; Respectively represent the coordinate vectors of the feature point and its neighborhood points in three-dimensional space; Represent the surface normal vectors of the feature point and its neighborhood points respectively; Represents the Euclidean distance between the feature point and the neighboring points.

[0130] 2) For all feature points Distance less than Neighborhood points Follow the method in step 1) to take these neighborhood points Search for a center with a distance less than 's neighborhood points, and then calculate the SPFH of the center point and the neighborhood points.

[0131] 3) Feature points SPFH and weighted neighborhood points The SPFH is counted to get the final FPFH descriptor. The formula is as follows:

[0132] ; (15)

[0133] In the above formula is the weight value, generally related to the key point With neighboring points The distance between them is related; Indicates the number of neighborhood points involved in the calculation.

[0134] The principle of achieving six-degree-of-freedom pose estimation of a ship through point cloud registration is as follows: first, create a template point cloud of the target ship, then make the template point cloud coincide with the point cloud to be measured through coordinate transformation, and finally estimate the pose of the point cloud to be measured by using the initial pose information of the template point cloud and the coordinate transformation matrix. The flowchart is as follows: Figure 4 As shown, the specific implementation steps are as follows:

[0135] 1) Obtain the complete point cloud of the target ship, and then build a template library of the target ship point cloud based on the feature points and feature histograms of the complete point cloud.

[0136] 2) Calculate the feature points and feature histogram of the point cloud to be registered.

[0137] 3) Coarse registration process: The feature points of the template point cloud library and the point cloud library to be registered are matched one by one, and the SAC-IA point cloud registration algorithm is used to overlap the feature points in the template point cloud library with the corresponding feature points in the point cloud library to achieve coarse registration of the point cloud.

[0138] 4) Fine registration process: The ICP point cloud registration algorithm is used to achieve precise registration of the point cloud based on the coarse registration, ultimately obtaining the precise coordinate transformation matrix of the point cloud and estimating the six-degree-of-freedom pose of the ship.

[0139] The core task of the AIS processing unit is to extract the vessel's location and trajectory from AIS messages. AIS messages provide key data such as the vessel's real-time position (latitude and longitude), speed, heading, and the vessel's MMSI (Medium Ship Size Identity). By decoding AIS messages, the AIS processing unit extracts the vessel's latitude and longitude, speed, and heading at a specific point in time, thereby accurately determining the vessel's current position.

[0140] Because AIS data may contain errors and noise, the decoded data needs to be preprocessed to ensure the accuracy of the extraction results. First, invalid MMSI codes are filtered out and messages with a length of less than 9 digits are eliminated. Second, the latitude and longitude data are checked for rationality and invalid location data that exceeds the preset geographical range is eliminated. For longitude and latitude with abnormal jumps, coordinates outside the reasonable range are eliminated or corrected according to the maximum speed limit of the ship. In the event of random errors in speed and heading, interpolation and repair are performed based on the longitude and latitude of the previous and next time nodes to ensure the continuity and consistency of the ship's speed and heading data.

[0141] After data cleaning and repair, the AIS processing unit attributes position information at different points in time to the same vessel based on the vessel's MMSI code, thereby generating a complete vessel trajectory. For vessels sailing intermittently, the system segments the trajectory based on a set time interval threshold to ensure that each segment of navigation data is continuous and stable. Through these processes, the AIS processing unit can provide reliable vessel position information and accurate navigation trajectory, providing data support for subsequent navigation analysis and monitoring.

[0142] In this embodiment, the method for dividing the dangerous water area into levels includes: this embodiment uses a weighted fusion method to fuse the longitudinal minimum safety distances obtained by two collision avoidance methods (a collision avoidance method based on dynamic trajectories and a collision avoidance method based on static rules), and divides the dangerous areas of the bridge area into these levels based on this.

[0143] The method for calculating the first longitudinal minimum safe distance and the ship width in the channel direction based on the collision avoidance method of dynamic trajectory is as follows:

[0144] ; (16)

[0145] ; (17)

[0146] in, Indicates the dynamic longitudinal minimum safety distance, Indicates the ship width in the dynamic channel direction, Indicates the current speed of the ship. represents the reaction time window, Indicates the human-machine reaction time, It indicates the effective deceleration (absolute value) that can be achieved when the ship takes emergency deceleration. Indicates the captain, Indicates the width of the ship. Indicates the angle between the heading and the course axis.

[0147] The method for calculating the second longitudinal minimum safe distance and the ship width in the channel direction based on the static rule collision avoidance method is as follows:

[0148] ; (18)

[0149] ; (19)

[0150] in, Indicates the static longitudinal minimum safety distance, Indicates the ship width in the static channel direction, They are the safety factors required for uplink and downlink respectively, It represents the empirical coefficient, usually between 1.1 and 1.5. Its value varies according to environmental conditions (such as water flow, wind force, etc.) and ship type. Table 1 below shows the corresponding coefficients for some ship types. Value range.

[0151] Table 1 Ship type corresponding to Value range

[0152]

[0153] The minimum longitudinal safety distance and the final channel direction ship width obtained by the weighted fusion formula of the two collision avoidance methods are as follows:

[0154] ; (20)

[0155] ; (twenty one)

[0156] in, and Represents the weighting coefficient. The greater the ship speed, the more reliable the real-time data. and The bigger.

[0157] like Figure 5 As shown in the figure, the longitudinal minimum safety distance obtained by weighting the two collision avoidance methods and the final channel direction ship width are used to determine the first-level dangerous area in the bridge area (where The second level of the bridge area is inflated by 1.5 times the first level, and the third level is inflated by 2 times. The average of the ship's position obtained by the three aforementioned methods (visible light and infrared, lidar, and AIS) is used to determine the ship's location in the danger zone. The specific ranges of the danger zones are shown in Tables 2-4.

[0158] Table 2 Scope of Level 1 Dangerous Area

[0159]

[0160] Table 3 Scope of secondary hazardous areas

[0161]

[0162] Table 4 Scope of Level 3 Dangerous Area

[0163]

[0164] The early warning module obtains a comprehensive risk value and divides the warning level based on the comprehensive risk value and the level of dangerous waters. The method of performing anti-collision warning based on the warning level includes: when the system detects a ship with a collision risk, it can quickly, accurately and hierarchically issue warning instructions to remind the target ship to take evasive measures, thereby effectively reducing the probability of an accident. The risk value is introduced to further quantify the actual risk of ship collision. The risk value comprehensively considers multiple factors such as the ship's heading deviation, attitude angle, height difference from the highest point of the mast to the water surface, and bridge clearance to calculate a comprehensive risk score. Comprehensive risk value The calculation formula is as follows:

[0165] ; (twenty two)

[0166] in, is the risk zone coefficient, which is determined according to the risk zone where the ship is located. For example, the first-level risk zone is , the second-level risk area , the third-level risk area ,in > > . is the heading deviation risk factor, reflecting the angular deviation between the ship's heading and the channel axis; is the heel angle risk factor, which indicates the risk of the ship swaying laterally; is the trim risk factor, which indicates the risk of the ship tilting longitudinally; is the clearance margin risk factor, which represents the difference between the height difference (the highest point of the mast and the water surface) and the bridge clearance; is the speed risk factor, which reflects the impact of the ship's navigation speed on the collision risk; is the weight coefficient of each risk factor, which sums up to 1. The following is the calculation of each risk factor:

[0167] ; (twenty three)

[0168] ; (twenty four)

[0169] in, is the ship’s heading deviation angle, is the maximum allowed heading deviation angle (e.g. 30°); is the ship's heel angle, is the maximum permissible heel angle (e.g. 10°); is the ship's trim angle, is the maximum permissible trim angle (e.g. 5°); is the current speed of the ship, is the set maximum safe speed; is the height difference between the highest point of the mast and the water surface. is the bridge clearance height.

[0170] According to the comprehensive risk value and risk area, the warning level is divided into three levels: low, medium and high. Exceeding the threshold When the ship is in the secondary risk area or the comprehensive risk value is Exceeding the threshold When the ship is in the first-level risk area or the comprehensive risk value is Exceeding the threshold When a high-level warning is triggered, the VHF radio broadcast and the strong sound and light alarm system will be activated to provide a comprehensive warning and recommend avoidance measures (such as changing the ship's route and adjusting the ship's speed).

[0171] Through a multi-mode, multi-level early warning mechanism, the system can monitor the process of ships passing through bridges around the clock, warn of potential bridge collision risks, and provide solid protection for bridge structures and shipping safety.

[0172] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A ship navigation bridge anti-collision warning system, characterized in that: include: A monitoring module is used to collect ship information within the warning area; the ship information includes visible light and infrared images, lidar point cloud data, and AIS information; An information processing module, configured to obtain a ship posture and a height difference between a highest point of the mast and a water surface based on the ship information; A dangerous water area classification module is used to calculate the minimum longitudinal safety distance and classify the dangerous water area based on the minimum longitudinal safety distance; An early warning module is used to calculate a comprehensive risk value based on the ship's posture and the height difference from the highest point of the mast to the water surface, and to divide the early warning level based on the comprehensive risk value and the level of dangerous waters in the area where the ship is located, and to perform anti-collision early warning based on the early warning level.

2. A ship navigation bridge anti-collision warning system according to claim 1, characterized in that: The information processing module includes: An image processing unit for identifying the position of a ship based on visible light and infrared images; A mast recognition unit, used to recognize masts based on visible light and infrared images and based on lidar point cloud data; LiDAR point cloud processing unit, used to estimate the ship's position and posture based on LiDAR point cloud data; The AIS processing unit is used to extract the ship's position and track data based on the AIS information.

3. A ship navigation bridge anti-collision warning system according to claim 2, characterized in that: The method for identifying the position of a ship based on visible light and infrared images includes: using a single-stage detector YOLO-G model to identify the position of the ship; The single-stage detector YOLO-G model includes a feature extractor and a YOLO interaction layer; The feature extractor is improved based on the Darknet-53 network. The improvement method includes: For visible light and infrared images with an input resolution of 416×416 and 3 channels, the deep network architecture design constructs a structure consisting of 12 convolutional layers and 6 residual layers, obtaining output feature maps of 4 different scales. All convolutional layers perform batch normalization and activation function operations in sequence. The first convolutional layer is configured with 32 3×3 convolution kernels to extract features from visible light and infrared images. The output is used as the input of the second layer, which uses 64 3×3 convolution kernels with a step size of 2 to complete the downsampling operation. Based on the residual connection mechanism of YOLOv7, a residual convolution group is formed by alternating stacking of 3×3 and 1×1 convolution layers, and the final output feature map of size 208×208 is obtained. In the residual convolution group: ； Where, Feature map for network input; For the The network output of the layer; Represents the splicing between feature maps; ( ) is a combination function of batch normalization, activation function and convolution operation, which is used to implement the first Nonlinear transformation of layers; in, The operation of ( ) is: Conv(1,1)-BN-Relu-Conv(3,3)-BN-Relu, further constructing 4 groups of residual convolution groups, and downsampling at 4 times, 8 times, 16 times, and 32 times, respectively, corresponding to the output size of 104×104, 52×52, 26×26, and 13×13 feature maps, and the feature maps are fused through the upsampling layer to construct a feature pyramid structure; The YOLO interaction layer is divided into four independent detection branches, each of which contains six convolutional layers and performs a 2x upsampling operation. Each branch is connected to the upsampling layer through tensor splicing to complete the multi-scale feature fusion of the shallow position information and deep semantic information of the input image. The multi-scale feature fusion formula is: ； ； Where, Indicates the feature map that needs to be fused; Represents an upsampling operation; Represents the concatenation of feature map tensors at the same scale; Represents the fused feature map; Indicates the reconstruction and recognition of the obtained fused feature map; Represents the feature map after reconstruction and recognition; After multi-scale feature fusion, the YOLO interaction layer finally outputs four feature maps with scales of 13×13, 26×26, 52×52, and 104×104.

4. The ship navigation bridge anti-collision warning system according to claim 2, characterized in that: The method for estimating a ship's pose based on laser radar point cloud data comprises: segmenting the laser radar point cloud data using a region growing algorithm based on an indexed eight-neighborhood to obtain a segmented point cloud; and estimating a ship's pose based on the segmented laser radar point cloud data. The process of cutting the laser radar point cloud data using the region growing algorithm based on the indexed eight-neighborhood includes: S1. Traverse the lidar point cloud data in scanning order and find the first unattributed data point, which is set as the seed point ; S2, seed point Centered, computing The eight neighborhood points of the index matrix ,if Satisfy the point cloud growth criteria, and into one category and Save in the stack; S3. Take the top point of the stack and use it as the new seed point Repeat S2 and S3; S4: When the stack is empty, it means that the region has grown. Return to S1 and look for a new seed point for the new region. S5. Repeat S1 to S4 until every data point in the index has an attribute.

5. The ship navigation bridge anti-collision warning system according to claim 2, characterized in that: The method for calculating the longitudinal minimum safety distance by the dangerous water area demarcation module includes: calculating the first longitudinal minimum safety distance by a collision avoidance method based on a dynamic trajectory and calculating the second longitudinal minimum safety distance by a collision avoidance method based on a static rule; Calculate the first longitudinal minimum safe distance based on the collision avoidance method of dynamic trajectory The calculation methods include: ； Where, Indicates the current speed of the ship; represents the reaction time window; Indicates the human-machine reaction time, Indicates the effective deceleration that can be achieved by the ship when emergency deceleration is adopted. Indicates the captain; Calculation of the second longitudinal minimum safety distance based on the collision avoidance method of static rules The calculation methods include: ； Where, 、 Respectively represent the safety factors required for uplink and downlink; The first longitudinal minimum safety distance and the second longitudinal minimum safety distance are weightedly integrated to obtain the longitudinal minimum safety distance: ； Where, Represents the weighting coefficient.

6. The ship navigation bridge anti-collision warning system according to claim 2, characterized in that: The calculation method of the comprehensive risk value includes: ； Where, is the risk zone coefficient; is the heading deviation risk factor; is the heel angle risk factor; is the pitch angle risk factor; is the headroom risk factor; is the speed risk factor; is the weight coefficient of the risk factor, which adds up to 1.

7. A ship navigation bridge anti-collision warning system according to claim 6, characterized in that: The risk factor calculation method includes: ； ； Where, is the ship’s course deviation angle; is the maximum allowed heading deviation angle; is the ship's heel angle, is the maximum permissible heel angle; is the ship's trim angle, is the maximum permissible trim angle; Indicates the current speed of the ship. is the set maximum safe speed; is the height difference between the highest point of the mast and the water surface. is the bridge clearance height.

Images