A ship navigation bridge anti-collision early warning system

Abstract

The application discloses a kind of ship navigation bridge anti-collision early warning system, belong to bridge anti-collision technical field.It includes: monitoring module, for collecting the ship information in early warning area;Information processing module, for obtaining ship pose, mast identification and the height difference of mast highest point to water surface based on ship information;Dangerous water area division module, for calculating longitudinal minimum safety distance, based on longitudinal minimum safety distance to divide dangerous water area grade;Early warning module, for obtaining comprehensive risk value based on ship pose, mast identification and the height difference of mast highest point to water surface;And based on comprehensive risk value and dangerous water area grade division early warning grade, based on early warning grade to carry out anti-collision early warning.The application is superior to existing bridge anti-collision technology in monitoring accuracy, environmental adaptability, early warning timeliness, alarm diversity and deployment economy.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of bridge anti-collision, and particularly relates to a ship navigation bridge anti-collision early warning system. BACKGROUND

[0002] The current mainstream bridge anti-collision technology mainly includes passive protection and active early warning. The passive protection technology mainly uses physical facilities (such as anti-collision fenders) to absorb collision energy and reduce bridge damage, but the passive protection cannot avoid accidents and can only reduce the consequences. The current active early warning technology mainly relies on a single technical route of video monitoring and AIS (ship automatic identification system). The traditional video monitoring is easily disturbed by fog, heavy rain, night low light and other environmental interference, and it is difficult to accurately capture the ship dynamic at all times. Although the AIS system can provide ship identity and trajectory data, it has the defect of serious dependence on active reporting of the ship. Therefore, it is of great theoretical research and engineering practical significance to study a ship navigation bridge anti-collision early warning system with high monitoring accuracy, timely and accurate early warning, and to reduce the occurrence of ship-bridge collision accidents.

[0003] For the ship navigation bridge anti-collision early warning system, the ship-bridge collision avoidance monitoring and early warning system research based on intelligent video processing technology, combined with maneuvering theory and motion mathematical model, carries out collision risk prediction, but the monitoring accuracy needs to be improved. The bridge anti-ship collision active early warning system design and experimental research based on video monitoring technology uses infrared, visible light and laser composite detection and multi-source information fusion to design the bridge anti-ship collision active early warning system, but the alarm mode is single and lacks a hierarchical response mechanism. SUMMARY

[0004] The application aims to solve the problems of the prior art and provides a ship navigation bridge anti-collision early warning system, which comprises:

[0005] A monitoring module is configured to collect ship information in a warning area; the ship information includes visible light and infrared images, laser radar point cloud data and AIS information;

[0006] An information processing module is configured to obtain a ship pose and a height difference between a highest point of a mast and a water surface based on the ship information;

[0007] A dangerous water area division module is configured to calculate a longitudinal minimum safety distance and divide a dangerous water area level based on the longitudinal minimum safety distance;

[0008] An early warning module is configured to calculate a comprehensive risk value based on the ship pose and the height difference between the highest point of the mast and the water surface, divide an early warning level based on the comprehensive risk value and a dangerous water area level of an area where the ship is located, and perform anti-collision early warning based on the early warning level.

[0009] Further preferably, the information processing module comprises:

[0010] an image processing unit configured to identify a ship position based on the visible light and infrared images;

[0011] a mast identification unit configured to identify a mast based on the visible light and infrared images and based on the lidar point cloud data;

[0012] a lidar point cloud processing unit configured to perform ship pose estimation based on the lidar point cloud data;

[0013] an AIS processing unit configured to extract ship position and ship trajectory data based on the AIS information.

[0014] Further preferably, the method of identifying a ship position based on visible light and infrared images comprises: identifying a ship position using a single-stage detector YOLO-G model;

[0015] the single-stage detector YOLO-G model comprises a feature extractor and a YOLO interaction layer;

[0016] the feature extractor is improved based on a Darknet-53 network, and the improvement method comprises:

[0017] for an input visible light and infrared image with a resolution of 416x416 and a channel number of 3, in the network deep architecture design, a structure containing 12 convolutional layers and 6 residual layers is constructed to obtain the output of 4 different scale feature maps; all convolutional layers sequentially perform batch normalization and activation function operations, the first convolutional layer is configured with 32 3x3 convolutional kernels, and the visible light and infrared image is subjected to feature extraction; the output is taken as the input of the second layer, the second layer uses 64 3x3 convolutional kernels and sets the step to 2 to complete the down-sampling operation; based on the residual connection mechanism of YOLOv7, residual convolution groups are formed by alternately stacking 3x3 and 1x1 convolutional layers, and finally the feature map with a size of 208x208 is output; in the residual convolution group:

[0018] ;

[0019] wherein, is a feature map of network input; is a network output of the i-th layer; is a network output of the i-th layer; represents splicing between feature maps; is a combination function of batch normalization, activation function and convolution operation, used to realize nonlinear transformation of the i-th layer; is a combination function of batch normalization, activation function and convolution operation, used to realize nonlinear transformation of the i-th layer;

[0020] wherein, The operation of ( ) is: Conv(1,1)-BN-Relu-Conv(3,3)-BN-Relu, which further constructs 4 groups of residual convolutional groups, and outputs feature maps of size 104×104, 52×52, 26×26, and 13×13 respectively at downsampling positions of 4x, 8x, 16x, and 32x. The feature maps are then fused through upsampling layers to construct a feature pyramid structure.

[0021] The YOLO interaction layer is divided into four independent detection branches, each containing six convolutional layers and undergoing a 2x upsampling operation. Each branch is connected to the upsampling layer via tensor concatenation, completing multi-scale feature fusion of shallow positional information and deep semantic information of the input image. The multi-scale feature fusion formula is as follows:

[0022] ;

[0023] ;

[0024] In the formula, This represents the feature maps that need to be fused. Indicates an upsampling operation; This represents the concatenation of feature map tensors at the same scale. This represents the fused feature map; This indicates the reconstruction and recognition of the obtained fused feature map; This represents the feature map after reconstruction and recognition.

[0025] After multi-scale feature fusion, the YOLO interaction layer finally outputs feature maps with four scales: 13×13, 26×26, 52×52, and 104×104.

[0026] More preferably, the method for ship pose estimation based on lidar point cloud data includes: segmenting the lidar point cloud data using an indexed eight-neighborhood-based region growing algorithm to obtain segmented point clouds; and estimating ship pose based on the segmented lidar point cloud data.

[0027] The process of segmenting the lidar point cloud data using the region growing algorithm based on indexed eight neighborhoods includes:

[0028] S1. Traverse the LiDAR point cloud data in scanning order, find the first unassigned data point, and set it as the seed point. ;

[0029] S2, using seed point Calculate with center The eight neighborhood points of the index matrix ,if Satisfy the point cloud growth criterion, will With As a class, and will Save in the stack;

[0030] S3, the point at the top of the stack, take it as a new seed Repeat S2, S3;

[0031] S4, when the stack is empty, indicating that the region growth is complete, return to S1, looking for a new seed point of the new region;

[0032] S5, repeat S1-S4, until the index of each data point has a home growth ends.

[0033] Further preferably, the dangerous water area division module calculates the longitudinal minimum safety distance method comprising: based on the dynamic trajectory collision avoidance mode to calculate the first longitudinal minimum safety distance and based on the static rule collision avoidance mode to calculate the second longitudinal minimum safety distance;

[0034] Based on the dynamic trajectory collision avoidance mode to calculate the first longitudinal minimum safety distance The calculation method comprises:

[0035] ;

[0036] In the formula, Indicates the current speed of the ship; Indicates the reaction time window; Indicates the man-machine reaction time, Indicates the effective deceleration that the ship can achieve when taking emergency deceleration, Indicates the length of the ship;

[0037] Based on the static rule collision avoidance mode to calculate the second longitudinal minimum safety distance The calculation method comprises:

[0038] ;

[0039] In the formula, , Indicate the safety factor required when going up and down, respectively;

[0040] The first longitudinal minimum safety distance and the second longitudinal minimum safety distance are weighted and fused to obtain the longitudinal minimum safety distance:

[0041] ;

[0042] In the formula, Indicates the weighting coefficient.

[0043] Further preferably, the method for calculating the comprehensive risk value comprises:

[0044] ;

[0045] wherein, is a risk zone coefficient; is a heading deviation risk factor; is a roll angle risk factor; is a pitch angle risk factor; is a clearance margin risk factor; is a speed risk factor; is a weight coefficient of the risk factor, and the sum is 1.

[0046] Further preferably, the method for calculating the risk factor comprises:

[0047] ;

[0048] ;

[0049] wherein, is a heading deviation angle of the ship; is a maximum allowed heading deviation angle; is a roll angle of the ship, is a maximum allowed roll angle; is a pitch angle of the ship, is a maximum allowed pitch angle; represents a current speed of the ship, is a set maximum safe speed; is a height difference between the highest point of the mast and the water surface, is a clearance height of the bridge.

[0050] Compared with the prior art, the application has the following beneficial effects:

[0051] The application realizes all-weather high-precision monitoring under fog, heavy rain and night conditions through the deep integration of visible light-infrared dual-spectrum cameras, laser radars and AIS, and overcomes the dependence on visibility and active reporting of the traditional single video or single AIS scheme. The system constructs a quantitative risk value based on multiple factors such as ship-bridge distance, ship speed, pose and the height difference between the highest point of the mast and the water surface, and can accurately predict the collision trend several minutes before the ship enters the bridge area. According to the risk value, the warning levels are divided into three levels of low, medium and high, and the disposal strategies are matched, so as to avoid the interference caused by the "one-size-fits-all" alarm. The VHF voice and bridge area sound and light signal linkage alarm makes the information directly reach the ship's duty station, shortens the response time. In summary, the application is superior to the existing bridge anti-collision technology in terms of monitoring accuracy, environmental adaptability, early warning timeliness, alarm diversity and deployment economy. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the present application, the drawings required to be used in the embodiments are briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0053] Figure 1 A distribution diagram of a camera and a laser radar of an embodiment of the present application;

[0054] Figure 2 A detection range expansion algorithm diagram of an embodiment of the present application;

[0055] Figure 3 A radar angle precision and detection range diagram of an embodiment of the present application;

[0056] Figure 4 A flowchart of estimating a ship pose based on laser point cloud data of an embodiment of the present application;

[0057] Figure 5 A dangerous area division diagram of an embodiment of the present application. DETAILED DESCRIPTION

[0058] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0059] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0060] Embodiment one:

[0061] The embodiment provides a ship navigation bridge anti-collision early warning system, which comprises: a monitoring module, which is used for collecting ship information in a warning area; the ship information comprises visible light and infrared images, laser radar point cloud data and AIS information; an information processing module, which is used for obtaining a ship pose and a height difference from a highest point of a mast to water surface based on the ship information; a dangerous water area division module, which is used for calculating a minimum longitudinal safety distance and dividing a dangerous water area level based on the minimum longitudinal safety distance; and an early warning module, which is used for calculating a comprehensive risk value based on the ship pose and the height difference from the highest point of the mast to the water surface, dividing an early warning level based on the comprehensive risk value and a dangerous water area level of an area where the ship is located, and performing anti-collision early warning based on the early warning level.

[0062] Further implementation is that the monitoring module comprises a beam body / bridge pier monitoring unit and an AIS information acquisition unit; the beam body / bridge pier monitoring unit is mainly composed of a laser radar and a camera, and is used to monitor the trajectory of the super-high ship in the early warning area in real time. The AIS information acquisition unit obtains the latitude and longitude position, speed, heading, water business identification code (MMSI), ship name and other information of the ship based on AIS. The AIS installation position is arbitrary, and the camera and laser radar installation positions are as shown in Figure 1 The positions and quantities of the camera and laser radar are set according to the length of the bridge, the lateral visual angle of the camera and the lateral coverage angle of the laser radar, and the principles are as follows: (1) the images of all cameras can splice the complete visible light and infrared images of the water surface; (2) the images of all laser radars can splice the complete laser point cloud image of the water surface.

[0063] Further implementation is that the information processing module comprises: an image processing unit, which is used to identify the ship position based on the visible light and infrared images; a mast recognition unit, which is used to identify the mast based on the visible light and infrared images and based on the laser radar point cloud data; a laser radar point cloud processing unit, which is used to perform ship pose estimation based on the laser radar point cloud data; and an AIS processing unit, which is used to extract the ship position based on the AIS information and obtain the ship trajectory data based on the AIS information.

[0064] The method for identifying the ship position based on the visible light and infrared images collected by the camera by using the single-stage detector YOLO-G model is as follows: in order to realize real-time identification and analysis of the ship and high identification accuracy of small targets, the network structure and identification algorithm of YOLOv7 are improved to obtain the single-stage detector YOLO-G model. It mainly includes two parts of a feature extractor Darknet-65 and a YOLO interaction layer.

[0065] (1) Feature extractor: The Darknet-53 network outputs three feature maps of different sizes, with the largest feature map size being 52x52, which can complete the task of identifying regular-sized ship targets. However, for cases where the sample resolution is low and the ship target size is extremely small, it cannot complete the ship target detection task. The network structure of the original Darknet-53 network is improved to obtain Darknet-65. For the input of visible light and infrared images with a resolution of 416x416 and 3 channels, a structure containing 12 convolutional layers and 6 residual (res) layers is constructed in the deep network architecture design, which realizes the output of four different scale feature maps. All convolutional layers perform batch normalization and activation function operations in turn to improve the stability and nonlinear expression ability of the model. Specifically, the first convolutional layer is configured with 32 3x3 convolutional kernels to extract features from the original image data. The output of the first layer is used as the input of the second layer, which uses 64 3x3 convolutional kernels and sets the step size to 2 to complete the downsampling operation. The subsequent structure borrows the residual connection mechanism of YOLOv7, and forms residual convolution groups by alternately stacking 3x3 and 1x1 convolutional layers, finally outputting a feature map with a size of 208x208. In the residual convolution group:

[0066] ; (1)

[0067] wherein, is the feature map input to the network; is the network output of the th layer; represents the concatenation between feature maps; ( ) is a combination function of batch normalization, activation function, and convolution operation, used to realize the nonlinear transformation of the th layer. Among them, ( ) operation is: Conv(1,1)-BN-Relu-Conv(3,3)-BN-Relu, further constructing 4 groups of residual convolution groups, and respectively at 4 times, 8 times, 16 times, 32 times downsampling positions, corresponding to output feature maps with sizes of 104x104, 52x52, 26x26, 13x13, and through the upsampling layer to fuse the feature maps, construct the feature pyramid structure, and realize the identification of extremely small ship targets.

[0068] (2) YOLO interaction layer: In the deep convolutional neural network, the target coordinate information of the shallow feature is accurate but the semantic information is poor, and the deep feature is rich in semantics but has a rough coordinate positioning problem. Therefore, YOLO-G adopts a multi-scale prediction strategy, divides the YOLO interaction layer into 4 independent detection branches, each branch contains 6 convolutional layers, and performs a 2x upsampling operation to enhance the feature pyramid learning ability. Subsequently, the branches are connected to the upsampling layer through tensor splicing to realize the interaction of feature maps at different levels, thereby highly fusing the position information of the shallow layer and the semantic information of the deep layer of the input image, helping the network to learn fine-grained features and obtain more valuable semantic content, and efficiently regress the ship boundary box and class label. The multi-scale feature fusion formula is:

[0069] (2)

[0070] (3)

[0071] In the formula, represents the feature map that needs to be fused; represents the upsampling operation; represents the tensor splicing of feature maps at different scales; represents the fused feature map; represents the reconstruction recognition of the obtained fused feature map; represents the feature map after reconstruction recognition.

[0072] After multi-scale feature fusion processing, the YOLO interaction layer finally generates 4 feature maps of different scales, with sizes of 13x13, 26x26, 52x52 and 104x104. Since the feature map size is negatively correlated with the receptive field size, these 4 scales correspond to the detection of large, medium, small and extremely small targets, respectively. In actual detection, taking a feature map of size SxS as an example, the network will divide the ship image into SxS grid cells. When the target center coordinates fall within a certain grid, the grid is responsible for the detection task of the target, and each grid cell will output 4-dimensional coordinate information and corresponding class labels.

[0073] In the ship small target recognition task, there is a problem of difficulty in distinguishing between foreground and background of input data, and the proportion of the target in the image is much lower than that of the background, resulting in a dataset dominated by negative samples. This sample imbalance will make it difficult for the network to learn effective information, and thus cause the model performance to degrade. To address this issue, a modulated loss function is selected to solve the sample imbalance problem:

[0074] (4)

[0075] In the formula, is the modulated loss value, is a coefficient in the range of 0 to 1, is a parameter greater than zero, is the correct class probability.

[0076] Through the construction of feature extractor and YOLO interaction layer, the target feature extraction is completed, the coordinate position information and class label of the target are obtained, the improved loss function is introduced, the recognition accuracy and recognition accuracy of the network model are improved, so as to constitute a new single-stage detector YOLO-G model.

[0077] For the detection of small target objects such as masts, the improved algorithm based on YOLOv7 is used to identify masts in visible light and infrared images in this embodiment. YOLOv7 is an anchor-based detection model, which has less parameter quantity and occupies less memory space, and is suitable for deployment on edge devices. This network model is mainly divided into three parts: 1) the backbone network is used to extract features of different output scales; 2) the feature fusion network is mainly responsible for feature fusion; 3) the detection head is responsible for the prediction output of the result.

[0078] The algorithm structure is adjusted in combination with the prior box and the model scale, the K-means clustering algorithm is used for clustering the ship data set through the idea of multi-scale fusion detection, in the K-means algorithm, the best centroid is solved by minimizing the total sum of squares, the anchor box clustering is realized based on the best centroid, and the K-Means clustering calculation formula of the CIoU distance measurement method is:

[0079] ; (5)

[0080] In the formula, is the average score of the centroid and all target boxes; is the real target box; is the centroid of clustering; is a similarity factor for measuring the aspect ratio, is a weight coefficient, is the Euclidean distance between the centroid and the center point of the target box; is the diagonal distance of the minimum closed rectangle of the anchor box and the target box; represents the cross ratio.

[0081] ​Attention mechanism helps neural networks accurately focus on local important feature regions and has become a research direction to improve the performance of deep neural networks. The commonly used attention mechanisms mainly include spatial attention and channel attention, which aim to capture the pair-wise relationship at the pixel level and channel dependency, respectively. Usually, the fusion of the two achieves better feature extraction performance, but it easily increases the computational burden. A more efficient SA mechanism is introduced without additional computational overhead. The idea of group convolution is used to make channel and spatial attention work together. The specific process is as follows:

[0082] 1) Grouping operation. Obtain group different feature maps, the purpose is to reduce the computational amount of the model and speed up the running speed of the network. The width and height of the feature layer after the grouping operation are unchanged, and the number of channels becomes , wherein represents the number of channels of the input feature map, and each group is processed using an SA unit. The unit contains a channel attention mechanism and a spatial attention mechanism, and the former is specifically implemented similar to the squeeze-and-excitation (SE) operation, and the latter uses a group normalization operation to achieve feature extraction (group norm, GN).

[0083] 2) Splicing operation. The feature map obtained after the SA unit is spliced by using the Concat method to realize the information fusion and interaction within the group.

[0084] 3) Rearrangement operation. Shuffle is used to rearrange the groups to make information flow between different groups to output a feature layer with an attention mechanism.

[0085] To solve the problem of insufficient features of small targets, existing researches mostly use ordinary image enhancement techniques. According to this method, the small target features provided are limited and cannot meet the needs of the scene. Therefore, a negative data enhancement method is used, which takes advantage of the easy highlighting of image edge information, combines negative effects and image sharpening mechanisms, and expands more effective samples. The specific process is as follows:

[0086] Input: Inland navigation dense small target ship data set .

[0087] Output: Original data set pixel information set , wherein is the width and height of the sample; is the number of samples. The data set after gray scale conversion is , the negative data set is , the data set after Gaussian blur processing is , and the target data set after USM sharpening is .

[0088] Step 1: When the set If not empty, repeat the following steps.

[0089] Step 2: Obtain Pixel information of medium samples Then, the sample to be processed is converted into a grayscale image. .

[0090] Step 3: From After iterating through all the pixels in the grayscale image of the dataset, the following formula is used:

[0091] ; (6)

[0092] In the formula: For set In China The grayscale value of a pixel before it is reversed; For set In China The grayscale value after the pixel is inverted.

[0093] For sets Invert the grayscale value of each pixel to obtain the set. .

[0094] Step 4: In the set Gaussian blur is used to obtain the set. .

[0095] Step 5: In the set The Unsharp Mask (USM) image sharpening enhancement algorithm is used to obtain a set. The specific formula is as follows:

[0096] ; (7)

[0097] In the formula: The sample key value corresponding to the set of three; and This is the scaling factor for image enhancement.

[0098] In this embodiment, VoxelNet algorithm is used to detect the ship mast based on lidar point cloud data. VoxelNet is suitable for processing sparse three-dimensional point cloud data, and can provide high detection accuracy while ensuring high real-time performance. The network structure of VoxelNet is divided into three parts, which are feature learning network, convolutional intermediate layer and RPN layer. The feature learning network includes voxel block, point cloud grouping, random sampling, multi-layer voxel feature coding and sparse tensor representation. First, set the fixed voxel grid size and divide the point cloud area into voxel grids. Then, group the point cloud according to the voxel grid, and randomly sample a fixed number of points in each voxel. Each convolutional intermediate layer includes a 3D convolution, batch normalization and ReLU nonlinear activation. Conv3D( ) describes a convolutional intermediate layer, Conv3D represents a three-dimensional convolution, , the convolution kernel size is , the step size is , and the padding size is . The RPN network contains three fully convolutional layer blocks, and the processing logic of each layer block is as follows: first, the size of the input feature map is compressed to 50% of the original size through a convolution layer with a step size of 2; then, three convolution layers with a step size of 1 are connected, and a batch normalization (BN) layer and a ReLU activation function are connected after each convolution layer to enhance the feature extraction performance and nonlinear expression ability of the network. The output of each layer block is concatenated after upsampling operation, thereby constructing a high-resolution feature map. The RPN layer finally forms two branches, one of which is used to output the class probability distribution, and the other of which is used to output the transformation process of Anchor to the real box.

[0099] In order to verify the accuracy of the laser point cloud mast recognition result, the image recognition result and the point cloud result are compared and fused in this embodiment. Specifically, the extracted mast point cloud is projected onto the image plane using the internal and external parameters of the sensor to obtain the image pixel position corresponding to the mast point cloud. Then the result is compared with the mast pixel position obtained based on visible and infrared image recognition, and the coincidence degree of the two is calculated at the pixel level. When the coincidence degree of the mast pixels in the image and the point cloud data exceeds 90% (i.e. more than 90% of the image mast pixels coincide with the mast pixel position obtained by point cloud projection), it is considered that the detection results of the two sensors are highly consistent, and the laser point cloud data is reliable and can be used to obtain the highest point of the mast.

[0100] Under the premise that the point cloud data is verified to be reliable, the distance from the highest point of the mast point cloud to the water surface is estimated. Specifically, when the height reference of the water surface in the laser radar coordinate system is known (for example, the water surface is set as =0 plane), the The value directly reflects its vertical height relative to the water surface. If the water surface height reference is unknown, the height of the water surface in the coordinate system can be determined by detecting the position of the horizontal plane in the point cloud or by using information such as the sensor installation height. Then, the height difference between the highest point of the mast and the water surface can be calculated. .

[0101] A further implementation involves estimating the ship's pose based on lidar point cloud data. Methods for obtaining point cloud pose estimation data include:

[0102] (1) Laser Point Cloud Segmentation Algorithm Design: This embodiment uses a region growing algorithm for laser point cloud segmentation, which generally consists of three steps: 1. Determining the seed points for growth; 2. Establishing the conditions for seed growth; 3. Setting the conditions for stopping growth. Now, this idea of ​​the region growing algorithm is applied to the field of point cloud segmentation. The main difference from image segmentation is the seed growth conditions. The core issue of the region growing algorithm is to establish the criteria for judging similar regions. In the field of image segmentation, the judgment criteria are mainly the difference in gray levels of pixels. In the field of point cloud segmentation, it is more complex. Generally, it is necessary to calculate the geometric parameters such as the normal vector and curvature of each point, and to judge whether to grow by comparing the geometric parameters (such as distance, curvature, normal vector, etc.) of two points. This region growing method can segment each face in the point cloud, and the segmentation effect is very good. However, when performing region growing using this method, a lot of computational processing of the point cloud is required. The point cloud segmentation purpose of this embodiment is to obtain each independent ship whole. Therefore, the above growth rules are obviously not suitable for the segmentation requirements. Therefore, this embodiment uses distance information to obtain the desired result. Since LiDAR acquires point cloud data sequentially, the resulting point cloud is an ordered point cloud. Therefore, similar to image segmentation methods, we can calculate the distance between two adjacent indexed points and use this distance as the growth criterion for region growing. The steps of the region growing algorithm based on the index eight-neighborhood are as follows:

[0103] S1. Traverse the point cloud data in the scanning order, find the first unassigned data point, and set this point as the seed point. .

[0104] S2, using seed point Calculate with center The eight neighborhood points of the index matrix ,if To meet the criteria for point cloud growth, and Combine them into one category, and Stored in the stack.

[0105] S3. Remove the top node from the stack and use it as the new seed. Repeat S2 and S3.

[0106] S4, when the stack is empty, indicating that the region growth is complete, return to S1, looking for a new seed point of the new region.

[0107] S5, repeat S1-S4 until each data point of the index has a home, and the growth ends.

[0108] The growth condition of the point cloud is the Euler distance between the candidate point and the seed point, and the calculation formula is as follows:

[0109] (8)

[0110] In the formula, indicates the Euler distance; indicates the three-dimensional coordinates of the candidate point; indicates the three-dimensional coordinates of the seed point.

[0111] Since the smallest size of the ship on the sea is also more than ten meters, the distance of the judgment whether a class is set to 5 meters, when the Euler distance of two points is less than 5 meters, the two points are classified into one class. Based on the region growing algorithm of eight neighborhoods, the segmentation speed is very fast, but there are too many places where the segmentation phenomenon occurs, resulting in unsatisfactory segmentation effect. In order to solve the above problems, the detection range of region growing can be expanded from eight neighborhoods, as Figure 2 shown, when the seed is in block A, if method 1, i.e. eight neighborhood method growth, is used, A and B will be divided into two classes, and if method 2, i.e. detection range diffusion algorithm, is used, the search range is expanded by one more circle, A and B will be connected together, thereby avoiding over-segmentation.

[0112] For targets with a relatively short distance, the density of the point cloud is very large, and if the expansion circle number is small, it may not be improved well, and for targets with a relatively long distance, if the expansion circle number is large, it may cause two adjacent ships to be segmented together. In order to take into account the near and far distance ship targets, the expansion circle number must be dynamic. The neighborhood expansion circle number to be set at different distances is calculated by fixing the chord length. As Figure 3 shown, the accuracy of the radar horizontal angle is defined as , the accuracy of the vertical angle is defined as , the horizontal detection distance is defined as , the vertical detection distance is defined as , and the distance between the obstacle and the radar is defined as .

[0113] Since the accuracy requirement here is not high, in order to calculate conveniently, the red arc length is used instead of the blue chord length, and then the calculation formula of the horizontal expansion number and the vertical expansion number is as follows:

[0114] (9)

[0115] The +1 part in the above formula ensures that the number of extended circles is at least 1, so as to ensure that at least eight neighbors grow. So far, the range of the neighborhood can be basically realized to dynamically change with the distance.

[0116] (2) The method for estimating the position and attitude of the ship based on the segmented laser point cloud comprises: when performing point cloud registration-based pose estimation, the registration process of the point cloud needs to detect feature points of the point cloud and describe the features of the feature points. The extracted feature points generally require stability, rotation invariance, anti-density interference and the like, and a Harris 3D extraction method is used to extract the feature points. Harris 3D is an extension of two-dimensional feature Harris on an image in a three-dimensional model. The detection principle is that a window is moved in the point cloud, and whether there is a feature point in the window is judged by the number of point clouds before and after the window is moved. The expected formula of the number of point clouds in the window is as follows:

[0117] (10)

[0118] In the formula, is a window function representing the weight size of point clouds at different positions in the window, and a Gaussian weighting function is generally taken; represents the attribute value (such as gray value, reflectivity or distance scalar information) of a given spatial coordinate in the point cloud (or voxel grid); represents the displacement vector in three directions.

[0119] The formula (10) is expressed in a matrix as follows:

[0120] (11)

[0121] (12)

[0122] In the formula, respectively represent the gradient of the pixel point in direction; contains the change of the gray value when the window moves in each direction. In order to directly use numerical values to represent the corner point condition in the window, the formula is designed as follows:

[0123] (13)

[0124] In the formula, is a constant; represents the feature intensity value; and det represents a matrix The determinant of; Representation matrix The trace, that is, the sum of the elements on the main diagonal; when When a point is greater than a specified threshold and is a local maximum, it is considered a key point.

[0125] When performing point cloud matching using key points, it is necessary to determine which key points in the two models (template point cloud library and point cloud library to be matched) correspond one-to-one. This requires describing the features of each key point in a certain form, and then matching between points can be achieved based on the features of the key points.

[0126] The Fast Point Feature Histogram (FPFH) descriptor quantifies geometric features such as the location and curvature of a neighborhood and encodes them into a multidimensional histogram. The specific calculation steps are as follows:

[0127] 1) For each feature point Find the distance to it using the kd_tree. neighborhood points Then calculate the triples between the feature point and its neighboring points. The formula for calculating a triple is as follows:

[0128] ;(14)

[0129] In the above formula, It is a unit vector in the three directions of the established local three-dimensional coordinates; These represent the coordinate vectors of the feature point and its neighboring points in three-dimensional space, respectively. These represent the surface normal vectors of the feature point and its neighboring points, respectively. This represents the Euclidean distance between a feature point and its neighboring points.

[0130] 2) For all feature points Distance less neighborhood points Following the method in step 1), these neighborhood points The search distance to the center is less than Find the neighborhood points, and then calculate the SPFH between the center point and the neighborhood points.

[0131] 3) Feature points SPFH and weighted neighborhood points The SPFH values ​​are statistically analyzed to obtain the final FPFH descriptor, as shown in the following formula:

[0132] ; (15)

[0133] In the above formula For the weight value, generally related to the key point and the distance between the neighborhood points ; represents the number of neighborhood points participating in the calculation.

[0134] The principle of realizing the six-degree-of-freedom pose estimation of the ship through point cloud registration is as follows: first, a template point cloud of the target ship is created, then the template point cloud is made to coincide with the to-be-measured point cloud through coordinate transformation, and finally the initial pose information of the template point cloud and the coordinate transformation matrix are used to realize the estimation of the pose of the to-be-measured ship point cloud. The flow chart is shown in Figure 4 , and the specific implementation steps are as follows:

[0135] 1) Obtain the complete point cloud of the target ship, and then establish a template library of the target ship point cloud based on the feature points and the feature histogram of the complete point cloud.

[0136] 2) Calculate the feature points and the feature histogram of the to-be-registered point cloud.

[0137] 3) Coarse registration process: the feature points of the template point cloud library and the to-be-registered point cloud library are one-to-one corresponding, the SAC-IA point cloud registration algorithm is used to make the feature points in the template point cloud library coincide with the corresponding feature points in the to-be-registered point cloud library, and the coarse registration of the point cloud is realized.

[0138] 4) Fine registration process: the ICP point cloud registration algorithm is used to realize the accurate registration of the point cloud on the basis of coarse registration, finally obtain the accurate coordinate conversion matrix of the point cloud, and estimate the six-degree-of-freedom pose of the ship.

[0139] The core task of the AIS processing unit is to extract the position information and navigation trajectory of the ship from the AIS message. The AIS message provides the real-time position (latitude and longitude), speed, heading, ship identification code (MMSI) and other key data of the ship. The AIS processing unit decodes the AIS message to extract the latitude, longitude, speed, heading and other information of the ship at a specific time point, thereby accurately obtaining the current position of the ship.

[0140] Since the AIS data may contain errors and noise, the decoded data needs to be preprocessed to ensure the accuracy of the extraction result. First, filter out invalid MMSI codes and eliminate messages with a length of less than 9 bits. Second, perform reasonableness check on the latitude and longitude data, and eliminate invalid position data that exceeds the preset geographical range. For latitude and longitude with abnormal jumps, eliminate or correct the coordinates that exceed the reasonable range according to the maximum speed limit of the ship. For the case of random errors in speed and heading, perform interpolation repair according to the latitude and longitude of the previous and next time nodes to ensure the continuity and consistency of the ship's speed and heading data.

[0141] After completing data cleaning and repair, the AIS processing unit attributes the position information at different time points to the same ship according to the MMSI code of the ship, thereby generating the complete voyage trajectory of the ship. For ships with non-continuous voyages, the system will segment the trajectory according to the set time interval threshold, ensuring that each segment of voyage data is continuous and stable. Through these processes, the AIS processing unit can provide reliable ship position information and accurate voyage trajectory, providing data support for subsequent voyage analysis and monitoring.

[0142] In this embodiment, the method for dividing the grades of dangerous water areas includes: the weighted fusion method is used to fuse the longitudinal minimum safety distance obtained by the two collision avoidance methods (collision avoidance method based on dynamic trajectory and collision avoidance method based on static rules), and the dangerous area of the bridge area is divided based on the fusion result.

[0143] The method for calculating the first longitudinal minimum safety distance and the channel direction ship width based on the dynamic trajectory is as follows:

[0144] ; (16)

[0145] ; (17)

[0146] wherein, represents the dynamic longitudinal minimum safety distance, represents the dynamic channel direction ship width, represents the current speed of the ship, represents the reaction time window, represents the man-machine reaction time, represents the effective deceleration (taking the absolute value) that the ship can achieve when taking emergency deceleration, represents the ship length, represents the ship width, represents the angle between the bow direction and the channel axis.

[0147] The method for calculating the second longitudinal minimum safety distance and the channel direction ship width based on the static rules is as follows:

[0148] ; (18)

[0149] ; (19)

[0150] wherein, represents the static longitudinal minimum safety distance, represents the static channel direction ship width, respectively represent the safety factors required when going upstream and downstream, represents an empirical coefficient, usually between 1.1 and 1.5, the value of which varies according to environmental conditions (such as water flow, wind force, etc.) and the type of ship, Table 1 below is a value range corresponding to some ship types

[0151]

[0152]

[0153] The longitudinal minimum safety distance and the final channel direction ship width obtained by the weighted fusion formula of the two collision avoidance modes are as follows:

[0154]

[0155]

[0156] wherein, and represent weighted coefficients, the greater the ship speed, the more reliable the real-time data, and the greater and .

[0157] As shown in Figure 5 , the longitudinal minimum safety distance and the final channel direction ship width obtained by the weighted fusion of the two collision avoidance modes determine the bridge water area first danger area (wherein represents the width of the bridge pier), the first danger area is inflated by 1.5 times as the bridge water area second danger area, and inflated by 2 times as the bridge water area third danger area. The average value of the ship positions obtained by the above three methods (visible light and infrared, laser radar, AIS) is taken to obtain the result for determining the ship in which danger area. The specific range of the danger area is shown in Tables 2-4.

[0158] Table 2 Range of the first danger area

[0159]

[0160] Table 3 Range of the second danger area

[0161]

[0162] Table 4 Range of the third danger area

[0163]

[0164] ​​​​​The early warning module obtains the comprehensive risk value, and divides the early warning level based on the comprehensive risk value and the dangerous water area level. The method for collision avoidance early warning based on the early warning level includes: when the system detects a ship with a collision risk, the early warning instruction can be quickly, accurately and hierarchically issued to remind the target ship to take evasive measures, thereby effectively reducing the probability of an accident. The risk value is introduced to further quantify the actual risk of ship collision. The risk value considers multiple factors such as the heading deviation of the ship, the attitude angle, the height difference between the highest point of the mast and the water surface, and the bridge clearance, and calculates a comprehensive risk score. The calculation formula of the comprehensive risk value is as follows:

[0165] ; (22)

[0166] wherein, is a risk zone coefficient determined according to the risk zone in which the ship is located. For example, the first risk zone takes , the second risk zone takes , and the third risk zone takes , wherein is a heading deviation risk factor reflecting the angle deviation of the ship heading and the channel axis; is a roll angle risk factor representing the risk of lateral rocking of the ship; is a pitch angle risk factor representing the risk of longitudinal inclination of the ship; is a clearance margin risk factor representing the difference between the height difference (the highest point of the mast and the water surface) and the bridge clearance; is a speed risk factor reflecting the influence of the ship's navigation speed on the collision risk; is the weight coefficient of each risk factor, and the sum is 1. The calculation of each risk factor is as follows:

[0167] ; (23)

[0168] ; (24)

[0169] wherein, is the heading deviation angle of the ship, is the maximum allowed heading deviation angle (for example, 30°); is the roll angle of the ship, is the maximum allowed roll angle (for example, 10°); is the pitch angle of the ship, is the maximum allowed pitch angle (for example, 5°); is the current speed of the ship, is the set maximum safe speed;​​​​ is the height difference between the highest point of the mast and the water surface, is the bridge clearance height.

[0170] According to the comprehensive risk value and the risk area, the early warning level is divided into three levels of low, medium and high. When the ship is in the third risk area or the comprehensive risk value exceeds the threshold value , a low-level early warning is triggered, and a warning light alarm is started to prompt; when the ship is in the second risk area or the comprehensive risk value exceeds the threshold value , a medium-level early warning is triggered, and an audible and visual alarm is started to prompt; when the ship is in the first risk area or the comprehensive risk value exceeds the threshold value , a high-level early warning is triggered, a VHF radio broadcast and a strong sound and light alarm system are started to conduct comprehensive early warning, and avoidance measures (such as changing the ship route and adjusting the ship speed) are suggested.

[0171] Through the multi-mode and multi-level early warning mechanism, the system can monitor the process of the ship passing through the bridge all day round, early warn the potential bridge collision risk, and provide solid protection for the bridge structure and shipping safety.

[0172] The above-described embodiments are only descriptions of the preferred modes of the present application, and do not limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements to the technical solutions of the present application made by those skilled in the art shall fall within the protection scope determined by the claims of the present application.

Claims

1. A ship navigation bridge collision avoidance early warning system, characterized in that, include: The monitoring module is used to collect ship information within the warning area; the ship information includes visible light and infrared images, lidar point cloud data, and AIS information. The information processing module is used to obtain the ship's attitude and the height difference between the highest point of the mast and the water surface based on the ship information. The information processing module includes: an image processing unit for identifying the ship's position based on visible light and infrared images; a mast identification unit for identifying masts based on visible light and infrared images and based on lidar point cloud data; a lidar point cloud processing unit for estimating the ship's pose based on lidar point cloud data; an AIS processing unit for extracting the ship's position and trajectory data based on AIS information; and a dangerous water area classification module for calculating the minimum longitudinal safety distance and classifying dangerous water areas based on the minimum longitudinal safety distance. The method for calculating the minimum longitudinal safety distance by the dangerous water area classification module includes: calculating a first minimum longitudinal safety distance based on a collision avoidance method using dynamic trajectories and calculating a second minimum longitudinal safety distance based on a collision avoidance method using static rules; the method for calculating the first minimum longitudinal safety distance based on a collision avoidance method using dynamic trajectories... The calculation methods include: ； In the formula, Indicates the ship's current speed; Indicates the reaction time window; Indicates human-machine reaction time. This indicates the effective deceleration that a ship can achieve when it takes emergency deceleration measures. The length of the ship is indicated; the minimum safe distance in the second longitudinal direction is calculated based on the static rules of collision avoidance. The calculation methods include: ； In the formula, , These represent the safety factors required for upward and downward movements, respectively. The first longitudinal minimum safety distance and the second longitudinal minimum safety distance are weighted and fused to obtain the longitudinal minimum safety distance: ； In the formula, Indicates the weighting coefficient; The early warning module is used to calculate a comprehensive risk value based on the ship's position and the height difference between the highest point of the mast and the water surface, and to classify the early warning level based on the comprehensive risk value and the dangerous water level of the area where the ship is located, and to issue a collision avoidance warning based on the early warning level; the method for calculating the comprehensive risk value includes: ； In the formula, It is the risk zone coefficient; It is a heading deviation risk factor; It is the lateral tilt risk factor; It is the pitch angle risk factor; It is the net air margin risk factor; It is a speed risk factor; These are the weighting coefficients of the risk factors, which add up to 1; Methods for calculating risk factors include: ； ； In the formula, It is the ship's heading deviation angle; It is the maximum allowable heading deviation angle; It is the ship's heel angle. It is the maximum permissible roll angle; It is the ship's pitch angle. It is the maximum permissible pitch angle; Indicates the ship's current speed. It is the set maximum safe speed; It is the height difference between the highest point of the mast and the water surface. It refers to the clearance height of the bridge.

2. The ship navigation bridge collision avoidance early warning system according to claim 1, characterized in that, Methods for identifying ship positions based on visible light and infrared images include: using a single-stage detector YOLO-G model to identify ship positions; The single-level detector YOLO-G model includes a feature extractor and a YOLO interaction layer; The feature extractor is an improvement on the Darknet-53 network, and the improvement methods include: For input visible light and infrared images with a resolution of 416×416 and 3 channels, the deep network architecture is designed to include 12 convolutional layers and 6 residual layers, resulting in output feature maps at 4 different scales. All convolutional layers sequentially perform batch normalization and activation function operations. The first convolutional layer is configured with 32 3×3 convolutional kernels to extract features from the visible light and infrared images. The output serves as the input to the second layer, which uses 64 3×3 convolutional kernels with a stride of 2 to perform downsampling. Based on the residual connection mechanism of YOLOv7, residual convolutional groups are formed by alternately stacking 3×3 and 1×1 convolutional layers, ultimately outputting a feature map with a size of 208×208. In the residual convolutional groups: ； In the formula, Feature maps are input to the network; For the first The network output of the layer; This indicates the concatenation of feature maps; ( ) is a combination function of batch normalization, activation function and convolution operation, used to implement the first... Nonlinear transformation of layers; in, The operation of ( ) is: Conv(1,1)-BN-Relu-Conv(3,3)-BN-Relu, which further constructs 4 groups of residual convolutional groups, and outputs feature maps of size 104×104, 52×52, 26×26, and 13×13 respectively at downsampling positions of 4x, 8x, 16x, and 32x. The feature maps are then fused through upsampling layers to construct a feature pyramid structure. The YOLO interaction layer is divided into four independent detection branches, each containing six convolutional layers and undergoing a 2x upsampling operation. Each branch is connected to the upsampling layer via tensor concatenation, completing multi-scale feature fusion of shallow positional information and deep semantic information of the input image. The multi-scale feature fusion formula is as follows: ； ； In the formula, This represents the feature maps that need to be fused. Indicates an upsampling operation; This represents the concatenation of feature map tensors at the same scale. This represents the fused feature map; This indicates the reconstruction and recognition of the obtained fused feature map; This represents the feature map after reconstruction and recognition. After multi-scale feature fusion, the YOLO interaction layer finally outputs feature maps with four scales: 13×13, 26×26, 52×52, and 104×104.

3. The ship navigation bridge collision avoidance early warning system according to claim 1, characterized in that, The method for ship pose estimation based on lidar point cloud data includes: segmenting the lidar point cloud data using an indexed eight-neighborhood-based region growing algorithm to obtain segmented point clouds; and estimating ship pose based on the segmented lidar point cloud data. The process of segmenting the lidar point cloud data using the region growing algorithm based on indexed eight neighborhoods includes: S1. Traverse the LiDAR point cloud data in scanning order, find the first unassigned data point, and set it as the seed point. ; S2, using seed point Calculate with center The eight neighborhood points of the index matrix ,if To meet the criteria for point cloud growth, and Combine them into one category, and Stored in the stack; S3. Remove the top point from the stack and use it as the new seed point. Repeat S2 and S3; S4. When the stack is empty, it means that the growth of this region has been completed. Return to S1 and find a new seed point for the new region. S5. Repeat S1~S4 until each data point in the index has a home, at which point the growth ends.

Images