A multi-source perception fusion high-precision bridge active anti-ship collision early warning system

Abstract

The application discloses a multi-source perception fusion high-precision bridge active anti-ship collision early warning system, and belongs to the technical field of bridge active anti-collision. The active anti-ship collision early warning system comprises: an overrun monitoring module comprising a plurality of laser radars and a radar water level gauge; the plurality of laser radars fuse ship space point clouds with each other, and convert water surface ship height and width; the radar water level gauge is used for acquiring a navigation water level height; a yaw monitoring module comprising two ultrasonic radars respectively installed on both sides of a navigation hole main beam, to monitor whether a ship deviates from a virtual channel; a video monitoring module comprising a high-definition camera installed at a mid-span vertex of the navigation hole main beam, to monitor overall navigation conditions of the channel, and to automatically capture an abnormal ship; a back-end management module responsible for data convergence, processing analysis, storage management and instruction issuing of the overrun monitoring module, the yaw monitoring module and the video monitoring module; and an abnormal early warning module used for warning overrun and yawing ships.

Description

Technical Field

[0001] This invention belongs to the field of active bridge collision avoidance technology, specifically relating to a high-precision active bridge collision avoidance and early warning system based on multi-source sensing fusion. Background Technology

[0002] With the rapid development of water transport in my country, the density of navigation has increased dramatically, and the tonnage of ships has also increased significantly. As a result, the risk of ship-bridge collisions is rising, and incidents of ships colliding with bridges on navigable waterways occur frequently. Once a ship-bridge collision occurs, it can cause serious consequences such as bridge damage and collapse, waterway obstruction, environmental pollution, loss of life and property, and result in huge economic losses.

[0003] To address the risk of ship collisions with bridges, bridge management units typically employ passive collision avoidance measures (including navigation aids, warning signs, crash barriers, sandbars, and crash shields) and manual observation. However, once a large cargo ship enters the warning zone of the bridge's navigation channel, patrol personnel often cannot alert the ship's navigator in time. While passive collision avoidance methods can mitigate damage from collisions, they cannot prevent them entirely, especially on high-grade inland waterways where vessel traffic is heavy and fast. Relying solely on manual assessment or facility protection is far from sufficient to meet the needs of bridge collision management. There is an urgent need to address the problems of traditional methods being passive, untimely, inefficient, and lacking in early warning. Therefore, it is imperative to improve the level of active collision avoidance for bridges, providing real-time remote warnings for vessels at risk of collision.

[0004] For early warning of bridge collisions, vessel identification is fundamental. Current vessel identification methods typically use synthetic aperture radar (SAR), cameras, and automatic identification systems (AIS) to identify vessel targets. SAR imaging is less affected by environmental conditions and can operate both day and night, but SAR images are mostly black and white, lacking some essential vessel features. Compared to SAR images, camera video images have richer target features, but camera accuracy is easily affected by insufficient lighting, rain, fog, and other environmental factors. Unlike the previous two vessel identification methods, AIS is a self-reporting system widely used in ship navigation. However, most AIS systems have low reporting frequency and accuracy, making it difficult to meet the needs of real-time early warning for vessels in waterways, and AIS systems can be manually turned off. Therefore, building a more stable and practical active bridge collision prevention system is a research direction for bridge maintenance departments to prevent ship collisions with bridges. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a high-precision bridge active ship collision prevention and early warning system that integrates multi-source sensing and fusion, thus solving the problems in the existing technologies.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] A high-precision bridge active ship collision avoidance early warning system based on multi-source sensing fusion includes:

[0008] The over-limit monitoring module includes multiple lidar sensors and a radar level gauge. The lidar sensors fuse the ship's spatial point cloud data to calculate the ship's height T and width W on the water surface. The radar level gauge is used to obtain the navigable water level T. w ;

[0009] Yaw monitoring module: includes two ultrasonic radars installed on both sides of the main beam of the navigation channel. Both ultrasonic radars are tilted towards the outside of the pier to emit ultrasonic signals. The area on the inside that is not covered by ultrasonic signals is a virtual channel to monitor whether the incoming ship deviates from the virtual channel.

[0010] Video monitoring module: includes a high-definition camera installed at the mid-span apex of the main beam of the navigation channel, used to monitor the overall navigation status of the channel and automatically capture abnormal vessels;

[0011] Backend management module: responsible for the aggregation, processing, analysis, storage management, and command issuance of data from the over-limit monitoring module, yaw monitoring module, and video surveillance module;

[0012] In addition, an anomaly warning module: used to warn vessels that exceed limits or deviate from course.

[0013] Furthermore, multiple lidar sensors are optimized in their deployment to scan all navigating vessels within the monitoring range; the optimization method includes the following steps:

[0014] Step 1: Establish a global coordinate system O-XYZ with the mid-span vertex of the main beam of the navigation channel as the origin, and use the lidar L... j The scanned point cloud is mapped to the coordinate point P in the global coordinate system O-XYZ. j (x j ,y j ,z j );

[0015] Step 2: Define the point cloud perception model based on the above coordinate system;

[0016] Step 3: Based on the point cloud perception model, obtain the number of lidars k and the parameters of the remaining lidars using a first-order optimization algorithm.

[0017] Furthermore, the objective function of the point cloud perception model is:

[0018] G j =min[F j (k j ,P oj V j H j ,v j,h j )-(1.1DT s +DL+2.2T s L)]

[0019] Among them, multiple lidar units have a size of D×1.1T. s In the point cloud obtained by scanning the spatial surface of ×L, D is the width of the navigation aperture; T s The design altitude is L, which represents the over-limit warning distance. Three points with the shortest distance are randomly selected and connected to form a micro-triangular grid. A total of m micro-triangular grids Δ can be divided from the entire point cloud. q (q=1,2,…,m), For Δ q The summation area; k is the number of lidar units, P oj (x oj ,y oj ,z oj ) represents the installation location of the lidar, V j For vertical field of view, H j For the horizontal field of view, v j For the vertical rotation angle of the laser, h j The horizontal rotation angle of the laser; This can then be written as a function F of the lidar parameters. j (k,P oj V j H j ,v j ,h j ).

[0020] Furthermore, the mapping relationship between the point cloud and the global coordinate system is as follows:

[0021]

[0022] Where, r j α represents the distance between each lidar point cloud and its installation location. j β is the horizontal rotation angle of the laser. j The vertical rotation angle of the laser.

[0023] Furthermore, the height T and width W of the surface vessel are respectively:

[0024] T = maxY - minY

[0025] jj

[0026] W = maxX - minX

[0027] jj

[0028] Among them, the x and y coordinate sets of the point clouds scanned by multiple lidars in the global coordinate system O-XYZ are respectively X j ={x1,x2,…,x k}、Y j ={y1,y2,…,y k}

[0029] Furthermore, with a distance of S between the two ultrasonic radars, a trapezoidal navigation area is defined within the existing designed waterway using ultrasonic radars. The trapezoidal navigation area is 2000m long, and the angle between the innermost beams of the two ultrasonic radars and the existing designed waterway is:

[0030]

[0031] Where D is the width of the navigation hole.

[0032] Furthermore, the snapshot capture step of the video surveillance module includes:

[0033] 1) When an over-height or eccentric vessel is detected, cluster analysis is performed on the point cloud data from multiple lidar scans to obtain a cluster center K* with coordinates (x*, y*, z*); the cluster center is mapped to the pixel coordinate system op-uv of the video image captured by the high-definition camera to obtain the pixel point P*(u*, v*).

[0034] 2) Mark the pixel P* in the video image captured by the high-definition camera. The ship where the pixel P* falls is the abnormal ship. Enlarge and crop the image of the abnormal ship, identify the ship number, and realize the automatic capture of abnormal ships.

[0035] Furthermore, the mapping relationship between the cluster center mapping and the pixel coordinate system is as follows:

[0036]

[0037] In the formula, f is the focal length of the camera, and x0 and y0 are the horizontal and vertical distances from the center of the imaging plate to the origin of the pixel coordinate system, respectively.

[0038] Furthermore, the rules for determining oversized vessels are as follows: if the design navigation height T... s ≤(TT w If D / 2 ≤ W, it indicates that the ship is too tall; if D / 2 ≤ W, it indicates that the ship is too wide.

[0039] A bridge equipped with the aforementioned active ship collision warning system.

[0040] The beneficial effects of this invention are:

[0041] 1. This invention integrates the advantages of sensors such as lidar, ultrasonic radar, high-definition camera and radar level gauge, which improves the accuracy of ship identification and positioning, and is less affected by real-world environmental factors, with strong system robustness.

[0042] 2. This invention has a multi-laser radar optimization layout function, which can achieve comprehensive scanning and monitoring with the optimal radar layout scheme.

[0043] 3. Existing bridge active collision avoidance technologies mainly include ship yaw and over-height warning. This invention also introduces ship over-width warning technology. For some small and medium-sized bridges in low-grade waterways, the navigable clearance is small, so ship over-width warning is very necessary, and the application scenarios of this invention are more diverse.

[0044] 4. This invention forms a virtual waterway by emitting ultrasonic signals. Once a ship deviates from the virtual waterway, the system will immediately issue an early warning signal, which is highly effective.

[0045] 5. Upon detecting an abnormal vessel, this invention can quickly and automatically capture images of it, identify its vessel number, and promptly report detailed information to the maritime authorities. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a schematic diagram of the system sensor layout;

[0048] Figure 2 This is a schematic diagram of lidar point cloud data fusion.

[0049] Figure 3 This is a schematic diagram of lidar parameters;

[0050] Figure 4 A schematic diagram illustrating the mapping of point cloud cluster centers to a two-dimensional pixel plane;

[0051] Figure 5 This is a schematic diagram of an ultrasonic virtual waterway.

[0052] Figure 6 This is a schematic diagram of the system structure and workflow. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] like Figure 1 and 6 As shown, a high-precision bridge active ship collision prevention and early warning system with multi-source sensing fusion includes an over-limit monitoring module, a yaw monitoring module, a video monitoring module, a back-end management module, and an anomaly early warning module. Among them, the over-limit monitoring module, the yaw monitoring module, and the video monitoring module are all front-end monitoring modules. They input the effective data features of statistical analysis into the back-end management module, which further analyzes the data and determines whether to send an early warning command to the anomaly early warning module.

[0055] The over-limit monitoring module is used to monitor the height and width of approaching vessels, and includes k lidar sensors and one radar level gauge. The k lidar sensors are optimized to scan all navigable vessels within the monitoring range, and the spatial point clouds of the vessels are fused to calculate the height T and width W of the vessels on the water surface. The radar level gauge is installed at the bottom of the main beam of the bridge's navigation span to obtain the navigable water level height T. w T+T w This refers to the actual navigable height of the target vessel, used to determine whether the vessel exceeds the height limit.

[0056] The yaw monitoring module is used to monitor whether incoming vessels deviate from the virtual channel. It includes two ultrasonic radars installed on both sides of the main beam of the navigation channel. Both ultrasonic radars are tilted outwards from the piers to emit ultrasonic signals. The area on the inner side not covered by ultrasonic signals is the virtual channel. If the ultrasonic radars receive the echo signal of an incoming vessel, it is determined that the vessel has deviated from the virtual channel.

[0057] The video monitoring module is used to monitor the overall navigation status of the waterway and automatically capture abnormal vessels; the equipment includes a high-definition camera, which is installed at the mid-span apex of the main beam of the navigation channel.

[0058] The backend management module is responsible for the aggregation, processing, analysis, storage management, and command issuance of data from the over-limit monitoring module, yaw monitoring module, and video surveillance module; the equipment includes a central server and a visualization platform. The central server cleans, repairs, performs refined analysis, and stores the input information, while the visualization platform displays the information characteristics of the central server and integrates the software operation interface.

[0059] The anomaly warning module is used to warn vessels of exceeding limits and deviating from their course. Its components include an AIS base station and a high-frequency audible and visual alarm. The AIS base station receives AIS data from the vessel and sends AIS warning text messages. The high-frequency audible and visual alarm transmits high-frequency audible and visual warning signals.

[0060] like Figure 2 and Figure 3 As shown, the optimal arrangement method for k lidar units is as follows:

[0061] Step 1: Establish a global coordinate system O-XYZ with the mid-span vertex of the main beam of the navigation channel as the origin, and use the lidar L... j The point cloud scanned by (j=1,2,…,k) is mapped to the coordinate point P in the global coordinate system O-XYZ. j (x j ,yj,z j ).

[0062] Step 2, based on the above coordinate system, define the point cloud perception model as follows:

[0063] k lidar units in a configuration of size D × 1.1T s ×L(D is the width of the navigation hole; T) s For the design navigation altitude; L is the over-limit warning distance, determined according to the owner's requirements) in the point cloud scanned from the spatial surface, the three closest points are randomly selected and connected to form a micro-triangular grid. A total of m micro-triangular grids Δ can be divided into the entire point cloud. q (q = 1, 2, ..., m); Δ q sum of areas The parameters of each lidar are determined by the number of lidars (k) and the installation location (P) of the lidars. oj (x oj ,y oj ,z oj Vertical field of view V j Horizontal field of view H j Laser vertical rotation angle v j Laser horizontal rotation angle h j ; This can then be written as a function F of the lidar parameters. j (k j ,P oj V j H j ,v j ,h j );

[0064] Define the optimization objective function G j :

[0065] G j =min[Fj (k j ,P oj V j H j ,v j ,h j )-(1.1DT s +DL+2.2T s L)]

[0066] Step 3: Based on the point cloud perception model, obtain the number of lidars k and the parameters of the remaining lidars using a first-order optimization algorithm;

[0067] The first-order optimization algorithm is as follows:

[0068] Minimize G j (k j ,P oj V j H j ,v j ,h j )

[0069] Subjected to

[0070]

[0071] In the formula, the upper and lower horizontal lines of the parameters represent the upper and lower limits of the corresponding parameters, respectively.

[0072] like Figure 4 As shown, each lidar L j The point cloud scanned by (j=1,2,…,k) is mapped to the coordinate point P in the global coordinate system O-XYZ. j (x j ,y j ,z j The mapping relationship is as follows:

[0073]

[0074] Where, r j α represents the distance between each lidar point cloud and its installation location. j β is the horizontal rotation angle of the laser. j The vertical rotation angle of the laser;

[0075] The specific calculation methods for the height and width of vessels on the water surface in the over-limit monitoring module are as follows:

[0076] The set of x and y coordinates of the point cloud scanned by k lidar sensors in the global coordinate system O-XYZ is X j ={x1,x2,…,x k}、Y j={y1,y2,…,y k}

[0077] The height T and width W of a surface vessel are as follows:

[0078] T = max Y j -min Y j

[0079] W = maxX - minX

[0080] jj

[0081] like Figure 3 As shown, the virtual channel division in the yaw monitoring module is as follows:

[0082] Two ultrasonic radars are symmetrically installed on the main beam of the navigation channel, with a spacing of S. Within the existing design channel, the ultrasonic radars divide the navigation area into a trapezoidal zone. To allow sufficient warning and braking distance, the trapezoidal navigation zone is 2000m long. The angle between the innermost beam of each ultrasonic radar and the existing design channel is:

[0083]

[0084] Where D is the width of the navigation hole.

[0085] like Figure 5 As shown, the specific methods for video surveillance module to capture images are as follows:

[0086] If a vessel is found to be too high or off course, the k-means clustering algorithm is used to perform cluster analysis on the point cloud data of k lidar scans to obtain a cluster center K* with coordinates (x*, y*, z*).

[0087] The total number of point clouds is n, and the objective function for clustering is:

[0088]

[0089] In the formula, W n P represents the distance between point clouds. i Let P be the i-th data center point; j Let j be the j-th data center point.

[0090] By mapping the cluster centers to the pixel coordinate system op-uv of the video images captured by the high-definition camera, we obtain the pixel point P*(u*,v*), and the mapping relationship is as follows:

[0091]

[0092] In the formula, f is the focal length of the camera, and x0 and y0 are the horizontal and vertical distances from the center of the imaging plate to the origin of the pixel coordinate system, respectively.

[0093] In the video images captured by the high-definition camera, pixel point P* is marked. The ship where pixel point P* falls is the abnormal ship. The abnormal ship image is enlarged and cropped, and the ship number is identified to realize the automatic capture of abnormal ships.

[0094] like Figure 6 As shown, the specific working methods of the backend management module and the anomaly warning module are as follows:

[0095] Determination of Oversized Vessels: Collects and stores the height T and width W of surface vessels, and the water level T in navigable waters, output from the oversized vessel monitoring module. w If the designed navigation altitude T s ≤(TT w If D / 2 ≤ W, it indicates that the ship is too tall; if D / 2 ≤ W, it indicates that the ship is too wide.

[0096] Determining a ship's yaw: Analyzing the output data of the ultrasonic radar, the detection of a ship's signal indicates that the ship is yawing.

[0097] Upon detecting an abnormal vessel, an early warning command is issued and executed by the abnormality early warning module.

[0098] Based on the abnormal vessel's registration number, an SMS alert is sent via AIS base station; if the abnormal vessel still does not respond, a high-frequency audible and visual alarm is activated to warn the abnormal vessel and to notify the maritime authorities.

[0099] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0100] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A high-precision bridge active ship collision prevention and early warning system based on multi-source sensing fusion, characterized in that, include: The over-limit monitoring module includes multiple lidar sensors and a radar level gauge. The lidar sensors fuse the ship's spatial point cloud data to calculate the ship's height T and width W on the water surface. The radar level gauge is used to obtain the navigable water level T. w ; Yaw monitoring module: includes two ultrasonic radars installed on both sides of the main beam of the navigation channel. Both ultrasonic radars are tilted towards the outside of the pier to emit ultrasonic signals. The area on the inside that is not covered by ultrasonic signals is a virtual channel to monitor whether the incoming ship deviates from the virtual channel. Video monitoring module: includes a high-definition camera installed at the mid-span apex of the main beam of the navigation channel, used to monitor the overall navigation status of the channel and automatically capture abnormal vessels; Backend management module: responsible for the aggregation, processing, analysis, storage management, and command issuance of data from the over-limit monitoring module, yaw monitoring module, and video surveillance module; And, an anomaly warning module: used to warn vessels that exceed limits or deviate from their course; The two ultrasonic radars are spaced S apart. Within the existing designed waterway, a trapezoidal navigation area is defined by the ultrasonic radars. The trapezoidal navigation area is 2000m long. The angle between the innermost beam of each ultrasonic radar and the existing designed waterway is: Where D is the width of the navigation hole; The video surveillance module's snapshot capture steps include: 1) When an overly tall or eccentric vessel is detected, the k-means clustering algorithm is used to perform cluster analysis on the point cloud data from multiple lidar scans to obtain a cluster center K. Coordinates are (x ,y ,z The cluster centers are mapped onto the pixel coordinate system op-uv of the video image captured by the high-definition camera to obtain pixel point P. (u ,v ); 2) Mark pixel P in the video image captured by the high-definition camera. Pixel P The vessel that is captured is considered an abnormal vessel. The image of the abnormal vessel is enlarged and cropped, and the vessel number is identified to achieve automatic capture of the abnormal vessel. The mapping relationship between cluster center mapping and pixel coordinate system is as follows: In the formula, f is the focal length of the camera, and x0 and y0 are the horizontal and vertical distances from the center of the imaging plate to the origin of the pixel coordinate system, respectively.

2. The high-precision bridge active ship collision prevention and early warning system according to claim 1, characterized in that, Multiple lidar sensors are optimally deployed to scan all navigating vessels within the monitoring range. The optimization method includes the following steps: Step 1: Establish a global coordinate system O-XYZ with the mid-span vertex of the main beam of the navigation channel as the origin. The point cloud scanned by the lidar Lj is mapped to the coordinate point Pj(xj,yj,zj) in the global coordinate system O-XYZ. Step 2: Define the point cloud perception model based on the above coordinate system; Step 3: Based on the point cloud perception model, obtain the number of lidars k and the parameters of the remaining lidars using a first-order optimization algorithm.

3. A high-precision bridge active ship collision prevention and early warning system according to claim 2, characterized in that, The objective function of the point cloud perception model is: In this system, multiple lidar sensors scan a point cloud on a spatial surface with dimensions D×1.1Ts×L, where D is the width of the navigation aperture, Ts is the designed navigation altitude, and L is the over-limit warning distance. The three closest points are randomly selected and connected to form a micro-triangular mesh. A total of m micro-triangular meshes Δq (q=1,2,…,m) can be divided from the entire point cloud. Let be the summation area of ​​Δq; k is the number of lidars; Poj(xoj,yoj,zoj) is the lidar installation position; Vj is the vertical field of view; Hj is the horizontal field of view; vj is the vertical rotation angle of the lidar; and hj is the horizontal rotation angle of the lidar. It can then be written as a function Fj(k, Poj, Vj, Hj, vj, hj) of the lidar parameters.

4. A high-precision bridge active ship collision prevention and early warning system according to claim 2, characterized in that, The mapping relationship between point clouds and the global coordinate system is as follows: Where rj is the distance between each lidar point cloud and its installation location. βj is the horizontal rotation angle of the laser, and βj is the vertical rotation angle of the laser.

5. A high-precision bridge active ship collision prevention and early warning system according to claim 4, characterized in that, The height T and width W of a surface vessel are as follows: Among them, the x and y coordinate sets of the point clouds scanned by multiple lidars in the global coordinate system O-XYZ are Xj={x1,x2,…, xk} and Yj={y1, y2,…, yk}, respectively.

6. A high-precision bridge active ship collision prevention and early warning system according to claim 1, characterized in that, The rule for determining oversized vessels is: if the design navigation height T s ≤(TT w If D / 2 ≤ W, it indicates that the ship is too tall; if D / 2 ≤ W, it indicates that the ship is too wide.

7. A bridge having an active ship collision warning system as described in any one of claims 1-6.

Images