A ship fog-penetration night navigation auxiliary enhancement system based on optical radar fusion

Abstract

The application provides a ship fog-penetration night navigation auxiliary enhancement system and method based on optical radar fusion, and belongs to the technical field of ship navigation assistance. The system comprises an optical camera device, a radar detection device, a data fusion processing unit and a display control terminal. The optical camera device collects real-time video images, and the radar detection device obtains distance, direction and speed information of surrounding ships. The data fusion processing unit accurately labels the radar target on the optical picture through coordinate transformation, time synchronization and target matching algorithm. The display control terminal superimposes and displays the position, distance, heading and speed information of the surrounding ships in an augmented reality mode. The application can effectively identify the surrounding ships under low-visibility conditions such as heavy fog, heavy rain and night, significantly improve the navigation safety, and solve the technical problems of failure of traditional optical equipment and non-intuitive radar interface.

Description

Technical Field

[0001] This invention relates to the field of ship navigation assistance technology, specifically to a ship fog-penetrating night navigation assistance enhancement system and method based on optical radar fusion, which is particularly suitable for ship navigation safety assistance in low visibility environments such as dense fog, heavy rain, and darkness. Background Technology

[0002] With the rapid development of water transport, the issue of ship navigation safety has become increasingly prominent. In adverse weather conditions, such as dense fog, heavy rain, blizzards, and nighttime navigation, ship operators face severe visual challenges. Traditional optical camera equipment suffers a significant drop in image quality in low visibility, making it almost impossible to effectively identify surrounding ship targets. While traditional marine radar offers all-weather detection capabilities, its interface is abstract, requiring specialized training for operators to accurately understand the position and movement of targets in radar images. Furthermore, radar cannot provide intuitive visual information about targets.

[0003] Existing ship navigation aids mainly fall into two categories: single optical systems and single radar systems. Single optical systems experience a sharp decline in performance under adverse weather conditions, making it difficult to meet the requirements for safe navigation. While single radar systems are unaffected by weather, their display interfaces lack intuitiveness, requiring the operator to mentally connect the radar image with the actual navigation environment. This cognitive conversion process is prone to errors in emergency situations.

[0004] Therefore, there is an urgent need for a system that can combine the intuitiveness of optical imaging with the precision of radar detection, enabling operators to intuitively understand the distribution and movement of surrounding vessels under various adverse weather conditions, thereby effectively improving navigation safety. Summary of the Invention

[0005] The purpose of this invention is to provide a ship fog-penetrating night navigation assistance enhancement system and method based on optical radar fusion, so as to solve the technical problems of insufficient ship target identification capability and unintuitive radar interface in the prior art under adverse weather conditions.

[0006] The technical solution adopted in this invention is as follows: A ship fog-penetrating night navigation aid enhancement system based on optical-radar fusion includes an optical camera device, a radar detection device, a data fusion processing unit, and a display and control terminal.

[0007] The optical camera device is used to acquire real-time video images around the ship. It preferably uses an ultra-low illumination CMOS image sensor with a minimum illumination of no more than 0.001 Lux and has optical fog-penetrating function, which can acquire clear video images in low light and low visibility environments.

[0008] The radar detection device is used to detect the distance, azimuth and speed information of surrounding ship targets. It preferably adopts X-band or S-band marine navigation radar with a detection range of not less than 12 nautical miles, an azimuth accuracy better than ±1 degree, and can work stably under various weather conditions.

[0009] The data fusion processing unit is connected to both the optical camera and the radar detection device, and is used to perform real-time fusion processing of radar data and optical images. The data fusion processing unit includes a coordinate transformation module, a time synchronization module, a target matching module, and an image enhancement module. The coordinate transformation module converts the target position coordinates detected by the radar into pixel coordinates consistent with the viewing angle of the optical image; the time synchronization module aligns the radar data with the optical image in time; the target matching module associates and matches the radar-detected target with the ship in the optical image based on the target's position and motion characteristics; and the image enhancement module overlays the position markers of the radar-detected ship target onto the optical image.

[0010] The display control terminal is connected to the data fusion processing unit and is used to display the enhanced image after fusion. It displays the position, distance, heading and speed information of surrounding ship targets on the optical video in a superimposed annotation manner, so that the driver can intuitively understand the navigation environment.

[0011] Furthermore, the system also includes an AIS receiving module for receiving AIS information from surrounding vessels; the data fusion processing unit also receives data from the AIS receiving module and combines it with the AIS information to perform target identification and classification, thereby improving the accuracy of target matching.

[0012] Furthermore, the system also includes a collision warning module, which calculates the collision risk of surrounding vessels based on fused data and issues a warning on the display control terminal.

[0013] Furthermore, the data fusion processing unit also includes an image enhancement processing submodule, used to perform defogging, brightening, and edge enhancement processing on optical images in low-light or hazy environments.

[0014] Furthermore, the system also includes a navigation data recording module for synchronously recording optical video, radar data, and fusion result data, supporting post-event playback and analysis.

[0015] Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) Optical radar dual perception: It can effectively identify surrounding ship targets under various adverse weather conditions by utilizing the intuitiveness of optical imaging and the accuracy of radar detection.

[0016] (2) Real-time data fusion: The data fusion processing unit completes the precise matching of radar data and optical images within 0.5 seconds, with low delay and good real-time performance.

[0017] (3) Augmented reality display: The radar targets displayed as symbols are accurately superimposed on the intuitive optical screen, so that the driver does not need to make a cognitive conversion, reducing the risk of misjudgment.

[0018] (4) Significantly improve visibility: Combined with advanced image processing algorithms, visibility is increased by more than 5 times in dense fog, effectively ensuring navigation safety.

[0019] (5) Multi-source data fusion: Combine AIS data to further improve the accuracy of target identification and support collision risk warning. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0021] Figure 1 This is a system composition block diagram of the present invention; Figure 2 This is a structural block diagram of the data fusion processing unit of the present invention; Figure 3 This is a flowchart of the coordinate transformation and target matching method of the present invention; Figure 4 This is a schematic diagram of the fused screen of the display control terminal of the present invention.

[0022] In the diagram: 1. Optical camera device; 2. Radar detection device; 3. Data fusion processing unit; 31. Coordinate transformation module; 32. Time synchronization module; 33. Target matching module; 34. Image enhancement module; 4. Display control terminal; 5. AIS receiving module; 6. Collision warning module; 7. Navigation data recording module. Detailed Implementation

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0025] like Figure 1 As shown, the present invention provides a ship fog-penetrating night navigation assistance enhancement system based on optical radar fusion, including: an optical camera device 1, a radar detection device 2, a data fusion processing unit 3, and a display control terminal 4.

[0026] The optical camera device 1 is used to acquire real-time video images around the ship. It preferably uses a 1 / 1.8-inch 4-megapixel ultra-low-light CMOS image sensor with a minimum illumination of 0.0005 Lux (color) / 0.0002 Lux (black and white), and has 53x optical zoom and optical fog penetration capabilities. The effective visibility distance is not less than 2000m in heavy fog (visibility 500m) and not less than 3000m in complete darkness.

[0027] Radar detection device 2 is used to detect the range, azimuth, and speed information of surrounding ship targets. Preferably, an X-band (9.4GHz) marine navigation radar is used, with a maximum detection range of 24 nautical miles, a range resolution of 20 meters, an azimuth accuracy of ±0.5 degrees, and the ability to simultaneously track no fewer than 100 targets.

[0028] The data fusion processing unit 3 is connected to both the optical camera device 1 and the radar detection device 2, and is used to perform real-time fusion processing of radar data and optical images. The core processor of the data fusion processing unit 3 uses an NPU with 6 TOPS computing power, which can process no less than 16 video streams simultaneously, with a data fusion latency of less than 0.5 seconds.

[0029] The display control terminal 4 is connected to the data fusion processing unit 3 and is used to display the enhanced image after fusion. The display control terminal 4 uses a high-brightness LCD screen, which remains clearly visible even in direct sunlight.

[0030] like Figure 2 As shown, the data fusion processing unit 3 includes: a coordinate transformation module 31, a time synchronization module 32, a target matching module 33, and an image enhancement module 34.

[0031] The coordinate transformation module 31 is used to convert the target position coordinates detected by the radar into pixel coordinates consistent with the viewing angle of the optical image. Specifically, it includes: converting the polar coordinates (range, azimuth) of the radar into Cartesian coordinates with the ship as the origin; converting the above coordinates into geographic coordinates according to the ship's position, heading and attitude information; and projecting the geographic coordinates onto the pixel coordinates of the optical image according to the interior and exterior orientation elements of the optical camera device 1.

[0032] The time synchronization module 32 is used to time-align radar data with optical images. The radar data update cycle is typically 1-2 seconds, while the optical image frame rate is typically 25-30 frames per second. The time synchronization module 32 employs timestamp alignment and interpolation algorithms to ensure that spatially consistent radar and optical data are obtained at any given time.

[0033] The target matching module 33 is used to associate and match radar-detected targets with ships in the optical image based on the target's position and motion characteristics. A multi-target tracking algorithm based on Kalman filtering is employed, performing data association and matching based on the target's predicted motion trajectory and position characteristics. Specific steps include: initializing the target list; calculating the predicted position of each radar-detected target in the optical image; extracting visual features of possible ship areas in the optical image; performing optimal matching based on the Hungarian algorithm; and updating the target status.

[0034] Image enhancement module 34 is used to overlay position markers of radar-detected ship targets onto the optical image. The markings include: target outline; target type identification (cargo ship, passenger ship, fishing vessel, etc.); target distance and bearing information; and target heading and speed information. The target marking accuracy and positional deviation are less than 5 pixels.

[0035] like Figure 3 The diagram shows the flow chart of the coordinate transformation and target matching method of the present invention, including the following steps: S1: Radar data acquisition, obtaining the distance, azimuth, and speed information of surrounding targets; S2: Converting the radar polar coordinates to Cartesian coordinates with the ship as the origin; S3: Combining the ship's position, heading, and attitude information, converting the Cartesian coordinates to geographic coordinates; S4: Projecting the geographic coordinates onto the pixel coordinates of the optical image based on the internal and external orientation elements of the optical camera device; S5: Time synchronization alignment of the radar data and the optical image; S6: Predicting the target's trajectory and calculating the predicted position; S7: Simultaneously extracting visual features of possible ship areas in the optical image; S8: Matching the radar target with the optical target based on a multi-target data association algorithm; S9: Updating the target tracking status based on the matching result and outputting the fused and enhanced image.

[0036] Furthermore, the system also includes an AIS receiver module 5, used to receive AIS information broadcast by surrounding vessels, including vessel identification codes, positions, headings, and speeds. The fusion of AIS data with radar and optical data can significantly improve the accuracy of target identification, especially for vessels already equipped with AIS.

[0037] Furthermore, the system also includes a collision warning module 6, which calculates the collision risk of surrounding vessels based on fused data. The algorithm considers factors including: the target vessel's distance, course, and speed; the ship's own course and speed; the closest encounter distance (DCPA) and the closest encounter time (TCPA); and the target vessel's maneuverability. When the collision risk exceeds a preset threshold, the collision warning module 6 issues an audible and visual warning on the display control terminal 4.

[0038] Furthermore, the data fusion processing unit 3 also includes an image enhancement processing submodule, used for defogging, brightening, and edge enhancement processing of optical images in low-light or hazy environments. Employing a deep learning-based image restoration algorithm, the contrast and clarity of hazy images can be effectively improved.

[0039] Furthermore, the system also includes a navigation data recording module 7, which is used to simultaneously record optical video, radar data and fusion result data, supporting post-event playback and analysis, and providing objective evidence for maritime accident investigations.

[0040] like Figure 4 As shown, the fused screen of the display control terminal 4 includes: a real-time optical video background layer; a radar target annotation layer superimposed on the optical video; an AIS information annotation layer; and a navigation information overlay layer. The driver can simultaneously obtain intuitive visual information and accurate radar data on a single screen without switching between multiple display interfaces.

[0041] During system operation, the optical camera device 1 continuously acquires real-time video images of the forward field of view and transmits them synchronously to the data fusion processing unit 3. The radar detection device 2 scans the surrounding sea area at fixed intervals, transmitting the position and motion information of all detected targets to the data fusion processing unit 3. The AIS receiving module 5 receives AIS broadcast information from surrounding vessels. The time synchronization module 32 in the data fusion processing unit 3 first performs time alignment on the three data streams, then the coordinate transformation module 31 transforms all target coordinates to a unified coordinate system before projecting them onto the optical image plane. The target matching module 33 correlates and fuses target data from different sources based on position and motion characteristics, eliminating duplicate targets. Finally, the image enhancement module 34 overlays the fusion result onto the optical video in an augmented reality manner, displaying it to the driver on the control terminal 4.

[0042] In a specific implementation, the optical camera device 1 adopts a rotatable gimbal design with a horizontal rotation range of 0-360 degrees and a pitch rotation range of -45 degrees to +60 degrees, covering a wide field of view around the ship. The radar detection device 2 adopts a dual-axis stabilization platform to compensate for the effects of ship roll and pitch, ensuring the stability of the detection data.

[0043] The data fusion processing unit 3 is equipped with multiple working modes, including: standard mode, which only performs target fusion display; early warning mode, which enhances collision risk monitoring and early warning on the basis of standard mode; recording mode, which enables navigation data recording function on the basis of standard mode; and night flight mode, which optimizes image enhancement parameters and display brightness for nighttime environments.

[0044] The system power supply adopts the standard 24VDC for marine applications and has overvoltage, overcurrent, and reverse connection protection functions. The operating temperature range is -40℃ to +70℃, and the protection rating is IP66, making it suitable for various harsh sea conditions.

Claims

1. A ship fog-penetrating night navigation aid enhancement system based on optical radar fusion, characterized in that: The system includes an optical camera device (1), a radar detection device (2), a data fusion processing unit (3), and a display control terminal (4). The optical camera device (1) is used to collect real-time video images around the ship. The radar detection device (2) is used to detect the distance, azimuth, and speed information of surrounding ship targets. The data fusion processing unit (3) is connected to the optical camera device (1) and the radar detection device (2) respectively, and is used to perform real-time fusion processing of radar data and optical images. The display control terminal (4) is connected to the data fusion processing unit (3) and is used to display the enhanced image after fusion.

2. The ship fog-penetrating night navigation aid enhancement system based on optical radar fusion according to claim 1, characterized in that: The data fusion processing unit (3) includes a coordinate transformation module (31), a time synchronization module (32), a target matching module (33), and an image enhancement module (34). The coordinate transformation module (31) is used to convert the target position coordinates detected by the radar into pixel coordinates consistent with the optical image viewpoint. The time synchronization module (32) is used to time-align the radar data with the optical image. The target matching module (33) is used to associate and match the radar-detected target with the ship in the optical image based on the target's position and motion characteristics. The image enhancement module (34) is used to overlay the position markers of the ship target detected by the radar onto the optical image.

3. The ship fog-penetrating night navigation aid enhancement system based on optical radar fusion according to claim 2, characterized in that: The target matching module (33) uses a multi-target tracking algorithm based on Kalman filtering to perform data association matching based on the target's motion trajectory prediction and position characteristics.

4. The ship fog-penetrating night navigation aid enhancement system based on optical radar fusion according to claim 1, characterized in that: The optical camera device (1) uses an ultra-low illumination CMOS image sensor with a minimum illumination of no more than 0.001 Lux and has optical fog-penetrating function.

5. The ship fog-penetrating night navigation aid enhancement system based on optical radar fusion according to claim 1, characterized in that: The radar detection device (2) is an X-band or S-band marine navigation radar with a detection range of not less than 12 nautical miles and an azimuth accuracy better than ±1 degree.

6. The ship fog-penetrating night navigation aid enhancement system based on optical radar fusion according to claim 1, characterized in that: The display control terminal (4) displays the type identification, distance, heading and speed information of surrounding ship targets on the fused screen by superimposing annotations.

7. The ship fog-penetrating night navigation aid enhancement system based on optical radar fusion according to claim 1, characterized in that: The system also includes an AIS receiving module (5) for receiving AIS information from surrounding vessels; the data fusion processing unit (3) also receives data from the AIS receiving module (5) and combines it with the AIS information to perform target identification and classification.

8. The ship fog-penetrating night navigation aid enhancement system based on optical radar fusion according to claim 1, characterized in that: The system also includes a collision warning module (6), which calculates the collision risk of surrounding ships based on fused data and issues a warning on the display control terminal (4).

9. The ship fog-penetrating night navigation aid enhancement system based on optical radar fusion according to claim 1, characterized in that: The data fusion processing unit (3) further includes an image enhancement processing submodule, which is used to perform defogging, brightening and edge enhancement processing on optical images in low light or haze environments.

10. The ship fog-penetrating night navigation aid enhancement system based on optical radar fusion according to claim 1, characterized in that: The system also includes a navigation data recording module (7) for synchronously recording optical video, radar data and fusion result data, supporting post-event playback and analysis.

Images