Ship lock gate dangerous area intelligent early warning system based on double cameras

Abstract

The invention provides a ship lock gate dangerous area intelligent early warning system based on double cameras, and the system comprises a thermal imaging camera which is installed at the top of a gate control room and is configured to cover a 180-degree visual field of a gate area; the visible light camera and the thermal imaging camera are coaxially mounted, the included angle between optical axes of the visible light camera and the thermal imaging camera is less than or equal to 3 degrees, and the space position is unchanged through support fixation; the edge calculation unit is connected with the thermal imaging camera and the visible light camera and is configured to receive and process image data acquired by the thermal imaging camera and the visible light camera; the alarm device is connected with the edge calculation unit, and the alarm device comprises an audible and visual alarm, a gate control center LED warning screen and a linkage control interface; the thermal imaging camera and the visible light camera are configured, dynamic adjustment is performed according to the environment temperature and the illumination intensity in combination with the edge calculation unit, effective detection of a ship target under various weather and illumination conditions is ensured, the layout cost is low, and the software and hardware combination degree is high.

Description

Technical Field

[0001] This invention relates to the field of waterway safety at lock gates, and more particularly to an intelligent early warning system for dangerous areas of lock gates based on dual cameras. Background Technology

[0002] Locks, hydraulic structures, and port areas serve as crucial waterway transportation hubs and water conservancy control nodes, making their safe operation paramount. Unauthorized entry into the danger zone of a lock could lead to collisions, channel blockages, or even safety accidents. Traditional ship early warning systems largely rely on single visible light camera monitoring or manual lookout. However, in practical applications, visible light camera monitoring is susceptible to environmental factors, such as overexposure due to strong nighttime light interference, insufficient light in winter or overcast weather leading to difficulties in image recognition, and image quality degradation caused by water ripples and changes in lighting. This can result in missed or false detections, leading to low reliability of early warning systems. Manual lookout, on the other hand, suffers from inherent drawbacks such as slow reaction time, high labor intensity, and susceptibility to human error.

[0003] Existing research has explored multimodal fusion technology, combining visible light and infrared images, and significantly improving recognition accuracy and recall through adaptive weighted fusion strategies. Some systems further incorporate multi-sensor fusion, such as radar and AIS, to enhance situational awareness and early warning response capabilities. However, these solutions generally face problems such as complex structure, insufficient real-time performance, and high deployment costs, and have not yet formed a simple and efficient intelligent recognition system suitable for gate areas. Summary of the Invention

[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: The intelligent early warning system for dangerous areas of ship lock gates based on dual cameras includes: A thermal imaging camera is installed on the top of the gate control room and configured to cover a 180° field of view of the gate area. A visible light camera is coaxially mounted with the thermal imaging camera, the optical axis angle between the two is ≤3°, and the camera is fixed by a bracket to ensure that the spatial position remains unchanged. An edge computing unit, connected to the thermal imaging camera and the visible light camera, is configured to receive and process image data acquired by the thermal imaging camera and the visible light camera; and An alarm device is connected to the edge computing unit. The alarm device includes an audible and visual alarm, a gate control center LED warning screen, and a linkage control interface.

[0005] Furthermore, the linkage control interface of the alarm device is configured as a Modbus interface for signal interaction with the gate control system.

[0006] Furthermore, the edge computing unit is configured to dynamically adjust the temperature difference threshold according to environmental parameters and switch between summer mode, winter mode and night mode for thermal imaging images.

[0007] Furthermore, the edge computing unit is configured to: activate summer mode when the ambient temperature is >25℃, prioritizing the detection of high-temperature abnormal areas; activate winter mode when the ambient temperature is <15℃, prioritizing the detection of low-temperature abnormal areas; and activate night mode when the light intensity is <10 lux, enhancing the contrast of low-temperature areas.

[0008] Furthermore, the edge computing unit is configured to perform contour analysis on thermal anomaly regions in thermal imaging images, including calculating shape regularity, edge smoothness, and hot spot continuity, and extracting the movement trajectory of thermal anomaly regions using optical flow.

[0009] Furthermore, the edge computing unit is configured to mark thermal anomaly regions that meet at least one threshold region as suspected ships. The threshold region includes a bounding rectangle aspect ratio between 1.5 and 5.0, a ratio of contour points to perimeter > 0.8, and an area change rate of < 20% for three consecutive frames.

[0010] Furthermore, the edge computing unit is configured to mark thermal anomaly areas that simultaneously satisfy the following conditions: the angle between the movement trajectory direction and the channel axis is ≤30°, and the speed is in the range of 0.5-5m / s, as suspected vessels.

[0011] Furthermore, the edge computing unit is configured to perform CLAHE enhancement processing on the visible light image and to identify the ship using a segmentation network and a classification model.

[0012] Furthermore, the edge computing unit is configured to spatially register and fuse the suspected ship area identified by the thermal imaging image with the ship target box identified by the visible light image, and determine it as a confirmed ship or a high-confidence ship when the preset conditions are met.

[0013] Furthermore, the edge computing unit is configured to preset a gate danger zone. When it is confirmed that the center coordinates of a ship or a high-confidence ship have entered the danger zone and no gate control center has been received to allow entry, the alarm device is controlled to perform a first-level or second-level warning operation.

[0014] Compared with the prior art, the beneficial effects of the present invention include: 1. By configuring dual cameras for thermal imaging and visible light, and combining them with edge computing units to dynamically adjust based on ambient temperature (summer / winter mode) and light intensity (night mode), it can effectively cope with scenarios that traditional single-mode vision systems cannot handle, such as strong light interference at night, low temperature in winter, and insufficient light. It ensures effective detection of ship targets under various weather and lighting conditions, and has low deployment cost and high hardware and software integration.

[0015] 2. The thermal imaging module establishes a baseline temperature field model, dynamically adjusts the temperature difference threshold, and analyzes the contour regularity, edge smoothness, hot spot continuity, and motion trajectory characteristics of thermal anomaly regions. Combined with the CLAHE enhancement, segmentation, and classification algorithms of the visible light module, it achieves refined preliminary screening and confirmation of ship targets. The information fusion mechanism of the edge computing unit can comprehensively utilize the advantages of both modalities, effectively reducing the false detection rate and false negative rate of single-modal recognition, and improving the accuracy of ship identification.

[0016] 3. The edge computing unit enables real-time image processing. By pre-setting dangerous areas on the gate and based on the vessel's intrusion location and authorization status, it quickly triggers level one or level two warnings with a short response time. The alarm device, through audible and visual alarms, LED screen displays, and linkage with the gate control system, achieves multi-level and multi-method early warning and response, effectively preventing vessel intrusion accidents. Attached Figure Description

[0017] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts. Wherein: Figure 1 This is a flowchart illustrating the working principle of this system. Detailed Implementation

[0018] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0019] The dual-mode early warning system proposed in this embodiment is applicable to gates, hydraulic structures, and port areas. The specific hardware configuration is as follows: A thermal imaging camera is installed on the top of the gate control room, covering a 180° field of view of the gate area; A visible light camera and a thermal imaging camera are coaxially mounted with an optical axis angle of ≤3° and fixed by a bracket to ensure that their spatial positions remain unchanged. An edge computing unit is responsible for real-time image processing and fusion algorithm operation; An alarm device includes an audible and visual alarm, an LED warning screen at the gate control center, and a linkage control interface.

[0020] The specific system module workflow is as follows, including: The thermal imaging detection module continuously acquires infrared images of the gate area to perform preliminary screening of ship targets. The specific steps are as follows: S110, after the system starts, it performs statistical analysis on the thermal imaging data of the previous 24 hours and extracts the average water surface temperature. (e.g., an average of 28 in summer) The average temperature in winter is 12. Temperature difference threshold (±3) The standard deviation of temperature in the still water zone (within 5m on both sides of the gate) is ≤0.5. Establish a reference temperature field model; Set initial temperature difference threshold And increase by 0.5 based on environmental parameters (e.g., when wind speed > 5 m / s). (The temperature will increase by 1°C during rainy weather) will be dynamically adjusted to... .

[0021] S120 uses a sliding time window (window length 5 minutes, sampling interval 1 second) to calculate the rate of change of temperature difference between the current frame and the previous 3 frames. If a certain area It was determined to be a rapid thermal change event; The concrete structure area of ​​the gate (where the temperature is relatively stable) was selected as the reference point. The temperature of any water body is Establish a local reference curve Short-term temperature fluctuations caused by water surface ripples are eliminated by using a moving average filtering method.

[0022] S130, for areas with thermal anomalies Perform contour analysis and calculate shape regularity (surrounding rectangle aspect ratio 1.5-5.0 is marked as a suspected ship), edge smoothness (contour point count / perimeter > 0.8 is marked as a suspected ship), and hot spot continuity (area change rate < 20% for 3 consecutive frames is marked as a suspected ship). The movement trajectory of the thermal anomaly area is extracted by optical flow method. If the angle between the trajectory direction and the channel axis is ≤30° and the speed is 0.5-5m / s (which is within the range of ship navigation speed), it is marked as a suspected ship. Summer mode (ambient temperature > 25°C) Prioritize detection of areas with abnormal high temperatures. ; Winter mode (ambient temperature < 15°C) Switch to low temperature anomaly detection ; Nighttime (light intensity < 10 lux): Enhance contrast in low-temperature areas (by expanding dynamic range through histogram equalization). Among other things, it can also use temperature change trends to identify moving targets, and can accurately detect even cold targets like ships.

[0023] S140 generates a binary heat source mask image, which includes the center coordinates of the suspected ship area (e.g., (x=15m, y=40m)), area (e.g., 200m²), and temperature difference (e.g., +3.2℃ in summer, -2.1℃ in winter).

[0024] The visible light image recognition module utilizes YOLO series algorithms to extract ship contours and confirm their categories, assisting in the verification of thermal imaging results. The specific steps are as follows: S210 performs CLAHE enhancement on visible light images (with a contrast threshold of 2.0) to improve details in backlight and low-light environments (such as improving the sharpness of ship hull edges by 40%). An improved YOLO-Seg lightweight segmentation network (input size 640×640) is adopted, and water surface reflection features are optimized through transfer learning (pre-trained dataset: such as COCO dataset) to output a candidate mask for ships (IOU≥0.75).

[0025] S220 uses the YOLOv8-nano model (with a water area attention mechanism added to the detection layer) to classify the segmented regions, outputting the ship category confidence score (e.g., 0.92) and the target bounding box coordinates. =1200px, =800px, =1800px, =1000px). Secondary validation is performed on low-confidence results (0.5 ≤ confidence < 0.8): edge detection of ship features (such as deck railings and cabin structure) (such as the Canny operator with a threshold of 50-150) is used to improve the reliability of recognition.

[0026] The image fusion and decision module, through spatial registration and information fusion, enables interactive verification of thermal imaging and visible light results. The specific steps are as follows: S310 employs the SIFT feature point matching algorithm to extract at least 50 matching feature points (such as gate edge corners and lighthouse outline points) from thermal and visible light images. False matches are eliminated using the RANSAC algorithm (interior point ratio ≥ 90%), and the affine transformation matrix is ​​calculated. (Error must be ≤1.5 pixels); based on the matrix Map the thermal imaging heat source mask to the visible light image coordinate system to achieve spatial alignment (the overlapping area needs to be ≥95%).

[0027] S320 performs pixel-level weighting (thermal imaging weight 0.6, visible light weight 0.4) on the thermal imaging temperature difference value and the visible light segmentation mask to generate a fused confidence map; If the IOU between the suspected ship area in the thermal imaging and the ship target bounding box in the visible light is ≥0.6, and the confidence level in the visible light is ≥0.8, then the ship is confirmed. If the single-modal detection meets the following criteria: thermal imaging spatiotemporal feature score ≥ 0.85 (when there is no visible light result) or visible light confidence score ≥ 0.95 (when there is no thermal imaging result), it is judged as a high-confidence ship; Other situations (such as thermal imaging misdetecting floating objects on the water surface, or visible light failing to detect small-tonnage vessels) are judged as invalid targets.

[0028] S330: Preset gate danger zone (a rectangular area 20m wide and 30m wide in front of the gate). If the center coordinates of a vessel or a vessel with high confidence are confirmed to have entered this area, and no permission signal for entry is received from the gate control center (obtained in real time via the AIS interface), an early warning will be triggered.

[0029] The alarm linkage module executes the following linkage operations when the system determines that a vessel has entered a dangerous area: S410, Level 1 Warning (ship 10-20m from the lock): The audible and visual alarm is activated (intermittent sounding, red light flashing), and the LED screen in the lock control center displays the ship's intrusion into the lock's dangerous area; S420, Level 2 Warning (vessel distance to gate < 10m): The alarm sounds continuously, sends an emergency gate closure signal to the gate control system via the Modbus interface, and records the trajectory of the intruding vessel (saving 1 frame of fused image per second for 30 seconds).

[0030] Example 1 - Nighttime Strong Light Interference Scenarios The visible light image was overexposed due to strong light reflection (pixel value in the hull area > 250), and the recognition confidence dropped to 0.42; The thermal imaging module detects the temperature difference (hull temperature 31℃ > water surface temperature 28℃ + 3℃) and analyzes the trajectory (speed 1.2m / s, direction pointing towards the gate) to output suspected vessels; The fusion module is primarily based on thermal imaging results and triggers a level-two warning in the unauthorized state, with a response time of less than 0.5 seconds.

[0031] Example 2 - Low Temperature Winter Scenario (Water Temperature 12℃, Hull Temperature 10℃): The thermal imaging module automatically switches to winter mode to detect areas with abnormal low temperatures (temperature difference -2℃ < -1.5℃ threshold), and determines them to be ships based on regular outlines (width-to-height ratio 3.2). The visible light image, due to insufficient illumination (brightness < 5 lux), was enhanced by CLAHE to segment the hull outline with a confidence level of 0.85. After fusion, a warning was confirmed, and a level 2 warning was triggered in the unauthorized state.

[0032] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. A smart early warning system for dangerous areas of ship lock gates based on dual cameras, characterized in that, include: A thermal imaging camera is installed on the top of the gate control room and configured to cover a 180° field of view of the gate area. A visible light camera is coaxially mounted with the thermal imaging camera, the optical axis angle between the two is ≤3°, and the camera is fixed by a bracket to ensure that the spatial position remains unchanged. An edge computing unit, connected to the thermal imaging camera and the visible light camera, is configured to receive and process image data acquired by the thermal imaging camera and the visible light camera; and An alarm device is connected to the edge computing unit. The alarm device includes an audible and visual alarm, a gate control center LED warning screen, and a linkage control interface.

2. The intelligent early warning system for dangerous areas of lock gates based on dual cameras according to claim 1, characterized in that, The alarm device's linkage control interface is configured as a Modbus interface for signal interaction with the gate control system.

3. The intelligent early warning system for dangerous areas of lock gates based on dual cameras according to claim 1, characterized in that, The edge computing unit is configured to dynamically adjust the temperature difference threshold according to environmental parameters and switch between summer mode, winter mode and night mode for thermal imaging images.

4. The intelligent early warning system for dangerous areas of lock gates based on dual cameras according to claim 3, characterized in that, The edge computing unit is configured to: activate summer mode when the ambient temperature is >25℃, prioritizing the detection of high-temperature abnormal areas; and activate winter mode when the ambient temperature is <15℃, prioritizing the detection of low-temperature abnormal areas. When the light intensity is less than 10 lux, activate the night mode to enhance the contrast in low-temperature areas.

5. The intelligent early warning system for dangerous areas of lock gates based on dual cameras according to claim 1, characterized in that, The edge computing unit is configured to perform contour analysis on thermal anomaly regions in thermal imaging images, including calculating shape regularity, edge smoothness, and hot spot continuity, and extracting the movement trajectory of thermal anomaly regions using optical flow.

6. The intelligent early warning system for dangerous areas of lock gates based on dual cameras according to claim 5, characterized in that, The edge computing unit is configured to mark thermal anomaly regions that meet at least one threshold region as suspected ships. The threshold regions include a bounding rectangle aspect ratio between 1.5 and 5.0, a ratio of contour points to perimeter greater than 0.8, and an area change rate of less than 20% for three consecutive frames.

7. The intelligent early warning system for dangerous areas of lock gates based on dual cameras according to claim 5, characterized in that, The edge computing unit is configured to mark thermal anomaly areas that simultaneously satisfy the following conditions: the angle between the movement trajectory direction and the channel axis is ≤30°, and the speed is in the range of 0.5-5m / s, as suspected ships.

8. The intelligent early warning system for dangerous areas of lock gates based on dual cameras according to claim 1, characterized in that, The edge computing unit is configured to perform CLAHE enhancement processing on visible light images and to identify ships using a segmentation network and a classification model.

9. The intelligent early warning system for dangerous areas of lock gates based on dual cameras according to claim 1, characterized in that, The edge computing unit is configured to spatially register and fuse the suspected ship area identified by the thermal imaging image with the ship target box identified by the visible light image, and determine it as a confirmed ship or a high-confidence ship when the preset conditions are met.

10. The intelligent early warning system for dangerous areas of lock gates based on dual cameras according to claim 1, characterized in that, The edge computing unit is configured to preset a gate danger zone. When it is confirmed that the center coordinates of a ship or a high-confidence ship have entered the danger zone and no gate control center has been received to allow entry, the alarm device is controlled to perform a first-level or second-level warning operation.

Images