Bimodal verification-based camouflage ship identification system and working method

Abstract

The invention discloses a disguise ship identification system based on dual-mode verification and a working method. The system comprises an FPGA circuit board, a width sensor, a dual-band light splitting synchronous imaging device, a light supplementing LED array and a computer. The bimodal imaging mechanism breaks through the limitation of a single mode, and through the light supplementing LED array and visible light and near-infrared bimodal fusion, identification can still be performed in severe scenes such as night and rainstorm, and 24-hour all-weather monitoring is realized. According to the method, near-infrared segmentation is guided through the visible light character contour, the smearing area is accurately positioned, and the recognition accuracy is high. According to the invention, through the combination of the number of visible light stroke intersection points and the area of the near-infrared connected region, the correction identification confidence is improved. According to the invention, electromagnetic interference is synchronously eliminated by adopting optical fiber transmission and PTP, the time error of a 10km link is less than 500ns, and the anti-radar interference capability is more than 90% better than that of a copper cable scheme.

Description

Technical Field

[0001] This invention belongs to the field of ship supervision technology, specifically relating to a disguised ship identification system and working method based on dual-modal verification, applicable to ship supervision scenarios in waterways such as ports, waterways, and dams. Background Technology

[0002] With the increasing frequency of maritime transport, fishing, and military activities, effective monitoring and identification of vessels has become crucial. In maritime safety monitoring, obscuring vessel names is a common camouflage tactic. By covering or altering the vessel's name characters (using materials such as paper, tape, or paint), the true identity is concealed to facilitate activities such as smuggling, illegal fishing, or military camouflage. Such camouflage poses a significant challenge to the real-time performance and accuracy of maritime monitoring systems.

[0003] With the development of image processing and pattern recognition technologies, image-based ship recognition technology has gradually emerged.

[0004] Single-modal image recognition, such as using only visible light images to identify altered ship names, has significant limitations: the reflectivity of the material used to paint the ship name in the visible light band is similar to that of the ship's hull or the original name characters, which can easily lead to confusion in color information during image segmentation. For example, when using paint similar to the background color of the ship name to paint the characters, edge extraction and binarization processing of the visible light image can be interfered with by false colors, resulting in the loss of character outlines and the identification of the incorrect altered ship name. Summary of the Invention

[0005] To address the aforementioned problems in existing technologies, this invention aims to design a dual-modal verification-based system and method for identifying camouflaged ships that can accurately identify altered ship names.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows: a camouflaged ship identification system based on dual-modal verification, comprising an FPGA circuit board, a width sensor, a dual-band beam splitting synchronous imaging device, a supplementary LED array, and a computer; The dual-band beam splitting synchronous imaging device is installed on one side of the waterway by a fixed bracket. The dual-band beam splitting synchronous imaging device includes an optical lens, a beam splitting prism, a visible light filter, a near-infrared filter, a visible light camera, and a near-infrared camera. After the incident light enters the optical lens, it is split into two paths by the beam splitting prism. One path enters the visible light camera through the visible light filter, and the other path enters the near-infrared camera through the near-infrared filter.

[0007] The width sensor is an ultrasonic sensor, which is installed on the front side of the dual-band beam splitting imaging device, i.e. the side closest to the entrance of the navigation channel, by a fixed bracket. It is used to detect whether a ship is passing through the channel. When a ship passes through, the width detected by the width sensor becomes narrower. At this time, the FPGA circuit board sends a signal to make the dual-band beam splitting synchronous imaging device and the supplementary LED array start working. The FPGA circuit board is fixedly mounted on the dual-band beam splitting synchronous imaging device. It is connected to the width sensor via a 485 communication line to receive signals transmitted from the width sensor; it is connected to the visible light camera via a network cable and to the near-infrared camera via a D-Sub cable for data transmission and reception.

[0008] The supplementary LED array is mounted 10cm above the dual-band beam splitting synchronous imaging device and angled downwards at 45° using a fixed bracket. It is connected to the GPIO interface of the near-infrared camera via a dedicated synchronous trigger line, with a supplementary lighting distance of 40-60m. The computer is connected to the FPGA circuit board via optical fiber and receives image information from the visible light camera and near-infrared camera after the FPGA circuit board has processed the image information.

[0009] A working method for a camouflaged ship identification system based on dual-modal verification includes the following steps: D. Simultaneous acquisition of dual-modal images; E. Image preprocessing and registration; F. Cross-modal feature extraction and fusion; D. Anomaly detection in fusion results; E. Results Output and Early Warning; Output the verification result. If the result indicates that the ship name has been altered, trigger an alert and stop the ship.

[0010] Furthermore, the method for simultaneous acquisition of dual-modal images described in step A is as follows: When a ship passes by, the width sensor detects a narrowing of the width. The FPGA circuit board transmits the detected width to the computer. When the computer detects that the narrowing of the width lasts for 5 seconds, it sends information to the FPGA circuit board. The FPGA circuit board then controls the visible light camera and the near-infrared camera to trigger synchronous operation. The shutter signal of the near-infrared camera controls the activation of the supplementary LED array, which uniformly illuminates the ship name area. The dual-band beam splitting synchronous imaging device uses a 45° beam splitter to split the incident light into two paths: visible light and near-infrared light. After passing through a 400-700nm bandpass filter and an 850nm narrow bandpass filter, the visible light camera and the near-infrared camera synchronously acquire images of the ship name area, sharing a lens to ensure overlapping fields of view. Furthermore, the image preprocessing and registration method described in step B is as follows: Visible light and near-infrared cameras transmit image information captured simultaneously to the FPGA circuit board. The computer acts as the PTP time server, i.e., the master clock. The PTP slave clock module inside the FPGA circuit board injects timestamps into both the visible light and near-infrared images, ensuring spatiotemporal alignment of the dual-modal images. The FPGA circuit board transmits the processed dual-modal images to the computer via fiber optic cable. Upon receiving the images, the computer performs dual-modal image calibration: calibrating the acquisition times of the visible light and near-infrared cameras using the PTP protocol, and utilizing feature point matching to ensure pixel-level alignment of the image regions. Furthermore, the method for cross-modal feature extraction and fusion described in step C is as follows: The computer performs cross-modal feature extraction on the calibrated bimodal images: Visible light images undergo adaptive histogram equalization to enhance character contrast, followed by binarization and grayscale processing to extract character features. Near-infrared images are processed with median filtering to remove salt-and-pepper noise. The binarized region of the visible light character mask (character outline) serves as a spatial reference to guide near-infrared image segmentation, precisely dividing the near-infrared ship name region into "character areas" and "background areas" that perfectly correspond to the visible light character positions. Key features from both modalities are fused: the "character position and shape" of the visible light image is used to anchor the analysis area, while the "material reflectance characteristics" of the near-infrared image are used to determine whether the area is abnormal. Furthermore, the method for detecting anomalies in the fusion result described in step D is as follows: Binarize the character area and background area, calculate the area of ​​the largest connected region of white pixels and the pixel ratio, and compare it with an empirical threshold to determine whether the ship name has been altered.

[0011] Compared with the prior art, the present invention has the following beneficial effects: 1. The dual-modal imaging mechanism in this invention overcomes the limitations of single-modal imaging. In existing single-modal technologies, visible light image recognition depends on illumination, and infrared images lack texture details. This invention, through supplementary LED arrays and visible light + near-infrared dual-modal fusion, can still recognize images in harsh environments such as nighttime and heavy rain, achieving 24-hour all-weather monitoring.

[0012] 2. Existing dual-modal solutions mostly use simple weighted fusion, resulting in low recognition accuracy. This invention guides near-infrared segmentation with visible light character outlines to accurately locate the smeared area, achieving high recognition accuracy.

[0013] 3. Existing fusion algorithms do not construct feature association models. This invention improves the confidence of tamper recognition by using a joint feature vector of "number of visible light stroke intersections + area of ​​near-infrared connected regions".

[0014] 4. This invention uses optical fiber transmission + PTP synchronization to eliminate electromagnetic interference. The time error of a 10km link is <500ns, and its anti-radar interference capability is more than 90% better than that of copper cable solutions.

[0015] 5. The dual-band beam splitting imaging device of the present invention uses a single optical lens, which is less expensive than the existing dual-camera dual-lens solution. At the same time, hardware-level parallax elimination (registration error < 0.5 pixels) does not require complex software correction. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the camouflage ship identification method of the present invention.

[0017] Figure 2 This is a schematic diagram illustrating an application example of the system of the present invention.

[0018] Figure 3 This is a schematic diagram of the dual-band beam splitting and synchronous imaging device. Detailed Implementation

[0019] To make the technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0020] This invention is applicable to ship identification verification scenarios in waterways such as ports, waterways, and dams. By analyzing the differences in the reflectivity of ship surface materials in the near-infrared band, it can accurately identify camouflage behaviors such as forged ship names and altered markings, and achieve early warning of interception of camouflaged ships.

[0021] like Figure 1 As shown, the key technical steps of this invention are divided into dual-modal image synchronous acquisition, image preprocessing and registration, cross-modal feature extraction and fusion, fusion result anomaly detection, and result output and early warning.

[0022] like Figure 2 As shown, the dual-band beam splitting synchronous imaging device is mounted on one side of the waterway using a fixed bracket. The width sensor is installed 5cm in front of it. The FPGA circuit board is mounted on the dual-band beam splitting synchronous imaging device. The supplementary LED array is mounted above the dual-band beam splitting synchronous imaging device via a fixed bracket and is connected to the GPIO interface of the near-infrared camera via a dedicated synchronous trigger line. The computer is connected to the FPGA circuit board via optical fiber. The FPGA circuit board is connected to the dual-band beam splitting synchronous imaging device and receives the image information sent by it.

[0023] like Figure 3 As shown, the dual-band beam splitting synchronization device consists of an optical lens, a beam splitting prism, a visible light filter, a near-infrared filter, a visible light camera, and a near-infrared camera.

[0024] When a ship enters the detection area, the width sensor detects the ship's passage, and the FPGA circuit board triggers the dual-band beam splitting synchronous imaging device to work, acquiring visible light and near-infrared images of the ship's name area. At this time, the supplementary LED array is turned on synchronously, enhancing the difference in reflectivity between the coating material and the ship's hull to form a bright area with a grayscale difference ≥15 in the near-infrared image. A 12mm fixed-focus optical lens converges the incident light (400~1100nm), and the amount of light entering the lens is controlled by an adjustable aperture (F1.4~F16). A beam splitter mounted at a 45° angle utilizes the coating properties to reflect 400~700nm visible light into the visible light channel and transmit 800~1100nm near-infrared light into the near-infrared channel, achieving physical-level dual-band separation with an optical axis parallelism error of <0.5°. The visible light channel uses a 400~700nm bandpass visible light filter to remove near-infrared stray light, while the near-infrared channel uses an 850nm narrowband near-infrared filter (bandwidth ≤50nm) to focus the reflection peak band of the coated material. The visible light camera and the near-infrared camera receive light signals of their respective bands, and the shared lens ensures complete overlap of the field of view. The supplementary LED array is triggered synchronously with the near-infrared camera shutter hardware (error < 1ms) to enhance the contrast of near-infrared images. After the images are acquired, they are transmitted to the FPGA circuit board. The FPGA circuit board uses PTP to achieve stamp synchronization (error < 1μs) and uses PTP to achieve time synchronization, providing a spatiotemporal consistency basis for subsequent cross-modal feature matching. PTP, or Precision Time Protocol, is a protocol for achieving sub-microsecond clock synchronization through network message exchange. In a dual-modal system, its core objectives are: to ensure that the timestamp error of image acquisition from visible light and near-infrared cameras is <1μs, eliminating spatiotemporal misalignment caused by ship movement; and to provide a unified time reference for "cross-modal feature matching," ensuring strict alignment of dual-band images from the same ship on the timeline. The GPS timing server (i.e., the computer) at the shore-based monitoring center serves as the master clock, and the PTP clock module on the FPGA circuit board acts as the slave clock, forming a star-shaped synchronization network via fiber optic links. The master clock (computer) sends Sync messages via fiber optic cable, and the PTP clock module on the FPGA circuit board hardware records the transmission time t1 (nanosecond precision). The slave clock receives the Sync message, hardware records t2, and obtains t1 through a Follow_Up message. The slave clock sends a Delay_Req message, recording t3; the master clock receives and records t4, feeding back t4 through a Delay_Resp message. The time deviation is calculated as follows: Path delay Td = (1) Time Deviation Offset = (2) The local clock is adjusted based on the offset to ensure a synchronization error with the master clock of <1μs (measured error <500ns for a 10km fiber optic link). During the initial calibration, 100 sets of PTP messages are sent to calculate the average one-way delay (e.g., 50μs delay for a 10km fiber optic link). Subsequent synchronizations will directly deduct this fixed delay.

[0025] After receiving the image information processed by the FPGA circuit board via optical fiber, the computer calibrates the acquisition times of the visible light camera and the infrared camera using the PTP protocol. Feature point matching (such as SIFT) is used to ensure pixel-level alignment of the ship name region in the dual-modal image. The visible light image undergoes adaptive histogram equalization to enhance character contrast: the ship name image acquired by the visible light camera is divided into 8×8 pixel sub-blocks, and the grayscale histogram of each sub-block is calculated. The equalization transformation of each sub-block is performed as follows: s k =T(r k )= (3) Where, r k Let be the k-th gray level value within the sub-block, ni be the number of times this gray level appears, n be the total number of pixels in the sub-block, and L be the gray level (usually 256). Contrast limiting is introduced to limit the peak value of the sub-block histogram, preventing noise from being highlighted due to excessive enhancement. The edges of the ship name area are sharpened by enhancing the stroke edge gradient using the Sobel operator to improve character recognition. Near-infrared images are filtered using median filtering to remove salt-and-pepper noise: the near-infrared image is divided into a 3×3 sliding window, the gray values ​​of 9 pixels within the window are collected and sorted, and the median value is used to replace the center pixel of the window. The visible light image is grayscaled and binarized to generate a clear black-and-white character mask using the classic grayscale conversion formula: Y = 0.299R + 0.587G + 0.114B (4) By using weighted coefficients to highlight the green component that is sensitive to the human eye and reducing redundant color information, the 3-channel color image is converted into a single-channel grayscale image, reducing the computational load of subsequent processing. An adaptive thresholding algorithm is used for binarization: the Otsu thresholding method automatically calculates the optimal segmentation threshold T, minimizing the inter-class variance based on the bimodal characteristic of the grayscale histogram (characters and background form two peaks). T=argmin t [ω1(t)σ1 2 (t)+ω2(t)σ2 2 (t)](5)where, ω1(t) = number of background pixels / total number of image pixels, ω2(t) = number of character pixels / total number of image pixels, ω1 + ω2 = 1, reflecting the distribution ratio of the two types of pixels; σ1 2 (t) = the grayscale variance of the background pixels, σ²2 (t) = the grayscale variance of the character pixels. The smaller the variance, the more concentrated the grayscale of this type of pixel. This method is highly adaptable to scenes with uneven lighting (such as ship shadows). Near-infrared image segmentation is guided by visible light character masks, dividing the near-infrared ship name region into character and background regions: the visible light character mask is used as a priori region, and watershed segmentation is performed after superimposing it with the near-infrared image. A gradient magnitude map is calculated for the near-infrared image; the region inside the mask is set as foreground seed points, and the region outside the mask is set as background seed points. The watershed algorithm is executed based on the gradient map, avoiding the oversegmentation problem of traditional threshold segmentation. The character and background regions are binarized, and the area of ​​the largest connected region of white pixels and the pixel ratios R1 and R2 are calculated. R1 represents the proportion of non-ship material in the character region, and R2 represents the proportion of non-ship material in the background region. The pixel ratio calculation formula is as follows: R1= (6) R12= (7) By comparing the pixel ratios R1 and R2 of the character area to the background area in the near-infrared image with empirical thresholds T1 and T2 (typically T1 is 0.15~0.25 and T2 is 0.10~0.18), it is determined whether there is any non-hull material covering. Combined with character integrity analysis of the visible light image, alteration patterns such as stroke additions / reductions and overlays are identified. Temporal analysis of five consecutive frames is performed; if abnormal reflectivity is observed in the same area for three consecutive frames, it is confirmed as deliberate alteration, excluding false features such as shadows caused by sea wave reflections and object occlusion. The ship name alteration status (normal / altered) is output, triggering an early warning and activating the stop device for ships with altered names.

[0026] The present invention has the following embodiments: The invention is installed at a fixed monitoring station in the port, monitoring a channel width of 80m. The dual-band beam splitting synchronous imaging device faces the centerline of the channel and is installed at a tilt angle of 15° (downward angle). The width sensor is installed 5cm in front of it. The FPGA circuit board is installed on the dual-band beam splitting synchronous imaging device. The 850nm supplementary light LED array is installed 10cm above the imaging device, illuminating downward at a 45° angle. The supplementary light distance is 50~80m. It is connected to the GPIO interface of the near-infrared camera through a dedicated trigger line. The synchronous trigger delay is <500μs to ensure clear imaging of the ship name area. Employing a 12mm lens with an adjustable aperture of F1.4~F16 and a field of view of 65°×49°, featuring a 45° tilt coated prism, the system boasts a visible light (400~700nm) reflectivity ≥95% and a near-infrared (800~1100nm) transmittance ≥90%. It utilizes a visible light 400~700nm bandpass filter and a near-infrared 850nm narrowband filter. Data is simultaneously acquired by a visible light camera (2-megapixel CMOS) and a near-infrared camera (InGaAs sensor). A 5km single-mode fiber optic cable is laid to the port monitoring center, supporting 10Gbps transmission. Dual-camera timestamp synchronization (error <500ns) is achieved via the PTP protocol: the master clock is a GPS timing server (UTC accuracy ±100ns), and the slave clock is a TI DP83640 chip integrated into both cameras, with a timestamp error <300ns. A Sync message is sent every 100ms to dynamically calibrate the transmission delay. When a vessel with altered name enters the monitoring area (e.g., changing the number "8" to "0"), the FPGA circuit board triggers the simultaneous activation of the dual-band beam splitting synchronous imaging device and the supplementary LED array. This acquires visible light and near-infrared images of the vessel's name within the same field of view, transmitting them to a computer via fiber optic cable. The computer performs adaptive histogram equalization and Otsu binarization on the visible light image, creating a clear black-and-white character mask. This mask guides near-infrared image segmentation, dividing the near-infrared vessel name region into a character area and a background area. The character and background areas are then binarized. The maximum connected region area and pixel ratio of white pixels are calculated: R1=0.28, R2=0.17. Empirical thresholds are T1=0.2, T2=0.15, R1>T1, R>T2, and this ratio is maintained for four consecutive [percentages]. Frame anomalies trigger warnings and coordinate with port interception devices to implement control measures. In this embodiment, the system achieves an 89.7% success rate in identifying altered vessels, with a nighttime false alarm rate of less than 5.3%, effectively verifying the engineering feasibility of dual-modal fusion recognition.

[0027] This invention is not limited to this embodiment. Any equivalent concept or modification within the technical scope disclosed in this invention shall be included within the protection scope of this invention.

Claims

1. A camouflaged ship identification system based on dual-modal verification, characterized in that: Includes an FPGA circuit board, a width sensor, a dual-band beam splitting and synchronous imaging device, a supplementary LED array, and a computer; The dual-band beam splitting synchronous imaging device is installed on one side of the waterway by a fixed bracket. The dual-band beam splitting synchronous imaging device includes an optical lens, a beam splitting prism, a visible light filter, a near-infrared filter, a visible light camera, and a near-infrared camera. After the incident light enters the optical lens, it is split into two paths by the beam splitting prism. One path enters the visible light camera through the visible light filter, and the other path enters the near-infrared camera through the near-infrared filter. The width sensor is an ultrasonic sensor, which is installed on the front side of the dual-band beam splitting imaging device, i.e. the side closest to the entrance of the navigation channel, by a fixed bracket. It is used to detect whether a ship is passing through the channel. When a ship passes through, the width detected by the width sensor becomes narrower. At this time, the FPGA circuit board sends a signal to make the dual-band beam splitting synchronous imaging device and the supplementary LED array start working. The FPGA circuit board is fixedly mounted on the dual-band beam splitting synchronous imaging device. It is connected to the width sensor via a 485 communication line to receive the signal transmitted from the width sensor; it is connected to the visible light camera via a network cable and to the near-infrared camera via a D-Sub cable for data transmission and reception. The supplementary LED array is mounted 10cm above the dual-band beam splitting synchronous imaging device and angled downwards at 45° using a fixed bracket. It is connected to the GPIO interface of the near-infrared camera via a dedicated synchronous trigger line, with a supplementary lighting distance of 40-60m. The computer is connected to the FPGA circuit board via optical fiber and receives image information from the visible light camera and near-infrared camera after the FPGA circuit board has processed the image information.

2. A working method for a camouflaged ship identification system based on dual-modal verification, characterized in that: Includes the following steps: A. Simultaneous acquisition of dual-modal images; B. Image preprocessing and registration; C. Cross-modal feature extraction and fusion; D. Anomaly detection in fusion results; E. Results Output and Early Warning; Output the verification result. If the result indicates that the ship name has been altered, trigger an alert and stop the ship.

3. The working method of the camouflage ship identification system based on dual-modal verification according to claim 2, characterized in that: The method for simultaneous acquisition of dual-modal images described in step A is as follows: When a ship passes by, the width sensor detects a narrowing of the width. The FPGA circuit board transmits the detected width to the computer. When the computer detects that the narrowing of the width lasts for 5 seconds, it sends information to the FPGA circuit board. The FPGA circuit board controls the visible light camera and the near-infrared camera to trigger synchronous operation. The shutter signal of the near-infrared camera controls the activation of the supplementary LED array, which uniformly illuminates the ship name area. The dual-band beam splitting synchronous imaging device splits the incident light into two paths, visible light and near-infrared light, through a 45° beam splitter prism. After passing through a 400~700nm bandpass filter and an 850nm narrow bandpass filter, the visible light camera and the near-infrared camera synchronously acquire images of the ship name area, sharing a lens to ensure overlapping fields of view.

4. The working method of the camouflage ship identification system based on dual-modal verification according to claim 2, characterized in that: The image preprocessing and registration method described in step B is as follows: The visible light camera and the near-infrared camera transmit image information captured at the same time to the FPGA circuit board; the computer acts as the PTP time server, i.e., the master clock, and the PTP slave clock module inside the FPGA circuit board injects timestamps into the visible light image and the near-infrared image to ensure spatiotemporal alignment of the dual-modal images; the FPGA circuit board transmits the processed dual-modal images to the computer through optical fiber, and the computer performs dual-modal image calibration after receiving the images: calibrating the acquisition time of the visible light camera and the near-infrared camera through the PTP protocol, and using feature point matching to ensure pixel-level alignment of the image regions.

5. The working method of the camouflage ship identification system based on dual-modal verification according to claim 2, characterized in that: The method for cross-modal feature extraction and fusion described in step C is as follows: The computer performs cross-modal feature extraction on the calibrated dual-modal images: the visible light image enhances character contrast through adaptive histogram equalization, and then extracts character features through binarization and grayscale processing; the near-infrared image uses median filtering to remove salt-and-pepper noise; the near-infrared image is segmented using the binarized region of the visible light character mask (i.e., the character outline) as a spatial reference, accurately dividing the near-infrared ship name region into "character area" and "background area" that completely correspond to the visible light character positions; the key features of the two modalities are fused: the "character position and shape" of the visible light is used to anchor the analysis area, and the "material reflection characteristics" of the near-infrared are used to determine whether the area is abnormal.

6. The working method of the camouflage ship identification system based on dual-modal verification according to claim 2, characterized in that: The method for detecting anomalies in the fusion results described in step D is as follows: Binarize the character area and background area, calculate the area of ​​the largest connected region of white pixels and the pixel ratio, and compare it with an empirical threshold to determine whether the ship name has been altered.

Images