A method and system for automatic identification of a black smoke emitting ship

Abstract

This invention discloses an automatic identification method and system for vessels emitting black smoke, aiming to solve the problem of rapid and accurate identification of such vessels. The method includes: recording initial parameters of a zoomable camera and acquiring channel video; capturing black smoke events using a multi-feature fusion black smoke detection algorithm; adjusting the camera via PID control and recording the vessel's 30-second navigation trajectory and heading; and filtering and matching identity information based on an "AIS + hull coding" dual-modal system, combined with vessel quantity and heading, supplemented by trajectory similarity and size error constraints to improve accuracy. The system includes modules such as a zoomable camera, an AI computing power unit, and a controller, integrating black smoke detection, trajectory tracking, identity matching, and data uploading. This invention achieves automatic identification of vessels emitting black smoke with strong real-time performance and high accuracy, enabling rapid evidence collection and providing technical support for channel vessel supervision.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection, and in particular to a method and system for automatic identification of ships emitting black smoke. Background Technology

[0002] Traditional methods of black smoke monitoring rely on manual on-site patrols and CCTV electronic patrols. However, when visibility is poor (such as in rain, snow, or fog) and when there is heavy ship traffic and the vessel is far from the waterway, these methods are insufficient to accurately identify the identity of suspected vessels.

[0003] Meanwhile, some solutions lack data security and have poor evidence integrity, making it difficult to achieve automated, high-precision identification and complete evidence recording of ships emitting black smoke. There is an urgent need for an automated, high-precision, and highly adaptable ship black smoke identification and identity association system. Summary of the Invention

[0004] To solve the above-mentioned technical problems, the present invention provides a method for automatic identification of ships emitting black smoke, comprising the following steps: S1 records the initial installation position, pitch angle, focal length, and distance S to the opposite channel of the zoom camera, and acquires video data for channel monitoring through the zoom camera; S2 uses a multi-feature fusion black smoke detection algorithm to process surveillance video data and automatically capture events of ships emitting black smoke in the video footage; S3 automatically adjusts the direction and focus of the camera through the controller to ensure that the black smoke vessel is always in the center of the frame, and records the direction of camera movement and the 30-second navigation trajectory of the black smoke vessel to determine the navigation direction d of the black smoke vessel, while recording evidence video. S4. Based on the AIS data of ships in the monitoring area and the ship code identification results obtained by the AIS receiver, a dual-modal identity matching system of "AIS + ship code" is constructed. Combining the number of ships and the navigation direction d, the identity information of ships emitting black smoke is screened and matched. The dual-modal identity matching system forms a two-way verification, main and backup combination, and real-time linkage identity confirmation mechanism.

[0005] Furthermore, in step S2, the multi-feature fusion black smoke detection algorithm integrates background filtering, detail enhancement, and multi-dimensional feature extraction techniques. The background filtering technology separates the channel background from the ship target area using an adaptive threshold segmentation algorithm. The threshold calculation formula for the adaptive threshold segmentation algorithm is as follows: Where μ(x,y) is the mean gray level of the local region centered at pixel coordinates (x,y), σ(x,y) is the standard deviation of the gray level of the local region, k is an adjustment coefficient, and T(x,y) is the adaptive segmentation threshold at pixel coordinates (x,y). The original pixel grayscale value, if If the pixel is positive, it corresponds to the candidate area for ships and black smoke; otherwise, it is a background interference pixel. The detail enhancement technique employs a multi-scale Retinex algorithm to improve the contrast and edge sharpness of the black smoke area, and strengthens the grayscale difference between the black smoke and the background. The formula for calculating the reflection component of the multi-scale Retinex algorithm is as follows: Where R(x,y) is the reflection component and N is the scale number. The original pixel grayscale value. For the nth scale weight, Let be the Gaussian kernel function at the nth scale. For convolution operations, It is a small positive number; then for Normalization and gain processing are performed: , where gain is the gain coefficient, min(R) and max(R) are the global minimum and maximum values ​​of R(x,y) in the current image, respectively, and R'(x,y) is the reflection component after normalized gain processing.

[0006] Furthermore, the multi-dimensional feature extraction technology extracts black smoke feature vectors from three dimensions: color space, texture features, and morphological features, and inputs them into an intelligent judgment model for classification and recognition. The intelligent judgment model employs an improved support vector machine (SVM) and optimizes the penalty parameter C and kernel function parameters through a grid search algorithm. To maximize classification accuracy on the validation set: Where Accuracy is the classification accuracy of the SVM model on the validation set. The number of samples in the validation set, where i is the sample index, ranging from 1 to... , For real labels, Predict labels for the model, This is an indicator function. Black smoke events are determined based on a preset classification confidence threshold. The confidence calculation formula is as follows: Where f(x) is the output value of the SVM decision function: p(x) is the confidence level. When p(x) is greater than a preset threshold, it is determined to be a black smoke event. Let be the Lagrange multiplier, and b be the bias term of the decision function. Let i be the true class label of the i-th support vector. Let x be the kernel function. i The samples selected as support vectors in the training set. For the new input sample currently being used for prediction and classification, This represents the number of support vectors.

[0007] Furthermore, in step S4, if only one vessel is detected in the monitoring area, the AIS data and hull code identification results of the vessel are extracted first. The consistency of the MMSI number in the AIS data and the hull code is verified. If the verification passes, the vessel is directly identified as the vessel emitting black smoke. If the AIS data is missing or invalid, the hull code is used as the core identity information to complete the identification.

[0008] Furthermore, in step S4, if two or more vessels are found in the monitoring area, firstly, vessels whose heading and sailing direction d are consistent in the AIS data are selected to form a candidate vessel set; then, the hull code of each vessel in the candidate vessel set is extracted and compared with the hull code identification result of the vessel emitting black smoke, and the vessel with the code that matches perfectly is locked to complete the identity confirmation.

[0009] Furthermore, combining the imaging principle and parameters of the zoom camera, a perspective projection model is used to calculate the theoretical pixel length occupied by each ship in the candidate ship set in the video frame: ,in, The pixel length of the ship in the image. Where f is the actual length of the ship, D is the real-time focal length of the camera, θ is the real-time distance from the camera to the ship, and s is the pixel-to-physical size conversion factor. Extract the physical length of the ship emitting black smoke from the video footage. Simultaneously calculate the theoretical deck area of ​​the ship. , The actual width of the ship, compared with the actual measured area of ​​the ship's deck in the video. Candidate ships are screened using multi-dimensional error E constraints: ,in, The weights for length error and area error are determined. When E is less than a preset threshold, the ship is considered to have passed the screening and is updated to the candidate ship set. Then, the target is locked again by comparing the hull code. If there are duplicate or unidentifiable hull codes among the candidate ships, the ship length and speed information in the AIS data are used to help distinguish them.

[0010] Furthermore, for the updated candidate vessel set, the Dynamic Time Warping (DTW) algorithm is used to quantify the similarity between the 30-second navigation trajectory of the vessel emitting black smoke and the navigation trajectories of the candidate vessels: ,in, This is a 30-second sequence of coordinates of the ship's trajectory emitting black smoke. The candidate ship AIS trajectory coordinate sequence. Represents the optimal alignment path that satisfies the DTW constraint, (i k , j k () represents the k-th alignment point pair in path W. For the first in the video trajectory Position coordinates at that moment The first one aligned with it in the AIS trajectory Time and location coordinates; normalized trajectory similarity The calculation formula is: ,in The maximum possible distance to the monitored area. A larger value indicates a higher degree of trajectory overlap.

[0011] Furthermore, in step S3, the recording of the evidence video begins from the identification of the black smoke event, and the video duration is determined by the following formula: ,in, For the final video recording duration, v represents the actual distance of the ship emitting black smoke over 30 seconds, and v is the ship's real-time speed. The video evidence must include a complete navigation trajectory of the vessel emitting black smoke and its hull code area. The clarity of the hull code area is determined using the following formula: Where R represents the area containing the ship's hull code in the video frame. This represents the total number of pixels in the region. Let be the Sobel gradient operators in the x and y directions, respectively, and C be the sharpness score; when the sharpness score... Greater than the resolution threshold When the code is clear and identifiable, it indicates that the encoded region is clearly identifiable. If there is no AIS signal feedback from ships within the monitored area, the video watermark is triggered under the following conditions: if the number of received AIS signal frames... Less than the AIS signal frame number threshold Then add a "No AIS signal" watermark to the video.

[0012] The present invention also provides an automatic identification system for ships emitting black smoke to implement the method, including a zoomable camera, an AI computing power stick, a controller, an AIS receiver, a 4G router, and a black smoke detection algorithm; The variable zoom camera is installed perpendicular to the waterway and has direction adjustment and zoom functions. It is used to intelligently capture black smoke from ships and obtain waterway monitoring video data. The AI ​​computing power stick is used to run the black smoke detection algorithm to automatically identify the phenomenon of ships emitting black smoke. The controller is used to automatically adjust the direction and focal length of the zoom camera based on the initial position and direction information of the black smoke vessel captured, so that the black smoke vessel is always in the center of the frame, and saves at least 30 seconds of evidence video. After the video recording ends, the camera position and direction are reset. The AIS receiver is used to receive AIS data from all ships within the camera's monitoring area; The 4G router is used to send automatically identified information about vessels emitting black smoke and video evidence to regulatory authorities.

[0013] Furthermore, the AIS data includes the vessel name, MMSI number, vessel length, speed, heading, and position information.

[0014] This invention integrates black smoke detection with ship identification, improving identification accuracy and real-time performance, effectively addressing the shortcomings of single identification methods, quickly securing evidence of violations, reducing regulatory costs, assisting in intelligent maritime supervision, and promoting environmental governance in shipping. Attached Figure Description

[0015] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0016] Figure 1 Flowchart of an automatic identification method for ships emitting black smoke.

[0017] Figure 2 This is a structural diagram of an automatic identification system module for ships emitting black smoke. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0019] Figure 1 This is a flowchart of the automatic identification method for ships emitting black smoke provided in an embodiment of the present invention. See also... Figure 1 This invention provides a method for automatic identification of ships emitting black smoke, comprising the following steps: S1 records the initial installation position, pitch angle, focal length, and distance S to the opposite channel of the zoom camera, and acquires video data for channel monitoring through the zoom camera; Specifically, the camera is an industrial-grade high-definition variable-focus network camera with a resolution of no less than 1920×1080 and a frame rate of 25fps. It supports 360° horizontal and -30° to 90° vertical electric adjustment. During initial installation, the distance S to the opposite channel is accurately calibrated using a laser rangefinder with an error controlled within ±0.5m. At the same time, the latitude and longitude coordinates and installation height of the camera are recorded. All initial parameters are stored in the parameter database of the local controller for easy retrieval and calibration later.

[0020] S2 uses a multi-feature fusion black smoke detection algorithm to process surveillance video data and automatically capture events of ships emitting black smoke in the video footage; Specifically, the algorithm runs on the embedded computing platform of the AI ​​computing stick, processing the monitoring video frame by frame. The frame processing time is no more than 40ms, which meets the requirements of real-time detection. When black smoke features are detected, the camera's capture command is triggered, and the original image, frame number, timestamp, and current camera parameters of the captured frame are saved simultaneously. The resolution of the captured image is consistent with that of the video frame, and the coordinates of the outer rectangle of the black smoke area are marked for subsequent analysis.

[0021] S3 automatically adjusts the camera's direction and focus via the controller to ensure the black smoke vessel remains centered in the frame. It records the camera's movement direction and the vessel's 30-second trajectory to determine its direction of travel (d), while simultaneously recording evidence video. Specifically, the controller uses a PID closed-loop control algorithm to adjust the camera, taking the deviation between the center pixel coordinates of the black smoke vessel area and the center pixel coordinates of the frame as input. It calculates the camera's horizontal and vertical adjustment angles and focus adjustment values, with an adjustment response time of less than 0.5 seconds, ensuring the vessel remains centered in the frame. The trajectory is recorded frame-by-frame by identifying the vessel's centroid coordinates and recording them in a time sequence. The coordinates are referenced to the camera's imaging plane. The sampling interval for the 30-second trajectory recording matches the video frame rate, totaling 750 sampling points. Finally, the trajectory is fitted using the least squares method, and the slope of the line represents the direction of travel (d), with directional accuracy controlled within ±1°.

[0022] S4. Based on the AIS data of ships in the monitoring area and the ship code identification results obtained by the AIS receiver, a dual-modal identity matching system of "AIS + ship code" is constructed. Combining the number of ships and the navigation direction d, the identity information of ships emitting black smoke is screened and matched. The dual-modal identity matching system forms a two-way verification, main and backup combination, and real-time linkage identity confirmation mechanism.

[0023] Specifically, the AIS receiver uses Class A shipborne AIS receiving equipment with receiving frequencies of 161.975MHz and 162.025MHz and a receiving distance of no less than 10km. It refreshes the ship AIS data within the monitoring area once per second, and extracts key information after data parsing and stores it in a temporary database. The ship hull code recognition is achieved by image cropping, character segmentation, and OCR recognition of the coded areas of the ship's bow and stern in the video image. It supports the recognition of numbers, letters, and maritime-specific coded characters, with a single character recognition accuracy of no less than 99% and a whole string code recognition accuracy of no less than 98%. The dual-modal identity matching system is integrated into the embedded software of the controller, with an independent matching thread that runs in parallel with the trajectory recording and camera adjustment threads to ensure the real-time performance of identity recognition.

[0024] Furthermore, in step S2, the multi-feature fusion black smoke detection algorithm integrates background filtering, detail enhancement, and multi-dimensional feature extraction techniques. Specifically, the three techniques adopt a pipelined processing architecture. First, the video frame is background filtered to remove background interference and obtain candidate regions for ships and black smoke. Then, the candidate regions are enhanced in detail to improve the recognizability of black smoke features. Finally, multi-dimensional features are extracted from the enhanced regions. The processing results of each step are cached in real time to avoid repeated calculations. At the same time, a feature validity verification step is set. If the extracted feature vector dimension is missing or the value is abnormal, the frame is directly discarded and no further classification and recognition are performed.

[0025] The background filtering technology separates the channel background from the ship target area using an adaptive threshold segmentation algorithm. The threshold calculation formula for the adaptive threshold segmentation algorithm is as follows: Where μ(x,y) is the mean gray level of the local region centered at pixel coordinates (x,y), σ(x,y) is the standard deviation of the gray level of the local region, k is an adjustment coefficient, and T(x,y) is the adaptive segmentation threshold at pixel coordinates (x,y). The original pixel grayscale value, if If the pixel is positive, it corresponds to the candidate area for ships and black smoke; otherwise, it is a background interference pixel. The detail enhancement technique employs a multi-scale Retinex algorithm to improve the contrast and edge sharpness of the black smoke area, and strengthens the grayscale difference between the black smoke and the background. The formula for calculating the reflection component of the multi-scale Retinex algorithm is as follows: Where R(x,y) is the reflection component and N is the scale number. The original pixel grayscale value. For the nth scale weight, Let be the Gaussian kernel function at the nth scale. For convolution operations, It is a small positive number; then for Normalization and gain processing are performed: , where gain is the gain coefficient, min(R) and max(R) are the global minimum and maximum values ​​of R(x,y) in the current image, respectively, and R'(x,y) is the reflection component after normalized gain processing; Specifically, the number of scales N is set to 3, corresponding to Gaussian kernel functions at small, medium, and large scales, with standard deviations of 15, 80, and 250 respectively, and corresponding weights. , , This design meets the requirements for enhancing multi-scale details in the black smoke region; the convolution operation is implemented using Fast Fourier Transform to reduce computational complexity; the small positive number ε is set to 1e-6 to avoid the case where the denominator is 0; the gain coefficient is dynamically adjusted according to the gray-level contrast of the black smoke region. For example, when the contrast is below 30, the gain is set to 1.5-2.0, and when the contrast is above 30, the gain is set to 1.0-1.5. The normalized reflection component values ​​are mapped to the gray-level range of 0~255 to ensure the compatibility of image display and subsequent processing.

[0026] The multi-dimensional feature extraction technology extracts black smoke feature vectors from three dimensions: color space, texture features, and morphological features. These vectors are then input into an intelligent judgment model for classification and recognition. The multi-dimensional feature vectors are fused using a weighted linear fusion formula: Where F is the fused feature vector, α1, α2, and α3 are the feature vectors extracted from color space, texture features, and morphological features, respectively, and the fusion weights corresponding to color space, texture features, and morphological features are respectively. Specifically, color space features are extracted from the HSV color space, selecting the mean and variance of the H channel, the mean and variance of the S channel, and the mean and variance of the V channel, for a total of six feature values, which constitute the color space characteristics. The dimension is 6×1; texture features are extracted using the gray-level co-occurrence matrix, selecting four texture parameters: contrast, correlation, energy, and homogeneity, to form a texture matrix. The dimension is 4×1; morphological features are extracted from the black smoke region using four parameters: area, perimeter, circularity, and rectangularity, to form a... The dimension is 4×1; the weights of each feature are determined through cross-validation on the training set, and the optimal value is... , , The fused feature vector F has a dimension of 14×1, and all feature values ​​are normalized and mapped to the range of 0~1.

[0027] Furthermore, the intelligent judgment model employs an improved Support Vector Machine (SVM), and optimizes the penalty parameter C and kernel function parameters through a grid search algorithm. To maximize classification accuracy on the validation set: Where Accuracy is the classification accuracy of the SVM model on the validation set. The number of samples in the validation set, where i is the sample index, ranging from 1 to... , For real labels, Predict labels for the model, For indicator functions; Specifically, the SVM model uses the radial basis function (RBF) as the kernel function to calculate the classification of the current sample. With the i-th support vector x i The similarity between them, expressed by the kernel function, is as follows: The parameter range for grid search is: , The step size is 2, and 5-fold cross-validation is used to evaluate the classification accuracy of each parameter combination. The parameter combination with the highest accuracy is selected as the optimal parameters of the model. The model training set contains 10,000 samples, including 5,000 positive samples (black smoke) and 5,000 negative samples (non-black smoke, such as white fog, ship exhaust, clouds, etc.). The validation set contains 2,000 samples, 1,000 positive samples and 1,000 negative samples. After the model is trained, the parameters are fixed to the AI ​​computing power stick and support offline operation.

[0028] Black smoke events are determined based on a preset classification confidence threshold. The confidence calculation formula is as follows: Where f(x) is the output value of the SVM decision function: p(x) is the confidence level. When p(x) is greater than a preset threshold, it is determined to be a black smoke event. Let be the Lagrange multiplier, and b be the bias term of the decision function. Let i be the true class label of the i-th support vector. Let x be the kernel function. i The samples selected as support vectors in the training set. For the new input sample currently being used for prediction and classification, This represents the number of support vectors.

[0029] Specifically, the classification confidence threshold is determined by the ROC curve of the validation set. The threshold corresponding to the point with the largest Youden index is selected, and the optimal threshold is set to 0.85. At this point, the precision of the model is not less than 95%, and the recall is not less than 94%. The number of support vectors should be 10% to 20% of the number of samples in the training set to reduce the computational cost of the model. is the Lagrange multiplier, b is the bias term of the decision function, and both are output parameters of the model training; when p(x) is greater than the threshold, the black smoke event judgment signal is immediately triggered, and the feature vector, confidence value, and decision function output value of the frame are recorded to the event log.

[0030] Furthermore, in step S4, if only one vessel is detected in the monitoring area, the AIS data and hull code identification results of the vessel are extracted first. The consistency of the MMSI number in the AIS data and the hull code is verified. If the verification passes, the vessel is directly identified as the vessel emitting black smoke. If the AIS data is missing or invalid, the hull code is used as the core identity information to complete the identification. Specifically, the consistency verification between the MMSI number and the hull code is achieved through the maritime vessel basic database. The controller accesses the remote maritime database through a 4G router, enters the MMSI number to query the corresponding registered hull code, and performs string matching with the locally identified hull code. During the matching, case sensitivity and spaces are ignored. If they match completely, the verification passes. The verification response time does not exceed 3 seconds. The criteria for judging missing or invalid AIS data are: if no AIS data for the vessel is received for 5 consecutive seconds, or if the received AIS data has missing fields or incorrect format, the locally identified hull code is directly used as the core identity information, and the identity recognition result is marked as "AIS data is invalid, hull code shall prevail".

[0031] Furthermore, in step S4, if two or more vessels are found in the monitoring area, firstly, vessels whose heading and sailing direction d are consistent in the AIS data are selected to form a candidate vessel set; then, the hull code of each vessel in the candidate vessel set is extracted and compared with the hull code identification result of the vessel emitting black smoke, and the vessel with the code that matches perfectly is locked to complete the identity confirmation. Specifically, the matching of heading and navigation direction d adopts an angle error tolerance mechanism. The heading in the AIS data is the actual track of the ship. When the angle difference between the heading and the navigation direction d does not exceed ±5°, it is determined to be consistent with the direction. The screening process is achieved through conditional queries in the database, which takes no more than 0.1 seconds. The hull code comparison adopts precise string matching and supports error tolerance processing for fuzzy characters in the code. If the identified hull code has one fuzzy character, it can be matched with information such as ship length and speed. If there are two or more fuzzy characters, the code matching result is discarded and filtered through other dimensions.

[0032] Furthermore, combining the imaging principle and parameters of the zoom camera, a perspective projection model is used to calculate the theoretical pixel length occupied by each ship in the candidate ship set in the video frame: ,in, The pixel length of the ship in the image. Where f is the actual length of the ship, D is the real-time focal length of the camera, θ is the real-time distance from the camera to the ship, and s is the pixel-to-physical size conversion factor. Specifically, the camera's field of view θ is calculated by interpolation based on the real-time focal length f using the focal length-field of view correspondence table provided by the manufacturer, with an accuracy of ±0.1°; the real-time distance D from the camera to the ship is measured in real time by a laser ranging module, with the measurement frequency consistent with the video frame rate, and the error controlled within ±1m; the pixel-physical size conversion factor s is the physical size of the camera's pixels, determined by the camera's hardware parameters, such as 3.75μm / pixel for a 1920×1080 resolution camera, and this parameter is fixed in the controller's parameter database; when calculating the theoretical pixel length, all parameters are taken from the same timestamp to ensure calculation accuracy, and the calculation result is retained to one decimal place.

[0033] Extract the physical length of the ship emitting black smoke from the video footage. Simultaneously calculate the theoretical deck area of ​​the ship. , The actual width of the ship, compared with the actual measured area of ​​the ship's deck in the video. Candidate ships are screened using multi-dimensional error E constraints: ,in, The weights for length error and area error are determined. When E is less than a preset threshold, the ship is considered to have passed the screening and is updated to the candidate ship set. Then, the target is locked again by comparing the hull code. If there are duplicate or unidentifiable hull codes among the candidate ships, the ship length and speed information in the AIS data are used to help distinguish them.

[0034] Specifically, the physical length of the ship in the video footage By identifying the pixel coordinates of the ship's bow and stern, and combining this with the pixel-to-physical-size conversion factor 's', the following calculation is performed: Where Δx is the difference in pixel x-coordinate between the bow and stern; the actual width of the ship. Extracted from AIS data; if AIS data is missing, it is queried from the maritime database based on MMSI number or hull code; actual measured deck area. The contour of the ship deck area in the video is extracted, and the pixel area of ​​the contour is calculated and then converted into the physical area; length error weighting. Area error weight The preset error threshold E is set to 0.15, meaning that when the comprehensive relative error between length and area does not exceed 15%, it is judged as passing the screening. When distinguishing by speed, ships whose real-time speed error with the ship emitting black smoke does not exceed ±1kn are selected. The speed data is extracted from AIS data and updated in real time.

[0035] Furthermore, for the updated candidate vessel set, the Dynamic Time Warping (DTW) algorithm is used to quantify the similarity between the 30-second navigation trajectory of the vessel emitting black smoke and the navigation trajectories of the candidate vessels: ,in, This is a 30-second sequence of coordinates of the ship's trajectory emitting black smoke. The candidate ship AIS trajectory coordinate sequence. Represents the optimal alignment path that satisfies the DTW constraint, (i k , j k () represents the k-th alignment point pair in path W. For the first in the video trajectory Position coordinates at that moment The first one aligned with it in the AIS trajectory Time and location coordinates; normalized trajectory similarity The calculation formula is: ,in The maximum possible distance to the monitored area. A larger value indicates a higher degree of trajectory overlap.

[0036] Specifically, the DTW algorithm employs dynamic programming, setting the alignment window width of the trajectory coordinate sequence to 50 sampling points to reduce computational load; the algorithm's processing time does not exceed 1 second. The maximum possible distance of the monitored area... The distance from the camera to the farthest point on the waterway is determined by the initial installation parameters and fixed in the controller; trajectory similarity. The value range is 0~1, when At that time, it was assumed that the trajectories of the two ships were highly overlapping. If so, the candidate vessel is directly excluded.

[0037] Identity matching and conflict resolution are performed using the following priority formula: ,in, For hull code matching degree, For trajectory overlap, To determine the validity of AIS data, β1, β2, and β3 are the weights corresponding to the ship hull code matching degree, trajectory similarity, and AIS data validity, respectively. The ship with the highest priority score is used as the initial target, and its final identity is determined through bidirectional verification of the ship hull code and AIS data. If there are multiple ships with a priority score difference less than a preset threshold and AIS data matching conflicts, the ship hull code recognition result is used as the priority judgment basis, and a manual review mechanism is triggered at the same time. Specifically, hull code matching degree It is either 0 or 1; 1 is used when the code is a complete match, and 0 is used otherwise; AIS data validity. The value is 0 or 1, with 1 indicating complete and real-time AIS data and 0 indicating missing, invalid, or delayed data. The weights of each dimension have been determined through extensive experiments to be β1=0.6, β2=0.2, and β3=0.2, highlighting the core matching role of the hull code. The preset score difference threshold is 0.05. If the score difference of multiple ships is less than this threshold, it is judged as a matching conflict. At this time, a manual review request is immediately sent to the regulatory department through the 4G router, along with the identity information, trajectory data, video clips, and recognition results of all candidate ships. The feedback results of the manual review will be synchronously updated to the system's event database.

[0038] Furthermore, in step S3, the recording of the evidence video begins from the identification of the black smoke event, and the video duration is determined by the following formula: ,in, For the final video recording duration, The actual navigation trajectory of the vessel emitting black smoke within 30 seconds is given by v, which represents the real-time speed of the vessel. The video duration must be no less than 30 seconds and must completely cover the vessel's navigation trajectory. Specifically, the actual navigation trajectory length of the vessel emitting black smoke within 30 seconds. The calculation is performed by accumulating the Euclidean distances of the trajectory coordinate sequences, in meters; the ship's real-time speed v is extracted from AIS data, in knots (kn), converted to meters per second (m / s) for calculation, using the following conversion formula: The evidence video uses H.265 encoding format with a bitrate of 8Mbps to ensure video clarity while saving storage and transmission bandwidth. The video is encapsulated in MP4 format and contains video stream and timestamp information. The video file name is named in the format of "device number_timestamp_ship MMSI code" for easy retrieval.

[0039] The video evidence must include a complete navigation trajectory of the vessel emitting black smoke and its hull code area. The clarity of the hull code area is determined using the following formula: Where R represents the area containing the ship's hull code in the video frame. This represents the total number of pixels in the region. Let be the Sobel gradient operators in the x and y directions, respectively, and C be the sharpness score; when the sharpness score... Greater than the resolution threshold When the code is clear and identifiable, it indicates that the encoded region is clearly identifiable. Specifically, the Sobel gradient operator uses a 3×3 convolution kernel, and the convolution operation is performed pixel-by-pixel, with the gradient value being the absolute value; the sharpness threshold... Through extensive experimental calibration, a value of 30 was set. When C>30, the OCR recognition accuracy of the hull code can be guaranteed to be above 98%. If C≤30 is detected, the controller will automatically increase the camera focal length to optimize the imaging effect of the coding area, and mark the position of the coding area in the evidence video to facilitate manual recognition.

[0040] If there is no AIS signal feedback from ships within the monitored area, the video watermark is triggered under the following conditions: if the number of received AIS signal frames... Less than the AIS signal frame number threshold Then add a "No AIS signal" watermark to the video; Specifically, the number of AIS signal frames The number of valid AIS signal frames received during the video recording for evidence purposes; AIS signal frame count threshold. Set to 10% of the total frames of the evidence video. The video was determined to have no valid AIS signal. The video watermark is a semi-transparent text watermark, added to the upper right corner of the video screen. The font is bold, the font size is 24, the color is red, and the transparency is 50%. It does not obscure the main body of the ship and the black smoke area. The watermark is displayed continuously from the beginning to the end of the video. At the same time, the "No AIS signal" tag is added to the video's metadata.

[0041] Based on another embodiment of the present invention, see [link to other embodiments]. Figure 2 Furthermore, an automatic identification system for ships emitting black smoke is provided to implement the aforementioned method, including a zoomable camera, an AI computing stick, a controller, an AIS receiver, a 4G router, and a black smoke detection algorithm; The variable zoom camera is installed perpendicular to the waterway and has direction adjustment and zoom functions. It is used to intelligently capture black smoke from ships and obtain waterway monitoring video data. Specifically, the camera is mounted on a monitoring pole or lighthouse beside the waterway using a dedicated bracket, at a height of no less than 10m to avoid being obstructed by ships or obstacles; the camera has an IP67 protection rating, supports anti-shake and anti-backlight, and is equipped with an infrared fill light, enabling clear imaging in the absence of light sources at night, with a fill light distance of no less than 50m; the camera's control commands are sent by the controller via the network, supporting remote parameter configuration and real-time preview, facilitating system debugging and maintenance.

[0042] The AI ​​computing stick is used to run the black smoke detection algorithm and automatically identify black smoke emitted by ships. Specifically, the AI ​​computing stick is an embedded neural network computing stick equipped with a quad-core ARM processor and a built-in neural network acceleration engine. Its computing power is no less than 8 TOPS and it supports the deployment of models in mainstream deep learning frameworks such as TensorFlow and PyTorch. The algorithm is burned into the computing stick in the form of an offline model. The model file size does not exceed 200MB. The computing stick operates at 5V and consumes no more than 10W. It can be directly powered by the controller and does not require an independent power supply.

[0043] The controller is used to automatically adjust the direction and focal length of the zoom camera based on the initial position and direction information of the black smoke vessel captured, so that the black smoke vessel is always in the center of the frame, and saves at least 30 seconds of evidence video. After the video recording ends, the camera position and direction are reset. Specifically, the controller uses an industrial-grade embedded microcontroller that can store the identification results of no less than 1,000 black smoke events and the corresponding evidence videos; the controller has a built-in real-time operating system RT-Thread, which supports multi-threaded task scheduling and has a response time of less than 1ms; the camera position and orientation reset command is sent within 1 second after the evidence video recording ends, restoring the camera to its initial installation position, pitch angle and focal length, ensuring the continuity of subsequent waterway monitoring.

[0044] The AIS receiver is used to receive AIS data from all ships within the camera's monitoring area. Specifically, the AIS receiver is equipped with a high-gain marine antenna, which is installed at a height higher than the camera to ensure the stability of the received signal. The receiver supports AIS data parsing and filtering, retaining only ship data within the monitoring area and filtering out invalid data outside the area. Data parsing follows the ITU-R M.1371 international standard. The receiver has a built-in data buffer that can cache AIS data from the most recent 5 minutes to avoid data loss. The cached data is sorted by timestamp for easy retrieval by the controller.

[0045] The 4G router is used to send automatically identified black smoke emitting vessel identification information and evidence videos to regulatory authorities. Specifically, the 4G router supports full-network 4G / 5G connectivity, with a downlink speed of no less than 100Mbps and an uplink speed of no less than 50Mbps to ensure fast transmission of evidence videos. The router supports wired network backup, automatically switching to wired broadband network when the 4G / 5G network fails. The transmitted data is encrypted using the AES-256 encryption algorithm to ensure data transmission security. The transmitted content is also packetized, with video files divided into 5MB packets to avoid transmission failures due to excessive data volume in a single transmission. The router has built-in traffic statistics and alarm functions; when monthly traffic exceeds a preset threshold, it sends traffic alarm information to regulatory authorities.

[0046] Furthermore, the AIS data includes the vessel name, MMSI number, vessel length, speed, heading, and position information; Specifically, AIS data also includes extended information such as vessel call sign, port of registry, draft, vessel type, and port of destination, facilitating comprehensive understanding of vessel status by regulatory authorities. Vessel position information is represented in WGS-84 latitude and longitude coordinates with an accuracy of 0.0001°, speed accuracy of 0.1 knots, and heading accuracy of 0.1°. All data is updated every 1 second to ensure real-time performance. The controller verifies the validity of the received AIS data, marking and discarding invalid data with missing fields, incorrect formats, or coordinates outside the monitoring area, retaining only valid data for identity matching.

[0047] It is understood that the system and units provided in this embodiment can also be used to implement the steps in the methods provided in other embodiments of the present invention.

[0048] The present invention also provides a computer device. The computer device is manifested in the form of a general-purpose computing device. The components of the computer device may include, but are not limited to: one or more processors or processing units, system memory, and buses connecting different system components.

[0049] Computer devices typically include a variety of computer system-readable media. These media can be any available media that can be accessed by a computer device, including volatile and non-volatile media, and removable and non-removable media.

[0050] The system memory may include a computer system readable medium in the form of volatile memory, and the memory may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0051] The processing unit executes various functional applications and data processing by running programs stored in the system memory, such as implementing the methods provided in other embodiments of the present invention.

[0052] The present invention also provides a storage medium containing computer-executable instructions and storing a computer program thereon, which, when executed by a processor, implements the methods provided in other embodiments of the present invention.

[0053] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for automatic identification of ships emitting black smoke, characterized in that, Includes the following steps: S1 records the initial installation position, pitch angle, focal length, and distance S to the opposite channel of the zoom camera, and acquires video data for channel monitoring through the zoom camera; S2 uses a multi-feature fusion black smoke detection algorithm to process surveillance video data and automatically capture events of ships emitting black smoke in the video footage; S3 automatically adjusts the direction and focus of the camera through the controller to ensure that the black smoke vessel is always in the center of the frame, and records the direction of camera movement and the 30-second navigation trajectory of the black smoke vessel to determine the navigation direction d of the black smoke vessel, while recording evidence video. S4. Based on the AIS data of ships in the monitoring area and the ship code identification results obtained by the AIS receiver, a dual-modal identity matching system of "AIS + ship code" is constructed. Combined with the number of ships and the navigation direction d, the identity information of ships emitting black smoke is screened and matched. The dual-modal identity matching system forms a two-way verification, main and backup combination, and real-time linkage identity confirmation mechanism. In step S4, if only one vessel is detected in the monitoring area, the AIS data and hull code identification results of the vessel are extracted first. The consistency between the MMSI number in the AIS data and the hull code is verified. If the verification passes, the vessel is directly identified as the vessel emitting black smoke. If the AIS data is missing or invalid, the hull code is used as the core identity information to complete the identification. If two or more vessels are detected in the monitoring area, vessels with the same heading and sailing direction d in the AIS data are first screened to form a candidate vessel set. Then, the hull codes of each vessel in the candidate vessel set are extracted and compared with the hull code identification results of the vessel emitting black smoke. The vessel with a completely matching code is locked to complete the identity confirmation. Based on the imaging principle and parameters of the zoom camera, a perspective projection model is used to calculate the theoretical pixel length occupied by each ship in the candidate ship set in the video frame: ,in, The pixel length of the ship in the image. Where f is the actual length of the ship, D is the real-time focal length of the camera, θ is the real-time distance from the camera to the ship, and s is the pixel-to-physical size conversion factor. Extract the physical length of the ship emitting black smoke from the video footage. Simultaneously calculate the theoretical deck area of ​​the ship. , The actual width of the ship, compared with the actual measured area of ​​the ship's deck in the video. Candidate ships are screened using multi-dimensional error E constraints: ,in, The weights for length error and area error are determined. When E is less than a preset threshold, the ship is considered to have passed the screening and is updated to the candidate ship set. Then, the target is locked again by comparing the hull code. If there are duplicate or unidentifiable hull codes among the candidate ships, the ship length and speed information in the AIS data are used to help distinguish them.

2. The method for automatic identification of ships emitting black smoke according to claim 1, characterized in that, In step S2, the multi-feature fusion black smoke detection algorithm integrates background filtering, detail enhancement, and multi-dimensional feature extraction techniques. Background filtering technology separates the channel background from the ship target area using an adaptive threshold segmentation algorithm. The threshold calculation formula for the adaptive threshold segmentation algorithm is as follows: Where μ(x,y) is the mean gray level of the local region centered at pixel coordinates (x,y), σ(x,y) is the standard deviation of the gray level of the local region, k is an adjustment coefficient, and T(x,y) is the adaptive segmentation threshold at pixel coordinates (x,y). The original pixel grayscale value, if If the pixel is positive, it corresponds to the candidate area for ships and black smoke; otherwise, it is a background interference pixel. The detail enhancement technology employs a multi-scale Retinex algorithm to improve the contrast and edge sharpness of the black smoke area, and strengthens the grayscale difference between the black smoke and the background. The formula for calculating the reflection component of the multi-scale Retinex algorithm is as follows: Where R(x,y) is the reflection component and N is the scale number. The original pixel grayscale value. For the nth scale weight, Let be the Gaussian kernel function at the nth scale. For convolution operations, It is a small positive number; then for Normalization and gain processing are performed: , where gain is the gain coefficient, min(R) and max(R) are the global minimum and maximum values ​​of R(x,y) in the current image, respectively, and R'(x,y) is the reflection component after normalized gain processing.

3. The method for automatic identification of ships emitting black smoke according to claim 2, characterized in that, The multi-dimensional feature extraction technology extracts black smoke feature vectors from three dimensions: color space, texture features, and morphological features. These vectors are then input into an intelligent judgment model for classification and identification. The intelligent judgment model employs an improved support vector machine (SVM) and optimizes the penalty parameter C and kernel function parameters using a grid search algorithm. To maximize classification accuracy on the validation set: Where Accuracy is the classification accuracy of the SVM model on the validation set. The number of samples in the validation set, where i is the sample index, ranging from 1 to... , For real labels, Predict labels for the model, For indicator functions; Black smoke events are determined based on a preset classification confidence threshold. The confidence calculation formula is as follows: Where f(x) is the output value of the SVM decision function: p(x) is the confidence level. When p(x) is greater than a preset threshold, it is determined to be a black smoke event. Let be the Lagrange multiplier, and b be the bias term of the decision function. Let K(x) be the true class label of the i-th support vector. i (x, ) is the kernel function, x i The samples selected as support vectors in the training set. For the new input sample currently being used for prediction and classification, This represents the number of support vectors.

4. The method for automatic identification of ships emitting black smoke according to claim 1, characterized in that, For the updated candidate vessel set, the Dynamic Time Warping (DTW) algorithm is used to quantify the similarity between the 30-second navigation trajectory of the vessel emitting black smoke and the navigation trajectories of the candidate vessels: ,in, This is a 30-second sequence of coordinates of the ship's trajectory emitting black smoke. The candidate ship AIS trajectory coordinate sequence. Represents the optimal alignment path that satisfies the DTW constraint, (i k , j k () represents the k-th alignment point pair in path W. For the first in the video trajectory Position coordinates at that moment The first one aligned with it in the AIS trajectory Time and location coordinates; normalized trajectory similarity The calculation formula is: ,in The maximum possible distance to the monitored area. A larger value indicates a higher degree of trajectory overlap.

5. The method for automatic identification of ships emitting black smoke according to claim 4, characterized in that, In step S3, the recording of the evidence video begins from the identification of the black smoke event, and the video duration is determined by the following formula: ,in, For the final video recording duration, v represents the actual distance of the ship emitting black smoke over 30 seconds, and v is the ship's real-time speed. The video evidence must include a complete navigation trajectory of the vessel emitting black smoke and its hull code area. The clarity of the hull code area is determined using the following formula: Where R represents the area containing the ship's hull code in the video frame. This represents the total number of pixels in the region. Let be the Sobel gradient operators in the x and y directions, respectively, and C be the sharpness score; when the sharpness score... Greater than the resolution threshold When the code is clear and identifiable, it indicates that the encoded region is clearly identifiable. If there is no AIS signal feedback from ships within the monitored area, the video watermark is triggered under the following conditions: if the number of received AIS signal frames... Less than the AIS signal frame number threshold Then add a "No AIS signal" watermark to the video.

6. An automatic identification system for ships emitting black smoke, used to implement the method according to any one of claims 1-5, characterized in that, This includes a zoom camera, an AI computing power stick, a controller, an AIS receiver, a 4G router, and a black smoke detection algorithm; The variable zoom camera is installed perpendicular to the waterway and has direction adjustment and zoom functions. It is used to intelligently capture black smoke from ships and obtain waterway monitoring video data. The AI ​​computing power stick is used to run the black smoke detection algorithm to automatically identify the phenomenon of ships emitting black smoke. The controller is used to automatically adjust the direction and focal length of the zoom camera based on the initial position and direction information of the black smoke vessel captured, so that the black smoke vessel is always in the center of the frame, and saves at least 30 seconds of evidence video. After the video recording ends, the camera position and direction are reset. The AIS receiver is used to receive AIS data from all ships within the camera's monitoring area; The 4G router is used to send automatically identified information about vessels emitting black smoke and video evidence to regulatory authorities.

7. The automatic identification system for ships emitting black smoke according to claim 6, characterized in that, The AIS data includes the vessel name, MMSI number, vessel length, speed, heading, and location information.

Images