A method and system for monitoring ship black smoke events

Abstract

The application provides a ship black smoke event monitoring method and system. The method comprises the following steps: acquiring a ship video stream, performing normal interval identification on the ship video stream at a first time interval based on a black smoke identification algorithm; in response to identifying a black smoke frame, recording a current first time, and performing interval encryption identification on the video stream at a second time interval based on the black smoke identification algorithm; in response to the continuous duration of the interval encryption identification no longer identifying the black smoke frame satisfying a third time interval, recording a current second time; and determining whether to generate black smoke event alarm information based on the ship video stream from the first time to the second time, and outputting the result. The method provided by the application enhances the reliability and monitoring effect of ship black smoke monitoring.

Description

Technical Field

[0001] This application relates to the field of intelligent detection, and more specifically, to a method and system for monitoring ship black smoke incidents. Background Technology

[0002] With increasing global emphasis on environmental protection, the regulation of ship black smoke emissions has become a crucial part of protecting the atmospheric environment. Traditional monitoring methods mainly rely on manual on-site inspections or fixed sensor networks, which have significant limitations in terms of coverage, real-time performance, and accuracy. Specifically, manual inspections not only consume a large amount of human resources but are also constrained by time and space, making it difficult to achieve comprehensive and continuous monitoring. While fixed sensors can provide accurate data at specific locations, their deployment costs are high, and they cannot flexibly adapt to changes in different sea areas and navigation routes.

[0003] Furthermore, existing video stream processing technologies typically employ a single-frequency detection strategy. This means either performing high-frequency, all-day, high-precision monitoring, leading to a significant waste of computing resources, or performing preliminary screening at a lower frequency, which can easily miss brief but severe black smoke events, resulting in false negatives. Meanwhile, in the face of complex and ever-changing marine environments, efficiently distinguishing genuine black smoke emissions from false positives caused by environmental factors (such as cloud shadows and sea fog) is also a problem that current technologies urgently need to solve.

[0004] Therefore, this application provides a method and system for monitoring ship black smoke incidents to solve one of the aforementioned technical problems. Summary of the Invention

[0005] The purpose of this application is to provide a method and system for monitoring ship black smoke incidents, which can solve at least one of the technical problems mentioned above. The specific solution is as follows:

[0006] According to a specific embodiment of this application, in a first aspect, this application provides a method for monitoring ship black smoke incidents, comprising:

[0007] The system acquires a ship video stream and performs routine, interval-by-interval identification of the video stream based on a black smoke recognition algorithm at a first time interval. Upon detecting a black smoke frame, it records the current first moment and performs encrypted, interval-by-interval identification of the video stream based on the black smoke recognition algorithm at a second time interval. Upon the period during which the encrypted, interval-by-interval identification no longer detects a black smoke frame, it records the current second moment. Based on the ship video stream from the first moment to the second moment, it determines whether to generate a black smoke event alarm and outputs the result.

[0008] In one embodiment, determining whether a black smoke event alarm is generated based on the ship's video stream from the first time point to the second time point includes: determining whether the time interval between the second time point and the first time point is greater than a fourth time interval; if yes, saving a first video slice from the first time point to the second time point, and determining whether a black smoke event alarm is generated based on the first video slice; if no, saving a second video slice from the first time point to the fourth time point, and determining whether a black smoke event alarm is generated based on the second video slice; wherein, the fourth time point is a time point following the first time point, and the fourth time point is separated from the first time point by the fourth time interval.

[0009] In one embodiment, determining whether to generate a black smoke event alarm based on the first video slice includes: calculating a first average Ringelmann blackness value of the first video slice, and determining whether the first average Ringelmann blackness value meets the alarm threshold for generating a black smoke event alarm; determining whether to generate a black smoke event alarm based on the second video slice includes: calculating a second average Ringelmann blackness value of the second video slice, and determining whether the second average Ringelmann blackness value meets the alarm threshold; wherein, the average Ringelmann blackness value is obtained by averaging the Ringelmann blackness values ​​of the black smoke frames in each frame of the video slice.

[0010] In one embodiment, the second time interval is set to be less than the first time interval; the third time interval is set to be greater than a preset multiple of the second time interval; and the fourth time interval is set to be greater than the third time interval.

[0011] In one embodiment, the first time interval is between 3 and 5 seconds; the second time interval is 0.5 seconds; the third time interval is 10 seconds; and the fourth time interval is 30 seconds.

[0012] In one embodiment, the method further includes: calculating the Ringelmann blackness value of the black smoke frame each time a black smoke frame is detected; and displaying the latest calculated Ringelmann blackness value in real time.

[0013] In one embodiment, the Ringelmann blackness value is calculated using the following formula: B =( I 0- I s ) / ( I 0- I 5); among which, B The Ringelmann blackness value. I s This represents the brightness value of the plume. I 0 represents the brightness value for a black level of 0.I 5 represents the brightness value for a black level of 5.

[0014] According to a specific embodiment of this application, in a second aspect, this application provides a ship black smoke event monitoring system, comprising:

[0015] The identification unit is used to acquire the ship video stream and perform interval-by-interval normal identification of the ship video stream based on the black smoke identification algorithm at a first time interval; the processing unit, in response to the identification of a black smoke frame, records the current first moment and performs interval-by-interval encrypted identification of the video stream based on the black smoke identification algorithm at a second time interval; in response to the continuous duration for which the interval-by-interval encrypted identification no longer identifies a black smoke frame satisfying a third time interval, records the current second moment; the judgment unit is used to determine whether to generate black smoke event alarm information based on the ship video stream from the first moment to the second moment, and output the result.

[0016] In one embodiment, the determining unit determines whether a black smoke event alarm message is generated based on the ship video stream from the first time point to the second time point in the following manner: determining whether the time interval between the second time point and the first time point is greater than a fourth time interval; if yes, then saving a first video slice from the first time point to the second time point, and determining whether a black smoke event alarm message is generated based on the first video slice; if no, then saving a second video slice from the first time point to the fourth time point, and determining whether a black smoke event alarm message is generated based on the second video slice; wherein, the fourth time point is a time point after the first time point, and the fourth time point is separated from the first time point by the fourth time interval.

[0017] In one embodiment, the first time interval is between 3 and 5 seconds; the second time interval is 0.5 seconds; the third time interval is 10 seconds; and the fourth time interval is 30 seconds.

[0018] Compared with the prior art, the above-described solution of this application has at least the following beneficial effects: This application provides a method for monitoring ship black smoke events. By regularly identifying and filtering video streams at intervals and reducing the consumption of computing resources, it automatically switches to encrypted identification mode when black smoke is detected to achieve intensive monitoring of suspected black smoke. This method not only improves the accuracy and timeliness of detection, but also avoids false alarms caused by short-term and intermittent black smoke, thereby enhancing the reliability and monitoring effect. Attached Figure Description

[0019] Figure 1 A flowchart of a method for monitoring ship black smoke incidents is shown;

[0020] Figure 2A flowchart illustrating a method for determining whether to generate a black smoke event alarm is shown.

[0021] Figure 3 A flowchart illustrating another method for determining whether to generate a black smoke event alarm is shown.

[0022] Figure 4 A complete flowchart for performing black smoke event monitoring is shown;

[0023] Figure 5 A block diagram of a ship black smoke event monitoring system is shown. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “said,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.

[0026] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0027] It should be understood that although the terms first, second, third, etc., may be used in the embodiments of this application for description, these descriptions should not be limited to these terms, which are only used to distinguish the descriptions. For example, without departing from the scope of the embodiments of this application, first may also be referred to as second, and similarly, second may also be referred to as first.

[0028] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0029] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a product or system comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a product or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the product or system that includes said element.

[0030] It should be noted that any symbols and / or numbers present in the specification that are not marked in the accompanying drawings are not reference numerals.

[0031] The optional embodiments of this application are described in detail below with reference to the accompanying drawings.

[0032] The embodiments provided in this application are embodiments of a method for monitoring ship black smoke incidents.

[0033] The following is combined Figure 1 The embodiments of this application will be described in detail.

[0034] Figure 1 A flowchart of a method for monitoring ship black smoke incidents is shown, such as... Figure 1 As shown, it includes the following steps.

[0035] Step S101: Obtain the ship video stream and perform interval-by-interval normal identification on the ship video stream based on the black smoke recognition algorithm at the first time interval.

[0036] In step S102, in response to the detection of the black smoke frame, the current first moment is recorded, and based on the black smoke recognition algorithm, the video stream is encrypted and recognized interval by interval at a second time interval.

[0037] Step S103: In response to the continuous duration of the interval-by-interval encryption identification no longer identifying the black smoke frame satisfying the third time interval, record the current second moment.

[0038] Step S104: Based on the ship video stream from the first moment to the second moment, determine whether to generate black smoke event alarm information and output the result.

[0039] Among them, encrypted identification is a high-frequency identification compared to normal identification, used to reduce the interval and improve the response.

[0040] Currently, traditional methods for monitoring ship black smoke typically require the continuous transmission of multiple video streams (e.g., 16 channels) to a central server for centralized processing. The massive amount of video data uploaded and analyzed places extremely high demands on the computing power of the central server, resulting not only in high consumption of computing resources and high operating costs, but also in significant delays during data transmission and processing, which affect the real-time performance of the monitoring.

[0041] To address this issue, this application proposes an optimized method for monitoring ship black smoke incidents. This method employs intelligent, phased detection at the front end: first, routine identification is performed at longer time intervals to reduce computational load under normal conditions; once suspected black smoke is detected, it automatically switches to high-frequency, encrypted identification to accurately capture the entire event process. This method effectively reduces unnecessary data processing and transmission, significantly lowers the system's dependence on central computing power, thereby greatly reducing costs while ensuring monitoring accuracy, improving response speed and real-time performance, and achieving efficient, economical, and low-latency black smoke incident monitoring.

[0042] Figure 2 A flowchart illustrating a method for determining whether to generate a black smoke event alarm is shown, such as... Figure 2 As shown, it includes the following steps.

[0043] In step S201, it is determined whether the time interval between the second time moment and the first time moment is greater than the fourth time interval.

[0044] In step S202a, if yes, the first video slice from the first moment to the second moment is saved, and it is determined whether to generate black smoke event alarm information based on the first video slice.

[0045] In step S202b, if no, the second video slice from the first moment to the fourth moment is saved, and it is determined whether to generate black smoke event alarm information based on the second video slice.

[0046] The fourth time point is the time point following the first time point, and the fourth time point is separated from the first time point by a time interval of four.

[0047] In some embodiments, the black smoke event alarm information can be determined based on the blackness value of the video slice.

[0048] As one feasible implementation, the average blackness value of the black smoke frames in each frame of the video slice can be used to determine the triggering of alarm information for black smoke events. Based on this, as a further feasible implementation, the blackness value can be calculated using the Ringelmann blackness calculation formula.

[0049] Figure 3 A flowchart illustrating another method for determining whether to generate a black smoke event alarm is shown, such as... Figure 3As shown, it includes the following steps.

[0050] In step S301, it is determined whether the time interval between the second time moment and the first time moment is greater than the fourth time interval.

[0051] In step S302a, if yes, the first video slice from the first moment to the second moment is saved, the first average Ringelmann blackness value of the first video slice is calculated, and it is determined whether the first average Ringelmann blackness value meets the alarm threshold for generating black smoke event alarm information.

[0052] In step S302b, if no, the second video slice from the first time to the fourth time is saved, the second average Ringelmann blackness value of the second video slice is calculated, and it is determined whether the second average Ringelmann blackness value meets the alarm threshold.

[0053] The average Ringelmann blackness value is obtained by averaging the Ringelmann blackness values ​​of the smoke frames in each frame of the video slice.

[0054] As one specific implementation method, the blackness rating is based on the Ringelmann blackness value, which is divided into levels 0 to 5. The blackness increases progressively from level 0 to level 5. For example, level 0 means blackness of 0%, and level 5 means blackness of 100%.

[0055] Based on this, the Ringelmann blackness value is calculated using the following formula:

[0056] B =( I 0- I s ) / ( I 0- I 5).

[0057] in, B The Ringelmann blackness value. I s This represents the brightness value of the plume. I 0 represents the brightness value for a black level of 0. I 5 represents the brightness value for a black level of 5.

[0058] Furthermore, when calculating the average Ringelmann blackness value based on the above embodiments, the Ringelmann blackness value of the black smoke frame in each video frame of the first video slice (or the second video slice) is first calculated using the Ringelmann blackness value calculation formula, and then the mean Ringelmann blackness value between video frames is calculated to obtain the average Ringelmann blackness value.

[0059] In this embodiment, by employing a sequential, routine identification method, the ship black smoke event monitoring method can efficiently perform preliminary screening of video streams, reducing unnecessary computational resource consumption. Once a black smoke frame is identified, the system automatically switches to encrypted identification mode. This response mechanism ensures intensive monitoring begins at the first moment of suspected black smoke detection, greatly improving the accuracy and timeliness of detection. After recording the first moment, if no black smoke is identified again within the time interval, the second moment is recorded, and the relevant video clips are saved based on the time interval. This method not only effectively captures the occurrence process of black smoke events but also avoids false alarms caused by brief, intermittent black smoke, improving the reliability of the system.

[0060] In some embodiments, the second time interval is set to be shorter than the first time interval. This is to ensure intensive monitoring at a higher frequency after a suspected black smoke event is detected. The longer time interval used in the normal identification phase is to reduce computational resource consumption, performing only preliminary screening. Once a black smoke frame is detected, the system immediately enters the encrypted identification phase, continuously tracking the presence of the black smoke at shorter time intervals. This design allows the system to maintain low power consumption under normal conditions and respond quickly and improve identification accuracy in the event of anomalies.

[0061] In some embodiments, the third time interval is set to a preset multiple greater than the second time interval, which is used to determine whether the black smoke has disappeared. When no black smoke frame is detected within multiple consecutive second time intervals during the encryption identification phase, it is considered that the current black smoke event may have ended, and this moment is recorded as the second time point. In this case, by setting the third time interval to the sum of multiple second time intervals, false termination due to brief obstruction or environmental interference can be effectively avoided, thereby improving the stability and robustness of the system.

[0062] In some embodiments, the fourth time interval is set to be greater than the third time interval. This setting is used to determine whether to ultimately save the video segment as valid evidence. If the time interval from the first moment (the starting point of black smoke detection) to the second moment (the point at which black smoke has not been detected since the last detection) exceeds the fourth time interval, it indicates that the black smoke event has a relatively long duration and high credibility, and should be recorded and alarmed. Conversely, if the duration is short, it may be occasional interference or false identification, and can be ignored. This design helps to further reduce the false alarm rate while ensuring that truly valid black smoke events are accurately captured.

[0063] In the above embodiments of this application, the hierarchical relationship of time intervals reflects the differentiated processing strategy for task objectives at different stages:

[0064] Routine identification phase (longest time interval): Focus on energy conservation and preliminary screening.

[0065] Encryption identification phase (short second time interval): Emphasis on high accuracy and real-time tracking.

[0066] Confirm the end of the black smoke phase (the third time interval is a multiple of the second time interval): to prevent misjudgment and termination.

[0067] The alarm determination stage (the fourth time interval is the longest): ensures the continuity of the event and filters out invalid information.

[0068] As one specific embodiment, the time intervals are configured as follows.

[0069] The first time interval ranges from 3 to 5 seconds.

[0070] The second time interval is 0.5 seconds.

[0071] The third time interval is 10 seconds.

[0072] The fourth time interval is 30 seconds.

[0073] Based on the specific values ​​mentioned above, the entire process is as follows: a detection is performed every 5 seconds. After black smoke is detected, the detection is switched to every 0.5 seconds. If no black smoke is detected for 10 consecutive seconds, the current time is recorded as the second moment. If the time difference between the first moment and the second moment is greater than 30 seconds, the video segment is saved and an alarm is generated; otherwise, the event is ignored.

[0074] Figure 4 A complete flowchart for performing black smoke event monitoring is shown.

[0075] like Figure 4 As shown, the system first initiates the normal video stream recognition phase. During this phase, the system performs preliminary detection on the video stream at a time interval of t0 per frame. This low-frequency detection aims to reduce computational resource consumption while simultaneously identifying potential black smoke events. If a black smoke frame is detected during the normal recognition process, the current time is immediately recorded as T1, and the system proceeds to the encrypted recognition phase.

[0076] As a specific implementation, during the encrypted identification phase, the system increases the detection frequency to an interval of t1 per frame, i.e., a high-frequency encrypted identification mode. The purpose of this phase is to more accurately capture black smoke characteristics and avoid missed detections. During the encrypted identification process, the system continuously calculates and displays the real-time Ringelmann blackness value so that monitoring personnel can understand the changes in smoke concentration in real time. If a black smoke frame is detected again within the t2 interval, encrypted identification continues; otherwise, the current time is recorded as T2, and the system proceeds to the next time determination step.

[0077] For example, in the time determination stage, the system will determine whether the time difference between T2 and T1 is greater than a preset threshold t3. If T2 - T1 > t3, it indicates that the black smoke event has a relatively long duration and has high alarm value. In this case, the system will save the video slice from T1 to T2. Conversely, if T2 - T1 ≤ t3, it indicates that the black smoke event has a relatively short duration and may be a brief interference or false detection. In this case, the system will save the video slice from T1 to T1 + t3 to ensure data integrity.

[0078] In some embodiments, after saving the video slice, the system further calculates the average Ringelmann blackness value of the saved video slice. This calculation process is obtained by averaging the Ringelmann blackness values ​​of all frames within the slice, aiming to quantify the black smoke concentration and provide a scientific basis for subsequent alarm determination. For example, if the calculated average blackness value exceeds a preset alarm threshold, the system will generate a black smoke event alarm message and output the result to notify relevant personnel for further processing.

[0079] In summary, this embodiment achieves efficient monitoring and accurate recording of black smoke events by dynamically adjusting the detection frequency, time interval determination, and quantifying the blackness value. The process logic is clear, resource consumption is low, and it is suitable for real-time monitoring needs in complex environments, effectively improving the monitoring efficiency and accuracy of black smoke events.

[0080] In this embodiment of the application, based on the above-mentioned time interval setting, the time management of the entire monitoring process is both flexible and reasonable. It can quickly respond to potential black smoke events and avoid excessive redundant operations that lead to resource waste. This design enables the system to provide reliable monitoring services while ensuring efficiency.

[0081] In some embodiments, each time a black smoke frame is detected, the Ringelmann blackness value of the black smoke frame is calculated and the latest calculated Ringelmann blackness value is displayed in real time, thereby providing a real-time display function of the Ringelmann blackness value in practical application scenarios.

[0082] As one specific implementation method, the blackness rating is based on the Ringelmann blackness value, which is divided into levels 0 to 5. The blackness increases progressively from level 0 to level 5. For example, level 0 means blackness of 0%, and level 5 means blackness of 100%.

[0083] Based on this, the Ringelmann blackness value is calculated using the following formula:

[0084] B =( I 0- I s ) / ( I 0- I 5).

[0085] in, B The Ringelmann blackness value. I s This represents the brightness value of the plume. I 0 represents the brightness value for a black level of 0. I 5 represents the brightness value for a black level of 5.

[0086] In some embodiments, the black smoke identification algorithm employs the YOLOv5s-CMBI network.

[0087] In some embodiments, the labeled dataset of the YOLOv5s-CMBI network is created in the following manner.

[0088] In some embodiments, the creation of the labeled dataset for the ship black smoke event monitoring method includes the following steps: First, a specified number of first labeled data are obtained by labeling the training dataset. Based on the first labeled data, the YOLOv5s-CMBI network is initially trained, and the pre-trained network is used to automatically label unlabeled data, generating second labeled data. Finally, the first and second labeled data are fine-tuned and merged to form a complete labeled dataset. As a specific embodiment, the efficient labeling mode of X-AnyLabeling can be adopted. First, a portion of the images are labeled and an initial model is trained. Then, this model is loaded into X-AnyLabeling for automatic labeling, and the labeling results are further optimized through fine-tuning, thereby significantly saving labeling time and cost.

[0089] In some embodiments, the training data is image data, such as images of ship smoke. The processing of the training dataset includes: selecting a bounding box around the smoke area in each image and removing the chimney portion; setting the label to "smock". Ensuring the bounding box position in each image remains within the specified area (e.g., the top left corner). Adjusting the image's width and height to multiples of 16 pixels. Generating YOLO format annotation files for the annotated images, where the .txt file corresponds to the image name, and each line records information for an annotation instance, including five columns: label category, the ratio of the bounding box center point coordinates to the image size, and the proportion of the bounding box width and height to the image size.

[0090] As a feasible implementation, data augmentation of the training set is necessary to improve the model's generalization ability. Common data augmentation methods include random cropping, rotation, scaling, brightness adjustment, and noise addition. For images in low-light scenes, histogram equalization, Gamma transformation, or the Retinex algorithm are used for enhancement to restore color information and improve contrast under low-light conditions. Among these, adaptive Gamma transformation can dynamically adjust parameters based on the image's average brightness, compressing pixels with excessively high or low gray levels to a normal range, thus adapting to the needs of black smoke feature extraction under different lighting conditions.

[0091] In some embodiments, this application considers setting the loss function as follows. Specifically, in YOLOv5, by using GIOU_Loss instead of the traditional Smooth L1 Loss function, the accuracy of ship black smoke detection is significantly improved. This loss function not only measures the overlap area between the predicted box and the ground truth box based on IoU, but also effectively handles the distance calculation problem when the two do not overlap by adding a penalty term, avoiding the error of traditional methods in separated scenes. However, GIOU_Loss degenerates into IoU_Loss when the predicted box and the ground truth box partially overlap or contain each other, and cannot accurately reflect the positional differences. To this end, this application subsequently optimized and introduced CIoU_Loss, which further integrates the comprehensive consideration of overlap area, center point distance, and aspect ratio. However, when the aspect ratio of the predicted box and the ground truth box is linearly related, its aspect ratio penalty term will fail, resulting in a decrease in regression accuracy. To address this limitation, this application replaces CIoU_Loss in YOLOv5s with EIoU_Loss. By splitting the aspect ratio influence factor into independent width and height difference calculations, the regression of width and height is optimized separately, thereby avoiding the ambiguity of the proportional relationship and significantly improving the accuracy of bounding box regression and model convergence speed. This provides more reliable loss function support for the accurate detection of non-rigid targets such as black smoke.

[0092] In some embodiments, negative samples can be added to the training dataset to address misidentification. Negative samples include data from non-black smoke scenes such as images of steel wire ropes and cloud shadows. Specifically, the generation method includes: selecting non-black smoke background images from the misidentified test set, performing Gaussian filtering, contrast enhancement, and sharpening on the selected images to generate a negative sample count equal to the positive sample count. The positive and negative samples are then divided into training, validation, and test sets in an 8:1:1 ratio. For example, black smoke images from scenes such as smoke, clear skies, and cloudy skies can be labeled, and the labeled images and YOLO format files can be uniformly organized into a training directory structure.

[0093] As a specific implementation, the YOLOv5s-CMBI network is an optimization based on the YOLOv5x network. The backbone network adds a Concurrent Attention (CBAM) module, which highlights key features and suppresses redundant information through channel attention and spatial attention mechanisms. The neck network replaces the original feature concatenation module with a Bidirectional Feature Pyramid Network (BiFPN_Concat), combined with an Adaptive Spatial Feature Fusion (ASFF) module to achieve efficient fusion of multi-scale features. The prediction module introduces multiple detection heads to adapt to the detection of black smoke targets of different sizes and shapes. Through these optimizations, the model can more accurately locate and classify black smoke regions in complex scenes.

[0094] In some embodiments, the YOLOv5s-CMBI network structure includes a backbone network, a neck network, a prediction module, a feature fusion module, and an output module. The backbone network extracts low- to mid-level visual features through a Spatial Pyramid Feature Enhancement (SPFE) module and Basic Convolutional Blocks (CBS). The neck network achieves bottom-up and top-down multi-scale feature fusion through BiFPN_Concat. The prediction module generates bounding boxes and class probabilities based on the fused features. The feature fusion module enhances the model's robustness to target scale variations through lateral stitching and vertical fusion. The output module covers the detection needs of targets of different sizes through multi-scale outputs (Pn out, Pn+1 out, Pn+2 out).

[0095] As a feasible implementation, the training parameters are set as follows: initial learning rate of 0.01, batch size of 8, total training epochs of 1000, image size of 640x640 pixels, and early stopping mechanism of 100 epochs. To alleviate overfitting, a 3-round warm-up learning rate strategy is adopted, and cosine annealing is used to adjust the learning rate after the warm-up. The remaining parameters follow the default YOLOv5 configuration.

[0096] As a specific implementation, in object detection tasks, the quality of a model is mainly evaluated from three aspects: detection accuracy, detection speed, and model complexity. Detection accuracy typically includes metrics such as precision, recall, mAP@0.5, and mAP@0.5:0.95. Detection speed focuses on frames per second (FPS). Model complexity considers model size, number of parameters, and computational cost (GFLOPs). For the binary classification problem discussed in this paper, which distinguishes between cases with and without black smoke detection boxes, the detection results can be categorized into four types: true positives (TP), false negatives (FN), false positives (FP), and true negatives (TN). For example, TP indicates that an actual black smoke box was correctly detected, while FN indicates that an actual black smoke box was incorrectly identified as having no black smoke box.

[0097] For example, precision reflects how many of the identified positive examples are actually positive, i.e., accuracy. Recall shows how many actual positive examples in the original sample are accurately predicted, i.e., recall. The overall performance of the model is measured by plotting PR curves at different thresholds and calculating the area under the curves as the average precision (AP). Generally, the average mAP values ​​when the Intersection over Union (IoU) is set to 0.5 or with a step size of 0.05 from 0.5 to 0.95 are denoted as mAP@0.5 and mAP@0.5:0.95, respectively. These are the main accuracy evaluation parameters of this method. mAP@0.5 refers to the average precision calculated when the IoU threshold is set to 0.5 in object detection. mAP@0.5 is a commonly used metric for calculating object detection accuracy; it represents the average precision across all categories when the IoU threshold is 0.5. A higher mAP value indicates higher detection accuracy. `mAP@0.5:meanAveragePrecision(IoU=0.5)` calculates the AP for all images in each class when IoU is set to 0.5, and then averages the AP across all classes, resulting in `mAP`. Here, AP50, AP60, AP70, etc., refer to IoU thresholds greater than 0.5, 0.6, 0.7, etc. Higher values ​​(larger thresholds) result in lower precision.

[0098] In this embodiment of the application, mAP and AP are calculated using the following formula:

[0099] ; ;

[0100] Where i represents the category, c represents the number of categories, and P(r) represents the precision at recall r.

[0101] In some specific implementations, the YOLOv5s-CMBI network significantly improves the accuracy and efficiency of black smoke detection through multi-scale feature fusion and attention mechanisms. When creating the dataset, the process of initial annotation, automatic annotation, and fine-tuning merging reduces the cost of manual annotation while ensuring the quality and diversity of the dataset. The standardized design of the YOLO format annotation files ensures accurate identification of the black smoke range in each image, providing high-quality input for model training. Data augmentation strategies for low-light images, combined with histogram equalization and Gamma transformation, further enhance the model's adaptability in low-light environments. By introducing negative samples and optimizing the network structure, the false alarm rate is effectively reduced, improving the overall robustness of the system.

[0102] In this embodiment, normal identification and encrypted identification are two core steps, which correspond to different video stream processing strategies at different stages. The following analysis is conducted from the perspectives of definition, implementation method and technical principle.

[0103] In some embodiments, routine identification refers to performing low-frequency, preliminary frame-by-frame detection on the ship's video stream at relatively long time intervals when no black smoke is detected. The aim is to reduce computational resource consumption while filtering out potential black smoke events.

[0104] For example, the specific implementation of normal recognition is as follows: Black smoke detection is performed on images in the video stream at a set first time interval (e.g., 3 to 5 seconds). This process relies on the YOLOv5s-CMBI network, which enhances feature extraction capabilities through optimizations including adding an attention mechanism module and using a bidirectional feature pyramid network, thereby more accurately identifying black smoke. When a black smoke frame is detected during normal recognition, it automatically switches to encrypted recognition mode. Encrypted recognition, after detecting black smoke, performs high-frequency, intensive frame-by-frame detection on the video stream at shorter time intervals (e.g., 0.5 seconds). Its purpose is to accurately capture the occurrence process of black smoke events and generate alarm information. When encrypted recognition starts, the current moment is recorded as the first moment, and the video stream is subjected to interval-by-interval encrypted recognition based on the same black smoke recognition algorithm at a second time interval (e.g., 0.5 seconds). If no black smoke frame is detected within a third time interval (e.g., 10 seconds), the current moment is recorded as the second moment, and the time interval between the first and second moments is used to determine whether to save the relevant video segment. The collaborative mechanism between normal recognition and encrypted recognition lies in a phased processing strategy. In the normal identification phase, low-frequency detection covers the entire video stream to initially screen for black smoke events. Once suspected black smoke is detected, the system immediately switches to encrypted identification mode for more frequent and detailed monitoring of suspected black smoke events. This dynamic switching mechanism not only improves detection efficiency but also effectively reduces the false alarm rate. Furthermore, by setting a time interval after which no black smoke is detected consecutively as a termination condition, flexibility and adaptability are ensured.

[0105] Compared to this application, traditional target recognition algorithms are mostly designed for rigid targets, characterized by clear outlines. However, black smoke, as a non-rigid target, has blurred boundaries and is easily mixed with other backgrounds, making recognition difficult. To address this, this application proposes a targeted solution: firstly, various background images that are easily misidentified are included as negative samples in the training set to effectively suppress interference; secondly, the CMBI network (an improved structure based on an attention mechanism) is selected from 16 network models optimized specifically for non-rigid targets and deeply optimized based on YOLOv5s. YOLOv5s is chosen because its lightweight nature reduces computational resource consumption, but the original version lacks accuracy. This application improves upon this by introducing the CBAM attention mechanism into the backbone network and adopting the BiFPN_Concat structure in the neck network, enabling the optimized YOLOv5s to balance lightweight design with high accuracy, significantly improving the accuracy and robustness of black smoke recognition.

[0106] This application also provides system embodiments that follow the above embodiments, for implementing the method steps of the above embodiments. The interpretation of the same names is the same as that of the above embodiments, and they have the same technical effects as those of the above embodiments, so they will not be repeated here.

[0107] like Figure 5 As shown, this application provides a ship black smoke event monitoring system, including:

[0108] The identification unit 501 is used to acquire the ship's video stream and perform routine, interval-by-interval identification of the video stream based on a black smoke identification algorithm at a first time interval. The processing unit 502, in response to the identification of a black smoke frame, records the current first moment and, based on the black smoke identification algorithm, performs encrypted, interval-by-interval identification of the video stream at a second time interval. In response to the continuous duration for which the encrypted, interval-by-interval identification no longer detects a black smoke frame satisfying a third time interval, records the current second moment. The judgment unit 503 is used to determine, based on the ship's video stream from the first moment to the second moment, whether to generate a black smoke event alarm message and output the result.

[0109] In one embodiment, the judgment unit 503 determines whether a black smoke event alarm message should be generated based on the ship's video stream from a first moment to a second moment in the following manner: It determines whether the time interval between the second moment and the first moment is greater than a fourth time interval. If yes, it saves a first video slice from the first moment to the second moment and determines whether a black smoke event alarm message should be generated based on the first video slice. If no, it saves a second video slice from the first moment to the fourth moment and determines whether a black smoke event alarm message should be generated based on the second video slice. Here, the fourth moment is the moment following the first moment, and the fourth moment is separated from the first moment by a fourth time interval.

[0110] In one embodiment, the first time interval is between 3 and 5 seconds. The second time interval is 0.5 seconds. The third time interval is 10 seconds. The fourth time interval is 30 seconds.

[0111] Regarding the system in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0112] Although the operations are described in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the operations shown to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0113] The methods and systems of this application can be implemented using standard programming techniques, utilizing rule-based logic or other logic to implement various method steps. It should also be noted that the terms "system" and "module" as used herein and in the claims are intended to include implementations using one or more lines of software code and / or hardware implementations and / or devices for receiving input.

[0114] Any step, operation, or procedure described herein may be performed or implemented using one or more hardware or software modules, either alone or in combination with other devices. In one embodiment, the software module is implemented using a computer program product comprising a computer-readable medium containing computer program code, which is executable by a computer processor to perform any or all of the described steps, operations, or procedures.

[0115] The foregoing description of implementations of this application has been provided for illustrative and descriptive purposes. The foregoing description is not exhaustive and is not intended to limit this application to the exact forms disclosed. Various modifications and variations may exist in accordance with the foregoing teachings, or may arise from practice of this application. These embodiments were chosen and described to illustrate the principles of this application and its practical application, enabling those skilled in the art to utilize this application in various implementations and modifications to suit the specific purpose of the concept.

[0116] Regarding the system in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0117] It can be further understood that, unless otherwise specified, "connection" includes both direct connections where no other components exist between the two parties and indirect connections where other components exist between them.

[0118] It is further understood that although the operations are described in a specific order in the accompanying drawings in the embodiments of this application, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all the operations shown to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0119] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the field of this application that are not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0120] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

[0121] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for monitoring ship black smoke incidents, characterized in that, Includes the following steps: The ship video stream is acquired, and based on the black smoke recognition algorithm, the ship video stream is subjected to interval-by-interval normal recognition at a first time interval; In response to the detection of the black smoke frame, the current first moment is recorded, and based on the black smoke recognition algorithm, the video stream is encrypted and recognized at intervals in a second time interval; In response to the continuous duration during which the interval-by-interval encrypted identification no longer detects the black smoke frame, satisfying the third time interval, the current second moment is recorded; Based on the ship video stream from the first moment to the second moment, determine whether to generate a black smoke event alarm message and output the result; The step of determining whether to generate a black smoke event alarm based on the ship video stream from the first time to the second time includes: Determine whether the time interval between the second time point and the first time point is greater than the fourth time interval; If so, the first video slice from the first moment to the second moment is saved, and a black smoke event alarm message is generated based on the first video slice. If not, save the second video slice from the first moment to the fourth moment, and determine whether to generate black smoke event alarm information based on the second video slice; Wherein, the fourth time point is the time point after the first time point, and the fourth time point is separated from the first time point by the fourth time interval.

2. The method according to claim 1, characterized in that, The step of determining whether to generate a black smoke event alarm based on the first video slice includes: Calculate the first average Ringelmann blackness value of the first video slice, and determine whether the first average Ringelmann blackness value meets the alarm threshold for generating black smoke event alarm information. The step of determining whether to generate a black smoke event alarm based on the second video slice includes: Calculate the second average Ringelmann blackness value of the second video slice, and determine whether the second average Ringelmann blackness value meets the alarm threshold; The average Ringelmann blackness value is obtained by averaging the Ringelmann blackness values ​​of the smoke frames in each frame of the video slice.

3. The method according to claim 1 or 2, characterized in that, The second time interval is set to be less than the first time interval; The third time interval is set to a preset multiple greater than the second time interval; The fourth time interval is set to be greater than the third time interval.

4. The method according to claim 3, characterized in that, The first time interval ranges from 3 seconds to 5 seconds; The second time interval is 0.5 seconds; The value of the third time interval is 10 seconds; The fourth time interval is 30 seconds.

5. The method according to claim 1, characterized in that, The method further includes: Each time a black smoke frame is detected, the Ringelmann blackness value of the black smoke frame is calculated. It displays the latest calculated Ringelmann blackness value in real time.

6. The method according to claim 2 or 5, characterized in that, The Ringelmann blackness value is calculated using the following formula: B =( I 0- I s ) / ( I 0- I 5)； in, B The Ringelmann blackness value. I s This represents the brightness value of the plume. I 0 represents the brightness value for a black level of 0. I 5 represents the brightness value for a black level of 5.

7. A ship black smoke incident monitoring system, characterized in that, Includes the following steps: The identification unit is used to acquire the ship video stream and perform interval-by-interval normal identification of the ship video stream based on the black smoke identification algorithm at a first time interval; The processing unit, in response to the detection of a black smoke frame, records the current first moment and performs interval-by-interval encrypted identification of the video stream based on the black smoke identification algorithm; in response to the continuous duration during which the interval-by-interval encrypted identification no longer detects a black smoke frame, records the current second moment. The judgment unit is used to determine whether to generate a black smoke event alarm information based on the ship video stream from the first time to the second time, and output the result; The judgment unit determines whether to generate a black smoke event alarm information based on the ship video stream from the first moment to the second moment in the following manner: Determine whether the time interval between the second time point and the first time point is greater than the fourth time interval; If so, the first video slice from the first moment to the second moment is saved, and a black smoke event alarm message is generated based on the first video slice. If not, save the second video slice from the first moment to the fourth moment, and determine whether to generate black smoke event alarm information based on the second video slice; Wherein, the fourth time point is the time point after the first time point, and the fourth time point is separated from the first time point by the fourth time interval.

8. The system according to claim 7, characterized in that, The first time interval ranges from 3 seconds to 5 seconds; The second time interval is 0.5 seconds; The value of the third time interval is 10 seconds; The fourth time interval is 30 seconds.

Images