Complex scene-oriented ship black smoke detection method and system
By constructing a dataset of ship black smoke in complex scenarios, adaptive preprocessing, and improving the YOLOv5s model, the problem of ship black smoke detection in complex scenarios was solved, and efficient identification and accurate detection were achieved under adverse weather conditions.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI JIUHENG TECHNOLOGY CO LTD
- Filing Date
- 2026-01-19
- Publication Date
- 2026-04-24
Smart Images
Figure CN121921745A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of ship black smoke identification technology, and in particular relates to a ship black smoke detection method and system for complex scenarios. Background Technology
[0002] Against the backdrop of economic globalization, shipping business has surged, leading to increased emissions of sulfur dioxide (SO2), hydrocarbons (HC), and nitrogen oxides during ship operations. Gases containing pollutants such as particulate matter (PM) form black smoke when their concentration reaches a critical point. Black smoke from ships not only accelerates engine wear and shortens the lifespan of diesel engines, but also seriously affects air quality at sea, posing a threat to the health of coastal residents. According to relevant data, ship emissions... Ship smoke accounts for 32.5% of total non-road mobile source emissions, with international shipping industry carbon emissions exceeding 1 billion tons, accounting for 2-3% of global total carbon emissions; ship smoke has become one of the important sources of air pollution.
[0003] Traditional methods for detecting black smoke from ships mainly rely on the Ringelmann blackness method and shore-based sniffing method. The Ringelmann blackness method is greatly affected by weather and is highly subjective; the shore-based sniffing method has a limited monitoring range, is affected by wind speed and direction, and the detection results are delayed, making it unable to meet the detection needs of ports or estuaries with heavy traffic.
[0004] With the development of deep learning technology, it has demonstrated advantages in the field of object detection, providing new ideas for ship black smoke detection. Existing deep learning-based black smoke detection methods are mostly designed for industrial or vehicle black smoke. Industrial black smoke combustion patterns are relatively stable, and the detection scenarios for vehicle black smoke differ significantly from those for ship black smoke. Ship black smoke is affected by factors such as wind speed and direction, exhibiting dynamic diffusion and instability. Furthermore, in adverse weather conditions (such as rain, snow, and fog), complex environmental interference can easily lead to feature extraction biases. Existing ship black smoke detection algorithms perform poorly in adverse weather conditions, making it difficult to meet the needs of accurate identification and effective monitoring of ship black smoke in practical monitoring scenarios. Summary of the Invention
[0005] To address the aforementioned technical problems, this invention proposes a method and system for detecting ship black smoke in complex scenarios, thereby resolving the issues present in the prior art.
[0006] To achieve the above objectives, the present invention provides a method for detecting ship black smoke in complex scenarios, comprising:
[0007] Collect video data of ship black smoke under different conditions, extract image frames from the ship black smoke video data, annotate the black smoke targets in the image frames, and obtain a ship black smoke annotation dataset;
[0008] The ship black smoke annotation dataset is augmented to obtain an augmented dataset;
[0009] Based on the YOLOv5s model, the detection head of the YOLOv5s model is replaced with a dynamic detection head, and a cross-dimensional feature fusion module is added to the Neck network of the YOLOv5s model to obtain a ship black smoke detection model.
[0010] The ship black smoke detection model is trained and evaluated based on the augmented dataset, and the trained ship black smoke detection model is used to achieve ship black smoke detection in complex scenarios.
[0011] Optionally, the process of augmenting the ship black smoke annotation dataset to obtain an augmented dataset includes:
[0012] Each image in the ship black smoke annotation dataset is divided into multiple non-overlapping sub-regions; a gray-level histogram is calculated for each sub-region; a histogram clipping threshold for each sub-region is calculated based on the clipping factor and the maximum allowable slope; the number of pixels in each sub-region that exceed the clipping threshold is clipped, and the clipped pixels are evenly distributed to gray-level regions below the clipping threshold within the corresponding sub-region; all processed sub-regions are fused using bilinear interpolation to obtain the enhanced dataset.
[0013] Optionally, the detection head includes a scale-aware attention module, a spatial-aware attention module, and a task-aware attention module.
[0014] Optionally, the processing procedure of the scale-aware attention module includes:
[0015] For each feature layer of the input feature tensor, a global average pooling operation is performed in both the spatial and channel dimensions to obtain the pooled feature representation. The pooled feature representation is then processed by the Sigmoid activation function to generate a weight vector corresponding to each feature layer. The generated weight vector is then multiplied element-wise with the input feature tensor in the feature layer dimension to obtain the weighted feature tensor.
[0016] Optionally, the processing procedure of the spatial awareness attention module includes:
[0017] The weighted feature tensor is sparsely sampled, the sampling point position is adjusted according to the spatial sampling position offset of self-learning, the feature value of the adjusted position is extracted, and the weighted sum is combined with the importance weight and position weight of self-learning. The processed feature tensor is then output to the task perception attention module.
[0018] Optionally, the processing procedure of the task-aware attention module includes:
[0019] The feature tensor processed by the spatial awareness attention module is sliced along the channel dimension to obtain feature slices. Each feature slice is then linearly transformed using weight and bias parameters. Finally, a maximum value function is applied to select the output from the two sets of transformation results to generate the final feature tensor as the output of the dynamic detection head.
[0020] Optionally, the cross-dimensional feature fusion module receives feature maps of different scales output by the Backbone network of the ship black smoke detection model; adjusts the number of channels of the feature maps through a 1×1 convolutional layer; samples the feature maps to make their sizes consistent; and stitches and fuses the feature maps with consistent sizes.
[0021] Optionally, it also includes replacing the loss function of the YOLOv5s model with a loss function based on the minimum point distance intersection-union ratio.
[0022] This invention also provides a ship black smoke detection system for complex scenarios, used to implement the above method, including:
[0023] The dataset construction module is used to collect video data of ship black smoke under different conditions, extract image frames from the video data of ship black smoke, and annotate the black smoke targets in the image frames to obtain a ship black smoke annotation dataset.
[0024] The preprocessing module is used to enhance the ship black smoke annotation dataset to obtain an enhanced dataset;
[0025] The model building module is used to replace the detection head of the YOLOv5s model with a dynamic detection head and add a cross-dimensional feature fusion module to the Neck network of the YOLOv5s model to obtain a ship black smoke detection model.
[0026] The model training and detection module is used to train and evaluate the ship black smoke detection model based on the augmented dataset, and to realize ship black smoke detection for complex scenarios based on the trained ship black smoke detection model.
[0027] Compared with the prior art, the present invention has the following advantages and technical effects:
[0028] This invention constructs a dataset of ship black smoke containing different resolutions and weather conditions, and uses adaptive histogram equalization to preprocess the images, which effectively enhances the contrast of black smoke images under severe weather conditions, reduces the impact of environmental interference on feature extraction, and provides high-quality data support for model training.
[0029] This invention replaces the original detection head of the YOLOv5s model with a dynamic detection head and utilizes scale-aware attention, spatial-aware attention, and task-aware attention to achieve dynamic weighting and key area focusing of black smoke features from ships of different scales. This improves the model's robustness to differences in ship size and smoke concentration and reduces background noise interference.
[0030] This invention adds a cross-dimensional feature fusion module to the Neck network, which enhances the spatial-channel feature interaction, effectively fuses feature information at different scales, captures the irregular contours of black smoke during the diffusion process, and improves the model's ability to identify multi-scale black smoke targets.
[0031] This invention employs a loss function based on minimum point distance intersection-union ratio (CIU). By minimizing the vertex distance between the predicted bounding box and the true bounding box, it solves the problem of inaccurate bounding box regression in small target black smoke detection using traditional loss functions, and significantly improves the model's ability to detect small target black smoke. Attached Figure Description
[0032] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:
[0033] Figure 1 The images shown are the original image (left) and the image after adaptive histogram equalization (right) according to an embodiment of the present invention.
[0034] Figure 2 This is a diagram of the Dynamic Head detection head architecture according to an embodiment of the present invention;
[0035] Figure 3 This is a diagram of the DyHead module architecture according to an embodiment of the present invention;
[0036] Figure 4 This is a model structure diagram of an embodiment of the present invention;
[0037] Figure 5 This is a comparison chart of mAP@0.5 before and after model optimization in an embodiment of the present invention;
[0038] Figure 6 This is a comparison chart of the accuracy before and after model optimization in an embodiment of the present invention.
[0039] Figure 7 This is a comparison chart of recall rates before and after model optimization in an embodiment of the present invention;
[0040] Figure 8 This is a comparison chart of various performance indicators before and after model optimization in an embodiment of the present invention;
[0041] Figure 9 This is a comparison chart of mAP@0.5 before and after loss function optimization in an embodiment of the present invention;
[0042] Figure 10 This is a comparison chart of recall before and after loss function optimization in an embodiment of the present invention;
[0043] Figure 11 This is a comparison chart of precision before and after loss function optimization in an embodiment of the present invention.
[0044] Figure 12 The images show the F1 curves of the loss function before (left) and after (right) optimization in this embodiment of the invention.
[0045] Figure 13 The images show the PR curves of the loss function before (left) and after (right) optimization in this embodiment of the invention.
[0046] Figure 14 The images shown are: a YOLOv5 recognition image (left), a recognition image with Dyhead+CCFM added (middle), and a recognition image with MPDIoU added (right) according to an embodiment of the present invention.
[0047] Figure 15 The images shown are: a YOLOv5 recognition image (left), a recognition image with Dyhead+CCFM added (middle), and a recognition image with MPDIoU added (right) according to an embodiment of the present invention. Detailed Implementation
[0048] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.
[0049] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0050] Example 1
[0051] This embodiment provides a method for detecting black smoke from ships in complex scenarios, including:
[0052] (a) Dataset image preprocessing;
[0053] (1) Dataset construction;
[0054] Two 10-minute video clips (3840*2160 resolution), one 1-minute video (2160*3840 resolution), one 6-minute video (2560*1440 resolution), one 12-minute video (1080*1920 resolution), and one 1-minute video (1280*720 resolution) were selected as the original dataset sources. FreeStudio software was used to extract images from every 10 frames, and irrelevant images were removed, resulting in a total of 8580 images. The two 3840*2160 video clips are black smoke videos from ships in severe weather, while the remaining clips are black smoke videos from ships in clear weather.
[0055] The Labelimg tool was used to annotate black smoke in the dataset images, collecting the target's location information and storing it as a txt file. The file was divided into a training set, validation set, and test set in a 6:2:2 ratio, resulting in 5148 training images, 1716 validation images, and 1716 test images. The txt file contains five parameters in the YOLO annotation format: category ID (label), x-center coordinate, y-center coordinate, width, and height. The category ID is a 0-based category index. The x-center coordinate is the ratio of the object's bounding box to the image width, the y-center coordinate is the ratio of the object's bounding box to the image height, and the x and y center coordinates represent the target's location information in the image.
[0056] (2) Dataset preprocessing;
[0057] Since the dataset contains images of black smoke under severe weather conditions, the dataset is preprocessed first, and image enhancement is used to improve the contrast of the black smoke under severe weather conditions.
[0058] For images that are generally dark (i.e., under severe weather conditions), the Contrast Limited Adaptive Histogram Equalization (CLAHE) method is used to enhance the image.
[0059] CLAHE divides an image into multiple regions and performs histogram equalization on each region to enhance contrast. To avoid blockiness caused by mapping differences between adjacent regions, adaptive histogram equalization calculates the mapping for each pixel by performing bilinear interpolation on the mappings of neighboring regions.
[0060] The principle of histogram equalization is as follows:
[0061] Let r and s be the normalized grayscale values of the original image and the grayscale values of the image after histogram transformation, respectively. When r=s=0, it represents black; when r=s=1, it represents white. r and s should be in the range of 0-1.
[0062] For any r, after transformation function Each of these can produce an s, and s = Among them, the function Satisfy: (1) In The inner part is a monotonically increasing function; (2) in Inside Therefore, there are .
[0063] From probability theory, the probability density function of the random variable r is given by... Since s is a function of r, the probability density function of s can be expressed by the probability density function of r. Let the distribution function of s be... Then we have:
[0064] (1)
[0065] Differentiating both sides with respect to s, we get:
[0066] (2)
[0067] It can be seen that, through the transformation function It can control the probability density function of image gray levels.
[0068] Because of the normalization assumption: ;
[0069] Then from equation (2), we have:
[0070] (3)
[0071] From equation (3), we can obtain that when the transformation function Using the cumulative distribution function of the original image histogram, histogram equalization can be achieved.
[0072] In images, gray levels are mostly discrete grayscale images. Using frequency instead of probability, we can obtain:
[0073] (4)
[0074] Therefore, it can be seen that the gray values of each pixel after equalization It can be calculated from the histogram of the original image.
[0075] The biggest problem with histogram equalization is that regional contrast is increased to its maximum. Adaptive histogram equalization limits the contrast to the desired level by cropping the histogram using a threshold β, where β is:
[0076] (5)
[0077] Where M is the number of pixels in the region, and N is the number of gray levels in the region. The maximum permissible slope is given by α, where α is the shearing factor. When α = 0, the shearing threshold equals M / N, meaning that pixels in all regions are uniformly distributed across all possible gray levels, forming an identity mapping, and the pixel values remain unchanged. When α = 100, the shearing threshold equals... The product of M and N represents the maximum permissible slope. .
[0078] The original histogram is modified according to the desired contrast variation limit of the image. The maximum limit for each gray level is set to β. Gray levels exceeding this threshold are evenly distributed to other regions below the threshold. Through multiple iterations, the shearing factor α is modified to achieve an adaptive histogram equalization method.
[0079] Adaptive histogram equalization enhances local contrast in an image by processing small patches; simultaneously, limiting contrast prevents noise from being excessively amplified during processing, thus maintaining image quality without sacrificing detail. The image after adaptive histogram equalization is shown below. Figure 1 As shown.
[0080] (ii) Algorithm model optimization;
[0081] (1) Optimization of the YOLO network model;
[0082] Because black smoke appears as low grayscale values in optical image features, and its overall shape is irregular with large differences in grayscale values within the image, its image features are greatly affected by external environmental factors such as wind speed and direction. As time goes by, the shape of black smoke will continue to change and eventually dissipate. At the same time, fog at sea will also have a significant impact on the detection of black smoke. Therefore, black smoke detection algorithms need to fully consider the multi-scale features of black smoke targets, interference from the external environment, and real-time requirements.
[0083] However, for the detection of ship black smoke, the smoke is often mixed with sea fog or deck steam, and in severe weather, it is greatly affected by background interference, making the fixed feature extraction mechanism of YOLOv5 susceptible to interference. Secondly, black smoke exhibits local diffusion characteristics in its initial stage, making it difficult for YOLOv5's default detection layer to effectively capture it. When the smoke is partially obscured by ship structures (such as funnels and masts), YOLOv5's bounding box regression mechanism is prone to failure. Therefore, this invention specifically improves the algorithm.
[0084] (1) Optimization of the detection head in the YOLO model;
[0085] The dynamic detection head of this invention is a Dynamic Head (DyHead module). It achieves uniformity in target detection by applying different attention mechanisms to different dimensions of target features. DynamicHead treats the image input as a three-dimensional tensor with hierarchy, space, and channels, and applies attention mechanisms to each of the three channels separately, forming a task-awareness module, a spatial-awareness module, and a scale-awareness module, such as... Figure 2 As shown. Figure 2 In the image, (a) represents the feature maps of the input image at various sizes. Figure 2 In (b), a single-layer feature map is divided into three parts by the Dynamic Head: layer, spatial, and channel. Figure 2 (c) in the diagram represents the size-aware attention mechanism. Figure 2 In the example, (d) represents spatial awareness attention. Figure 2 (e) in the text represents task-aware attention.
[0086] Figure 2 In the DyHead module, the image F is divided into four tensors, namely... ,
[0087] Where L is the layer number, W and H are the width and height of the image, respectively, and C represents the number of channels for the layer features. Therefore, the image is reshaped into The scale-aware attention focuses on channel L, which is the input to the dynamic head. Multiple DyHead modules, including scale-aware, spatial-aware, and task-aware attention, are stacked together as spatial-aware attention, focusing on channel S. Finally, task-aware attention is output, focusing on channel C. The DyHead modules are as follows: Figure 3 As shown, it consists of three parts.
[0088] For scale-aware attention, the formula is:
[0089] (6)
[0090] Where F is the input feature tensor with shape L*S*C. This is the activation function.
[0091] From formula (6), we can see that, Average pooling is performed on the space and channels of each feature layer to obtain the global feature representation of each feature layer, thus completing the importance assessment of the feature layers. The activation function SIGMOID reflects the relative importance of each feature layer in the current input. This involves multiplying the weight matrix generated by the SIGMOID function with each feature tensor to obtain a weighted tensor, ensuring that features at different scales are appropriately weighted. When dealing with black smoke images of different scales, smaller-scale black smoke is assigned higher weights, while larger-scale black smoke is assigned lower weights, creating a dynamic adjustment effect for feature weights. Using average pooling can better capture global information and better handle targets at different scales.
[0092] For spatial awareness attention, the formula is:
[0093] (7)
[0094] in, This represents the number of sparse sampling locations. Sampling location, This is the spatial sampling position offset for self-learning. Weighting the importance of self-learning Let k be the weight of the image at sampling position k in layer l.
[0095] From formula (7), we can see that, It integrates features from different feature layers. The sampling position is dynamically adjusted. Indicated in the feature layer ,Location The passage is located at [location]. eigenvalues. and Multiplication represents the multiplication of feature layers. and sampling location The eigenvalues are weighted, and and Multiplication indicates that the sampling locations are weighted. It is to evaluate different feature layers With position The relative importance of these factors allows the model to better integrate multi-scale information, while This involves evaluating each location. The importance of this allows for better focus on the key areas of the target. Therefore, when detecting target smoke, the model can better focus on the smoke representation of the target, ignore other background interference information, and further consider the multi-scale features of the smoke.
[0096] For the task-aware attention module, the formula is as follows:
[0097] (8)
[0098] in, This represents a feature slice of the feature tensor F on channel C. These are the weight parameters and the bias parameters, respectively.
[0099] The weight parameter is used to amplify or reduce the feature value of the current channel. A larger weight means that the channel is more helpful for feature extraction.
[0100] The bias parameter adjusts the baseline of the activation values, enabling the model to better capture the features of the target.
[0101] Task-aware attention uses the max function to maximize the weighted function, ensuring that the model selects the most relevant feature channels for activation, thereby improving the model's detection accuracy for features such as texture and edges of black smoke.
[0102] (2) Add a cross-dimensional feature fusion module (abbreviated as: CCFM module);
[0103] The CCFM module is a key component of the Real-Time Detection Transformer (RT-DETRS) module. It can fuse features at different scales to enhance the model's ability to identify targets at multiple scales.
[0104] Figure 4 The diagram shows the model structure, with the red dashed line representing the CCFM module. As shown, the CCFM module receives feature maps at different scales, extracts features, and then passes them through a 1x1 convolutional layer to reduce the number of channels. Upsampling and downsampling are then performed to achieve feature concatenation and fusion, effectively combining detailed features with contextual information to improve the model's predictive ability across multiple scales. This invention adds a CCFM module to the YOLO model, enhancing its ability to detect black smoke at different scales, especially for small targets. Simultaneously, it expands the model's receptive field and enhances its semantic understanding capabilities.
[0105] (3) Loss function optimization;
[0106] The black smoke emitted by actual ships is irregular in shape and constantly changes over time. To adapt to this characteristic, the model needs to enhance its multi-scale detection capabilities. YOLOv5 uses CIoU as the loss function, improving target detection accuracy by considering center point distance and aspect ratio consistency. However, the center point distance and aspect ratio have little impact on small targets, and CIoU performs poorly when detecting irregular targets. Therefore, the loss function needs to be optimized to enhance its multi-scale target detection capabilities and further improve the accuracy of small target detection. Through comparative studies, this invention selects Minimum Point Distance Intersection over Union (MPDIoU) as the loss function for the project.
[0107] MPDIoU's design comprehensively considers geometric features such as overlap rate, center point distance, and width-to-height deviation. It improves the accuracy and efficiency of bounding box regression by minimizing the distances between the top-left and bottom-right corners of the predicted bounding box and the ground truth bounding box. The following is the calculation process for MPDIoU:
[0108] Let the width and height of the input image be W and H, respectively. Predict the detection box A and the ground truth bounding box B. The coordinates of the top-left corner of A are (W, H). The coordinates of the lower right corner are The coordinates of the top left corner of B are... The coordinates of the lower right corner are .
[0109] Then predict the distance from the top-left corner of bounding box A to the top-left corner of the true bounding box B. for:
[0110] ;
[0111] Similarly, predict the distance from the bottom right corner of bounding box A to the bottom right corner of the actual bounding box B. for:
[0112] ;
[0113] Then MPDIoU:
[0114] ;
[0115] The loss formula for MPDIoU is:
[0116] ;
[0117] Therefore, all factors of the loss function can be represented using the four coordinates mentioned above, where:
[0118] ;
[0119] Therefore, MPDIoU can also be represented as:
[0120] ;
[0121] During training, this invention aims to minimize the loss function, i.e.:
[0122]
[0123] When IOU=0, meaning the predicted bounding box and the ground truth bounding box have no intersection, the minimization training performed by MPDIoU is actually minimizing the distance between the vertices of the predicted bounding box and the ground truth bounding box. This makes MPDIoU extremely sensitive to small positional errors and suitable for small object detection.
[0124] Furthermore, the MPDIoU loss function directly optimizes the vertex distance between the predicted bounding box and the ground truth bounding box, an optimization method independent of the target's specific scale. Therefore, whether the target is large or small, MPDIoU can effectively guide the model to learn more accurate bounding box locations. This also allows MPDIoU to perform multi-scale processing, better addressing multi-scale detection problems.
[0125] This embodiment also provides a ship black smoke detection system for complex scenarios, used to implement the above method, including:
[0126] The dataset construction module is used to collect video data of ship black smoke under different conditions, extract image frames from the video data of ship black smoke, and annotate the black smoke targets in the image frames to obtain a ship black smoke annotation dataset.
[0127] The preprocessing module is used to enhance the ship black smoke annotation dataset to obtain an enhanced dataset;
[0128] The model building module is used to replace the detection head of the YOLOv5s model with a dynamic detection head and add a cross-dimensional feature fusion module to the Neck network of the YOLOv5s model to obtain a ship black smoke detection model.
[0129] The model training and detection module is used to train and evaluate the ship black smoke detection model based on the augmented dataset, and to realize ship black smoke detection for complex scenarios based on the trained ship black smoke detection model.
[0130] (III) Results Analysis;
[0131] (1) Comparison of test set results;
[0132] (a) Comparison of the model before and after optimization;
[0133] This invention demonstrates the comparison between models by plotting the results of different models on a test set onto the same image. Comparison images are created using result.csv and result.png files obtained after running various models. Figures 5-8 In this series, "Dyhead" represents a YOLO model with the detection head replaced by a Dynamic Head, "CCFM" represents a model with the CCFM module added, and "MPDIoU" represents a model with the loss function changed to MPDIoU. All models will be displayed in a "+" format to indicate the changes made to the model. Figures 5-8 The left image in the series compares various metrics before and after model optimization, while the right image compares the effects after magnification with epochs between 35 and 130 and the vertical axis between 0.7 and 1.0.
[0134] Figures 5-8 In this context, mAP@0.5 represents the average precision at IoU=0.5. The larger the mAP@0.5, the stronger the model's predictive ability. mAP@0.5:0.95 (mAP@[0.5:0.95]) represents the mAP value at different IoU thresholds (from 0.5 to 0.95, with a step size of 0.05).
[0135] Precision is the proportion of positive samples that a model correctly predicts. A model is considered to have made a correct prediction when the predicted bounding box matches the true bounding box.
[0136] Recall is an evaluation of the proportion of true positive samples that a model can find. When the true bounding box coincides with the predicted bounding box, the sample is considered to have been correctly recalled.
[0137] Depend on Figure 5 , Figure 8 It can be seen that after replacing the Dynamic Head with a detection head and adding the CCFM module, the model's mAP@0.5 and mAP@0.5:0.95 values increased by an average of about 0.03. Figure 6 It can be seen that the accuracy rate increases by an average of approximately 0.01. From... Figure 7 It can be seen that the recall rate increased by an average of about 0.015. Overall, the various indicators show that replacing the Dynamic Head detector and adding the CCFM module further improved the model's predictive ability.
[0138] (b) Comparison chart before and after the change of loss function;
[0139] The comparison chart of model performance after changing the loss function to MPDIoU is provided. Figures 9-13 As shown. Figures 9-13 The left-middle chart compares various indicators, while the right-middle chart is an enlarged image with the vertical axis ranging from 0.7 to 1.0 and the epoch ranging from 35 to 130.
[0140] The “F1-confidence” reflects the relationship between the confidence threshold and the F1 score. The setting of the confidence threshold affects the precision and recall of the detection results, while the F1 score is the harmonic mean of precision and recall, which is a comprehensive measure of classification.
[0141] The "PR_curve" curve represents the relationship between precision and recall at different thresholds. In the PR curve, the horizontal axis represents recall, and the vertical axis represents precision. The PR curve illustrates the trade-off between recall and precision.
[0142] Depend on Figure 12 As can be seen from the F1 curve, before changing the loss function, the threshold for the model to reach an F1 value of 0.87 was 0.507, while after changing the loss function, the model can reach a higher F1 value of 0.90 at a higher threshold of 0.756.
[0143] Depend on Figure 9 It can be seen that after changing the loss function to MPDIoU, the average values of mAP@0.5 and mAP@0.5:0.95 of the model increased by approximately 0.02. From... Figure 10 It can be seen that the accuracy rate increases by an average of approximately 0.002. From... Figure 11 It can be seen that the recall rate increased by an average of about 0.03.
[0144] Depend on Figure 13 The PR curves show that the average precision before changing the loss function was 0.894, while the average precision after using MPDIoU was 0.931.
[0145] Table 1 compares the various metrics of the model in the test set results. "Ours" represents the algorithm proposed in this invention. It can be clearly seen that the algorithm proposed in this invention has good improvements in precision, recall, F1 score and mAP 0.5.
[0146] Table 1
[0147] Accuracy Recall rate mAP0.5 F1 YOLOv5s 0.855 0.882 0.876 0.868 YOLOv5s + Dyhead + CCFM 0.864 0.896 0.894 0.879 Ours 0.866 0.922 0.931 0.893
[0148] In summary, the algorithm proposed in this invention demonstrates superior black smoke detection capabilities on the test set.
[0149] (2) Comparison of target detection results;
[0150] To demonstrate that the model can identify black smoke under different weather conditions, a picture with poor weather conditions and a picture with clear weather were selected for a side-by-side comparison.
[0151] Figure 14 and Figure 15 The detection results of three models, YOLOv5s, YOLOv5s+Dynamichead+CCFM, and YOLOv5s+Dynamichead+CCFM+MPDIoU, are shown on these two images.
[0152] Depend on Figure 14 As can be seen, all three models can detect black smoke images under clear weather conditions. The confidence score of YOLOv5s is 0.7, while the model with the DynamicHead detector and CCFM module shows an improved confidence score of 0.75. The model with the MPDIoU loss function significantly improves its confidence score to 0.89. In conclusion, the YOLO model with the DynamicHead detector, CCFM module, and MPDIoU has good recognition capabilities for black smoke in clear weather.
[0153] like Figure 15 As shown, even in poor weather conditions, all three models can detect black smoke. The confidence score of YOLOv5s is 0.7, while the model with the DynamicHead detection head and CCFM module shows a slight improvement to 0.75. The model with the loss function changed to MPDIoU significantly improves its confidence score to 0.83. In conclusion, the YOLO model with the DynamicHead detection head, CCFM module, and MPDIoU can effectively detect black smoke in adverse weather conditions.
[0154] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for detecting black smoke from ships in complex scenarios, characterized in that, Includes the following steps: Collect video data of ship black smoke under different conditions, extract image frames from the ship black smoke video data, annotate the black smoke targets in the image frames, and obtain a ship black smoke annotation dataset; The ship black smoke annotation dataset is augmented to obtain an augmented dataset; Based on the YOLOv5s model, the detection head of the YOLOv5s model is replaced with a dynamic detection head, and a cross-dimensional feature fusion module is added to the Neck network of the YOLOv5s model to obtain a ship black smoke detection model. The ship black smoke detection model is trained and evaluated based on the augmented dataset, and the trained ship black smoke detection model is used to achieve ship black smoke detection in complex scenarios.
2. The method for detecting ship black smoke in complex scenarios according to claim 1, characterized in that, The process of augmenting the labeled ship black smoke dataset to obtain the augmented dataset includes: Each image in the ship black smoke annotation dataset is divided into multiple non-overlapping sub-regions; a gray-level histogram is calculated for each sub-region; a histogram clipping threshold for each sub-region is calculated based on the clipping factor and the maximum allowable slope; the number of pixels in each sub-region that exceed the clipping threshold is clipped, and the clipped pixels are evenly distributed to gray-level regions below the clipping threshold within the corresponding sub-region; all processed sub-regions are fused using bilinear interpolation to obtain the enhanced dataset.
3. The method for detecting ship black smoke in complex scenarios according to claim 1, characterized in that, The detection head includes a scale-aware attention module, a spatial-aware attention module, and a task-aware attention module.
4. The method for detecting ship black smoke in complex scenarios according to claim 3, characterized in that, The processing steps of the scale-aware attention module include: For each feature layer of the input feature tensor, a global average pooling operation is performed in both the spatial and channel dimensions to obtain the pooled feature representation. The pooled feature representation is then processed by the Sigmoid activation function to generate a weight vector corresponding to each feature layer. The generated weight vector is then multiplied element-wise with the input feature tensor in the feature layer dimension to obtain the weighted feature tensor.
5. The method for detecting ship black smoke in complex scenarios according to claim 4, characterized in that, The processing steps of the spatial awareness attention module include: The weighted feature tensor is sparsely sampled, the sampling point position is adjusted according to the spatial sampling position offset of self-learning, the feature value of the adjusted position is extracted, and the weighted sum is combined with the importance weight and position weight of self-learning. The processed feature tensor is then output to the task perception attention module.
6. The method for detecting ship black smoke in complex scenarios according to claim 5, characterized in that, The processing procedure of the task-aware attention module includes: The feature tensor processed by the spatial awareness attention module is sliced along the channel dimension to obtain feature slices. Each feature slice is then linearly transformed using weight and bias parameters. Finally, a maximum value function is applied to select the output from the two sets of transformation results to generate the final feature tensor as the output of the dynamic detection head.
7. The method for detecting ship black smoke in complex scenarios according to claim 1, characterized in that, The cross-dimensional feature fusion module receives feature maps of different scales output by the Backbone network of the ship black smoke detection model; adjusts the number of channels of the feature maps through a 1×1 convolutional layer; samples the feature maps to make their sizes consistent; and splices and fuses the feature maps with consistent sizes.
8. The method for detecting ship black smoke in complex scenarios according to claim 1, characterized in that, It also includes replacing the loss function of the YOLOv5s model with a loss function based on the minimum point distance intersection-union ratio.
9. A ship black smoke detection system for complex scenarios, used to implement the method described in any one of claims 1-8, characterized in that, include: The dataset construction module is used to collect video data of ship black smoke under different conditions, extract image frames from the video data of ship black smoke, and annotate the black smoke targets in the image frames to obtain a ship black smoke annotation dataset. The preprocessing module is used to enhance the ship black smoke annotation dataset to obtain an enhanced dataset; The model building module is used to replace the detection head of the YOLOv5s model with a dynamic detection head and add a cross-dimensional feature fusion module to the Neck network of the YOLOv5s model to obtain a ship black smoke detection model. The model training and detection module is used to train and evaluate the ship black smoke detection model based on the augmented dataset, and to realize ship black smoke detection for complex scenarios based on the trained ship black smoke detection model.