An automatic identification algorithm for ship black smoke emission events based on video.

Abstract

This invention discloses an automatic identification algorithm for ship black smoke incidents based on video. The steps are as follows: S1, video is collected from a designated water area, and black smoke and ship identification are performed on each frame of the video to obtain a set of marked images with black smoke identification boxes and ship identification boxes; S2, it is determined whether there is a ship black smoke incident in the video; S3, based on the start and end times of the black smoke incident of ship i that has a violation determined in step S2, the violation video of ship i is extracted from the video collected in step S1. This automatic identification algorithm is not affected by the instability of ship exhaust emissions and the defects of black smoke identification algorithms, and can effectively eliminate some erroneous black smoke identification results. It has the advantages of simplicity, efficiency and high accuracy.

Description

Technical Field

[0001] This invention relates to the field of ship exhaust gas monitoring technology, and in particular to an automatic identification algorithm for ship black smoke emission events based on video. Background Technology

[0002] Black smoke from ships is a severe form of air pollution visible to the naked eye. Currently, maritime law enforcement officers primarily determine the opacity of ship exhaust by visual observation and comparison with Ringelmann blackness diagrams, which is inefficient. With the development of artificial intelligence technology and the widespread use of video surveillance systems, using AI to identify ships or black smoke in footage is technically feasible. However, from a maritime regulatory perspective, brief black smoke emissions caused by temporary acceleration or deceleration are not considered violations in practice; only continuous black smoke emissions while the ship is sailing at a constant speed constitute a violation. Therefore, a ship black smoke capture system must not only identify each frame of the video to accurately determine whether black smoke or other similar phenomena are present, but also be able to determine whether multiple frames of black smoke emissions represent a single event or two or more independent black smoke emissions from different ships.

[0003] Because ship exhaust is a non-rigid target, characterized by its volatile shape and varying degrees of blackness, the misidentification and missed identification rates for ship exhaust fumes in images are significantly higher and harder to avoid in current image recognition systems compared to identifying ships in images. This poses a considerable challenge to the automatic identification of subsequent ship exhaust black smoke incidents. Therefore, the key to developing ship black smoke capture and identification systems is how to accurately identify ship exhaust black smoke incidents despite the high misidentification and missed identification rates. Summary of the Invention

[0004] The purpose of this invention is to provide a video-based automatic identification algorithm for ship black smoke emissions that solves the above-mentioned technical problems.

[0005] Therefore, the technical solution of the present invention is as follows:

[0006] An automatic identification algorithm for ship black smoke emission events based on video, comprising the following steps:

[0007] S1. Video is captured in the designated water area, and black smoke and ships are identified in each frame of the video, resulting in a set of marked images with black smoke and ship identification boxes.

[0008] S2. Identify whether there is a ship emitting black smoke in the video. The specific steps are as follows:

[0009] S201. Calculate the center distance d between each black smoke identification box and each ship identification box in each marked image.i,t The calculation formula is as follows:

[0010] d i,t =ps j,t -pv i,t ,

[0011] In the formula, t is the acquisition time of the marked image; ps j,t ρ is the pixel x-coordinate of the center point of the black smoke identification box; j is the index of the black smoke identification box in the marked image, pv i,t is the pixel x-coordinate of the center point of the ship identification box; i is the ship number, different ships have different numbers, and the same ship has the same number in different frame images;

[0012] S202, the distance d between all center points of the same vessel i i,t Find the average value D i ;

[0013] S203. Calculate the distance d between the center points of the same ship i. i,t The corresponding absolute value of distance deviation b i,t The calculation formula is as follows:

[0014] b i,t =∣d i,t -D i |;

[0015] S204. Based on the set maximum distance deviation upper limit value b max Calculate the absolute value b of the distance deviation occurring in the same ship i. i,t Less than the maximum distance deviation upper limit b max number of times n i ;

[0016] S205. Based on the set minimum number of times n can be detected for ship black smoke... min and with n i Comparison: when n i ≥n min If ship i has committed a violation by emitting black smoke, then ship i has committed a violation by emitting black smoke once; when n i <n min If there is no violation of regulations regarding black smoke emitted by vessel i;

[0017] S3. The absolute value b of all distance deviations corresponding to vessel i, which was identified in step S2 as having one violation involving black smoke emission. i,t In the process, select those that satisfy the condition of being less than the upper limit of the maximum distance deviation b. max The absolute value of multiple distance deviations b i,t And find the minimum value t of the marked image acquisition time t. min and maximum value t maxIn step S1, the video is captured using t min For video start time, t max Extract the violation video of vessel i based on the video end time.

[0018] Furthermore, in step S1, the video image acquisition frequency is 2 frames / second to 5 frames / second.

[0019] Furthermore, in step S1, the identification and marking of black smoke in the image is completed by a black smoke identification algorithm, and the identification and marking of ships in the image is completed by a ship identification algorithm.

[0020] Furthermore, in step 4) of step S2, the upper limit value of the maximum distance deviation is b. max Set to 50-100.

[0021] Furthermore, in step 5) of step S2, the minimum number of times n is used to identify ship black smoke is determined. min Set to 10 to 100 times.

[0022] This application's video-based automatic identification algorithm for ship exhaust black smoke incidents can not only compensate for the high misidentification and omission rates in current ship exhaust image recognition, but also utilize the fact that ships are rigid targets and the motion characteristics of ship exhaust relative to the ship to achieve accurate identification of ship exhaust black smoke, and based on this, complete the judgment of illegal black smoke incidents from ships.

[0023] Compared with existing technologies, the beneficial effects of this video-based automatic identification algorithm for ship black smoke incidents include: 1) This method is not affected by the instability of ship exhaust emissions, and multiple indirect black smoke emissions from the same ship can be accurately identified as a single complete ship black smoke incident; 2) This method is not affected by defects in black smoke identification algorithms, and even if some shapes and colors of black smoke in the video are not accurately identified, it does not affect the final identification result; 3) This method can effectively eliminate some erroneous black smoke identification results (such as cloud shadows on cloudy days, smoke emitted from distant fixed pollution sources, some sunsets at dusk, and the projection of tall buildings on the ship's hull). In summary, this method has the advantages of simplicity, efficiency, and high accuracy, and has a good application and promotion prospect for maritime regulatory departments in the automatic identification of ship black smoke incidents based on video. Attached Figure Description

[0024] Figure 1 The flowchart shows the video-based automatic identification algorithm for ship black smoke emissions according to the present invention.

[0025] Figure 2 This is a schematic diagram of the marked image obtained in step S1 of the video-based automatic identification algorithm for ship black smoke events of the present invention;

[0026] Figure 3 This is a schematic diagram of step S1 of the video-based automatic identification algorithm for ship black smoke incidents of the present invention, in which a marked image is obtained based on an existing black smoke identification algorithm, showing a building projected as black smoke on the hull.

[0027] Figure 4 This is a schematic diagram of step S1 of the video-based automatic identification algorithm for ship black smoke incidents of the present invention, in which a marked image with a cloud base shadow is obtained based on an existing black smoke identification algorithm. Detailed Implementation

[0028] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the following embodiments are by no means intended to limit the present invention.

[0029] Example 1

[0030] See Figure 1 This embodiment applies the method of this application to images captured on January 9, 2024, by a camera installed at a wharf in the Shanghai waters of the Yangtze River estuary. The specific implementation steps are as follows:

[0031] S1. A camera positioned at a high point on the dock captures video of the designated waterway at an image acquisition frequency of 2 frames per second. Each frame of the video is then used for black smoke identification and vessel identification, resulting in a set of marked images with black smoke and vessel identification boxes. (See [link to documentation]). Figure 2 .

[0032] The identification and marking of ships in the images are accomplished using existing black smoke recognition algorithms. Specifically, ships in the images are identified and marked with their smallest bounding rectangles, which serve as the ship identification boxes. Similarly, the identification and marking of black smoke in the images are accomplished using existing black smoke recognition algorithms. Specifically, black smoke portions in the images are identified and marked with their smallest bounding rectangles, which serve as the black smoke identification boxes. However, because dark clouds and black reflections of objects in this type of image are similar to images of ship exhaust smoke, misidentification is prone to occur. Furthermore, when the color difference between the black smoke and the sky at the time of image acquisition is small, ship exhaust smoke is easily missed.

[0033] In practical applications, black smoke identification and ship identification are performed independently based on two different image recognition algorithms, and are marked separately in the images. Since the images are captured by the same camera and have the same resolution, a pixel coordinate system is constructed for the images, and the number of columns and rows of pixel units in the image is used to represent the coordinates of points in the image, so as to obtain the x-coordinate ps of the center point of the black smoke identification box in each image. j,tand the x-coordinate of the center point of the ship identification frame (pv) i,t ;

[0034] S2. Identify whether there is a ship emitting black smoke in the video. The specific steps are as follows:

[0035] In the video captured by the camera on January 9, 2024, 39 frames of images of the same ship were automatically identified between 7:06:16 and 7:36.5. Among them, due to the misidentification and omission problems of the existing black smoke identification algorithm, the ship identification box was identified 39 times in the 39 frames, but only the black smoke identification box was identified 31 times.

[0036] Table 1 below shows the center pixel coordinates of the black smoke and ship identification boxes identified at various acquisition times for 39 frames of images. In Table 1, ps j,t The blank column data means that the black smoke may not have been identified in the image acquired at that time due to a lack of identification, and therefore there is no center point coordinate data of the black smoke identification box.

[0037] Table 1:

[0038]

[0039] S201. Based on the formula: di,t=psj,t-pvi,t, calculate the center distance d between each black smoke identification box and the different ship identification boxes in each marked image. i,t In the formula, t is the acquisition time of the marked image; ps j,t ρ is the pixel x-coordinate of the center point of the black smoke identification box; j is the index of the black smoke identification box in the marked image, pv i,t is the pixel x-coordinate of the center point of the ship identification frame; i is the ship number;

[0040] S202, the total center-to-center distance d of the vessel i i,t Find the average value D i ;

[0041] S203, According to the formula: b i,t =∣d i,t -D i | Calculate the distance d between the center points of ship i. i,t The corresponding absolute value of distance deviation b i,t ;

[0042] In this embodiment, the center distance d between each black smoke identification frame and the different ship identification frames in step S201 i,t The specific calculation results are shown in Table 1; based on the center spacing d obtained in step S201 i,t The average value D was calculated. i =-45.4; the distance d between each center point in step S202i,t The corresponding absolute value of distance deviation b i,t The specific calculation results are shown in Table 1. From Table 1, we can see that the absolute value of the distance deviation b i,t The minimum value is 0.4, and the maximum value is 40.6;

[0043] S204, Set the maximum distance deviation upper limit value b max Given a value of 50, count the absolute value b of the distance deviation occurring in ship i. i,t Less than the maximum distance deviation upper limit b max number of times n i For 30 times;

[0044] S205. Set the minimum number of times n is needed to identify black smoke from ships. min For 10 times, since n i >n min It was determined that vessel i had committed a violation involving the emission of black smoke.

[0045] S3, the absolute value of all distance deviations corresponding to vessel i, b i,t In the process, select those that satisfy the condition of being less than the upper limit of the maximum distance deviation b. max The absolute value of multiple distance deviations b i,t And find the minimum value t of the marked image acquisition time t. min and maximum value t max The video footage from the 17th second to the 36.5th second is collected in step S1 and then saved as the violation video of vessel i.

[0046] To further verify the validity of this application, video recordings from 7:00 to 7:10 were retrieved from the video collected in step S1. The video recordings clearly show the entry and exit of vessel i from 0:16 to 36.5 seconds, and black smoke exhaust emanating from the vessel is clearly visible during its entry and exit, consistent with the results automatically identified in this embodiment. Furthermore, although no black smoke was detected in the images at 16 seconds, 16.5 seconds, 18 seconds, 24.5 seconds, 25 seconds, 25.5 seconds, 28 seconds, and 28.5 seconds during the data processing in this embodiment, this does not affect the accurate determination of the fact that vessel i committed a violation by emitting black smoke. Additionally, see... Figure 3 The video image processing contained instances where the building was mistakenly projected onto the ship's hull and marked with black smoke boxes. However, this did not affect the accuracy of the application data results, thus proving that the method of this application can indeed accurately and automatically identify illegal black smoke emissions from ships even when there are defects in the identification of black smoke from ship exhaust.

[0047] Example 2

[0048] See Figure 1 This embodiment applies the method of this application to images captured on January 22, 2024, by a camera installed at a wharf in the Shanghai waters of the Yangtze River estuary. The specific implementation steps are as follows:

[0049] S1. Using the same image acquisition method as in the embodiment, video is acquired from the designated water area, and black smoke and ship identification are performed on each frame of the video to obtain a set of marked images with black smoke marking black smoke identification boxes and ship marking ship identification boxes in the images;

[0050] S2. Identify whether there is a ship emitting black smoke in the video. The specific steps are as follows:

[0051] In the video captured by the camera on January 22, 2024, 28 frames of images of the same ship were automatically identified between 16:09:10 and 16:25. Similarly, due to the problem of dropped frames, the ship identification frame was identified 28 times in the 28 frames, but only the black smoke identification frame was identified 22 times.

[0052] Table 2 below shows the center pixel coordinates of the black smoke identification box and the ship identification box identified at each acquisition time in 28 frames of images, as shown in Table 2; similarly, ps j,t The presence of blank data indicates that no black smoke was detected in the image acquired at that moment.

[0053] Table 2:

[0054] S201. Calculate the center distance d between each black smoke identification box and each ship identification box in each marked image. i,t For detailed calculation results, please refer to Table 2.

[0055] S202, the total center-to-center distance d of the vessel i i,t Find the average value D i D i = -277.9;

[0056] S203. Calculate the distance d between the center points of ship i. i,t The corresponding absolute value of distance deviation b i,t The specific calculation results are shown in Table 2; as can be seen from Table 2, the absolute value of the distance deviation b i,t The minimum value is 1.9, and the maximum value is 42.1;

[0057] S204, Set the maximum distance deviation upper limit value b max Given a value of 50, count the absolute value b of the distance deviation occurring in ship i. i,t Less than the maximum distance deviation upper limit bmax number of times n i It was 22 times;

[0058] S205. Set the minimum number of times n is needed to identify black smoke from ships. min For 10 times, since n i >n min It was determined that vessel i had committed a violation involving the emission of black smoke.

[0059] S3, the absolute value of all distance deviations corresponding to vessel i, b i,t In the process, select those that satisfy the condition of being less than the upper limit of the maximum distance deviation b. max The absolute value of multiple distance deviations b i,t And find the minimum value t of the marked image acquisition time t. min and maximum value t max The video footage from the 10th second to the 25th second is collected in step S1 and then saved as the violation video of vessel i.

[0060] To further verify the validity of this application, the video recording from 16:05 to 16:15 was retrieved from the video collected in step S1. The video recording shows the entry and exit of vessel i from 9:10 to 9:25. It also clearly shows vessel i emitting noticeable black smoke exhaust from 10 to 25 seconds, consistent with the results automatically identified in this embodiment. Therefore, although no black smoke was detected in the image from 14.5 to 17 seconds during data processing, this does not affect the accurate determination of a violation involving vessel i emitting black smoke using the method of this application. Furthermore, the distance deviation b obtained according to step S203... i,t The minimum and maximum values ​​in the data show that although the initial data has a large discrepancy, this method can filter out the effective data; thus, it proves that the method of this application can indeed accurately and automatically identify illegal black smoke emission from ships even when there are deficiencies in the identification of black smoke from ship exhaust.

[0061] Example 3

[0062] See Figure 1 This embodiment applies the method of this application to images captured on January 22, 2024, by a camera installed at a dock in the Huangpu River in Shanghai. The specific implementation steps are as follows:

[0063] S1. Using the same image acquisition method as in the embodiment, video is acquired from the designated water area, and black smoke and ship identification are performed on each frame of the video to obtain a set of marked images with black smoke marking black smoke identification boxes and ship marking ship identification boxes in the images;

[0064] S2. Identify whether there is a ship emitting black smoke in the video. The specific steps are as follows:

[0065] In the video captured by the camera on January 22, 2024, 27 frames of images of the same ship were automatically identified between 14:25:00 and 14:14.5. Similarly, due to the problem of dropped frames, the ship identification frame was identified 27 times in the 27 frames, but only 11 black smoke identification frames were identified.

[0066] Table 3 below shows the center pixel coordinates of the black smoke identification box and the ship identification box identified at each acquisition time in 27 frames of images, as shown in Table 3 below; similarly, ps j,t The presence of blank data indicates that no black smoke was detected in the image acquired at that moment.

[0067] Table 3:

[0068]

[0069] S201. Calculate the center distance d between each black smoke identification box and each ship identification box in each marked image. i,t For detailed calculation results, please refer to Table 3.

[0070] S202, the total center-to-center distance d of the vessel i i,t Find the average value D i D i = -544.9;

[0071] S203. Calculate the distance d between the center points of ship i. i,t The corresponding absolute value of distance deviation b i,t The specific calculation results are shown in Table 3; as can be seen from Table 3, the absolute value of the distance deviation b i,t The minimum value is 110.1, and the maximum value is 745.1;

[0072] S204, Set the maximum distance deviation upper limit value b max Given a value of 50, count the absolute value b of the distance deviation occurring in ship i. i,t Less than the maximum distance deviation upper limit b max number of times n i It was 0 times;

[0073] S205. Set the minimum number of times n is needed to identify black smoke from ships. min For 10 times, since n i <n min The system determines that there is no violation of regulations regarding the emission of black smoke from vessel i.

[0074] S3. Since there are no violations involving black smoke emitted by vessel i, no violation videos need to be saved.

[0075] To further verify the validity of this application, the video recording from 14:15 to 14:35 was retrieved from the video collected in step S1. The video recording shows the entry and exit process of vessel i, and as... Figure 4 As shown, in this embodiment, when identifying black smoke in the image, dark clouds were consistently mistakenly identified as black smoke, hence the pv data in Table 3 was incorrect. i,t The value increases continuously from 119 to 1836, while the position of the black smoke detection frame remains basically unchanged, i.e., p st The value fluctuated between 115 and 133, therefore, the vessel i did not have any illegal black smoke emission problem; this proves that the method of this application can indeed accurately eliminate interference data when there are defects in the identification of black smoke from ship exhaust, and realize the automatic identification of illegal black smoke emission behavior of ships.

Claims

1. An automatic identification algorithm for ship black smoke emission events based on video, characterized in that, Here are the steps: S1. Video is captured in the designated water area, and black smoke and ships are identified in each frame of the video, resulting in a set of marked images with black smoke and ship identification boxes. S2. Identify whether there is a ship emitting black smoke in the video. The specific steps are as follows: S201. Calculate the center distance d between each black smoke identification box and each ship identification box in each marked image. i,t The calculation formula is as follows: d i,t =ps j,t -pv i,t , In the formula, t is the acquisition time of the marked image; ps j,t ρ is the pixel x-coordinate of the center point of the black smoke identification box; j is the index of the black smoke identification box in the marked image, pv i,t is the pixel x-coordinate of the center point of the ship identification frame; i is the ship number; S202, the distance d between all center points of the same vessel i i,t Find the average value D i ; S203. Calculate the distance d between the center points of the same ship i. i,t The corresponding absolute value of distance deviation b i,t The calculation formula is as follows: b i,t =∣d i,t -D i ∣; S204. Based on the set maximum distance deviation upper limit value b max Calculate the absolute value b of the distance deviation occurring in the same ship i. i,t Less than the maximum distance deviation upper limit b max number of times n i ; S205. Based on the set minimum number of times n can be detected for ship black smoke... min and with n i Comparison: when n i ≥n min If ship i has committed a violation by emitting black smoke, then ship i has committed a violation by emitting black smoke once; when n i <n min If there is no violation of regulations regarding black smoke emitted by vessel i; S3. The absolute value b of all distance deviations corresponding to vessel i, which was identified in step S2 as having one violation involving black smoke emission. i,t In the process, select those that satisfy the condition of being less than the upper limit of the maximum distance deviation b. max The absolute value of multiple distance deviations b i,t And find the minimum value t of the marked image acquisition time t. min and maximum value t max In step S1, the video is captured using t min For video start time, t max Extract the violation video of vessel i based on the video end time.

2. The video-based automatic identification algorithm for ship black smoke emissions according to claim 1, characterized in that, In step S1, the video image acquisition frequency is 2 frames / second to 5 frames / second.

3. The video-based automatic identification algorithm for ship black smoke emissions according to claim 1, characterized in that, In step S1, the identification and marking of black smoke in the image is completed by a black smoke identification algorithm, and the identification and marking of ships in the image is completed by a ship identification algorithm.

4. The video-based automatic identification algorithm for ship black smoke emissions according to claim 1, characterized in that, In step 4) of step S2, the upper limit of the maximum distance deviation is b. max Set to 50-100.

5. The video-based automatic identification algorithm for ship black smoke emissions according to claim 1, characterized in that, In step 5) of step S2, the minimum number of times n is required to identify ship black smoke. min Set to 10 to 100 times.

Images