A method and apparatus for monitoring a port for polluting vessels

Abstract

The application provides a port pollution ship monitoring method and device, the method comprises the following steps: acquiring a video image of a port area shot by a UAV, the video image is composed of continuous video frames; using a mixed Gaussian background model to perform foreground detection on each video frame pixel by pixel to obtain a foreground image containing a potential area; performing optical flow calculation on each foreground pixel point in the foreground image to determine the motion information of each foreground pixel point; according to the motion speed data and the motion direction information in the motion information, screening abnormal pixel points from the foreground pixel points that meet a preset abnormal condition; performing connected domain analysis on the abnormal pixel points, and determining a pollution area and a pollution type according to the analysis result; based on the detection time and the spatial position of each pollution area, combining ship automatic identification system data, calculating the correlation probability of each pollution area and a candidate ship to determine a pollution ship. The pollution ship is traced, and the port pollution is monitored in a wide coverage and all-weather mode.

Description

Technical Field

[0001] This invention relates to the field of ship monitoring technology, and in particular to a method and apparatus for monitoring polluting ships in ports. Background Technology

[0002] Currently, visual monitoring of oil spills and sewage discharges from port vessels mainly relies on manual patrols or fixed sensors, making it difficult to achieve real-time, continuous coverage of large areas of port waters. This technological need stems from the high frequency and dynamic nature of pollution incidents in port waters, necessitating technological means to overcome the temporal and spatial limitations of traditional monitoring methods.

[0003] Specifically, existing technologies suffer from the following drawbacks: Manual patrols rely on periodic inspections, resulting in high costs and limited coverage. Furthermore, monitoring effectiveness is heavily dependent on weather conditions, light intensity, and the experience of patrol personnel, making 24 / 7 monitoring impossible and leading to delayed detection and response to pollution incidents. While fixed sensor deployments can provide precise water quality data at specific locations, their fixed monitoring range, large blind spots, and high deployment and maintenance costs make them ill-suited to the dynamic changes in port water surfaces. This hinders flexible and continuous visual perception and event capture, creating a technical bottleneck of "monitoring blind spots and data fragmentation." Additionally, existing vision systems can only identify surface pollution incidents but lack the ability to track and correlate the behavior of polluting vessels in real time, making it difficult to quickly identify the responsible party after a pollution incident, thus impacting pollution control efficiency and the regulatory loop. Summary of the Invention

[0004] This invention provides a method and apparatus for monitoring polluting vessels in ports, which uses a Gaussian mixture background model to determine the foreground image and combines abnormal pixels with data from an automatic ship identification system to monitor polluting vessels.

[0005] In a first aspect, the present invention provides a method for monitoring polluting vessels in ports, comprising: Acquire video images of the port area captured by a drone, the video images being composed of consecutive video frames; Using a Gaussian mixture background model, foreground detection is performed pixel by pixel on each of the video frames to obtain a foreground image containing potential regions; Optical flow calculation is performed on each foreground pixel in the foreground image to determine the motion information of each foreground pixel; Based on the motion speed data and motion direction information in the motion information, abnormal pixels that meet the preset abnormal conditions are selected from the foreground pixels. Connectivity analysis is performed on the abnormal pixels, and the contaminated areas and their contamination types are determined based on the analysis results. Based on the detection time and spatial location of each polluted area, and combined with data from the Automatic Identification System (AIS), the association probability between each polluted area and candidate vessels is calculated to identify the polluted vessels.

[0006] Optionally, using a Gaussian mixture background model, foreground detection is performed pixel-by-pixel on each of the video frames to obtain a foreground image containing the potential region, including: For each pixel, initialize multiple Gaussian distributions containing mean, variance, and weight parameters; Calculate the matching degree between the pixel value of each pixel in the video frame and the plurality of Gaussian distributions to determine the matching result; Based on the matching results, the plurality of Gaussian distributions are updated; Based on the updated weights of the Gaussian distributions, the Gaussian distributions with cumulative weights greater than a preset background threshold are selected as the background model. The pixel value is matched with the background model. If the matching degree of the pixel value with all distributions in the background model does not meet the preset foreground condition, it is determined to be a foreground pixel. The foreground image is composed of the foreground pixels.

[0007] Optionally, the plurality of Gaussian distributions are updated based on the matching results, including: If the matching result is a Gaussian distribution with matching degree satisfying the preset matching condition, then the corresponding Gaussian distribution is determined as the matching distribution, and the weight, mean and variance of the corresponding matching distribution are dynamically updated according to the current pixel value. If the matching result is that the matching degree of all Gaussian distributions does not meet the preset matching condition, then a new Gaussian distribution is initialized with the current pixel value, and the distribution with the smallest weight in the original Gaussian distribution is replaced.

[0008] Optionally, based on the motion speed data and motion direction information in the motion information, abnormal pixels that meet preset abnormal conditions are filtered from the foreground pixels, including: Calculate the optical flow vector for each of the foreground pixels to obtain the corresponding actual motion speed; Obtain the water flow direction vector for each of the foreground pixels, and calculate the angle between the corresponding motion direction and the water flow direction; If the actual movement speed is greater than the preset movement speed and the included angle is greater than the preset angle, then it is determined to be an abnormal pixel.

[0009] Optionally, the pollution types include: oil film pollution areas and wastewater pollution areas; connected component analysis is performed on the abnormal pixels, and the pollution areas and their pollution types are determined based on the analysis results, including: The abnormal pixels are labeled with connected components to construct at least one connected region; Calculate the ratio of the area to the perimeter of each of the connected regions; If the ratio is greater than the first preset threshold, the connected region is determined to be an oil film contamination region; if the ratio is less than or equal to the first preset threshold and greater than the second preset threshold, the connected region is determined to be a sewage contamination region.

[0010] Optionally, based on the detection time and spatial location of each polluted area, and combined with data from the Automatic Identification System (AIS), the association probability between each polluted area and candidate vessels is calculated to identify the polluted vessels, including: Based on the detection time of the polluted area, a time window is dynamically selected, and all the Automatic Identification System (AIS) data reported within that window are filtered out. The image coordinates of the polluted area and the geographical coordinates of the vessels in the filtered Automatic Identification System (AIS) data are converted to the same coordinate system using a georeferencing function. Based on the transformed coordinates, the spatial correlation between the polluted area and each ship is dynamically calculated; the spatial correlation is determined based on the Euclidean distance between the polluted area and the candidate ship and the directional difference between the expansion direction of the polluted area and the ship's heading. The vessel with the highest spatial correlation is selected as the polluting vessel associated with the polluted area.

[0011] Secondly, the present invention provides a monitoring device for polluting vessels in ports, comprising: The acquisition module is used to acquire video images of the port area captured by the UAV, the video images being composed of consecutive video frames; The foreground detection module is used to perform foreground detection pixel by pixel in each video frame using a Gaussian mixture background model to obtain a foreground image containing potential regions. The optical flow calculation module is used to perform optical flow calculation on each foreground pixel in the foreground image to determine the motion information of each foreground pixel. The filtering module is used to filter abnormal pixels that meet preset abnormal conditions from the foreground pixels based on the motion speed data and motion direction information in the motion information. The connected component analysis module is used to perform connected component analysis on the abnormal pixels and determine the contaminated area and its contamination type based on the analysis results. The polluted vessel identification module is used to calculate the association probability between each polluted area and candidate vessels based on the detection time and spatial location of each polluted area, combined with data from the Automatic Identification System (AIS), in order to identify the polluted vessels.

[0012] Thirdly, the present invention provides an electronic device including a processor and a memory, the memory storing computer-readable instructions that, when executed by the processor, perform the steps of the method provided in the first aspect above.

[0013] Fourthly, the present invention provides a storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method provided in the first aspect above.

[0014] Fifthly, the present invention provides a computer program product comprising a computer program that, when executed by a processor, performs the steps of the method provided in the first aspect above.

[0015] As can be seen from the above technical solutions, the present invention has the following advantages: This invention provides a method and apparatus for monitoring polluting vessels in ports. The method includes: acquiring video images of a port area captured by a drone, the video images consisting of consecutive video frames; performing foreground detection pixel-by-pixel on each video frame using a Gaussian mixture background model to obtain a foreground image containing potential areas; performing optical flow calculation on each foreground pixel in the foreground image to determine the motion information of each foreground pixel; filtering out abnormal pixels that meet preset abnormality conditions from the foreground pixels based on the motion speed data and motion direction information in the motion information; performing connected component analysis on the abnormal pixels and determining the polluted areas and their pollution types based on the analysis results; calculating the association probability between each polluted area and candidate vessels based on the detection time and spatial location of each polluted area, combined with data from an Automatic Identification System (AIS), to identify polluting vessels. The method utilizes a Gaussian mixture model to extract the foreground image from the video images acquired by the drone, then combines optical flow to filter abnormal pixels and exclude normal fluctuations, classifies pollution types by the area-to-perimeter ratio of connected components, and finally integrates AIS data to calculate the association probability based on distance and direction difference, thereby tracing polluting vessels and achieving wide coverage and all-weather monitoring of port pollution. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the steps of a method for monitoring polluting vessels in a port according to the present invention. Figure 2This is a flowchart illustrating the steps of a second embodiment of the method for monitoring polluting vessels in a port according to the present invention. Figure 3 This is a structural block diagram of an embodiment of a port pollution monitoring device for vessels according to the present invention. Detailed Implementation

[0018] This invention provides a method and apparatus for monitoring polluting vessels in ports, which uses a Gaussian mixture background model to determine the foreground image and combines abnormal pixels with data from an automatic ship identification system to monitor polluting vessels.

[0019] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0020] Example 1 Please see Figure 1 , Figure 1 This is a flowchart illustrating the steps of a method for monitoring polluting vessels in a port according to the present invention. The method includes: Step S101: Acquire video images of the port area captured by the drone, wherein the video images are composed of consecutive video frames; In this embodiment of the application, a drone deployed in the port is equipped with a camera device to continuously film the water surface of the port area according to a preset route or hovering position, and obtain video images containing scenes such as port ships, water surface, and docks.

[0021] Step S102: Using a Gaussian mixture background model, perform foreground detection pixel by pixel on each video frame to obtain a foreground image containing potential regions; In this embodiment, multiple Gaussian distributions with mean, variance, and weights are initialized for each pixel. For each pixel in the current frame, the Mahalanobis distance between its RGB value and each Gaussian distribution is calculated, and the distribution parameters are dynamically updated based on the matching results. After sorting by weight, the distribution with a cumulative sum greater than a preset background threshold is selected as the background model. The current pixel is compared with the background model; if it does not match any of the background distributions, it is determined to be a foreground pixel, and all foreground pixels constitute the foreground image.

[0022] Step S103: Perform optical flow calculation on each foreground pixel in the foreground image to determine the motion information of each foreground pixel; In this embodiment, optical flow is used to calculate the optical flow vector of each pixel in adjacent frames of the foreground image sequence. The optical flow vector represents the motion information of the pixel, including its speed and direction.

[0023] Step S104: Based on the motion speed data and motion direction information in the motion information, filter out abnormal pixels that meet the preset abnormal conditions from the foreground pixels. In this embodiment, the actual speed of the foreground pixel is calculated using the magnitude, conversion factor and frame rate of the optical flow vector, and the angle of motion direction is obtained by combining the water flow direction vector; if the speed is greater than a preset threshold and the angle is greater than a preset angle, it is determined to be an abnormal pixel to exclude normal water flow fluctuations.

[0024] Step S105: Perform connected component analysis on the abnormal pixels and determine the contaminated area and its contamination type based on the analysis results; In this embodiment, abnormal pixels are labeled as connected components, and adjacent abnormal pixels are merged into several connected regions. The area and perimeter of each region are calculated, and the area-to-perimeter ratio is obtained. The pollution type is determined based on the area-to-perimeter ratio, thus obtaining the spatial range and type of each polluted region.

[0025] Step S106: Based on the detection time and spatial location of each polluted area, and combined with the data from the Automatic Identification System (AIS), calculate the association probability between each polluted area and the candidate vessel to determine the polluted vessel.

[0026] In this embodiment, a time window is selected based on the detection time of the polluted area, and AIS data within the window is filtered; the coordinates of the polluted area and the ship are unified through georegistration, and the Euclidean distance between their centroids and the difference between their expansion direction and heading are calculated; the correlation probability is calculated based on the Gaussian kernel function, and the ship corresponding to the maximum value is the polluted ship.

[0027] This invention provides a method for monitoring polluting vessels in ports. The method involves acquiring video images of a port area captured by a drone, where the video images consist of consecutive video frames. Using a Gaussian mixture model (Gaussian background model), foreground detection is performed pixel-by-pixel on each video frame to obtain a foreground image containing potential areas. Optical flow calculation is performed on each foreground pixel in the foreground image to determine its motion information. Based on the motion speed and direction information, abnormal pixels that meet preset abnormality conditions are selected from the foreground pixels. Connectivity analysis is performed on the abnormal pixels, and the polluted areas and their pollution types are determined based on the analysis results. Based on the detection time and spatial location of each polluted area, combined with data from the Automatic Identification System (AIS), the association probability between each polluted area and candidate vessels is calculated to identify the polluting vessels. The method utilizes a Gaussian mixture model to extract the foreground image from the video images acquired by the drone, then uses optical flow to filter abnormal pixels and exclude normal fluctuations. Pollution types are then classified using the area-to-perimeter ratio of connected components. Finally, AIS data is fused to calculate the association probability based on distance and direction difference, thereby tracing polluting vessels and achieving broad coverage and all-weather monitoring of port pollution.

[0028] Example 2 Please see Figure 2 , Figure 2 This is a flowchart illustrating a second embodiment of the method for monitoring polluting vessels in a port according to the present invention. The steps include: Step S201: Acquire video images of the port area captured by the drone, wherein the video images are composed of consecutive video frames; In this embodiment, a drone deployed at the port, equipped with a high-definition camera, continuously films the water surface of the port area according to a preset route or hovering position. The drone transmits the collected video stream to the ground processing system in real time or near real time. After receiving the data, the system decomposes it into continuous video images, each frame of which is an RGB color image containing information about the port water surface, ships, dock facilities, etc.

[0029] Step S202: Initialize multiple Gaussian distributions containing mean, variance and weight parameters for each pixel; In this embodiment of the application, for each pixel in the video image, the system constructs a preset number (usually 3 to 5) Gaussian distributions. The mean vector of each Gaussian distribution is initialized to the RGB pixel value of that pixel in the first frame of the video; the variance vector is initialized to a large preset value (such as 30-50) to accommodate initial fluctuations; and the weights are initialized to the average of all distributions (i.e., 1 / K, where K is the number of Gaussian distributions). This initialization lays the foundation for subsequent dynamic background modeling.

[0030] Step S203: Calculate the matching degree between the pixel value of each pixel in the video frame and the plurality of Gaussian distributions to determine the matching result; For each pixel in the current video frame, calculate the Mahalanobis distance between its RGB pixel value and each Gaussian distribution. The Mahalanobis distance takes into account the variance differences of each color channel, and the formula is: ; in, The Mahalanobis distance, The red channel pixel value of the pixel. The mean of the Gaussian distribution along the red channel. Let Variance be the Gaussian distribution along the red channel. The green channel pixel value of the pixel. The mean of the Gaussian distribution along the green channel. Let Variance be the Gaussian distribution along the green channel. The blue channel pixel value of the pixel. The mean of the Gaussian distribution in the blue channel. Let V be the variance of the Gaussian distribution in the blue channel.

[0031] The calculated Mahalanobis distance is compared with a preset distance threshold (e.g., 2.5). If the Mahalanobis distance is less than the threshold, the pixel is considered to match the Gaussian distribution; otherwise, it is considered not to match. The final matching result between the pixel and each Gaussian distribution is then determined.

[0032] Step S204: Update the plurality of Gaussian distributions according to the matching results; In this embodiment of the application, if the matching result is that there is a Gaussian distribution whose matching degree meets the preset matching condition, then the corresponding Gaussian distribution is determined as the matching distribution, and the weight, mean and variance of the corresponding matching distribution are dynamically updated according to the current pixel value; if the matching result is that the matching degree of all Gaussian distributions does not meet the preset matching condition, then a new Gaussian distribution is initialized with the current pixel value, and the distribution with the smallest weight in the original Gaussian distribution is replaced.

[0033] In the specific implementation, if a Gaussian distribution exists that satisfies the preset matching conditions, then this distribution is determined as the matching distribution, and the weights are updated according to the preset learning rate combined with the weight update formula. Simultaneously, the rate is updated based on the probability density function value negatively correlated with Mahalanobis distance, combined with the rate update formula, and then the mean and variance are updated using the mean update formula and the variance update formula. If the matching degree of all Gaussian distributions does not satisfy the preset matching conditions, a new Gaussian distribution is initialized with the current pixel value, its mean is set to X, its variance is set to a relatively large initial value, and its weights are set to a relatively small initial value (e.g., 0.01), replacing the distribution with the smallest weight among the original Gaussian distributions.

[0034] The weight update formula is as follows: ; The rate update formula is: ; The mean update formula is: ; The variance update formula is: ; in, For the updated weights, For learning rate, The original weights, The probability density function value is related to the Mahalanobis distance. , Proportional to, The mean, For update rate, This is the updated mean. This is the original mean. For the updated variance, This represents the original variance.

[0035] Step S205: Based on the updated weight sorting of the Gaussian distribution, select the Gaussian distribution with a cumulative weight greater than the preset background threshold as the background model; In this embodiment, all updated Gaussian distributions are sorted in descending order according to their weights or weight / variance ratios, and the weights of these distributions are accumulated sequentially. When the accumulated sum first exceeds a preset background threshold, a preset number of Gaussian distributions with the largest values ​​are selected as the background model. These selected distributions represent the most common pixel value change patterns of that pixel, i.e., background features.

[0036] Step S206: Match the pixel value with the background model. If the matching degree of the pixel value with all distributions in the background model does not meet the preset foreground condition, it is determined to be a foreground pixel. The foreground image is composed of the foreground pixels. In this embodiment, the Mahalanobis distance between the pixel value of the current pixel and each Gaussian distribution in the background model is calculated. If the Mahalanobis distance between the pixel value and all distributions in the background model is greater than a preset foreground threshold, the pixel is determined to be uninterpretable by the background model and belongs to the foreground pixel; otherwise, if it matches any background distribution, it is determined to be a background pixel. After completing the above judgment for all pixels in the current frame, all foreground pixels are combined to form a binary foreground image.

[0037] Step S207: Perform optical flow calculation on each foreground pixel in the foreground image to determine the motion information of each foreground pixel; In this embodiment of the application, the optical flow method is used to calculate the foreground pixels of adjacent frames in the foreground image sequence to obtain the motion vector of each foreground pixel. The motion vector is the motion information of the pixel, including the motion direction and motion amplitude.

[0038] Step S208: Based on the motion speed data and motion direction information in the motion information, filter out abnormal pixels that meet the preset abnormal conditions from the foreground pixels; In this embodiment, the optical flow vector of each foreground pixel is calculated to obtain the corresponding actual motion speed; the water flow direction vector of each foreground pixel is obtained, and the angle between the corresponding motion direction and the water flow direction is calculated; if the actual motion speed is greater than the preset motion speed and the angle is greater than the preset angle, it is determined to be the abnormal pixel.

[0039] In the specific implementation, the actual motion speed of each foreground pixel is first calculated using the magnitude of the optical flow vector, the conversion factor from pixel to actual distance, and the video frame rate. ,in, The actual speed of motion, As the conversion factor, The frame rate is used to obtain the water flow direction vector of that pixel. Calculate the angle between the direction of motion and the direction of water flow. .

[0040] If the actual movement speed is greater than 0.3 m / s and the included angle is greater than 45°, the pixel is determined to be an abnormal pixel to exclude normal water flow fluctuations and preliminarily locate possible contaminated areas.

[0041] Step S209: Perform connected component analysis on the abnormal pixels and determine the contaminated area and its contamination type based on the analysis results; It should be noted that the pollution types include: oil film contamination areas and wastewater contamination areas.

[0042] In this embodiment of the application, the abnormal pixels are marked with connected components to construct at least one connected region; the ratio of the area to the perimeter of each connected region is calculated; if the ratio is greater than a first preset threshold, the connected region is determined to be an oil film contamination region; if the ratio is less than or equal to the first preset threshold and greater than a second preset threshold, the connected region is determined to be a sewage contamination region.

[0043] In the specific implementation, the abnormal pixels selected in step S208 are labeled with connected components, and adjacent abnormal pixels are merged into several connected regions, each representing a potential contaminated area. Then, the area of ​​each connected region is calculated. That is, the total number of pixels in the region, where, For area, The region is connected; its perimeter is also calculated. ,in For the number of boundary pixels, The distance between adjacent boundary pixels (1 for horizontal or vertical adjacency, 0 for diagonal adjacency). Then, the ratio of the area to the perimeter of each connected region is obtained. The pollution type is determined based on this ratio: if the ratio is greater than the first preset threshold (e.g., 1.5), the connected area is determined to be an oil film pollution area; if the ratio is greater than the second preset threshold (e.g., 1.2) and less than or equal to the first preset threshold, the connected area is determined to be a sewage pollution area.

[0044] Step S210: Based on the detection time of the polluted area, dynamically select a time window and filter out all the Automatic Identification System (AIS) data reported within that window; In this embodiment, for each polluted area, spatiotemporal alignment is first performed, selecting a preset time window centered on the detection time of that area. All Automatic Identification System (AIS) data reported within this window are then filtered to form a candidate vessel set. ,in For the detection time of the contaminated area, For the first Automatic Identification System (AIS) data reporting time for each vessel Set a preset time tolerance (e.g., 10 seconds). Indicates the first A ship.

[0045] This application only considers ships that are active within a short period of time before and after the pollution occurs, because ships can only move a limited distance within 10 seconds, and the time synchronization error between systems can be tolerated. This effectively narrows down the candidate range while ensuring the accuracy of the correlation, and improves the efficiency of subsequent calculations.

[0046] Step S211: The image coordinates of the polluted area and the geographical coordinates of the vessels in the selected Automatic Identification System (AIS) data are converted to the same coordinate system using a georeferencing function. In this embodiment of the application, the centroid pixel coordinates of the polluted area are converted into actual geographic coordinates using a georegistration function, i.e. ,in, For actual geographic coordinates, For geographic registration functions, The centroid pixel coordinates are used; simultaneously, the geographic coordinates of each selected ship are converted into image pixel coordinates using an inverse georeferencing function, i.e. ,in, The image pixel coordinates of the ship. It is an inverse geographic registration function. The coordinates are the ship's geographic coordinates. Through bidirectional transformation, the location information of the polluted area and the ship are unified into the same spatial reference system.

[0047] Step S212: Based on the transformed coordinates, dynamically calculate the spatial correlation between the polluted area and each ship; the spatial correlation is determined based on the Euclidean distance between the polluted area and the candidate ship and the directional difference between the expansion direction of the polluted area and the ship's heading. In this application, for each candidate vessel, the Euclidean distance between the center of mass of the polluted area and the center of mass of the vessel, and the absolute difference between the direction angle of the pollution area's expansion and the vessel's heading angle are calculated. Combining the distance and direction difference, the correlation probability between the two is calculated using the correlation concept calculation formula. The correlation probability calculation formula is: ; in, For the probability of association, Euclidean distance. For the maximum association distance, The maximum associated distance tolerance (e.g., 1000 meters). This is the maximum permissible directional deviation tolerance (e.g., 30°).

[0048] Step S213: Select the ship with the highest spatial correlation as the polluting vessel associated with the polluted area.

[0049] In this embodiment, for each polluted area, its association probability with all candidate vessels is compared, and the vessel with the highest probability is selected as the polluting vessel for that area. If the association probability of a vessel is significantly higher than that of other vessels (e.g., >0.9), the determination result is highly reliable; if all association probabilities are low, it may indicate that the pollution source is a vessel that has not activated its Automatic Identification System (AIS) or other non-vessel sources. The final output includes information on the polluted area and its associated polluting vessels, including vessel name, MMSI number, and pollution type, for subsequent enforcement or recording.

[0050] This invention provides a method for monitoring polluting vessels in a port. The method involves acquiring video images of a port area captured by a drone, where the video images consist of consecutive video frames. Using a Gaussian mixture background model, foreground detection is performed pixel-by-pixel on each video frame to obtain a foreground image containing potential regions. Optical flow calculation is performed on each foreground pixel in the foreground image to determine its motion information. Based on the motion speed and direction information in the motion information, abnormal pixels that meet preset abnormality conditions are selected from the foreground pixels. Connectivity analysis is performed on the abnormal pixels, and the polluted areas and their pollution types are determined based on the analysis results. Based on the detection time and spatial location of each polluted area, combined with data from an Automatic Identification System (AIS), the association probability between each polluted area and candidate vessels is calculated to identify the polluting vessels. By acquiring port video images using drones, a multi-Gaussian distribution is established for each pixel and dynamically updated. Combined with Mahalanobis distance matching and weighted sorting, dynamic interference such as waves and changes in lighting is effectively overcome, and the foreground area is accurately extracted. Then, abnormal pixels are screened using optical flow to exclude normal fluctuations. Pollution types are then classified by the ratio of area to perimeter of connected components. Based on time windows and geographic registration, the correlation probability is calculated using Euclidean distance and direction difference, which significantly improves the accuracy of tracing polluting vessels. This solves the technical problems of limited range of manual inspection, large blind spots of fixed sensors, and inability to trace polluting vessels, and achieves precise monitoring of port pollution.

[0051] To facilitate those skilled in the art to understand the beneficial effects of the present invention, an example of a method for detecting the etching thickness of button glass according to the present invention is described below.

[0052] This example uses a drone to acquire video images of the port. Through foreground detection and anomaly pixel filtering, a contaminated area is identified at 10:30:25. The image pixel centroid is (350, 480), and the contaminated area's expansion direction angle is 120° (image coordinate system). The area is determined to be an oil film contamination area based on its area-to-perimeter ratio. The image coordinates of this contaminated area are then converted to actual geographic coordinates (122.5°E, 30.8°N) using a georeferencing function. With a time tolerance Δt = 10 seconds, ships reporting AIS data between 10:30:15 and 10:30:35 were selected, resulting in three candidate ships: Ship A (reporting time 10:30:20, position 122.501°E, 30.799°N, heading 115°), Ship B (reporting time 10:30:28, position 122.503°E, 30.802°N, heading 130°), and Ship C (reporting time 10:30:32, position 122.495°E, 30.805°N, heading 95°). The geographic coordinates of Ship A were inversely converted to image pixel coordinates (352, 478). The Euclidean distance between the centroid of the contaminated area and the centroid of Ship A was calculated to be 250 meters, and the directional difference between the expansion direction of 120° and the ship's heading of 115° was 5°. Similarly, calculate the distance to ship B: 800 meters, direction difference 10°; and the distance to ship C: 1200 meters, direction difference 25°. Set the correlation parameters σ_d = 1000 meters, σ_ =30°, and the association probability of each ship was calculated based on the Gaussian kernel function: Ship A is 0.956, Ship B is 0.687, and Ship C is 0.344. Comparing the association probabilities, Ship A has the highest probability, therefore Ship A is determined to be the polluting vessel corresponding to this polluted area.

[0053] Therefore, the port pollution monitoring method of the present invention can accurately trace the polluting vessel. Compared with traditional manual inspection and fixed sensors, this method achieves wide coverage of port pollution, all-weather monitoring and accurate identification of the responsible vessel, significantly improving regulatory efficiency.

[0054] Example 3 Please see Figure 3 , Figure 3 This is a structural block diagram of an embodiment of a port pollution monitoring device for vessels according to the present invention. The device includes: The acquisition module 301 is used to acquire video images of the port area captured by the UAV, the video images being composed of consecutive video frames; Foreground detection module 302 is used to perform foreground detection pixel by pixel in each video frame using a Gaussian mixture background model to obtain a foreground image containing potential regions; The optical flow calculation module 303 is used to perform optical flow calculation on each foreground pixel in the foreground image to determine the motion information of each foreground pixel. The filtering module 304 is used to filter abnormal pixels that meet preset abnormal conditions from the foreground pixels based on the motion speed data and motion direction information in the motion information. The connected component analysis module 305 is used to perform connected component analysis on the abnormal pixels and determine the contaminated area and its contamination type based on the analysis results. The polluted vessel identification module 306 is used to calculate the association probability between each polluted area and the candidate vessel based on the detection time and spatial location of each polluted area and combined with the data from the Automatic Identification System for Ships, so as to identify the polluted vessel.

[0055] In an optional embodiment, the foreground detection module 302 includes: The initialization submodule is used to initialize multiple Gaussian distributions containing mean, variance, and weight parameters for each pixel. The matching degree calculation submodule is used to calculate the matching degree between the pixel value of each pixel in the video frame and the plurality of Gaussian distributions to determine the matching result; The Gaussian distribution submodule is used to update the plurality of Gaussian distributions based on the matching results; The background model selection submodule is used to select a Gaussian distribution with a cumulative weight greater than a preset background threshold as the background model based on the updated weight sorting of the Gaussian distribution. The matching submodule is used to match the pixel value with the background model. If the matching degree of the pixel value with all distributions in the background model does not meet the preset foreground conditions, it is determined to be a foreground pixel. The foreground image is composed of the foreground pixels.

[0056] In an optional embodiment, the Gaussian distribution submodule includes: The unit number is dynamically updated. If the matching result is a Gaussian distribution with a matching degree that satisfies the preset matching conditions, the corresponding Gaussian distribution is determined as the matching distribution, and the weight, mean and variance of the corresponding matching distribution are dynamically updated according to the current pixel value. The replacement unit is used to initialize a new Gaussian distribution with the current pixel value and replace the distribution with the smallest weight in the original Gaussian distribution if the matching result is that the matching degree of all Gaussian distributions does not meet the preset matching condition.

[0057] In an optional embodiment, the filtering module 304 includes: The optical flow calculation submodule is used to calculate the optical flow vector of each of the foreground pixels to obtain the corresponding actual motion speed; Angle calculation submodule is used to obtain the water flow direction vector of each foreground pixel and calculate the angle between the corresponding motion direction and the water flow direction; The abnormal point judgment submodule is used to determine an abnormal pixel point if the actual movement speed is greater than the preset movement speed and the included angle is greater than the preset angle.

[0058] In an optional embodiment, the pollution type includes: oil film contamination areas and wastewater contamination areas; the connectivity analysis module 305 includes: The region construction submodule is used to mark the abnormal pixels as connected components and construct at least one connected region. The ratio calculation submodule is used to calculate the ratio of the area to the perimeter of each of the connected regions; The region determination submodule is used to determine that the connected region is an oil film contamination region if the ratio is greater than a first preset threshold; and to determine that the connected region is a sewage contamination region if the ratio is less than or equal to the first preset threshold and greater than a second preset threshold.

[0059] In an optional embodiment, the polluting vessel determination module 306 includes: The window selection submodule is used to dynamically select a time window based on the detection time of the polluted area, and filter out all the Automatic Identification System (AIS) data reported within that window. The conversion submodule is used to convert the image coordinates of the polluted area and the geographical coordinates of the vessels in the filtered Automatic Identification System (AIS) data to the same coordinate system using a georegistration function. The correlation calculation submodule is used to dynamically calculate the spatial correlation between the polluted area and each ship based on the transformed coordinates; the spatial correlation is determined based on the Euclidean distance between the polluted area and the candidate ship and the directional difference between the expansion direction of the polluted area and the ship's heading. The polluted vessel identification submodule is used to select the vessel with the highest spatial correlation as the polluted vessel associated with the polluted area.

[0060] Example 4 This invention also provides an electronic device, including a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of a method for monitoring polluting vessels in a port, according to any embodiment.

[0061] Example 5 This invention also provides a computer storage medium storing a computer program thereon, which, when executed by the processor, implements the steps of a method for monitoring polluting vessels in a port according to any embodiment.

[0062] Example 6 This invention also provides a computer program product having a computer program stored thereon, wherein the computer program, when executed by the processor, implements the steps of a method for monitoring polluting vessels in a port according to any embodiment.

[0063] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0064] In the several embodiments provided in this application, it should be understood that the methods, apparatuses, electronic devices, and storage media disclosed in this invention can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0065] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0066] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0067] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0068] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for monitoring polluting vessels in ports, characterized in that, include: Acquire video images of the port area captured by a drone, the video images being composed of consecutive video frames; Using a Gaussian mixture background model, foreground detection is performed pixel by pixel on each of the video frames to obtain a foreground image containing potential regions; Optical flow calculation is performed on each foreground pixel in the foreground image to determine the motion information of each foreground pixel; Based on the motion speed data and motion direction information in the motion information, abnormal pixels that meet the preset abnormal conditions are selected from the foreground pixels. Connectivity analysis is performed on the abnormal pixels, and the contaminated areas and their contamination types are determined based on the analysis results. Based on the detection time and spatial location of each polluted area, and combined with data from the Automatic Identification System (AIS), the association probability between each polluted area and candidate vessels is calculated to identify the polluted vessels.

2. The method for monitoring polluting vessels in ports according to claim 1, characterized in that, Using a Gaussian mixture background model, foreground detection is performed pixel-by-pixel on each of the video frames to obtain a foreground image containing potential regions, including: For each pixel, initialize multiple Gaussian distributions containing mean, variance, and weight parameters; Calculate the matching degree between the pixel value of each pixel in the video frame and the plurality of Gaussian distributions to determine the matching result; Based on the matching results, the plurality of Gaussian distributions are updated; Based on the updated weights of the Gaussian distributions, the Gaussian distributions with cumulative weights greater than a preset background threshold are selected as the background model. The pixel value is matched with the background model. If the matching degree of the pixel value with all distributions in the background model does not meet the preset foreground condition, it is determined to be a foreground pixel. The foreground image is composed of the foreground pixels.

3. The method for monitoring polluting vessels in ports according to claim 2, characterized in that, Based on the matching results, the plurality of Gaussian distributions are updated, including: If the matching result is a Gaussian distribution with matching degree satisfying the preset matching condition, then the corresponding Gaussian distribution is determined as the matching distribution, and the weight, mean and variance of the corresponding matching distribution are dynamically updated according to the current pixel value. If the matching result is that the matching degree of all Gaussian distributions does not meet the preset matching condition, then a new Gaussian distribution is initialized with the current pixel value, and the distribution with the smallest weight in the original Gaussian distribution is replaced.

4. The method for monitoring polluting vessels in ports according to claim 1, characterized in that, Based on the motion speed data and motion direction information in the motion information, abnormal pixels that meet preset abnormality conditions are filtered from the foreground pixels, including: Calculate the optical flow vector for each of the foreground pixels to obtain the corresponding actual motion speed; Obtain the water flow direction vector for each of the foreground pixels, and calculate the angle between the corresponding motion direction and the water flow direction; If the actual movement speed is greater than the preset movement speed and the included angle is greater than the preset angle, then it is determined to be an abnormal pixel.

5. The method for monitoring polluting vessels in ports according to claim 1, characterized in that, The pollution types include: oil film pollution areas and wastewater pollution areas; connected component analysis is performed on the abnormal pixels, and the pollution areas and their pollution types are determined based on the analysis results, including: The abnormal pixels are labeled with connected components to construct at least one connected region; Calculate the ratio of the area to the perimeter of each of the connected regions; If the ratio is greater than the first preset threshold, the connected region is determined to be an oil film contamination region; if the ratio is less than or equal to the first preset threshold and greater than the second preset threshold, the connected region is determined to be a sewage contamination region.

6. The method for monitoring polluting vessels in ports according to claim 1, characterized in that, Based on the detection time and spatial location of each contaminated area, and combined with data from the Automatic Identification System (AIS), the association probability between each contaminated area and candidate vessels is calculated to identify the contaminated vessels, including: Based on the detection time of the polluted area, a time window is dynamically selected, and all the Automatic Identification System (AIS) data reported within that window are filtered out. The image coordinates of the polluted area and the geographical coordinates of the vessels in the filtered Automatic Identification System (AIS) data are converted to the same coordinate system using a georeferencing function. Based on the transformed coordinates, the spatial correlation between the polluted area and each ship is dynamically calculated; the spatial correlation is determined based on the Euclidean distance between the polluted area and the candidate ship and the directional difference between the expansion direction of the polluted area and the ship's heading. The vessel with the highest spatial correlation is selected as the polluting vessel associated with the polluted area.

7. A monitoring device for polluting vessels in a port, characterized in that, include: The acquisition module is used to acquire video images of the port area captured by the UAV, the video images being composed of consecutive video frames; The foreground detection module is used to perform foreground detection pixel by pixel in each video frame using a Gaussian mixture background model to obtain a foreground image containing potential regions. The optical flow calculation module is used to perform optical flow calculation on each foreground pixel in the foreground image to determine the motion information of each foreground pixel. The filtering module is used to filter abnormal pixels that meet preset abnormal conditions from the foreground pixels based on the motion speed data and motion direction information in the motion information. The connected component analysis module is used to perform connected component analysis on the abnormal pixels and determine the contaminated area and its contamination type based on the analysis results. The polluted vessel identification module is used to calculate the association probability between each polluted area and candidate vessels based on the detection time and spatial location of each polluted area, combined with data from the Automatic Identification System (AIS), in order to identify the polluted vessels.

8. An electronic device, characterized in that, It includes a processor and a memory, the memory storing computer-readable instructions that, when executed by the processor, perform the method as described in any one of claims 1-6.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it performs the method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-6.

Images