A method, device and system for real-time monitoring of water pollution of a river flowing into the sea

Abstract

The present application relates to the field of image processing, in particular to a water pollution real-time monitoring method, device and system of an estuary river, which is used to solve the technical problem of low water pollution monitoring accuracy caused by the interference of the water surface reflection area of the estuary river. The method comprises: collecting continuous frame images of the water surface of the estuary river; converting the continuous frame images into pulse count maps, identifying the reflection area according to the pulse count maps, and determining the static reflection degree of the reflection area; determining the dynamic reflection degree of the reflection area based on the optical flow information of the continuous frame images; weakening the brightness of each pixel in the reflection area according to the static reflection degree and the dynamic reflection degree; inputting the image after brightness weakening into a target detection model to monitor the water pollution of the estuary river.

Description

Technical Field

[0001] This invention relates to the field of image processing, specifically to a method, equipment, and system for real-time monitoring of water pollution in rivers flowing into the sea. Background Technology

[0002] With the continuous development of industry and urban life along riverbanks, large amounts of sewage are discharged into the ocean through rivers flowing into the sea, leading to increasingly serious pollution problems in estuaries and nearshore waters, posing a severe threat to marine ecosystems. Therefore, it is currently necessary to conduct effective real-time monitoring of the water quality and floating pollutants in rivers flowing into the sea in order to promptly detect pollution situations.

[0003] Currently, real-time monitoring of rivers flowing into the sea can be performed using computer vision. However, in practical applications, under natural light, the water surface forms complex reflective areas of uneven intensity that dynamically change with the water flow. A single color in these reflective areas can be mistaken for garbage pollution, easily leading to false alarms from the monitoring system and severely impacting its accuracy. Therefore, reducing the interference from reflective areas on the water surface of rivers flowing into the sea and improving the accuracy of water pollution monitoring has become an urgent technical problem to be solved. Summary of the Invention

[0004] To address the technical problem of low accuracy in water pollution monitoring caused by interference from reflective areas on the surface of rivers flowing into the sea, this invention aims to provide a real-time water pollution monitoring method for rivers flowing into the sea. The specific technical solution adopted is as follows:

[0005] Collect continuous frame images of the river surface flowing into the sea;

[0006] The continuous frame images are converted into pulse count maps, and reflective areas are identified based on the pulse count maps, and the static reflectivity of the reflective areas is determined.

[0007] Based on the optical flow information of consecutive frame images, the dynamic reflectivity of reflective areas is determined.

[0008] Based on the static and dynamic reflectivity, the brightness of each pixel in the reflective area is reduced;

[0009] The image with reduced brightness is input into the object detection model to monitor water pollution in rivers flowing into the sea.

[0010] In one possible implementation, identifying reflective regions in consecutive frame images includes: clustering pixels in a pulse count map based on brightness features; identifying reflective pixel clusters from the clusters based on the brightness features of each cluster; and performing edge detection and contour extraction based on the positional distribution of pixels in the reflective pixel clusters in the image to determine the reflective regions.

[0011] In one possible implementation, reflective pixel clusters are identified from the clusters based on the brightness statistical characteristics of each cluster, including: calculating the brightness statistical characteristic value of each cluster based on the multi-channel brightness values ​​of all pixels in the pulse count map within each cluster; the brightness statistical characteristic value is used to characterize the overall brightness level of the cluster; determining a feature sequence composed of the brightness statistical characteristic values ​​of all clusters; determining a first threshold based on the feature sequence, the first threshold being the sum of the mean and standard deviation of the elements in the feature sequence; and identifying clusters with brightness statistical characteristic values ​​greater than the first threshold as reflective pixel clusters.

[0012] In one possible implementation, determining the static reflectivity of a reflective region includes: for each reflective region, constructing a feature matrix of the pixels in the reflective region in a pulse count map; decomposing the feature matrix based on a singular value decomposition algorithm, and reconstructing an approximate matrix of the feature matrix based on the first singular value obtained after decomposition and its corresponding singular vector; determining the difference between the feature matrix and the approximate matrix; and determining the static reflectivity of the reflective region based on the difference; wherein the difference is negatively correlated with the static reflectivity.

[0013] In one possible implementation, determining the dynamic reflectivity of a reflective region based on optical flow information from consecutive frame images includes: calculating the motion vector of each pixel based on consecutive frame images; generating a fluctuation intensity map of the consecutive frame images based on the motion vectors; the fluctuation intensity map characterizes the inconsistency of motion vectors in the local region where each pixel is located; and determining the dynamic reflectivity of the reflective region based on the intensity values ​​within the reflective region in the fluctuation intensity map.

[0014] In one possible implementation, generating a volatility intensity map of a series of frames based on motion vectors includes: determining the Gaussian weighted variance of the motion vector magnitudes of all pixels within a sliding window centered on a pixel in the series of frames; determining the coefficient of variation of the motion vector orientation angles of all pixels within the sliding window; normalizing and linearly fusing the Gaussian weighted variance and the coefficient of variation, and using the fusion result as a volatility intensity value of a pixel; traversing all pixels in the series of frames to obtain the volatility intensity values ​​of all pixels; and generating a volatility intensity map of the series of frames based on the volatility intensity values ​​of all pixels.

[0015] In one possible implementation, the brightness of each pixel in the reflective region is reduced based on the static and dynamic reflectivity, including: determining the individual reflectivity of the target pixel, wherein the individual reflectivity is directly proportional to the brightness value of the target pixel, directly proportional to the sum of the static and dynamic reflectivity, and inversely proportional to the sum of the brightness of all pixels in the reflective region; the target pixel is any pixel in a continuous frame image; determining the brightness reduction coefficient of the target pixel based on its individual reflectivity; wherein the brightness reduction coefficient is negatively correlated with the individual reflectivity, and the value of the brightness reduction coefficient is less than or equal to 1; and multiplying the brightness values ​​of the R, G, and B channels of the target pixel by the brightness reduction coefficient to obtain the reduced pixel brightness value of the target pixel.

[0016] In one possible implementation, before inputting the brightness-reduced image into the target detection model to monitor water pollution in rivers flowing into the sea, the method further includes: acquiring multiple brightness-reduced images of the river surface and labeling the brightness-reduced images of the river surface with water pollution; and training the initial model based on the labeled images to obtain the target detection model.

[0017] The present invention also provides a real-time monitoring device for water pollution in rivers flowing into the sea, the device comprising:

[0018] A communication unit is used to acquire continuous frame images of the surface of rivers flowing into the sea;

[0019] The processing unit is configured to convert the continuous frame images into a pulse count map, identify reflective areas based on the pulse count map, and determine the static reflectivity of the reflective areas.

[0020] The processing unit is also used to determine the dynamic reflectivity of the reflective area based on the optical flow information of the continuous frame images;

[0021] The processing unit is further configured to reduce the brightness of each pixel in the reflective area based on the static reflectivity and the dynamic reflectivity.

[0022] The processing unit is also used to input the reduced-brightness image into the target detection model to monitor water pollution in the river flowing into the sea.

[0023] The present invention also provides a real-time monitoring system for water pollution in rivers flowing into the sea, comprising: a data acquisition device and a real-time monitoring device for water pollution in rivers flowing into the sea; the data acquisition device is used to acquire continuous frame images of the water surface of the river flowing into the sea and send the acquired continuous frame images to the real-time monitoring device for water pollution in rivers flowing into the sea; the real-time monitoring device for water pollution in rivers flowing into the sea is used to execute the aforementioned real-time monitoring method for water pollution in rivers flowing into the sea based on the continuous frame images.

[0024] This invention offers the following advantages: By converting images into pulse count maps to identify reflective areas and calculate static reflectivity, combined with dynamic reflectivity calculated based on optical flow information, this invention can not only spatially identify uniformly overexposed reflective areas but also temporally identify dynamically flickering areas. This significantly improves the accuracy of reflective area identification and achieves precise reduction of pixel brightness in reflective areas. Thus, this invention can effectively reduce the interference of water surface reflection on image analysis and improve the accuracy of water pollution monitoring. Attached Figure Description

[0025] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A method flow chart for real-time monitoring of water pollution in rivers flowing into the sea, as provided in one embodiment of the present invention. Figure 1 ;

[0027] Figure 2 A method flow chart for real-time monitoring of water pollution in rivers flowing into the sea, as provided in one embodiment of the present invention. Figure 2 ;

[0028] Figure 3 A method flow chart for real-time monitoring of water pollution in rivers flowing into the sea, provided as an embodiment of the present invention. Figure 3 ;

[0029] Figure 4 A method flow chart for real-time monitoring of water pollution in rivers flowing into the sea, as provided in one embodiment of the present invention. Figure 4 ;

[0030] Figure 5 A method flow chart for real-time monitoring of water pollution in rivers flowing into the sea, as provided in one embodiment of the present invention. Figure 5 ;

[0031] Figure 6 This is a schematic diagram of the structure of a real-time water pollution monitoring device for a river flowing into the sea, provided in one embodiment of the present invention;

[0032] Figure 7 This is a system architecture diagram of a real-time water pollution monitoring system for a river flowing into the sea, provided as an embodiment of the present invention. Detailed Implementation

[0033] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a real-time monitoring method for water pollution in rivers flowing into the sea proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0035] The following describes in detail, with reference to the accompanying drawings, a specific scheme for a real-time monitoring method for water pollution in rivers flowing into the sea provided by the present invention.

[0036] Please see Figure 1 It illustrates a flowchart of a method for real-time monitoring of water pollution in a river flowing into the sea, according to an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0037] Step 101: Collect continuous frame images of the river surface flowing into the sea.

[0038] One possible implementation involves setting up a high-definition pan-tilt unit (PTZ) at the source of the target river flowing into the sea, and acquiring continuous frame images of the river based on the river's flow velocity. After acquiring these continuous frame images, RAW format data from the high-definition PTG sensor is obtained. The faster the river flow, the shorter the acquisition interval. As an example, for a moderate river flow velocity between 0.5 m / s and 1.5 m / s, the acquisition interval in this application is set to 2 seconds.

[0039] It is important to note that the acquisition parameters of the HD pan-tilt unit (PTZ) must remain constant during the acquisition process to avoid the influence of accidental factors. For example, if the acquisition angle or position of the HD pan-tilt unit changes during acquisition, the information differences between adjacent frames in consecutive acquired images will be significant, leading to serious errors in subsequent processing. Therefore, the acquisition angle and position of the HD pan-tilt unit must be kept constant during acquisition. Furthermore, if the angle or position of the HD pan-tilt unit changes due to accidental factors (such as incorrect settings or accidental button presses), the image can be divided into multiple parts through manual selection and processed separately. For example, the image acquired at the point where the HD pan-tilt unit changed position can be used as a segmentation point to divide the overall image into multiple parts, and each part can be processed separately.

[0040] It is understandable that during the acquisition process, the high-definition pan-tilt sensor may be affected by its own internal or surrounding environment (e.g., slight vibrations of the sensor itself, or splashing water in the image due to fast water flow), resulting in some noise in the image. In this case, denoising algorithms can be used to denoise all river surface images to eliminate external interference. For example, the denoising algorithm in this application can be Gaussian filtering, median filtering, wavelet denoising, etc., and this application is not limited to these algorithms.

[0041] Step 102: Convert the continuous frame images into a pulse count map, identify the reflective areas based on the pulse count map, and determine the static reflectivity of the reflective areas.

[0042] In one possible implementation, this application simulates the pulse firing mechanism of neurons to perform nonlinear response processing on the original brightness values ​​in the image, which are linearly related to light intensity, generating a pulse count map with a wider dynamic range, thereby effectively distinguishing reflective areas that are saturated due to overexposure. Subsequently, the image is divided into multiple types of regions based on the brightness characteristics of different areas using methods such as image segmentation or pixel clustering, and regions that meet the reflective characteristics are selected as reflective areas from these multiple types of regions.

[0043] Optionally, this application can determine the degree of static reflectivity by quantifying the uniformity or consistency of the brightness values ​​within the region, wherein the more uniform and consistent the brightness within the region, the stronger the degree of static reflectivity is determined.

[0044] It should be noted that when sunlight shines on the surface of a river, the water reflects the sunlight, creating reflective areas on the water surface. At the same time, because the angle of incidence of the sun is the same, the degree of reflection produced by the water surface closer to the sunlight is different from that of the water surface relatively far away. At the same time, the fixed position of the high-definition pan-tilt unit results in uneven reflective areas in the collected images of the river surface.

[0045] The greater the intensity of sunlight, the greater the reflection from the river surface. Consequently, the captured river surface image will be overexposed in reflective areas, resulting in pixels with high brightness values ​​in these areas. Since RGB values ​​in an RGB image are derived through linear interpolation, with each channel's brightness ranging from 0 to 255, the RGB values ​​in the reflective areas will become saturated (displaying a maximum value of 255), making the colors in these areas monotonous and severely damaging image details (e.g., color and texture). Therefore, this application uses static reflectivity to represent the reflectivity of an image, providing a basis for subsequent weakening of reflective areas.

[0046] It should be noted that the river surface images acquired in this application are all based on RAW format data output by high-definition pan-tilt sensors (bit depth is typically 12-bit, 14-bit, etc., with a dynamic range far exceeding that of standard 8-bit images). This RAW data did not reach the sensor's hardware saturation threshold and still fully preserved the brightness gradient information of the reflective areas. The aforementioned RGB value saturation to 255 is due to software-level information truncation caused by dynamic range compression when the RAW data is converted to a standard 8-bit RGB image (dynamic range limited to 0-255), rather than the loss of light intensity signals at the sensor hardware level. This ensures that subsequent algorithms, such as pulse counting to expand the dynamic range and calculating static reflectivity, can uncover the untrunculated highlight details in the RAW data, providing a physical basis for precise weakening of reflective areas.

[0047] Step 103: Determine the dynamic reflectivity of the reflective area based on the optical flow information of consecutive frame images.

[0048] Optionally, this application analyzes motion information between consecutive frames of images to quantify the temporal fluctuation characteristics of reflective regions, thereby distinguishing optical reflections from physical contaminants from a dynamic perspective. In other words, this application first calculates optical flow information based on consecutive frames of images to obtain the motion vector of each pixel, which includes motion magnitude and direction information. Subsequently, a fluctuation intensity map is generated based on the motion vector, which characterizes the degree of disorder in the motion of each local region in the image. Finally, based on the fluctuation intensity map and the identified reflective regions, the dynamic reflectivity of each reflective region is determined through aggregation calculation, wherein the fluctuation intensity and the dynamic reflectivity are negatively correlated.

[0049] It should be noted that in the reflective area of ​​a river surface, the reflective wavefront possesses strong edge and brightness information. In consecutive frame images, the river is constantly flowing, and its random undulations cause the reflective wavefront to continuously change. This can lead to reflective areas being misidentified as dynamic targets (e.g., household waste moving with the river's flow). This means that in adjacent frame images, the reflective area of ​​the water surface exhibits local fluctuations. Because the higher brightness values ​​of pixels in the reflective area may weaken the fluctuations of the river surface, the intensity of the fluctuations in the reflective area is inversely proportional to the degree of reflectivity. Therefore, this application can identify dynamic reflective areas based on the dynamic degree of reflectivity, providing a basis for subsequent weakening of reflective areas.

[0050] Step 104: Reduce the brightness of each pixel in the reflective area based on the static and dynamic reflectivity.

[0051] Optionally, this application generates pixel-level brightness reduction coefficients by fusing static and dynamic reflectivity evaluation results, achieving targeted local brightness adjustment to effectively suppress reflectivity interference while preserving image details. In other words, firstly, the static and dynamic reflectivity of the reflective area are fused to obtain a comprehensive reflectivity characterization of the area. Subsequently, based on the comprehensive reflectivity and the relative brightness contribution of each pixel within the reflective area, the individual reflectivity of each pixel is determined. Finally, a corresponding brightness reduction coefficient is generated based on the individual reflectivity, and this coefficient is applied to adjust the brightness values ​​of each color channel of the pixel, completing the refined brightness reduction processing of the reflective area.

[0052] Step 105: Input the image with reduced brightness into the target detection model to monitor water pollution in rivers flowing into the sea.

[0053] As one possible implementation, the target detection model takes as input data the river surface image data after brightness reduction, and outputs as the pollution type and location of the pollution area on the river surface.

[0054] Optionally, the target detection model in this application is a pre-trained model for detecting river water pollution. The model training process includes: acquiring multiple images of the river surface with reduced brightness, and labeling these images with water pollution information; training the initial model based on the labeled images to obtain the target detection model. Specifically, historical images of the river surface are acquired, and the brightness of all river surface images is reduced according to the scheme of this application. The LabelMe annotation tool is used to label all the reduced-brightness river surface images, and the labeled targets are various types of water pollution, such as oil pollution and domestic waste. All reduced-brightness river surface images and labels are used as input, the optimizer is set to Adam or SGD, the loss function is set to CIoU or DIoU, the number of iterations is set to 600, and the YOLOv8 algorithm is used to train all the reduced-brightness river surface images to obtain the target detection model.

[0055] For the image to be detected, the brightness of the image to be detected is reduced using the method of this application. The trained target detection model is used to detect the target in the image to be detected. If the target is detected, it means that there is water pollution in the image to be detected. If the target is not detected, it means that there is no water pollution in the image to be detected.

[0056] Based on the above technical solution, this application identifies reflective areas and calculates static reflectivity by converting the image into a pulse count map. Combined with dynamic reflectivity calculated based on optical flow information, it can not only spatially identify uniformly overexposed reflective areas but also temporally identify dynamically flickering areas, thereby significantly improving the accuracy of reflective area identification and achieving precise reduction of pixel brightness in reflective areas. Thus, this application can effectively reduce the interference of water surface reflection on image analysis and improve the accuracy of water pollution monitoring.

[0057] As one possible implementation, the conversion of consecutive frame images into pulse counting maps in step 102 above can be achieved as follows: For the t-th frame image, read the original RGB values ​​that are linearly related to light intensity from the RAW format data of the high-definition pan-tilt sensor. Use the min-max normalization algorithm to normalize the R, G, and B values ​​of each color channel. Set the time step parameter of the integral emission model to 0.1ms (range [0.1ms, 1.0ms]) and the simulation duration parameter to 500ms (range [100ms, 1000ms]), with the remaining parameters using default values. Initialize three pulse counting matrices and a membrane potential matrix with the same size as the image, all of which are all zero matrices.

[0058] Normalized RGB values ​​for each channel are used as input, and an integral emission model is applied to process each pixel and each color channel. This model simulates the membrane potential integration and pulse firing mechanism of biological neurons, performing a non-linear transformation on the linear RGB values ​​to generate the corresponding pulse sequence. The total number of pulses fired by each pixel in each channel is recorded in the corresponding position of the pulse counting matrix, ultimately forming the pulse count map of frame t.

[0059] Each value in the pulse count map represents the total number of pulses fired by the corresponding pixel in the corresponding color channel. In the integral emission model, the pulse firing rate (i.e., the ratio of the total number of pulses fired to the simulation duration) is proportional to the normalized RGB value of the corresponding channel. It should be noted that the range of RGB values ​​in traditional images is limited to 0-255, making reflective areas prone to saturation distortion. The pulse count map, however, has a wider numerical range, effectively avoiding saturation and accurately distinguishing brightness differences within reflective areas. Furthermore, the non-linear enhancement of linear RGB values ​​by the integral emission model improves the system's robustness to noise.

[0060] like Figure 2 As shown, in one possible implementation, step 102 above, which identifies reflective areas in consecutive frame images, can be achieved through the following process:

[0061] Step 201: Cluster the pixels in the pulse count map based on brightness features.

[0062] As one possible implementation, the clustering process can be as follows: The total number of pulses fired at corresponding positions in the three pulse counting images of the t-th frame are concatenated to form a series of triplet data, and then fused to generate a comprehensive pulse counting image A. The number of clusters is set to 3, and the K-means algorithm, using Euclidean distance as the metric, is used to perform cluster analysis on all triples in pulse counting image A.

[0063] Step 202: Identify reflective pixel clusters from the clusters based on the brightness characteristics of each cluster.

[0064] As one possible implementation, the process can be as follows: based on the multi-channel brightness values ​​of all pixels within each cluster in the pulse count map, calculate the brightness statistical feature value of each cluster; the brightness statistical feature value is used to characterize the overall brightness level of the cluster; determine a feature sequence composed of the brightness statistical feature values ​​of all clusters; determine a first threshold based on the feature sequence, the first threshold being the sum of the mean and standard deviation of the elements in the feature sequence; and identify clusters with brightness statistical feature values ​​greater than the first threshold as reflective pixel clusters.

[0065] For example, taking the i-th cluster as an example, calculate the mean of all triplet data within this cluster, and then calculate the proportionally weighted sum, denoted as . The weighted sum of all clusters forms a sequence Z, and the mean μ and standard deviation σ of this sequence are calculated. Based on the statistical characteristics of sequence Z, if... If μ ≥ σ, the cluster is determined to consist of pixels in the reflective region, and all pixels within it are marked as 0; if If the value is ≤μ-σ, the cluster is determined to consist of pixels from the shaded area; otherwise, it is determined to be pixels from the normal water surface area. Based on this, reflective clusters are identified from the clusters.

[0066] Step 203: Based on the positional distribution of pixels in the reflective pixel cluster in the image, perform edge detection and contour extraction to determine the reflective area.

[0067] As one possible implementation, in this step, based on the pixel-level correspondence between the pulse counting map A and the t-th frame image, all pixels marked as 0 in the map are identified as candidate reflective pixels. The original image is then binarized based on this marking result, with pixels marked as 0 designated as foreground and the remaining pixels as background. Edge detection is performed on the resulting binary map using a pre-defined image processing library, identifying potential boundaries by analyzing pixel gradient changes. Subsequently, a contour extraction algorithm is applied to identify closed regions formed by continuous edge point sets as effective reflective regions, ultimately achieving precise localization of the reflective regions in the t-th frame image. Based on the above three-step operation of binarization, edge detection, and contour extraction, a transformation from pixel-level marking to region-level recognition can be achieved, providing accurate spatial definition for subsequent reflectivity analysis.

[0068] Based on the above technical solution, this application identifies reflective pixel clusters by combining cluster analysis with adaptive threshold judgment, and further determines the precise boundary of the reflective area through edge detection. This can effectively distinguish the reflective area from the normal water surface and shadow area, avoid misjudgment caused by relying solely on a single brightness threshold, improve the automation and spatial accuracy of reflective area identification, and provide a precise target area for subsequent reflection reduction processing.

[0069] like Figure 3 As shown, in one possible implementation, the process of determining the static reflectivity of the reflective area in step 102 above can be specifically achieved through the following steps:

[0070] Step 301: For each reflective area, construct the feature matrix of the pixels in the reflective area in the pulse count map.

[0071] As one possible implementation, taking the i-th cluster as an example, the process can be implemented as follows: determine the magnitude of the weighted sum of the means of each triplet, and map each triplet into a 3×N-dimensional feature matrix in ascending order to obtain the feature matrix. Where N is the total number of triples in the cluster.

[0072] Step 302: Decompose the feature matrix based on the singular value decomposition algorithm, and reconstruct an approximate matrix of the feature matrix based on the first singular value obtained after decomposition and its corresponding singular vector.

[0073] One possible implementation is as follows: Perform singular value decomposition on the matrix to obtain its singular value matrix and corresponding left and right singular matrices. Select the first singular value and its corresponding left and right singular vectors, and construct the fundamental matrix through outer product operations. The fundamental matrix Multiplying by the first singular value yields an approximate matrix.

[0074] Step 303: Determine the degree of difference between the characteristic matrix and the approximate matrix.

[0075] As one possible implementation, this application can use matrix difference measures such as the Frobenius norm to calculate the original matrix. With approximate matrix The degree of difference between them.

[0076] As an example, a matrix With approximate matrix The degree of difference between them satisfies the following formula:

[0077]

[0078] in, Indicates to proceed Norm calculation Representation of the characteristic matrix With approximate matrix Perform matrix subtraction. For parameter tuning coefficients, if If it is 0, then set it to 0.01. If it is not 0, then set it to 0.

[0079] Step 304: Based on the degree of difference, determine the static reflectivity of the reflective area.

[0080] Among them, the degree of difference is negatively correlated with the degree of static reflectivity.

[0081] As one possible implementation, this application can utilize a matrix difference algorithm, for example: Algorithms such as norm and spectral norm are used to calculate matrices. With approximate matrix The difference between them is then used to calculate the static reflectivity of the i-th reflective region in the t-th frame image.

[0082] As an example, the static reflectivity of the i-th reflective region in the t-th frame of the image satisfies the following formula:

[0083]

[0084] It should be pointed out that, The larger the value, the more characteristic the matrix. With approximate matrix The higher the similarity between them. The smaller the value, the stronger the static reflectivity of the i-th reflective area. This is a data normalization function used to scale data to a specific range (e.g., [0, 1]) according to rules, unifying the data scale. For example... It is a function for normalizing the maximum and minimum values.

[0085] Based on the above technical solution, this application quantifies the degree of static reflectivity by constructing a feature matrix and calculating its difference from the approximate matrix. Based on this, this application can accurately reflect the overexposure degree and texture feature loss of the reflective area, and conduct a refined evaluation of different degrees of reflectivity, providing an accurate quantitative basis for subsequent brightness reduction.

[0086] like Figure 4 As shown, in one possible implementation, the process of determining the dynamic reflectivity of the reflective area based on the optical flow information of consecutive frame images in step 103 above can be specifically implemented through the following steps:

[0087] Step 401: Calculate the motion vector of each pixel based on consecutive frame images.

[0088] As one possible implementation, in this step, the images of frame t and frame (t-1) can be used as input, and the motion vector of each pixel in the image of frame t can be calculated using an optical flow algorithm. Optionally, optical flow algorithms include, but are not limited to, the Brox Flow algorithm and the Lucas-Kanade algorithm, which will not be elaborated upon in this application.

[0089] Optionally, in determining the motion vector of a pixel Next, determine the pixel's modulus. Optical flow vector direction angle .

[0090] Step 402: Generate a wave intensity map of consecutive frame images based on the motion vectors.

[0091] Among them, the volatility intensity map represents the inconsistency of motion vectors in the local region where each pixel is located.

[0092] As one possible implementation, this step can be specifically implemented as follows: determine the Gaussian weighted variance of the motion vector magnitudes of all pixels within a sliding window centered on a pixel in the continuous frame image; determine the coefficient of variation of the motion vector orientation angles of all pixels within the sliding window; normalize and linearly fuse the Gaussian weighted variance and the coefficient of variation, and use the fusion result as a fluctuation intensity value of a pixel; traverse all pixels in the continuous frame image to obtain the fluctuation intensity values ​​of all pixels; and generate a fluctuation intensity map of the continuous frame image based on the fluctuation intensity values ​​of all pixels.

[0093] For example, a sliding window with a step size of 1 and a size of n×n is set (n can be adjusted according to the actual situation; in this embodiment, n=3). In the t-th frame image, each pixel is used as the center point, and the window slides from left to right and from top to bottom. Using a two-dimensional Gaussian kernel function, the Gaussian weight of each pixel within the j-th sliding window in the t-th frame image is calculated. The Gaussian weight of each pixel is normalized using the min-max algorithm, and the Gaussian kernel composed of the normalized Gaussian weights of each pixel within the j-th sliding window is obtained. The Gaussian weighted variance of the motion vector magnitude within the j-th sliding window is calculated, denoted as . The Gaussian weighted variance is obtained by calculating the variance of the normal magnitude variance, taking the product of the magnitude of each pixel and its normalized Gaussian weight, and then calculating the variance.

[0094] Calculate the mean and standard deviation of the optical flow vector direction angles of all pixels within the j-th sliding window. Then, use the quotient of the standard deviation and the mean to calculate the coefficient of variation of the optical flow vector direction angles of all pixels within the j-th sliding window, denoted as [formula missing]. , The closer the value is to 0, the more consistent the local orientation of the j-th sliding window. The min-max algorithm is then used to... and Normalization and linear fusion are performed to obtain the fluctuation intensity value of the center pixel of the j-th sliding window. The wave intensity value of each pixel in the t-th frame image can be calculated. The wave intensity value of each pixel in the t-th frame image is obtained and converted into a wave intensity map.

[0095] As an example, the fluctuation intensity value of the center pixel of the j-th sliding window Satisfy the following formula:

[0096]

[0097] in, Indicates the weighting coefficient. The value of can be set by those skilled in the art based on experience; for example, The value is 0.5.

[0098] Step 403: Determine the dynamic reflectivity of the reflective area based on the intensity value within the reflective area in the fluctuation intensity diagram.

[0099] As one possible implementation, this step can be implemented as follows: using the Minkowski norm to aggregate the fluctuation intensity map converted from the image of frame t and all reflective regions in the image of frame t to obtain an aggregation result, and obtaining the dynamic reflectivity based on the aggregation result.

[0100] Optionally, the Minkowski norm order in this application ranges from [2, 6], and the specific value needs to be adjusted based on the area of ​​the reflective region. When the reflective region area is > 500 pixels, the order is 4-6 to enhance the region aggregation accuracy; when the reflective region area is < 100 pixels, the order is 2-3 to reduce calculation errors; in this embodiment, the reflective region is of medium size (100-500 pixels), so the exemplary value is 4.

[0101] Taking the i-th reflective region as an example, the normalized value of its aggregation result is Then, the dynamic reflectivity of the i-th reflective region is calculated based on the aggregation results. .

[0102] Based on the above technical solution, this application evaluates the degree of dynamic reflectivity by calculating pixel-level motion vectors and generating a wave intensity map, capturing the motion characteristics of the reflective area in continuous frames. In this way, this application can effectively utilize the difference in motion patterns between the reflective area and the real pollutants to identify false moving targets caused by reflection, further improving the accuracy of pollution identification in dynamic scenes.

[0103] like Figure 5 As shown, in one possible implementation, the process of reducing the brightness of each pixel in the reflective area based on the static and dynamic reflectivity in step 104 above can be achieved through the following steps:

[0104] Step 501: Determine the individual reflectivity of the target pixel.

[0105] Among them, the individual reflectivity is directly proportional to the brightness value of the target pixel, directly proportional to the sum of static reflectivity and dynamic reflectivity, and inversely proportional to the sum of the brightness of all pixels in the reflective area; the target pixel is any pixel in a continuous frame image.

[0106] As an example, the static and dynamic reflectivity of the i-th reflective region is diffused to each pixel to obtain the reflectivity of the x-th pixel. Satisfy the following formula:

[0107]

[0108] in, These represent the normalized R, G, and B values ​​of the x-th pixel, respectively. Let R, G, and B represent the mean normalized R, G, and B values ​​of all pixels within the i-th reflective region, respectively. This represents the static reflectivity of the i-th reflective region. This represents the dynamic reflectivity of the i-th reflective region. This represents the min-max normalization algorithm. For parameter tuning coefficients, if If it is 0, then set it to 0.01. If it is not 0, then set it to 0.

[0109] Step 502: Determine the brightness reduction coefficient of the target pixel based on the individual reflectivity of the target pixel.

[0110] Among them, the brightness attenuation coefficient is negatively correlated with the individual reflectivity, and the value of the brightness attenuation coefficient is less than or equal to 1.

[0111] As an example, the brightness attenuation factor is: (1- ).

[0112] Step 503: Multiply the brightness values ​​of the R, G, and B channels of the target pixel by the brightness reduction coefficient to obtain the reduced pixel brightness value of the target pixel.

[0113] As one possible implementation, this step can be achieved by: dividing the RGB value of the x-th pixel by (1- The product of these factors reduces the brightness of the x-th pixel in the reflective region. Similarly, the reflectivity of each pixel in all reflective regions of the t-th frame is calculated, and the brightness of each pixel is reduced based on this reflectivity. It should be noted that since dynamic reflectivity cannot be calculated for the first frame, only static reflectivity is used when constructing the reflectivity map.

[0114] Based on the above technical solution, this application determines the degree of individual reflectivity by comprehensively considering the pixel's own brightness, the degree of regional reflectivity, and the sum of regional brightness, thus achieving pixel-level fine-grained brightness adjustment. This proportional reduction method can effectively suppress reflectivity while preserving image details, avoiding the image information loss caused by suppressing reflective areas based on a single dimension, and maintaining the visual naturalness and usability of the reduced image.

[0115] This invention also provides a structural schematic diagram of a real-time water pollution monitoring device for rivers flowing into the sea, as shown below. Figure 6 As shown, the device includes:

[0116] The communication unit 601 is used to acquire continuous frame images of the surface of the river flowing into the sea.

[0117] The processing unit 602 is used to convert continuous frame images into pulse count maps, identify reflective areas based on the pulse count maps, and determine the static reflectivity of the reflective areas.

[0118] The processing unit 602 is also used to determine the dynamic reflectivity of the reflective area based on the optical flow information of the continuous frame images.

[0119] The processing unit 602 is also used to reduce the brightness of each pixel in the reflective area based on the static reflectivity and the dynamic reflectivity.

[0120] The processing unit 602 is also used to input the brightness-reduced image into the target detection model to monitor water pollution in rivers flowing into the sea.

[0121] As one possible implementation, the processing unit 602 is specifically used to: cluster the pixels in the pulse counting image based on brightness features; identify reflective pixel clusters from the clusters based on the brightness features of each cluster; and perform edge detection and contour extraction based on the positional distribution of pixels in the reflective pixel clusters in the image to determine the reflective area.

[0122] As one possible implementation, the processing unit 602 is specifically used to: calculate the brightness statistical feature value of each cluster based on the multi-channel brightness values ​​of all pixels in the pulse count map within each cluster; the brightness statistical feature value is used to characterize the overall brightness level of the cluster; determine a feature sequence composed of the brightness statistical feature values ​​of all clusters; determine a first threshold based on the feature sequence, the first threshold being the sum of the mean and standard deviation of the elements in the feature sequence; and designate clusters with brightness statistical feature values ​​greater than the first threshold as reflective pixel clusters.

[0123] As one possible implementation, the processing unit 602 is specifically used to: for each reflective region, construct a feature matrix of the pixels in the reflective region in the pulse counting map; decompose the feature matrix based on the singular value decomposition algorithm, and reconstruct an approximate matrix of the feature matrix based on the first singular value obtained after decomposition and its corresponding singular vector; determine the degree of difference between the feature matrix and the approximate matrix; and determine the static reflectivity of the reflective region based on the degree of difference; wherein the degree of difference is negatively correlated with the static reflectivity.

[0124] As one possible implementation, the processing unit 602 is specifically used for: calculating the motion vector of each pixel based on the continuous frame images; generating a wave intensity map of the continuous frame images based on the motion vectors; the wave intensity map characterizes the inconsistency of the motion vectors in the local region where each pixel is located; and determining the dynamic reflectivity of the reflective region based on the intensity value in the reflective region in the wave intensity map.

[0125] As one possible implementation, the processing unit 602 is specifically used to: determine the Gaussian weighted variance of the motion vector magnitudes of all pixels within a sliding window centered on a pixel in a continuous frame image; determine the coefficient of variation of the motion vector direction angles of all pixels within the sliding window; normalize and linearly fuse the Gaussian weighted variance and the coefficient of variation, and use the fusion result as a fluctuation intensity value of a pixel; traverse all pixels in the continuous frame image to obtain the fluctuation intensity values ​​of all pixels; and generate a fluctuation intensity map of the continuous frame image based on the fluctuation intensity values ​​of all pixels.

[0126] As one possible implementation, the processing unit 602 is specifically used to: determine the individual reflectivity of the target pixel, wherein the individual reflectivity is directly proportional to the brightness value of the target pixel, directly proportional to the sum of the static reflectivity and the dynamic reflectivity, and inversely proportional to the sum of the brightness of all pixels in the reflective area; the target pixel is any pixel in a continuous frame image; based on the individual reflectivity of the target pixel, determine the brightness reduction coefficient of the target pixel; wherein the brightness reduction coefficient is negatively correlated with the individual reflectivity, and the value of the brightness reduction coefficient is less than or equal to 1; multiply the brightness values ​​of the R, G, and B channels of the target pixel by the brightness reduction coefficient respectively to obtain the reduced pixel brightness value of the target pixel.

[0127] As one possible implementation, the processing unit 602 is specifically used to: acquire multiple images of the river surface after brightness reduction, and perform water pollution labeling processing on the images of the river surface after brightness reduction; and train the initial model based on the labeled images to obtain the target detection model.

[0128] This invention also provides a system architecture diagram for a real-time monitoring system of water pollution in rivers flowing into the sea, such as... Figure 7 As shown, the system includes: a data acquisition device 701 and a real-time monitoring device for water pollution of rivers flowing into the sea 702; the data acquisition device 701 is used to acquire continuous frame images of the water surface of the rivers flowing into the sea and send the acquired continuous frame images to the real-time monitoring device for water pollution of rivers flowing into the sea 702; the real-time monitoring device for water pollution of rivers flowing into the sea 702 is used to execute the real-time monitoring method for water pollution of rivers flowing into the sea as described in the foregoing embodiments based on the continuous frame images.

[0129] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0130] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for real-time monitoring of water pollution in rivers flowing into the sea, characterized in that, The method includes: Collect continuous frame images of the water surface of rivers flowing into the sea; The continuous frame images are converted into pulse count maps, and reflective areas are identified based on the pulse count maps, and the static reflectivity of the reflective areas is determined. Based on the optical flow information of the consecutive frame images, the dynamic reflectivity of the reflective area is determined; Based on the static and dynamic reflectivity, the brightness of each pixel in the reflective area is reduced. The image with reduced brightness is input into the target detection model to monitor water pollution in the river flowing into the sea; The process of identifying reflective regions in the consecutive frame images includes: The pixels in the pulse counting map are clustered based on brightness features; Based on the brightness characteristics of each cluster, reflective pixel clusters are identified from the clusters; Edge detection and contour extraction are performed based on the positional distribution of pixels in the reflective pixel cluster in the image to determine the reflective region; Determining the static reflectivity of the reflective area includes: For each reflective region, construct the feature matrix of the pixels in the reflective region in the pulse count map; The feature matrix is ​​decomposed based on the singular value decomposition algorithm, and an approximate matrix of the feature matrix is ​​reconstructed based on the first singular value obtained after decomposition and its corresponding singular vector. Determine the degree of difference between the feature matrix and the approximate matrix; Based on the difference, the static reflectivity of the reflective area is determined; wherein the difference is negatively correlated with the static reflectivity. The determination of the dynamic reflectivity of the reflective region based on the optical flow information of the consecutive frame images includes: Based on the consecutive frame images, calculate the motion vector of each pixel; Based on the motion vectors, a wave intensity map of the consecutive frame images is generated; the wave intensity map represents the inconsistency of motion vectors in the local region where each pixel is located; The dynamic reflectivity of the reflective region is determined based on the intensity value within the reflective region in the fluctuation intensity diagram. Generating the wave intensity map of the consecutive frame images based on the motion vector includes: Determine the Gaussian-weighted variance of the motion vector magnitudes of all pixels within a sliding window centered on a pixel in the consecutive frame images; Determine the coefficient of variation of the motion vector direction angle of all pixels within the sliding window; The Gaussian weighted variance and the coefficient of variation are normalized and linearly fused, and the fusion result is used as the fluctuation intensity value of a pixel. Traverse all pixels in the consecutive frame images to obtain the fluctuation intensity value of all pixels; Based on the fluctuation intensity values ​​of all the pixels, a fluctuation intensity map of the consecutive frame images is generated.

2. The method for real-time monitoring of water pollution in rivers flowing into the sea according to claim 1, characterized in that, Based on the brightness characteristics of each cluster, reflective pixel clusters are identified from the clusters, including: Based on the multi-channel brightness values ​​of all pixels within each cluster in the pulse count map, a brightness statistical feature value for each cluster is calculated; the brightness statistical feature value is used to characterize the overall brightness level of the cluster. Determine the feature sequence consisting of the brightness statistical characteristic values ​​of all clusters; A first threshold is determined based on the feature sequence, wherein the first threshold is the sum of the mean and standard deviation of the elements in the feature sequence; Clusters whose brightness statistical feature values ​​are greater than the first threshold are designated as the reflective pixel clusters.

3. The method for real-time monitoring of water pollution in rivers flowing into the sea according to claim 1, characterized in that, Based on the static and dynamic reflectivity, the brightness of each pixel in the reflective area is reduced, including: The individual reflectivity of a target pixel is determined, wherein the individual reflectivity is directly proportional to the brightness value of the target pixel, directly proportional to the sum of the static reflectivity and the dynamic reflectivity, and inversely proportional to the sum of the brightness of all pixels within the reflective area; the target pixel is any pixel in the consecutive frame images. Based on the individual reflectivity of the target pixel, a brightness reduction coefficient for the target pixel is determined; wherein the brightness reduction coefficient is negatively correlated with the individual reflectivity, and the value of the brightness reduction coefficient is less than or equal to 1; The brightness values ​​of the R, G, and B channels of the target pixel are multiplied by the brightness reduction coefficient to obtain the reduced pixel brightness value of the target pixel.

4. The method for real-time monitoring of water pollution in rivers flowing into the sea according to claim 1, characterized in that, Before inputting the brightness-reduced image into the target detection model to monitor water pollution in the river flowing into the sea, the method further includes: Multiple images of the river surface with reduced brightness were acquired, and water pollution annotation was performed on the images of the river surface with reduced brightness. The initial model is trained based on the labeled images to obtain the target detection model.

5. A real-time monitoring device for water pollution in rivers flowing into the sea, characterized in that, The device includes: A communication unit is used to acquire continuous frame images of the surface of rivers flowing into the sea; The processing unit is configured to convert the continuous frame images into a pulse count map, identify reflective areas based on the pulse count map, and determine the static reflectivity of the reflective areas. The processing unit is also used to determine the dynamic reflectivity of the reflective area based on the optical flow information of the continuous frame images; The processing unit is further configured to reduce the brightness of each pixel in the reflective area based on the static reflectivity and the dynamic reflectivity. The processing unit is also used to input the image with reduced brightness into the target detection model to monitor water pollution in the river flowing into the sea; Specifically, the processing unit is used for: The pixels in the pulse counting map are clustered based on brightness features; Based on the brightness characteristics of each cluster, reflective pixel clusters are identified from the clusters; Edge detection and contour extraction are performed based on the positional distribution of pixels in the reflective pixel cluster in the image to determine the reflective region; The processing unit is further configured to: For each reflective region, construct the feature matrix of the pixels in the reflective region in the pulse count map; The feature matrix is ​​decomposed based on the singular value decomposition algorithm, and an approximate matrix of the feature matrix is ​​reconstructed based on the first singular value obtained after decomposition and its corresponding singular vector. Determine the degree of difference between the feature matrix and the approximate matrix; Based on the difference, the static reflectivity of the reflective area is determined; wherein the difference is negatively correlated with the static reflectivity. The processing unit is further configured to: Based on the consecutive frame images, calculate the motion vector of each pixel; Based on the motion vectors, a wave intensity map of the consecutive frame images is generated; the wave intensity map represents the inconsistency of motion vectors in the local region where each pixel is located; The dynamic reflectivity of the reflective region is determined based on the intensity value within the reflective region in the fluctuation intensity diagram. The processing unit is further configured to: Determine the Gaussian-weighted variance of the motion vector magnitudes of all pixels within a sliding window centered on a pixel in the consecutive frame images; Determine the coefficient of variation of the motion vector direction angle of all pixels within the sliding window; The Gaussian weighted variance and the coefficient of variation are normalized and linearly fused, and the fusion result is used as the fluctuation intensity value of a pixel. Traverse all pixels in the consecutive frame images to obtain the fluctuation intensity value of all pixels; Based on the fluctuation intensity values ​​of all the pixels, a fluctuation intensity map of the consecutive frame images is generated.

6. A real-time water pollution monitoring system for rivers flowing into the sea, characterized in that, The system includes: a data acquisition device and a real-time monitoring device for water pollution in rivers flowing into the sea; the data acquisition device is used to acquire continuous frame images of the water surface of the rivers flowing into the sea and send the acquired continuous frame images to the real-time monitoring device for water pollution in rivers flowing into the sea; the real-time monitoring device for water pollution in rivers flowing into the sea is used to execute the real-time monitoring method for water pollution in rivers flowing into the sea as described in any one of claims 1-4 based on the continuous frame images.

Images