Anti-glare water surface floating object detection method based on deep learning and image enhancement

Abstract

This invention discloses an anti-glare floating object detection method based on deep learning and image enhancement. To address the issues of strong reflections and local overexposure caused by direct sunlight in water surface monitoring scenarios such as rivers and reservoirs, leading to reduced contrast and missed detections of floating objects, as well as the problems of numerous false targets and unstable detection results caused by dynamic backgrounds such as water ripples, waves, and currents, this invention decomposes water surface image sequences into specular and diffuse reflection components to generate glare and overexposure masks. A diffusion repair network is used to repair and reconstruct overexposed areas of the diffuse reflection component. During instance segmentation, multi-scale features are weighted and suppressed based on the glare mask to reduce glare interference. Furthermore, a temporal mask propagation network is employed for cross-frame propagation and consistency correction, outputting the floating object instance outline and confidence level. This achieves improved detection accuracy, reduced false detections, and continuous and stable results for floating objects under strong glare and dynamic backgrounds.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing, and in particular to an anti-glare method for detecting floating objects on water surfaces based on deep learning and image enhancement. Background Technology

[0002] Floating object detection is a crucial technical aspect of river management, lake and reservoir cleaning, and water environment monitoring. It typically relies on shore-based monitoring cameras or unattended video equipment to continuously image the water surface, and then uses image processing or intelligent recognition algorithms to automatically detect and alert on floating objects. Early solutions employed traditional image processing methods such as threshold segmentation, edge detection, morphological processing, background modeling, and inter-frame differencing to highlight floating object areas and suppress the background. With the development of deep learning, convolutional neural network-based target detection and instance segmentation methods are widely used for floating object recognition, improving small target detection capabilities through multi-scale feature extraction. Simultaneously, to improve imaging quality under complex lighting conditions, some solutions introduce preprocessing techniques such as image enhancement, dehazing, and brightness correction, and combine these with temporal methods such as optical flow estimation, target tracking, or video segmentation to enhance the continuity and stability of detection.

[0003] However, in open water scenarios such as rivers and reservoirs, existing technologies still have the following shortcomings: First, strong reflections from direct sunlight and local overexposure can lead to a loss of target texture information and reduced contrast, making it easy for traditional segmentation methods and deep learning models to miss or falsely detect objects. Second, dynamic backgrounds caused by water ripples, waves, and currents generate a large number of pseudo-targets that resemble floating objects, causing the detection results to fluctuate between adjacent frames and making it difficult to output stable and continuous results. Third, conventional image enhancement methods usually lack precise localization and targeted repair of glare and overexposed areas, and the enhancement results are prone to introducing pseudo-textures or noise, which further affects the subsequent detection and temporal consistency processing effects.

[0004] Therefore, there is a need for an anti-glare method for detecting floating objects on the water surface that can overcome the shortcomings of the existing technologies. Summary of the Invention

[0005] One objective of this invention is to propose an anti-glare floating object detection method based on deep learning and image enhancement. Addressing the problems of existing technologies in water surface monitoring scenarios, such as the susceptibility to strong reflections from direct sunlight and local overexposure leading to reduced contrast and missed detections, as well as the issues of numerous false targets and unstable detection results across frames caused by dynamic backgrounds from water ripples, waves, and currents, the following technical solution is proposed: Acquire a sequence of water surface images arranged chronologically; decompose the current initial image frame into specular and diffuse reflection components to generate a glare mask for representing strongly reflective areas and an overexposure mask for representing locally overexposed areas; use a diffusion repair network to repair the overexposure of the diffuse reflection component image using the overexposure mask to obtain a repaired diffuse reflection image; perform floating object instance segmentation on the repaired diffuse reflection image, and apply weighted suppression to the network features based on the glare mask to reduce glare interference; further utilize a temporal mask propagation network to propagate the temporally consistent instance mask from the previous frame across frames and establish a correspondence, fusion, and consistency screening with the current frame segmentation result, outputting a temporally consistent floating object detection result. This invention has the technical effect of improving the accuracy of floating object detection, reducing false detections, and enhancing continuous and stable output across frames under strong glare and dynamic background conditions.

[0006] This invention provides an anti-glare water surface floating object detection method based on deep learning and image enhancement, comprising:

[0007] S1. Acquire the initial image sequence collected by the water surface monitoring equipment in chronological order, containing multiple initial image frames, and determine the initial image frame to be processed as the current initial image frame; S2. Perform reflectance decomposition processing on the current initial image frame, so that the reflectance decomposition network outputs the specular reflection component map and diffuse reflection component map corresponding to the current initial image frame. Generate a glare mask to characterize the strong reflective area based on the specular reflection component map, and generate an overexposure mask to characterize the local overexposure area based on the current initial image frame; S3. Perform overexposure repair processing on the diffuse reflection component map based on the overexposure mask, so that the diffusion repair network outputs the repaired diffuse reflection map; S4. Perform floating object instance segmentation processing on the repaired diffuse reflection map, so that the instance segmentation network outputs the current initial... S5. Perform temporal consistency correction on the floating object instance mask set. Determine the previous initial image frame based on the initial image sequence, and use the previous temporally consistent floating object instance mask set corresponding to the previous initial image frame to propagate across frames through the temporal mask propagation network and perform consistency correction on the floating object instance mask set. Output the temporally consistent floating object instance mask set corresponding to the current initial image frame. S6. Generate and output the floating object detection result based on the temporally consistent floating object instance mask set. The floating object detection result includes the floating object instance contour information and the confidence information corresponding to the instance contour information.

[0008] Optionally, S1 includes:

[0009] Acquire multiple frames of images continuously collected by the water surface monitoring equipment within a preset sampling period and generate an initial image sequence according to the acquisition time order;

[0010] Each frame in the initial image sequence is assigned a corresponding timestamp to form an initial image frame;

[0011] And select the initial image frame to be processed from the initial image sequence according to the timestamp as the current initial image frame;

[0012] Furthermore, acquiring the initial image sequence includes: acquiring at least two frames of images with different exposure parameters at the same sampling time, and performing alignment and fusion on the images with different exposure parameters to generate a high dynamic range image or exposure fusion image used as the initial image frame.

[0013] Optionally, S2 includes:

[0014] The current initial image frame is subjected to reflectance decomposition processing. Based on the imaging constraints of specular reflection and diffuse reflection, the reflectance decomposition network outputs the specular reflection component map and diffuse reflection component map corresponding to the current initial image frame.

[0015] A glare mask is generated based on the region where the pixel intensity in the specular reflection component image is greater than a preset specular reflection threshold. Connectivity filtering and morphological processing are then performed on the glare mask to remove isolated noise and fill holes.

[0016] An overexposure mask is generated based on the saturated pixel region formed by pixels that meet the saturation determination condition in the current initial image frame. Connectivity filtering and morphological processing are then performed on the overexposure mask to obtain an overexposure mask for characterizing local overexposure regions.

[0017] Optionally, S3 includes:

[0018] The area to be repaired is determined based on the overexposed mask, and the diffuse reflection component map is degraded within the area to be repaired to generate an initial repair map. The degradation construction includes replacing the pixel values ​​within the area to be repaired with preset noise or random sampled values.

[0019] The diffusion repair network performs iterative denoising and reconstruction within a preset number of denoising steps based on the initial repair map to obtain the reconstruction result of the region to be repaired.

[0020] The reconstruction result is fused with the diffuse reflection component map based on the overexposure mask. The reconstruction result is used in the area to be repaired, while the diffuse reflection component map is maintained in the non-repaired area. Figure 1 Output the repaired diffuse reflection map;

[0021] Furthermore, the conditional inputs of the diffusion inpainting network also include a diffuse component map or a repaired diffuse map corresponding to at least one initial image frame adjacent to the current initial image frame, so as to use cross-frame redundancy information to apply reference constraints to the region to be repaired.

[0022] Optionally, S4 includes:

[0023] Multi-scale feature maps are obtained by performing multi-scale feature extraction on the repaired diffuse reflection map using an instance segmentation network.

[0024] The glare mask is scaled up to match the spatial resolution of each multi-scale feature map, and multiplicative weighted suppression is performed on the corresponding multi-scale feature map based on the scaled glare mask to reduce the feature response of the glare region.

[0025] Predict the instance segmentation results of floating objects based on the weighted and suppressed multi-scale feature map and generate a set of floating object instance masks;

[0026] The set of floating object instance masks is then subjected to confidence threshold filtering and area threshold filtering to remove low-confidence instance masks and small-area noise instance masks.

[0027] Furthermore, the input to the instance segmentation network is a multi-channel input, which includes at least a repaired diffuse reflection map and at least one of a specular reflection component map and a glare mask, so that the instance segmentation network can use at least one of the specular reflection component map and the glare mask to perform feature decoupling and false detection suppression.

[0028] Optionally, S5 includes:

[0029] Based on the temporal order of the initial image sequence, determine the previous initial image frame adjacent to the current initial image frame, and obtain the previous time-consistent floating object instance mask set corresponding to the previous initial image frame.

[0030] The pixel-level motion relationship between the current initial image frame and the previous initial image frame is estimated based on the repaired diffuse map using a temporal mask propagation network. The previous temporally consistent floating object instance mask set is then propagated to the current initial image frame according to the pixel-level motion relationship to generate a propagation instance mask set.

[0031] An instance correspondence is established based on the mask overlap between the propagation instance mask set and the floating object instance mask set, and the instance masks with the established correspondence are fused to obtain a fused instance mask;

[0032] The fused instance masks and instance masks without established correspondence are summarized, and consistency filtering is performed on the summarized results to remove instance masks that are inconsistent across frames. The set of time-consistent floating object instance masks corresponding to the current initial image frame is output.

[0033] Optionally, S6 includes:

[0034] The instance outline information of each floating object instance is determined based on the time-consistent floating object instance mask set, and the bounding box is calculated for each floating object instance;

[0035] Based on the confidence information output by the instance segmentation network and the instance correspondence, cross-frame weighted fusion is performed on the confidence information to generate fused confidence information corresponding to the instance contour information;

[0036] The instance contour information, the bounding box, and the fusion confidence information are combined to form the floating object detection result.

[0037] The beneficial effects of this invention are:

[0038] 1. A glare mask is generated by decomposing specular reflection and diffuse reflection components, and glare region features are weighted and suppressed during instance segmentation to reduce the interference of strong reflection on feature extraction and classification, thereby reducing false detections and false negatives caused by glare.

[0039] 2. By locating locally overexposed areas using an overexposure mask, and employing a diffusion repair network to iteratively reconstruct and fuse the overexposed areas of the diffuse reflection component, the target texture and edge details are restored, improving the detectability and segmentation accuracy of floating objects under local overexposure conditions.

[0040] 3. The temporally consistent instance mask of the previous frame is propagated across frames through the temporal mask propagation network, and correspondence, fusion and consistency screening are established with the instance segmentation results of the current frame. This suppresses the dynamic background false targets and detection jitter caused by water surface ripples and water flow, and realizes the continuous and stable output of floating object detection results across frames. Attached Figure Description

[0041] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0042] Figure 1 This is a flowchart of an anti-glare floating object detection method based on deep learning and image enhancement proposed in this invention. Detailed Implementation

[0043] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0044] refer to Figure 1 A method for detecting floating objects on water surfaces with anti-glare based on deep learning and image enhancement, comprising:

[0045] S1. Acquire the initial image sequence collected by the water surface monitoring equipment in chronological order, containing multiple initial image frames, and determine the initial image frame to be processed as the current initial image frame; S2. Perform reflectance decomposition processing on the current initial image frame, so that the reflectance decomposition network outputs the specular reflection component map and diffuse reflection component map corresponding to the current initial image frame. Generate a glare mask to characterize the strong reflective area based on the specular reflection component map, and generate an overexposure mask to characterize the local overexposure area based on the current initial image frame; S3. Perform overexposure repair processing on the diffuse reflection component map based on the overexposure mask, so that the diffusion repair network outputs the repaired diffuse reflection map; S4. Perform floating object instance segmentation processing on the repaired diffuse reflection map, so that the instance segmentation network outputs the current initial... S5. Perform temporal consistency correction on the floating object instance mask set. Determine the previous initial image frame based on the initial image sequence, and use the previous temporally consistent floating object instance mask set corresponding to the previous initial image frame to propagate across frames through the temporal mask propagation network and perform consistency correction on the floating object instance mask set. Output the temporally consistent floating object instance mask set corresponding to the current initial image frame. S6. Generate and output the floating object detection result based on the temporally consistent floating object instance mask set. The floating object detection result includes the floating object instance contour information and the confidence information corresponding to the instance contour information.

[0046] In this specific embodiment, S1 includes:

[0047] The water surface monitoring equipment uses fixed, shore-based cameras to continuously collect data on the same monitored water area, with the data acquisition and control module controlling the sampling at a constant period. Trigger imaging, and write the continuously acquired multiple frames of images into the initial image sequence in the order of acquisition time;

[0048] The first in the initial image sequence The timestamp corresponding to the initial image frame satisfies:

[0049] ;

[0050] in, Indicates the first The timestamp of the initial image frame. Indicates the starting timestamp of the initial image sequence. This represents the frame number and increments from 0. This indicates the sampling period and is a fixed value set by the acquisition control module;

[0051] At each sampling time The camera continuously captures two frames of images with different exposure parameters within the same field of view to extend the dynamic range. The short exposure image corresponding to the first exposure parameter is denoted as... The long exposure image corresponding to the second exposure parameter is denoted as Both frames have a resolution of 1000. With the lens aperture and camera focal length remaining constant, a short exposure image is produced. The shutter speed is set to And keep the analog gain at Long exposure image The shutter speed is set to And keep the analog gain at This allows for the acquisition of unexposed highlight information and more adequate shadow details when strong glare and shadow areas coexist.

[0052] To eliminate the geometric misalignment caused by water surface dynamics and minute jitter between two images with different exposures, the following measures were taken: Perform alignment processing and use As a reference frame, the alignment model adopts an affine registration model based on enhanced correlation coefficient, and the affine transformation parameters are denoted as follows. And initialized to an identity affine matrix, during alignment... and Convert to grayscale and construct a three-layer image pyramid, iteratively estimating from coarse to fine. The maximum number of iterations per layer is set to 50, and the iteration termination threshold is set to... And use bilinear resampling to through The transformation yields an aligned image;

[0053] After alignment With Alignment Perform exposure fusion to generate an exposure-fused image used as the initial image frame. Exposure fusion employs a Laplacian pyramid fusion framework, constructing a five-layer Laplacian pyramid for each input image. Simultaneously, a weight map is built with contrast, saturation, and appropriate exposure as evaluation terms, and all three weight indices are set to 1. For each pixel location, the weights of the two input images are normalized, and then layer-by-layer weighted fusion is performed to reconstruct the image. ,in Preserve more effective texture and reduce the impact of local overexposure on subsequent processing;

[0054] The initial image sequence consists of multiple frames. Composition and according to The initial image frames to be processed are stored incrementally, and are retrieved sequentially according to their timestamps and identified as the current initial image frames.

[0055] In this specific embodiment, S2 includes:

[0056] Set the current initial image frame An input reflection decomposition network is used to perform reflection decomposition processing. The reflection decomposition network employs an encoder-decoder structure and has two parallel output branches at the decoding end to output specular reflection component maps respectively. With diffuse reflection component map The encoder employs a residual network backbone consisting of four downsampling stages and sequentially outputs channels with the following number of channels: The decoder uses four upsampling stages corresponding to the downsampling stage and fuses features of the same scale as the encoder through skip connections to preserve edge details. Each upsampling stage includes one bilinear upsampling and two... Convolution, both output branches use Convolution maps features to 3 channels and constrains pixel values ​​to 3 channels using Sigmoid activation. Thus Used to characterize the intensity of specular high reflection and The weight parameters of the reflection decomposition network, used to characterize the texture information of the target in the diffuse component, are trained offline before deployment and remain fixed and not updated during the inference phase.

[0057] based on When generating a glare mask, first... The specular reflection intensity map is obtained by converting the image to grayscale using a fixed coefficient, and a specular reflection threshold is set. Then, the initial binary value of the glare mask is obtained according to the following formula, and connected component filtering and morphological processing are performed:

[0058] ;

[0059] in, Indicates pixel position The glare mask at the location takes values ​​and the set of values ​​is: Indicates pixel position The overexposure mask value at the location is and the set of values ​​is Represents pixel coordinates and This indicates an indicator function that outputs 1 if the condition within the parentheses is true, and outputs 0 otherwise. Represents the specular reflection component diagram At pixel position The pixel intensity at that location has been normalized to Indicates the specular reflection threshold. Indicates the current initial image frame At pixel position Color channels pixel values ​​and These represent the red channel, green channel, and blue channel, respectively. Indicates the saturation threshold and sets it to And in 8-bit quantization Determining localized overexposure within a given space;

[0060] Initial binary values ​​for glare mask When performing connected component filtering, 8-neighbor connectivity is used, and regions with a connected component area of ​​less than 200 pixels are deleted to remove isolated noise. Then, morphological closing and hole filling are performed sequentially to connect broken highlights and fill internal holes. The structuring element of the closing operation is... The square structural element is filled with holes using a flooding method that uses the image boundary as the background seed.

[0061] Binary initial values ​​for overexposed masks Similarly, 8-neighbor connected component filtering is used to filter and delete regions with an area less than 100 pixels. Then, morphological dilation is performed again to cover the half-saturated transition zone at the edge of the saturated region. The structuring element of the dilation is... Square structural elements were used to obtain glare masks for characterizing highly reflective areas. And overexposure masks used to characterize local overexposure areas and the diffuse reflection component map Glare mask With overexposed mask .

[0062] In this specific embodiment, S3 includes:

[0063] Based on overexposure mask Identify the area to be repaired And based on the diffuse component map corresponding to the current initial image frame. As the initial image for basic image construction and repair ;

[0064] Repair the initial image The construction rule is to maintain consistency with the non-repair area. In the area to be repaired, pixel values ​​are replaced with uniformly distributed random noise, denoted as . And satisfy ,in Indicates that it is defined in A continuous and uniform distribution on top Represents pixel coordinates and ;

[0065] Will With overexposed mask The diffuse inpainting network is input to perform iterative denoising and reconstruction of the region to be inpainted. The diffuse component map or the inpainted diffuse map corresponding to at least one adjacent initial image frame is used as a conditional input to provide cross-frame redundancy reference, wherein the adjacent frame is selected from the inpainted diffuse map of the previous initial image frame. ,when When Set as with Use zero-size images of the same size to ensure that the dimensions of the conditional input are determined;

[0066] The diffusion repair network employs a conditional diffusion denoising model and performs a fixed number of denoising steps during the inference phase. Its denoising network employs a two-dimensional network with time-step embedding. The Net structure contains four scale levels at both the encoding and decoding ends, with the number of channels arranged sequentially as follows: Each scale level consists of two sets of " The model consists of convolution, group normalization, and SiLU activation. Downsampling is achieved through convolution with a stride of 2, and upsampling is achieved through bilinear upsampling. A multi-head self-attention module is added to the bottleneck layer with the number of attention heads set to 4 to enhance the modeling ability of large-scale textures. The temporal embedding dimension is set to 256 and then added to each residual block after being mapped through two fully connected layers.

[0067] Conditional inputs for diffusion repair networks The structure is constructed by splicing together channels as follows:

[0068] ;

[0069] in This indicates splicing by channel dimension. This indicates pixel-by-pixel multiplication. This indicates the overexposure of the binary overexposed mask. The non-overexposure mask obtained by pixel-by-pixel inversion is used by the diffusion inpainting network at each time step based on the current noise state, time step embedding, and conditional input. Predict noise and follow a fixed noise schedule The update is performed, and the noise schedule adopts a linear schedule and satisfies... After the iteration is complete, the reconstructed diffuse reflection map is output. ;

[0070] Will Compared with the original diffuse reflection component map Based on overexposure mask Perform region fusion to obtain the restored diffuse reflection map The fusion relationship satisfies:

[0071] ;

[0072] in This indicates the location of the repaired diffuse map at the pixel position. Pixel value at that location, Indicates the overexposure mask at the pixel position. The mask value at the location and the set of values ​​is This represents the diffuse component map at the pixel location. Pixel value at that location, This indicates the reconstructed diffuse map at pixel location. The pixel value at that location, constant 1 represents a scalar with a value of 1, used to achieve pixel-level selection of "overexposed areas replaced, non-overexposed areas preserved".

[0073] In this specific embodiment, S4 includes:

[0074] Repair the diffuse reflection image Diagram of specular reflection components and glare masks The multi-channel inputs that together constitute the instance segmentation network are denoted as . And in order of passage The three color channels The 3 color channels and It is obtained by stitching together a single-channel binary mask, so that The number of channels is 7, and for and Perform numerical normalization to make the pixel value range 10 ... ,right Preserving the set of binary values constant;

[0075] The instance segmentation network adopts a two-stage instance segmentation structure with Mask R-CNN as the overall framework. The backbone network uses ResNet-50, and the size of the first-layer convolutional kernel is [missing information]. With a stride of 2 and 64 output channels, the size of the first-layer convolutional weight tensor is set to accommodate a 7-channel input. And in the backbone network A feature pyramid network is built on top of the feature layer. To obtain multi-scale feature maps The superscripts (2) to (5) represent the FPN levels and correspond to the input resolution respectively. The number of channels in each layer of the feature map is uniformly 256;

[0076] Glare mask Scaling the images individually yields scaled glare masks with spatial resolution consistent with the feature maps of each layer. Scaling employs nearest-neighbor downsampling to preserve binary boundaries and avoid introducing intermediate gray levels, and multiplicative weighted suppression is performed on each feature map layer to obtain suppressed multi-scale feature maps. The inhibition relationship is as follows:

[0077] ;

[0078] in, Indicates the FPN level. Indicates hierarchy The original multi-scale feature map, Indicates hierarchy The multi-scale feature map after glare suppression. This indicates element-wise multiplication. Indicates scaling to hierarchy A glare mask with a resolution and a set of values. , Indicates the glare suppression coefficient and sets it to 0. The constant 1 represents a scalar with a value of 1 and is used to construct the suppression weights corresponding to the scaled glare mask;

[0079] Based on multi-scale feature maps after glare suppression to Perform candidate box generation and instance segmentation prediction, where candidate box generation uses a Region Proposal Network (RPN) with anchor box scale set to [value missing]. And the aspect ratio is Non-maximum suppression is performed on the RPN output with a threshold set to 0.7, and the top 1000 candidate boxes with the highest scores are retained for RoIAlign. The features of each candidate region output by RoIAlign are fed into a classification regression head to output class confidence and bounding box regression values, and simultaneously fed into a mask head to output the corresponding binary instance mask. The mask head consists of four... The system consists of convolutional layers, each with 256 output channels, followed by a single... Deconvolution upsampling and through Convolution output The mask logits are obtained and the mask probability map is obtained by Sigmoid. The categories are set as "floating object" single class and "background" single class.

[0080] Binarize the mask probability map with a threshold of 0.5 and map it back to the original image size to obtain the set of floating object instance masks. and for the set Each instance mask in the dataset is associated with the confidence level of its classification head output. Subsequently, confidence threshold filtering and area threshold filtering are performed to remove low-confidence instances and small-area noise instances, where the confidence threshold is set to [value missing]. The area threshold is set to Output the set of floating object instance masks corresponding to the current initial image frame, where each instance mask is counted by the number of pixels with a value of 1 within it. .

[0081] In this specific embodiment, S5 includes:

[0082] Based on the time sequence of the initial image sequence, the current initial image frame is... The previous initial image frame was determined as And read the set of floating object instance masks corresponding to the previous time sequence of the previous initial image frame. and with The set of previous fusion confidence scores corresponding to each instance mask in the middle ,in This represents the set of temporally consistent floating object instance masks output from the previous initial image frame. Represents a set The Middle Fusion confidence of individual instance masks;

[0083] when When Set to an empty set and directly mask the floating object instances. The set of temporally consistent floating object instance masks for the current initial image frame Output;

[0084] when At that time, a temporal mask propagation network is used to estimate the pixel-level motion relationship between the current initial image frame and the previous initial image frame based on the repaired diffuse reflection map. The temporal mask propagation network is an end-to-end optical flow estimation network and its input is a channel-stitched sequence. ,in This represents the restored diffuse image from the previous initial image frame. Represents the restored diffuse map of the current initial image frame, Concat This indicates concatenation by channel dimension, resulting in an input of 6 channels and a pixel value range of [value missing]. ;

[0085] The temporal mask propagation network adopts an encoder-decoder structure and outputs a two-dimensional displacement field with the same resolution as the input. ,in A two-channel floating-point image with each pixel position The corresponding displacement vector is , Represents pixel coordinates and This represents the horizontal pixel displacement from the previous initial image frame to the current initial image frame. It represents the vertical pixel displacement from the previous initial image frame to the current initial image frame;

[0086] The encoder contains 4 downsampling levels, and the number of output channels for each level is as follows: Each level consists of two The decoder is constructed using convolutions, with a stride of 2 used for downsampling in the first convolution. After each convolution, cascaded group normalization and SiLU activation are performed. The decoder contains four upsampling levels symmetrical to the encoder, employing bilinear upsampling and skip connections to fuse corresponding encoded features. Two upsampling steps are applied after each upsampling level. Convolution, group normalization, and SiLU activation are ultimately achieved through a single step. Convolution output 2-channel displacement field Furthermore, the output layer does not use an activation function to maintain the shift as a signed real number;

[0087] Using displacement field The mask set of the previous time-consistent floating object instances Perform cross-frame propagation to generate a set of propagation instance masks. The mode of transmission is to Each binary instance mask in the process is first converted to Floating-point mask probability graph and using a method based on The reverse bilinear sampling is used for pixel-level distortion, and then binarized with a threshold of 0.5 to obtain the propagated binary instance mask, thereby enabling... Align with the current initial image frame space;

[0088] Based on the propagation instance mask set With floating object instance mask set The mask overlap between instances establishes a correspondence, and the mask overlap is determined by the cross-union ratio. Calculate and satisfy:

[0089] ;

[0090] in, Represents a binary mask With binary mask The intersection and union ratio, Represents the set of propagation instance masks A mask for a specific instance, Represents the set of floating object instances mask A mask for a specific instance, The intersection operation represents the set of pixels. The union operation represents the set of pixels. The cardinality of the pixel set is equal to the number of pixels in the set;

[0091] After calculating the intersection, union, and comparison (IUCN) of all propagation instance masks with the current instance mask pairwise to form a cost matrix, the Hungarian matching algorithm is used to obtain one-to-one matching pairs, and only those that satisfy the condition are retained. The matching pairs are considered as valid correspondences, where This indicates the intersection-union ratio threshold, which is set to 0.3.

[0092] For each valid correspondence, the propagation instance mask is fused with the current instance mask to obtain a fused instance mask. The fusion method involves normalizing and weighting the probability maps of the two masks according to their corresponding confidence levels, then linearly superimposing them and binarizing them with a threshold of 0.5. The confidence level of the current instance mask is denoted as... And by steps The instance segmentation network classification head output, and the confidence of propagating the instance mask, uses the previous fusion confidence of its source. The instance identifier corresponding to the instance mask will be inherited and propagated through the fusion instance mask to ensure consistency across frames.

[0093] All fusion instance masks are aggregated into a set. and the set No corresponding relationship has been established and the confidence level is satisfied. The current instance masks are collectively aggregated to form a candidate set, where The threshold for accepting new instances is set to 0.9. Then, all propagation instance masks without established relationships are removed to filter out inconsistent instances across frames, resulting in a set of temporally consistent floating object instance masks corresponding to the current initial image frame. .

[0094] In this specific embodiment, S6 includes:

[0095] Based on the time-consistent floating object instance mask set corresponding to the current initial image frame Generate and output the floating object detection results;

[0096] right Each binary instance mask in the process performs instance contour extraction to obtain instance contour information. The instance contour extraction adopts a boundary tracking algorithm based on 8 neighborhoods and outputs a closed contour point sequence in clockwise order. Then, Douglas-Peucker polygon approximation is performed on the contour point sequence to obtain an instance contour polyline with no more than 200 vertices and an approximation error threshold of 2 pixels, so that the instance contour information can be directly stored and displayed in the form of a pixel coordinate sequence.

[0097] The bounding box is calculated based on the contour information of the instance. The bounding box is an axis-aligned rectangle, and the coordinates of the upper left corner are obtained by traversing the sequence of contour points. and the coordinates of the bottom right corner ,in and These are the minimum and maximum x-coordinates in the contour point sequence, respectively. and These are the minimum and maximum values ​​of the ordinate in the contour point sequence, respectively, and both coordinates are based on the current initial image frame. Represented by pixel coordinates;

[0098] Based on the confidence information output by the instance segmentation network for the current initial image frame and the established instance correspondence, cross-frame weighted fusion is performed on the confidence information to generate fused confidence information. For the step... The set of floating object instance masks is consistent with the previous time series. For instances that establish a correspondence, let the original confidence level of the instance corresponding to the current frame be . The fusion confidence level for this instance in the previous frame was 1. The fusion weight is and let the fusion weight Equal to steps The intersection-union ratio of the correspondence is limited to the interval. To avoid weight degradation, the fusion confidence of the current frame is calculated using the following formula:

[0099] ;

[0100] in, Indicates the current initial image frame The Middle Fusion confidence of time-consistent floating object instances Indicates the weights used for cross-frame fusion. This represents the original confidence level corresponding to the instance mask of the current frame. This represents the previous fusion confidence level in the previous initial image frame that establishes a correspondence with this instance, and the constant 1 represents a scalar with a value of 1;

[0101] For steps For newly created instances for which no corresponding relationship has been established, directly... To ensure that the detection confidence of the current frame is not diluted by cross-frame fusion;

[0102] The instance outline information, bounding box, and fusion confidence information of each floating object instance are structurally summarized to form the floating object detection result, which is then output as a result with a timestamp. Related result records.

[0103] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

[0104] This invention addresses the critical contradictions in water surface monitoring scenarios: "strong reflections and local overexposure leading to loss of target contrast and texture, and dynamic water surface backgrounds causing false targets and result jitter." It employs a synergistic link of "reflection decomposition, overexposure repair, instance segmentation, and temporal correction" to generate a cumulative effect. First, by decomposing specular and diffuse reflection components, the influence of strong reflections is explicitly removed from the image, and a glare mask is generated from the specular component to identify high-light interference areas. Next, the overexposure mask locates saturated areas, and diffuse reflection components are repaired using diffusion to restore texture and edge information caused by local overexposure. Then, a glare mask suppression mechanism is introduced in the instance segmentation stage to reduce the interference of high-light areas on feature extraction and discrimination. Finally, by combining temporal mask propagation and cross-frame fusion consistency screening, redundant information from adjacent frames is used to suppress false targets and cross-frame jitter caused by water ripples and currents, thereby achieving continuous and stable floating object detection output and improving accuracy.

[0105] In terms of algorithm structure, this invention makes targeted improvements to address the aforementioned technical problems: First, strong reflection and local overexposure are decoupled and modeled using glare masks and overexposure masks respectively, and diffusion repair is limited to the overexposure area of ​​the diffuse reflection component, avoiding unnecessary reconstruction disturbances to the normal area and improving the reliability of repair and detection; Second, glare mask weighted suppression is performed at the multi-scale feature level of the instance segmentation network, so that the high-light interference is consistently weakened at each layer of the feature pyramid, enhancing the separability of small targets and low-contrast targets; Third, mask cross-frame propagation based on pixel-level motion relationships is introduced, and a stable temporally consistent mask set is formed through instance correspondence, fusion, and consistency screening, making the detection results more adaptable to dynamic water surface backgrounds, further improving the stability of continuous detection and the ability to resist false targets.

Claims

1. A method for detecting floating objects on water surfaces with anti-glare based on deep learning and image enhancement, characterized in that, include: S1. Obtain the initial image sequence collected by the water surface monitoring equipment in chronological order, which includes multiple initial image frames, and determine the initial image frame to be processed as the current initial image frame; S2. Perform reflection decomposition processing on the current initial image frame, so that the reflection decomposition network outputs the specular reflection component map and diffuse reflection component map corresponding to the current initial image frame. Generate a glare mask to characterize the strong reflective area based on the specular reflection component map, and generate an overexposure mask to characterize the local overexposure area based on the current initial image frame. S3. Perform overexposure repair processing on the diffuse reflection component map based on the overexposure mask, so that the diffusion repair network outputs the repaired diffuse reflection map; S4. Perform floating object instance segmentation processing on the repaired diffuse reflection map, so that the instance segmentation network outputs the floating object instance mask set corresponding to the current initial image frame, and suppress the interference of the glare region on the floating object instance segmentation processing based on the glare mask. S5. Perform temporal consistency correction on the floating object instance mask set. Determine the previous initial image frame based on the initial image sequence, and use the previous temporally consistent floating object instance mask set corresponding to the previous initial image frame to propagate across frames through the temporal mask propagation network and perform consistency correction on the floating object instance mask set. Output the temporally consistent floating object instance mask set corresponding to the current initial image frame. S6. Generate and output floating object detection results based on the time-consistent floating object instance mask set. The floating object detection results include the instance contour information of the floating object and the confidence information corresponding to the instance contour information.

2. The method for detecting anti-glare floating objects on water surfaces based on deep learning and image enhancement according to claim 1, characterized in that, S1 includes: Acquire multiple frames of images continuously collected by the water surface monitoring equipment within a preset sampling period and generate an initial image sequence according to the acquisition time order; Each frame in the initial image sequence is assigned a corresponding timestamp to form an initial image frame; The initial image frame to be processed is selected from the initial image sequence according to the timestamp as the current initial image frame.

3. The anti-glare floating object detection method based on deep learning and image enhancement according to claim 1, characterized in that, S2 include: The current initial image frame is subjected to reflectance decomposition processing. Based on the imaging constraints of specular reflection and diffuse reflection, the reflectance decomposition network outputs the specular reflection component map and diffuse reflection component map corresponding to the current initial image frame. A glare mask is generated based on the region where the pixel intensity in the specular reflection component image is greater than a preset specular reflection threshold. Connectivity filtering and morphological processing are then performed on the glare mask to remove isolated noise and fill holes. An overexposure mask is generated based on the saturated pixel region formed by pixels that meet the saturation determination condition in the current initial image frame. Connectivity filtering and morphological processing are then performed on the overexposure mask to obtain an overexposure mask for characterizing local overexposure regions.

4. The method for detecting anti-glare floating objects on water surfaces based on deep learning and image enhancement according to claim 1, characterized in that, S3 includes: The area to be repaired is determined based on the overexposed mask, and the diffuse reflection component map is degraded within the area to be repaired to generate an initial repair map. The degradation construction includes replacing the pixel values ​​within the area to be repaired with preset noise or random sampled values. The diffusion repair network performs iterative denoising and reconstruction within a preset number of denoising steps based on the initial repair map to obtain the reconstruction result of the region to be repaired. Based on the overexposure mask, the reconstruction result is fused with the diffuse reflection component map, wherein the reconstruction result is used in the area to be repaired, and the area to be repaired is kept consistent with the diffuse reflection component map, and the repaired diffuse reflection map is output.

5. The anti-glare water surface floating object detection method based on deep learning and image enhancement according to claim 1, characterized in that, S4 include: Multi-scale feature maps are obtained by performing multi-scale feature extraction on the repaired diffuse reflection map using an instance segmentation network. The glare mask is scaled up to match the spatial resolution of each multi-scale feature map, and multiplicative weighted suppression is performed on the corresponding multi-scale feature map based on the scaled glare mask to reduce the feature response of the glare region. Predict the instance segmentation results of floating objects based on the weighted and suppressed multi-scale feature map and generate a set of floating object instance masks; The set of floating object instance masks is then subjected to confidence threshold filtering and area threshold filtering to remove low-confidence instance masks and small-area noise instance masks.

6. The method for detecting anti-glare floating objects on water surfaces based on deep learning and image enhancement according to claim 1, characterized in that, S5 include: Based on the temporal order of the initial image sequence, determine the previous initial image frame adjacent to the current initial image frame, and obtain the previous time-consistent floating object instance mask set corresponding to the previous initial image frame. The pixel-level motion relationship between the current initial image frame and the previous initial image frame is estimated based on the repaired diffuse map using a temporal mask propagation network. The previous temporally consistent floating object instance mask set is then propagated to the current initial image frame according to the pixel-level motion relationship to generate a propagation instance mask set. An instance correspondence is established based on the mask overlap between the propagation instance mask set and the floating object instance mask set, and the instance masks with the established correspondence are fused to obtain a fused instance mask; The fused instance masks and instance masks without established correspondence are summarized, and consistency filtering is performed on the summarized results to remove instance masks that are inconsistent across frames. The set of time-consistent floating object instance masks corresponding to the current initial image frame is output.

7. The anti-glare water surface floating object detection method based on deep learning and image enhancement according to claim 1, characterized in that, S6 include: The instance outline information of each floating object instance is determined based on the time-consistent floating object instance mask set, and the bounding box is calculated for each floating object instance; Based on the confidence information output by the instance segmentation network and the instance correspondence, cross-frame weighted fusion is performed on the confidence information to generate fused confidence information corresponding to the instance contour information; The instance contour information, the bounding box, and the fusion confidence information are combined to form the floating object detection result.

8. The method for detecting anti-glare floating objects on water surfaces based on deep learning and image enhancement according to claim 2, characterized in that, Acquiring the initial image sequence includes: acquiring at least two frames of images with different exposure parameters at the same sampling time, and performing alignment and fusion on the images with different exposure parameters to generate a high dynamic range image or exposure-fused image used as the initial image frame.

9. The method for detecting anti-glare floating objects on water surfaces based on deep learning and image enhancement according to claim 4, characterized in that, The conditional inputs of the diffusion inpainting network also include the diffuse component map or the inpainted diffuse map corresponding to at least one initial image frame adjacent to the current initial image frame, so as to use cross-frame redundancy information to apply reference constraints to the region to be inpainted.

10. The method for detecting anti-glare floating objects on water surfaces based on deep learning and image enhancement according to claim 5, characterized in that, The instance segmentation network receives a multi-channel input, which includes at least a repaired diffuse reflection map and at least one of a specular reflection component map and a glare mask, so that the instance segmentation network can use at least one of the specular reflection component map and the glare mask to perform feature decoupling and false detection suppression.

Images