An image recognition-based water surface floating object classification and measurement method

Abstract

The application discloses a water surface floating object classification and measurement method based on image recognition, which comprises the following steps: placing a calibration board, collecting an image base map and an initial focal length map, and calibrating four vertex pixel feature points of the calibration board; marking a water mask and a floating object mask, identifying the water mask and generating a floating object detection range, training a floating object identification model by using the floating object mask and a MaskDINO model; identifying the position of the calibration board, calculating a current perspective transformation matrix according to the position of the calibration board and the physical size of the calibration board, mapping the floating object identified by the MaskDINO model to a physical plane by using the current perspective transformation matrix, and calculating the physical area of the floating object; identifying the type and position of the floating object based on a zoom map, matching the initial focal length map and the zoom map identification result; collecting the pixel detection position of the floating object in the initial focal length map, converting the relative physical detection position of the floating object by using the current perspective transformation matrix, and converting the flow rate of the floating object. The accurate classification problem of the floating object is effectively solved.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a method for classifying and measuring floating objects on the water surface based on image recognition. Background Technology

[0002] With the development of computer vision and image processing technologies, using fixed cameras to monitor floating objects on the water surface has gradually become a mainstream method. This technology involves setting up cameras at fixed locations around the water area to continuously capture images of the water surface, and then using image recognition algorithms to analyze the acquired video images to identify and track floating objects.

[0003] In terms of ecological and environmental protection, accurate classification of floating debris can quickly pinpoint pollution sources, such as distinguishing between household waste, aquatic plants, and duckweed, providing data support for tracing the source and assigning responsibility, and assisting in the formulation of targeted governance policies. In the safety and management of water conservancy projects, the area measurement and flow velocity calculation of floating debris can help assess the impact risk of water flow on dams and sluice gates; especially during the flood season, large floating debris may cause structural damage when it collides with hydraulic structures, and real-time flow velocity data can be linked to early warning systems to activate anti-collision protection measures in advance.

[0004] However, existing technologies for monitoring floating objects on water surfaces using fixed cameras still face many challenges. The water surface and surrounding environment are complex and variable; dynamic changes in light intensity can lead to uneven image brightness, easily causing false detections. Floating objects in rivers vary in distance from the camera, and in a fixed view, distant floating objects are difficult to classify accurately due to resolution limitations. Measuring the area and calculating the flow velocity of floating objects relies on image calibration, but dynamic changes in water level make efficient and adaptive calibration a challenge. Summary of the Invention

[0005] Based on this, this application provides an image recognition-based method for classifying and measuring floating objects on the water surface to address the problems of false detection, inaccurate classification of floating objects, and reliance on image calibration for measuring the area and flow velocity of floating objects in existing technologies.

[0006] This application provides a method for classifying and measuring floating objects on the water surface based on image recognition, including:

[0007] A calibration board is placed on the water surface to be monitored, and the image base map P_base and the initial focal length map P are acquired. The four vertex pixel feature points of the calibration board are then calibrated.

[0008] Images of the water surface are acquired, and water masks and floating object masks are labeled. The Segformer semantic segmentation model is used to identify the water masks, generate the floating object detection range, and train the floating object recognition model using the floating object mask and the MaskDINO model.

[0009] Based on the initial focal length map P, floating objects are identified using the MaskDINO model; the position of the calibration board is identified according to the initial focal length map P; the current perspective transformation matrix is ​​calculated based on the position and physical dimensions of the calibration board; the floating objects identified by the MaskDINO model are mapped to a physical plane using the current perspective transformation matrix; and the physical area of ​​the floating objects is calculated. The floating object identification results need to be filtered by the floating object detection range, and the floating object detection range is updated periodically.

[0010] Using a PTZ camera, the floating object monitoring area is captured by optical zoom to obtain a zoom image P_zoom;

[0011] Based on the zoom image P_zoom, the type and location of floating objects are identified, and the initial focal length image P and the zoom image P_zoom identification results are matched.

[0012] By collecting the floating object pixel detection positions of the initial focal length image P, the floating objects are tracked using the sort algorithm. At the same time, the floating objects' relative physical detection positions are calculated by calibrating the current perspective transformation matrix, and the floating object's flow velocity is obtained.

[0013] The generation of the floating object detection range includes,

[0014] Water body segmentation is performed on the base image P_base using the trained semantic segmentation Segformer model. The first water surface area mask MaskA is extracted, and the initial dilation coefficient is used. Perform a morphological dilation operation on the first water surface range mask MaskA to obtain the water surface floating object detection range Mask_range1 at the initial moment;

[0015] Set the update frequency, use the semantic segmentation Segformer algorithm to segment the water surface of the acquired image, and extract the second water surface range mask MaskB.

[0016] Calculate the minimum distance from all outer contour points of the second water surface area mask MaskB to all outer contour points of the first water surface area mask MaskA. If the outer contour points of the second water surface area mask MaskB are within the first water surface area mask MaskA, the minimum distance is negative; otherwise, the minimum distance is positive.

[0017] include,

[0018] Count the number of minimum distances that are negative and the number of minimum distances that are positive. Based on the number of minimum distances that are negative and the number of minimum distances that are positive, determine whether the water level of the second water surface area mask (MaskB) is rising or falling relative to the water level of the first water surface area mask (MaskA). If the number of minimum distances that are positive is greater than the number of minimum distances that are negative, then the water level is rising; otherwise, the water level is falling.

[0019] If the water level of the second water surface area mask MaskB rises relative to the first water surface area mask MaskA, the new expansion coefficient λ is:

[0020] λ=Max(P)+abs(Min(N));

[0021] If the water level of the second water surface area mask MaskB drops relative to the first water surface area mask MaskA, the new expansion coefficient λ is:

[0022] λ = abs(Min(N));

[0023] Where P represents the maximum positive distance and N represents the minimum negative distance;

[0024] The second water surface range mask MaskB is morphologically expanded using a new expansion coefficient λ to obtain the current water surface floating object detection range Mask_range2.

[0025] The identification of the calibration board position includes,

[0026] Extract the bounding rectangle ROI image imgA_ROI from the base image P_base. The bounding rectangle is expanded by 30 pixels based on the positions of the four vertex pixel feature points of the calibration reference box. Perform template matching between imgA_ROI and the initial focal length image P.

[0027] Based on the ROI region obtained by matching, the center point is taken and the open source SAM2 model is used for segmentation to obtain the binary mask image of the calibration board.

[0028] For the binary mask image of the calibration plate, the convex hull calculation function of the OPENCV algorithm library is used to obtain the set of vertex coordinates of the convex polygon;

[0029] The approxPolyDP polygon approximation function from the OpenCV algorithm library is used to process the point set, reducing the number of contour points while maintaining shape approximation, and obtaining the quadrilateral vertex coordinate set.

[0030] After sorting and matching the quadrilateral vertex coordinate set with the physical coordinates, four pixel feature points of the calibration board are obtained. Based on the four pixel points of the calibration board and the corresponding physical coordinate points, the current perspective transformation matrix is ​​calculated.

[0031] If multiple calibration plates are placed on the water surface, the position of each calibration plate is identified, and for each floating object in the image, the calibration plate with the closest vertical distance is selected.

[0032] The acquisition of the zoom image P_zoom includes,

[0033] The water surface area is divided into regions according to a certain pattern, and four zoom magnification preset positions are set. Based on the area ratio of floating objects in each sub-region, the most suitable preset position is selected to zoom and capture zoom images P_zoom.

[0034] The operations for identifying the type and location of floating objects based on the zoom image P_zoom include,

[0035] Based on the image P_zoom, the floating object recognition model is used for secondary recognition.

[0036] The operation of matching the initial focal length image P and the zoom image P_zoom recognition results includes:

[0037] Identify the floating object masks of the initial focal length image P and the zoom image P_zoom, and extract the ROI image of the original image corresponding to each connected component in the floating object mask;

[0038] Extract the HSV components of the ROI image and generate an HS component histogram. Normalize the number of pixels in the HS component histogram.

[0039] Traverse the histograms of the ROI images in the initial focal length image P, calculate the ROI image with the highest similarity to each identified floating object in the ROI image in the initial focal length image P and the zoom image P_zoom, and use the corresponding ROI classification result in the zoom image P_zoom as the corresponding ROI classification result in the initial focal length image P.

[0040] The HISTCMP_CORREL relevance matching algorithm was used to assess similarity. Perform calculations.

[0041] ;

[0042] Where I represents the index of the histogram, The histogram representing the ROI image of the initial focal length image P. This represents the histogram mean of the initial focal length plot P. The histogram representing the ROI image of the zoom image P_zoom. This represents the histogram mean of the zoom image P_zoom.

[0043] Beneficial effects: This invention automatically generates the detection range and improves the Mask-DINO training model, thereby enhancing the model's detection performance;

[0044] This invention proposes a method that combines optical zoom function with floating object classification, which effectively solves the problem that floating objects at the far end of the river are difficult to classify accurately due to insufficient resolution;

[0045] This invention proposes an adaptive water level change water surface calibration algorithm and applies it to the measurement of floating object area and flow velocity. The algorithm can automatically adjust calibration parameters according to dynamic changes in water level and other conditions, achieving efficient and accurate image calibration and providing data support for flood season early warning.

[0046] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0047] The accompanying drawings are provided for a better understanding of this solution and do not constitute a limitation of this application. Wherein:

[0048] Figure 1 This is based on the flowchart provided in this application;

[0049] Figure 2 It is based on the calibration diagram provided in this application;

[0050] Figure 3 This is based on the PTZ camera zoom setting diagram provided in this application.

[0051] Figure 4 This is a line chart comparing the validation set metrics of the MaskDINO model provided in this application and traditional methods;

[0052] Figure 5 It is based on the initial focal length diagram provided in this application;

[0053] Figure 6 It is based on the zoom image provided in this application;

[0054] Figure 7 This is a screenshot of the ROI of the calibration board provided in this application;

[0055] Figure 8 It is the matching result of the base map P_base provided in this application and the calibration board template of the real-time video;

[0056] Figure 9 The image shows the fitting result of the circumscribed quadrilateral of the identified profile based on the calibration plate provided in this application.

[0057] Figure 10 This application provides an initial focal length floating ROI hue component feature histogram;

[0058] Figure 11 This application provides a histogram of saturation component characteristics of the initial focal length floating ROI.

[0059] Figure 12 This application provides a histogram of tonal component features of the ROI of the zoom image.

[0060] Figure 13 This application provides a zoom image of the ROI saturation component characteristic histogram. Detailed Implementation

[0061] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0062] like Figure 1 As shown, this application provides a method for classifying and measuring floating objects on the water surface based on image recognition, including:

[0063] S1: Place a calibration board on the water surface to be monitored, acquire the base image P_base and the initial focal length image P, and calibrate the four vertex pixel feature points of the calibration board. It should be noted that:

[0064] In practical applications, calibration boards of different sizes and colors can be arranged at different longitudinal distances according to the length of the monitored water surface image in the longitudinal direction. The four vertex pixel feature points Z1, Z2, Z3, and Z4 of the calibration board are then sequentially calibrated (e.g., ...). Figure 2 (As shown), for use in subsequent measurements. Figure 7 This is the result of extracting the calibration plate position image region on the base map P_base based on the pre-calibrated coordinates of the four vertices Z1-Z4;

[0065] The identification of the calibration board position includes,

[0066] Extract the bounding rectangle (ROI) image (imgA_ROI) of the calibration reference bounding box from the base image P_base. The bounding rectangle is extended 30 pixels outward based on the positions of the four vertex pixel feature points of the calibration. Perform template matching between imgA_ROI and the initial focal length image P (e.g., ...). Figure 8 (as shown)

[0067] Based on the ROI region obtained by matching, the center point is taken and the open source SAM2 model is used for segmentation to obtain the binary mask image of the calibration board.

[0068] For the binary mask image of the calibration plate, the convex hull calculation function of the OPENCV algorithm library is used to obtain the set of vertex coordinates of the convex polygon;

[0069] The approxPolyDP polygon approximation function from the OpenCV algorithm library is used to process the point set, reducing the number of contour points while maintaining shape approximation, and obtaining the quadrilateral vertex coordinate set.

[0070] Figure 9 To extract the fitting result image of the circumscribed quadrilateral of the calibration board recognition contour based on real-time video frames.

[0071] After sorting and matching the quadrilateral vertex coordinate set with the physical coordinates, four pixel feature points of the calibration board are obtained. Based on the four pixel points of the calibration board and the corresponding physical coordinate points, the current perspective transformation matrix is ​​calculated.

[0072] Let the coordinates of the four matched pixel feature points be (x1,y1), (x2,y2), (x3,y3), and (x4,y4), and sort them counterclockwise. Based on the physical size, let the physical coordinates be (0,0), (0,win_size), (win_size,win_size), and (win_size,0), where win_size represents the physical size of the calibration reference object. Here, we choose 1 meter. Based on the four pixel points and the corresponding physical coordinate points, calculate the perspective transformation 3*3 matrix.

[0073] If multiple calibration plates are placed on the water surface, the position of each calibration plate is identified, and for each floating object in the image, the calibration plate with the closest vertical distance is selected.

[0074] S2: Acquire images of the water surface, label the water mask and floating object mask, use the semantic segmentation Segformer model to identify the water mask, generate the floating object detection range, and use the floating object mask and MaskDINO model to train the floating object recognition model;

[0075] Images from security surveillance cameras are collected and floating objects are labeled. The floating objects are divided into four categories: household waste, duckweed, aquatic plants, and water hyacinth. Due to the diversity of floating objects and their shapes, as well as the presence of reflections and glare on the water surface and interference from the shore environment, floating object identification is prone to missed detections and false detections. Therefore, this invention uses the MaskDINO multi-task learning algorithm, which combines object detection and semantic segmentation, for floating object detection. The training method is improved and the model performance is enhanced by adopting the following balanced negative sample mining strategy.

[0076] The training of the floating object recognition model includes,

[0077] The MaskDINO model, a multi-task learning algorithm combining object detection and semantic segmentation, is used to detect floating objects. A balanced negative sample mining strategy is employed to improve the training of the floating object recognition model.

[0078] Object detection training requires identifying positive and negative samples and labeling ground truth boxes; the classification loss is then calculated using positive samples. cls Confidence loss conf and location loss loc Based on classification loss cls Confidence loss conf and location loss loc Calculate the negative sample confidence loss (Loss).

[0079] ;

[0080] in, , , These represent the weight coefficients of classification loss, confidence loss, and localization loss, respectively.

[0081] The definition rule for positive samples is as follows:

[0082] Iterate through all labeled ground truth boxes and calculate the Intersection over Union (IOU) with all predicted boxes. The predicted box with the highest IOU is taken as the ground truth predicted box. At the same time, set the number of image labeled ground truth boxes in each training batch to N, then the number of positive samples selected for training is also N.

[0083] Negative samples are filtered based on the IOU (Intersection over Union) between the training predicted bounding boxes and the labeled ground truth bounding boxes, and the confidence scores of the training predicted bounding boxes.

[0084] Method 1: Sort the training predicted boxes by IOU with the labeled ground truth boxes, and not the predicted boxes that are determined to be positive samples. The higher the IOU, the greater the difficulty of learning negative samples.

[0085] Method 2: Sort the training prediction boxes by confidence level, and not the prediction boxes that are determined to be positive samples. The higher the confidence level, the greater the difficulty of learning negative samples.

[0086] The positive to negative samples are selected in a 1:5 ratio, meaning the total number of negative samples is 5*N. The negative samples are then selected based on a 1:4 ratio between methods 1 and 2, meaning the total number of negative samples learned using method 1 is N, and the total number of negative samples learned using method 2 is 4*N. Therefore, the negative sample loss weights are calculated as follows:

[0087] ;

[0088] ;

[0089] in, Representation 1: Negative sample loss weights Method 2: Negative sample loss weights, where num_n represents the total number of predicted negative samples, x represents the IOU value, y represents the confidence level, and u and v represent the probability distribution used to adjust the selection of negative samples.

[0090] The generation of the floating object detection range includes,

[0091] Water body segmentation is performed on the base image P_base using the trained semantic segmentation Segformer model. The first water surface area mask MaskA is extracted, and the initial dilation coefficient is used. Perform a morphological dilation operation on the first water surface range mask MaskA to obtain the water surface floating object detection range Mask_range1 at the initial moment;

[0092] Set the update frequency, use the semantic segmentation Segformer algorithm to segment the water surface of the acquired image, and extract the second water surface range mask MaskB.

[0093] Calculate the minimum distance from all outer contour points of the second water surface area mask MaskB to all outer contour points of the first water surface area mask MaskA. If the outer contour points of the second water surface area mask MaskB are within the first water surface area mask MaskA, the minimum distance is negative; otherwise, the minimum distance is positive.

[0094] include,

[0095] Count the number of minimum distances that are negative and the number of minimum distances that are positive. Based on the number of minimum distances that are negative and the number of minimum distances that are positive, determine whether the water level of the second water surface area mask (MaskB) is rising or falling relative to the water level of the first water surface area mask (MaskA). If the number of minimum distances that are positive is greater than the number of minimum distances that are negative, then the water level is rising; otherwise, the water level is falling.

[0096] If the water level of the second water surface area mask MaskB rises relative to the first water surface area mask MaskA, the new expansion coefficient λ is:

[0097] λ=Max(P)+abs(Min(N));

[0098] If the water level of the second water surface area mask MaskB drops relative to the first water surface area mask MaskA, the new expansion coefficient λ is:

[0099] λ = abs(Min(N));

[0100] Where P represents the maximum positive distance and N represents the minimum negative distance;

[0101] The second water surface range mask MaskB is morphologically expanded using a new expansion coefficient λ to obtain the current water surface floating object detection range Mask_range2.

[0102] Water body segmentation is performed on the base image P_base using the trained semantic segmentation Segformer model. The first water surface range mask MaskA is extracted, and the initial dilation coefficient of 20 is substituted. The update frequency is set to 1 minute. The second water surface range mask Mask is then extracted. The minimum distance from all outer contour points of MaskB to all outer contour points of MaskA is 41. Therefore, the new dilation coefficient λ is 41. Using the new dilation coefficient and MaskB, the current water surface floating object detection range Mask_range2 is obtained.

[0103] S3: Based on the initial focal length map P, identify floating objects using the MaskDINO model; identify the position of the calibration board according to the initial focal length map P; calculate the current perspective transformation matrix according to the position and physical size of the calibration board; use the current perspective transformation matrix to map the floating objects identified by the MaskDINO model onto the physical plane; and calculate the physical area of ​​the floating objects. The floating object identification results need to be filtered by the floating object detection range, and the floating object detection range is updated periodically.

[0104] For the acquired real-time video stream, floating objects within the image are identified at a certain frame rate. The MaskDINO model is used for floating object recognition. Since the PTZ camera has optical zoom capabilities, recognition is based on the initial focal length image without optical zoom magnification. The masking results for floating object recognition are then filtered by the floating object detection range (Mask_range) to remove false detections of floating objects outside the range. Figure 4 As shown, the MaskDINO model provided by this invention, compared with the traditional method, improves the mIOU (mean intersection-union ratio) by 0.45% after 320,000 iterations.

[0105] like Figure 5 As shown, the location of the floating object (red outline) is identified at the initial focal length, and the current perspective transformation matrix is ​​calculated based on the calibration board, identifying the floating object area as 0.45 square meters.

[0106] S4: Use a PTZ camera to optically zoom and capture images of the floating object monitoring area, obtaining a zoomed image P_zoom. It should be noted that:

[0107] The acquisition of the zoom image P_zoom includes,

[0108] The water surface area is divided into regions according to a certain pattern, and four zoom magnification preset positions are set. Based on the area ratio of floating objects in each sub-region, the most suitable preset position is selected to zoom and capture zoom images P_zoom.

[0109] Figure 3 A schematic diagram for setting preset positions for the PTZ camera is provided. Zoom areas 1-4 are in the corresponding areas, and the PTZ camera focuses and magnifies according to the corresponding preset positions.

[0110] S5: Based on the zoom image P_zoom, identify the type and location of floating objects, and match the initial focal length image P with the zoom image P_zoom identification results. It should be noted that:

[0111] The operations for identifying the type and location of floating objects based on the zoom image P_zoom include,

[0112] Based on the image P_zoom, the floating object recognition model is used for secondary recognition.

[0113] Figure 6 In the optical zoom magnification image, the location of the floating object (red outline) is identified and classified as household waste.

[0114] The operation of matching the initial focal length image P and the zoom image P_zoom recognition results includes:

[0115] Identify the floating object masks of the initial focal length image P and the zoom image P_zoom, and extract the ROI image of the original image corresponding to each connected component in the floating object mask;

[0116] Figure 8 This represents the matching result between the initial focal length image P and the zoom image P_zoom.

[0117] Extract the HSV components of the ROI image and generate an HS component histogram. Normalize the number of pixels in the HS component histogram. The size of the HS component histogram is set to [18, 8], and the pixel value range is set to [[0, 180], [0, 256]].

[0118] Traverse the histograms of the ROI images in the initial focal length image P, and calculate the ROI image with the highest similarity to each identified floating object in the ROI images of the initial focal length image P and the zoom image P_zoom (e.g., Figures 10 to 13 As shown in the figure, the ROI classification and recognition results in the zoom image P_zoom are used as the corresponding ROI classification results in the initial focal length image P.

[0119] The HISTCMP_CORREL correlation matching algorithm (a correlation matching algorithm in OpenCV used for histogram comparison, which measures similarity by calculating the correlation coefficient between two histograms) was used to assess similarity. Perform calculations.

[0120] ;

[0121] Where I represents the index of the histogram, The histogram representing the ROI image of the initial focal length image P. This represents the histogram mean of the initial focal length plot P. The histogram representing the ROI image of the zoom image P_zoom. This represents the histogram mean of the zoom image P_zoom.

[0122] S6: By acquiring the floating object pixel detection positions in the initial focal length image P, the floating objects are tracked using the sort algorithm. Simultaneously, the relative physical detection positions of the floating objects are calculated by calibrating the current perspective transformation matrix, thus determining the floating object's velocity. It should be noted that:

[0123] Based on the pixel coordinates of the target center point of the floating object appearing in the first moment of image A, the virtual physical plane coordinates (phy_x1, phy_x1) are obtained using the current perspective transformation matrix, in meters;

[0124] Based on the pixel coordinates of the center point of the floating object at time B, the virtual physical plane coordinates (phy_x2, phy_x2) are obtained using the current perspective transformation matrix.

[0125] Calculate the velocity of the floating material at time B:

[0126] ;

[0127] Multiple frames may be captured of floating objects within the field of view, and the final output is based on the average velocity of the floating objects identified multiple times.

[0128] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An image recognition based water surface floating object classification and measurement method, characterized by, The method comprises the following steps: placing a calibration board on the water surface to be monitored, collecting an image base P_base and an initial focal length image P, and calibrating the pixel feature points of the four vertices of the calibration board; collecting a water area image, labeling a water mask and a floating object mask, identifying the water mask using a semantic segmentation Segformer model, generating a floating object detection range, and training a floating object identification model using the floating object mask and a MaskDINO model; based on the initial focal length image P, identifying the floating object using the MaskDINO model; identifying the position of the calibration board according to the initial focal length image P, calculating the current perspective transformation matrix according to the position of the calibration board and the physical size of the calibration board, mapping the floating object identified by the MaskDINO model to the physical plane using the current perspective transformation matrix, and calculating the physical area of the floating object, wherein the floating object identification result needs to be filtered using the floating object detection range, and the floating object detection range is updated regularly; using a ball machine to optically zoom in on the floating object monitoring area to capture a zoomed image P_zoom; based on the zoomed image P_zoom, identifying the type and position of the floating object, and matching the identification results of the initial focal length image P and the zoomed image P_zoom; by collecting the floating object pixel detection position of the initial focal length image P, tracking the floating object using a sort algorithm, and converting the relative physical detection position of the floating object by using the current perspective transformation matrix, the flow rate of the floating object is obtained; wherein the generation of the floating object detection range comprises, The water body segmentation is performed on the image base P_base, the segmentation adopts a trained semantic segmentation Segformer model, a first water surface range mask MaskA is extracted, and an initial inflation coefficient is used A morphological inflation operation is performed on the first water surface range mask MaskA to obtain a water surface floating object detection range Mask_range1 at an initial time. setting an update frequency, segmenting the collected image using a semantic segmentation Segformer algorithm, and extracting a second water surface range mask MaskB; calculating the minimum distance from all the contour points of the second water surface range mask MaskB to all the contour points of the first water surface range mask MaskA, if the contour points of the second water surface range mask MaskB are within the first water surface range mask MaskA, the minimum distance is negative, otherwise, the minimum distance is positive.

2. The image recognition based water surface floating object classification and measurement method according to claim 1, characterized in that: comprises, counting the values of the minimum distance being negative and the values of the minimum distance being positive, and determining whether the water level of the second water surface range mask MaskB rises or falls relative to the water level of the first water surface range mask MaskA according to the values of the minimum distance being negative and the values of the minimum distance being positive, if the values of the minimum distance being positive are greater than the values of the minimum distance being negative, the water level rises, otherwise, the water level falls; if the water level of the second water surface range mask MaskB rises relative to the water level of the first water surface range mask MaskA, the new dilation coefficient λ is: λ= Max(P)+ abs(Min(N)); if the water level of the second water surface range mask MaskB falls relative to the water level of the first water surface range mask MaskA, the new dilation coefficient λ is: λ= abs(Min(N)); wherein P represents the maximum positive distance, and N represents the minimum negative distance; performing a morphological dilation operation on the second water surface range mask MaskB using the new dilation coefficient λ to obtain the current water surface floating object detection range Mask_range2.

3. The image recognition based water surface floating object classification and measurement method according to claim 1, characterized in that: The identification of the position of the calibration board comprises, Cut out the calibration reference frame target circumscribed rectangle ROI image imgA_ROI from the base map P_base, and expand the circumscribed rectangle based on the four vertex pixel feature point positions by 30 pixels. Perform template matching between imgA_ROI and the initial focal length map P; According to the obtained ROI region, take the center point and use the open source SAM2 model for segmentation to obtain a calibration plate binary mask image; For the calibration plate binary mask image, use the OPENCV algorithm library convexHull convex hull calculation function to obtain a convex polygon vertex coordinate point set; Use the OPENCV algorithm library approxPolyDP polygon approximation function to process the point set, reduce the number of contour points, and maintain the shape approximation to obtain a quadrilateral vertex coordinate set; Sort and match the quadrilateral vertex coordinate set with the physical coordinates to obtain the four pixel feature points of the calibration plate. Based on the four pixel points and the corresponding physical coordinate points, calculate the current perspective transformation matrix. If multiple calibration plates are placed on the water surface, the position of each calibration plate is identified and the longitudinal distance of each floating object in the image is selected.

4. The image recognition based water surface floating object classification and measurement method according to claim 3, characterized in that: The acquisition of the zoom map P_zoom includes, Divide the water surface area according to a certain rule, and preset four zoom-in preset positions. According to the area proportion of the floating object in each sub-region, select the most suitable preset position for zoom-in and capture the zoom map P_zoom.

5. The image recognition based water surface floating object classification and measurement method according to claim 4, characterized in that: The operation of identifying the floating object type and position based on the zoom map P_zoom includes, Based on the image P_zoom, use the floating object recognition model for secondary recognition.

6. The image recognition based water surface floating object classification and measurement method according to claim 5, characterized in that: The operation of matching the initial focal length map P and the zoom map P_zoom recognition result includes, Identify the floating object mask of the initial focal length map P and the zoom map P_zoom, and extract the ROI image of each connected domain in the floating object mask from the original image. Extract the HSV component of the ROI image and generate an HS component histogram. Normalize the pixel number of the HS component histogram. Traverse the histogram of the ROI image of the initial focal length map P, calculate the similarity between the ROI image of the initial focal length map P and each identified floating object in the zoom map P_zoom, and take the classification recognition result of the corresponding ROI in the zoom map P_zoom as the classification result of the corresponding ROI in the initial focal length map P.

7. The image recognition based water surface floating object classification and measurement method according to claim 6, characterized in that: The similarity is calculated using a HISTCMP_CORREL correlation matching algorithm ​ ； where I denotes the index of the histogram, denotes the histogram of the ROI image of the initial focus map P, denotes the histogram mean value of the initial focus map P, denotes the histogram of the ROI image of the zoom map P_zoom, denotes the histogram mean value of the zoom map P_zoom.

Images