Virtual water gauge water level identification method based on multi-mask matching segmentation network

Abstract

The invention discloses a virtual water gauge water level identification method based on a multi-mask matching segmentation network. The method comprises the steps that S1, a real-time image of a target water body area is acquired through a preset position camera; S2, a virtual water gauge geometric model is constructed; s3, extracting a region with high confidence as a candidate region of the segmentation model; s4, designing a new training data structure optimization method to help the model better understand and utilize information of different levels; s5, respectively carrying out maximum matching on the rows and the columns, finding out an optimized matching path with the minimum total cost, and obtaining a finer segmentation result; s6, identifying a water body boundary in the high-precision water body segmentation map through a Canny edge detection algorithm, and extracting a waterline position; and S7, calculating an actual water level value, and for the water body with dynamic movement, smoothing the water level reading by adopting an interval time sampling moving average method. The defects of a traditional physical water gauge and an existing image processing technology are overcome, and an innovative solution is provided for water level monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of water level recognition, and in particular to a virtual water gauge water level recognition method based on a multi-mask matching segmentation network. Background Art

[0002] In the field of water level monitoring and hydrological analysis, accurate water level measurement is of great significance for flood warning, irrigation management, water resource scheduling, etc. Traditional semantic segmentation algorithms have certain limitations in water body segmentation tasks. For example, although the open source PaddleSeg semantic segmentation model performs well in conventional scenarios, its segmentation results are not accurate and robust in complex water environments, and it is difficult to meet the requirements of high-precision water level recognition. To this end, researchers proposed an improved Segment Anything Model (SAM) algorithm, which improves the accuracy and robustness of water body segmentation by introducing technologies such as high-confidence image point guidance and multi-scale feature fusion. However, in practical applications, how to effectively combine the improved algorithms to form a systematic water level recognition optimization method still requires further research and exploration. Summary of the invention

[0003] One purpose of the present invention is to propose a virtual water gauge water level recognition method based on a multi-mask matching segmentation network. The present invention overcomes the shortcomings of traditional physical water gauges and existing image processing technology and provides an innovative solution for water level monitoring.

[0004] A virtual water gauge water level recognition method based on a multi-mask matching segmentation network according to an embodiment of the present invention comprises the following steps:

[0005] S1, obtaining a real-time image of the target water area through a preset position camera, and preprocessing the real-time image using image processing technology;

[0006] S2, perform multi-point calibration on the real-time image and construct a virtual water gauge geometric model;

[0007] S3, use the PaddleSeg semantic segmentation model to perform preliminary segmentation and extract areas with high confidence as candidate areas for the segmentation model;

[0008] S4. Design new training data;

[0009] S5, taking the training data obtained in S4 and the candidate regions obtained in S3 as input, a new segmentation algorithm based on multi-mask matching is used to extract image features through the feature extractor Transformer, perform many-to-many matching between different ground truths and masks, perform maximum matching on rows and columns respectively, find the optimized matching path with the minimum total cost, and obtain a more refined segmentation result;

[0010] S6, identifying the water body boundary in the segmentation result through the Canny edge detection algorithm and extracting the waterline position;

[0011] S7. Calculate the actual water level value using the intersection of the virtual water gauge scale and the waterline position. For water bodies with dynamic movement, use the interval time sampling sliding average method to smooth the water level readings.

[0012] Optionally, S1 includes the following sub-steps:

[0013] S11. Install pre-positioned cameras at specific locations in the target water area: select appropriate installation locations by conducting a detailed analysis of the regional terrain and monitoring requirements to ensure that the camera image can cover the lowest and highest water levels of the target water body;

[0014] S12, obtain the real-time image of the target water area through the preset position camera {I t}: The camera transmits the real-time video stream to the monitoring center through a wired or wireless network. After the monitoring center performs frame extraction processing, the image is stored in the database, where I t represents the image acquired at time point t;

[0015] S13, the real-time image {I t}Preprocessing:

[0016] Projection distortion processing: Detect whether there is projective distortion in the virtual water gauge preset position image, and determine whether correction is needed. For images with large projective distortion, select multiple pairs of projective reference points P i Perform calibration and calculate the 3×3 projection transformation matrix H;

[0017] Use the projection transformation matrix H to correct the distorted image and get the corrected image I t ';

[0018] Denoising: Use Gaussian filtering algorithm to smooth the image and remove high-frequency noise in the image;

[0019] Grayscale transformation: grayscale transformation is performed on the real-time image after denoising, and the color image is converted into a grayscale image;

[0020] Edge enhancement: Perform edge enhancement on the grayscale transformed image, use the Sobel edge detection algorithm to quickly process the image, and obtain an edge-enhanced image sequence

[0021] Optionally, S2 includes the following sub-steps:

[0022] The S2 comprises the following sub-steps:

[0023] S21, in real-time image {I t} to calibrate the position and scale information of the virtual water ruler and generate a calibration image containing scale information {I t ,G t}, use image processing algorithms to automatically identify the virtual water ruler in the real-time image, calibrate its geometric position in the image and the position of each scale line, and superimpose the calibration information on the real-time image to generate a calibration image, where G t Indicates real-time image I t The geometric position and scale information of the virtual water ruler;

[0024] S22, selecting multiple reference points in the image for calibration by a multi-point calibration method, establishing a mapping relationship between the spatial coordinate system of the virtual water gauge and the image coordinate system, selecting multiple obvious and stable reference points in the image, recording the pixel coordinates and actual spatial coordinates of each reference point in the image, and calculating the mapping relationship between the spatial coordinate system of the virtual water gauge and the image coordinate system by a multi-point calibration method;

[0025] S23, constructing a geometric model of the virtual water ruler by calculating the image coordinates of each reference point at different time points {P i}, where P i Represents the spatial coordinates of the i-th reference point.

[0026] Optionally, S3 includes the following sub-steps:

[0027] S31, the image sequence after denoising, grayscale conversion and edge enhancement processing Input into the improved PaddleSeg semantic segmentation model. The improved PaddleSeg semantic segmentation model adds multi-scale feature extraction and context attention fusion. The multi-scale feature extraction formula is as follows:

[0028]

[0029] in, represents the feature map of the kth scale, and b k are the convolution kernel weight and bias term respectively, (i, j) represents the offset of the convolution kernel in space;

[0030] S32, fuse multi-scale features through the contextual attention mechanism to generate a segmentation probability map P t :

[0031] A t (x,y)=∑ m,n softmax·V t (m,n);

[0032]

[0033] Among them, σ is the sigmoid activation function, V t (m,n) represents the feature map of the pixel value (m,n) at time point t, α k is the weight of the k-th scale feature, P t (x,y) represents the probability that the pixel point (x,y) belongs to the water area;

[0034] S33, in the generated segmentation probability map P t Based on this, the threshold θ is set by an adaptive threshold adjustment algorithm based on image features. t :

[0035]

[0036] Among them, μ t P is the segmentation probability map t , λ is the control parameter, N is the total number of pixels in the image, which is used to adjust the sensitivity of the threshold;

[0037] Binarize the segmentation result to obtain the preliminary segmentation result image S t :

[0038]

[0039] S34, the preliminary segmentation result image S t Post-processing is performed, and morphological operations based on connected domain analysis are used to eliminate isolated pixels and small noise areas to obtain preliminary segmentation results.

[0040] S35, find the minimum outer rectangular frame of the polygonal area. Since each connected area obtained in S34 represents an independent high-confidence area, and the high-confidence area is polygonal, for the connected area k, assume that the bounding box is

[0041] S36, add noise to the Ground Truth to simulate inaccurate candidate box input, and use the noise box as a spatial cue for the decoder. Add noise to the Ground Truth to simulate inaccurate candidate box input, and set the noise box to The noise boxes will serve as spatial cues for the decoder;

[0042] Among them, ∈ k Represents noise, which is generated by Gaussian noise or other noise models. Noise boxes are used during model training. As input, the model is trained to reconstruct the original box and mask given the noise box, and the reconstructed box output by the model is set to and the reconstruction mask is

[0043]

[0044] In the process of matching the candidate region with GroundTruth, a certain buffer δ is added.

[0045] Optionally, S4 includes the following sub-steps:

[0046] S41. Provide multiple GroundTruth annotations of different granularities for each candidate region;

[0047] S42, extract features from each GroundTruth annotation;

[0048] S43, performing cluster analysis on the extracted features, and clustering GroundTruth annotations with similar features into the same group;

[0049] S44, reorganizing the data structure based on the clustering results so that the GroundTruth annotations in the same group have similar characteristics;

[0050] S45. According to the clustering grouping result, the GroundTruth annotations are grouped according to similarity.

[0051] Optionally, S5 includes the following sub-steps:

[0052] S51, reorganize the data D obtained in S4 i Input to the segmentation model;

[0053] S52, introduced different semantic annotations, including multi-granularity segmentation, partial segmentation, and universal segmentation, and used a text encoder to encode the three text annotations for performing full and partial segmentation;

[0054] S53, capturing masks of different granularities;

[0055] S54, the candidate region b obtained in S3 k Encoded as a location cue and combined with X different textual cues, where each content cue is represented as a trainable embedding vector at a given level of granularity;

[0056] S55. Construct a weight matrix W, where each element W ij Represents a prediction mask P i With a GroundTruthG j The similarity score between them is calculated based on the overlapping area, shape similarity, and boundary matching indicators;

[0057] S56. Subtract the minimum value from each row and each column so that each row and each column in the matrix has at least one zero element;

[0058] S57. Find a maximum match of zero elements in the matrix, and find a way so that at most one zero element is selected in each row and column;

[0059] S58. Find the solution with the minimum total matching cost by optimizing the matching path through the Hungarian algorithm;

[0060] S59, query feature Q, candidate region and the classification label vector F CLS_token Input the Decoder of the segmentation model and get the output feature O corresponding to each Q t :

[0061] O t =Decoder(Q,b k ,F CLS_token );

[0062] Among them, O t =(o1,o2,…,o X ) means that for each q i , the output o obtained by the model i ;

[0063] S510, each o i =(c i ,m i ) contains the predicted semantic category c i and mask m i , which are used to construct classification loss and mask prediction loss respectively. The classification loss is mainly used to evaluate the accuracy of the model in identifying the object category of the candidate area:

[0064]

[0065] Among them, y i is the true category label, is the predicted category probability, N is the number of categories, and the mask loss is used to evaluate the accuracy of the model for semantic segmentation. The present invention adopts Dice loss:

[0066]

[0067] Where M represents the total number of pixels, p i Indicates the value of the i-th pixel predicted by the model, usually the predicted probability value, ranging between [0,1], g i Represents the value of the i-th pixel in the Ground Truth, indicating the true label;

[0068] The total loss consists of a weighted sum of the classification loss and the mask prediction loss to ensure that the model is optimized in both category prediction and mask prediction:

[0069] L total =αL cls +βL mask ;

[0070] Where α and β are weight coefficients used to balance the impact of classification loss and mask loss.

[0071] Optionally, S6 includes the following sub-steps:

[0072] S61, the generated accurate water body segmentation map O t Perform Gaussian filtering;

[0073] S62, the result after Gaussian filtering The Sobel operator is used to calculate the gradient magnitude and direction of the image in the horizontal and vertical directions respectively:

[0074]

[0075] Among them, G x and G y Represents the Sobel convolution kernel in the horizontal and vertical directions;

[0076] S63, performing non-maximum suppression on the gradient amplitude, retaining the local maximum value, and suppressing the non-edge part;

[0077] S64, apply dual threshold detection to divide the gradient amplitude into strong edge, weak edge and non-edge, and set the high threshold T h and low threshold T l :

[0078]

[0079] S65, by connecting the strong edge and the weak edge, forming the final edge graph B t (x,y);

[0080] S66. Perform morphological processing on the edge detection results, use dilation and erosion operations to eliminate noise and small edge discontinuities, and obtain a processed boundary image

[0081] S67, after processing the boundary image In the ,connected domain analysis algorithm is used to identify continuous boundary segments and extract the complete water body boundary;

[0082] S68, fitting the identified water body boundary, and using a polynomial fitting method to generate a waterline position W t .

[0083] Optionally, S7 includes the following sub-steps:

[0084] S71, extract the waterline position W t With virtual water gauge scale information G t Match and identify the intersection of the waterline position and the virtual water ruler scale

[0085] S72. Calculate the waterline position W at each time point t t The height h in the image coordinate system t (x);

[0086] S73, using the geometric calibration model P of the virtual water gauge i and the waterline height h in the image coordinate system t (x), calculate the actual water level value L t If there are abnormal fluctuations in the water body, the water level readings are smoothed using the sliding average method of interval sampling:

[0087] S74, the water level value L at each time point t calculated t Perform time interval sampling, set the sampling interval Δt, and obtain the sampling point sequence

[0088] S75, sampling point sequence Perform sliding average processing, set the sliding window size N, and calculate the smoothed water level value

[0089] S76, smoothed water level value Conduct analysis to detect water level trends and abnormal fluctuations and generate smooth water level curves.

[0090] The beneficial effects of the present invention are:

[0091] The present invention proposes a new virtual water gauge water level recognition scheme, including: obtaining candidate areas through the fuzzy matching PaddleSeg method, then using a many-to-many matching segmentation algorithm to obtain accurate segmentation areas, and finally obtaining an accurate water level line and then a water level value through the Canny edge algorithm.

[0092] The present invention proposes a many-to-many matching water body segmentation scheme for matching candidate region masks with ground truths. Different from the traditional one-to-one and many-to-one matching methods of segmentation models, the present invention uses the Hungarian algorithm to achieve matching of multiple predicted masks and multiple ground truths for each candidate region by constructing a weight matrix and finding the optimal matching path, thereby improving the accuracy of the segmentation model.

[0093] The present invention proposes a PaddleSeg method based on fuzzy matching, which improves the accuracy and robustness of candidate regions in a multi-mask matching segmentation model by adding noise boxes in the Ground Truth as spatial cues for the decoder and using learnable markers instead of real label embedding. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0095] Figure 1 A flowchart of a virtual water gauge water level recognition method based on a multi-mask matching segmentation network proposed by the present invention;

[0096] Figure 2 The present invention provides an algorithm flow framework for a virtual water gauge water level recognition method based on a multi-mask matching segmentation network;

[0097] Figure 3 This is a diagram of the segmentation network structure based on multi-mask matching proposed in the present invention;

[0098] Figure 4 This is the water level measurement effect of the virtual water gauge proposed in the present invention (see Figure b), and the comparison between the segmentation model based on multi-mask matching and the improved segmentation model (see Figures c and d). DETAILED DESCRIPTION

[0099] The present invention will now be described in further detail with reference to the accompanying drawings. The accompanying drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0100] refer to Figure 1-Figure 4 , a virtual water gauge water level recognition method based on a multi-mask matching segmentation network, comprising the following steps:

[0101] S1, obtaining a real-time image of the target water area through a preset position camera, and preprocessing the real-time image using image processing technology;

[0102] S2, perform multi-point calibration on the real-time image and construct a virtual water gauge geometric model;

[0103] S3, use the PaddleSeg semantic segmentation model to perform preliminary segmentation and extract areas with high confidence as candidate areas for the segmentation model;

[0104] S4. Design new training data;

[0105] S5, taking the training data obtained in S4 and the candidate regions obtained in S3 as input, a new segmentation algorithm based on multi-mask matching is used to extract image features through the feature extractor Transformer, perform many-to-many matching between different GroundTruth and masks, perform maximum matching on rows and columns respectively, find the optimized matching path with the minimum total cost, and obtain a more refined segmentation result;

[0106] S6, identifying the water body boundary in the segmentation result through the Canny edge detection algorithm and extracting the waterline position;

[0107] S7. Calculate the actual water level value using the intersection of the virtual water gauge scale and the waterline position. For water bodies with dynamic movement, use the interval time sampling sliding average method to smooth the water level readings.

[0108] In this implementation, S1 includes the following sub-steps:

[0109] S11. Install pre-positioned cameras at specific locations in the target water area: select appropriate installation locations by conducting a detailed analysis of the regional terrain and monitoring requirements to ensure that the camera image can cover the lowest and highest water levels of the target water body;

[0110] S12, obtain the real-time image of the target water area through the preset position camera {I t}: The camera transmits the real-time video stream to the monitoring center through a wired or wireless network. After the monitoring center performs frame extraction processing, the image is stored in the database, where I t represents the image acquired at time point t;

[0111] S13, the real-time image {I t}Preprocessing:

[0112] Projection distortion processing: Detect whether there is projective distortion in the virtual water gauge preset position image, and determine whether correction is needed. For images with large projective distortion, select multiple pairs of projective reference points P i Perform calibration and calculate the 3×3 projection transformation matrix H;

[0113] Use the projection transformation matrix H to correct the distorted image and get the corrected image I t ';

[0114] Denoising: Use Gaussian filtering algorithm to smooth the image and remove high-frequency noise in the image;

[0115] Grayscale transformation: grayscale transformation is performed on the real-time image after denoising, and the color image is converted into a grayscale image;

[0116] Edge enhancement: Perform edge enhancement on the grayscale transformed image, use the Sobel edge detection algorithm to quickly process the image, and obtain an edge-enhanced image sequence

[0117] In this implementation, S2 includes the following sub-steps:

[0118] S2 includes the following sub-steps:

[0119] S21, in real-time image {I t} to calibrate the position and scale information of the virtual water ruler and generate a calibration image containing scale information {I t ,G t}, use image processing algorithms to automatically identify the virtual water ruler in the real-time image, calibrate its geometric position in the image and the position of each scale line, and superimpose the calibration information on the real-time image to generate a calibration image, where G t Indicates real-time image I t The geometric position and scale information of the virtual water ruler;

[0120] S22, selecting multiple reference points in the image for calibration by a multi-point calibration method, establishing a mapping relationship between the spatial coordinate system of the virtual water gauge and the image coordinate system, selecting multiple obvious and stable reference points in the image, recording the pixel coordinates and actual spatial coordinates of each reference point in the image, and calculating the mapping relationship between the spatial coordinate system of the virtual water gauge and the image coordinate system by a multi-point calibration method;

[0121] S23, constructing a geometric model of the virtual water ruler by calculating the image coordinates of each reference point at different time points {P i}, where P i Represents the spatial coordinates of the i-th reference point.

[0122] In this implementation, S3 includes the following sub-steps:

[0123] S31, the image sequence after denoising, grayscale conversion and edge enhancement processing Input into the improved PaddleSeg semantic segmentation model. The improved PaddleSeg semantic segmentation model adds multi-scale feature extraction and context attention fusion. The multi-scale feature extraction formula is as follows:

[0124]

[0125] in, represents the feature map of the kth scale, and b k are the convolution kernel weight and bias term respectively, (i, j) represents the offset of the convolution kernel in space;

[0126] S32, fuse multi-scale features through the contextual attention mechanism to generate a segmentation probability map P t :

[0127] A t (x,y)=∑ m,n softmax·V t (m,n);

[0128]

[0129] Among them, σ is the sigmoid activation function, V t (m,n) represents the feature map of the pixel value (m,n) at time point t, α k is the weight of the k-th scale feature, P t (x,y) represents the probability that the pixel point (x,y) belongs to the water area;

[0130] S33, in the generated segmentation probability map P t Based on this, the threshold θ is set by an adaptive threshold adjustment algorithm based on image features. t :

[0131]

[0132] Among them, μ t P is the segmentation probability map t , λ is the control parameter, N is the total number of pixels in the image, which is used to adjust the sensitivity of the threshold;

[0133] Binarize the segmentation result to obtain the preliminary segmentation result image S t :

[0134]

[0135] S34, the preliminary segmentation result image S t Post-processing is performed, and morphological operations based on connected domain analysis are used to eliminate isolated pixels and small noise areas to obtain preliminary segmentation results.

[0136] S35, find the minimum outer rectangular frame of the polygonal area. Since each connected area obtained in S34 represents an independent high-confidence area, and the high-confidence area is polygonal, for the connected area k, assume that the bounding box is

[0137] S36, add noise to the Ground Truth to simulate inaccurate candidate box input, and use the noise box as a spatial cue for the decoder. Add noise to the Ground Truth to simulate inaccurate candidate box input, and set the noise box to The noise boxes will serve as spatial cues for the decoder;

[0138] Among them, ∈ k Represents noise, which is generated by Gaussian noise or other noise models. Noise boxes are used during model training. As input, the model is trained to reconstruct the original box and mask given the noise box, and the reconstructed box output by the model is set to and the reconstruction mask is

[0139]

[0140] In the process of matching the candidate region with the Ground Truth, a certain buffer δ is added.

[0141] In this implementation, S4 includes the following sub-steps:

[0142] S41. Provide multiple Ground Truth annotations of different granularities for each candidate region;

[0143] S42, extract features from each Ground Truth annotation;

[0144] S43, performing cluster analysis on the extracted features, and clustering Ground Truth annotations with similar features into the same group;

[0145] S44. Reorganize the data structure based on the clustering results so that the Ground Truth annotations in the same group have similar features;

[0146] S45. According to the clustering grouping result, the Ground Truth annotations are grouped according to similarity.

[0147] In this implementation, S5 includes the following sub-steps:

[0148] S51, reorganize the data D obtained in S4 i Input to the segmentation model;

[0149] S52, introduced different semantic annotations, including multi-granularity segmentation, partial segmentation, and universal segmentation, and used a text encoder to encode the three text annotations for performing full and partial segmentation;

[0150] S53, capturing masks of different granularities;

[0151] S54, the candidate region b obtained in S3 k Encoded as a location cue and combined with X different textual cues, where each content cue is represented as a trainable embedding vector at a given level of granularity;

[0152] S55. Construct a weight matrix W, where each element W ij Represents a prediction mask P i With a GroundTruthG j The similarity score between them is calculated based on the overlapping area, shape similarity, and boundary matching indicators;

[0153] S56. Subtract the minimum value from each row and each column so that each row and each column in the matrix has at least one zero element;

[0154] S57. Find a maximum match of zero elements in the matrix, and find a way so that at most one zero element is selected in each row and column;

[0155] S58. Find the solution with the minimum total matching cost by optimizing the matching path through the Hungarian algorithm;

[0156] S59, query feature Q, candidate region and the classification label vector F CLS_token Input the Decoder of the segmentation model and get the output feature O corresponding to each Q t :

[0157] O t =Decoder(Q,b k ,F CLS_token );

[0158] Among them, O t =(o1,o2,…,o X ) means that for each q i , the output o obtained by the model i ;

[0159] S510, each o i =(c i ,m i ) contains the predicted semantic category c i and mask m i , which are used to construct classification loss and mask prediction loss respectively. The classification loss is mainly used to evaluate the accuracy of the model in identifying the object category of the candidate area:

[0160]

[0161] Among them, y i is the true category label, is the predicted category probability, N is the number of categories, and the mask loss is used to evaluate the accuracy of the model for semantic segmentation. The present invention adopts Dice loss:

[0162]

[0163] Where M represents the total number of pixels, p i Indicates the value of the i-th pixel predicted by the model, usually the predicted probability value, ranging between [0,1], g i Represents the value of the i-th pixel in the Ground Truth, indicating the true label;

[0164] The total loss consists of a weighted sum of the classification loss and the mask prediction loss to ensure that the model is optimized in both category prediction and mask prediction:

[0165] L total =αL cls +βL mask ;

[0166] Where α and β are weight coefficients used to balance the impact of classification loss and mask loss.

[0167] In this implementation, S6 includes the following sub-steps:

[0168] S61, the generated accurate water body segmentation map O t Perform Gaussian filtering;

[0169] S62, the result after Gaussian filtering The Sobel operator is used to calculate the gradient magnitude and direction of the image in the horizontal and vertical directions respectively:

[0170]

[0171] Among them, G x and G y Represents the Sobel convolution kernel in the horizontal and vertical directions;

[0172] S63, performing non-maximum suppression on the gradient amplitude, retaining the local maximum value, and suppressing the non-edge part;

[0173] S64, apply dual threshold detection to divide the gradient amplitude into strong edge, weak edge and non-edge, and set the high threshold T h and low threshold T l :

[0174]

[0175] S65, by connecting the strong edge and the weak edge, forming the final edge graph B t (x,y);

[0176] S66. Perform morphological processing on the edge detection results, use dilation and erosion operations to eliminate noise and small edge discontinuities, and obtain a processed boundary image

[0177] S67, after processing the boundary image In the ,connected domain analysis algorithm is used to identify continuous boundary segments and extract the complete water body boundary;

[0178] S68, fitting the identified water body boundary, and using a polynomial fitting method to generate a waterline position W t .

[0179] In this implementation, S7 includes the following sub-steps:

[0180] S71, extract the waterline position W t With virtual water gauge scale information G t Match and identify the intersection of the waterline position and the virtual water ruler scale

[0181] S72. Calculate the waterline position W at each time point t t The height h in the image coordinate system t (x);

[0182] S73, using the geometric calibration model P of the virtual water gauge i and the waterline height h in the image coordinate system t (x), calculate the actual water level value L t If there are abnormal fluctuations in the water body, the water level readings are smoothed using the sliding average method of interval sampling:

[0183] S74, the water level value L at each time point t calculated t Perform time interval sampling, set the sampling interval Δt, and obtain the sampling point sequence L ti ;

[0184] S75, sampling point sequence L ti Perform sliding average processing, set the sliding window size N, and calculate the smoothed water level value

[0185] S76, smoothed water level value Conduct analysis to detect water level trends and abnormal fluctuations and generate smooth water level curves.

[0186] The present invention proposes a new virtual water gauge water level recognition scheme, including: obtaining candidate areas through the fuzzy matching PaddleSeg method, then using a many-to-many matching segmentation algorithm to obtain accurate segmentation areas, and finally obtaining an accurate water level line and then a water level value through the Canny edge algorithm.

[0187] The present invention proposes a many-to-many matching water body segmentation scheme for matching candidate region masks with ground truths. Different from the traditional one-to-one and many-to-one matching methods of segmentation models, the present invention uses the Hungarian algorithm to achieve matching of multiple predicted masks and multiple ground truths for each candidate region by constructing a weight matrix and finding the optimal matching path, thereby improving the accuracy of the segmentation model.

[0188] The present invention proposes a PaddleSeg method based on fuzzy matching, which improves the accuracy and robustness of candidate regions in a multi-mask matching segmentation model by adding noise boxes in the Ground Truth as spatial cues for the decoder and using learnable markers instead of real label embedding.

[0189] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A virtual water gauge water level recognition method based on multi-mask matching segmentation network, characterized in that: The following steps are involved: S1, obtaining a real-time image of the target water area through a preset position camera, and preprocessing the real-time image using image processing technology; S2, perform multi-point calibration on the real-time image and construct a virtual water gauge geometric model; S3, use the PaddleSeg semantic segmentation model to perform preliminary segmentation and extract areas with high confidence as candidate areas for the segmentation model; S4. Design new training data; S5, taking the training data obtained in S4 and the candidate regions obtained in S3 as input, a new segmentation algorithm based on multi-mask matching is used to extract image features through the feature extractor Transformer, perform many-to-many matching between different ground truths and masks, perform maximum matching on rows and columns respectively, find the optimized matching path with the minimum total cost, and obtain a more refined segmentation result; S6, identifying the water body boundary in the segmentation result through the Canny edge detection algorithm and extracting the waterline position; S7. Calculate the actual water level value using the intersection of the virtual water gauge scale and the waterline position. For water bodies with dynamic movement, use the interval time sampling sliding average method to smooth the water level readings.

2. According to the method for virtual water gauge water level recognition based on multi-mask matching segmentation network in claim 1, it is characterized in that: The S1 comprises the following sub-steps: S11. Install pre-positioned cameras at specific locations in the target water area: select appropriate installation locations by conducting a detailed analysis of the regional terrain and monitoring requirements to ensure that the camera image can cover the lowest and highest water levels of the target water body; S12, obtain the real-time image of the target water area through the preset position camera {I t }: The camera transmits the real-time video stream to the monitoring center through a wired or wireless network. After the monitoring center performs frame extraction processing, the image is stored in the database, where I t represents the image acquired at time point t; S13, the real-time image {I t }Preprocessing: Projection distortion processing: Detect whether there is projective distortion in the virtual water gauge preset position image, and determine whether correction is needed. For images with large projective distortion, select multiple pairs of projective reference points P i Perform calibration and calculate the 3×3 projection transformation matrix H; Use the projection transformation matrix H to correct the distorted image and get the corrected image I t '; Denoising: Use Gaussian filtering algorithm to smooth the image and remove high-frequency noise in the image; Grayscale transformation: grayscale transformation is performed on the real-time image after denoising, and the color image is converted into a grayscale image; Edge enhancement: Perform edge enhancement on the grayscale transformed image, use the Sobel edge detection algorithm to quickly process the image, and obtain an edge-enhanced image sequence 3. According to the method of virtual water gauge water level recognition based on multi-mask matching segmentation network in claim 1, it is characterized in that: The S2 comprises the following sub-steps: The S2 comprises the following sub-steps: S21, in real-time image {I t } to calibrate the position and scale information of the virtual water ruler and generate a calibration image containing scale information {I t ,G t }, use image processing algorithms to automatically identify the virtual water ruler in the real-time image, calibrate its geometric position in the image and the position of each scale line, and superimpose the calibration information on the real-time image to generate a calibration image, where G t Indicates real-time image I t The geometric position and scale information of the virtual water ruler; S22, selecting multiple reference points in the image for calibration by a multi-point calibration method, establishing a mapping relationship between the spatial coordinate system of the virtual water gauge and the image coordinate system, selecting multiple obvious and stable reference points in the image, recording the pixel coordinates and actual spatial coordinates of each reference point in the image, and calculating the mapping relationship between the spatial coordinate system of the virtual water gauge and the image coordinate system by a multi-point calibration method; S23, constructing a geometric model of the virtual water ruler by calculating the image coordinates of each reference point at different time points {P i }, where P i Represents the spatial coordinates of the i-th reference point.

4. According to the method for virtual water gauge water level recognition based on multi-mask matching segmentation network in claim 1, it is characterized in that: The S3 comprises the following sub-steps: S31, the image sequence after denoising, grayscale conversion and edge enhancement processing Input into the improved PaddleSeg semantic segmentation model. The improved PaddleSeg semantic segmentation model adds multi-scale feature extraction and context attention fusion. The multi-scale feature extraction formula is as follows: in, represents the feature map of the kth scale, and b k are the convolution kernel weight and bias term respectively, (i, j) represents the offset of the convolution kernel in space; S32, fuse multi-scale features through the contextual attention mechanism to generate a segmentation probability map P t : TO t (x,y)=∑ m,n softmax·V t (m,n); Among them, σ is the sigmoid activation function, V t (m,n) represents the feature map of the pixel value (m,n) at time point t, α k is the weight of the k-th scale feature, P t (x,y) represents the probability that the pixel point (x,y) belongs to the water area; S33, in the generated segmentation probability map P t Based on this, the threshold θ is set by an adaptive threshold adjustment algorithm based on image features. t : Among them, μ t P is the segmentation probability map t , λ is the control parameter, N is the total number of pixels in the image, which is used to adjust the sensitivity of the threshold; Binarize the segmentation result to obtain the preliminary segmentation result image S t : S34, the preliminary segmentation result image S t Post-processing is performed, and morphological operations based on connected domain analysis are used to eliminate isolated pixels and small noise areas to obtain preliminary segmentation results. S35, find the minimum outer rectangular frame of the polygonal area. Since each connected area obtained in S34 represents an independent high-confidence area, and the high-confidence area is polygonal, for the connected area k, assume that the bounding box is S36, add noise to the Ground Truth to simulate inaccurate candidate box input, and use the noise box as a spatial cue for the decoder. Add noise to the Ground Truth to simulate inaccurate candidate box input, and set the noise box to The noise boxes will serve as spatial cues for the decoder; Among them, ∈ k Represents noise, which is generated by Gaussian noise or other noise models. Noise boxes are used during model training. As input, the model is trained to reconstruct the original box and mask given the noise box, and the reconstructed box output by the model is set to and the reconstruction mask is In the process of matching the candidate region with the Ground Truth, a certain buffer δ is added.

5. The method for virtual water gauge water level recognition based on multi-mask matching segmentation network according to claim 1 is characterized in that: The S4 comprises the following sub-steps: S41. Provide multiple Ground Truth annotations of different granularities for each candidate region; S42, extract features from each Ground Truth annotation; S43, performing cluster analysis on the extracted features, and clustering Ground Truth annotations with similar features into the same group; S44. Reorganize the data structure based on the clustering results so that the Ground Truth annotations in the same group have similar features; S45. According to the clustering grouping result, the Ground Truth annotations are grouped according to similarity.

6. The method for virtual water gauge water level recognition based on multi-mask matching segmentation network according to claim 1 is characterized in that: The S5 comprises the following sub-steps: S51, reorganize the data D obtained in S4 i Input to the segmentation model; S52, introduced different semantic annotations, including multi-granularity segmentation, partial segmentation, and universal segmentation, and used a text encoder to encode the three text annotations for performing full and partial segmentation; S53, capturing masks of different granularities; S54, the candidate region b obtained in S3 k Encoded as a location cue and combined with X different textual cues, where each content cue is represented as a trainable embedding vector at a given level of granularity; S55. Construct a weight matrix W, where each element W ij Represents a prediction mask P i With a Ground TruthG j The similarity score between them is calculated based on the overlapping area, shape similarity, and boundary matching indicators; S56. Subtract the minimum value from each row and each column so that each row and each column in the matrix has at least one zero element; S57. Find a maximum match of zero elements in the matrix, and find a way so that at most one zero element is selected in each row and column; S58. Find the solution with the minimum total matching cost by optimizing the matching path through the Hungarian algorithm; S59, query feature Q, candidate region and the classification label vector F CLS_token Input the Decoder of the segmentation model and get the output feature O corresponding to each Q t : O t =Decoder(Q,b k ,F CLS_token ); Among them, O t =(o1,o2,…,o X ) means that for each q i , the output o obtained by the model i ; S510, each o i =(c i ,m i ) contains the predicted semantic category c i and mask m i , which are used to construct classification loss and mask prediction loss respectively. The classification loss is mainly used to evaluate the accuracy of the model in identifying the object category of the candidate area: Among them, y i is the true category label, is the predicted category probability, N is the number of categories, and the mask loss is used to evaluate the accuracy of the model for semantic segmentation. The present invention adopts Dice loss: Where M represents the total number of pixels, p i Indicates the value of the i-th pixel predicted by the model, usually the predicted probability value, ranging between [0,1], g i Represents the value of the i-th pixel in the Ground Truth, indicating the true label; The total loss consists of a weighted sum of the classification loss and the mask prediction loss to ensure that the model is optimized in both category prediction and mask prediction: L total =αL cls +βL mask ; Where α and β are weight coefficients used to balance the impact of classification loss and mask loss.

7. The method for virtual water gauge water level recognition based on multi-mask matching segmentation network according to claim 1 is characterized in that: The S6 comprises the following sub-steps: S61, the generated accurate water body segmentation map O t Perform Gaussian filtering; S62, the result after Gaussian filtering The Sobel operator is used to calculate the gradient magnitude and direction of the image in the horizontal and vertical directions respectively: Among them, G x and G y Represents the Sobel convolution kernel in the horizontal and vertical directions; S63, performing non-maximum suppression on the gradient amplitude, retaining the local maximum value, and suppressing the non-edge part; S64, apply dual threshold detection to divide the gradient amplitude into strong edge, weak edge and non-edge, and set the high threshold T h and low threshold T l : S65, by connecting the strong edge and the weak edge, forming the final edge graph B t (x,y); S66. Perform morphological processing on the edge detection results, use dilation and erosion operations to eliminate noise and small edge discontinuities, and obtain a processed boundary image S67, after processing the boundary image In the ,connected domain analysis algorithm is used to identify continuous boundary segments and extract the complete water body boundary; S68, fitting the identified water body boundary, and using a polynomial fitting method to generate a waterline position W t .

8. The method for virtual water gauge water level recognition based on multi-mask matching segmentation network according to claim 1 is characterized in that: The S7 comprises the following sub-steps: S71, extract the waterline position W t With virtual water gauge scale information G t Match and identify the intersection of the waterline position and the virtual water ruler scale S72, calculate the waterline position W at each time point t t The height h in the image coordinate system t (x); S73, using the geometric calibration model P of the virtual water gauge i and the waterline height h in the image coordinate system t (x), calculate the actual water level value L t If there are abnormal fluctuations in the water body, the water level readings are smoothed using the sliding average method of interval sampling: S74, calculate the water level value L at each time point t t Perform time interval sampling, set the sampling interval Δt, and obtain the sampling point sequence S75, sampling point sequence Perform sliding average processing, set the sliding window size N, and calculate the smoothed water level value S76, smoothed water level value Conduct analysis to detect water level trends and abnormal fluctuations and generate smooth water level curves.

Images