Water level monitoring method based on SAM image segmentation model

Abstract

The present invention relates to the field of hydrological measurement technology, and in particular to a water level monitoring method based on a SAM image segmentation model, comprising: spatially correcting a collected water gauge image to obtain a water gauge correction image; establishing a SAM image segmentation model for water gauge image segmentation based on a visual self-attention mechanism; obtaining prompt point information on the water gauge correction image, and inputting the water gauge correction image and prompt point information into the SAM image segmentation model to obtain a water gauge region segmentation result; establishing a water level measurement model between mask pixels and the measured water level, and using the water level measurement model to process the water gauge region segmentation result to obtain a measured water level value. The present invention realizes automatic monitoring of water level through the SAM image segmentation model, solves the problem of poor robustness and applicability of existing image recognition water level methods, improves the accuracy and long-term stability of water level monitoring, and reduces monitoring costs.

Description

Technical Field

[0001] The invention relates to the technical field of hydrological testing, and in particular to a water level monitoring method based on a SAM image segmentation model. Background Art

[0002] Water level measurement is crucial to fields and industries such as flood warning and hydrological forecasting, water resources management, water ecological protection, hydropower generation and shipping. Traditional water level measurement requires dispatching technicians to the site to read the water level with the help of a water gauge. This manual measurement method not only takes a lot of time and personnel training costs, but also brings challenges to water level measurement in river basins with complex terrain or during flood seasons.

[0003] In recent years, non-contact water level measurement methods based on image processing have significantly improved the efficiency and safety of hydrological surveys. Existing image recognition water level methods are mainly divided into two types: one is to extract water level mark lines based on image processing methods such as thresholds or morphology, but the recognition accuracy is limited by the environmental conditions of the observed water area (lighting, shadows, aquatic plants, etc.); the other is to extract water level features based on machine learning methods to realize the simulation calculation of water level height, but due to the sparsity of hydrological observation data, the generalization performance of the model is often limited, and the robustness and applicability are poor. Summary of the invention

[0004] The invention provides a water body water level monitoring method based on the SAM image segmentation model, and solves the technical problem that the existing image water level recognition method has poor robustness and applicability.

[0005] In order to solve the above technical problems, the present invention provides a water level monitoring method based on the SAM image segmentation model.

[0006] In a first aspect, the present invention provides a water level monitoring method based on a SAM image segmentation model, the method comprising the following steps:

[0007] Acquire a water gauge image, and perform spatial correction on the water gauge image to obtain a water gauge corrected image;

[0008] Based on the visual self-attention mechanism, a SAM image segmentation model for water gauge image segmentation is established;

[0009] Acquire the cue point information on the water gauge correction image, and input the water gauge correction image and the cue point information into the SAM image segmentation model to obtain a water gauge area segmentation result;

[0010] A water level measurement model between mask pixels and the measured water level is established, and the water level measurement model is used to process the water gauge area segmentation result to obtain the measured water level value.

[0011] In a further embodiment, the step of performing spatial correction on the water gauge image to obtain a water gauge corrected image comprises:

[0012] Obtaining an average deflection angle of the water gauge edge line or the center axis in the vertical direction, and establishing a rotation matrix according to the average deflection angle;

[0013] Performing an affine transformation on the water gauge image using the rotation matrix to obtain a rotation-corrected image;

[0014] Performing distortion correction on the rotation-corrected image to generate a deformity-corrected image;

[0015] The missing values ​​in the deformity correction image are interpolated using a preset interpolation algorithm to obtain a water gauge correction image.

[0016] In a further embodiment, the SAM image segmentation model includes an input layer, an output layer, and an encoding module and a mask decoding module sequentially connected between the input layer and the output layer, wherein the encoding module includes an image encoding module and a cue point encoding module both connected to the mask decoding module;

[0017] The image encoding module includes an image reorganization vector encoding module, a preset number of image self-attention mechanism modules and two first convolutional layers; each of the image self-attention mechanism modules includes a normalization module connected by residuals, a multi-head self-attention mechanism layer and a first multi-layer perceptron;

[0018] The mask decoding module comprises a first decoding layer, a second decoding layer and a feature output layer connected in sequence, wherein each of the decoding layers comprises an image cross attention mechanism and an information self-attention mechanism, an information cross attention mechanism and a second multi-layer perceptron connected in sequence, wherein the image cross attention mechanism is connected to the second multi-layer perceptron, the image encoding module and the information self-attention mechanism, and the image cross attention mechanism and the second multi-layer perceptron are both connected to the feature output layer;

[0019] The feature output layer includes a sequentially connected fusion cross attention mechanism layer, a second convolutional layer and a third multi-layer perceptron.

[0020] In a further embodiment, the step of inputting the water gauge correction image and the cue point information into a trained SAM image segmentation model to obtain a water gauge area segmentation result comprises:

[0021] Recombining and encoding the water gauge correction image through the image encoding module to obtain water gauge image features;

[0022] Performing superposition encoding on the cue point information by the cue point encoding module to generate superposition features;

[0023] Using two decoding layers to decode the water gauge image features and the superposition features to obtain decoded output features; wherein the decoded output features include water gauge image output features and superposition output features;

[0024] Taking the superimposed output feature as a query vector, the superimposed output feature and the water gauge image output feature are passed through the fusion cross attention mechanism layer to capture external correlation, and transposed convolution is performed through the second convolution layer to obtain the final water gauge image feature;

[0025] The mask feature is separated from the superimposed output feature, the dimension is adjusted by the third multi-layer perceptron and multiplied with the final water gauge image feature to output the water gauge area segmentation result.

[0026] In a further embodiment, the step of recombining and encoding the water gauge correction image by the image encoding module to obtain water gauge image features, and superimposing and encoding the cue point information by the cue point encoding module to generate superimposed features includes:

[0027] The water gauge correction image is convolved or linearly mapped to obtain a sub-block vector, and the sub-block vector is summed with the position vector to obtain an image input feature, so as to encode the image input feature through the image self-attention mechanism module to obtain a water gauge image feature;

[0028] Acquire learnable output features, perform feature extraction on the cue point information to obtain cue point features, and superimpose the learnable output features on the cue point features to generate superimposed features; the learnable output features include intersection-over-union features and mask features, and the intersection-over-union features are used to obtain the intersection-over-union score of the final water gauge area segmentation result.

[0029] In a further embodiment, the two decoding layers are used to decode the water gauge image feature and the superposition feature to obtain a decoded output feature, including:

[0030] In the first decoding layer, the information self-attention mechanism is used to capture the internal correlation of the cue point superimposed features to obtain the superimposed internal update features;

[0031] Taking the superimposed internal updated features as query vectors, capturing the external correlation between the superimposed internal updated features and the water gauge image features through an information cross attention mechanism, and performing feature transformation on the results output by the information cross attention mechanism through a multi-layer perceptron to obtain updated superimposed features;

[0032] Taking the water gauge image feature as a query vector, capturing external correlations between the water gauge image feature and the updated superimposed feature through the image cross attention mechanism, and obtaining an updated water gauge image feature;

[0033] According to the above steps, the updated superposition features and the updated water gauge image features are decoded twice using the second decoding layer to obtain decoded output features.

[0034] In a further embodiment, the water level measurement model is:

[0035] S=Hk(y max -y min )

[0036] In the formula, S represents the measured water level value; H represents the water gauge range; k represents the regression coefficient; y min Indicates the ordinate of the upper boundary of the effective mask in the water gauge area segmentation result; y max Indicates the ordinate of the lower boundary of the effective mask in the water gauge area segmentation result.

[0037] In a second aspect, the present invention provides a water level monitoring device based on the SAM image segmentation model, the device comprising:

[0038] An image correction module is used to collect a water gauge image and perform spatial correction on the water gauge image to obtain a water gauge correction image;

[0039] Model building module, used to build a SAM image segmentation model for water gauge image segmentation based on the visual self-attention mechanism;

[0040] An image segmentation module, used for obtaining the cue point information on the water gauge correction image, and inputting the water gauge correction image and the cue point information into the SAM image segmentation model to obtain a water gauge area segmentation result;

[0041] The water level measurement module is used to establish a water level measurement model between the mask pixels and the measured water level, and use the water level measurement model to process the water gauge area segmentation result to obtain the measured water level value.

[0042] The present invention provides a water level monitoring method for a water body based on a SAM image segmentation model, the method comprising: performing spatial correction on a water gauge image to obtain a water gauge correction image; obtaining prompt point information on the water gauge correction image, and inputting the water gauge correction image and the prompt point information into a SAM image segmentation model with a visual self-attention mechanism as a framework to obtain a water gauge region segmentation result; establishing a water level measurement model between mask pixels and the measured water level, and using the water level measurement model to process the water gauge region segmentation result to obtain a measured water level value. Compared with the prior art, the method uses a SAM image segmentation model with a visual self-attention mechanism as a framework to measure the water level, not only has good segmentation and recognition capabilities for water gauges in various environments or observation angles, avoids the influence of interference factors such as light and aquatic plants, improves the accuracy of the measurement results and the adaptability of the method application, but also reduces the construction difficulty and construction cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic flow chart of a water level monitoring method based on a SAM image segmentation model provided by an embodiment of the present invention;

[0044] Figure 2 is a schematic diagram of a water ruler image with prompt points provided by an embodiment of the present invention;

[0045] Figure 3 It is a schematic diagram of the structure of the SAM image segmentation model provided by an embodiment of the present invention;

[0046] Figure 4 is a schematic diagram of a binary mask in a water gauge area segmentation result provided by an embodiment of the present invention;

[0047] Figure 5 It is a schematic diagram of a water gauge image with prompt points and masks output by the SAM image segmentation model provided in an embodiment of the present invention;

[0048] Figure 6 It is a block diagram of a water level monitoring device based on the SAM image segmentation model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following specifically illustrates the implementation mode of the present invention in conjunction with the accompanying drawings. The embodiments are provided for illustrative purposes only and are not to be construed as limitations of the present invention. The accompanying drawings are provided for reference and illustration only and do not constitute limitations on the scope of patent protection of the present invention, because many changes may be made to the present invention without departing from the spirit and scope of the present invention.

[0050] refer to Figure 1 , the embodiment of the present invention provides a water level monitoring method based on the SAM image segmentation model, such as Figure 1 As shown, the method comprises the following steps:

[0051] S1. Collect a water gauge image, perform spatial correction on the water gauge image, and obtain a water gauge corrected image.

[0052] In this embodiment, a water gauge is vertically installed in the target water area, and a monitoring camera is arranged at a preset distance from the water gauge and at a position where observation is relatively easy. The angle of the monitoring camera is adjusted so that the lens is facing the water gauge, and the picture taken by the monitoring camera is checked to ensure that the entire length of the water gauge is located in the picture and is as close to the center of the picture as possible. The monitoring personnel can clearly see the water gauge scale from the picture and read the water level more accurately. After adjustment in the above steps, the working mode of the monitoring camera is switched to the photo shooting mode, and the shooting interval is set to 10 minutes. The monitoring camera regularly shoots water gauge pictures, which are simultaneously uploaded to the cloud and transmitted to the image processing module for water level calculation. The monitoring personnel or spot checkers can view and download water gauge pictures from the cloud.

[0053] Since the collected water gauge image has problems such as deflection and distortion, the present embodiment needs to first perform spatial correction on the image. In the present embodiment, the step of performing spatial correction on the water gauge image to obtain the water gauge corrected image includes:

[0054] Obtaining an average deflection angle of the water gauge edge line or the center axis in the vertical direction, and establishing a rotation matrix according to the average deflection angle;

[0055] Using the rotation matrix, the water gauge image is affine transformed to rotate it to a vertical direction to obtain a rotation-corrected image;

[0056] Performing distortion correction on the rotation-corrected image to generate a deformity-corrected image;

[0057] The missing values ​​in the deformity correction image are interpolated using a preset interpolation algorithm to obtain a water gauge correction image.

[0058] It should be noted that since the rotation-corrected image after tilt correction still has perspective distortion, this embodiment needs to perform distortion correction according to the distortion parameters inside the camera. The calculation formula for distortion correction is:

[0059]

[0060]

[0061]

[0062] Where (u, v) represents the pixel coordinates of the rotation-corrected image; (x′, y′) represents the pixel coordinates of the distortion-corrected image; A represents the spatial perspective transformation matrix, which is determined by the camera's intrinsic parameter matrix and distortion coefficient; a ijRepresents the elements in the spatial perspective transformation matrix.

[0063] After the water gauge image is distorted and corrected, if there are missing pixel values ​​in some areas, this embodiment copies the pixel value of the nearest non-missing point to the pixel point or uses Lagrange interpolation to obtain an image with missing value repaired.

[0064] S2. Based on the visual self-attention mechanism, a SAM image segmentation model for water gauge image segmentation is established.

[0065] S3. Obtain the prompt point information on the water gauge correction image, and input the water gauge correction image and the prompt point information into the SAM image segmentation model to obtain the water gauge area segmentation result.

[0066] Figure 3 It is a schematic diagram of the structure of the SAM image segmentation model. In this embodiment, the SAM (Segment Anything Model) image segmentation model includes an input layer, an output layer, and a coding module and a mask decoding module connected in sequence between the input layer and the output layer, wherein the coding module includes an image coding module and a prompt point coding module both connected to the mask decoding module; the image coding module includes an image reorganization vector coding module, a preset number of image self-attention mechanism modules, and two first convolutional layers; each of the image self-attention mechanism modules includes a normalization module connected by residuals, a multi-head self-attention mechanism layer, and a first multi-layer perceptron; the mask decoding module includes a first decoding layer, a second decoding layer, and a feature output layer connected in sequence, wherein each of the decoding layers includes an image cross-attention mechanism and an information self-attention mechanism, an information cross-attention mechanism, and a second multi-layer perceptron connected in sequence, wherein the image cross-attention mechanism is connected to the second multi-layer perceptron, the image coding module, and the adaptive attention mechanism, and the image cross-attention mechanism and the second multi-layer perceptron are both connected to the feature output layer; the feature output layer includes a fusion cross-attention mechanism layer, a second convolutional layer, and a third multi-layer perceptron connected in sequence.

[0067] In this embodiment, the step of inputting the water gauge correction image and the cue point information into the constructed SAM image segmentation model to obtain the water gauge area segmentation result includes:

[0068] Recombining and encoding the water gauge correction image through the image encoding module to obtain water gauge image features;

[0069] Performing superposition encoding on the cue point information by the cue point encoding module to generate superposition features;

[0070] Using two decoding layers to decode the water gauge image features and the superposition features to obtain decoded output features; wherein the decoded output features include water gauge image output features and superposition output features;

[0071] Specifically, the superimposed output feature is used as a query vector, the superimposed output feature and the water gauge image output feature are passed through the fused cross attention mechanism layer to capture external correlation, and transposed convolution is performed through the second convolution layer to obtain the final water gauge image feature, the mask feature is separated from the superimposed output feature, the dimension is adjusted by the third multi-layer perceptron and multiplied with the final water gauge image feature, and the water gauge area segmentation result is output, and the intersection-over-union ratio feature is used to obtain the intersection-over-union ratio score of the final water gauge area segmentation result.

[0072] In this embodiment, the step of recombining and encoding the water gauge correction image by the image encoding module to obtain water gauge image features, and superimposing and encoding the cue point information by the cue point encoding module to generate superimposed features includes:

[0073] The water gauge correction image is convolved or linearly mapped to obtain a sub-block vector, and the sub-block vector is summed with the position vector to obtain an image input feature, so as to encode the image input feature through the image self-attention mechanism module to obtain a water gauge image feature;

[0074] The learnable output feature is obtained, and the feature extraction is performed on the prompt point information to obtain the prompt point feature, and the learnable output feature is superimposed on the prompt point feature to generate a superimposed feature; the learnable output feature includes an intersection-over-union feature and a mask feature.

[0075] In this embodiment, two decoding layers are used to decode the water gauge image feature and the superposition feature to obtain a decoded output feature, including:

[0076] In the first decoding layer, the information self-attention mechanism is used to capture the internal correlation of the superimposed features to obtain the superimposed internal update features;

[0077] Taking the superimposed internal updated features as query vectors, capturing the external correlation between the superimposed internal updated features and the water gauge image features through an information cross attention mechanism, and performing feature transformation on the results output by the information cross attention mechanism through a multi-layer perceptron to obtain updated superimposed features;

[0078] Taking the water gauge image feature as a query vector, capturing external correlations between the water gauge image feature and the updated superimposed feature through the image cross attention mechanism, and obtaining an updated water gauge image feature;

[0079] According to the above steps, the updated superposition features and the updated water gauge image features are decoded twice using the second decoding layer to obtain decoded output features.

[0080] Specifically, Figure 2 As shown, in this embodiment, four prompt points are marked on the water gauge correction image, the SAM image segmentation model is input, and the water gauge area segmentation result above the water surface is obtained as output, such as Figure 3 Different from the traditional convolutional neural network, the SAM image segmentation model has been pre-trained by the data engine, and can achieve good image segmentation results in most scenarios without the need to input specific instances for re-training. In this embodiment, the SAM image segmentation model is based on VIT (Vision Transformer, visual self-attention mechanism) model is used as the framework, and prompt engineering technology is introduced to realize accurate segmentation of the target mask. In this embodiment, the size of the water gauge correction image is 3×3648×2736. After inputting the SAM image segmentation model, it is divided into N=38988 sub-blocks of 16×16 size. Each sub-block is encoded into a vector by a convolution or linear projection method. The number of channels changes from 3 to 768. The sub-block images can be spliced ​​into a whole image according to a certain order and rule. Therefore, the position of the sub-block image in the whole image is crucial for model feature extraction. In this embodiment, the Bert model is used to encode the sub-image position, and the position vector and the sub-block vector are summed to obtain the image input feature. The image input feature passes through 16 image self-attention mechanism modules (Transformer modules). Each Transformer module is obtained by two normalization layers, a multi-head self-attention mechanism layer and a first multi-layer perceptron residual connection. Finally, two convolution layers are used to reduce the dimension to 256, and a 256×64×64 feature tensor m is output. 16 , to realize the encoding of the image, the calculation process of the Transformer module is shown as follows:

[0081]

[0082] m' l =MSA(LN(m l-1 ))+m l-1

[0083] m l =MLP(LN(m′ l ))+m′ l

[0084] In the formula, Represents the feature vector set of the Nth image block; x class Represents a classification vector, which is used to perform image classification tasks; Z pos Represents the position vector set of the image sub-block; m 0Represents the image input features of the first Transformer module; m l represents the features calculated by the lth Transformer module; m l ′ represents the output result of the multi-head self-attention mechanism layer; MSA represents the multi-head self-attention mechanism layer; LN represents the normalization function; MLP represents the first multi-layer perceptron.

[0085] The SAM image segmentation model realizes the segmentation of the target object according to the correlation between the water level image features and the prompt points. In this embodiment, the prompt points are divided into foreground prompt points and background prompt points. The foreground prompt points are representative points of the segmentation area, marking the area where the model should focus on, and the background prompt points mark the area that the model should avoid paying attention to or ignore. The number and position of the prompt points are not limited to the embodiments of the present invention. Those skilled in the art will make attempts in the specific implementation process according to the characteristics of the segmentation target and the performance of the computer, and select the prompt points with the goal of obtaining prompt points that can efficiently complete the target area segmentation task. In this embodiment, three foreground prompt points are preferentially marked on the water level correction image. The coordinates of the three foreground prompt points are (1190, 1118), (1076, 715) and (1095, 1762), respectively. Figure 2 In the figure, it is represented by three five-pointed stars from top to bottom. At the same time, since the water ruler produces a reflection on the water surface, it will affect the image segmentation effect. Therefore, a background prompt point is marked in the reflection area with coordinates (1114, 3041). Figure 2 It is represented by the five-pointed star at the bottom. The SAM image segmentation model encodes the position coordinates and background indications of all cue points, where the background indication refers to whether the cue point is a foreground cue point (input is 1) or a background cue point (input is 0). The feature size of the encoded cue point is N×256.

[0086] Execute decoding step: This embodiment converts a set of learnable output features (similar to the category feature x of the VIT model) into class ) is superimposed on the prompt point feature. The learnable output feature consists of two parts: one is the intersection over union (IoU) feature. The mask predicted by the SAM image segmentation model has an IoU with the real mask. The mean square error between the IoU and the IoU calculated by the SAM image segmentation model constitutes the loss function of the feature. The other is the mask feature, which is supervised by a weighted combination of cross entropy loss (focal loss) and dice loss (dice loss) at a ratio of 20:1. The superimposed feature obtained after superposition is calculated by the information self-attention mechanism to find the internal correlation and is calculated by the softmax function:

[0087]

[0088] δ=Bv

[0089] In the formula, B represents the weight matrix; q represents the query vector; k represents the queried vector, k T Transpose k to satisfy the matrix multiplication requirement; D h Represents the feature dimension; v represents the value vector; δ represents the superimposed internal update feature, and all variables are continuously updated during the training process.

[0090] Then, the present embodiment performs a decoding process, which specifically uses the superimposed internal update feature as the query vector q, and uses the information cross-attention mechanism to find external correlations with the water gauge image feature, updates the superimposed internal update feature, and implements feature transformation through a multi-layer perceptron network to obtain updated superimposed features; then, the present embodiment uses the water gauge image feature as the query vector q, and uses the image cross-attention mechanism to implement self-update with the updated superimposed feature. Repeat the decoding process, and after decoding through the second decoding layer, the second decoding layer outputs the decoded output feature; wherein the decoded output feature includes the water gauge image output feature and the superimposed output feature; finally, the present embodiment uses the superimposed output feature as the query vector q, and performs another cross-attention calculation, that is, the superimposed output feature and the water gauge image output feature are captured through the fusion cross-attention mechanism layer, and transposed convolution is performed through the second convolution layer to obtain the final water gauge image feature. In this embodiment, the updated water gauge image features are subjected to two transposed convolutions using a convolution kernel of size 2 and stride 2, and the feature size is upsampled to 4 times of the original size to obtain the final water gauge image features. It should be noted that, in order to prevent information loss, each attention mechanism and multi-layer perceptron calculation has a residual connection.

[0091] It should be noted that the self-attention calculation within the water gauge image features is an internal correlation exploration, for example: the correlation between the first sub-block and the second sub-block of the image. The internal correlation is mainly used in the encoding process. Since the second sub-block is connected to the first sub-block, the encoding of the second sub-block needs to consider the characteristics of the first sub-block; the cross-attention calculation of the image features and the cue point information is an external correlation exploration, which is mainly used in the decoding process. During the decoding process, each sub-block is traversed, and the cue point and the sub-block features are calculated. If the correlation is greater than the threshold, it is 1, and if it is less than the threshold, it is 0.

[0092] In this embodiment, the water gauge correction image and the prompt point information of the four prompt points are input into the SAM image segmentation model, and the SAM image segmentation model outputs the mask with the highest confidence, such as Figure 4 As shown, the mask is a binary matrix, the water gauge area is 1, and the background area is 0; the water gauge image with mask and prompt points is as follows Figure 5As shown; it should be noted that the mask feature is separated from the superimposed output feature, and the mask feature dimension is adjusted to be consistent with the image feature through the multi-layer perceptron. The two are multiplied to obtain the target mask, and the intersection-over-union feature in the superimposed output feature generates an intersection-over-union score prediction for the mask. During the training process, the model supervises the prediction results based on the actual results, back-propagates, and continuously updates the parameters.

[0093] S4. Establish a water level measurement model between the mask pixels and the measured water level, and use the water level measurement model to process the water gauge area segmentation result to obtain the measured water level value; in this embodiment, the water level measurement model is:

[0094] S=Hk(y max -y min )

[0095] In the formula, S represents the measured water level value; H represents the water gauge range; k represents the regression coefficient; y min Indicates the ordinate of the upper boundary of the effective mask in the water gauge area segmentation result; y max Indicates the ordinate of the lower boundary of the effective mask in the water gauge area segmentation result.

[0096] For the water gauge area segmentation result, this embodiment first determines whether the mask in the water gauge area segmentation result is valid. Specifically, the sum of the elements in each row of the mask matrix in the water gauge area segmentation result is calculated, and the sum of the elements in each row is compared with the preset mask threshold. If it is greater than the mask threshold, the elements in this row are judged to be valid masks, otherwise they are invalid masks. This embodiment filters invalid masks through the mask threshold. In this embodiment, since the average width of the water gauge in the image is 207 pixels, the mask threshold is set to 100. This embodiment calculates the number of pixels occupied by the water gauge in the image along the height direction through the height of the effective mask: the ordinate of the upper boundary of the effective mask is y min , the ordinate of the lower boundary (the intersection with the water surface) is y max , the number of pixels is (y max -y min ), read the water level through the water gauge scale line, establish a linear relationship between the mask pixels and the real water level, that is, the water level measurement model. For example, in this embodiment, the water gauge range is measured to be 100 cm, and the mask occupies 2164 pixels along the height direction, and the following real water level value calculation formula can be obtained:

[0097] S = 100 - 0.0377 × n pix

[0098] Where n pix Indicates the pixel height occupied by the mask.

[0099] The embodiment of the present invention provides a water level monitoring method based on the SAM image segmentation model. The method obtains the water gauge area segmentation result by using the SAM image segmentation model established with the visual self-attention mechanism as the framework; establishes a water level measurement model between the mask pixel and the measured water level, and uses the water level measurement model to process the water gauge area segmentation result to obtain the measured water level value. Compared with the traditional method of manually monitoring the water level, the water level monitoring method proposed in this embodiment only needs to input the water gauge image taken by the monitoring camera to output the water level value corresponding to the image, which can not only realize the uninterrupted, rapid and automatic monitoring of the water level in any water area, reducing the work intensity of the detection personnel, but also has excellent recognition performance for the water level in various environments or observation angles, significantly reducing the influence of interference factors such as light and aquatic plants, and improving the accuracy and stability of the water level monitoring results.

[0100] It should be noted that the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0101] In one embodiment, Figure 6 As shown, an embodiment of the present invention provides a water level monitoring device based on the SAM image segmentation model, the device comprising:

[0102] The image correction module 101 is used to collect a water gauge image and perform spatial correction on the water gauge image to obtain a water gauge corrected image;

[0103] A model building module 102 is used to establish a SAM image segmentation model for water gauge image segmentation based on a visual self-attention mechanism;

[0104] The image segmentation module 103 is used to obtain the cue point information on the water gauge correction image, and input the water gauge correction image and the cue point information into the SAM image segmentation model to obtain a water gauge area segmentation result;

[0105] The water level measurement module 104 is used to establish a water level measurement model between the mask pixels and the measured water level, and use the water level measurement model to process the water gauge area segmentation result to obtain the measured water level value.

[0106] For the specific definition of a water level monitoring device based on the SAM image segmentation model, please refer to the above-mentioned definition of a water level monitoring method based on the SAM image segmentation model, which will not be repeated here. Those of ordinary skill in the art will appreciate that the various modules and steps described in conjunction with the embodiments disclosed in this application can be implemented in hardware, software, or a combination of both. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0107] An embodiment of the present invention provides a water level monitoring device based on the SAM image segmentation model. The device realizes non-contact water level monitoring through the SAM image segmentation model and the water level measurement model, can identify the current water level height with high precision, and avoids the situation where the traditional water level monitoring method is easily affected by light, resulting in large water level monitoring errors. The device has the characteristics of small renovation project volume, strong stability, wide application range, etc., and can be promoted and applied on a large scale.

[0108] The above-mentioned embodiments only express several preferred implementation modes of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in the technical field, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be regarded as the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be based on the protection scope of the claims.

Claims

1. A water level monitoring method based on SAM image segmentation model, characterized in that: The following steps are involved: Acquire a water gauge image, and perform spatial correction on the water gauge image to obtain a water gauge corrected image; Based on the visual self-attention mechanism, a SAM image segmentation model for water gauge image segmentation is established; Acquire the cue point information on the water gauge correction image, and input the water gauge correction image and the cue point information into the SAM image segmentation model to obtain a water gauge area segmentation result; Establishing a water level measurement model between mask pixels and the measured water level, and using the water level measurement model to process the water gauge area segmentation result to obtain the measured water level value; The SAM image segmentation model includes an input layer, an output layer, and a coding module and a mask decoding module connected in sequence between the input layer and the output layer, wherein the coding module includes an image coding module and a cue point coding module both connected to the mask decoding module; The image encoding module includes an image reorganization vector encoding module, a preset number of image self-attention mechanism modules and two first convolutional layers; each of the image self-attention mechanism modules includes a normalization module connected by residuals, a multi-head self-attention mechanism layer and a first multi-layer perceptron; The mask decoding module comprises a first decoding layer, a second decoding layer and a feature output layer connected in sequence, wherein each decoding layer comprises an image cross attention mechanism and an information self-attention mechanism, an information cross attention mechanism and a second multi-layer perceptron connected in sequence, wherein the image cross attention mechanism is connected to the second multi-layer perceptron, the image encoding module and the information self-attention mechanism, and the image cross attention mechanism and the second multi-layer perceptron are both connected to the feature output layer; The feature output layer includes a sequentially connected fusion cross attention mechanism layer, a second convolutional layer, and a third multi-layer perceptron; The step of inputting the water gauge correction image and the prompt point information into a trained SAM image segmentation model to obtain a water gauge area segmentation result comprises: Recombining and encoding the water gauge correction image through the image encoding module to obtain water gauge image features; Performing superposition encoding on the cue point information by the cue point encoding module to generate superposition features; Using two decoding layers to decode the water gauge image features and the superposition features to obtain decoded output features; wherein the decoded output features include water gauge image output features and superposition output features; Taking the superimposed output feature as a query vector, the superimposed output feature and the water gauge image output feature are passed through the fusion cross attention mechanism layer to capture external correlation, and transposed convolution is performed through the second convolution layer to obtain the final water gauge image feature; Separating the mask feature from the superimposed output feature, adjusting the dimension through a third multi-layer perceptron and multiplying the feature with the final water gauge image feature, and outputting a water gauge region segmentation result; The step of performing superposition encoding on the cue point information by the cue point encoding module to generate superposition features includes: obtaining a learnable output feature, performing feature extraction on the cue point information to obtain a cue point feature, superimposing the learnable output feature on the cue point feature to generate a superposition feature; the learnable output feature includes an intersection-and-union feature and a mask feature, and the intersection-and-union feature is used to obtain an intersection-and-union score of a final water gauge area segmentation result; The water level measurement model is: In the formula, S represents the measured water level value; H represents the water gauge range; k represents the regression coefficient; y min Indicates the ordinate of the upper boundary of the effective mask in the water gauge area segmentation result; y max Indicates the ordinate of the lower boundary of the effective mask in the water gauge area segmentation result; The two decoding layers are used to decode the water gauge image feature and the superposition feature to obtain a decoded output feature, including: In the first decoding layer, the information self-attention mechanism is used to capture the internal correlation of the superimposed features to obtain the superimposed internal update features; Taking the superimposed internal updated features as query vectors, capturing the external correlation between the superimposed internal updated features and the water gauge image features through an information cross attention mechanism, and performing feature transformation on the results output by the information cross attention mechanism through a multi-layer perceptron to obtain updated superimposed features; Taking the water gauge image feature as a query vector, capturing external correlations between the water gauge image feature and the updated superimposed feature through the image cross attention mechanism, and obtaining an updated water gauge image feature; According to the above steps, the updated superposition feature and the updated water gauge image feature are decoded again by using the second decoding layer to obtain a decoded output feature; The cue points are divided into foreground cue points and background cue points. The foreground cue points are representative points of the segmented area, marking the areas that the SAM image segmentation model should focus on. The background cue points mark the areas that the SAM image segmentation model should avoid paying attention to.

2. A water level monitoring method based on the SAM image segmentation model as claimed in claim 1, characterized in that: The step of performing spatial correction on the water gauge image to obtain a water gauge corrected image comprises: Obtaining an average deflection angle of the water gauge edge line or the center axis in the vertical direction, and establishing a rotation matrix according to the average deflection angle; Performing an affine transformation on the water gauge image using the rotation matrix to obtain a rotation-corrected image; Performing distortion correction on the rotation-corrected image to generate a deformity-corrected image; The missing values ​​in the deformity correction image are interpolated using a preset interpolation algorithm to obtain a water gauge correction image.

3. The water level monitoring method based on the SAM image segmentation model according to claim 1, characterized in that: The step of recombining and encoding the water gauge correction image by the image encoding module to obtain water gauge image features comprises: The water gauge correction image is convolved or linearly mapped to obtain a sub-block vector, and the sub-block vector is summed with the position vector to obtain an image input feature, so as to encode the image input feature through the image self-attention mechanism module to obtain a water gauge image feature.

4. A water level monitoring device based on the SAM image segmentation model, characterized in that: The device comprises: An image correction module is used to collect a water gauge image and perform spatial correction on the water gauge image to obtain a water gauge correction image; Model building module, used to build a SAM image segmentation model for water gauge image segmentation based on the visual self-attention mechanism; An image segmentation module, used for obtaining the cue point information on the water gauge correction image, and inputting the water gauge correction image and the cue point information into the SAM image segmentation model to obtain a water gauge area segmentation result; A water level measurement module is used to establish a water level measurement model between mask pixels and the measured water level, and use the water level measurement model to process the water gauge area segmentation result to obtain the measured water level value; The SAM image segmentation model includes an input layer, an output layer, and a coding module and a mask decoding module connected in sequence between the input layer and the output layer, wherein the coding module includes an image coding module and a cue point coding module both connected to the mask decoding module; The image encoding module includes an image reorganization vector encoding module, a preset number of image self-attention mechanism modules and two first convolutional layers; each of the image self-attention mechanism modules includes a normalization module connected by residuals, a multi-head self-attention mechanism layer and a first multi-layer perceptron; The mask decoding module comprises a first decoding layer, a second decoding layer and a feature output layer connected in sequence, wherein each decoding layer comprises an image cross attention mechanism and an information self-attention mechanism, an information cross attention mechanism and a second multi-layer perceptron connected in sequence, wherein the image cross attention mechanism is connected to the second multi-layer perceptron, the image encoding module and the information self-attention mechanism, and the image cross attention mechanism and the second multi-layer perceptron are both connected to the feature output layer; The feature output layer includes a sequentially connected fusion cross attention mechanism layer, a second convolutional layer, and a third multi-layer perceptron; The step of inputting the water gauge correction image and the prompt point information into a trained SAM image segmentation model to obtain a water gauge area segmentation result comprises: Recombining and encoding the water gauge correction image through the image encoding module to obtain water gauge image features; Performing superposition encoding on the cue point information by the cue point encoding module to generate superposition features; Using two decoding layers to decode the water gauge image features and the superposition features to obtain decoded output features; wherein the decoded output features include water gauge image output features and superposition output features; Taking the superimposed output feature as a query vector, the superimposed output feature and the water gauge image output feature are passed through the fusion cross attention mechanism layer to capture external correlation, and transposed convolution is performed through the second convolution layer to obtain the final water gauge image feature; Separating the mask feature from the superimposed output feature, adjusting the dimension through a third multi-layer perceptron and multiplying the feature with the final water gauge image feature, and outputting a water gauge region segmentation result; The step of performing superposition encoding on the cue point information by the cue point encoding module to generate superposition features includes: obtaining a learnable output feature, performing feature extraction on the cue point information to obtain a cue point feature, superimposing the learnable output feature on the cue point feature to generate a superposition feature; the learnable output feature includes an intersection-and-union feature and a mask feature, and the intersection-and-union feature is used to obtain an intersection-and-union score of a final water gauge area segmentation result; The water level measurement model is: In the formula, S represents the measured water level value; H represents the water gauge range; k represents the regression coefficient; y min Indicates the ordinate of the upper boundary of the effective mask in the water gauge area segmentation result; y max Indicates the ordinate of the lower boundary of the effective mask in the water gauge area segmentation result; The two decoding layers are used to decode the water gauge image feature and the superposition feature to obtain a decoded output feature, including: In the first decoding layer, the information self-attention mechanism is used to capture the internal correlation of the superimposed features to obtain the superimposed internal update features; Taking the superimposed internal updated features as query vectors, capturing the external correlation between the superimposed internal updated features and the water gauge image features through an information cross attention mechanism, and performing feature transformation on the results output by the information cross attention mechanism through a multi-layer perceptron to obtain updated superimposed features; Taking the water gauge image feature as a query vector, capturing external correlations between the water gauge image feature and the updated superimposed feature through the image cross attention mechanism, and obtaining an updated water gauge image feature; According to the above steps, the updated superposition feature and the updated water gauge image feature are decoded again by using the second decoding layer to obtain a decoded output feature; The cue points are divided into foreground cue points and background cue points. The foreground cue points are representative points of the segmented area, marking the areas that the SAM image segmentation model should focus on. The background cue points mark the areas that the SAM image segmentation model should avoid paying attention to.

Images