Machine Vision-Based Water Body Segmentation and Water Level Line Extraction System and Method

Abstract

The present invention discloses a water body segmentation and water level line extraction system and method based on machine vision, including: S1: Collect and upload the monitoring video images of the river in the selected regional watershed; S2: Roughly outline the foreground and background of the image using a brush; S3: Precisely segment the water body and the irrelevant background area and extract the water body part; S4: Select and draw the area of the water level line and draw the water level line within the area. The present invention uses the camera device of the hydrological sluice station to collect the monitoring video of a specific water area, segments the water body through the improved image segmentation algorithm and line detection algorithm, and then obtains the water level line in real time. It can realize the real-time dynamic monitoring of the reservoir environment. The improved image segmentation algorithm and line detection algorithm have strong interactivity, small computational amount, and do not require a large amount of storage space.

Description

Technical Field

[0001] The present invention belongs to the technical field of scene analysis in the water conservancy industry, and particularly relates to a water body segmentation and water level line extraction system and method based on machine vision. Background Technique

[0002] At present, the problem of insufficient monitoring and early warning capabilities of reservoir water conditions is relatively prominent, and the emergency response capabilities are seriously insufficient. Therefore, the importance of monitoring the reservoir water level, analyzing data in a timely manner, and grasping the real-time changes of the reservoir water level is even more prominent.

[0003] The current monitoring schemes for reservoir water levels include the following several types:

[0004] Traditional manual monitoring: Manually read the water level gauge in the scene with a water gauge. This scheme has a high labor cost and is greatly affected by the environment. Water level gauge + river monitoring terminal: Install monitoring equipment poles and brackets, generate water level process curves and data statistical tables through sensors, and upload data through various communication methods such as GPRS, CDMA, and Beidou satellites. This scheme can achieve the function of water level monitoring, but has a high cost and complex installation.

[0005] The traditional manual monitoring scheme has inaccuracies and dangers in human eye recognition, and the water level cannot be observed in harsh environments or when the water gauge is damaged. The water level gauge + river monitoring terminal scheme requires special instruments, and the installation and maintenance costs are expensive. The water level monitoring is easily affected by wind and waves, resulting in large errors. Summary of the Invention

[0006] In order to overcome the deficiencies of the above-mentioned existing technologies, the purpose of the present invention is to provide a water body segmentation and water level line extraction system and method based on machine vision, which uses the camera device of a hydrological sluice station to collect monitoring videos of a specific water area, segments the water body through an improved image segmentation algorithm and a line detection algorithm, and then obtains the water level line in real time. The present invention is suitable for conditions without a water gauge or when the water gauge is damaged, and is suitable for various different reservoir environments such as harsh environmental conditions or unattended situations. It has a low labor cost, a simple device, and can achieve real-time dynamic monitoring of the reservoir environment. The water body segmentation algorithm and the line detection algorithm have strong interactivity, small computational complexity, and do not require a large amount of storage space.

[0007] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0008] A water body segmentation and water level line extraction system based on machine vision includes a video image acquisition part, an image processing part, a result display part, a multi-thread processing and interaction part;

[0009] The video image acquisition part includes image acquisition, storing the original video image information of the water area to be detected, and image encoding and decoding. The image acquisition inputs the acquired original video image into the storage of the original video image information of the water area to be detected, and through image encoding and decoding, frame images are obtained and transferred to the image processing part for the next step of processing.

[0010] The image processing part includes three major modules: image preprocessing, water body segmentation, and water level line detection. The image preprocessing is used to adjust the image size, grayscale the image, and perform morphological processing of the images acquired by the video image acquisition part to reduce the influence of different weather conditions on subsequent operations. The water body segmentation part is responsible for segmenting the concerned water body area and other irrelevant areas, and roughly outlines the water body range of the water area to be detected through manual interaction to improve the accuracy of water body segmentation. The water level line detection includes drawing the water body edge line within the selected area, data estimation and filtering to achieve the purpose of water level detection.

[0011] The result display part includes three modules: image rendering, data display, and data recording. Among them, the image rendering module converts the original video and the graphic calculation results of the image processing part into visible pixels. The data display module is responsible for displaying the results of water body segmentation and water level line drawing. Data recording is responsible for storing the foreground and background models and masks generated by water body segmentation, the endpoint coordinates of the water level line generated by water level line detection, and the drawn result images. Recording the parameters is beneficial to the transplantation of the system and the warm start of the segmentation model.

[0012] The multi-thread processing runs through the three parts: the video image acquisition part, the image processing part, and the result display part.

[0013] The interaction system runs through the three parts: the video image acquisition part, the image processing part, and the result display part.

[0014] The multi-thread processing is used to coordinate the video image acquisition and storage thread, the image processing thread, and the result display thread to synchronously process and execute at the same time, and improve the system efficiency by increasing the resource utilization rate.

[0015] The interaction system receives the user's intention through manual operations to improve the user experience and system applicability. In the video image acquisition part, the user selects the video access method and encoding and decoding format. In the image processing part, the user needs to use the mouse as a brush to move and draw a rough mask. In the result display part, the user selects whether to save the parameters of the foreground and background models, mask images, water level line endpoint coordinates, and drawn result images. The parameters are used for the transplantation of this system and the warm start of the segmentation model.

[0016] The image acquisition selects two video loading methods: reading the network video stream or the local video file; the image encoding and decoding uses a codec to encode the video obtained by the image acquisition part, compress the image for transmission, and save the bandwidth occupied by the communication transmission.

[0017] The usage method of the water body segmentation and water level line extraction system based on machine vision includes the following steps;

[0018] S1: Collect and upload the monitoring video images of the river in the selected area basin;

[0019] S2: Roughly outline the foreground and background of the image using a brush;

[0020] S3: Accurately segment the water body and the irrelevant background area, and extract the water body part;

[0021] S4: Select and draw the water level line area and draw the water level line within the area.

[0022] The specific step S1 is as follows:

[0023] 1) Video reading: The video image acquisition part reads the network video stream or loads the local video image, obtains the current display frame, frame image size, and frame rate information through image encoding and decoding, and obtains the original video image and the first frame image of the video;

[0024] 2) Video display: Perform image fusion on the mask and the first frame image in a 1:1 ratio, perform preprocessing on the first frame image by adjusting the size, contrast, and brightness, and display the preprocessed first frame image and the fusion result. If the local video file is read in the previous step, the video display is interrupted after counting to the total number of video frames;

[0025] 3) Video storage: Optionally store the network video stream as a video file.

[0026] The specific S2 is as follows:

[0027] 1) Foreground and background area initialization: Initialize the mask for segmentation, identify the foreground and background with white and gray respectively, and default the pixels at the four corner positions of the mask to the foreground, and all other pixels to the background;

[0028] 2) Drawing settings: Initialize the brush and the foreground and background modes. The default initial radius of the brush is 15px, the adjustment range is 5 - 40px, the brush default draws in the foreground mode, and switches between the foreground and background modes according to the drawing needs;

[0029] 3) Mouse event binding: Bind the mouse movement event to draw the mask, and bind the mouse click event to select two points of the rectangular box;

[0030] 4) Mask drawing: Move the mouse to draw the foreground and background parts, which are passed as model parameters to the interactive GrabCut for the next segmentation.

[0031] Specifically, S3 is as follows:

[0032] The original image, the mask during the iteration process, and the foreground and background models are passed as parameters to the water body segmentation algorithm - interactive GrabCut. With the goal of minimizing the Gibbs energy, the water body of the image is segmented, the n-links between the pixel points in the foreground area and the pixel points in the background area are cut off, the irrelevant t-links are deleted, the connection relationship between the terminal nodes and the ordinary nodes is adjusted manually, and the water body segmentation algorithm is iteratively used to obtain the optimal segmentation map, realizing the segmentation of the foreground water body and the irrelevant background area and displaying it on the window.

[0033] Specifically, S4 is as follows:

[0034] To detect the water level line, it is necessary to first frame the mask map of the rectangular area, reset the coordinates of the mask, and then perform edge detection and line detection.

[0035] 1) Edge detection: The original image is an RGB image, which is converted to a single-channel grayscale image to start edge detection: First, perform Gaussian filtering to reduce noise, reducing the impact of Gaussian noise and salt-and-pepper noise on subsequent operations; then, calculate the difference between the matrix of the grayscale image and the Sobel horizontal operator and vertical operator to solve the image gradient and direction. According to the solved gradient direction, perform maximum value suppression on the amplitude to avoid misidentifying non-edge pixels as edges, clarify the boundary, and perform single or multiple dilation operations on the detection result to avoid the situation where the water level line cannot be detected.

[0036] 2) Line detection: Use the probabilistic Hough line detection algorithm to perform line detection. Randomly select edge points in the image that have not been marked as a certain line for Hough transformation and cumulative calculation. Select the points greater than the threshold, find the endpoints of its line and calculate the line length. When the line length is greater than the threshold, it is recognized as a line. After multiple calculations, the endpoint positions of the multiple lines obtained are fitted as the final result.

[0037] 3) Water level line drawing: Draw the water level line in the rectangular area according to the final fitting result of the line detection.

[0038] 4) Window display: Render the processing result to the window and display the water level line result.

[0039] Advantages of the present invention:

[0040] The present invention can realize all-weather real-time water level monitoring for unattended small watersheds, large water conservancy project facilities that cannot be covered, stations without water gauges or with ineffective water gauges where hydrological information monitoring work cannot be completed.

[0041] This system has a simple device and only requires installing cameras that meet the requirements for observation; the model is simple, without the need for a large amount of computing resources or a large amount of data for model training, and only a few model parameters are needed to achieve work with a similar degree of completion to deep learning;

[0042] The model of the present invention has high accuracy, and human-computer interaction can compensate for errors caused by algorithm errors and other reasons, and the detection effect is relatively good;

[0043] The system of the present invention has high adaptability and strong portability. Using the saved model parameters, the system can be hot-started and can be transplanted to other systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic diagram of the system architecture of the present invention.

[0045] Figure 2 It is a schematic diagram of image segmentation of the present invention.

[0046] Figure 3 It is a schematic diagram of the steps of Canny operator edge detection of the present invention.

[0047] Figure 4 It is a schematic diagram of the steps of probabilistic Hough line detection of the present invention.

[0048] Figure 5 It is a flowchart of the technical solution of the present invention.

[0049] Figure 6 It is a schematic diagram of collecting and uploading monitoring videos of the present invention.

[0050] Figure 7 It is a schematic diagram of custom brush for drawing foreground and background of the present invention.

[0051] Figure 8 It is a schematic diagram of selecting and drawing the water level line area of the present invention.

[0052] Figure 9 It is a schematic diagram of displaying the results of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] The present invention will be further described in detail below with reference to the accompanying drawings.

[0054] Embodiment: As Figures 1-5 shown:

[0055] S1: Collect and upload the monitoring video images of the river in the selected area basin:

[0056] 1) Video reading: The video image acquisition part reads the network video stream or loads the local video image, obtains the current display frame, frame image size, and frame rate information through image encoding and decoding, and obtains the original video image and the first frame image of the video;

[0057] 2) Video display: Perform image fusion on the mask and the first frame image in a 1:1 ratio, perform preprocessing such as resizing, contrast adjustment, and brightness adjustment on the first frame image, and display the preprocessed first frame image and the fusion result. If a local video file is read in the previous step, the video display is interrupted after counting to the total number of video frames.

[0058] 3) Video storage: Optionally store the network video stream as a video file.

[0059] S2: Roughly outline the foreground and background of the image using a brush:

[0060] 1) Foreground and background area initialization: Initialize the mask for segmentation, identify the foreground and background with white and gray respectively. By default, the pixels at the four corner positions of the mask are the foreground, and all other pixels are the background.

[0061] 2) Drawing settings: Initialize the brush and foreground / background modes. The default initial radius of the brush is 15px, and the adjustable range is 5 - 40px. The brush defaults to the foreground drawing mode, and the foreground and background modes can be switched according to the drawing needs.

[0062] 3) Mouse event binding: Bind the mouse movement event to draw the mask, and bind the mouse click event to select two points of the rectangular box.

[0063] 4) Mask drawing: Draw the foreground and background parts during mouse movement, and pass them as model parameters to the improved GrabCut algorithm for the next segmentation step.

[0064] S3: Accurately segment the water body and the irrelevant background area using the algorithm, and extract the water body part;

[0065] Pass the original image, the mask and the foreground / background model during the iteration process as parameters to the water body segmentation algorithm. Targeting the minimum of Gibb energy, segment the water body of the image, cut off the n-links between the foreground area pixels and the background area pixels, delete the irrelevant t-links, manually interact to adjust the connection relationship between the terminal nodes and the ordinary nodes, and iteratively use the water body segmentation algorithm to obtain the optimal segmentation map, realizing the segmentation of the foreground water body and the irrelevant background area, and display it on the window.

[0066] S4: Select and draw the water level line area and draw the water level line within the area;

[0067] To detect the water level line, it is necessary to first select the mask image of the rectangular area, reset the coordinates of the mask, and then perform edge detection and line detection.

[0068] 1) Edge detection: The original image is an RGB image, which is converted into a single-channel grayscale image to start edge detection. First, Gaussian filtering is performed to reduce noise, minimizing the impact of Gaussian noise and salt-and-pepper noise on subsequent operations. Then, the matrix of the grayscale image is differentially calculated with Sobel horizontal and vertical operators to solve for the image gradient and direction. Based on the solved gradient direction, non-maximum suppression is performed on the amplitude to avoid misidentifying non-edge pixels as edges and to clarify the boundaries. The detection result is subjected to single or multiple dilation operations to prevent the situation where the water level line cannot be detected.

[0069] 2) Line detection: The probabilistic Hough line detection algorithm is used for line detection. Edge points that have not been marked as a certain line are randomly selected from the image for Hough transform and cumulative calculation. Points greater than the threshold are selected, and their line endpoints are found and the line length is calculated. When the line length is greater than the threshold, it is recognized as a line. After multiple calculations, the endpoint positions of multiple lines obtained are fitted as the final result.

[0070] 3) Water level line drawing: Based on the final fitting result of line detection, the water level line is drawn in the rectangular area.

[0071] 4) Window display: The processing result is rendered into the window to display the water level line result.

[0072] The water body segmentation algorithm of the present invention - Interactive GrabCut does not need to use the method of bounding box selection. Only by roughly outlining the foreground and background on a single-frame image with a custom brush to draw the mask, and through multiple iterations, the segmentation of the water body and irrelevant regions is realized, improving the calculation speed and reducing the probability of mis-segmentation.

[0073] The specific principle of the water body segmentation algorithm - Interactive GrabCut in the present invention is as follows:

[0074] An undirected graph G = (V, E) is established to represent the image to be segmented. V and E are the sets of vertices (vertex) and edges (edge) respectively. Each ordinary vertex corresponds to each pixel in the image, and each edge corresponds to connecting two adjacent ordinary vertices. On this basis, terminal vertices S and T are added, which are connected to each ordinary vertex respectively. The edges connecting ordinary vertices are n-links, and the edges connecting to terminal vertices are t-links. Each edge has a non-negative weight, that is, a cost function cost. Image segmentation is to cut off the n-links to complete the separation of the S graph and the T graph. The result of image segmentation is to separate the S graph and the T graph to minimize the cost function. The schematic diagram of image segmentation is as Figure 2 shown. The white nodes represent the pixel points in the foreground area, and the gray nodes represent the pixel points in the background area. Cut the image according to the minimum value of the cost function, retain the solid arcs, and delete the dashed arcs.

[0075] During the use of this system, masks of GC_BG (background), GC_PR_BG (possible background), GC_FG (foreground), and GC_PR_FG (possible foreground) are drawn on the fused image using a custom brush, replacing the method of drawing masks by box selection and manual fixation in the original GrabCut algorithm, and iterating by accurately depicting the foreground and background regions. In the RGB color space, a Gaussian mixture model (GMM) with K Gaussian components is used to model the foreground and background respectively, and an additional vector k = {k1, k2, …, k n , …, k N} represents the Gaussian component corresponding to the nth pixel. The pixels belonging to the foreground and background are clustered into k classes respectively through the k-means clustering algorithm, that is, the k Gaussian models in the GMM.

[0076] Each Gaussian model in the GMM obtains a pixel sample set, and the mixture Gaussian density model is in the following form:

[0077]

[0078]

[0079] The parameters mean vector μ, covariance matrix Σ, and weight π are determined respectively by the RGB values of the pixels in the sample set, the ratio of the number of pixels of the Gaussian component to the total number. It is determined whether the pixel comes from the Gaussian component of the target GMM or the Gaussian component of the background GMM. The Gibbs energy of the entire image is determined for image segmentation, and the formula is as follows:

[0080] E( α , k, θ , z) = U( α , k, θ , z) + V( α , z)

[0081]

[0082]

[0083] Among them, the Gibbs energy includes the region term U( α , k, θ , z) and the edge energy term V( α , z), U( α , k, θ , z) measures the penalty for being classified as the target or background, and V( α , z) measures the penalty for the discontinuity of neighboring pixels m and n.

[0084] For each pixel, a Gaussian component in the GMM is assigned:

[0085]

[0086] For the given image data Z, learn and optimize the parameters of the GMM:

[0087]

[0088] Segmentation estimation:

[0089]

[0090] Repeat the above three steps to optimize the GMM model, and finally obtain the image segmentation result.

[0091] Water level line detection principle - Edge extraction + Probabilistic Hough line detection algorithm;

[0092] Water level line detection mainly applies edge detection and line detection algorithms. Edge detection is a basic problem in image processing and computer vision. The purpose is to reduce the data volume of the image without damaging the image content, create a line graph, and focus on understanding the image content. Edge detection algorithms are mainly divided into first-order edge detection operators, second-order edge detection operators, and other edge detection operators. Representative edge detection operators include Sobel operator, Roberts operator, Prewitt operator, and Canny operator, etc. Select the BSDS dataset to test the performance of the above edge detection methods, and use the detection ratio to evaluate the edge detection performance: The Roberts operator performs outstandingly for images with a single background, but is highly sensitive to noise; The detection effects of the Sobel operator and the Prewitt operator are at the same level, and have a certain smoothing effect on noise; The edge detection of the Canny operator processes image noise by introducing Gaussian filtering. After appropriately adjusting the parameters, it can be better applied to edge detection in different scenarios. Above, select the Canny operator for edge detection.

[0093] Line detection algorithms are commonly used in special tasks such as lane line detection and length measurement in the field of computer vision. Commonly used line detection algorithms include Hough line detection, Probabilistic Hough line detection, LSD line detection, etc. Hough line detection extracts the image edge and completes the detection by changing the edge point coordinates to the parameter space, but the detection result is affected by thresholds, Hough space resolution, etc., and can only detect edge lines and cannot detect thick lines with multiple pixel widths; LSD line detection is a method of perceptual clustering, and the detection process is complex, and multiple algorithm parameters need to be adjusted to obtain better detection effects. Above, after comprehensive evaluation, select the Probabilistic Hough line detection algorithm.

[0094] In the present invention, the Canny operator is used to detect the water body edge line, and the probabilistic Hough transform is used to fit the water level line. In Canny edge detection, here the mask obtained in GrabCut is converted from a four-value image to a binary RGB image, and the RGB image is converted to a single-channel grayscale image for edge detection. A 3*3 dilation kernel is constructed to dilate the edge detection result and thicken the edge line to avoid the situation where a straight line cannot be found. The Canny operator further refines the effect and more accurately locates compared with operators such as Sobel and Prewitt. Its detection steps are as follows Figure 3 shown.

[0095] The main steps of Canny edge detection: The first step is to use Gaussian smoothing to reduce the noise in the image; the second step is to use Sobel horizontal and vertical operators to calculate the image gradient; the third step is to perform non-maximum suppression on the amplitude according to the angle; the fourth step is to detect and connect edges using a double-threshold algorithm.

[0096] The essence of the Hough line detection is to convert the straight line detection problem in the image space to the parameter space, and complete the straight line detection task by finding peaks in the parameter space. The probabilistic Hough line detection algorithm randomly selects some edge points for processing, reduces the computational amount, and saves the required memory space. The steps of the probabilistic Hough line detection are as follows Figure 4 shown.

[0097] The steps of the system interaction in the embodiment of the present invention are as follows, and the effect diagram is as follows Figures 6-9 shown.

[0098] Step 1: Open the video

[0099] During the system operation, starting from the image acquisition and storage module, if you choose to read a local video file, use the image decoder module to decode the video and capture the first frame image. At the same time, the first frame image is preprocessed and then displayed on the left side of the interface, and the frame image is fused with the initial mask and displayed for water body segmentation in the image processing part; if you choose to read a network video stream, the video stream collected by the image is compressed by the codec, and the storage space of the video file is reduced while ensuring a certain clarity for communication. The captured first frame image is preprocessed and fused with the initial mask and then displayed on the left and right sides of the interface respectively, and is transmitted to the image processing part for the next step of processing.

[0100] Step 2: Draw the mask and click to segment;

[0101] During system operation, the water body segmentation module is mainly called. The water body segmentation module uses the improved image segmentation algorithm to process the first frame image and the mask image in Step 1: taking the mouse coordinates as the brush coordinates, outlining the foreground and background regions on the mask according to the drawing mode, and passing the mask, the original image, and the foreground and background models as parameters into the GrabCut algorithm for segmentation to obtain a new mask and foreground and background models for continued iteration. After repeated drawing and processing by the segmentation algorithm, finally, the mask and foreground and background model parameters for detecting the water level line are obtained.

[0102] Step 3: Select and adjust the region

[0103] During system operation, this step mainly calls the water level line detection module. The water level line detection module receives the final mask and foreground and background model parameters, clicks the mouse to select two coordinates as the upper left corner point and the lower right corner point coordinates of the selected region to draw a rectangle, and at the same time retains the mask selected region. Click the "Redraw" button to clear the coordinate list, restore the unpolluted image, and then redraw. Click the "Up", "Down", "Left", "Right", "Bigger", "Smaller" buttons, and the system changes the coordinates of the corner points by a step of 5 pixels according to the different selections. Click the "Next" button to use the probabilistic Hough line detection algorithm to detect the water level line in the local region.

[0104] Step 4: Save the video and related parameters

[0105] During system operation, this step mainly calls the image rendering module, the data display module, and the data recording module. The data display module passes the model processing results to the image rendering module. The data recording module saves the model parameters, etc. The image rendering module receives the data passed by the data display module and converts the model processing results stored in the memory into the actual drawing on the interface.

Claims

1. A water body segmentation and water level line extraction system based on machine vision, characterized in that, It includes a video image acquisition part, an image processing part, a result display part, a multi-threaded processing and interaction part; The video image acquisition part includes image acquisition, storing the original video image information of the water area to be detected, and image encoding and decoding; The image acquisition inputs the obtained original video image into the storage of the original video image information of the water area to be detected, obtains frame images through image encoding and decoding, and transfers them to the image processing part for the next step of processing; The image processing part includes three major modules: image preprocessing, water body segmentation, and water level line detection; the image preprocessing is used to adjust the image size, grayscale the image, and perform morphological processing of the images collected by the video image acquisition part to reduce the influence of different weather conditions on subsequent operations. The water body segmentation part is responsible for segmenting the concerned water body area and other irrelevant areas, and roughly outlines the water body range of the water area to be detected through manual interaction to improve the accuracy of water body segmentation; the water level line detection includes drawing the water body edge line within the selected area, data estimation and filtering to achieve the purpose of water level detection; The result display part includes three modules: image rendering, data display, and data recording. Among them, the image rendering module converts the original video and the graphic calculation results of the image processing part into visible pixels. The data display module is responsible for displaying the results of water body segmentation and water level line drawing. Data recording is responsible for storing the foreground and background models and masks generated by water body segmentation, the water level line endpoint coordinates generated by water level line detection, and the drawn result images. Recording the parameters is beneficial to the transplantation of the system and the warm start of the segmentation model; The multi-threaded processing runs through the three parts: the video image acquisition part, the image processing part, and the result display part; The interaction part runs through the three parts: the video image acquisition part, the image processing part, and the result display part; The way of manual interaction is to roughly outline the foreground and background of the image with a brush; draw the foreground and background parts by moving the mouse, and transfer them as model parameters to the interactive GrabCut for the next step of segmentation.

2. The water body segmentation and water level line extraction system based on machine vision according to claim 1, characterized in that, The multi-threaded processing is used to coordinate the video image acquisition and storage thread, the image processing thread, and the result display thread to synchronously process and execute at the same time, and improve the system efficiency by improving resource utilization.

3. The water body segmentation and water level line extraction system based on machine vision according to claim 1, characterized in that, The interaction part receives the user's intention through manual operations to improve the user experience and system applicability. In the video image acquisition part, the user selects the video access method and encoding and decoding format. In the image processing part, the user needs to use the mouse as a paintbrush to move and draw a rough mask. In the result display part, the user selects whether to save the parameters of the foreground and background models, mask images, water level line endpoint coordinates, and drawn result images. The parameters are used for the transplantation of this system and the warm start of the segmentation model.

4. The water body segmentation and water level line extraction system based on machine vision according to claim 1, characterized in that, The image acquisition selects two video loading methods: reading network video streams or local video files; image encoding and decoding is to use a codec to encode the video obtained by the image acquisition part, compress the image for transmission, and save the bandwidth occupied by communication transmission.

5. The usage method of the machine vision-based water body segmentation and water level line extraction system according to any one of claims 1-4, characterized in that, It includes the following steps; S1: Collect and upload the monitoring video images of the river in the selected area basin; S2: Roughly outline the foreground and background of the image with a brush; S3: Precisely segment the water body from the irrelevant background area and extract the water body part; S4: Select and draw the water level line area and draw the water level line within the area.

6. The method of using the water body segmentation and water level line extraction system based on machine vision according to claim 5, characterized in that, The specific steps of S1 are as follows: 1) Video reading: The video image acquisition part reads the network video stream or loads the local video image, obtains the current display frame, frame image size, and frame rate information through image encoding and decoding, and obtains the original video image and the first frame image of the video; 2) Video display: Perform image fusion on the mask and the first frame image at a ratio of 1:1, preprocess the first frame image by resizing, adjusting contrast and brightness, and display the preprocessed first frame image and the fusion result. If a local video file is read in the previous step, the video display is interrupted after counting to the total number of frames of the video; 3) Video storage: Optionally store the network video stream as a video file.

7. The method of using the machine vision-based water body segmentation and water level line extraction system according to claim 5, characterized in that, The specific steps of S2 are as follows: 1) Initialization of foreground and background areas: Initialize the mask for segmentation, identify the foreground and background with white and gray respectively, default the pixels at the four corner positions of the mask as the foreground, and the rest of all pixels as the background; 2) Drawing settings: Initialize the brush and foreground and background modes. The default initial radius of the brush is 15px, and the adjustment range is 5 - 40px. The brush defaults to the foreground drawing mode, and switches between the foreground and background modes according to the drawing needs; 3) Mouse event binding: Bind the mouse movement event to draw the mask, and bind the mouse click event to select two points of the rectangular box; 4) Mask drawing: Draw the foreground and background parts during mouse movement and pass them as model parameters to the interactive GrabCut for the next segmentation.

8. The method for using the machine vision-based water body segmentation and water level line extraction system according to claim 5, characterized in that The specific steps of S3 are as follows: Transfer the original image, the mask during the iteration process, and the foreground and background models as parameters to the water body segmentation algorithm - interactive GrabCut. Targeting the minimization of Gibb energy, segment the water body of the image, cut off the n-links between the foreground area pixels and the background area pixels, delete the irrelevant t-links, manually interact to adjust the connection relationship between the terminal nodes and the ordinary nodes, and iteratively use the water body segmentation algorithm to obtain the optimal segmentation map to achieve the segmentation of the foreground water body and the irrelevant background area, and display it on the window.

9. The method of using the water body segmentation and water level line extraction system based on machine vision according to claim 5, characterized in that, The specific steps of S4 are as follows: To detect the water level line, it is necessary to first select and draw the mask image of the rectangular area, reset the coordinates of the mask, and then perform edge detection and line detection; 1) Edge detection: The original image is an RGB image, convert it to a single-channel grayscale image to start edge detection: First, perform Gaussian filtering to reduce noise, reducing the influence of Gaussian noise and salt-and-pepper noise on subsequent operations; Then, calculate the image gradient and direction by taking the difference between the matrix of the grayscale image and the Sobel horizontal operator and vertical operator, suppress the maximum value of the amplitude according to the solved gradient direction to avoid misidentifying non-edge pixels as edges, clarify the boundary, and perform single or multiple dilation operations on the detection result to avoid the situation where the water level line cannot be detected; 2) Straight line detection: Use the probabilistic Hough line detection algorithm to perform straight line detection. Randomly extract edge points in the image that are not marked as a certain straight line for Hough transformation and cumulative calculation. Select points greater than the threshold, find the endpoints of its straight line and calculate the straight line length. When the straight line length is greater than the threshold, it is recognized as a straight line. After multiple calculations, fit the endpoint positions of the multiple straight lines obtained as the final result; 3) Water level line drawing: Draw the water level line in the rectangular area according to the final fitting result of the straight line detection; 4) Window display: Render the processing result into the window and display the water level line result.

Images