A river channel siltation intelligent early warning method based on visual perception

Abstract

The application provides a river channel silting intelligent early warning method based on visual perception, comprising: collecting image data of a to-be-tested river channel, and pre-processing the image data; constructing an image data set, dividing it into a training set and a verification set, constructing a silting early warning model for determining river channel silting risk, outputting silting early warning information, training the silting early warning model using the training set to obtain a trained silting early warning model; inputting the verification set into the trained silting early warning model to generate a verification result, evaluating through a model evaluation index to obtain a trained silting early warning model; inputting the to-be-tested image into the trained silting early warning model to output corresponding silting early warning information. The application cooperatively analyzes the influence degree of the shoreline change rate and the floating object density change rate on the river channel silting, realizes the graded early warning of the silting risk, reduces the false alarm risk of a single early warning index, and effectively improves the accuracy and reliability of the silting early warning.

Description

Technical Field

[0001] This invention belongs to the field of water environment monitoring technology, specifically relating to an intelligent early warning method for river siltation based on visual perception. Background Technology

[0002] River siltation is a core challenge in water environment management, leading not only to reduced flood control capacity and water quality deterioration but also to disruption of ecosystem balance. Currently, river siltation monitoring primarily employs methods such as manual inspection, sensor monitoring, and image recognition. While traditional manual inspection is intuitive and reliable, it suffers from low monitoring efficiency, poor early warning timeliness, and strong subjectivity, failing to accurately quantify siltation risk. Siltation monitoring and early warning based on flow velocity sensors are largely limited by high deployment costs, maintenance difficulties, and susceptibility to equipment obstruction, making it difficult to reflect the overall siltation trend of the river. With the development of computer vision and artificial intelligence technologies, river monitoring methods based on image recognition or multi-device combinations are gradually becoming a research hotspot, but some shortcomings still exist.

[0003] Chinese patent CN113155102B discloses a comprehensive river management system. This system utilizes flow velocity sensors to acquire real-time water flow velocity information at key river points and simultaneously monitors and acquires video information of the river surface at these key points. By statistically analyzing the proportion of floating debris at locations of abnormal flow velocity, it determines the location of blockages and issues early warnings. While this system can provide early warnings of river blockages, it is still limited by the high cost and susceptibility to sensor failures, and the monitoring range is directly related to the sensor deployment location.

[0004] Chinese patent CN116503767B proposes a river floating object identification system based on semantic image processing. This system uses unmanned aerial photography to acquire image information of each monitored sub-river, analyzes it to obtain obstacle information for each sub-river, including the type and actual area of ​​each obstacle, and then filters out and issues warnings for each sub-river. However, while this system can identify floating objects in multiple rivers over a large area to assess the risk of river siltation, it struggles with real-time monitoring, and relying solely on obstacle accumulation for river blockage warnings fails to fully guarantee the reliability of the warning information.

[0005] In summary, existing technologies have the following shortcomings in river siltation monitoring and early warning:

[0006] 1. Traditional river siltation monitoring mainly relies on manual inspections, which has limitations such as low efficiency, narrow coverage, poor real-time performance, and strong subjectivity.

[0007] 2. Sensor-based monitoring methods are expensive to maintain and are prone to data distortion or even damage due to siltation or floating debris. Furthermore, the warning range is limited by the sensor deployment location, making it difficult to achieve large-scale coverage of the river and resulting in high operation and maintenance costs.

[0008] 3. Most existing visual monitoring technologies rely on a single indicator (such as the proportion of floating objects) for early warning of river siltation, which is prone to false alarms and has significant shortcomings in accuracy and reliability.

[0009] Therefore, there is an urgent need for a low-cost, highly reliable, and multi-scenario-adaptable intelligent early warning method for river siltation. Summary of the Invention

[0010] To address the issues of poor accuracy and reliability in existing water environment monitoring for river siltation risk early warning, and to achieve real-time monitoring and precise early warning of river siltation, this invention provides a visual perception-based intelligent early warning method for river siltation, comprising the following steps:

[0011] A visual perception-based intelligent early warning method for river siltation includes:

[0012] S1. Collect image data of the river channel to be measured, and preprocess the image data to obtain preprocessed image data;

[0013] S2. Based on the preprocessed image data, construct an image dataset, randomly divide the image dataset into a training set and a validation set according to a certain ratio, construct a siltation early warning model, which is used to determine the risk of river siltation and output siltation early warning information. Use the training set to train the siltation early warning model to obtain the trained siltation early warning model.

[0014] The siltation early warning model includes a shoreline extraction module, a floating object identification module, and an early warning analysis module;

[0015] The shoreline extraction module is used to obtain the shoreline coordinates in the image data;

[0016] The floating object recognition module is used to identify the distribution density of floating objects in the image data;

[0017] The early warning analysis module is used to analyze the shoreline change rate and floating object density change rate within the monitoring period, and output the corresponding siltation early warning information.

[0018] S3. Input the validation set into the trained siltation early warning model to generate validation results. Evaluate the validation results using model evaluation metrics. If the validation results meet the evaluation criteria, the trained siltation early warning model is obtained and proceeds to S4. If the validation results do not meet the evaluation criteria, return to S2 for retraining.

[0019] S4. Input the image to be tested into the trained siltation early warning model and output the siltation early warning information corresponding to the image to be tested.

[0020] Preferably, step S1 further includes:

[0021] The image data includes images of the river channel under test, taken in different weather conditions and at different times, and containing the shoreline; the preprocessing includes sequentially removing noise and enhancing the image data.

[0022] Preferably, step S2 further includes:

[0023] The image dataset is randomly divided into the training set and the validation set in an 8:2 ratio.

[0024] The shoreline extraction module includes distinguishing water features and riverbank features in the image data using optical flow, outputting a water-shore segmentation mask, extracting shoreline information from the image data using an edge detection algorithm, outputting an edge image, performing logical operations on the edge image and the water-shore segmentation mask, and extracting the shoreline coordinates.

[0025] The floating object identification module includes using a target detection algorithm to obtain the center point coordinates of the floating objects, and using a density clustering algorithm to cluster the floating objects based on the center point coordinates to obtain the distribution density of the floating objects;

[0026] The early warning analysis module includes outputting corresponding siltation early warning information based on the contribution of the shoreline change rate and the floating object density change rate to river siltation.

[0027] The shoreline change rate is the ratio of the change in shoreline coordinates within the monitoring period to the monitoring period itself, and the floating object density change rate is the ratio of the change in floating object distribution density within the monitoring period to the floating object distribution density at the start time.

[0028] Preferably, the method further includes:

[0029] The optical flow method is the Farneback optical flow method, which calculates the optical flow amplitude of pixels in the shoreline neighborhood, sets the optical flow amplitude threshold to 0.5, and obtains the shoreline segmentation mask.

[0030] The edge detection algorithm is a multi-scale edge detection algorithm based on the Canny operator. It uses the Canny operator to perform edge extraction and weighted fusion on the image data at at least three spatial resolutions to obtain the edge image. The edge image is then subjected to an AND operation with the waterfront segmentation mask to obtain the shoreline coordinates.

[0031] The target detection algorithm is an improved YOLOv8 algorithm, which identifies the bounding box of the floating object and uses the center coordinates of the bounding box as the center point coordinates of the floating object.

[0032] The density clustering algorithm is the DBSCAN algorithm, which clusters the floating objects based on the coordinates of the center point by setting the clustering radius and the minimum number of clustering points, and obtains the floating object accumulation area. The ratio of the accumulation area to the water area in the image data is defined as the floating object distribution density.

[0033] The contribution rate includes the degree of influence of the shoreline change rate and the floating debris density change rate on the river channel siltation, and its expression is:

[0034]

[0035] In the formula, I represents the contribution level. and These represent the rate of change of the shoreline and the rate of change of the density of the floating objects, respectively. and The weighting coefficients representing the rate of change of the shoreline and the rate of change of the density of floating objects are respectively set to 0.4 and 0.6.

[0036] The siltation early warning information includes the following: if the contribution I < 0.3, the output is no siltation phenomenon; if 0.3 ≤ I < 0.5, the output is to increase the monitoring frequency; if 0.5 ≤ I < 1, the output is that staff need to be arranged to confirm the degree of siltation on site; if I > 1, the output is that serious siltation has occurred and the dredging plan needs to be activated immediately.

[0037] Preferably, the method further includes:

[0038] The spatial resolution is generated by a three-layer Gaussian pyramid with a size ratio of 1:0.5:0.25.

[0039] The improved YOLOv8 algorithm includes adding a dynamic attention head between the Backbone and Neck of the YOLOv8 algorithm, and introducing river context features as additional input to the YOLOv8 algorithm;

[0040] The cluster radius is obtained through a K-distance map, where the k value of the K-distance map is set to 2, and the minimum number of cluster points is 3.

[0041] Preferably, the method further includes:

[0042] The dynamic attention head includes large-scale features, medium-scale features, and small-scale features, with initial weights of 0.4, 0.3, and 0.3, respectively. The large-scale features are used to adapt to the needs of small target detection, the medium-scale features are used to balance detection accuracy and speed, and the small-scale features are used to improve the accuracy of large target localization.

[0043] The river context features include water reflectivity, water flow direction, and illumination conditions. The water reflectivity is used to quantify the degree of water reflectivity and help distinguish between reflective areas of the water and the floating objects. The water flow direction is used to help determine the trajectory of the floating objects. The illumination conditions are used to adapt the detection parameters under different illumination conditions.

[0044] Preferably, step S3 further includes:

[0045] The model evaluation metrics include accuracy, which is expressed as follows:

[0046]

[0047] In the formula, Acc represents the accuracy rate, TP represents the total number of verification results that are consistent with the actual situation, and FN represents the total number of verification results that are inconsistent with the actual situation;

[0048] The evaluation criteria include an accuracy rate (Acc) ≥ 0.8.

[0049] The advantages of this invention compared to existing methods are:

[0050] (1) High shoreline extraction accuracy and strong environmental adaptability. Considering the difficulty and poor accuracy of shoreline identification in complex environments, this invention uses the Farneback optical flow method to calculate the pixel motion vector of the pixels in the shoreline neighborhood, effectively distinguishing dynamic water features from static riverbank features. Combined with a multi-scale edge extraction algorithm based on the Canny operator, it achieves accurate and complete shoreline extraction, significantly improving the robustness and accuracy of shoreline dynamic perception, and ensuring accurate acquisition of shoreline coordinates and their rate of change during the monitoring period.

[0051] (2) Accurate floating object detection and strong quantification capabilities. Addressing the issues of high false negative and missed detection rates for multi-scale floating objects and subjective accumulation identification, this invention significantly improves the detection accuracy and anti-interference capabilities of multi-scale floating objects by adding a dynamic attention head between the Backbone and Neck of the YOLOv8 algorithm and introducing river context features as additional input. Furthermore, DBSCAN clustering combined with K-distance maps enables objective and accurate identification of floating object accumulation areas, improving the accuracy of calculating floating object density and its rate of change during the monitoring period, and avoiding subjective human error.

[0052] (3) High reliability and practicality of early warning. This invention uses a dual-indicator collaborative analysis of the rate of change of shoreline coordinates and the rate of change of floating debris density to achieve graded early warning of river siltation, effectively reducing the risk of false alarms from single early warning indicators and significantly improving the accuracy and reliability of river siltation early warning. At the same time, through this non-contact closed-loop management mechanism of sensing, analysis and early warning, the monitoring range and early warning efficiency of river siltation early warning are significantly improved. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a flowchart illustrating a visual perception-based intelligent early warning method for river siltation in an embodiment of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention more apparent and understandable, the technical solutions of this invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0056] Please see Figure 1 As shown in the figure, an intelligent early warning method for river siltation based on visual perception provided by an embodiment of the present invention includes the following steps:

[0057] S1. Collect image data of the river channel to be measured, and preprocess the image data to obtain preprocessed image data.

[0058] Step S1 in this embodiment of the invention specifically includes:

[0059] Image data of the river channel under test was collected under different weather conditions (sunny, rainy, foggy) and at different times of day (morning, noon, evening), including clear shorelines, various floating objects, and characteristics of light changes. The image data was then subjected to noise removal and image enhancement. Noise removal included using a bilateral filtering algorithm to denoise the image data, effectively removing noise while preserving edge and detail information to the greatest extent. Image enhancement included using a multi-scale retinal enhancement algorithm with color restoration (MSRCR) to enhance the denoised image data, improving brightness and contrast in low-light or degraded images while maintaining color accuracy and detail information through color restoration technology, making it more consistent with human visual perception. This resulted in preprocessed image data. A mapping matrix between the preprocessed image data and the world coordinate system was established to provide a basis for subsequent pixel coordinate conversion to geographic coordinates.

[0060] In this embodiment of the invention, the spatial domain standard deviation of the bilateral filtering algorithm is set to 5, which is used to control the spatial neighborhood weight, avoid excessive blurring of edges, and ensure the clarity of the subsequent shoreline and floating object outlines; the gray domain standard deviation of the bilateral filtering algorithm is set to 20, which is used to control the gray similarity weight, focus on filtering high gray noise such as reflections, and retain the detailed features of the floating objects.

[0061] The MSRCR employs three Gaussian filter kernels of different scales: a small-scale kernel, a medium-scale kernel, and a large-scale kernel, with their radii set to 15, 80, and 150, respectively. The small-scale kernel is used to enhance local details of floating objects and highlight their outlines. The medium-scale kernel is used to balance the overall brightness of the river area, avoiding local overexposure or underexposure. The large-scale kernel is used to eliminate large areas of backlight shadows, ensuring image quality under complex lighting conditions such as overcast skies and backlighting.

[0062] Four fixed geographic landmarks near the river channel are selected from the preprocessed image data. Their corresponding geographic coordinates in the world coordinate system are obtained, and their corresponding pixel coordinates are marked in the pixel coordinate system. The mapping matrix is ​​then solved using the least squares method, and its expression is as follows:

[0063]

[0064] In the formula, P represents the mapping matrix, which is a 3×3 matrix, i.e. X and Y represent the horizontal and vertical coordinates of the geographic coordinates, respectively, and x and y represent the horizontal and vertical coordinates of the pixel coordinates, respectively.

[0065] The matrix parameters of the mapping matrix P are solved using the geographic coordinates and pixel coordinates of the four fixed geographic landmarks to achieve mutual conversion between the pixel coordinate system and the world coordinate system.

[0066] This step effectively solves the problems of image degradation and measurement distortion caused by water surface reflection, backlighting and shadows, floating object boundary noise, and coordinate mismatch in river scenes by sequentially applying bilateral filtering algorithm to the image data, using MSRCR for image enhancement, and establishing a mapping matrix between the image data in the pixel coordinate system and the world coordinate system. This ensures that high-quality image data is input in subsequent steps.

[0067] S2. Based on the preprocessed image data, construct an image dataset, randomly divide the image dataset into a training set and a validation set according to a certain ratio, construct a siltation early warning model, which is used to determine the risk of river siltation and output siltation early warning information. Use the training set to train the siltation early warning model to obtain the trained siltation early warning model.

[0068] Step S2 in this embodiment of the invention specifically includes:

[0069] The preprocessed image data is used to construct an image dataset. This dataset is then randomly flipped and its brightness adjusted to expand the dataset and improve the model's generalization ability. The image dataset is then randomly divided into a training set and a validation set in an 8:2 ratio.

[0070] A siltation early warning model is constructed, comprising a shoreline extraction module, a floating object identification module, and an early warning analysis module. This model is used to determine the risk of river siltation, and its output is siltation early warning information. The shoreline extraction module acquires shoreline coordinates from the image data, the floating object identification module identifies the distribution density of floating objects in the image data, and the early warning analysis module analyzes the shoreline change rate and floating object density change rate within the monitoring period, outputting the corresponding siltation early warning information to achieve tiered early warning for river siltation.

[0071] The siltation early warning model is iteratively trained using the training set. A loss function is constructed based on shoreline fitting loss and floating object distribution density detection loss, which is used to calculate the loss value of the siltation early warning model during iterative training. By setting optimizer parameters and introducing an early stopping mechanism, the siltation early warning model is trained and optimized to obtain a trained siltation early warning model.

[0072] In this embodiment, the shoreline extraction module specifically includes:

[0073] The preprocessed image data is converted into a grayscale image using a weighted average method, preserving the grayscale difference between the shoreline and the water area. The Farneback optical flow method is used to calculate the optical flow vector of each pixel within the shoreline neighborhood in two consecutive grayscale images, and the optical flow amplitude of the optical flow vector is extracted. The shoreline neighborhood is the area within 10 pixels above and below the shoreline.

[0074] The expression for the optical flow amplitude is:

[0075]

[0076] In the formula, mag represents the optical flow amplitude, and u and v represent the components of the optical flow vector in the horizontal and vertical directions, respectively.

[0077] The optical flow amplitude threshold M is set to 0.5. If the optical flow amplitude mag of a pixel is less than the optical flow amplitude threshold M, the pixel is determined to be in a riverbank region, and its corresponding mask value is set to 0. Otherwise, it is determined to be in a water area, and its mask value is set to 1. In addition, the mask values ​​of all pixels outside the shoreline neighborhood are set to 0 to obtain a water area segmentation mask.

[0078] By constructing a three-layer Gaussian pyramid with a size ratio of 1:0.5:0.25, the grayscale image is sequentially generated into an original-size grayscale image, a 1 / 2-size grayscale image, and a 1 / 4-size grayscale image using a scaling factor of 0.5. The low threshold of the Canny operator is set to 50 and the high threshold to 150. Using a multi-scale edge detection algorithm based on the Canny operator, the edges of the three grayscale images are extracted to obtain the corresponding initial edge images. The initial edge images corresponding to the 1 / 2-size and 1 / 4-size grayscale images are upsampled to the original size using linear interpolation. These initial edge images are then fused with the initial edge images corresponding to the original-size grayscale image using weights of 0.3, 0.3, and 0.4. The aim is to complete the detail information with a small resolution, balance the fusion accuracy with a medium resolution, and ensure the overall contour with the original resolution, ultimately obtaining the edge image.

[0079] The edge image and the water segmentation mask are subjected to an AND operation to filter out dynamic interference and non-shoreline edges, retaining only shoreline pixels. By performing segmented fitting and splicing on the filtered shoreline pixels, a continuous smooth shoreline is obtained. The coordinates of the shoreline pixels on the smooth shoreline are extracted, and the mapping matrix P is used to convert the shoreline pixel coordinates in the pixel coordinate system to the corresponding shoreline coordinates in the world coordinate system.

[0080] The floating object identification module specifically includes:

[0081] An improved YOLOv8 algorithm is constructed by adding a dynamic attention head between the Backbone and Neck of the YOLOv8 algorithm and introducing river context features as additional input to the YOLOv8 algorithm.

[0082] The dynamic attention head includes large-scale features, medium-scale features, and small-scale features, with initial weights of 0.4, 0.3, and 0.3, respectively. The large-scale features have a feature map size of 80×80, corresponding to small floating objects, and their weights are used to adapt to the detection requirements of small targets. The medium-scale features have a feature map size of 40×40, corresponding to medium-sized floating objects, and their weights are used to balance detection accuracy and speed. The small-scale features have a feature map size of 20×20, corresponding to large floating objects, and their weights are used to emphasize the accuracy of large target localization. During model training, the weights can be dynamically adjusted based on the detection error.

[0083] The river context features include water reflectance intensity, water flow direction, and lighting conditions. The water reflectance intensity is obtained by averaging the V channel in the HSV color space to quantify the degree of water reflectance and help distinguish between water reflectance and floating objects. The water flow direction is determined by the optical flow trend in the Farneback optical flow method to help determine the trajectory of the floating objects. The lighting conditions are determined by calculating the grayscale mean of the image data. If the grayscale mean is less than 80, the lighting conditions are determined to be cloudy / nighttime; otherwise, they are determined to be sunny. This also helps predict the accumulation trend of floating objects.

[0084] The improved YOLOv8 algorithm is used to identify the floating objects in the image data, and outputs the bounding box corresponding to each floating object. The center point pixel coordinates of the bounding box are extracted and converted into the corresponding geographic coordinates using the mapping matrix P, which are used as the center point coordinates of the floating object.

[0085] The minimum number of cluster points for the DBSCAN algorithm is set to 3, and the k value of the K-distance graph is set to 2. The distance from the coordinates of each center point to its k-th nearest neighbor is calculated. The distances are arranged in ascending order and drawn into a K-distance graph. The distance corresponding to the "inflection point" of the K-distance graph is used as the clustering radius of the DBSCAN algorithm.

[0086] The floating objects are clustered based on the cluster radius, resulting in multiple floating object accumulation regions. The floating object distribution density is defined as the ratio of the total area of ​​the floating object accumulation regions to the total area of ​​the water area within the monitoring range.

[0087]

[0088] In the formula, ρ represents the distribution density of the floating objects, and S C With S R These represent the total area of ​​the floating debris accumulation area and the total area of ​​the water area, respectively.

[0089] The early warning analysis module specifically includes:

[0090] Within the monitoring period T, the change L of each shoreline coordinate on the shoreline is calculated from the start time to time T. The average value ΔL of the changes in all shoreline coordinates is obtained. The ratio of the average value ΔL to the monitoring period T is defined as the shoreline change rate.

[0091]

[0092] Similarly, the distribution density of the floating objects at the start time and time T is calculated. and The ratio of the difference in the density of the floating objects at time T and the starting time to the density of the floating objects at the starting time is defined as the rate of change of the floating object density, i.e.

[0093]

[0094] The analytic hierarchy process (AHP) was used to analyze the shoreline change rate R1 and the floating debris density change rate R2. The decision matrix of the AHP is as follows: By analyzing the contribution of the shoreline change rate R1 and the floating debris density change rate R2 to river siltation, corresponding siltation early warning information is output. The contribution can be expressed as:

[0095]

[0096] In the formula, I represents the contribution level. and The weighting coefficients for the rate of change of the shoreline and the rate of change of the density of floating objects are 0.4 and 0.6, respectively.

[0097] If the contribution rate I < 0.3, the river channel under test is considered to be unblocked, and the output siltation warning information is no siltation phenomenon. If 0.3 ≤ I < 0.5, the siltation warning information is to increase the monitoring frequency. If 0.5 ≤ I < 1, the siltation warning information is to arrange staff to confirm the degree of siltation on site. If I > 1, the siltation warning information is that serious siltation has occurred, and the dredging plan needs to be activated immediately.

[0098] The expression for the loss function is:

[0099]

[0100] In the formula, Let L1 and L2 represent the shoreline fitting loss and the floating object distribution density detection loss, respectively, and let α and β represent the preset loss weights, with values ​​of 0.3 and 0.7, respectively.

[0101] The expression for the shoreline fitting loss is:

[0102]

[0103] In the formula, N represents the total number of pixels corresponding to the shoreline coordinates, and D and E represent the shoreline coordinates obtained by the siltation early warning model and the actual shoreline coordinates, respectively.

[0104] The expression for the detection loss of the floating object distribution density is:

[0105]

[0106] In the formula, S E This represents the total area of ​​the actual floating debris accumulation zone.

[0107] The Adam optimizer was selected as the optimization algorithm during training. The learning rate of the Adam optimizer was set to 1e-3, the decay rate for first-order moment estimation was set to 0.9, the decay rate for second-order moment estimation was set to 0.999, and the numerical stability parameter was set to 1e-7. The parameters of the Adam optimizer were dynamically adjusted after each round of training iterations.

[0108] The batch size is set to 8 and the maximum number of iterations is set to 50. An early stopping mechanism is introduced during the iterative training process to prevent overfitting. The early stopping mechanism includes stopping the iterative training if the loss value of the loss function does not decrease in 10 consecutive rounds of iterative training, saving the current optimal model parameters, and obtaining the trained siltation early warning model. Otherwise, the parameters of the last round of iteration are used as the model parameters of the trained siltation early warning model.

[0109] This step unifies the preprocessed image data into an image dataset and randomly divides it according to a certain ratio. This ensures the diversity of training samples while retaining an independent validation set for unbiased evaluation. This provides the model with a stable and reproducible data foundation for subsequent shoreline change capture and floating object density change capture, avoiding overfitting or evaluation distortion due to data imbalance.

[0110] By employing a three-pronged mechanism—combining optical flow with multi-scale edge detection algorithms to accurately extract shoreline coordinates, improving target detection algorithms with density clustering to identify and quantify floating debris density, and constructing a loss function to achieve end-to-end model training—this approach effectively addresses the problems of poor shoreline capture accuracy, difficulty in quantifying floating debris accumulation, and challenges in balancing multi-task optimization in existing technologies. The constructed siltation early warning model continuously self-corrects shoreline coordinate and density estimates during the training phase, providing a stable and reliable decision-making basis for accurate early warning information output. Furthermore, by establishing a multi-level siltation early warning system, the workload of on-site personnel for inspections can be significantly reduced.

[0111] S3. Input the validation set into the trained siltation early warning model to generate validation results. Evaluate the validation results using model evaluation metrics. If the validation results meet the evaluation criteria, the trained siltation early warning model is obtained and proceeds to S4. If the validation results do not meet the evaluation criteria, return to S2 for retraining.

[0112] Step S3 in this embodiment of the invention specifically includes:

[0113] The validation set is input into the trained siltation early warning model, and its output is the siltation early warning information corresponding to the image data in the validation set. The siltation early warning information corresponding to the validation set is evaluated by introducing an accuracy rate. An evaluation standard for the accuracy rate is set. If the accuracy rate meets the evaluation standard, the trained siltation early warning model is obtained and proceeds to S4. Otherwise, it returns to S2 and iterates and trains again.

[0114] In this embodiment, the expression for the accuracy rate is:

[0115]

[0116] In the formula, Acc represents the accuracy rate, TP represents the total number of instances where the siltation warning information and the siltation conclusions drawn by on-site personnel in the validation set are consistent, and FN represents the total number of instances where the warning information and the siltation conclusions drawn by on-site personnel in the validation set are inconsistent. The siltation conclusion refers to the siltation warning conclusion reached by on-site personnel after conducting an on-site survey of the river channel under test in the real environment corresponding to the image data. This conclusion includes the absence of siltation, the need to increase monitoring frequency, the need to confirm the degree of siltation, and the need to immediately initiate a dredging plan.

[0117] The evaluation criterion is set as Acc ≥ 0.8. If the evaluation criterion is met, the trained siltation early warning model is obtained and proceeds to S4; otherwise, it returns to S2 to re-perform the iterative training.

[0118] This step aims to validate the siltation early warning model using a validation set. By comparing the consistency between the siltation early warning information output by the model and the siltation conclusions under real conditions, a well-trained siltation early warning model can be obtained, ensuring the accuracy and reliability of the siltation early warning model output during actual deployment.

[0119] S4. Input the image to be tested into the trained siltation early warning model and output the siltation early warning information corresponding to the image to be tested.

[0120] Step S4 in this embodiment of the invention specifically includes:

[0121] The image of the river channel to be tested, which is not in the dataset, is input into the trained siltation early warning model. The shoreline coordinates of the image are extracted by the Farneback optical flow method combined with the multi-scale edge detection algorithm based on the Canny operator. The floating object distribution density of the image is identified by the improved YOLOv8 algorithm combined with the DBSCAN density clustering algorithm. By analyzing the rate of change of the shoreline coordinates and the rate of change of the floating object density within the monitoring period, the siltation early warning information corresponding to the image is output.

[0122] This embodiment acquires image data of the river channel under test at different time periods, including the shoreline, and performs bilateral filtering denoising and MSRCR image enhancement processing. Based on fixed geographic landmarks, a mapping matrix between the pixel coordinate system and the world coordinate system is established to construct an image dataset and a siltation early warning model. The shoreline coordinates are extracted using optical flow method combined with multi-scale edge detection algorithm, and the distribution density of floating objects is identified by improving the target detection algorithm combined with density clustering algorithm. By comprehensively analyzing the rate of change of shoreline coordinates and the rate of change of floating object distribution density within the monitoring period, a graded early warning of river siltation is achieved.

[0123] This method can accurately acquire changes in shoreline coordinates and floating debris distribution density, and through collaborative analysis of their contribution to river siltation, it enables tiered early warning of river siltation risk. This effectively reduces the risk of false alarms from single early warning indicators, significantly improving the accuracy and reliability of siltation warnings. Furthermore, by employing purely visual perception technology, it avoids the limitations of existing sensor solutions, such as high maintenance costs, limited monitoring range, and low monitoring efficiency, making it applicable to various complex river systems.

[0124] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of the present invention.

Claims

1. A visual perception-based intelligent warning method for river channel silting, characterized in that, include: S1. Collect image data of the river channel to be measured, and preprocess the image data to obtain preprocessed image data; S2. Based on the preprocessed image data, construct an image dataset, randomly divide the image dataset into a training set and a validation set according to a certain ratio, construct a siltation early warning model, which is used to determine the risk of river siltation and output siltation early warning information. Use the training set to train the siltation early warning model to obtain the trained siltation early warning model. The siltation early warning model includes a shoreline extraction module, a floating object identification module, and an early warning analysis module; The shoreline extraction module is used to obtain the shoreline coordinates in the image data; The floating object recognition module is used to identify the distribution density of floating objects in the image data; The early warning analysis module is used to analyze the shoreline change rate and floating object density change rate within the monitoring period, and output the corresponding siltation early warning information. The image dataset is randomly divided into the training set and the validation set in an 8:2 ratio. The shoreline extraction module includes distinguishing water features and riverbank features in the image data using optical flow, outputting a water-shore segmentation mask, extracting shoreline information from the image data using an edge detection algorithm, outputting an edge image, performing logical operations on the edge image and the water-shore segmentation mask, and extracting the shoreline coordinates. The floating object identification module includes using a target detection algorithm to obtain the center point coordinates of the floating objects, and using a density clustering algorithm to cluster the floating objects based on the center point coordinates to obtain the distribution density of the floating objects; The early warning analysis module includes outputting corresponding siltation early warning information based on the contribution of the shoreline change rate and the floating object density change rate to river siltation. The shoreline change rate is the ratio of the change in shoreline coordinates within the monitoring period to the monitoring period itself, and the floating object density change rate is the ratio of the change in floating object distribution density within the monitoring period to the floating object distribution density at the start time. S3. Input the validation set into the trained siltation early warning model to generate validation results. Evaluate the validation results using model evaluation metrics. If the validation results meet the evaluation criteria, the trained siltation early warning model is obtained and proceeds to S4. If the validation results do not meet the evaluation criteria, return to S2 for retraining. S4. Input the image to be tested into the trained siltation early warning model and output the siltation early warning information corresponding to the image to be tested.

2. The method of claim 1, wherein, Step S1 also includes: The image data includes images of the river channel under test, taken in different weather conditions and at different times, and containing the shoreline; the preprocessing includes sequentially removing noise and enhancing the image data.

3. The method of claim 1, wherein, The method includes: The optical flow method is the Farneback optical flow method, which calculates the optical flow amplitude of pixels in the shoreline neighborhood, sets the optical flow amplitude threshold to 0.5, and obtains the shoreline segmentation mask. The edge detection algorithm is a multi-scale edge detection algorithm based on the Canny operator. It uses the Canny operator to perform edge extraction and weighted fusion on the image data at at least three spatial resolutions to obtain the edge image. The edge image is then subjected to an AND operation with the waterfront segmentation mask to obtain the shoreline coordinates. The target detection algorithm is an improved YOLOv8 algorithm, which identifies the bounding box of the floating object and uses the center coordinates of the bounding box as the center point coordinates of the floating object. The density clustering algorithm is the DBSCAN algorithm, which clusters the floating objects based on the coordinates of the center point by setting the clustering radius and the minimum number of clustering points, and obtains the floating object accumulation area. The ratio of the accumulation area to the water area in the image data is defined as the floating object distribution density. The contribution rate includes the degree of influence of the shoreline change rate and the floating debris density change rate on the river channel siltation, and its expression is: ; In the formula, I represents the contribution level. and These represent the rate of change of the shoreline and the rate of change of the density of the floating objects, respectively. and The weighting coefficients representing the rate of change of the shoreline and the rate of change of the density of floating objects are respectively set to 0.4 and 0.

6. The siltation early warning information includes the following: if the contribution I < 0.3, the output is no siltation phenomenon; if 0.3 ≤ I < 0.5, the output is to increase the monitoring frequency; if 0.5 ≤ I < 1, the output is that staff need to be arranged to confirm the degree of siltation on site; if I > 1, the output is that serious siltation has occurred and the dredging plan needs to be activated immediately.

4. The method of claim 3, wherein, The method further includes: The spatial resolution is generated by a three-layer Gaussian pyramid with a size ratio of 1:0.5:0.

25. The improved YOLOv8 algorithm includes adding a dynamic attention head between the Backbone and Neck of the YOLOv8 algorithm, and introducing river context features as additional input to the YOLOv8 algorithm; The cluster radius is obtained through a K-distance map, where the k value of the K-distance map is set to 2, and the minimum number of cluster points is 3.

5. The method of claim 4, wherein, The method further includes: The dynamic attention head includes large-scale features, medium-scale features, and small-scale features, with initial weights of 0.4, 0.3, and 0.3, respectively. The large-scale features are used to adapt to the needs of small target detection, the medium-scale features are used to balance detection accuracy and speed, and the small-scale features are used to improve the accuracy of large target localization. The river context features include water reflectivity, water flow direction, and illumination conditions. The water reflectivity is used to quantify the degree of water reflectivity and help distinguish between reflective areas of the water and the floating objects. The water flow direction is used to help determine the trajectory of the floating objects. The illumination conditions are used to adapt the detection parameters under different illumination conditions.

6. The method of claim 1, wherein, Step S3 also includes: The model evaluation metrics include accuracy, which is expressed as follows: ; In the formula, Acc represents the accuracy rate, TP represents the total number of verification results that are consistent with the actual situation, and FN represents the total number of verification results that are inconsistent with the actual situation; The evaluation criteria include an accuracy rate (Acc) ≥ 0.8.

Images