A small reservoir water level intelligent identification method and system based on general pictures

Abstract

The application provides a small reservoir water level intelligent identification method and system based on general pictures, which comprises the following steps: firstly, a three-dimensional twin grid model of a reservoir dam is constructed, and structural feature point three-dimensional coordinates are extracted and stored in a database. A monitoring image is acquired, an initial water level line is preliminarily identified, and two-dimensional pixel coordinates of corresponding feature points in the image are matched. A real-time pose matrix of a camera is solved based on matching pairs of three-dimensional and two-dimensional coordinates. The three-dimensional model is projected to the image by using the pose matrix to generate a virtual measurement grid with three-dimensional information. Pixels on the initial water level line are back-projected to the physical space to form a three-dimensional point cloud by the grid, and an optimal water body plane is fitted by iterative optimization. Finally, the real physical height difference between the water body plane and the dam top is calculated, and the absolute height of the dam top is subtracted from the height difference to obtain the final real-time absolute water level height. The application has the effect of improving the accuracy of water level identification based on image processing.

Description

Technical Field

[0001] This invention belongs to the field of image recognition technology, specifically relating to a method and system for intelligent recognition of water levels in small reservoirs based on general images. Background Technology

[0002] Traditional methods for monitoring small reservoirs mainly include manual inspections and the installation of dedicated water level sensing equipment. However, manual inspections have inherent drawbacks such as slow response times, high labor intensity, and high risks during the flood season. While deploying professional water level gauges such as float-type, pressure-type, or ultrasonic gauges can achieve automated monitoring, they face a series of problems, including high equipment costs, difficulties in laying power supply and signal transmission lines, a large amount of subsequent maintenance work, and susceptibility to natural or man-made damage. There are significant obstacles to their widespread adoption in the numerous small reservoirs. Therefore, utilizing security monitoring cameras already widely deployed in reservoirs for water level identification has become the mainstream technological direction today.

[0003] A preliminary image processing-based approach typically involves placing a graduated physical water gauge on the dam and then using algorithms to identify the boundary between the water body and the gauge in the image to read the value. However, this method is severely limited in accuracy and reliability. Surveillance cameras are not precise measuring instruments; their installation positions can experience minute pose shifts due to factors such as wind, vibration, and temperature changes. These seemingly insignificant angular or positional changes, under perspective projection, can cause significant displacement of pixels at the far end of the image, rendering any pre-defined pixel-elevation relationship completely invalid and resulting in serious measurement errors. Summary of the Invention

[0004] This invention provides a method and system for intelligent water level recognition in small reservoirs based on general images, in order to solve the above-mentioned technical problems.

[0005] In a first aspect, the present invention provides a method for intelligent identification of water levels in small reservoirs based on general images, the method comprising the following steps: A three-dimensional model of the target reservoir dam is constructed, and a three-dimensional twin mesh model of the target reservoir dam is built. The three-dimensional coordinates of multiple structural feature points in a preset physical coordinate system are extracted from the three-dimensional twin mesh model to form a feature array database. The system acquires real-time monitoring images from surveillance cameras in the target reservoir dam. It then processes these images using an image segmentation model to initially identify the initial water level line and initial dam crest line in the real-time monitoring images, as well as the two-dimensional pixel coordinates of the target structural feature points matched in the feature array database in the real-time monitoring images. Based on the three-dimensional coordinates and corresponding matching two-dimensional pixel coordinates in the feature array database, the real-time pose matrix of the surveillance camera relative to the physical coordinate system is solved by perspective projection geometry. Based on the real-time pose matrix, the three-dimensional twin mesh model is projected onto the real-time monitoring image to generate a virtual measurement mesh that is registered with the image content in the real-time monitoring image and each image pixel of the real-time monitoring image is accompanied by three-dimensional coordinate information. The pixel set on the initial water level line is back-projected into the physical coordinate system using a virtual measurement grid to form a three-dimensional point cloud. The three-dimensional point cloud is then processed by an iterative optimization algorithm based on physical plane constraints to obtain the optimal water body plane equation. Calculate the actual physical height difference between the optimal water body plane and the crest of the target reservoir dam in three-dimensional space; The real-time absolute water level elevation of the reservoir where the target reservoir dam is located is calculated by subtracting the actual physical elevation difference from the preset absolute elevation value of the dam crest.

[0006] Optionally, the step of performing three-dimensional modeling of the target reservoir dam, constructing a three-dimensional twin mesh model of the target reservoir dam, and extracting the three-dimensional coordinates of multiple structural feature points in a preset physical coordinate system from the three-dimensional twin mesh model to form a feature array database includes the following steps: The target reservoir dam was photographed from multiple angles using drones to obtain multi-view image data of the target reservoir dam; A 3D dense point cloud of the target reservoir dam is generated based on multi-view image data and using a motion reconstruction structure algorithm. Mesh reconstruction and texture mapping are performed on the 3D dense point cloud to generate a 3D twin mesh model of the target reservoir dam; Multiple candidate structural feature points were initially selected on the 3D twin mesh model using a feature detection algorithm; For different combinations of feature points composed of multiple candidate structural feature points, the spatial resolution index of different feature point combinations within the field of view of the monitoring camera in the target reservoir dam is calculated respectively. Feature point combinations with optimal spatial resolution are selected as structural feature points, and the three-dimensional coordinates of the structural feature points in a preset physical coordinate system are stored in a preset database to form a feature array database.

[0007] Optionally, the step of processing the real-time monitoring image using an image segmentation model to initially identify the initial water level line and initial dam crest line in the real-time monitoring image, as well as the two-dimensional pixel coordinates of the target structural feature points matched in the feature array database in the real-time monitoring image, includes the following steps: A training image dataset containing the initial water level line, initial dam crest line, and structural feature points was constructed based on historical monitoring images of the target reservoir dam and manually labeled. Data augmentation processing is performed on the training image dataset, including simulating image transformations of rain, fog, different light intensities, and viewpoint jitter. Construct and configure an image segmentation model, which includes a cyclic feature aggregation module for capturing linear feature context information; The image segmentation model is trained using the enhanced training image dataset until the segmentation loss function of the image segmentation model converges. The real-time monitoring image is input into the trained image segmentation model. The image segmentation model identifies the initial water level line and initial dam crest line in the real-time monitoring image, as well as the two-dimensional pixel coordinates of the target structure feature points matched in the feature array database in the real-time monitoring image.

[0008] Optionally, the step of solving the real-time pose matrix of the surveillance camera relative to the physical coordinate system based on the three-dimensional coordinates and corresponding matching two-dimensional pixel coordinates in the feature array database through perspective projection geometry includes the following steps: Select a subset of core feature points, consisting of multiple structural feature points, from all target structural feature points for preliminary pose estimation; The initial real-time pose matrix is ​​obtained by combining the three-dimensional coordinates of the core feature point subset and the corresponding matching two-dimensional pixel coordinates and solving the PnP algorithm. Using the initial real-time pose matrix, the three-dimensional coordinates of all structural feature points in the feature array database that were not selected into the core feature point subset are projected onto the image plane of the real-time monitoring image to generate the predicted two-dimensional pixel coordinates of each of the unselected structural feature points. Search for the actual matching two-dimensional pixel coordinates within the neighborhood of the predicted two-dimensional pixel coordinates to construct a validated target feature point matching set. The real-time pose matrix of the surveillance camera relative to the physical coordinate system is obtained by matching the target feature points and using the PnP algorithm.

[0009] Optionally, the step of selecting a subset of core feature points, consisting of multiple structural feature points, from all target structural feature points for preliminary pose estimation includes the following steps: Obtain the three-dimensional coordinates of all target structural feature points; All target structural feature points are combined into multiple candidate feature point subsets by a preset number; For each subset of candidate feature points, a geometric stability index is calculated based on the geometric distribution of the three-dimensional coordinates of all target structural feature points contained in the subset. The geometric stability index is used to quantify the ability of the subset of candidate feature points to resist noise and collinearity when solving the PnP algorithm. The subset of candidate feature points with the highest geometric stability index is selected as the core feature point subset.

[0010] Optionally, the step of using a virtual measurement grid to project the pixel set on the initial water level line into a physical coordinate system to form a three-dimensional point cloud, and then processing the three-dimensional point cloud using an iterative optimization algorithm based on physical plane constraints to fit the optimal water body plane equation includes the following steps: The pixel set on the initial water level line is back-projected onto the physical coordinate system through a virtual measurement grid to obtain the initial three-dimensional point cloud; The initial 3D point cloud was processed using a random sampling consensus algorithm, and the initial water body plane equation was obtained by fitting and the initial set of interior points was identified. The spatial intersection of the initial water body plane equation and the three-dimensional twin mesh model is calculated, and the spatial intersection is projected onto the image plane according to the real-time pose matrix to form the theoretical water level line; Based on the residuals of each inlier in the initial inlier set on the image plane between the theoretical water level and the inlier in the initial inlier set, all inliers in the initial inlier set are weighted, and the optimal water body plane equation is obtained by refitting the initial inlier set based on the weighted initial inlier set.

[0011] Optionally, the step of weighting all inliers in the initial inlier set based on the residuals between the theoretical water level and each inlier in the initial inlier set on the image plane, and then re-fitting the plane based on the weighted initial inlier set to obtain the optimal water body plane equation includes the following steps: For each inlier in the initial set of inliers, the vertical projection distance of the inlier on the image plane to the theoretical water level is calculated as the pixel residual, and the density of other inliers within a preset neighborhood is calculated as the spatial consistency score. The absolute value of the pixel residual is mapped to the residual confidence score through a preset confidence function; Multiply the residual confidence score by the spatial consistency score to obtain the composite weight of each inlier in the initial inlier set; After normalizing all composite weights, the final weights corresponding to each interior point in the initial interior point set are obtained when performing weighted plane fitting. Based on the weighted initial interior point set, plane fitting is performed again to obtain the optimal water body plane equation.

[0012] Optionally, the method further includes the following steps: Continuously record and store historical time-series data composed of real-time pose matrices; Time series analysis algorithms are applied to identify and quantify the long-term pose drift caused by permanent physical displacement of surveillance cameras from historical time series data; When the long-term pose drift exceeds the preset system stability threshold, a system maintenance alarm for abnormal pose of the surveillance camera is generated, and the long-term pose drift is fed back as a correction parameter to the calculation step of the real-time pose matrix to compensate for the systematic error caused by the permanent physical displacement of the surveillance camera.

[0013] Secondly, the present invention also provides a small reservoir water level intelligent recognition system based on general images, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the small reservoir water level intelligent recognition method based on general images as described in the first aspect.

[0014] Thirdly, the present invention also provides a computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the intelligent water level recognition method for small reservoirs based on general images according to any one of the first aspects.

[0015] The beneficial effects of this invention are: This invention establishes a stable physical coordinate system as a measurement benchmark by pre-constructing a 3D twin model of the target dam and extracting structural feature points. Then, using matched feature points, the external parameter matrix of the monitoring camera is solved in real time, thereby accurately determining the camera's instantaneous spatial pose. Based on this instantaneous spatial pose, the 3D twin model can be accurately projected onto a 2D image, generating a virtual measurement grid with accompanying 3D coordinate information. This transforms non-standard monitoring images into standard, scientifically quantifiable views. Subsequent back-projection of the water level, 3D point cloud fitting, and iterative optimization based on physical plane constraints provide optimal estimation of the water level plane in real physical space, effectively filtering out noise and errors from the image segmentation stage. The final output is the absolute elevation in a unified physical coordinate system, rather than a relative value dependent on pixel changes. This solves the problem of severe measurement errors caused by camera pose offset and improves the accuracy of water level identification based on image processing. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a method for intelligent water level recognition of small reservoirs based on general images, according to one embodiment of this application.

[0017] Figure 2 This is a schematic diagram of a manually labeled dataset image in one embodiment of this application.

[0018] Figure 3 This is a schematic diagram of the architecture design of the RESA architecture in one embodiment of this application.

[0019] Figure 4 This is a schematic diagram of the detection results output by the image segmentation model after training, according to one embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0021] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0022] Figure 1 This is a flowchart illustrating a method for intelligent water level recognition in a small reservoir based on general images, as shown in one embodiment. It should be understood that, although... Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are executed, and they can be performed in other orders. Furthermore, Figure 1 At least some steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps. For example Figure 1 As shown, the intelligent water level recognition method for small reservoirs based on general images disclosed in this invention specifically includes the following steps: S101. Perform three-dimensional modeling of the target reservoir dam, construct a three-dimensional twin mesh model of the target reservoir dam, and extract the three-dimensional coordinates of multiple structural feature points in the preset physical coordinate system from the three-dimensional twin mesh model to form a feature array database.

[0023] The process involved using drones equipped with a high-precision global navigation satellite system and inertial measurement unit to perform multi-flight, multi-angle oblique photogrammetry of the dam and its surrounding environment, acquiring high-resolution image data from all angles. After acquisition, these multi-view image data were input into professional photogrammetry processing software. The relative positions and attitudes between images were calculated using a structure-of-motion (SOG) algorithm, followed by dense matching to generate a dense point cloud of the dam containing millions or even hundreds of millions of 3D coordinate points. Based on this, the point cloud was meshed using Poisson surface reconstruction or similar algorithms to construct a 3D mesh model with topological structure and geometric accuracy. Texture mapping was then performed using the original images to ultimately create a highly realistic 3D twin mesh model. Once the model was built, several structures that maintained stable geometric shapes under different lighting and seasonal conditions were selected on its surface, such as dam corners, gate column vertices, or specific markers. The 3D coordinates of these structural feature points in a preset physical world coordinate system were precisely extracted and compiled into a dedicated feature array database, serving as the benchmark for subsequent camera pose calculations.

[0024] S102. Obtain real-time monitoring images from surveillance cameras in the target reservoir dam. Process the real-time monitoring images using an image segmentation model to initially identify the initial water level line and initial dam crest line in the real-time monitoring images, as well as the two-dimensional pixel coordinates of the target structural feature points matched in the feature array database in the real-time monitoring images.

[0025] The process involves obtaining the 3D model and feature database of the dam, then connecting to the real-time video stream from the front-end monitoring cameras of the target reservoir dam, and extracting monitoring images frame by frame. For each frame of the real-time monitoring image, a pre-trained deep learning image segmentation model is invoked for semantic analysis. The core of this model is a recurrent feature aggregation network structure, specifically designed to enhance the perception of slender linear targets, effectively handling complex environments such as rain, fog, and low light at night. After receiving the image input, the model outputs a pixel-level segmentation mask, clearly marking the initial water level line and the initial dam crest line in the image. Simultaneously, the feature point detection branch integrated within the model locates and matches the corresponding target structural feature points in the current image based on feature information from the feature array database, outputting the two-dimensional pixel coordinates of these feature points in the image coordinate system. The completion of this step signifies the successful extraction of key geometric information for 3D spatial perception from unstructured image data, namely two initial contour lines and a set of corresponding 2D-3D point pairs.

[0026] S103. Based on the three-dimensional coordinates and corresponding matching two-dimensional pixel coordinates in the feature array database, the real-time pose matrix of the surveillance camera relative to the physical coordinate system is solved by perspective projection geometry.

[0027] Once the 3D coordinates of the structural feature points in the physical coordinate system and their 2D pixel coordinates in the real-time monitoring image are obtained, the real-time spatial pose of the monitoring camera can be solved. This process essentially solves the perspective n-point problem in computer vision. The core camera perspective projection model is used, which describes the mapping relationship from 3D spatial points to 2D image points. Its mathematical expression is as follows: Where s is the scale factor, These are the homogeneous two-dimensional pixel coordinates of the feature points on the image. These are the homogeneous 3D coordinates of the corresponding feature points in the physical world coordinate system. K is the camera intrinsic parameter matrix, containing inherent properties such as focal length and principal point, which is usually obtained through pre-calibration. The solution to be found is... Let R be the extrinsic parameter matrix of the camera, where R is a 3x3 rotation matrix defining the camera's orientation, and t is a 3x1 translation vector defining the camera's position. By substituting at least three pairs of matching 2D-3D point coordinates into this equation and solving it using direct linear transformation or iterative optimization algorithms, the real-time pose matrix can be accurately calculated. This allows for the quantification of the spatial state of the camera.

[0028] S104. Based on the real-time pose matrix, project the three-dimensional twin mesh model onto the real-time monitoring image to generate a virtual measurement mesh that is registered with the image content in the real-time monitoring image and has three-dimensional coordinate information attached to each image pixel of the real-time monitoring image.

[0029] After accurately calculating the real-time pose matrix of the surveillance camera, the previously constructed 3D twin mesh model can be virtually projected from its physical world coordinate system using this pose matrix, precisely superimposed onto the 2D real-time surveillance image. This process can be understood as simulating the camera's imaging process, transforming each vertex of the 3D model to the camera coordinate system according to the pose matrix, and then projecting it onto the image plane through the camera's intrinsic parameter matrix. This generates a virtual measurement mesh perfectly registered with the real-time surveillance image content. The unique aspect of this mesh is that each pixel in the image establishes a direct mapping relationship with a specific 3D spatial point on the 3D twin mesh model through this mesh. In other words, any pixel on the screen is no longer just a color value, but is assigned its 3D coordinate information in the real world. The generation of this virtual measurement mesh is equivalent to constructing a 3D index for the 2D image, providing a crucial spatial measurement benchmark for subsequent reverse calculation of the 3D water body plane from 2D water level information.

[0030] S105. Using a virtual measurement grid, the pixel set on the initial water level line is back-projected into the physical coordinate system to form a three-dimensional point cloud. The three-dimensional point cloud is then processed by an iterative optimization algorithm based on physical plane constraints to obtain the optimal water body plane equation.

[0031] In this process, using the virtual measurement grid generated in the previous step, all pixels on the initial water level line identified by the image segmentation model can be projected back to the physical coordinate system one by one. Specifically, the 3D coordinates of each pixel on the initial water level line within the virtual measurement grid are queried, and all these 3D coordinates are collected to form an initial 3D point cloud describing the boundary between the water body and the dam. Considering that the initial water level line may contain noise introduced by factors such as image recognition errors, water surface ripples, or floating objects, directly fitting the point cloud to a plane may yield inaccurate results. Therefore, an iterative optimization algorithm based on physical plane constraints, such as the Random Sample Consensus (RANSAC) algorithm, is used to process this 3D point cloud. This algorithm repeatedly randomly selects a subset of the point cloud to fit candidate planes and evaluates the plane quality based on the number of points supporting the plane (interior points), ultimately finding the optimal water body plane supported by the most data points.

[0032] S106. Calculate the actual physical height difference between the optimal water body plane and the crest of the target reservoir dam in three-dimensional space.

[0033] Once the equation for the optimal water surface is successfully fitted, the next step is to calculate the actual physical elevation difference between this water surface and the crest of the target reservoir dam in three-dimensional space. Since the 3D twin mesh model of the dam is pre-built, the geometric information of the dam crest is fully known. The 3D coordinates of a series of points representing the dam crest elevation can be extracted from the 3D model, and a unified dam crest elevation value can be determined by averaging or other methods. In hydraulic engineering, the water surface can be approximated as a horizontal plane in a static state, therefore its normal vector mainly points in the vertical direction. In this case, the equation for the optimal water surface... The A and B components in the equation are close to zero. The elevation of the water level can be approximately calculated from -D / C. Therefore, the true physical height difference between the water level and the dam crest, commonly referred to as the safety superelevation, can be obtained by calculating the coordinate difference between the two in the vertical direction (usually the Z-axis). This height difference calculation is entirely based on the geometric relationship in three-dimensional space and is unaffected by factors such as camera angle and distortion, thus ensuring the physical authenticity and accuracy of the measurement results.

[0034] S107. Subtract the actual physical elevation difference from the preset absolute elevation of the target reservoir dam crest to calculate the real-time absolute water level elevation of the reservoir where the target reservoir dam is located.

[0035] Finally, the calculated actual physical elevation difference is converted into the real-time absolute water level elevation of the reservoir, which has practical application value. In reservoir construction and management, the dam crest usually has an absolute elevation value that has been precisely surveyed and set. This dam crest absolute elevation value is pre-input into the system as a known parameter. Based on the actual physical elevation difference between the water surface and the dam crest calculated in the previous steps, the current real-time absolute water level elevation of the reservoir can be obtained through a simple subtraction operation. This calculation result directly reflects the current elevation of the reservoir water surface and is a standardized, key data that can be directly compared and used by different systems and departments. It can not only be used for daily water situation monitoring, but also for real-time comparison with key elevations such as flood control limit water levels and design flood levels, providing accurate and reliable data support for the scientific scheduling, flood warning, and safe operation of the reservoir.

[0036] In one implementation, the target reservoir dam is three-dimensionally modeled to construct a three-dimensional twin mesh model of the target reservoir dam. The three-dimensional coordinates of multiple structural feature points in a preset physical coordinate system are extracted from the three-dimensional twin mesh model to form a feature array database. This process includes the following steps: The target reservoir dam was photographed from multiple angles using drones to obtain multi-view image data of the target reservoir dam; A 3D dense point cloud of the target reservoir dam is generated based on multi-view image data and using a motion reconstruction structure algorithm. Mesh reconstruction and texture mapping are performed on the 3D dense point cloud to generate a 3D twin mesh model of the target reservoir dam; Multiple candidate structural feature points were initially selected on the 3D twin mesh model using a feature detection algorithm; For different combinations of feature points composed of multiple candidate structural feature points, the spatial resolution index of different feature point combinations within the field of view of the monitoring camera in the target reservoir dam is calculated respectively. Feature point combinations with optimal spatial resolution are selected as structural feature points, and the three-dimensional coordinates of the structural feature points in a preset physical coordinate system are stored in a preset database to form a feature array database.

[0037] In this embodiment, to construct a high-precision digital model of the dam, a professional-grade multi-rotor UAV is first employed, integrating a high-resolution full-frame camera, a differential GPS system, and a high-frequency inertial measurement unit. Before data acquisition, precise flight path planning is required, designing a combination of orthogonal and oblique flight paths involving multiple sorties. The orthogonal flight path flies parallel to the dam's axis, taking images perpendicular to the ground to ensure complete coverage of the dam crest and surrounding terrain. The oblique flight path, however, flies around the dam from different directions at preset angles, aiming to fully capture the detailed textures of the dam slope, dam facade, and vertical structures such as gates. Throughout the flight, extremely high image overlap must be maintained, typically with a forward overlap of no less than 80% and a lateral overlap of no less than 70%, to ensure that the subsequent 3D reconstruction algorithm can find a sufficient number of corresponding points. Simultaneously, to ensure the absolute geographic accuracy of the model, several ground control points are evenly distributed within the survey area, and their 3D coordinates are precisely determined using real-time dynamic measurement technology. These multi-view image data, containing high-precision position and attitude information, collectively constitute the foundational dataset for refined 3D reconstruction.

[0038] After acquiring multi-view image data, the next stage is the generation of a 3D dense point cloud. The core of this stage is the Structure in Motion (SIM) algorithm. This algorithm first automatically detects and extracts stable feature points such as scale-invariant feature transforms from all acquired images. It then matches feature descriptors across different images to establish cross-image correspondences of corresponding points. Based on these correspondences, through incremental or global optimization methods, the algorithm can simultaneously solve for the camera extrinsic parameters (i.e., spatial position and orientation at the time of capture) and the preliminary 3D spatial coordinates of these feature points for each image, forming a sparse point cloud. Next, based on the sparse point cloud and accurate camera pose, the Multi-View Stereo (MVS) algorithm is applied. The MVS algorithm utilizes known camera geometry to perform dense matching on all images at the pixel level, deeply mining the 3D spatial position of each pixel, thereby generating a 3D dense point cloud containing millions or even hundreds of millions of points. After obtaining the dense point cloud, it needs to be converted into a 3D mesh model with continuous surfaces for subsequent geometric analysis and applications. This step typically employs the Poisson surface reconstruction algorithm. This algorithm treats the input point cloud data as sampling points of a continuous 3D surface. By calculating the normal vector information of each point and its neighbors, it constructs an implicit scalar field function. The zero-value surface of this function is the desired 3D surface. Using algorithms such as the moving cube algorithm, isosurfaces are extracted from this scalar field, generating a closed triangular mesh model with a well-structured topology. After model generation, texture mapping is performed to give the model a realistic appearance. The texture mapping process involves back-projecting the initially acquired high-resolution UAV imagery, based on the calculated camera pose, onto the corresponding facets of the 3D mesh model. The algorithm intelligently selects the best-viewing and clearest image patch as the texture for each triangular facet, performs color fusion and correction, eliminates uneven lighting and stitching gaps, and ultimately generates a 3D twin mesh model with highly consistent appearance and precise dimensions with respect to the real dam.

[0039] After generating a refined 3D twin mesh model, structural feature points suitable for long-term stable tracking need to be selected from the model. To ensure objectivity and efficiency in the selection process, automated feature detection algorithms are used to replace manual selection. These algorithms are primarily based on the analysis of the model's local geometric properties. For example, the 3D Harris corner detection operator can be applied to find corner points in 3D space—locations where the surface changes significantly in multiple directions—by calculating the gradient matrix of geometric changes within the neighborhood of each vertex. Another method is to analyze the principal curvature of the surface, using the extreme points of Gaussian curvature or mean curvature as candidate feature points. These points typically correspond to convex corners, concave corners, edge intersections, or unique geometric structures on the dam surface. Then, the feature point combinations composed of different candidate points need to be quantitatively evaluated. The core of the evaluation is calculating the spatial resolution index of each combination under a specific surveillance camera viewpoint. This index measures the amplification effect of small observation errors of feature points on the image plane on the final calculated 3D pose error of the camera. A geometrically well-configured feature point combination has strong spatial resolution and is insensitive to noise during the pose calculation process.

[0040] In practical implementation, for each candidate feature point combination, a Jacobian matrix J can be constructed for its perspective projection model with respect to the camera's six-DOF pose parameters. The spatial resolution index can then be determined using the condition number of this Jacobian matrix. To quantify it, its calculation formula is: ,in The condition number represents the matrix norm. A smaller condition number indicates a well-state problem, meaning the solution is less sensitive to input errors, i.e., higher spatial resolution. By traversing or heuristically searching different combinations of feature points and calculating their respective condition numbers, an optimal combination that minimizes the condition number can be found. After evaluating the spatial resolution of all candidate feature point combinations, the combination with the optimal index (i.e., the minimum condition number) is selected as the final structural feature point used for system calibration and real-time tracking. This selected combination is not only geometrically widely distributed, avoiding collinear or coplanar degradation, but also maximizes the constraint of the camera's six degrees of freedom, ensuring the accuracy and robustness of pose calculation. After selecting the final structural feature points, the three-dimensional coordinates of these points in a preset physical coordinate system (e.g., a geodetic coordinate system with a fixed point on the dam as the origin) are accurately extracted from the 3D twin mesh model. The unique identifiers of these points and their corresponding high-precision 3D coordinates are structured and stored to form a dedicated feature array database. Once established, this database becomes the static reference for the entire intelligent recognition system. In subsequent real-time operation, it is only necessary to identify these specific structural feature points in the image to perform accurate pose calculations using the 3D coordinates stored in this database, without having to rely on the complete 3D model again.

[0041] In one embodiment, processing the real-time monitoring image using an image segmentation model to initially identify the initial water level line and initial dam crest line in the real-time monitoring image, as well as the two-dimensional pixel coordinates of the target structural feature points matched in the feature array database in the real-time monitoring image, includes the following steps: A training image dataset containing the initial water level line, initial dam crest line, and structural feature points was constructed based on historical monitoring images of the target reservoir dam and manually labeled. Data augmentation processing is performed on the training image dataset. Data augmentation processing includes simulating image transformations such as rain, fog, different light intensities, and viewpoint jitter. Build and configure an image segmentation model, which includes a cyclic feature aggregation module for capturing contextual information of linear features; The image segmentation model is trained using the enhanced training image dataset until the segmentation loss function of the image segmentation model converges. The real-time monitoring image is input into the trained image segmentation model. The image segmentation model identifies the initial water level line and initial dam crest line in the real-time monitoring image, as well as the two-dimensional pixel coordinates of the target structure feature points matched in the feature array database in the real-time monitoring image.

[0042] In this embodiment, a series of image frames that comprehensively reflect the reservoir's operational status are selected from long-archived historical monitoring images of the target reservoir dam. These images cover different seasons, different times of day (early morning, noon, evening, and night), diverse weather conditions (sunny, cloudy, rainy, and foggy), and significantly varying water levels. After selecting the images, they are annotated using image annotation software and manually by experienced technicians. The annotated images are shown below. Figure 2 As shown, the annotation content mainly falls into three categories: First, using polygon or polyline tools to accurately delineate the initial water level line at the boundary between the water body and the dam body or bank slope; second, similarly, marking a clearly identifiable outline of the dam top as the initial dam crest line; and finally, according to the definition of the feature array database, precisely locating the positions of each structural feature point on the image using keypoint annotation. The annotation results for each image are saved in a structured data format (e.g., a JSON file), which records detailed pixel coordinate information for each type of annotated object, thus forming a complete training image dataset containing the original image and corresponding ground truth labels.

[0043] To enhance the generalization ability and robustness of image segmentation models in complex and ever-changing real-world application scenarios, a series of data augmentation processes must be performed on the original manually labeled training image dataset. The essence of data augmentation is to programmatically transform images without altering the semantic meaning of the image labels to simulate various disturbances that may occur in the real world, thereby greatly expanding the diversity of training samples. Specific augmentation strategies include: simulating rainy weather by overlaying synthetic rain streaks or raindrops; applying specific image filtering algorithms, such as simulating foggy environments by increasing global atmospheric light scattering effects; randomly adjusting the brightness, contrast, and saturation of images to cover different lighting intensities and camera exposure settings; and applying small, randomized affine transformations, such as translation, rotation, and scaling, to simulate camera shake caused by wind or slight vibrations. All these image-level transformations are simultaneously applied to the corresponding labeled data, ensuring that the spatial correspondence between labels and transformed images remains accurate.

[0044] The image segmentation model employs a deep neural network architecture optimized for line feature detection, typically based on a mature encoder-decoder design paradigm. The encoder uses a powerful pre-trained convolutional neural network, such as ResNet-50, as its backbone to extract multi-level feature maps from the input image, ranging from low-level texture to high-level semantics. The key innovation of the model lies in the integration of a specially designed recurrent feature aggregation module between the encoder and decoder, or within the upsampling path of the decoder. Figure 3 The recurrent feature aggregation module can be based on the RESA architecture. Inspired by the spatial continuity prior of linear targets, the recurrent feature aggregation module can iteratively move and aggregate feature information along multiple directions, such as horizontal and vertical, on the feature layer surface. This unique recursive information propagation mechanism ensures that the features of each pixel not only contain information from its local neighborhood but also efficiently capture contextual information from distant objects on the same line, thus greatly enhancing the model's ability to perceive and segment slender, continuous targets such as water levels and dam crests. When configuring the model, it is necessary to define the standard resolution of the input image, the number of output categories (such as background, water level, dam crest, etc.), and the specific hyperparameters of the recurrent aggregation module.

[0045] Supervised training of a configured image segmentation model is performed using a data-augmented training image dataset. The training process is an iterative optimization process designed to allow the model's internal adjustable parameters (i.e., network weights) to learn the complex mapping from the input image to the correct segmentation mask. In each iteration, a small batch of images and their corresponding ground truth labels are randomly selected from the training set and fed into the model. The model performs a forward propagation on the input images, generating a set of predicted segmentation results. Subsequently, a predefined segmentation loss function is used to quantify the difference between the model's predictions and the ground truth labels. A commonly used loss function is the classification cross-entropy loss. Its expression is Where M is the total number of categories, It is an indicator variable; it is 1 if the true category is c, and 0 otherwise. This is the probability that the model predicts a pixel belongs to class c. The calculated loss value is then used to calculate the gradient of the loss with respect to each model parameter through backpropagation. Stochastic gradient descent or its variant optimizer (such as Adam) then fine-tunes and updates the model parameters based on this gradient, aiming to reduce the loss in the next iteration. This process is repeated until the model's loss value on an independent validation set no longer decreases significantly, reaching a convergence state, marking the completion of model training.

[0046] Once the image segmentation model has been trained and reached convergence, it can be deployed in practical applications to process real-time monitoring images from surveillance cameras. For each frame of the transmitted real-time image, the image first needs to undergo preprocessing operations identical to those used in the training phase. This includes resizing the image to the standard input size accepted by the model and normalizing it to ensure consistent data distribution. After preprocessing, the image tensor is input into the trained model for a fast forward inference computation. The model outputs a multi-channel probability map with the same size as the input image. Each channel corresponds to a category (background, initial water level, initial dam crest line, structural feature points, etc.), and the value of each pixel in the map represents the probability that the pixel belongs to the corresponding category. By applying a maximum value operation to all channel probability values ​​at each pixel location, the final pixel-level segmentation mask map can be generated. From this mask map, the pixel sets marked as initial water level and initial dam crest lines can be clearly extracted, and their vectorized representations can be obtained through algorithms such as contour detection. Simultaneously, for regions marked as structural feature points, their geometric centers can be calculated, thereby obtaining the precise two-dimensional pixel coordinates of these points in the current image, such as... Figure 4 As shown.

[0047] In one implementation, the real-time pose matrix of the surveillance camera relative to the physical coordinate system is solved by perspective projection geometry based on the three-dimensional coordinates in the feature array database and the corresponding matching two-dimensional pixel coordinates, including the following steps: Select a subset of core feature points, consisting of multiple structural feature points, from all target structural feature points for preliminary pose estimation; The initial real-time pose matrix is ​​obtained by combining the three-dimensional coordinates of the core feature point subset and the corresponding matching two-dimensional pixel coordinates and solving the PnP algorithm. Using the initial real-time pose matrix, the three-dimensional coordinates of all structural feature points in the feature array database that were not selected into the core feature point subset are projected onto the image plane of the real-time monitoring image to generate the predicted two-dimensional pixel coordinates of each of the unselected structural feature points. Search for the actual matching two-dimensional pixel coordinates within the neighborhood of the predicted two-dimensional pixel coordinates to construct a validated target feature point matching set. The real-time pose matrix of the surveillance camera relative to the physical coordinate system is obtained by matching the target feature points and using the PnP algorithm.

[0048] In this implementation, to ensure the robustness of the initial pose estimation, a small subset of core feature points with extremely stable geometric configuration is strategically selected from all known target structural feature points. This selection process aims to maximize the resistance of subsequent solution algorithms to image noise and small matching errors. Specifically, a subset size is first set, for example, selecting 5 feature points. Then, all possible combinations of 5 points formed by all target structural feature points are traversed, forming multiple candidate feature point subsets. For each candidate subset, the three-dimensional coordinates of its contained feature points in the physical coordinate system are extracted, and a geometric configuration matrix is ​​constructed based on these coordinates. The condition number of this matrix is ​​then calculated. This metric quantifies the goodness of distribution of the feature point combination in three-dimensional space. A lower condition number indicates a wide distribution of points, without tending towards collinearity or coplanarity, thus providing strong geometric constraints for pose estimation. By sorting the condition numbers of all candidate subsets, the candidate feature point subset with the lowest condition number is finally selected as the optimal core feature point subset for initial pose estimation. After determining the core feature point subset, initial real-time camera pose estimation can be performed. This step combines the known, precise 3D world coordinates of each feature point in the core subset with the corresponding 2D pixel coordinates identified by the image segmentation model in the current real-time monitoring image, forming a reliable 2D-3D point correspondence. Using this correspondence, the camera pose is calculated by solving the classic Perspective-n-Point (PnP) problem.

[0049] Specifically, an efficient PnP algorithm can be used, which can robustly handle cases with four or more points. Its core is based on a camera perspective projection model. Where s is the scale factor, These are the homogeneous two-dimensional pixel coordinates of the feature points on the image. These are the homogeneous 3D coordinates of the corresponding feature points in the physical world coordinate system, and K is a pre-calibrated camera intrinsic parameter matrix. This is the camera extrinsic parameter matrix to be solved, which includes the rotation matrix R and the translation vector t. The PnP algorithm cleverly represents the 3D control points as a linear combination of four virtual control points, transforming the problem into a linear least squares solution, thus efficiently solving for the camera's initial real-time pose matrix. Although this matrix is ​​preliminary, it can roughly describe the camera's spatial position and orientation relative to the physical coordinate system. After obtaining the initial real-time pose matrix of the camera, this preliminary pose information can be used to verify and expand the matching set of feature points. Specifically, all remaining structural feature points in the feature array database that were not selected into the core feature point subset are projected forward one by one. For each such point, its three-dimensional coordinates in the physical coordinate system are known. Using the complete camera projection formula, the three-dimensional coordinate point is transformed and projected using the initial pose matrix and the camera intrinsic parameter matrix to obtain the position that the point should appear on the image plane of the current real-time monitoring image, i.e., the predicted two-dimensional pixel coordinates. This calculation process A corresponding 2D image location prediction was generated for each unused 3D feature point.

[0050] After generating the predicted 2D pixel coordinates of all non-core structural feature points, the next step is to search for feature points actually detected by the image segmentation model within the local neighborhood of these predicted locations to construct a validated and more comprehensive target feature point matching set. For each predicted coordinate, a fixed-size search window is defined around it, such as a 20x20 pixel square region. Then, the output of the image segmentation model is queried to check if any identified feature points fall within this search window. If one or more detection points exist within the window, the Euclidean distance between each detection point and the predicted coordinate is calculated, and the detection point with the smallest distance is selected as a candidate match. To further ensure the reliability of the matching, a distance threshold is set; only when the minimum distance is less than this threshold is the pair of predicted and detected points considered a valid match. Finally, based on the validated and larger-scale target feature point matching set from the previous step, the PnP algorithm is applied again for a more accurate pose calculation, thus obtaining the final real-time camera pose matrix. Because the input data used for this solution, namely the set of corresponding 2D and 3D points, is much larger, and potential mismatches and undetected points due to occlusion have been eliminated through prediction and search verification processes, its overall data quality is far superior to the initial input consisting only of a core subset. In this step, we can continue to use the efficient PnP algorithm, or, for higher accuracy, employ an iterative nonlinear optimization algorithm, such as the Levenberg-Marquardt algorithm, using the initial pose obtained in the previous step as the starting point for optimization. This optimization process minimizes the reprojection error between the 3D coordinates of all matching points projected onto the image and their actual detected 2D coordinates. After this refined solution, the final output real-time pose matrix more accurately reflects the instantaneous position and orientation of the surveillance camera in the physical world coordinate system.

[0051] In one implementation, selecting a subset of core feature points, consisting of multiple structural feature points, from all target structural feature points for preliminary pose estimation includes the following steps: Obtain the three-dimensional coordinates of all target structural feature points; All target structural feature points are combined into multiple candidate feature point subsets by a preset number; For each subset of candidate feature points, a geometric stability index is calculated based on the geometric distribution of the three-dimensional coordinates of all target structural feature points contained in the subset. The geometric stability index is used to quantify the ability of the subset of candidate feature points to resist noise and collinearity when solving the PnP algorithm. The subset of candidate feature points with the highest geometric stability index is selected as the core feature point subset.

[0052] In this embodiment, the three-dimensional coordinate data of all available target structural feature points are acquired. This data originates from a previously established feature array database. Each feature point possesses a unique, high-precision three-dimensional coordinate system within a predefined, fixed physical world coordinate system. After obtaining the complete set of coordinates of all target structural feature points, a systematic subset of all possible candidate feature points is generated. The purpose of this process is to evaluate various combinations of feature points to ensure that the final selected core subset is globally optimal.

[0053] First, a subset size needs to be preset, i.e., the number of feature points contained in each subset, denoted as k. The choice of this number requires a trade-off between computational efficiency and solution robustness; typically, a value slightly larger than the minimum requirement of the PnP algorithm (3 non-collinear points) is chosen to provide sufficient geometric constraints and redundancy. Based on the total number of feature points N and the preset subset size k, all unique feature point combinations can be generated using the principles of combinatorics. The total number of candidate subsets that can be generated is given by the combinatorics formula. The decision is made by implementing a combinatorial generation algorithm, such as using recursive backtracking, to iterate through every unique combination of k feature points without repetition or omission. The output of this step is a large list where each element is a subset of candidate feature points, containing the 3D coordinates of the k points that make up that subset.

[0054] For each subset of candidate feature points generated in the previous step, an independent geometric stability evaluation is required. The core of this evaluation is calculating an index that quantifies the quality of the point set's 3D spatial distribution—the geometric stability index. An ideal point set should be as dispersed as possible in 3D space, avoiding the degenerate situation where all points are approximately collinear or coplanar, as this configuration makes the PnP algorithm's solution unstable and extremely sensitive to input noise. An effective evaluation method is based on covariance analysis of the point set. For a candidate subset containing k 3D coordinate points, first, the geometric centers of these k points are calculated. Then, a 3x3 covariance matrix is ​​constructed, describing the distribution and correlation of these points relative to the geometric centers in each dimension. Subsequently, the covariance matrix is ​​decomposed into eigenvalues, yielding three eigenvalues. These three eigenvalues ​​geometrically correspond to the distribution variance of the point set along three mutually orthogonal principal directions. The geometric stability index can be defined as the smallest eigenvalue. The larger the value of the geometric stability index, the larger the distribution range of the point set in the weakest direction. The closer the shape of the point cloud is to a sphere rather than a thin sheet or line, the more stable its geometric configuration is.

[0055] After calculating the geometric stability indices for all candidate feature point subsets, the final step is to select the optimal combination. This process first stores all candidate subsets and their corresponding stability indices as an evaluation result list. Then, this list is traversed or sorted to find the entry with the highest geometric stability index value. The candidate feature point subset corresponding to this maximum value is ultimately determined as the core feature point subset for subsequent preliminary pose estimation. The theoretical basis for selecting this subset is that it has the strongest resistance to noise and collinearity among all possible combinations. When the PnP algorithm is executed using the 3D coordinates of this subset and their matching 2D coordinates in the image, the solution process will be the most robust, and the accuracy of the resulting initial pose matrix will be the most guaranteed. This selection process ensures that the entire pose calculation process starts from the most solid geometric foundation, greatly reducing the risk of introducing large initial errors due to poor feature point geometry.

[0056] In one implementation, a three-dimensional point cloud is formed by back-projecting the pixel set on the initial water level line into the physical coordinate system using a virtual measurement grid. The three-dimensional point cloud is then processed by an iterative optimization algorithm based on physical plane constraints to fit the optimal water body plane equation, including the following steps: The pixel set on the initial water level line is back-projected onto the physical coordinate system through a virtual measurement grid to obtain the initial three-dimensional point cloud; The initial 3D point cloud was processed using a random sampling consensus algorithm, and the initial water body plane equation was obtained by fitting and the initial set of interior points was identified. The spatial intersection of the initial water body plane equation and the three-dimensional twin mesh model is calculated, and the spatial intersection is projected onto the image plane according to the real-time pose matrix to form the theoretical water level line; Based on the residuals of each inlier in the initial inlier set on the image plane between the theoretical water level and the inlier in the initial inlier set, all inliers in the initial inlier set are weighted, and the optimal water body plane equation is obtained by refitting the initial inlier set based on the weighted initial inlier set.

[0057] In this embodiment, the virtual measurement grid assigns three-dimensional coordinates in the physical world coordinate system to each pixel in the real-time monitoring image. Therefore, for each identified pixel (u, v) on the initial water level line, its corresponding three-dimensional coordinates (X, Y, Z) can be obtained directly by querying the virtual measurement grid. Summarizing the three-dimensional coordinates of all pixels on the initial water level line constitutes an initial three-dimensional point cloud. This point cloud spatially depicts the boundary between the water body and the dam and its surrounding environment, serving as the direct data source for subsequent water body plane fitting. Since image segmentation results may contain noise, such as water surface ripples, reflections, or minor segmentation errors, this initial three-dimensional point cloud may contain some outliers that deviate from the actual water surface. After obtaining the initial three-dimensional point cloud containing potential noise, a fitting algorithm is needed to estimate the main plane of the water body and simultaneously identify which points conform to the plane and which are outliers. The Random Sample Consensus (RANSAC) algorithm is an ideal choice for this task. The core idea of ​​RANSAC is to perform model assumptions and verifications iteratively.

[0058] In each iteration, the algorithm randomly selects the minimum set of data points required to fit a plane from the initial 3D point cloud (3 non-collinear points are needed for a plane). Based on these 3 points, a candidate plane equation can be uniquely determined. The algorithm then iterates through all remaining points in the initial 3D point cloud, calculating the distance from each point to the candidate plane and setting a distance threshold. Points with a distance less than this threshold are considered interior points supporting the candidate plane. The algorithm records the number of interior points for the current candidate plane. This iterative process is repeated a preset number of times, and the candidate plane with the most interior points is selected as the initial water plane that best represents the main body of the data. Simultaneously, all points supporting this optimal plane are identified to form the initial interior point set. After obtaining the initial water plane equation and the corresponding interior point set using the RANSAC algorithm, the results can be further optimized using the dam's 3D geometric priors. Specifically, the geometric intersection line between the initial water plane and the high-precision 3D twin mesh model of the dam in 3D space is calculated. This calculation involves performing plane-to-plane intersection tests on each triangular facet in the 3D model. Connecting all intersecting line segments forms a continuous 3D spatial intersection line. This intersection line theoretically precisely represents the contour where the water surface meets the dam surface at the current water plane height. Subsequently, using the known real-time camera pose matrix and intrinsic parameter matrix, all points on this three-dimensional spatial intersection line are projected forward onto the image plane of the real-time monitoring image. The resulting two-dimensional curve is the theoretical water level line.

[0059] Finally, the initial inlier set is refined and weighted using the theoretical waterline, followed by final plane fitting. For each 3D point in the initial inlier set, it is first projected back onto the image plane to obtain its corresponding 2D pixel coordinates. Then, the vertical distance from this 2D pixel to the theoretical waterline generated in the previous step is calculated; this distance is defined as the image plane residual for that inlier point. The smaller the residual, the closer the position of the inlier point on the image matches the position predicted by the geometric model, and the higher its reliability. Based on this residual, a weight can be calculated for each inlier point, for example, using a Gaussian function. ,in It is the weight of the i-th interior point. It is its image plane residual. This is a parameter that controls the rate of weight decay. Thus, inliers with smaller residuals receive higher weights, while inliers with larger residuals receive lower weights. After assigning weights to all initial inliers, a weighted least-squares plane fitting is performed. This fitting prioritizes points with higher weights, effectively suppressing the negative impact of points that, although identified as inliers by RANSAC, have poor consistency with the geometric model. The final fit yields a more accurate and robust optimal water plane equation.

[0060] In one implementation, the initial 3D point cloud is processed using a random sampling consensus algorithm to fit the initial water body plane equation and identify the initial set of interior points, including the following steps: While identifying the initial water level line through the image segmentation model, the segmentation confidence score corresponding to each pixel on the initial water level line is obtained; When the set of pixels on the initial water level line is back-projected into the physical coordinate system to form the initial three-dimensional point cloud, the segmentation confidence score corresponding to each pixel is mapped together so that each three-dimensional point in the initial three-dimensional point cloud is accompanied by a segmentation confidence score. When performing the iterative sampling step of the random sampling consensus algorithm, a confidence-weighted non-uniform sampling strategy is adopted, in which the probability of selecting any three-dimensional point from the initial three-dimensional point cloud is proportional to the segmentation confidence score attached to the three-dimensional point. The candidate water body plane is fitted using a 3D point set selected through a confidence-weighted non-uniform sampling strategy; The candidate water plane that receives the most interior point support is selected as the final initial water plane equation.

[0061] In this embodiment, during the forward inference process of the image segmentation model, in addition to generating the final pixel-level segmentation mask, richer probabilistic information can be obtained from the penultimate layer of the network, namely the Softmax activation layer. For each pixel in the image, the Softmax layer outputs a probability distribution vector, where each element represents the probability that the pixel belongs to the corresponding category. Specifically, for the category of waterline, its confidence score... It can be derived from the formula The calculation shows that, among which This represents the raw logistic values ​​output by the network for the waterline categories before the Softmax layer, where M is the total number of categories. When a pixel is determined to belong to the initial waterline by taking the highest probability value, its corresponding probability value is recorded as the pixel's segmentation confidence score. This score ranges from 0 to 1, intuitively quantifying the model's confidence in classifying the pixel. Typically, pixels located in the center of the waterline with clear edges will receive a high score close to 1, while pixels located on blurred boundaries or affected by reflections may receive a lower score.

[0062] In the process of converting a two-dimensional initial water level line into a three-dimensional point cloud, the segmentation confidence score obtained in the previous step needs to be attached as an important attribute to each generated three-dimensional point. Specifically, for any pixel on the initial water level line, its corresponding physical world three-dimensional coordinates are first obtained by querying a virtual measurement grid. At the same time, the segmentation confidence score of that pixel, calculated in the previous step, is extracted. Then, this three-dimensional coordinate and confidence score are combined into a single data unit. This process is repeated for all pixels on the initial waterline, resulting in an initial 3D point cloud that is no longer a simple set of coordinates, but an augmented dataset. Each 3D point carries metadata quantifying the reliability of its source pixel identification. This point cloud, with its accompanying confidence score, provides strong prior information for subsequent robust plane fitting algorithms, enabling the algorithm to distinguish the quality of data points within the point cloud and thus perform more intelligent processing.

[0063] Traditional random sampling consensus algorithms treat all points in the dataset equally during iterative sampling, employing uniform random sampling. To utilize the confidence information obtained in the previous step, a key improvement to the sampling strategy is made here: a confidence-weighted non-uniform sampling strategy is adopted. Specifically, when selecting a point in each iteration of RANSAC, it is no longer randomly selected with equal probability from the initial 3D point cloud. Instead, it first considers the segmentation confidence scores attached to all N points in the point cloud. Construct a discrete probability distribution. The probability that the i-th point is selected is... It is set to be proportional to its confidence score, which can be expressed by the formula. Normalization calculations are performed. Then, random sampling is conducted based on this constructed probability distribution using methods such as roulette wheel selection or inverse transform sampling. In this way, 3D points originating from high-confidence pixels have a higher probability of being selected to form the initial point set for the fitting candidate plane. This strategy greatly increases the probability of selecting three high-quality interior points in early iterations, thereby accelerating the convergence speed of the algorithm and improving the overall quality of the generated candidate planes.

[0064] In each iteration of RANSAC, a confidence-weighted non-uniform sampling strategy is used to select three points from the initial 3D point cloud, denoted as . These three points were used to fit a candidate water body plane. Due to the optimized sampling strategy, these three points are highly likely to be high-quality interior points. The process of fitting the plane follows standard analytical geometry methods. First, two vectors lying in the plane are constructed using these three points, for example... and Next, by calculating the cross product of these two vectors, the normal vector of the candidate plane can be obtained. The three components (A, B, C) of the normal vector constitute the plane equation. The first three coefficients. Finally, by substituting the coordinates of any one of the three sampling points into the plane equation, the fourth coefficient can be obtained. After a predetermined number of iterations, generating a large number of candidate water planes, the optimal one needs to be selected as the final initial water plane equation.

[0065] For a given candidate plane, all points in the initial 3D point cloud are traversed. For each point, its perpendicular distance to the candidate plane is calculated. This distance is then compared to a pre-defined distance threshold (i.e., an inlier threshold). If a point's distance is less than the threshold, it is marked as an inlier of this candidate plane, and the inlier counter is incremented. During the validation phase, the split confidence scores of all points are no longer used; all points are treated equally based on pure geometric distance. This avoids discrimination against points that, while having lower confidence scores, perfectly fit the planar model. After traversing all candidate planes and counting inliers, the candidate water plane that receives the most inlier support (i.e., has the largest number of inliers) is considered the best fit. This plane best represents the mainstream trend in the data and most effectively filters out the influence of noise and outliers, and is therefore selected as the final initial water plane equation.

[0066] In one implementation, the process of weighting all inliers in the initial inlier set based on the residuals between the theoretical water level and each inlier in the initial inlier set on the image plane, and then re-fitting the plane based on the weighted initial inlier set to obtain the optimal water body plane equation includes the following steps: For each inlier in the initial set of inliers, the vertical projection distance of the inlier on the image plane to the theoretical water level is calculated as the pixel residual, and the density of other inliers within a preset neighborhood is calculated as the spatial consistency score. The absolute value of the pixel residual is mapped to the residual confidence score through a preset confidence function; Multiply the residual confidence score by the spatial consistency score to obtain the composite weight of each inlier in the initial inlier set; After normalizing all composite weights, the final weights corresponding to each interior point in the initial interior point set are obtained when performing weighted plane fitting. Based on the weighted initial interior point set, plane fitting is performed again to obtain the optimal water body plane equation.

[0067] In this implementation, for each 3D point in the initial set of inliers, its quality needs to be finely evaluated from two independent dimensions. The first dimension is geometric consistency, quantified by calculating pixel residuals. Specifically, the coordinates of each 3D inlier are reprojected back onto the 2D image plane based on the known real-time camera pose matrix and intrinsic parameter matrix, resulting in 2D pixel coordinates. Then, the Euclidean distance from this pixel coordinate to the previously generated theoretical waterline, i.e., the shortest vertical distance, is calculated. This distance value, in pixels, is defined as the pixel residual of the inlier. A smaller residual means that the image observation position of the inlier closely matches the ideal position calculated based on the 3D geometric model. The second dimension is spatial clustering, measured by calculating a spatial consistency score. For this purpose, a spherical neighborhood of a fixed radius is defined around each inlier in 3D space. Then, the number of other inliers falling within this spherical neighborhood is counted. This number directly reflects the local point cloud density around the inlier and is used as its spatial consistency score. A higher score indicates that the inlier is not an isolated noise point, but belongs to a continuous, dense cluster of points, making it more likely to be a real water surface sampling point.

[0068] After obtaining the pixel residuals for each inlier, they need to be converted from an error metric to a more intuitive confidence score. This conversion is accomplished using a pre-defined, non-linear confidence function. The function is chosen to map smaller pixel residuals to higher confidence scores and larger pixel residuals to lower confidence scores, and this mapping should be smooth. A commonly used function is the Gaussian function. Specifically, the confidence score of the residuals for an inlier... It can be calculated using the following formula: In this formula, The absolute value of the pixel residual calculated in the previous step is used as the input to the function. This represents the standard deviation of the Gaussian function, controlling the rate at which the confidence score decays as the pixel residual increases. A smaller... A high value will cause the score to decay very quickly; only pixels with extremely small pixel residuals can obtain high scores. A larger value will result in a lower score. The value makes the decay process smoother, allowing points with some pixel residuals to obtain a considerable confidence score. Through this function, the original unbounded pixel residuals are elegantly mapped to a standardized residual confidence score between 0 and 1, which facilitates subsequent weight fusion.

[0069] After calculating the residual confidence score and spatial consistency score for each inlier separately, these two indicators, which evaluate point quality from different perspectives, need to be merged into a comprehensive composite weight. The most direct and effective way to achieve this fusion is through multiplication. Specifically, for any inlier in the initial set of inliers, its composite weight... It is determined by its residual confidence score Its spatial consistency score The product obtained by multiplication is The spatial consistency score is the result of normalizing the number of points within the aforementioned neighborhood to ensure that its scale matches the residual confidence score. If an inlier performs poorly in any dimension (e.g., spatially clustered but off-center from the theoretical waterline, or accurately located but surrounded by empty space), its corresponding score will be low, significantly lowering the final composite weight. This mechanism effectively identifies and suppresses the most unreliable inliers.

[0070] The final step is to use the calculated composite weights to perform a weighted plane fitting on the initial set of inliers to obtain the final optimal water body plane equation. Before fitting, the composite weights of all inliers need to be normalized to ensure that the sum of all weights is 1. The final weight of the i-th inlier... Depend on The calculation shows that k is the total number of interior points in the initial interior point set.

[0071] The normalized final weights are used in a weighted least squares fitting problem. The goal of this problem is to find a set of plane parameters (A, B, C, D) that minimizes the sum of squared weighted distances from all inliers to the plane. This process no longer treats all inliers equally, but instead gives greater weight to those inliers deemed more reliable in previous steps (i.e., those with higher final weights). During the fitting process, these high-weighted points exert a greater influence on the position and orientation of the final plane, while the influence of low-weighted points is correspondingly reduced. By solving this weighted optimization problem, the resulting optimal water plane equation can approximate the most reliable data points to the greatest extent possible, thus achieving a superior effect compared to the standard RANSAC algorithm in filtering out various subtle noises and outliers, ensuring the highest accuracy of the water plane model.

[0072] In one embodiment, the method further includes the following steps: Continuously record and store historical time-series data composed of real-time pose matrices; Time series analysis algorithms are applied to identify and quantify the long-term pose drift caused by permanent physical displacement of surveillance cameras from historical time series data; When the long-term pose drift exceeds the preset system stability threshold, a system maintenance alarm for abnormal pose of the surveillance camera is generated, and the long-term pose drift is fed back as a correction parameter to the calculation step of the real-time pose matrix to compensate for the systematic error caused by the permanent physical displacement of the surveillance camera.

[0073] In this implementation, each real-time pose matrix essentially contains information describing the six degrees of freedom in the camera space: a translation vector consisting of three translational components and a rotation matrix consisting of three rotational components. To facilitate time-series analysis, the 3x3 matrix representing rotation is first converted into a more intuitive parametric form, such as Euler angles, which can be decomposed into the camera's yaw, pitch, and roll angles. Therefore, at each timestamp, a complete real-time pose matrix is ​​resolved into a six-dimensional state vector containing three translational coordinate values ​​and three rotational angle values. This state vector and its corresponding timestamp are continuously stored as a data unit in a dedicated time-series database. In this way, over time, six independent historical data streams are formed, precisely recording the minute changes in each degree of freedom of camera motion over time.

[0074] After accumulating a sufficiently long period of historical time-series data, time-series analysis algorithms can be applied to accurately identify and quantify irreversible, permanent physical displacements caused by factors such as camera bracket loosening and foundation settlement. Unlike short-term, high-frequency jitter caused by wind or vibration, which has zero-mean characteristics, long-term pose drift manifests as a continuous, slow, unidirectional change in the overall mean of the data series. To effectively extract this trend, an exponentially weighted moving average filter can be applied to each of the six time-series data points. The core idea of ​​this filter is to assign a weight to each data point in the time series, with more recent data points having higher weights. The calculation formula is as follows: ,in It is the smoothed mean at time point k. These are the original pose parameter observations at that time point. This is a smoothing factor between 0 and 1, controlling the weight distribution between new and old data. Through this filter, a smoothed mean sequence representing the long-term trend can be obtained. Comparing the smoothed mean at the current moment with the baseline mean during the initial system installation and calibration, the difference represents the long-term pose drift in that degree of freedom.

[0075] Once the calculated long-term pose drift is confirmed, a closed-loop alarm and correction mechanism is initiated. First, the long-term pose drifts across the six degrees of freedom are combined into a six-dimensional drift vector. The norm of this vector is calculated and compared to a pre-defined system stability threshold representing the maximum tolerable pose deviation. If the norm of the drift vector exceeds this threshold, the system automatically generates a detailed alarm for abnormal camera pose, notifying management to check the camera's physical condition. Simultaneously, to ensure the accuracy of measurement data before physical maintenance is completed, the system uses this quantized long-term pose drift as a dynamic correction parameter, feeding it back into the real-time pose matrix calculation process. Specifically, the six-dimensional drift vector is converted into a homogeneous transformation matrix with the same structure as the original pose matrix—the correction matrix. In each subsequent pose calculation, the newly calculated real-time pose matrix is ​​multiplied by the inverse of this correction matrix. In this way, known systematic drift errors can be offset from the original measurement results in real time and online, thereby outputting a compensated and more accurate pose matrix, ensuring the long-term reliability of subsequent water level elevation calculations.

[0076] The present invention also discloses a small reservoir water level intelligent recognition system based on general images, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the small reservoir water level intelligent recognition method based on general images as described above.

[0077] The processor can be a central processing unit (CPU). Of course, depending on the actual use, it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc., and this application does not limit it.

[0078] The memory can be an internal storage unit of a computer device, such as a hard disk or RAM, or an external storage device, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD), or flash memory card (FC) provided on the computer device. Furthermore, the memory can be a combination of internal storage units and external storage devices of a computer device. The memory is used to store computer programs and other programs and data required by the computer device. The memory can also be used to temporarily store data that has been output or will be output. This application does not limit this.

[0079] The present invention also discloses a computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to be configured to perform the intelligent water level recognition method for small reservoirs based on general images as described in any of the above embodiments.

[0080] The computer program can be stored in a machine-readable medium. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or certain middleware. The machine-readable medium includes any entity or device capable of carrying computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the machine-readable medium includes, but is not limited to, the above-mentioned components.

[0081] The method for intelligent identification of water level in small reservoirs based on general images described in the above embodiments is stored in the computer-readable storage medium and loaded and executed on the processor to facilitate the storage and application of the above method.

[0082] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of protection of this application is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of one or more embodiments of this application as described above, which are not provided in detail for the sake of brevity.

[0083] One or more embodiments in this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments in this application should be included within the protection scope of this application.

Claims

1. A method for intelligent water level recognition in small reservoirs based on general images, characterized in that, The steps include the following: A three-dimensional model of the target reservoir dam is constructed, and a three-dimensional twin mesh model of the target reservoir dam is built. The three-dimensional coordinates of multiple structural feature points in a preset physical coordinate system are extracted from the three-dimensional twin mesh model to form a feature array database. The system acquires real-time monitoring images from surveillance cameras in the target reservoir dam. The images are then processed using an image segmentation model to preliminarily identify the initial water level line and initial dam crest line in the real-time monitoring images, as well as the two-dimensional pixel coordinates of the target structural feature points that match the structural feature points in the feature array database. Select a subset of core feature points, consisting of multiple structural feature points, from all the structural feature points corresponding to the target structural feature points for preliminary pose estimation; The initial real-time pose matrix is ​​obtained by combining the three-dimensional coordinates of the core feature point subset and the corresponding matching two-dimensional pixel coordinates and solving the PnP algorithm. Using the initial real-time pose matrix, the three-dimensional coordinates of all structural feature points in the feature array database that were not selected into the core feature point subset are projected onto the image plane of the real-time monitoring image to generate the predicted two-dimensional pixel coordinates of each of the unselected structural feature points. Search for the actual matching two-dimensional pixel coordinates within the neighborhood of the predicted two-dimensional pixel coordinates to construct a validated target feature point matching set. The real-time pose matrix of the surveillance camera relative to the physical coordinate system is obtained by matching the target feature points and solving the PnP algorithm. Based on the real-time pose matrix, the three-dimensional twin mesh model is projected onto the real-time monitoring image to generate a virtual measurement mesh that is registered with the image content in the real-time monitoring image and each image pixel of the real-time monitoring image is accompanied by three-dimensional coordinate information. The pixel set on the initial water level line is back-projected into the physical coordinate system using a virtual measurement grid to form a three-dimensional point cloud. The three-dimensional point cloud is then processed by an iterative optimization algorithm based on physical plane constraints to obtain the optimal water body plane equation. Calculate the actual physical height difference between the optimal water body plane and the crest of the target reservoir dam in three-dimensional space; The real-time absolute water level elevation of the reservoir where the target reservoir dam is located is calculated by subtracting the actual physical elevation difference from the preset absolute elevation value of the dam crest.

2. The intelligent water level recognition method for small reservoirs based on general images according to claim 1, characterized in that, The process of creating a 3D twin mesh model of the target reservoir dam, extracting the 3D coordinates of multiple structural feature points in a preset physical coordinate system from the 3D twin mesh model, and forming a feature array database includes the following steps: The target reservoir dam was photographed from multiple angles using drones to obtain multi-view image data of the target reservoir dam; A 3D dense point cloud of the target reservoir dam is generated based on multi-view image data and using a motion reconstruction structure algorithm. Mesh reconstruction and texture mapping are performed on the 3D dense point cloud to generate a 3D twin mesh model of the target reservoir dam; Multiple candidate structural feature points were initially selected on the 3D twin mesh model using a feature detection algorithm; For different combinations of feature points composed of multiple candidate structural feature points, the spatial resolution index of different feature point combinations within the field of view of the monitoring camera in the target reservoir dam is calculated respectively. Feature point combinations with optimal spatial resolution are selected as structural feature points, and the three-dimensional coordinates of the structural feature points in a preset physical coordinate system are stored in a preset database to form a feature array database.

3. The intelligent water level recognition method for small reservoirs based on general images according to claim 1, characterized in that, The process of processing real-time monitoring images using an image segmentation model to initially identify the initial water level line and initial dam crest line in the real-time monitoring images, as well as the two-dimensional pixel coordinates of the target structural feature points in the real-time monitoring images that match the structural feature points in the feature array database, includes the following steps: A training image dataset containing the initial water level line, initial dam crest line, and structural feature points was constructed based on historical monitoring images of the target reservoir dam and manually labeled. Data augmentation processing is performed on the training image dataset, including simulating image transformations of rain, fog, different light intensities, and viewpoint jitter. Construct and configure an image segmentation model, which includes a cyclic feature aggregation module for capturing linear feature context information; The image segmentation model is trained using the enhanced training image dataset until the segmentation loss function of the image segmentation model converges. The real-time monitoring image is input into the trained image segmentation model. The image segmentation model identifies the initial water level line and initial dam crest line in the real-time monitoring image, as well as the two-dimensional pixel coordinates of the target structural feature points that match the structural feature points in the feature array database in the real-time monitoring image.

4. The intelligent water level recognition method for small reservoirs based on general images according to claim 1, characterized in that, The process of back-projecting the pixel set on the initial water level line into the physical coordinate system using a virtual measurement grid to form a three-dimensional point cloud, and then processing the three-dimensional point cloud using an iterative optimization algorithm based on physical plane constraints to fit the optimal water body plane equation includes the following steps: The pixel set on the initial water level line is back-projected onto the physical coordinate system through a virtual measurement grid to obtain the initial three-dimensional point cloud; The initial 3D point cloud was processed using a random sampling consensus algorithm, and the initial water body plane equation was obtained by fitting and the initial set of interior points was identified. The spatial intersection of the initial water body plane equation and the three-dimensional twin mesh model is calculated, and the spatial intersection is projected onto the image plane according to the real-time pose matrix to form the theoretical water level line; Based on the residuals of each inlier in the initial inlier set on the image plane between the theoretical water level and the inlier in the initial inlier set, all inliers in the initial inlier set are weighted, and the optimal water body plane equation is obtained by refitting the initial inlier set based on the weighted initial inlier set.

5. The method for intelligent water level recognition of small reservoirs based on general images according to claim 4, characterized in that, The process of weighting all inliers in the initial inlier set based on the residuals between the theoretical water level and each inlier in the initial inlier set on the image plane, and then re-fitting the plane based on the weighted initial inlier set to obtain the optimal water body plane equation includes the following steps: For each inlier in the initial set of inliers, the vertical projection distance of the inlier on the image plane to the theoretical water level is calculated as the pixel residual, and the density of other inliers within a preset neighborhood is calculated as the spatial consistency score. The absolute value of the pixel residual is mapped to the residual confidence score through a preset confidence function; Multiply the residual confidence score by the spatial consistency score to obtain the composite weight of each inlier in the initial inlier set; After normalizing all composite weights, the final weights corresponding to each interior point in the initial interior point set are obtained when performing weighted plane fitting. Based on the weighted initial interior point set, plane fitting is performed again to obtain the optimal water body plane equation.

6. The intelligent water level recognition method for small reservoirs based on general images according to claim 1, characterized in that, The method further includes the following steps: Continuously record and store historical time-series data composed of real-time pose matrices; Time series analysis algorithms are applied to identify and quantify the long-term pose drift caused by permanent physical displacement of surveillance cameras from historical time series data; When the long-term pose drift exceeds the preset system stability threshold, a system maintenance alarm for abnormal pose of the surveillance camera is generated, and the long-term pose drift is fed back as a correction parameter to the calculation step of the real-time pose matrix to compensate for the systematic error caused by the permanent physical displacement of the surveillance camera.

7. A small-scale reservoir water level intelligent recognition system based on general images, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the intelligent water level recognition method for small reservoirs based on general images as described in any one of claims 1 to 6.

8. A computer-readable storage medium storing instructions thereon, characterized in that, When executed by the processor, the instruction causes the processor to be configured to perform the intelligent water level recognition method for small reservoirs based on general images according to any one of claims 1 to 6.

Images