Water conservancy monitoring intelligent interactive tracking method and system based on visual number fusion

Abstract

The application relates to the technical field of water conservancy monitoring, and discloses a water conservancy monitoring intelligent interactive tracking method and system based on view number fusion, which comprises the following steps: collecting a monitoring video, recording a video time, obtaining a corresponding water level reading, determining a left gate groove fixed transverse position, a right gate groove fixed transverse position, a left waterline height, a right waterline height, a left waterline point and a right waterline point; obtaining a candidate area from a current video frame, establishing a water level line track and configuring a tracking number; determining a left waterline change amount, a right waterline change amount, a water level change amount, a bilateral same direction value, a view number same direction value, a bilateral difference value and a propelling belt available value based on the left waterline height, the right waterline height and the water level reading of adjacent video times; forming a waterline propelling belt in an available state, and restraining candidate area screening, water level line track continuation and manual click position correction; and falling back to regular visual tracking in an unavailable state.

Description

Technical Field

[0001] This invention relates to the field of water conservancy monitoring technology, and more specifically, to a smart interactive tracking method and system for water conservancy monitoring based on visual-data fusion. Background Technology

[0002] At sluice gates and canal monitoring points, fixed cameras are responsible for continuously outputting live footage, water level gauges provide water level data, and edge devices or platforms are responsible for display, alarms, and archiving. During operation, the water surface is affected by sunlight, rainfall, nighttime supplemental lighting, floating objects, foam, and gate projection, so the water level line often appears in the image as locally discontinuous, with abrupt changes in brightness and dark areas, and mixed boundaries.

[0003] Current identification processes mostly rely on single-frame image boundaries, color segmentation, or the continuity of adjacent image positions to determine water levels, primarily using water level gauge data for reading display, alarm judgment, or result verification. Such processes are prone to including non-water level boundaries in the trajectory when there is strong image interference or short-term occlusion, causing trajectory jumps. Even after manual dragging or clicking corrections, subsequent images may still continue updating along the incorrect boundaries. For water conservancy monitoring systems requiring long-term monitoring and remote playback, it is also necessary to maintain a traceable temporal correspondence between video, sensor data, and manual operation, ensuring that automatic tracking results and manual correction records can stably serve monitoring displays, anomaly alerts, and historical queries. Summary of the Invention

[0004] This invention provides a smart interactive tracking method and system for water conservancy monitoring based on visual-data fusion, which solves the technical problems mentioned in the background art.

[0005] To achieve the above-mentioned objectives, the present invention provides the following technical solution.

[0006] A smart interactive tracking method for water conservancy monitoring based on view-to-data fusion is used for tracking water level lines in water conservancy monitoring images, including: Collect monitoring video and record the video time, and obtain the water level reading corresponding to the video time; Determine the fixed horizontal position of the left gate slot and the fixed horizontal position of the right gate slot, read the height of the left waterline and the height of the right waterline, and determine the waterline point on the left and the waterline point on the right. Obtain candidate regions from the current video frame, establish water level trajectory, and configure tracking numbers for the water level trajectory; Based on the left and right waterline heights and water level readings of adjacent video times, the changes in the left and right waterline, the water level, the bilateral same-direction values, the same-direction values ​​of the number of views, and the bilateral difference are determined. The consistency of the movement direction of the changes in the left and right waterlines is verified based on the bilateral same-direction values. The imaging relationship between the direction of waterline movement and the direction of water level reading change is verified based on the same-direction values ​​of the number of views. The difference between the changes in the left and right waterlines is limited based on the bilateral difference and the upper limit of the bilateral difference, so as to determine the available value of the propulsion zone. When the available value of the push zone is available, a waterline push zone is formed based on the left and right waterline points of adjacent video times, and the waterline push zone is used to constrain the selection of candidate areas, the continuation of waterline trajectory, and the correction of manual click positions. When the available values ​​for the propulsion zone are unavailable, the corresponding left and right waterline heights are not used to update the waterline trajectory, and the system reverts to regular visual tracking.

[0007] Compared with the prior art, the present invention has the following substantial features and significant progress: This invention integrates camera footage, water level gauge readings, left and right gate boundaries, and interactive terminal operations into a single processing flow. By using a waterline propulsion belt to link water level changes, bilateral waterline movement, and candidate areas, it can reduce water level trajectory jumps under interference from reflections, ripples, occlusions, and shadows. When data is abnormal, it reverts to conventional visual tracking, reducing the impact of abnormal intersections on subsequent images. After manual clicks are constrained by the waterline propulsion belt, the water level trajectory is updated and the tracking number is retained, which helps improve the continuity of water level recognition, trajectory continuation, and manual correction in water conservancy monitoring images. Attached Figure Description

[0008] Figure 1 This is a schematic diagram of the modules and data flow of the intelligent interactive tracking system for water conservancy monitoring based on visual-data fusion according to the present invention. Detailed Implementation

[0009] The following description is provided in conjunction with the accompanying drawings. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the invention; those skilled in the art can make equivalent substitutions or combinations for the specific implementations.

[0010] Example 1: This embodiment is used for tracking water level lines in a water conservancy monitoring screen. The monitoring screen is continuously captured by cameras 100 installed at monitoring points of the water conservancy facility, and water level readings are output by water level gauges 110 that have a fixed correspondence with each camera 100. This fixed correspondence ensures a stable correspondence between the monitoring video, water level readings, video time, and spatial range of the water conservancy facility at the same monitoring point, preventing the incorrect use of water level readings from other monitoring points in the current video frame during subsequent calculations.

[0011] In monitoring footage of fixed sluice gates, channels, or river sections, water levels are typically affected by reflections, ripples, raindrops, obstructions from floating objects, and gate leaf shadows. If relying solely on water surface color, edge intensity, or positional continuity between adjacent video frames, the water level trajectory may jump to foam edges, gate leaf shadows, or bank textures. This implementation utilizes the correspondence between the fixed lateral positions of the left and right gate slots, the left and right waterline heights, and the water level readings to transform changes in water level readings into waterline advancement zones that constrain candidate areas. This ensures that candidate area selection, water level trajectory continuity, and manual click position correction are all constrained by the same view-to-view fusion result.

[0012] like Figure 1 As shown, camera 100 outputs monitoring video to video acquisition unit 120, and water level gauge 110 outputs water level readings to numerical acquisition unit 130. Video acquisition unit 120 outputs the current video frame and video time to waterline end position extraction unit 140, and outputs the current video frame to candidate region acquisition unit 150. Numerical acquisition unit 130 outputs valid water level readings to waterline advancement zone generation unit 160. Waterline end position extraction unit 140 determines the fixed lateral position of the left gate slot, the fixed lateral position of the right gate slot, the left waterline height, the right waterline height, the left waterline point, and the right waterline point. Candidate region acquisition unit 150 obtains candidate regions. Waterline advancement zone generation unit 160 determines the available values ​​of the advancement zone and forms the waterline advancement zone. Candidate matching and tracking unit 170 uses the waterline advancement zone to constrain the selection of candidate regions and the continuation of the waterline trajectory. Interactive correction unit 180 uses the waterline advancement zone to constrain the manual click position correction. Interactive display terminal 190 displays the waterline trajectory, tracking number, waterline advancement zone range prompts, and correction results.

[0013] In this embodiment, the fixed edges of the left and right gate slots serve as existing fixed space references on-site for the hydraulic facilities. The intersection relationship between the water surface and the fixed edges of the left and right gate slots serves as the basis for reading the water level height on the left and right sides. The water level reading output by the water level gauge 110 serves as the basis for numerical constraints.

[0014] Example 2: When the monitoring point of the water conservancy facility is activated, the video acquisition unit 120 receives the monitoring video output by the camera 100. The camera 100 can be a fixedly installed water conservancy monitoring camera, whose shooting range covers the gate opening, water surface, fixed edge of the left gate slot, fixed edge of the right gate slot, and gate leaf. The video acquisition unit 120 records the video time for each current video frame. The video time can be generated by a unified clock provided by the camera 100, edge computing device, or monitoring server. The purpose of the video time is to establish a traceable time correspondence between the current video frame and the water level reading.

[0015] The numerical acquisition unit 130 acquires the water level reading corresponding to the video time. The water level reading is output by the water level gauge 110 at the same water conservancy facility monitoring point, representing the water surface height measured by the water level gauge 110 at the corresponding moment in the video. To ensure that the water level reading corresponds to the video time based on the acquisition time, the system establishes a fixed correspondence between the camera 100 and the water level gauge 110 during the initialization phase, and records the video time and the acquisition time of the water level reading using a unified clock. For a monitoring point, the device number of the camera 100, the device number of the water level gauge 110, the communication address, the sampling period, and the operating status are written into the scene registration record.

[0016] When recording video time and water level reading acquisition time using a unified clock, both video acquisition unit 120 and numerical acquisition unit 130 write data to the time buffer queue. The time buffer queue is sorted according to the reception time or the unified clock time, so that the current video frame, video time, water level reading, and water level reading acquisition time have a unified search basis. When the sampling period of water level gauge 110 is greater than the video frame period of camera 100, the most recently marked valid water level reading can be used between two adjacent water level readings; when the water level reading exceeds the device range, communication is interrupted, or the allowable delay is exceeded, numerical acquisition unit 130 marks the water level reading as temporarily unavailable.

[0017] When acquiring water level readings corresponding to video timestamps, the system uses the video time as a reference and searches for valid water level readings in the time buffer queue. If the time difference between the acquisition time of the water level reading and the video time does not exceed a preset allowable value, the water level reading is used as the valid water level reading for the current video frame to determine the usable value of the propulsion zone. If the time difference exceeds the preset allowable value, the water level reading is marked as temporarily unavailable. The preset allowable value can be pre-configured according to the video frame period of camera 100, the sampling period of water level gauge 110, and the normal communication delay of the monitoring platform. Its purpose is only to ensure that the acquisition time of the video time and the water level reading has an engineering-acceptable correspondence.

[0018] During the numerical validity check, the system performs range checks, abrupt change checks, and communication status checks on the water level readings. The range check confirms that the water level reading does not exceed the equipment range of the water level gauge 110; the abrupt change check identifies readings that clearly do not conform to the characteristics of continuous water level changes; and the communication status check confirms that the data link between the water level gauge 110 and the numerical acquisition unit 130 is not interrupted. If a water level reading is marked as temporarily unavailable, the waterline propulsion zone generation unit 160 does not use the water level reading to determine the available propulsion zone value and reverts subsequent processing to conventional visual tracking.

[0019] The preset tolerance value is determined based on the video frame period of camera 100 and the sampling period of water level gauge 110. During implementation, the video frame period of camera 100 and the sampling period of water level gauge 110 are first read. When the sampling period of water level gauge 110 is greater than the video frame period of camera 100, the preset tolerance value is a time not less than the sampling period of water level gauge 110 and not greater than two sampling periods of water level gauge 110. When the sampling period of water level gauge 110 is not greater than the video frame period of camera 100, the preset tolerance value is a time not less than the video frame period of camera 100 and not greater than two video frame periods of camera 100. If there is a fixed transmission delay in the on-site communication link, this fixed transmission delay is added to the above time range.

[0020] The mutation check is performed based on the changes in adjacent water level readings and the continuous sampling time. During implementation, the numerical acquisition unit 130 calculates the change in water level reading between adjacent sampling times and divides this change by the difference between adjacent sampling times to obtain the rate of change. If the rate of change exceeds the maximum allowable rate of change specified in the equipment manual for the water level gauge 110, or if the direction of water level reading change significantly conflicts with the current operating status of the hydraulic facility during two consecutive sampling times, the water level reading is marked as temporarily unavailable. If the water level gauge 110 does not provide a maximum rate of change, three times the rate of change of water level readings continuously and stably collected during the initialization phase is used as the mutation check threshold.

[0021] The range check uses the equipment range of water level gauge 110 as the direct basis. When the water level reading is less than the lower limit of the range of water level gauge 110 or greater than the upper limit of the range of water level gauge 110, the data acquisition unit 130 marks the water level reading as temporarily unavailable; when the water level reading is within the range but the communication status is abnormal, the water level reading is still marked as temporarily unavailable. Communication interruption is determined by the time during which no data is continuously received from water level gauge 110. When this time reaches two sampling cycles of water level gauge 110, communication interruption is considered. After communication is restored, the data acquisition unit 130 must receive at least two consecutive valid water level readings before remarking the water level reading as valid.

[0022] Example 3: The waterline dual-end position extraction unit 140 receives the current video frame and video time output by the video acquisition unit 120, and determines the fixed lateral positions of the left and right gate slots in the initialization frame. The fixed lateral position of the left gate slot is the stable frame column position of the fixed edge of the left gate slot in the monitoring screen, and the fixed lateral position of the right gate slot is the stable frame column position of the fixed edge of the right gate slot in the monitoring screen. Both remain unchanged as image calibration constants after initialization and are only updated during manual recalibration or when waiting for unobstructed frames to be re-registered.

[0023] In each current video frame, the waterline end position extraction unit 140 extracts the waterline contour from the video frame. The waterline contour can be obtained by conventional visual recognition models, conventional edge detection, conventional water surface region segmentation, or a combination of the above conventional methods. This processing is only used to obtain the waterline contour in the current video frame and is not limited to specific model structures, training data, or algorithm frameworks. After obtaining the waterline contour, the waterline end position extraction unit 140 reads the left waterline height corresponding to the intersection position of the waterline contour with the left side of the fixed edge of the left gate slot, and reads the right waterline height corresponding to the intersection position of the waterline contour with the right side of the fixed edge of the right gate slot.

[0024] Water level contour extraction can be achieved using conventional image processing procedures. The video acquisition unit 120 converts the current video frame into a grayscale image and a brightness-equalized grayscale image. The water level double-end position extraction unit 140 performs edge extraction within the gate opening area, retaining edge segments whose length is greater than one-third of the gate opening width, whose absolute value of the edge segment's direction angle is not greater than the upper limit of the horizontal angle, and whose vertical distance between the edge segment and the upper boundary of the water surface area is not greater than the height of the upper boundary's neighborhood. For the retained edge segments, the water level double-end position extraction unit 140 merges connected components according to the vertical position continuity of the edge segment pixels to obtain the water level contour. If a pre-trained conventional visual recognition model is used, the model's output must be converted into a water level contour pixel set of the same form. Subsequent intersection calculations, candidate region acquisition, and candidate overlap ratio calculations are all performed based on this pixel set.

[0025] The upper limit of the horizontal angle and the height of the upper boundary neighborhood are determined during the initialization phase. The upper limit of the horizontal angle is based on the horizontal rightward direction, taking an angle of not less than eight degrees and not more than fifteen degrees; if no on-site adjustment is performed, the upper limit of the horizontal angle is taken as ten degrees. The dimension of the height of the upper boundary neighborhood is image pixels, taken as one-twentieth of the gate opening height; if the result is less than six image pixels, the upper boundary neighborhood height is taken as six image pixels; if the result is greater than thirty image pixels, the upper boundary neighborhood height is taken as thirty image pixels. When camera 100 is reinstalled, the video resolution is changed, or the gate opening range is re-registered, the upper limit of the horizontal angle and the height of the upper boundary neighborhood are re-determined.

[0026] The readings of the left and right intersection positions are performed according to the fixed sampling line intersection method. The image column corresponding to the fixed horizontal position of the left gate slot is used as the left fixed sampling line, and the image column corresponding to the fixed horizontal position of the right gate slot is used as the right fixed sampling line. If the water level line outline has multiple intersection points with the left fixed sampling line, the intersection point with the smallest longitudinal distance to the left water level height of the previous video time is selected as the left intersection position; if the left water level height of the previous video time is not available, the intersection point located within the gate opening range and conforming to the fixed camera imaging relationship with the direction of water level reading change is selected. The right intersection position is determined using the same rule. This rule ensures that the left and right water level heights have definite outputs.

[0027] Occlusion determination is based on contour continuity, local brightness abrupt changes, and consistency of bilateral changes. If the waterline contour within the break determination neighborhood width on both sides of the fixed sampling line breaks, or the brightness gradient of the candidate edge segment exceeds three times the average brightness gradient of the normal waterline contour during the initialization phase, or the waterline change on one side is opposite in direction to the waterline change on the other side and lasts for at least two video frames, then the intersection position on that side is determined to be occluded or affected by false edge interference. The break determination neighborhood width is one percent of the current video frame width; if the result is less than five frame pixels, the break determination neighborhood width is five frame pixels; if the result is greater than twenty frame pixels, the break determination neighborhood width is twenty frame pixels. The occluded side corresponding to the left or right waterline height is marked as temporarily unavailable.

[0028] When the intersection point on the left or right is obscured by floating objects, foam, raindrops, reflections, or the shadow of a door leaf, the waterline end position extraction unit 140 marks the corresponding left or right waterline height as temporarily unavailable, and does not use the reflective edge or door leaf shadow as a substitute, thereby avoiding forcibly using the false edge as the waterline contour in subsequent calculations. If only one side is obscured, the corresponding left or right waterline height on that side is temporarily unavailable; if both sides are obscured, both the left and right waterline heights are temporarily unavailable.

[0029] During the video Below, the fixed horizontal position of the left door slot is denoted as The fixed horizontal position of the right door slot is denoted as The height of the waterline on the left is recorded as The height of the waterline on the right is recorded as The left waterline point is determined by the fixed lateral position of the left gate slot and the height of the left waterline; the right waterline point is determined by the fixed lateral position of the right gate slot and the height of the right waterline. The methods for determining the left and right waterline points are as follows: ; in, Indicates video duration The lower left waterline point, This indicates that the left door slot is fixed in a horizontal position. Indicates video duration The height of the left waterline. The fixed lateral position of the left gate slot is determined by the registration of the fixed edge of the left gate slot during the initialization frame or manual recalibration. The height of the left waterline is determined by the intersection of the waterline outline and the left side of the fixed edge of the left gate slot in the current video frame.

[0030] ; in, Indicates video duration The right side waterline point below, This indicates that the right door slot is fixed in a horizontal position. Indicates video duration The height of the right waterline is determined by the fixed horizontal position of the right gate slot, which is determined during the initialization frame or manual recalibration. The height of the right waterline is determined by the intersection of the waterline outline and the right side of the fixed edge of the right gate slot in the current video frame.

[0031] The left and right waterline points are written into the current frame's visual result and, along with the current video frame, video time, candidate region, and valid water level readings, are processed further. Since the fixed lateral positions of the left and right gate slots do not change with short-term water level fluctuations, the variations in the left and right waterline points between adjacent video times are primarily reflected by the left and right waterline heights. This design allows the waterline propulsion zone to adhere to the fixed boundary of the gate opening, forming a stable spatial range.

[0032] Example 4: The candidate region acquisition unit 150 receives the current video frame output by the video acquisition unit 120 and obtains candidate regions from the current video frame. The candidate region is the image area in the current video frame that may correspond to the water level contour, and can be obtained by conventional water surface region segmentation, conventional edge extraction, conventional object detection, or conventional image candidate generation methods.

[0033] The candidate region acquisition unit 150 generates candidate regions based on the waterline contour pixel set. In practice, the waterline contour pixel set is first divided into connected components; then, each connected component is expanded upwards and downwards by half the base expansion width according to its minimum bounding rectangle, while restricting the expansion result to not exceed the current video frame boundary; subsequently, two candidate regions whose overlapping area is more than half the area of ​​the smaller candidate region are merged. Each merged image region is treated as a candidate region, and candidate numbers are assigned in a top-to-bottom and left-to-right order.

[0034] The candidate region output format includes a candidate number, the pixel set of the candidate region, the bounding rectangle of the candidate region, the pixel set of the corresponding waterline contour, and the recognition confidence score. The recognition confidence score is obtained by normalizing the continuous edge length of the waterline contour, its deviation from the horizontal direction, and the confidence score output by a conventional visual recognition model. When a conventional visual recognition model is not used, the recognition confidence score is obtained by normalizing the continuous edge length and its deviation from the horizontal direction. The normalized recognition confidence score value is located at... to between.

[0035] For duplicate candidate regions, the candidate region acquisition unit 150 retains the candidate region with higher recognition confidence; when the recognition confidence is the same, the candidate region with a smaller longitudinal distance from the water level trajectory of the previous video time is retained. If no candidate region is formed in the current video frame, the candidate matching and tracking unit 170 does not calculate the candidate overlap ratio and sets the water level trajectory to a lost waiting state or reverts to normal visual tracking.

[0036] When establishing a water level trajectory, the candidate matching and tracking unit 170 associates candidate regions in consecutive video frames that meet the constraints of spatial location, recognition confidence, and waterline advancement zone into the same water level trajectory, and assigns a tracking number to the water level trajectory. The tracking number is used to distinguish different trajectory states, ensuring that the water level trajectory maintains the same identity across automatic tracking, loss waiting, re-matching, and manual click position correction. After manual click position correction occurs, if the corresponding water level trajectory already has a tracking number, that tracking number is retained, and no duplicate trajectory is created.

[0037] For ordinary targets, the candidate matching and tracking unit 170 can continue to perform routine position distance comparison, bounding box overlap comparison, and color or texture similarity comparison, and combine the target's motion direction from the previous frame to predict its position in the current video frame. If an ordinary target meets the matching conditions, it retains its original tracking number; if it does not meet the matching conditions, a new number is created or it enters a lost-wait state. For water levels, the candidate matching and tracking unit 170 prioritizes using the waterline advancement zone to constrain the selection of candidate regions and the continuation of the water level trajectory, making the water level trajectory distinct from the conventional continuation method of ordinary target trajectories.

[0038] Candidate region acquisition unit 150 can assign a candidate number to each candidate region. and output video duration. Next Candidate regions Reliability of identification The confidence score can be derived from the detection confidence, segmentation confidence, or edge confidence of the candidate region by a conventional visual recognition model. Recognition confidence only indicates the degree of confidence the visual side has in the candidate region and cannot solely determine whether the waterline trajectory continues. In waterline tracking, recognition confidence needs to be combined with the candidate overlap ratio and the available values ​​of the propulsion zone to form the candidate matching score.

[0039] Example 5: The waterline propulsion zone generation unit 160 receives the left and right waterline points output by the waterline end position extraction unit 140, and receives valid water level readings output by the numerical acquisition unit 130. Only when the left waterline height, right waterline height, and water level readings are all valid, the waterline propulsion zone generation unit 160 determines the left waterline change, right waterline change, water level change, bilateral same-direction value, visual same-direction value, and bilateral difference value based on the left waterline height, right waterline height, and water level readings of adjacent video times. If any data is marked as temporarily unavailable, the available propulsion zone values ​​are in an unavailable state.

[0040] To describe the overall waterline position in the current video frame, the waterline advance zone generation unit 160 can calculate the average waterline height. The average waterline height is used as an auxiliary calculation to determine the overall positional change of the waterline. The average waterline height is calculated as follows: ; in, Indicates video duration Average height of the waterline below Indicates video duration The height of the waterline on the lower left side, Indicates video duration The height of the waterline on the right side below. The units of the average waterline height are the same as those of the left and right waterline heights, all of which are vertical positions in the monitoring screen.

[0041] Between adjacent video frames, the changes in the left and right water lines, and the water level change, respectively reflect the movement of the water level lines on both sides of the frame and the actual reading changes output by the water level gauge 110. The calculation method is as follows: ; in, Indicates video duration Relative to video time The change in the waterline on the left side, Indicates video duration The height of the waterline on the lower left side, Indicates video duration The height of the waterline on the left below.

[0042] ; in, Indicates video duration Relative to video time The change in the waterline on the right side, Indicates video duration The height of the waterline on the right side below, Indicates video duration The height of the waterline on the right side below.

[0043] ; in, Indicates video duration Relative to video time The change in water level, Indicates video duration The corresponding water level reading, Indicates video duration The corresponding water level reading. The dimensions of the water level change are consistent with the water level reading output by water level gauge 110.

[0044] The fixed edges of the left and right gate slots are located on opposite sides of the same real water surface. When the water level rises or falls, the heights of the left and right water lines should show the same trend of change. Based on this physical relationship, the waterline propulsion zone generation unit 160 determines the bilateral unidirectional value to verify the consistency of the movement direction of the changes in the left and right water lines. The calculation method for the bilateral unidirectional value is as follows: ; in, Indicates video duration The bilateral same-direction values ​​below This indicates the change in the waterline on the left. This indicates the change in the waterline on the right. This represents a sign function. A sign function outputs when its input is greater than zero. Output when input equals zero Output when input is less than zero When the values ​​on both sides, representing the changes in the waterline on the left and right sides, do not show any movement in opposite directions, the requirement for consistency in the direction of movement is met.

[0045] The fixed camera imaging relationship is used to describe the correspondence between the direction of waterline movement in the image and the direction of water level reading change. Typically, the vertical coordinate of the monitoring image increases downwards. When the actual water level rises, the water level line in the image moves upwards, and the height of the waterline on the left usually decreases; conversely, when the actual water level falls, the water level line in the image moves downwards, and the height of the waterline on the left usually increases. The waterline propulsion zone generation unit 160 determines the same-direction value of the number of views based on this, which is used to verify the imaging relationship between the direction of waterline movement and the direction of water level reading change. The calculation method for the same-direction value of the number of views is as follows: ; in, Indicates video duration The number of views in the same direction is below. This indicates the change in the waterline on the left. This indicates the change in water level. The negative sign is used to express the opposite direction in the imaging relationship of a fixed camera. When the same value in the same direction of the viewpoint indicates that the direction of water line movement in the image matches the direction of water level reading change in the imaging relationship of a fixed camera, the water level reading and the image change in the current video frame mutually support each other.

[0046] To prevent abnormal jumps caused by one side being obstructed by foam, reflections, floating debris, or the shadow of the door leaf, the waterline propulsion zone generation unit 160 also determines a bilateral difference. The bilateral difference is used to limit the difference between the changes in the waterline on the left and right sides. The calculation method for the bilateral difference is as follows: ; in, Indicates video duration The two-sided difference under the following conditions This indicates the change in the waterline on the left. This indicates the change in the waterline on the right side. Upper limit of the difference between the two sides. It can be pre-configured during initialization based on video resolution, the distance between the fixed edges of the left and right gate slots in the image, and the normal image error under normal water level fluctuation conditions.

[0047] The calibration of the upper limit of the bilateral difference is completed during the initialization phase. The waterline end position extraction unit 140 selects no less than thirty consecutive video frames as calibration frames for normal steady-state operation, and the left and right intersection positions within the calibration frames are not obstructed, and the water level readings are marked as valid. The normal steady-state calibration frames are confirmed by the interactive display terminal 190 after displaying the current video frame, waterline contour, left intersection position, right intersection position, and valid water level reading status; during confirmation, the left and right intersection positions should be located on the same continuous waterline contour, the water level readings should be valid within the corresponding acquisition time, and the candidate matching and tracking unit 170 should not output a waterline trajectory loss waiting status. For each calibration frame, the left waterline change, the right waterline change, and the bilateral difference are calculated, and the sum of the average of the bilateral differences in the calibration frame and three times the standard deviation is taken as the upper limit of the bilateral difference; if the result is less than two screen pixels, the upper limit of the bilateral difference is taken as two screen pixels. After the camera 100 is reinstalled and the left or right door slot is re-registered for horizontal position, the calibration is performed again.

[0048] When the bilateral same-direction values, representing the changes in the waterline on the left and right sides, do not show opposite directions of movement, and the same-direction value, representing the direction of waterline movement in the image, matches the direction of water level reading change with the fixed camera imaging relationship, and the bilateral difference does not exceed the upper limit of the bilateral difference, the waterline propulsion zone generation unit 160 sets the available propulsion zone value to an available state; otherwise, it sets the available propulsion zone value to an unavailable state. The binary expression of the available propulsion zone value is as follows: ; in, Indicates video duration The available value of the propulsion belt. This indicates that the propulsion belt is in an available state. This indicates that the available values ​​for the propulsion zone are unavailable. When the available values ​​for the propulsion zone are unavailable, the corresponding left and right waterline heights are not used to update the waterline trajectory.

[0049] Example 6: When the available value of the propulsion band is in an available state, the waterline propulsion band generation unit 160 forms a waterline propulsion band based on the left and right waterline points of adjacent video times. In order to enable the waterline propulsion band to cover slight fluctuations and adapt to situations with rapid water level changes, the width of the propulsion band is first determined based on the influence of the base extension width, the water level extension coefficient, and the amount of water level change on the range of waterline movement in the picture.

[0050] The on-screen water level conversion ratio is used to express the real-time conversion relationship between changes in water level readings and the movement of the water line on the screen. The calculation method for the on-screen water level conversion ratio is as follows: ; in, Indicates video duration The water level conversion ratio in the image below. This indicates the change in the waterline on the left. This indicates the change in the waterline on the right. Indicates the change in water level. This indicates the zero-prevention constant. The zero-prevention constant is a fixed small quantity used to prevent division by zero when the water level change is zero or close to zero. The zero-prevention constant can be pre-configured according to the resolution of the water level gauge 110 and the minimum effective change in water level reading; its function is only to stabilize the normal value.

[0051] The dimension of the zero-prevention constant is the same as that of the water level change. During implementation, the zero-prevention constant is taken from the resolution of the water level gauge 110; if the water level gauge 110 does not output a resolution, it is taken as half of the minimum non-zero change in the water level reading during the initialization phase. The zero-prevention constant is only used for numerical stability and does not participate in the validity judgment of the water level reading. When the water level reading unit changes, the zero-prevention constant is synchronously converted to the same unit to ensure that the order of magnitude of the water level conversion ratio on the screen does not change due to the unit switch.

[0052] The propulsion zone width is used to determine the width of the image as the waterline propulsion zone extends outwards. The propulsion zone width is calculated as follows: ; in, Indicates video duration The width of the propulsion belt below, Indicates the base extension width. Indicates the water level expansion coefficient. This indicates the water level conversion ratio on the screen. This indicates the amount of water level change. The base expansion width is used to cover minor fluctuations, while the water level expansion coefficient is used to widen the candidate area when the water level changes rapidly. The base expansion width and water level expansion coefficient can be pre-configured based on video resolution, the range of image jitter of the water level contour under normal steady-state conditions, and on-site management requirements.

[0053] The basic expansion width is measured in pixels. During implementation, the maximum longitudinal deviation of the waterline profile relative to the average waterline height is calculated in the calibration frame under normal steady-state conditions. The basic expansion width is then twice this maximum longitudinal deviation; if this result is less than four pixels, the basic expansion width is set to four pixels. The waterline expansion coefficient is a dimensionless coefficient, initially set to a value of... If the actual water level profile falls outside the waterline advance zone when the water level changes rapidly in the calibration frame, then... The water level spread factor is increased by adjusting the step size until the true water level contours in at least ten consecutive valid video frames fall within the waterline advance zone. The true water level contours are determined by the same continuous water level contour in the normal steady-state calibration frames, and this same continuous water level contour simultaneously passes through both the left and right intersection points. The water level spread factor does not exceed [a certain value]. This is to avoid the waterline advance zone being too wide, which could cause false edges to enter the candidate region for screening.

[0054] The upper and lower limits of the propulsion zone width are defined according to the current video frame size. When the propulsion zone width is less than the basic extension width, the basic extension width is used; when the propulsion zone width is greater than one-tenth of the current video frame height, one-tenth of the current video frame height is used. By using these upper and lower limits, the waterline propulsion zone can cover situations with slight fluctuations and rapid changes in water level, without expanding to cover the entire water surface.

[0055] When forming the waterline propulsion zone, based on video time The left and right waterline points and video time are shown below. Using the left and right waterline points as four vertices, a frame region is first formed by the left and right waterline points of adjacent video times. Then, this frame region is expanded according to the width of the advance band to obtain the waterline advance band. Its expression is as follows: ; in, Indicates video duration The waterline propulsion belt, Indicates video duration The lower left waterline point, Indicates video duration The right side waterline point below, Indicates video duration The right side waterline point below, Indicates video duration The lower left waterline point, Indicates the width of the propulsion belt. This indicates that a closed quadrilateral region is formed based on the input points. This indicates that the closed quadrilateral region is expanded outward in a conventional manner according to the width of the propulsion zone.

[0056] When forming a closed quadrilateral region, the four vertices are arranged according to... , , , The sequence of connections is as follows. If the left and right waterline points of adjacent video times cause the closed quadrilateral region to intersect, the waterline advancement band generation unit 160 does not use the closed quadrilateral region to update the waterline advancement band and sets the available value of the advancement band to an unavailable state. If the advancement band width is zero, it is expanded according to the basic expansion width. If the expanded waterline advancement band exceeds the boundary of the current video frame, the waterline advancement band is trimmed to within the boundary of the current video frame.

[0057] The pixel set of the waterline propulsion zone can be obtained through conventional polygon filling. In practice, a closed quadrilateral region is first filled, then expanded outwards along the boundary of this closed quadrilateral region with the width of the propulsion zone as the radius, and finally cropped to the boundary of the current video frame. When the expansion result is empty, the candidate matching and tracking unit 170 does not calculate the candidate overlap ratio and sets the waterline trajectory to a lost waiting state. This boundary rule ensures that the waterline propulsion zone still has a definite output even under extreme input conditions.

[0058] The front and rear boundaries of the waterline advance zone are derived from the actual water level positions of adjacent video times. The two side boundaries of the waterline advance zone are attached to the fixed edges of the left and right gate slots. Therefore, the waterline advance zone simultaneously includes the direction of waterline movement, the range of image movement, and the spatial boundaries of the gate opening. The waterline advance zone does not simply perform general color segmentation of the entire water surface, nor does it rely solely on water level readings for alarm judgment. Instead, it converts the left and right waterline points, along with the effective water level readings, into image areas that can participate in the candidate area selection.

[0059] When the available values ​​for the waterline advance zone are unavailable, the candidate matching and tracking unit 170 does not update the waterline trajectory using the corresponding left and right waterline heights, and reverts to regular visual tracking. In this case, the previous waterline advance zone can be retained for reference during subsequent rematching, but the current abnormal intersection point is not used as the new waterline advance zone boundary. This reverting process avoids skewing the entire waterline trajectory when a short-term anomaly occurs at a single-sided intersection position.

[0060] Example 7: The candidate matching and tracking unit 170 receives the candidate region output by the candidate region acquisition unit 150, and simultaneously receives the waterline propulsion zone and its available value output by the waterline propulsion zone generation unit 160. When the available value of the propulsion zone is available, the candidate matching and tracking unit 170 prioritizes determining the spatial overlap between the candidate region and the waterline propulsion zone. The higher the degree of overlap between the candidate region and the waterline propulsion zone, the more the candidate region conforms to the spatial constraints of waterline movement in adjacent video times.

[0061] For video time Next For each candidate region, the candidate matching and tracking unit 170 calculates the candidate overlap ratio between the candidate region and the waterline propulsion zone. The calculation method for the candidate overlap ratio is as follows: ; in, Indicates video duration Next The overlap ratio between candidate regions and candidate waterline advance zones, Indicates video duration Next Candidate regions, Indicates video duration The waterline propulsion belt, This represents the area of ​​intersection between the candidate region and the waterline advance zone. This represents the union area of ​​the candidate region and the waterline advance zone. If the union area is zero, the candidate region is not considered a valid candidate for continuing the waterline trajectory.

[0062] The candidate matching and tracking unit 170 combines the recognition confidence level, candidate overlap ratio, and available propulsion zone value into a candidate matching score. The candidate matching score is calculated as follows: ; in, Indicates video duration Next Candidate matching scores for each candidate region Indicates the credibility weight. Indicates the credibility of the identification. Indicates the overlap ratio weight. Indicates the percentage of overlapping candidates. Indicates the available value weight. This indicates the available value of the propulsion zone. The confidence weight, overlap ratio weight, and available value weight can be pre-configured based on the stability of the monitoring screen, the output range of the conventional visual recognition model, and the on-site tracking requirements, and can be normalized to fixed weights.

[0063] The credibility weight, overlap ratio weight, and usable value weight are all dimensionless weights, and their sum is 1. During implementation, the overlap ratio weight should not be less than the credibility weight, and the credibility weight should not be less than the usable value weight; before on-site calibration is completed, the credibility weight should be set to [value missing]. Overlap ratio weighting The value weight can be used to take After completing the on-site calibration, the candidate matching and tracking unit 170 calculates the difference in candidate matching scores between the real water level candidate region and the pseudo edge candidate region in the normal steady-state calibration frame, and adjusts the three weights to ensure that the candidate matching score of the real water level candidate region is consistently higher than that of the pseudo edge candidate region. The real water level candidate region is a candidate region that includes the outline of the real water level and whose bounding rectangle covers the set of pixels of the water level outline between the left and right intersection positions; the pseudo edge candidate region is a candidate region that does not simultaneously cover the left and right intersection positions and is formed by reflective edges, door leaf shadows, foam edges, or shoreline textures.

[0064] The lower limit of the matching score is determined during the on-site calibration phase. The candidate matching and tracking unit 170 selects a calibration frame containing both the real water level candidate region and common pseudo-edge candidate regions, and calculates the candidate matching score for each. Common pseudo-edge candidate regions are selected from the pseudo-edge candidate regions and displayed to the operator for confirmation by the interactive display terminal 190 within the same calibration frame. After confirmation, the scores are written into the calibration sample record. The lower limit of the matching score is the midpoint between the minimum candidate matching score of the real water level candidate region and the maximum candidate matching score of the pseudo-edge candidate region. If there is overlap, the lower limit of the matching score is the lower quartile of the candidate matching score of the real water level candidate region, and the overlap ratio is increased before recalibration until the real water level candidate region continuously meets the lower limit of the matching score.

[0065] After obtaining the candidate matching scores, the candidate matching and tracking unit 170 selects the candidate region with the highest candidate matching score that also meets the lower limit of the matching score to continue the water level trajectory. (Lower limit of matching score) The system can be pre-configured based on the recognition confidence range output by a conventional visual recognition model, the conventional distribution of candidate overlap ratios, and the on-site false alarm tolerance. If no candidate region meets the lower limit of the matching score, the waterline trajectory is set to a lost waiting state, and the previous waterline advance zone is retained for subsequent re-matching.

[0066] When in a lost-wait state, the candidate matching and tracking unit 170 does not immediately delete the waterline trajectory or create a new duplicate trajectory number. Instead, it continues to receive candidate regions, waterline advance zones, and available values ​​for the advance zones in subsequent video frames. If the candidate region in a subsequent video frame again meets the candidate matching score condition, the original waterline trajectory and original tracking number continue. If the water level gauge 110 is temporarily unavailable or the left and right intersection positions are obstructed, the system reverts to regular visual tracking and re-forms the waterline advance zone after the effective data is recovered.

[0067] Example 8: The interactive display terminal 190 displays the current video frame, water level trajectory, tracking number, waterline advance zone range indication, and correction results. Operators can click on the water level line or drag water level line control points on the interactive display terminal 190 to create manual click positions. These manual click positions are input to the interactive correction unit 180 from the interactive display terminal 190 via dashed arrows, such as... Figure 1 As shown, the interactive correction unit 180 then feeds back the corrected click position and the updated water level trajectory to the candidate matching and tracking unit 170.

[0068] When an operator clicks on the water level line or drags its control point, the interactive correction unit 180 determines whether the clicked position is within the waterline advance zone. If the clicked position is within the waterline advance zone, the interactive correction unit 180 corrects the clicked position to the water level line outline position in the nearest candidate area within the waterline advance zone, obtaining the corrected click position. The corrected click position is expressed as follows: ; in, Indicates the location of the manual click. This indicates the clicked location after calibration. Indicates video duration The waterline propulsion belt, This indicates that the manually clicked location is projected onto the waterline outline of the nearest candidate region within the waterline advance zone. This projection can be achieved using conventional nearest-point projection or nearest-candidate-location selection, and its purpose is to restrict the manually clicked location to a reasonable area within the frame represented by the waterline advance zone.

[0069] When a manual click is made to project a location, the interactive correction unit 180 first determines whether a water level contour position exists within a candidate region of the waterline advance zone. If it does, it calculates the Euclidean distance from the manually clicked location to the water level contour position in each candidate region and selects the location with the smallest distance as the corrected click location. If multiple locations have the same distance, it selects the location within the candidate region with the highest recognition confidence. If the recognition confidence is still the same, it selects the location with the smallest longitudinal distance from the water level trajectory of the previous video time. If no water level contour position exists within a candidate region of the waterline advance zone, the interactive display terminal 190 outputs a waterline advance zone range prompt and does not automatically update the water level trajectory.

[0070] When the manual click location is outside the waterline propulsion zone and forced correction is performed, the system writes the manual click location, video time, current available propulsion zone value, and operator confirmation result into the manual operation log. The number of consecutive recovered frames after forced correction is no less than three video frames; in these video frames, only when the left waterline height, right waterline height, and water level reading are all valid, and the bilateral same-direction values, the same-direction viewpoint value, and the bilateral difference value all meet the condition that the propulsion zone is in an available state, will the candidate matching and tracking unit 170 restore the automatic constraint of the waterline propulsion zone.

[0071] After obtaining the corrected click position, the interactive correction unit 180 updates the water level trajectory with the corrected click position and retains the tracking number. The candidate matching and tracking unit 170 continues to filter candidate regions and continue the water level trajectory from the next video frame onwards, constrained by the waterline advance zone. Because the tracking number remains unchanged, historical playback, anomaly alerts, and manual operation records can be continuously associated with the same water level trajectory.

[0072] When the manually clicked position is outside the waterline advance zone, the interactive display terminal 190 outputs a waterline advance zone range prompt. This waterline advance zone range prompt is used to inform the operator of the reasonable screen range under the current available state. If there are special obstructions on site or the waterline outline is partially obscured, the operator can perform forced correction; after forced correction, the system writes the current manually clicked position into the calibration record, and re-extracts the left and right waterline heights in several consecutive subsequent frames. Only when the bilateral same-direction values, the same-direction values ​​of the number of views, and the bilateral difference values ​​meet the conditions for the advance zone to be in an available state will the automatic constraint of the waterline advance zone be restored.

[0073] Example 9: In the At the start of each processing cycle, the video acquisition unit 120 first receives the current video frame captured by the camera 100 and records the video duration. Simultaneously, the numerical acquisition unit 130 obtains the water level reading from the water level gauge 110. and based on video time The system searches the time buffer queue for a valid water level reading as a baseline. This is done if the water level reading acquisition time is different from the video time. If the time difference between the two values ​​does not exceed the preset allowable value, then the water level reading is taken as the valid water level reading of the current video frame.

[0074] Subsequently, the waterline double-end position extraction unit 140 determines the fixed horizontal position of the left gate slot based on the current video frame. Fix the horizontal position of the right door slot The height of the left waterline is read from the position where the waterline outline intersects with the left side of the fixed edge of the left gate slot. The height of the right waterline is read from the position where the waterline outline intersects with the right side of the fixed edge of the right gate slot. And determine the left waterline point. and the right waterline point Candidate region acquisition unit 150 acquires candidate regions from the current video frame. The candidate matching and tracking unit 170 maintains the existing water level trajectory and tracking number.

[0075] Next, the waterline propulsion belt generation unit 160 reads the video time. The height of the waterline on the left side has already been saved. Right side waterline height and water level readings Calculate the change in the waterline on the left side. Changes in the waterline on the right side and water level change Then calculate the bilateral same-direction values. , Viewpoints in the same direction and two-sided difference And based on the bilateral same-direction value, the same-direction value of the number of views, the bilateral difference, and the upper limit of the bilateral difference. Determine the available value of the propulsion belt .

[0076] If the propulsion belt has available value In an available state, the waterline propulsion zone generation unit 160 further calculates the water level conversion ratio on the screen. and propulsion belt width A waterline advance zone is formed by the left and right waterline points of adjacent video times within the frame area. The candidate matching and tracking unit 170 calculates candidate regions. With waterline propulsion belt Candidate overlap ratio Then calculate the candidate matching score. If the candidate region with the highest matching score satisfies the lower limit of the matching score. If so, the candidate region is used to continue the water level trajectory and the tracking number is retained.

[0077] If the propulsion belt has available value In an unavailable state, the candidate matching and tracking unit 170 does not update the waterline trajectory using the corresponding left and right waterline heights, but instead reverts to regular visual tracking. If no candidate region in the current video frame meets the lower limit of the matching score... If the waterline trajectory is lost, the previous waterline advance zone is retained for subsequent rematching. The current video frame, video time, valid water level reading status, available advance zone value, waterline trajectory status, and tracking number are all saved in the history.

[0078] When the operator is at During each processing cycle, when the water level line is clicked or the water level line control point is dragged via the interactive display terminal 190, the interactive correction unit 180 obtains the manual click position. If the location is manually clicked. Located in the waterline propulsion zone Within, the corrected click position is generated. The water level trajectory is updated with the corrected click location, and the tracking number is retained. (If the location is manually clicked...) Located in the waterline propulsion zone In addition, the interactive display terminal 190 outputs a waterline propulsion belt range prompt and decides whether to perform forced correction based on the operator's confirmation.

[0079] As an optional implementation, the water surface reflection is weak, and both the left and right intersection positions can be stably read from the water level line contour. The water level gauge 110 continuously outputs valid water level readings. At this time, the directions of the water line change on the left and right sides are consistent, the difference between the two sides does not exceed the upper limit of the difference between the two sides, the direction of water line movement in the image conforms to the imaging relationship of the fixed camera, and the available value of the propulsion zone is in an available state. The water line propulsion zone generation unit 160 forms the water line propulsion zone, the candidate matching and tracking unit 170 selects the candidate area with the highest candidate matching score and meets the lower limit of the matching score to continue the water level line trajectory, and the interactive display terminal 190 continuously displays the same tracking number.

[0080] As an optional implementation, the water level readings show a significant change in water level between adjacent video times, with the left and right waterline heights changing synchronously with the actual water surface. The waterline propulsion zone generation unit 160 determines a large propulsion zone width based on the basic expansion width, water level expansion coefficient, image water level conversion ratio, and water level change, enabling the waterline propulsion zone to cover the range of water level movement between adjacent video times. The candidate matching and tracking unit 170 prioritizes candidate regions within the waterline propulsion zone, reducing the possibility of candidate regions being misjudged as deviating from the original waterline trajectory due to rapid water level changes.

[0081] As an optional implementation, floating objects, foam, or door leaf shadows obscure the left or right intersection positions. The waterline end position extraction unit 140 marks the corresponding left or right waterline height as temporarily unavailable and does not use reflective edges or door leaf shadows as substitutes. Since the left waterline height, right waterline height, and water level reading are not simultaneously valid, the waterline propulsion band generation unit 160 makes the available propulsion band values ​​unavailable. The candidate matching and tracking unit 170 does not use the corresponding left and right waterline heights to update the waterline trajectory and reverts to conventional visual tracking, thereby avoiding abnormal edges from deviating the waterline trajectory.

[0082] As an optional implementation, a pseudo waterline contour exists in the current video frame due to abrupt changes in brightness. The candidate region acquisition unit 150 may output the pseudo-edge as a candidate region, but this candidate region typically has a low degree of spatial overlap with the waterline advance zone. After the candidate matching and tracking unit 170 calculates the candidate overlap ratio, the candidate matching score of the candidate region corresponding to the pseudo-edge is lower than the candidate matching score of the real waterline candidate region, or does not meet the lower limit of the matching score. Therefore, the waterline trajectory will not extend to the pseudo-edge.

[0083] In one optional implementation, if the water level gauge 110 experiences a communication interruption, exceeds its measurement range, or the time difference exceeds a preset allowable value, the data acquisition unit 130 marks the water level reading as temporarily unavailable. At this time, the waterline propulsion zone generation unit 160 does not use the water level reading to determine the available propulsion zone value, and the candidate matching and tracking unit 170 reverts to conventional visual tracking, retaining the previous waterline propulsion zone for subsequent rematching. Once the water level reading is remarked as valid, and both the left and right waterline heights are valid, the system resumes waterline propulsion zone generation.

[0084] As an optional implementation, if an operator notices a deviation in the water level trajectory and clicks on the water level line on the interactive display terminal 190, and if the click location is within the waterline advance zone, the interactive correction unit 180 corrects the click location to the water level line outline position in the nearest candidate area within the waterline advance zone, obtaining the corrected click location. The water level trajectory is then updated with the corrected click location, and the tracking number is retained. If the click location is outside the waterline advance zone, the interactive display terminal 190 outputs a waterline advance zone range indicator, enabling the operator to identify the relationship between the current click location and the waterline advance zone.

[0085] It should be noted that the camera 100 can be an industrial camera or a water conservancy monitoring camera fixedly installed near a sluice gate, canal, or river cross-section. The water level gauge 110 can be an existing water level gauge on site, outputting water level readings. The video acquisition unit 120, the numerical acquisition unit 130, the waterline dual-end position extraction unit 140, the candidate area acquisition unit 150, the waterline propulsion zone generation unit 160, the candidate matching and tracking unit 170, and the interactive correction unit 180 can be executed by an edge computing device, a monitoring server, or both collaboratively. The interactive display terminal 190 can be a monitoring center display terminal or a duty terminal.

[0086] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A smart interactive tracking method for water conservancy monitoring based on view-to-data fusion, used for tracking water level lines in water conservancy monitoring screens, characterized in that... include: Acquire monitoring video and record video time, obtain water level readings corresponding to the video time; determine the fixed lateral position of the left gate slot and the fixed lateral position of the right gate slot, read the left waterline height and the right waterline height, and determine the left waterline point and the right waterline point; obtain candidate regions from the current video frame, establish water level line trajectories and configure tracking numbers for the water level line trajectories; based on the left waterline height, right waterline height and water level readings of adjacent video times, determine the left waterline change, right waterline change, water level change, bilateral same-direction value, visual number same-direction value and bilateral difference, and verify the consistency of the movement direction of the left waterline change and the right waterline change based on the bilateral same-direction value, verify the imaging relationship between the waterline movement direction and the water level reading change direction based on the visual number same-direction value, and limit the difference between the left waterline change and the right waterline change based on the bilateral difference and the upper limit of the bilateral difference to determine the available value of the propulsion zone; When the available value of the propulsion zone is available, a waterline propulsion zone is formed based on the left and right waterline points of adjacent video times. The waterline propulsion zone is used to constrain the selection of candidate regions, the continuation of the waterline trajectory, and the correction of manual click positions. When the available value of the propulsion zone is unavailable, the corresponding left and right waterline heights are not used to update the waterline trajectory, and the system reverts to regular visual tracking.

2. The intelligent interactive tracking method for water conservancy monitoring based on visual-to-digital fusion as described in claim 1, characterized in that, The process of acquiring and recording surveillance video includes: based on the fixed correspondence between the camera and the water level gauge, recording the video time and the acquisition time of the water level reading using a unified clock, so that the water level reading can be correlated with the acquisition time and the video time.

3. The intelligent interactive tracking method for water conservancy monitoring based on visual-to-digital fusion according to claim 1, characterized in that, Obtaining the water level reading corresponding to the video time includes: using the video time as a reference, searching for the water level reading marked as valid in the time buffer queue; when the time difference does not exceed a preset allowable value, using the corresponding water level reading as the valid water level reading of the current video frame to determine the usable value of the propulsion zone; when the time difference exceeds the preset allowable value, marking the water level reading as temporarily unavailable.

4. The intelligent interactive tracking method for water conservancy monitoring based on visual-to-digital fusion according to claim 1, characterized in that, Reading the left and right waterline heights includes: extracting the waterline contour from the video frame, reading the left waterline height corresponding to the intersection of the waterline contour with the left side of the fixed edge of the left gate slot, and reading the right waterline height corresponding to the intersection of the waterline contour with the right side of the fixed edge of the right gate slot; when the left or right intersection is occluded, the corresponding left or right waterline height is marked as temporarily unavailable, and the reflective edge or gate leaf shadow is not used as a substitute.

5. The intelligent interactive tracking method for water conservancy monitoring based on visual-to-digital fusion according to claim 1, characterized in that, Determining the usable value of the advance zone includes: when the bilateral same-direction values, representing the changes in the waterline on the left and right sides, do not show any opposite movement, and the bilateral same-direction values, representing the direction of waterline movement in the image, conform to the imaging relationship of the fixed camera and the direction of water level reading changes, and the bilateral difference does not exceed the upper limit of the bilateral difference, the usable value of the advance zone is set to the usable state; otherwise, the usable value of the advance zone is set to the unusable state.

6. The intelligent interactive tracking method for water conservancy monitoring based on visual-to-digital fusion according to claim 5, characterized in that, The formation of the waterline advance zone includes: determining the width of the advance zone based on the influence of the basic expansion width, the water level expansion coefficient, and the water level change on the range of waterline movement in the picture; the basic expansion width is used to cover slight fluctuations, and the water level expansion coefficient is used to widen the candidate area when the water level changes rapidly; the picture area enclosed by the left and right waterline points of adjacent video times is expanded according to the width of the advance zone to obtain the waterline advance zone.

7. The intelligent interactive tracking method for water conservancy monitoring based on visual-to-digital fusion according to claim 6, characterized in that, The selection of candidate regions and the continuation of waterline trajectories constrained by the waterline advance zone include: calculating the candidate overlap ratio between the candidate region and the waterline advance zone; combining the recognition confidence, candidate overlap ratio, and available values ​​of the advance zone into a candidate matching score; selecting the candidate region with the highest candidate matching score that meets the lower limit of the matching score to continue the waterline trajectory; when no candidate region meets the lower limit of the matching score, the waterline trajectory is set to a lost waiting state, and the previous waterline advance zone is retained for subsequent rematching.

8. The intelligent interactive tracking method for water conservancy monitoring based on visual-to-digital fusion according to claim 7, characterized in that, The manual click position correction using the waterline advance zone constraint includes: when the operator clicks on the waterline or drags the waterline control point, determining whether the manual click position is within the waterline advance zone; if the manual click position is within the waterline advance zone, correcting the manual click position to the waterline outline position in the nearest candidate area within the waterline advance zone, obtaining the corrected click position, updating the waterline trajectory with the corrected click position and retaining the tracking number; if the manual click position is outside the waterline advance zone, outputting a waterline advance zone range prompt.

9. A smart interactive tracking system for water conservancy monitoring based on visual-data fusion, characterized in that, include: The video acquisition unit is used to acquire surveillance video and record the video duration. The numerical acquisition unit is used to acquire water level readings corresponding to video time; the waterline double-end position extraction unit is used to determine the fixed lateral position of the left gate slot and the fixed lateral position of the right gate slot, read the waterline height on the left and the waterline height on the right, and determine the waterline point on the left and the waterline point on the right. The candidate region acquisition unit is used to obtain candidate regions from the current video frame; the waterline propulsion zone generation unit is used to determine the available value of the propulsion zone based on the change in the left waterline, the change in the right waterline, the change in water level, the same value on both sides, the same value of the number of views on both sides, and the difference between the two sides, and to form the waterline propulsion zone when the available value of the propulsion zone is in an available state; The candidate matching and tracking unit is used to filter candidate regions and continue the waterline trajectory by using the waterline propulsion zone constraint, and to fall back to conventional visual tracking when the available value of the propulsion zone is unavailable. An interactive correction unit is used to correct the position of manual clicks by using a waterline propulsion belt to constrain the position.

Images