A high-concurrency urban river floating object management method and system based on a layered architecture

Abstract

The application provides a high-concurrency urban river floating object management method and system based on a layered architecture, relates to the technical field of artificial intelligence and water conservancy engineering management, and comprises the following steps: obtaining multi-modal original data based on a sensing device; performing adaptive frame extraction on a video stream formed by the video data based on the coupling of flow velocity constraints and scene complexity; and performing lightweight feature extraction within the field of view of a single camera, candidate region confirmation, cross-frame tracking and event discrimination on the image frames obtained after frame extraction, and outputting floating object target information. According to the adaptive frame extraction mechanism coupled with flow velocity constraints and scene complexity, the frame extraction interval is dynamically adjusted according to factors such as river flow velocity, floating object level, picture structure change, target density, risk level and environmental disturbance, so that invalid frame processing is greatly reduced under the premise that key events are not lost.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and water conservancy project management technology, specifically to a method and system for managing floating debris in urban rivers based on a hierarchical architecture. Background Technology

[0002] Floating debris in urban waterways (including domestic waste, agricultural waste, plant remains, and industrial waste) is a significant issue affecting urban water quality, flood control, and ecological health. Especially during the flood season, the amount of floating debris can increase 5-10 times compared to normal, drastically increasing the pressure on its management.

[0003] Currently, the management of floating debris in urban waterways mainly relies on manual inspections and fixed camera monitoring, which has the following technical shortcomings: I. Insufficient high-concurrency processing capability Existing video analytics-based river floating object detection solutions mostly employ fixed frame extraction or frame-by-frame inference methods. Taking 100 cameras and 25fps video input per channel as an example, the original frame rate can reach 2500 frames per second. Using a general large model for frame-by-frame inference would generate enormous computational overhead, making it difficult to meet real-time requirements. Although some solutions introduce object detection frameworks for front-end screening, they lack adaptive frame extraction mechanisms for river flow patterns and floating object movement characteristics. In scenarios with dense targets, such as during flood season, problems such as high computational load, missed detection of critical events, or repeated triggering still exist.

[0004] Third, the decision-making process lacks professional knowledge support. Existing systems rely heavily on human experience or generic question-and-answer generation in the assessment and disposal decision-making stages of floating debris, lacking the systematic application and reasoning capabilities to professional knowledge such as river management standards, floating debris classification criteria, historical cases, and legal provisions. The generation of disposal plans often lacks a traceable chain of evidence, making it difficult to quickly output stable, compliant, and resource-appropriate disposal plans for special scenarios such as flood seasons and algal blooms.

[0005] IV. Business processes are not in a closed loop. In current river debris management, the processes of target identification, task assessment, work order issuance, on-site disposal, and effect evaluation are fragmented, lacking a unified event status tracking and coordination mechanism. There is a lack of standardized work order flow channels between identification results and disposal conclusions, and the disposal effects lack quantitative evaluation and feedback correction, preventing the system from continuously optimizing identification parameters, decision-making logic, and solution matching effectiveness based on historical experience. Furthermore, existing solutions generally lack digital twin support capabilities, making it difficult to achieve floating debris outbreak trend forecasting, risk warning, diffusion path simulation, and intelligent plan recommendation.

[0006] In summary, existing technologies for managing floating debris in urban waterways suffer from several drawbacks, including insufficient high-concurrency processing capabilities, difficulty in integrating multi-source heterogeneous data, lack of professional knowledge support for decision-making, and the absence of a closed-loop business process. There is an urgent need for an intelligent management solution that can achieve efficient identification, multi-source fusion, knowledge-driven approaches, and closed-loop control. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a high-concurrency urban river floating debris management method and system based on a hierarchical architecture, which solves the problems of insufficient high-concurrency processing capabilities, difficulty in integrating multi-source heterogeneous data, lack of professional knowledge support for decision-making, and failure to form a closed loop in business processes in existing technologies.

[0008] To achieve the above objectives, the present invention provides the following technical solution: A high-concurrency urban river debris management method based on a hierarchical architecture includes the following steps: Step S1: Based on the sensing device, acquire multimodal raw data; the multimodal raw data includes: video data and river data; Step S2: Based on the coupling of flow rate constraints and scene complexity, adaptive frame extraction is performed on the video stream composed of the video data; and lightweight feature extraction, candidate region confirmation, cross-frame tracking and event discrimination are performed on the image frames obtained after frame extraction within the field of view of a single camera, and floating object target information is output. Step S3: Perform three-level fusion on the multimodal raw data and the floating object target information, namely, data-level alignment, feature-level joint representation and decision-level conflict resolution, and unify the coordinates from different sources to the same geographic coordinate system; Step S4: Based on preset rules, each intelligent agent is driven to work collaboratively to perform identification, analysis, handling, and evaluation collaboratively; the intelligent agent is an autonomous collaborative unit that realizes the business functions of river management. Step S5: Call the professional knowledge base and attribute knowledge graph to perform enhanced retrieval generation, related reasoning, and disposal plan generation; Step S6: Implement closed-loop management and digital twin simulation of the entire process of river floating debris management.

[0009] Preferably, the sensing devices include: fixed visual acquisition devices, mobile visual inspection devices, and hydrological parameter sensing devices.

[0010] Preferably, the adaptive frame extraction in step S2 includes: Based on the river flow velocity in the river data and the floating object level in the current frame image of the video data, the equivalent velocity of the floating object is determined through a preset physical model; and the shortest dwell time of the floating object within the camera's field of view is calculated. The frame extraction interval is calculated based on the video frame rate, effective response time window, and dynamic adjustment items. The effective response time window is the smaller of the conservative correction value of the shortest dwell time and the alarm time limit value. The dynamic adjustment items include at least the multidimensional structural change index, the target density in the image, the normalized value of the event risk level, and the environmental disturbance factor formed by the image jitter, water level fluctuation, wind and wave disturbance, and sudden changes in illumination. The multidimensional structural change index is determined based on the changes in the target spatial structure, the shift in category distribution, and sudden changes in motion patterns. The frame skipping interval is positively correlated with the product of the frame rate and the effective response time window, negatively correlated with the sum of the dynamic adjustment terms, and the minimum value is 1. Adaptive frame extraction is performed based on the frame extraction interval.

[0011] Preferred options also include: When the multidimensional structural change index exceeds the preset adaptive threshold, the continuous frames are aggregated into event segments; otherwise, the lightweight tracking module is used.

[0012] Specifically, when the multidimensional structural change index exceeds a preset threshold, the current frame is marked as the event start frame, and subsequent frames are continuously monitored until the multidimensional structural change index is lower than the threshold or the event duration exceeds a preset window, and the continuous frames between the start frame and the end frame are aggregated into an event segment.

[0013] The lightweight tracking module employs a combination of Kalman filtering and IoU matching: the Kalman filter predicts the target's position in the current frame based on its historical motion state (position, velocity); then, it performs IoU matching between the predicted position and the foreground region obtained through simple background modeling. Targets that successfully match inherit their original ID, and the trajectory is updated. This module's single-frame processing time is less than 3 milliseconds, only 1 / 5 to 1 / 10 of the time required for complete detection, thus significantly reducing the computational load while ensuring tracking continuity. (The "lightweight tracking module" is a low-cost, high-efficiency target position prediction and update method used to maintain cross-frame tracking of existing targets in non-critical frames where image changes are insignificant, avoiding repeated execution of computationally intensive complete detection.)

[0014] Preferably, the adaptive frame extraction in step S2 further includes: An upper and lower bound value is configured for the frame extraction interval, so that the frame extraction interval is constrained between the lower and upper bound values; the lower and upper bound values ​​are configured in segments according to the river segment type.

[0015] The lower bound is generally a value greater than 1, and the specific segmentation configuration is based on the river segment type; if the lower bound is less than 1, the adaptive frame extraction interval constraint greater than or equal to 1 in step S2 is used as a fallback.

[0016] Preferably, it further includes: calculating the effective throughput for each river segment, wherein the effective throughput is determined based on the number of video acquisition devices in the river segment, the video frame rate, the adaptive frame extraction interval, the scheduling efficiency correction coefficient, and the low-value event filtering rate; The effective throughput of each river section is weighted and summed to obtain the assessment result of the concurrent capacity of the entire river.

[0017] Preferably, step S3, which performs a three-level fusion of data-level alignment, feature-level joint representation, and decision-level conflict resolution on the multimodal raw data and the floating object target information, includes: Data-level fusion: Complete the temporal alignment, spatial alignment, and format standardization of video data and river data to form a unified basic data foundation; Feature-level fusion: The visual features of visual targets in video data are jointly encoded with the hydrological operational features in river data to generate a unified event feature vector for identification and analysis. Visual features include category, area, color, trajectory, and drift speed; hydrological operational features include river flow velocity, water level, flow rate, and river section attributes. An event token compression mechanism is used to compress the temporal target set and trajectory state set of events into structured events through an event aggregation function. Decision-level fusion: The confidence level of multi-source identification results, rule-based judgment results and collaborative judgment results are weighted, conflict is resolved and consistency is checked, and the final floating object category, risk level, location and disposal recommendations are output according to the fusion confidence level.

[0018] Preferably, step S3 converts coordinates from different sources to the same geographic coordinate system—using a coordinate transformation link of “pixel coordinate system—device local coordinate system—river engineering coordinate system—WGS84 geographic coordinate system”.

[0019] Preferably, step S4 specifically includes: For each event fragment, calculate the probability of dynamic invocation; The dynamic call probability is the value obtained by mapping the weighted sum of information entropy, structural complexity, and business risk through the Sigmoid function. When the dynamic call probability exceeds a preset threshold, the multimodal large model is triggered to perform deep reasoning, and the deep reasoning result of the multimodal large model is used as the judgment conclusion. When the dynamic call probability does not exceed the preset threshold, the floating object target information is directly used as the judgment conclusion.

[0020] A high-concurrency urban river floating debris management system based on a hierarchical architecture includes: The perception access layer acquires multimodal raw data based on perception devices; the multimodal raw data includes video data and river data. The layered recognition layer performs adaptive frame extraction based on the coupling of flow rate constraints and scene complexity on the video stream composed of the video data, and performs target detection, cross-frame tracking and event discrimination on the image frames obtained after frame extraction, and outputs floating object target information. The data fusion layer performs three-level fusion on the multimodal raw data and the floating object target information: data-level alignment, feature-level joint representation, and decision-level conflict resolution, and uniformly transforms coordinates from different sources to the same geographic coordinate system. The agent collaboration layer contains multiple agents. Each agent shares event states through a state bus, transmits control permissions through task tokens, and works collaboratively according to preset rules to achieve task distribution, result feedback, and closed-loop control. The knowledge support layer is used to store professional knowledge bases and attribute knowledge graphs, and to perform retrieval enhancement generation, correlation reasoning and disposal solution generation. The closed-loop management layer implements full-process closed-loop management and digital twin simulation of river floating debris control.

[0021] This invention provides a high-concurrency urban river floating debris management method and system based on a hierarchical architecture. It has the following beneficial effects: 1. This invention employs an adaptive frame extraction mechanism coupled with flow velocity constraints and scene complexity. It dynamically adjusts the frame extraction interval based on factors such as river flow velocity, floating object level, image structure changes, target density, risk level, and environmental disturbances, significantly reducing invalid frame processing while ensuring no key events are lost. Simultaneously, it triggers key frame aggregation through multi-dimensional structural change indicators, compressing continuous frames into event fragments. This, combined with a lightweight tracking module, maintains target tracking on non-key frames, effectively avoiding redundant detection and computation. In a typical scenario with 100 cameras and 25fps per channel, this invention achieves an effective throughput of 1500-1800 targets / second, an improvement of 8-10 times compared to traditional visual solutions without frame extraction or collaborative deduplication. The invalid trigger rate per camera decreases by 25%-40%, and the effective recognition stability improves by 8%-15%.

[0022] 2. This invention constructs a multimodal perception system consisting of a fixed camera, a drone, and IoT sensors. By unifying the time scale, spatial reference, and attribute coding rules, it converges observation results scattered across different devices into a single semantic unit for business events. A four-level coordinate transformation link—pixel coordinate system—device local coordinate system—river engineering coordinate system—WGS84 geographic coordinate system—is adopted, and compensation mechanisms such as time-varying scale correction, water level and elevation compensation, attitude disturbance correction, recalibration, and time sequence smoothing are introduced, significantly improving positioning stability under actual working conditions such as water level fluctuations, camera jitter, and occlusion. The drone side employs RTK + flight attitude compensation to achieve meter-level dynamic target positioning. Testing shows that this invention can achieve floating object positioning accuracy of 1-3 meters and a traceability accuracy of 80%-88%, which is superior to single-vision positioning schemes or schemes without unified coordinate transformation. Attached Figure Description

[0023] Figure 1 This is a flowchart of a high-concurrency urban river floating debris management method based on a hierarchical architecture proposed in this invention; Figure 2 This is a diagram illustrating the collaborative and state routing structure of various agents in a high-concurrency urban river floating debris management method based on a hierarchical architecture proposed in this invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Example 1: like Figures 1-2 As shown, this embodiment of the invention provides a high-concurrency urban river floating debris management method based on a hierarchical architecture, comprising the following steps: Step S1: Based on the sensing device, acquire multimodal raw data; the multimodal raw data includes: video data and river data.

[0026] Sensing devices include: fixed visual acquisition devices, mobile visual inspection devices, and hydrological parameter sensing devices; for example, IPC (Internet Protocol Camera) cameras fixed along the river, supporting dual-mode switching of visible light and infrared, with a video acquisition frame rate of 25fps-30fps, accessing the system via the RTSP protocol; drone swarms equipped with RTK (Real-Time Dynamic Differential) positioning modules, with positioning accuracy of ±0.1m-0.3m and flight time of 30min-45min; drones conduct dynamic area inspections according to preset routes or manual remote control, transmitting GPS coordinates, flight attitude, and inspection image data captured by onboard cameras via the MAVLink protocol; flow sensors, water level gauges, and Doppler current meters deployed at key river sections, accessing via MQTT or HTTP protocols; flow velocity measurement accuracy of ±0.05m / s-0.1m / s, water level accuracy of ±0.01m-0.03m, and collected data fields including flow velocity v (m / s), water level h (m), and flow rate Q (m³ / s). 3 / s) and the corresponding timestamp.

[0027] All sensing devices are uniformly connected to the multimodal data gateway through a dedicated 5G communication network and time synchronization is performed using NTP (Network Time Protocol), with a synchronization error of ≤10ms; the coordinate system adopts the WGS84 geographic coordinate system.

[0028] Step S2: Based on the coupling of flow rate constraints and scene complexity, adaptive frame extraction is performed on the video stream composed of the video data; and lightweight feature extraction, candidate region confirmation, cross-frame tracking and event discrimination are performed on the image frames obtained after frame extraction within the field of view of a single camera, and floating object target information is output.

[0029] Based on the volume, duration of floating debris, potential hazards, and urgency of disposal, the objects to be addressed can be classified into four levels, L1 to L4: L1 consists of large, high-risk floating debris that may affect flood control safety; L2 consists of continuously accumulating floating debris that significantly impacts navigation or the landscape; L3 consists of scattered floating debris that can be included in routine inspections and disposal; and L4 consists of short-lived, minimally impactful, and continuously observable low-risk floating debris. According to this classification, this embodiment focuses on addressing L1 and L2 floating debris.

[0030] Table 1 shows the detailed characteristics and parameters of the four levels of governance objects from L1 to L4:

[0031] Table 1 Constraints: Floating objects in urban waterways move with the water flow; the equivalent velocity v of the graded floating objects... debris With water flow velocity v flow The relationship is constrained by the L1-L4 classification of floating objects.

[0032] The equivalent velocity of the L1-level floating object is calculated as follows: ; This is the L1 high-velocity weighting coefficient.

[0033] The equivalent velocity of the L2-level floating object is calculated as follows: ; This is the L2 high-velocity weighting coefficient.

[0034] The optimal frame extraction interval is calculated based on the above constraints. The specific processing flow is as follows: Step S21: Calculate the shortest dwell time T of the floating object within the camera's field of view. min : ; in, The length of the river section covered by the camera (m); To determine the equivalent velocity (m / s) of the floating debris, the image is first processed using intelligent recognition to classify the target debris into L1 or L2. Then, the water flow velocity v directly obtained from the Doppler current meter is used. flow The equivalent velocity of the graded floating objects is calculated using the equivalent velocity calculation method corresponding to the above-mentioned classification of the treatment objects (L1 or L2). .

[0035] Step S22: Determine the effective response time window Ensure at least K samples are collected. min =2 frames: ; in, As a conservative coefficient to cope with factors such as camera shake and water level fluctuations, As a conservative correction value for the shortest dwell time, the effective response time window is taken. As an alarm response time limit parameter, the relevant parameters can be set according to the specific river scenario and business requirements; for example, they can be dynamically configured according to the risk level of the river section: 20 seconds for Level 1 high-risk river sections, 30 seconds for Level 2 medium-risk river sections, and 60 seconds for Level 3 low-risk river sections, etc.

[0036] To accurately quantify inter-frame structural changes and avoid false triggering with fixed thresholds, a multi-dimensional structural change index Δt is introduced, which comprehensively considers the target spatial structural changes d. box Category distribution offset d cls and sudden changes in movement patterns d motion Three dimensions: ; in, ∈[0,1], is used to measure the change in the target spatial structure (based on the average IoU change and the displacement of the center point). ∈[0,1], is a measure of class distribution shift (based on Jensen-Shannon divergence); ∈[0,1], is used to measure abrupt changes in motion patterns (based on differences in trajectory velocity / direction / acceleration). , , These are the weighting coefficients, with a default value of (0.5, 0.3, 0.2), and they satisfy the following conditions: + + =1 constraint requirement.

[0037] when Exceeding the preset adaptive threshold When the keyframe flag gt=1 is triggered, the system enters event organization mode, aggregating consecutive frames into event fragment E. k =[t ks , t ke Otherwise, if gt=0, the lightweight tracking module is used (the lightweight tracking module is a low-cost, high-speed target tracking method used to maintain continuous tracking of existing targets on non-critical frames (i.e., frames where the image structure does not change significantly) without re-performing full target detection). This mechanism reduces unnecessary high-cost inference triggers and improves the processing efficiency of the event organization phase.

[0038] Step S23: Calculate the optimal frame extraction interval : ; in, K0 is the video frame rate, K1 is the structural change adjustment coefficient, K2 is the scene congestion adjustment coefficient, K3 is the business risk adjustment coefficient, and K4 is the environmental disturbance adjustment coefficient; ρt is the current target density of the image, Rt is the normalized value of the current event risk level, and Jt is the environmental disturbance factor formed by image jitter, water level fluctuation, wind and wave disturbance, and sudden changes in illumination.

[0039] Among them, the coefficients K0, K1, K2, K3, and K4 are all non-negative parameters, and preferably satisfy K0∈[2,4], K1∈[1,3], K2∈[0.5,2], K3∈[0.5,2], and K4∈[0.5,1.5]. ρt, Rt, and Jt can be normalized and limited to the [0,1] interval. When structural changes increase, target density rises, event risk increases, or environmental disturbances increase, the denominator increases, and the frame extraction interval automatically decreases to improve the retention rate of key segments and reduce the probability of missed triggering of high-risk events.

[0040] To avoid overly dense or sparse sampling in local extreme scenarios using a single frame-sampling formula, this embodiment sets upper and lower bound constraints N for frame-sampling. min ≤N frame ≤N max For example, N can be classified according to river segment type, monitoring coverage length, and response time limit. min and N max Segmented configurations are implemented; and different parameter groups can be used for the main river channel, the slow-flowing tributary area, and the easily deposited area at the sluice gate.

[0041] In a typical scenario analysis example, the coverage length L of the river segment can be set. fov The equivalent velocity v of the floating object debris Frame rate f ps Response time T alarm The above adjustment parameters are substituted into the formula to calculate the frame extraction interval, and the actual execution parameters are selected according to the project requirements.

[0042] Adaptive frame extraction is performed based on the frame extraction interval; the single-path inference frequency can be calculated by combining the original video frame rate with the frame extraction interval; the reduction ratio of inference load can be calculated based on the number of frames or events actually involved in inference before and after frame extraction, thereby verifying the comprehensive effect of the parameter joint adjustment mechanism on suppressing computational load and retaining key events in high-concurrency scenarios.

[0043] Lightweight feature extraction, candidate region confirmation, cross-frame tracking, and event discrimination are performed on the image frames obtained after frame extraction within the single camera's field of view. Specifically, lightweight feature extraction uses the MobileNet-SSD network to output candidate target bounding boxes and category confidence scores. Candidate region confirmation uses non-maximum suppression (NMS) to filter overlapping boxes. Cross-frame tracking uses a combination of Kalman filtering and IoU matching to assign a unique ID to each target and record its trajectory. Event discrimination is based on the following conditions: the target appears in at least 3 frames, the trajectory crosses a preset warning line (e.g., within 5 meters of the nearshore area), or the category is high-risk (e.g., oil drums, large garbage). If any of these conditions are met, it is determined to be a valid event, and floating object target information is output. The output floating object target information includes information on the discovery of floating object targets and information on the continuous confirmation of floating object targets.

[0044] It can also calculate the concurrent capacity of the entire river segment, and calculate the effective event throughput Ceff(s) (target / second) that the river segment s can stably complete per unit time; Ceff(s) = η sched × Ncam × fps × (1 - L) invalid ) / N frame ; Where Ceff(s) is the effective event throughput (target / second) that river segment s can stably complete per unit time; N cam The number of cameras participating in collaborative processing within the river section; fps is the effective input frame rate of a single video stream; N frame The adaptive frame skipping interval is calculated jointly based on flow velocity constraints, structural changes, risk levels, and environmental disturbances; η sched L is a scheduling efficiency correction coefficient used to characterize the impact of state routing, task token passing, and concurrent queuing on actual throughput capacity; its preferred value is 0.85-0.98. invalid The low-value event filtering rate is used to characterize the proportion of events that no longer enter the subsequent processing chain after being filtered out by lightweight rules, cache hit reduction, and short-term repeated trigger suppression. The preferred value is 0.08-0.25. By removing cross-camera correlation factors, the concurrency calculation is made more consistent with the single-camera independent recognition mode.

[0045] For example, η sched In the scenarios of main river channels, tributaries, and lake bays, values ​​of 0.93, 0.91, and 0.88 can be taken respectively; L invalid The values ​​can be set to 0.10, 0.15, and 0.22 respectively to reflect the differences in the filtering intensity of low-value events in different river sections.

[0046] Compared to existing coarse estimation methods that include cross-camera deduplication, this invention adopts a simplified throughput calculation structure of "scheduling efficiency correction + low-value event filtering", which can reduce the concurrency capacity estimation error from 12%-18% to 4%-7%, improve the work order resource pre-allocation matching degree by about 10%-18%, and is more suitable for high-concurrency and fast recognition scenarios with a single camera.

[0047] By calculating independently for each river section and then weighting and summing the results, a more accurate result of the full-river concurrency capability can be obtained, which is closer to the actual single-camera recognition load.

[0048] Table 2 shows the weighted summary after independent calculations for each river segment:

[0049] Table 2 Step S3 involves performing a three-level fusion process on the multimodal raw data and the floating object target information, which includes data-level alignment, feature-level joint representation, and decision-level conflict resolution, and transforming coordinates from different sources to the same geographic coordinate system.

[0050] Coordinate transformation: A unified coordinate transformation link of "pixel coordinate system - equipment local coordinate system - river engineering coordinate system - WGS84 geographic coordinate system" is adopted.

[0051] For example, after calibration of internal and external parameters, the fixed IPC camera maps the target pixel position to the river engineering coordinates; the UAV combines RTK position, attitude angle, and gimbal parameters to complete the geolocation of the target point; the IoT sensor position uses pre-deployed geographic coordinates. After unified time synchronization, all multi-source data is converted to the WGS84 coordinate system for storage and calculation, and then projected to the business GIS coordinate system when necessary to ensure consistent coordinate standards throughout the entire process of detection, fusion, work order dispatch, and handling feedback.

[0052] The conversion from IPC pixel coordinates to geographic coordinates can be achieved by combining pre-calibrated camera parameters, river surface elevation compensation, attitude disturbance correction, and reference control point recalibration. ; Where st is the time-varying scale correction factor, satisfying ; , and To normalize the scene parameters, all values ​​range from [0,1]. Preferably, Take a value of 0.95-1.05. Take a value between 0.08 and 0.18. Take a value of 0.10-0.25. Take 0.12-0.30, and + + ≤ 0.60, to avoid excessive scaling that could cause positioning drift. The elevation compensation caused by water level changes is preferably expressed as... = · ( - ),in Take a value between 0.60 and 1.20. This is the current water level. For reference water level; This is the local water surface normal vector. The correction term for external parameter disturbances caused by gimbal jitter, installation deflection, or attitude drift is preferably controlled within the range of 0.05m-0.30m. For backcalibration items obtained based on shoreline control points, known landmarks, or historically stable anchor points, it is preferred that the corresponding plane backcalibration deviation is no greater than 0.50m. The smoothing window is preferably 3-7 frames, which is formed by combining the continuous trajectory of adjacent frames and the stable observation results of the target. , , To correspond with the adjusted weights, the preferred option is to satisfy... ∈[0.15,0.30]、 ∈[0.25,0.45]、 ∈[0.20,0.35], and + + =1; when > 0.60 or When < 0.40, increase and The value of is selected to improve positioning stability under occlusion and distant conditions.

[0053] By introducing the time-varying scale correction factor, water surface elevation compensation term, attitude disturbance correction term, recalibration term, and time-series smoothing correction term, this invention can perform scene-based correction of general projection relationships under conditions of river water level fluctuations, slight camera deflection, reflective occlusion, and changes in depth of field. More preferably, when the river is under routine inspection conditions, this invention can employ... : : = 0.20:0.45:0.35; When the monitoring screen is stable and the reference control point is clear, this ratio can be used. : : = 0.25:0.40:0.35; When there is shaking, occlusion, or small distant targets, this can be used. : : = 0.15:0.35:0.50. Through the above numerical optimization, the present invention can improve the consistency, continuity and disturbance resistance stability of the spatial positioning of floating objects, which is different from the coordinate transformation method that only relies on fixed intrinsic and extrinsic parameters for inversion.

[0054] Dynamic target coordinate correction for UAVs (RTK + flight attitude compensation): The final target geographic coordinates : ; For RTK real-time location; The target is offset relative to the machine body; The attitude rotation matrix is ​​used, and the positioning accuracy can be determined based on the RTK status, flight attitude compensation accuracy, and project measurement results.

[0055] The multimodal raw data and the floating object target information are fused at three levels: data-level alignment, feature-level joint representation, and decision-level conflict resolution. Data-level fusion (Level-1): Complete the time alignment, spatial alignment, and format standardization of IPC video streams, drone image streams, and IoT data such as traffic flow and water level, forming a unified basic data foundation; Feature-level fusion (Level-2): Visual features such as target category, area, color, trajectory, and drift speed are jointly encoded with hydrological operational features such as river flow velocity, water level, flow rate, and river section attributes to generate a unified event feature vector that can be identified and analyzed.

[0056] Based on feature-level fusion, an event token compression mechanism can be used to compress the time-series target set { } and trajectory state set { } Through event aggregation functions Compression into structured event representation : = ({ | t ∈ }, { | j ∈ }); It includes structured fields such as target summary, trajectory statistics, spatial relationships and event labels, which are mapped to structured input sequences. This can reduce redundant input in the subsequent large model inference stage and maintain the effective transmission of key semantic information.

[0057] In one embodiment, under the same sample set, the same model version, and the same task requirements, the frame-by-frame input method and the event aggregation input method can be compared and statistically analyzed based on the average number of input tokens per event to illustrate the technical effect of event compression.

[0058] Decision-level fusion (Level-3): This process weights the multi-source identification results, rule-based judgment results, and collaborative analysis results with confidence levels, resolves conflicts, and verifies consistency. Based on the fusion confidence level, it outputs the final floating object category, risk level, location, and disposal recommendations. ; in, Visual recognition confidence level is used to characterize the reliability of fixed monitoring terminals or mobile patrol terminals in identifying the types and outlines of floating objects. Determined jointly by detection confidence and viewing angle quality, it can be expressed as: = · + · ,in , satisfy + =1, with preferred values ​​of 0.70-0.85 and 0.15-0.30 respectively. Hydrological consistency confidence is used to characterize the degree of matching between the motion state of floating objects and operational parameters such as flow velocity, water level, and flow direction. It can be determined by the combination of velocity deviation, position drift and water level change adaptability, and is limited to the range of 0.50-0.95. The confidence level for rule determination is used to characterize the degree to which the detection results conform to the preset governance rules, risk classification rules, and prohibition and restriction conditions; when the detection results simultaneously meet the floating object category rules, area rules, and time period rules, The preferred value is 0.75-0.95; if it is partially satisfied, the preferred value is 0.45-0.75. To collaboratively assess confidence levels, which characterize the degree of consistency among multiple processing units in their conclusions about the same event, it is preferably calculated based on a combination of the consensus rate of voting by the assessment units, the historical hit rate, and the feedback correction results, with an optimal range of 0.55-0.98. This is a conflict penalty term used to weaken the fusion output when there are significant contradictions between conclusions from different sources. It is preferably represented as follows: ,in For category conflict degree, For location conflict degree, For risk level conflict degree, For time consistency deviation, , , , satisfy + + + =1, with preferred values ​​being 0.30-0.40, 0.20-0.30, 0.20-0.30, and 0.10-0.20. The fusion weights for visual information, hydrological information, rule-based information, and collaborative analysis information are respectively determined to meet the following requirements. =1, and 0≤λc≤0.35. Preferably, the fusion weight is dynamically allocated according to the quality factors of each source, and can be expressed as: ,in For the quality score of the corresponding source, These are the basic weighting coefficients.

[0059] Under normal sunny and stable flow conditions The preferred values ​​are 0.35-0.50, 0.15-0.25, 0.15-0.25, and 0.15-0.25, respectively; in scenarios involving nighttime, enhanced shading, or rapid water level fluctuations, the values ​​can be appropriately reduced. And improve or To enhance the scenario adaptability of the fusion results, this invention, by dynamically optimizing the above parameters, differs from the simple summation method with fixed weights. It improves the consistency and stability of the final floating object category determination, risk classification, and disposal recommendations output, even in situations of uneven multi-source information quality, rule conflicts, or increased environmental disturbances.

[0060] Step S4: Based on preset rules, each intelligent agent is driven to work collaboratively to perform identification, analysis, handling, and evaluation collaboratively; the intelligent agent is an autonomous collaborative unit that realizes the business functions of river management. The agent collaboration layer adopts a state-driven task orchestration structure, setting up multiple functional units such as master control, perception, analysis, handling, and evaluation. Each functional unit is triggered sequentially or conditionally based on event state, risk level, and resource consumption, and cross-stage connection is achieved through unified state storage. Its agent collaboration and state routing structure is as follows: Figure 2 As shown.

[0061] 1) Responsibilities and Communication of Four Types of Intelligent Agents To avoid triggering high-cost multimodal large-model inference for every event fragment, this invention designs a dynamic probabilistic invocation mechanism. The decision to invoke the large model is based on a comprehensive assessment of three dimensions: uncertainty, structural complexity, and business risk. Specifically, under the same sample set, model version, and task requirements, the frame-by-frame input method and the event aggregation input method can be compared, and the event compression effect can be evaluated by the average number of input tokens per event. (State call probability) The calculation method is as follows: ; in, Information entropy is used to quantify the uncertainty and semantic ambiguity of events; Structural complexity (number of targets / relationship density / Token length); The level of business risk (including the type of hazardous floating objects, the equipment involved, and other factors); α, β, and γ are weights (default 1 / 3, 1 / 3, 1 / 3); σ is the Sigmoid activation function.

[0062] when Exceeding the threshold Large model inference is triggered when conditions are met; otherwise, lightweight L1 or L2 results are output directly. Through this mechanism, the system can reduce unnecessary high-cost model calls in low-uncertainty and low-complexity scenarios, thereby reducing the overall inference load and improving response efficiency. The percentage reduction in call frequency, latency optimization effect, and performance improvement should be determined in conjunction with implementation test data or project test data.

[0063] 2) State routing and rule orchestration structure The intelligent agent function is sequentially linked through a rule constraint layer, a state routing layer, a parameter calling layer, and a feedback correction layer to form a scalable task processing chain. Control information is transmitted between the layers through event tokens, priority parameters, and feedback scores, thereby highlighting the collaborative control mechanism of this invention in high-concurrency scenarios (the above is used to describe the task orchestration principle and is not limited to a specific external chain framework implementation).

[0064] The process employs a directed flowchart to orchestrate processing nodes, route conditions, pass states, and provide closed-loop feedback. High-risk events are prioritized for rapid response, while ordinary events are handled through a standard response channel. Evaluation results are used to adjust subsequent routing thresholds. This process can be implemented using a software state machine or a combination of a rules engine and a message queue, without relying on any specific generic framework name.

[0065] 3) Floating object intent recognition and differentiated routing The master control agent routes floating object events to corresponding handling solutions through an intent classifier, achieving differentiated management of floating objects at four levels; the accuracy of classification can be evaluated based on training data, model configuration, and project test results.

[0066] 4) Specialized skills for dealing with floating objects in river channels; The system has a built-in module for specialized knowledge and skills related to river debris management, which includes at least the following reference knowledge documents: debris-types.md: Classification criteria, characteristic descriptions, and source identification for four levels of floating debris; cleanup-standards.md: Standards, equipment configurations, and safety requirements for cleaning up various floating debris; report-template.md: Standardized handling report template, supports automatic filling to generate compliant work orders.

[0067] Step S5: Call the professional knowledge base and attribute knowledge graph to perform enhanced retrieval generation, related reasoning, and disposal plan generation; This layer includes at least two parts: knowledge base and knowledge graph construction.

[0068] 1) Knowledge base construction This invention constructs a multi-category professional knowledge base for river management, including a basic river knowledge base, a floating debris knowledge base, a contingency plan knowledge base, and a historical case knowledge base.

[0069] Table 3 contains information related to the knowledge base:

[0070] Table 3 Document processing and segmentation; Multi-format parsing: Supports document formats such as PDF (including scanned OCR), Word, Excel and Markdown, and can use OCR and structured parsing components to extract text and table content; the table recognition effect can be determined based on document quality, layout complexity and project test results.

[0071] Hierarchical segmentation: Regulatory documents can be hierarchically segmented by chapter, section, and clause; general documents can be segmented by semantic units; and the availability of retrieval can be enhanced by combining question-and-answer pairs with automatic generation methods; the recall improvement can be determined based on the knowledge base content and test set results.

[0072] Metadata annotation: Each knowledge block can be appended with metadata such as knowledge base ID, file ID, river section range, floating object type, date, and priority to support precise filtering and fast retrieval; the specific retrieval latency can be determined based on the index configuration and deployment environment.

[0073] Incremental updates: Adding a new document can trigger an incremental index update, eliminating the need for a full rebuild every time; the update latency can be determined based on the document size and deployment environment.

[0074] After fine-tuning, the relevance and stability of the mixed search results can be further improved. The specific recall effect and the improvement compared to a single search method can be determined based on the test samples and the actual test results of the project.

[0075] Knowledge graph construction; The attribute knowledge graph construction technology is used to realize the automated organization and relational reasoning of river management knowledge. The technical path includes entity extraction, relation construction, graph indexing and retrieval enhancement application.

[0076] Table 4 contains information related to knowledge graphs:

[0077] Table 4 Three-channel high-efficiency retrieval; The coverage and association analysis capabilities of attribute graph retrieval are enhanced by using a three-channel parallel approach of local retrieval, global retrieval, and vector retrieval.

[0078] Attribute graph retrieval enhancement is achieved through a three-channel parallel approach: local association retrieval, global topic retrieval, and semantic nearest neighbor retrieval. Local association retrieval is used to capture details of entity neighborhoods, global topic retrieval is used to obtain trend information across river segments, and semantic nearest neighbor retrieval is used to supplement similar cases and rule entries. The results of the three channels are then fused using a unified scoring function.

[0079] Global Search: Searching based on topic cluster summaries in a graph structure, used for macro trend analysis and cross-domain association reasoning; Semantic nearest neighbor retrieval: Node representations are encoded and written into a vector index to support semantic nearest neighbor matching, and high-frequency key nodes are cached to optimize access efficiency; Status of business knowledge graph achievements; Table 5 contains information related to the business knowledge graph:

[0080] Table 5 The graph can construct an attribute relationship network around entities such as river channels, river sections, floating debris types, treatment equipment, treatment units, legal provisions, event work orders, and historical cases. Indicators such as graph size, number of relationships, average node degree, and embedding matching accuracy can be confirmed after the specific project is completed, based on the actual imported data and retrieval test results.

[0081] Step S6: Implement closed-loop management and digital twin simulation of the entire process of river floating debris management.

[0082] The closed-loop management layer enables system collaboration and full-process closed-loop management, and the system fully supports the "four predictions" (forecasting, early warning, rehearsal, and contingency plan) capabilities of water conservancy digital twins.

[0083] Table 6 contains relevant information on the system's comprehensive support for the "four predictions" of water conservancy digital twins:

[0084] Table 6 The entire process is a six-stage closed loop; Table 7 shows the relevant information for the six-stage closed loop of the entire process:

[0085] Table 7 Work order automation and tracking; Automated Work Orders: After the analysis is completed, standardized electronic work orders are automatically generated, including information such as event number, river section, GPS coordinates, pollution level, floating debris type, disposal elements, responsible person, deadline, and acceptance criteria; the specific generation time can be determined based on system configuration and project measurement results. Multiple distribution channels: Work orders can be distributed simultaneously through multiple channels such as APP push, SMS and web; the delay for the responsible person to receive and confirm can be determined according to the network environment and system deployment method; Real-time tracking: The person in charge can report the progress status such as arrival, in progress, and completion through the APP, and the drone can provide on-site video verification in real time; the status update delay can be determined according to the on-site network and equipment conditions; Timeout escalation: When a task is nearing its response deadline, it can be automatically expedited; if no response is received within the time limit, it can be automatically escalated to the superior supervisor to improve the efficiency of closed-loop management.

[0086] Intelligent evaluation algorithm for treatment effectiveness; Overall evaluation score: Visual appeal score × 0.5 + Timeliness score × 0.3 + Completeness score × 0.2 System observability; The system can be integrated with an observable platform to achieve full-link tracking of intelligent agents, including token consumption, processing latency and call cost of large model calls, RAG retrieval results, end-to-end time distribution of work order processing, and abnormal alarms and automatic degradation strategies.

[0087] Example 2: A high-concurrency urban river floating debris management system based on a layered architecture includes: a perception access layer, a layered identification layer, a data fusion layer, an intelligent agent collaboration layer, a knowledge support layer, and a control closed-loop layer.

[0088] The perception access layer acquires multimodal raw data based on sensing devices; the multimodal raw data includes video data and river data; the hierarchical recognition layer performs adaptive frame extraction on the video stream composed of the video data, based on the coupling of flow velocity constraints and scene complexity, and performs target detection, cross-frame tracking, and event discrimination on the image frames obtained after frame extraction, outputting floating object target information; the data fusion layer performs three-level fusion on the multimodal raw data and the floating object target information, namely data-level alignment, feature-level joint representation, and decision-level conflict resolution, and uniformly transforms coordinates from different sources to the same geographic coordinate system; the intelligent agent collaboration layer contains multiple intelligent agents, each of which shares event states through a state bus, transmits control permissions through task tokens, and works collaboratively according to preset rule constraints to achieve task distribution, result feedback, and closed-loop control; the knowledge support layer is used to store professional knowledge bases and attribute knowledge graphs, and is used to perform retrieval enhancement generation, correlation reasoning, and disposal scheme generation; the management and control closed-loop layer performs full-process closed-loop management and digital twin simulation of river floating object governance.

[0089] The modules work together to implement the high-concurrency urban river floating debris management method based on a hierarchical architecture in Example 1.

[0090] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A high-concurrency urban river floating debris management method based on a hierarchical architecture, characterized in that, Includes the following steps: Step S1: Based on the sensing device, acquire multimodal raw data; the multimodal raw data includes: video data and river data; Step S2: Based on the coupling of flow rate constraints and scene complexity, adaptive frame extraction is performed on the video stream composed of the video data; and lightweight feature extraction, candidate region confirmation, cross-frame tracking and event discrimination are performed on the image frames obtained after frame extraction within the field of view of a single camera, and floating object target information is output. Step S3: Perform three-level fusion on the multimodal raw data and the floating object target information, namely, data-level alignment, feature-level joint representation and decision-level conflict resolution, and unify the coordinates from different sources to the same geographic coordinate system; Step S4: Based on preset rules, each intelligent agent is driven to work collaboratively to perform identification, analysis, handling, and evaluation collaboratively; the intelligent agent is an autonomous collaborative unit that realizes the business functions of river management. Step S5: Invoke the professional knowledge base and attribute knowledge graph to perform enhanced retrieval generation, related reasoning, and processing solution generation; Step S6: Implement closed-loop management and digital twin simulation of the entire process of river floating debris management; The adaptive frame extraction in step S2 includes: Based on the river flow velocity in the river data and the floating object level in the current frame image of the video data, the equivalent velocity of the floating object is determined through a preset physical model; and the shortest dwell time of the floating object within the camera's field of view is calculated. The frame extraction interval is calculated based on the video frame rate, effective response time window, and dynamic adjustment items. The effective response time window is the smaller of the conservative correction value of the shortest dwell time and the alarm time limit value. The dynamic adjustment items include at least the multidimensional structural change index, the target density in the picture, the normalized value of the event risk level, and the environmental disturbance factor formed by the combination of picture jitter, water level fluctuation, wind and wave disturbance and sudden change in illumination. The multidimensional structural change index is determined based on the target spatial structure change, category distribution shift and sudden change in motion mode. The frame skipping interval is positively correlated with the product of the frame rate and the effective response time window, negatively correlated with the sum of the dynamic adjustment terms, and the minimum value is 1. Adaptive frame extraction is performed based on the frame extraction interval.

2. The method for managing floating debris in urban rivers based on a hierarchical architecture according to claim 1, characterized in that, The sensing devices include: fixed visual acquisition devices, mobile visual inspection devices, and hydrological parameter sensing devices.

3. The method for managing floating debris in urban rivers based on a hierarchical architecture according to claim 1, characterized in that, Also includes: When the multidimensional structural change index exceeds the preset adaptive threshold, the continuous frames are aggregated into event segments; otherwise, the lightweight tracking module is used.

4. The method for managing floating debris in urban rivers based on a hierarchical architecture according to claim 1, characterized in that, The adaptive frame extraction in step S2 also includes: Configure an upper and lower bound for the frame skipping interval so that the frame skipping interval is constrained between the lower and upper bounds. The lower and upper bound values ​​are configured in segments according to the river segment type.

5. A method for managing floating debris in urban rivers based on a hierarchical architecture according to claim 1, characterized in that, Also includes: The effective throughput is calculated for each river segment. The effective throughput is determined based on the number of video acquisition devices in the river segment, the video frame rate, the adaptive frame extraction interval, the scheduling efficiency correction coefficient, and the low-value event filtering rate. The effective throughput of each river section is weighted and summed to obtain the assessment result of the concurrent capacity of the entire river.

6. The method for managing floating debris in urban rivers based on a hierarchical architecture according to claim 3, characterized in that, Step S3 performs a three-level fusion of the multimodal raw data and the floating object target information, including data-level alignment, feature-level joint representation, and decision-level conflict resolution: Data-level fusion: Complete the temporal alignment, spatial alignment, and format standardization of video data and river data to form a unified basic data foundation; Feature-level fusion: The visual features of visual targets in video data are jointly encoded with the hydrological operational features in river data to generate a unified event feature vector for identification and analysis; visual features include category, area, color, trajectory, and drift speed; hydrological operational features include river flow velocity, water level, flow rate, and river section attributes; and an event token compression mechanism is used to compress the temporal target set and trajectory state set of events into structured events through an event aggregation function; Decision-level fusion: The confidence level of multi-source identification results, rule-based judgment results and collaborative judgment results are weighted, conflict is resolved and consistency is checked, and the final floating object category, risk level, location and disposal recommendations are output according to the fusion confidence level.

7. A method for managing floating debris in urban rivers based on a hierarchical architecture according to claim 1, characterized in that, Step S3 unifies the coordinates from different sources to the same geographic coordinate system by adopting a coordinate transformation link of "pixel coordinate system - device local coordinate system - river engineering coordinate system - WGS84 geographic coordinate system".

8. The method for managing floating debris in urban rivers based on a hierarchical architecture according to claim 1, characterized in that, Step S4 specifically includes: For each event fragment, calculate the probability of dynamic invocation; The dynamic call probability is the value obtained by mapping the weighted sum of information entropy, structural complexity, and business risk through the Sigmoid function. When the dynamic call probability exceeds a preset threshold, the multimodal large model is triggered to perform deep reasoning, and the deep reasoning result of the multimodal large model is used as the judgment conclusion. When the dynamic call probability does not exceed the preset threshold, the floating object target information is directly used as the judgment conclusion.

9. A high-concurrency urban river floating debris management system based on a hierarchical architecture, characterized in that, The method for implementing the high-concurrency urban river floating debris management method based on a hierarchical architecture as described in any one of claims 1-8 specifically includes: The perception access layer acquires multimodal raw data based on perception devices; the multimodal raw data includes video data and river data. The layered recognition layer performs adaptive frame extraction based on the coupling of flow rate constraints and scene complexity on the video stream composed of the video data, and performs target detection, cross-frame tracking and event discrimination on the image frames obtained after frame extraction, and outputs floating object target information. The data fusion layer performs three-level fusion on the multimodal raw data and the floating object target information: data-level alignment, feature-level joint representation, and decision-level conflict resolution, and uniformly transforms coordinates from different sources to the same geographic coordinate system. The agent collaboration layer contains multiple agents. Each agent shares event states through a state bus, transmits control permissions through task tokens, and works collaboratively according to preset rules to achieve task distribution, result feedback, and closed-loop control. The knowledge support layer is used to store professional knowledge bases and attribute knowledge graphs, and to perform retrieval enhancement generation, correlation reasoning and disposal solution generation. The closed-loop management layer implements full-process closed-loop management and digital twin simulation of river floating debris control.

Images