Traffic monitoring method and system based on video image analysis and processing

Abstract

The present invention relates to the technical field of water flow monitoring, and specifically discloses a flow monitoring method and system based on video image analysis and processing. The method includes: performing communication collection of image monitoring and water level monitoring at multiple monitoring and collection sites; performing video frame extraction and image preprocessing; feature point detection and displacement analysis; converting into water surface flow velocities at multiple sites, and performing fitting correction; calculating site flow data at multiple monitoring and collection sites. The present invention obtains water surface image data through image monitoring and water level monitoring communication collection at multiple monitoring and collection sites, combines video frame extraction and image preprocessing, calculates site pixel flow velocities through feature point detection and displacement analysis, converts to corrected water surface flow velocities through a spatial projection model, and calculates site flow data, thereby realizing non-contact full-process monitoring, eliminating the safety hazards of traditional methods that require equipment to be arranged in water, significantly reducing maintenance difficulty, and being particularly suitable for flow monitoring during flood seasons and dangerous waters.

Description

Technical Field

[0001] The present invention belongs to the technical field of water flow monitoring, and in particular relates to a flow monitoring method and system based on video image analysis and processing. Background Art

[0002] Water flow monitoring is a technical means of understanding the dynamic changes in water resources by measuring and recording the flow rate of water through a specific section in real time. It is a fundamental task in hydrology, water conservancy, water resources management, and environmental protection, and is widely used in rivers, lakes, reservoirs, channels, and drainage systems. The primary purpose of water flow monitoring is to obtain accurate water volume information, providing a scientific basis for flood control, water resource allocation, water ecological protection, and project operations.

[0003] In the existing technology, water flow monitoring usually requires placing a large number of equipment in the water to perform contact measurements on the water body, which poses safety hazards and difficult maintenance. It is not suitable for flood seasons or dangerous waters and is easily interfered with by sediment, floating objects, and turbid water quality, affecting the flow monitoring effect. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a flow monitoring method and system based on video image analysis and processing, aiming to solve the problems raised in the background technology.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0006] The flow monitoring method based on video image analysis and processing specifically comprises the following steps:

[0007] Determine a plurality of monitoring and collection sites, perform communication collection of image monitoring and water level monitoring at the plurality of monitoring and collection sites, and obtain image monitoring data and water level monitoring data;

[0008] Performing video frame extraction and image preprocessing on the image monitoring data to obtain water surface image data of a plurality of monitoring and collection sites;

[0009] Performing feature point detection and displacement analysis on the plurality of water surface image data, and calculating the site pixel flow velocities of the plurality of monitoring and collection sites;

[0010] Constructing a spatial projection model, converting the pixel flow velocities of the plurality of stations into water surface flow velocities of the plurality of stations through the spatial projection model, and performing fitting correction to obtain a plurality of corrected water surface flow velocities;

[0011] Combined with the multiple corrected water surface flow velocities and the multiple water level monitoring data, the site flow data of the multiple monitoring and collection sites are calculated and visualized.

[0012] A flow monitoring system based on video image analysis and processing, the system comprising:

[0013] A site monitoring and collection unit is used to determine multiple monitoring and collection sites, perform communication collection of image monitoring and water level monitoring at the multiple monitoring and collection sites, and obtain image monitoring data and water level monitoring data;

[0014] A video image processing unit, configured to perform video frame extraction and image preprocessing on the image monitoring data to obtain water surface image data of a plurality of monitoring and collection sites;

[0015] A pixel flow rate calculation unit, configured to perform feature point detection and displacement analysis on the plurality of water surface image data, and calculate the pixel flow rates of the plurality of monitoring and collection sites;

[0016] A flow rate fitting and correction unit is used to construct a spatial projection model, convert the pixel flow rates of the plurality of stations into water surface flow rates of the plurality of stations through the spatial projection model, and perform fitting and correction to obtain a plurality of corrected water surface flow rates;

[0017] The flow data display unit is used to combine the multiple corrected water surface flow velocities and the multiple water level monitoring data to calculate the site flow data of the multiple monitoring and collection sites and perform visual display.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] The present invention acquires water surface image data through image monitoring and water level monitoring communication collection at multiple monitoring and collection sites, combined with video frame extraction and image preprocessing. The pixel flow velocity at the site is then calculated through feature point detection and displacement analysis. Finally, the data is converted into a corrected water surface flow velocity and the site flow data is calculated through a spatial projection model. This realizes non-contact full-process monitoring, completely eliminates the safety hazards of traditional methods that require equipment to be arranged in water, significantly reduces the difficulty of maintenance, and is particularly suitable for flow monitoring during flood seasons and dangerous waters.

[0020] The present invention constructs a multi-directional water pattern fingerprint map and an anti-interference multispectral image to eliminate interference, combines buoy measurement data to generate virtual-real fusion feature points, and uses the river flow field topology map to verify the direction of movement. It effectively overcomes the interference effects of sediment, floating objects and turbid water quality, so that the feature point detection remains stable in extreme environments, ensures the accuracy of the site pixel flow velocity calculation, and solves the key defect of traditional contact measurement that is easily affected by water quality interference.

[0021] The present invention realizes dynamic adjustment of the adaptive detection domain through the water level-texture fusion layer, combines the spectral weight configuration table and the spatiotemporal joint probability model to correct the flow data, and finally completes the multi-dimensional display on the visualization platform, creating a full-domain adaptability during flood and dry seasons, so that the site-time flow calculation can not only capture the instantaneous flood peak changes but also reflect the long-term hydrological laws, providing unprecedented decision-making support accuracy for flood control scheduling and water resources management. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention.

[0023] Figure 1 A flow chart of a method provided by an embodiment of the present invention is shown.

[0024] Figure 2 A flow chart of communication collection for image monitoring and water level monitoring in the method provided by an embodiment of the present invention is shown.

[0025] Figure 3 A flowchart of video frame extraction and image preprocessing in the method provided by an embodiment of the present invention is shown.

[0026] Figure 4 A flow chart showing feature point detection and displacement analysis in the method provided by an embodiment of the present invention is shown.

[0027] Figure 5 The application architecture diagram of the system provided by the embodiment of the present invention is shown.

[0028] Figure 6 The structure block diagram of the site monitoring and collection unit in the system provided by the embodiment of the present invention is shown.

[0029] Figure 7 The structure block diagram of the video image processing unit in the system provided by the embodiment of the present invention is shown.

[0030] Figure 8 The figure shows a structural block diagram of a pixel flow rate calculation unit in a system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0032] It is understandable that in the existing technology, for water flow monitoring, it is usually necessary to arrange a large number of equipment in the water and perform contact measurement of the water body, which poses safety hazards and maintenance difficulties. It is not suitable for flood seasons or dangerous waters, and is easily interfered by sediment, floating objects, and turbid water quality, affecting the flow monitoring effect.

[0033] To solve the above problems, an embodiment of the present invention determines multiple monitoring and collection sites, performs communication collection of image monitoring and water level monitoring at the multiple monitoring and collection sites, and obtains image monitoring data and water level monitoring data; performs video frame extraction and image preprocessing on the image monitoring data to obtain water surface image data of the multiple monitoring and collection sites; performs feature point detection and displacement analysis on the multiple water surface image data to calculate the site pixel flow velocities of the multiple monitoring and collection sites; constructs a spatial projection model, and converts the multiple site pixel flow velocities into multiple site water surface flow velocities through the spatial projection model, and performs fitting correction to obtain multiple corrected water surface flow velocities; combines the multiple corrected water surface flow velocities and the multiple water level monitoring data to calculate the site flow data of the multiple monitoring and collection sites and visualize them.

[0034] Figure 1 A flow chart of a method provided by an embodiment of the present invention is shown.

[0035] Specifically, the traffic monitoring method based on video image analysis and processing includes the following steps:

[0036] Step S101 : determining a plurality of monitoring and collection sites, performing communication collection of image monitoring and water level monitoring at the plurality of monitoring and collection sites, and acquiring image monitoring data and water level monitoring data.

[0037] In an embodiment of the present invention, by determining multiple monitoring and collection sites and performing communication function analysis on the multiple monitoring and collection sites, multiple relay collection sites are selected from the multiple monitoring and collection sites, and multiple monitoring and collection instructions are generated according to multiple preset monitoring parameter data (including: monitoring angle, shooting frame rate, shooting resolution, collection interval, etc.). The multiple monitoring and collection instructions are then sent to the multiple relay collection sites, so that the multiple relay collection sites can perform monitoring and communication transmission. Through the multiple relay collection sites, communication collection and recording of image monitoring are performed at the multiple monitoring and collection sites to obtain image monitoring data. Moreover, through the multiple relay collection sites, communication collection and recording of water level monitoring are performed at the multiple monitoring and collection sites to obtain water level monitoring data.

[0038] Specifically, Figure 2 A flow chart of communication collection for image monitoring and water level monitoring in the method provided by an embodiment of the present invention is shown.

[0039] In a preferred embodiment of the present invention, determining a plurality of monitoring and collection sites, performing communication collection of image monitoring and water level monitoring at the plurality of monitoring and collection sites, and obtaining image monitoring data and water level monitoring data specifically include the following steps:

[0040] Step S1011, determining multiple monitoring and collection sites;

[0041] Step S1012, selecting multiple relay collection sites from the multiple monitoring collection sites;

[0042] Step S1013, generating and sending monitoring and collection instructions to the plurality of relay collection sites according to the plurality of preset monitoring parameter data;

[0043] Step S1014, performing communication collection and recording of image monitoring at multiple monitoring collection sites through multiple relay collection sites to obtain image monitoring data;

[0044] Step S1015 , performing communication collection and recording of water level monitoring at multiple monitoring and collection sites through multiple relay collection sites to obtain water level monitoring data.

[0045] Furthermore, the flow monitoring method based on video image analysis and processing also includes the following steps:

[0046] Step S102 : performing video frame extraction and image preprocessing on the image monitoring data to obtain water surface image data of the plurality of monitoring and collection sites.

[0047] In an embodiment of the present invention, video frame extraction is performed on image monitoring data to obtain multiple frame-by-frame image data, and then the multiple frame-by-frame image data are enhanced (defogging, contrast enhancement, edge sharpening, etc.) to obtain multiple enhanced image data. Thereafter, offset elimination is performed on the multiple enhanced image data (eliminating the screen offset caused by device jitter) to obtain multiple offset-eliminated image data. From the multiple optimized image data, multiple basic water surface data are extracted (dynamic water surface areas are extracted using a Gaussian mixture model or inter-frame difference), and then interference elimination is performed on the multiple basic water surface data (interference factors such as floating objects and reflections are identified and eliminated) to obtain water surface image data corresponding to multiple monitoring and collection sites.

[0048] Specifically, Figure 3 A flowchart of video frame extraction and image preprocessing in the method provided by an embodiment of the present invention is shown.

[0049] Among them, in the preferred embodiment provided by the present invention, the video frame extraction and image preprocessing of the image monitoring data to obtain the water surface image data of multiple monitoring and collection sites specifically include the following steps:

[0050] Step S1021, extracting video frames from the image monitoring data to obtain a plurality of frame-by-frame image data;

[0051] Step S1022, performing enhancement processing on the plurality of frame-by-frame image data to obtain a plurality of enhanced image data;

[0052] Step S1023, performing offset elimination on the plurality of enhanced image data to obtain a plurality of offset eliminated image data;

[0053] Step S1024, extracting a plurality of basic water surface data from the plurality of offset-eliminated image data;

[0054] Step S1025 , performing interference elimination on the plurality of the basic water surface data to obtain the water surface image data of the plurality of the monitoring and collection sites.

[0055] In a preferred embodiment of the present invention, the step of removing interference from the plurality of basic water surface data to obtain the water surface image data of the plurality of monitoring and collection sites specifically includes the following steps:

[0056] Step S10251, extracting the HSV color space brightness channel of the enhanced image data to generate a water surface brightness fluctuation map;

[0057] Step S10252: performing multi-scale Gabor filtering on the offset-eliminated image data to extract watermark texture features in different directions to construct a multi-directional watermark fingerprint map;

[0058] Step S10253, performing a multi-frame differential operation on the offset-eliminated image data, marking the moving areas of consecutive frames, and generating a moving area marking map;

[0059] Step S10254: Based on the water surface brightness fluctuation map and the moving area label map, a spatiotemporal joint probability model is established, and interference confidence is calculated using the spatiotemporal joint probability model to generate an interference heat map;

[0060] Step S10255, performing a morphological opening operation on the basic water surface data to segment independent connected domains and form a water surface connected domain segmentation graph;

[0061] Step S10256, performing feature matching on the multi-directional watermark fingerprint map and the water surface connected domain segmentation map, identifying abnormal texture patterns, and obtaining abnormal texture areas;

[0062] Step S10257, performing Bayesian fusion on the interference heat map and the texture abnormality area to eliminate the falsely detected interference area and generate an accurate interference map;

[0063] Step S10258: determine and obtain the multi-scale structure of the surrounding clean water surface, extract candidate interference regions based on the precise interference map, and calculate the similarity between the extracted candidate interference regions and the multi-scale structure of the surrounding clean water surface to obtain the true interference region;

[0064] Step S10259: creating a direction-aware repair template in the real interference area according to the topological flow direction of the multi-directional watermark fingerprint map to generate an anisotropic repair template;

[0065] Step S102510, using the texture features of the adjacent clean water surface in the enhanced image data, performing texture migration and restoration on the real interference area to generate restored water surface data;

[0066] Step S102511: perform optical flow-driven fusion on the anisotropic restoration template and the restored water surface data to maintain the continuity of the water flow and generate motion-consistent restoration data;

[0067] Step S102512: performing turbulence feature enhancement on the motion-consistent repair data to enhance the visual significance of vortex and jet structures, thereby generating turbulence-enhanced repair data;

[0068] In step S102513, the turbulence enhanced repair data is fused with the basic water surface data to generate a complete water surface image, and the complete water surface image is subjected to adaptive gamma correction to unify the lighting conditions, and finally the water surface image data of the plurality of monitoring and collection sites are obtained.

[0069] Furthermore, the flow monitoring method based on video image analysis and processing also includes the following steps:

[0070] Step S103 , performing feature point detection and displacement analysis on the plurality of water surface image data, and calculating the site pixel flow velocities of the plurality of monitoring and collection sites.

[0071] In an embodiment of the present invention, feature point detection is performed on multiple water surface image data to determine multiple water surface feature points (including feature points such as water surface bright spots and texture features). Then, an optical flow method or a particle tracking method is used to perform feature matching of adjacent frame images for the multiple water surface feature points. The displacements of the multiple feature points are recorded, and the inter-frame interval time of the multiple water surface image data is obtained. Then, according to the displacements of the multiple feature points and the corresponding inter-frame interval time, the site pixel flow rate of the multiple monitoring and collection sites is calculated.

[0072] Specifically, Figure 4 A flow chart showing feature point detection and displacement analysis in the method provided by an embodiment of the present invention is shown.

[0073] In a preferred embodiment of the present invention, the performing of feature point detection and displacement analysis on the plurality of water surface image data and the calculation of the pixel flow rates of the plurality of monitoring and collection sites specifically include the following steps:

[0074] Step S1031, performing feature point detection on the plurality of water surface image data to determine a plurality of water surface feature points;

[0075] Step S1032, using an optical flow method or a particle tracking method, performing feature matching of adjacent frame images on the plurality of water surface feature points, and recording the displacements of the plurality of feature points;

[0076] Step S1033, obtaining the inter-frame interval time of a plurality of the water surface image data;

[0077] Step S1034 : calculating the site pixel flow rates of the plurality of monitoring and collection sites according to the plurality of feature point displacements and the corresponding inter-frame intervals.

[0078] In a preferred embodiment of the present invention, the step of performing feature point detection on the plurality of water surface image data to determine the plurality of water surface feature points specifically includes the following steps:

[0079] Step S10311: Perform height mapping on the water level monitoring data and overlay the fluctuation pattern of the basic water surface data to generate a water level-texture fusion layer. The water level-texture fusion layer is then used to intelligently shrink the monitoring area to generate an adaptive detection domain. The intelligent shrinking of the monitoring area includes focusing on the center of the river during high water levels, avoiding floating objects on the shore, and expanding to the marginal shallows during low water levels.

[0080] Step S10312: Determine and obtain the buoy measurement data, analyze the motion trajectory of the buoy measurement data, generate virtual feature points at equal intervals on the trajectory line, and reversely map the positions of the virtual feature points to lock the actual motion direction of the water surface to obtain a set of virtual feature points guided by the buoy;

[0081] Step S10313: parse the flow velocity values ​​in the buoy measurement data, configure spectral channel weights based on the turbidity state, generate a spectral weight configuration table, apply the spectral weight configuration table to the enhanced image data to perform channel reorganization to obtain a pseudo near-infrared layer, establish an interference fingerprint library based on the basic water surface data, offset-eliminated image data, buoy measurement data, and water level monitoring data, fuse the enhanced image data and the pseudo near-infrared layer to obtain a fused image, use the interference fingerprint library to detect and remove matching areas in the fused image, and obtain an interference-resistant multispectral image;

[0082] Step S10314: Based on the adaptive detection domain, physical feature point monitoring is performed to obtain the detected physical point positions, and the virtual feature point set guided by the buoy is coupled with the detected physical point positions to generate virtual-real fusion feature points;

[0083] Step S10315: extract the generated water level-texture fusion layer, construct a river flow field topology map, and use the river flow field topology map to verify the movement direction of the virtual-real fusion feature points, eliminate floating objects on the water surface, and generate a topology verification feature point set;

[0084] Step S10316: Track the topology verification feature point set in the anti-interference multispectral image, retain key points that persist and whose displacement variance is lower than a preset threshold, and obtain stable water surface feature points;

[0085] Step S10317: Determine multiple water surface feature points based on the stable water surface feature points.

[0086] In a preferred embodiment of the present invention, calculating the pixel flow rates of the plurality of monitoring and collection sites according to the displacements of the plurality of feature points and the corresponding inter-frame intervals specifically includes the following steps:

[0087] Step S10341, using the spatial distribution data of the stable water surface feature points, constructing a feature point motion trajectory topology network to generate an initial trajectory network;

[0088] Step S10342: Based on the water surface curvature characteristics of the water level-texture fusion layer, the initial trajectory network is subjected to curvature-driven resampling to generate a curvature optimized network;

[0089] Step S10343, based on the motion trajectory of the buoy measurement data, performing motion consistency correction on the curvature optimization network to generate a correction trajectory network;

[0090] Step S10344: extracting the mainstream belt vector field based on the river flow field topology map, and performing tensor fusion on the correction trajectory network and the mainstream belt vector field to generate a tensor fusion trajectory;

[0091] Step S10345: extracting the water ripple flow direction field based on the anti-interference multispectral image, and projecting the tensor fusion trajectory into the water ripple flow direction field for directional filtering to generate directional filtering displacement;

[0092] Step S10346, constructing a displacement reliability evaluation matrix based on the confidence weights of the virtual-real fusion feature points to generate a displacement weight matrix;

[0093] Step S10347, verifying the motion continuity of the feature point set based on the topology, performing time series smoothing processing on the directional filter displacement, and generating a smoothed displacement sequence;

[0094] Step S10348: performing weighted fusion on the displacement weight matrix and the smoothed displacement sequence to eliminate low-confidence displacement points and generate a weighted displacement field;

[0095] Step S10349, converting the weighted displacement field into an instantaneous velocity field in combination with the inter-frame interval time;

[0096] Step S103410: establishing a velocity variation coefficient map based on the adaptive detection domain, identifying turbulence anomaly areas, and generating a turbulence anomaly map;

[0097] Step S103411, identifying and removing turbulence outliers in the instantaneous velocity field according to the turbulence anomaly map, retaining the laminar motion component, and generating a laminar velocity field;

[0098] Step S103412, performing spatial correlation correction on the laminar flow velocity field and the reference velocity of the virtual feature point set guided by the buoy to generate a corrected flow velocity field;

[0099] Step S103413, spatially integrate the corrected velocity field, calculate the average pixel velocity of each monitoring and collection site, and obtain the preliminary site pixel velocity;

[0100] Step S103414: performing spectral sensitivity compensation on the preliminary site pixel flow velocity based on the spectral weight configuration table, and finally calculating the site pixel flow velocity of the plurality of monitoring and collection sites.

[0101] Furthermore, the flow monitoring method based on video image analysis and processing also includes the following steps:

[0102] Step S104 : constructing a spatial projection model, converting the pixel flow velocities of the plurality of stations into water surface flow velocities of the plurality of stations through the spatial projection model, and performing fitting correction to obtain a plurality of corrected water surface flow velocities.

[0103] In an embodiment of the present invention, a spatial projection model from image coordinates to world coordinates is constructed, and then the pixel flow velocities of multiple stations are converted into water surface flow velocities of multiple stations through the spatial projection model. Then, the water surface flow velocities of multiple stations are fitted and corrected through buoy measurement data to obtain multiple corrected water surface flow velocities.

[0104] Among them, in the preferred embodiment provided by the present invention, the construction of the spatial projection model, converting the pixel flow velocities of the plurality of stations into the water surface flow velocities of the plurality of stations through the spatial projection model, and performing fitting correction to obtain the plurality of corrected water surface flow velocities specifically comprises the following steps:

[0105] Step S1041, constructing a spatial projection model from image coordinates to world coordinates;

[0106] Step S1042, converting the pixel flow velocities of the plurality of stations into water surface flow velocities of the plurality of stations using the spatial projection model;

[0107] Step S1043: performing fitting correction on the water surface flow velocities at the plurality of stations using the buoy measurement data to obtain a plurality of corrected water surface flow velocities.

[0108] Furthermore, the flow monitoring method based on video image analysis and processing also includes the following steps:

[0109] Step S105 , combining the multiple corrected water surface flow velocities and the multiple water level monitoring data, calculating the site flow data of the multiple monitoring and collection sites and visually displaying them.

[0110] In an embodiment of the present invention, multiple corrected water surface flow rates and multiple water level monitoring data are combined to calculate the site section flow rates of multiple monitoring and collection sites, and based on the multiple site section flow rates, multiple site time period flow rates are calculated, and the multiple site section flow rates and multiple site time period flow rates are comprehensively sorted to obtain site flow data, and then the site flow data is visualized in a preset visualization platform.

[0111] In a preferred embodiment of the present invention, the combining of the plurality of corrected water surface flow velocities and the plurality of water level monitoring data to calculate the site flow data of the plurality of monitoring and collection sites and to perform a visual display specifically includes the following steps:

[0112] Step S1051, combining the multiple corrected water surface flow rates and the multiple water level monitoring data to calculate the site cross-sectional flow rates of the multiple monitoring and collection sites;

[0113] Step S1052, establishing a timestamp matrix of site section flows according to the plurality of site section flows, to generate a timestamp flow matrix;

[0114] Step S1053, extracting the water level change curve of the water level monitoring data, calculating the water level change rate per unit time, and generating a water level change rate curve;

[0115] Step S1054, converting the water level change rate curve into a flow change impact factor to generate a water level-flow impact factor;

[0116] Step S1055 , performing a convolution operation on the water level-flow impact factor and the timestamp flow matrix to generate a water level correction flow matrix;

[0117] Step S1056, obtaining the motion acceleration value of the buoy measurement data to obtain a water flow acceleration sequence;

[0118] Step S1057, dynamically weighting and fusing the water flow acceleration sequence with the water level correction flow matrix to generate the acceleration correction flow;

[0119] Step S1058, extracting the traffic change pattern of the same period in history from the visualization platform to generate a historical traffic pattern library;

[0120] Step S1059, performing similarity matching between the acceleration-corrected flow rate and the historical flow pattern library to generate pattern-matched flow rate;

[0121] Step S10510: Based on the spatial distribution of the modified water surface velocity, establish a flow transfer relationship between sites and generate a watershed flow transfer network;

[0122] Step S10511, performing spatial consistency optimization on the pattern matching flow through the watershed flow transfer network to generate spatially optimized flow;

[0123] Step S10512, analyzing the light intensity change curve of the image monitoring data to generate a circadian rhythm coefficient;

[0124] Step S10513, applying the circadian rhythm coefficient to adjust the spatial optimized flow to generate a circadian regulated flow;

[0125] Step S10514, extracting thermal infrared channel data from the anti-interference multispectral image to obtain a water surface temperature distribution map;

[0126] Step S10515, calculating the evaporation loss compensation value based on the water surface temperature distribution map and the water level-flow influencing factor to generate the evaporation compensation value;

[0127] Step S10516, superimposing the diurnal and night-time regulated flow rate and the evaporation compensation value to calculate the flow rate for multiple stations during the period;

[0128] Step S10517, comprehensively sorting out the cross-sectional flow rates of the plurality of sites and the time period flow rates of the plurality of sites to obtain site flow data;

[0129] Step S10518: Visually display the site traffic data on a preset visualization platform.

[0130] Further, Figure 5 The application architecture diagram of the system provided by the embodiment of the present invention is shown.

[0131] In another preferred embodiment of the present invention, a flow monitoring system based on video image analysis and processing includes:

[0132] The site monitoring and collection unit 101 is used to determine multiple monitoring and collection sites, perform communication collection of image monitoring and water level monitoring at the multiple monitoring and collection sites, and obtain image monitoring data and water level monitoring data.

[0133] In an embodiment of the present invention, the site monitoring and collection unit 101 determines multiple monitoring and collection sites and performs communication function analysis on the multiple monitoring and collection sites. From the multiple monitoring and collection sites, multiple relay collection sites are selected, and multiple monitoring and collection instructions are generated according to multiple preset monitoring parameter data (including: monitoring angle, shooting frame rate, shooting resolution, collection interval, etc.). The multiple monitoring and collection instructions are then sent to the multiple relay collection sites, so that the multiple relay collection sites can perform monitoring and communication transmission. Through the multiple relay collection sites, communication collection and recording of image monitoring are performed at the multiple monitoring and collection sites to obtain image monitoring data. Communication collection and recording of water level monitoring are performed at the multiple monitoring and collection sites through the multiple relay collection sites to obtain water level monitoring data.

[0134] Specifically, Figure 6 FIG. 1 shows a structural block diagram of the site monitoring and collection unit 101 in the system provided by an embodiment of the present invention.

[0135] In a preferred embodiment of the present invention, the site monitoring and collection unit 101 specifically includes:

[0136] A site determination module 1011 is used to determine multiple monitoring and collection sites;

[0137] The relay selection module 1012 is used to select multiple relay collection sites from the multiple monitoring collection sites;

[0138] The instruction sending module 1013 is used to generate and send monitoring and collection instructions to the plurality of relay collection sites according to a plurality of preset monitoring parameter data;

[0139] The image monitoring and acquisition module 1014 is used to acquire image monitoring data by communicating, collecting, and recording image monitoring data at multiple monitoring and acquisition sites through multiple relay acquisition sites.

[0140] The water level monitoring and collection module 1015 is used to collect and record water level monitoring data at multiple monitoring and collection sites through multiple relay collection sites to obtain water level monitoring data.

[0141] Furthermore, the traffic monitoring system based on video image analysis and processing also includes:

[0142] The video image processing unit 102 is used to perform video frame extraction and image preprocessing on the image monitoring data to obtain water surface image data of multiple monitoring and collection sites.

[0143] In an embodiment of the present invention, the video image processing unit 102 extracts video frames from the image monitoring data to obtain a plurality of frame-by-frame image data, and then performs enhancement processing (defogging, contrast enhancement, edge sharpening, etc.) on the plurality of frame-by-frame image data to obtain a plurality of enhanced image data. Thereafter, the plurality of enhanced image data are subjected to offset elimination (eliminating the screen offset caused by device jitter) to obtain a plurality of offset eliminated image data. From the plurality of optimized image data, a plurality of basic water surface data are extracted (dynamic water surface areas are extracted using a Gaussian mixture model or inter-frame difference), and then interference elimination is performed on the plurality of basic water surface data (interference factors such as floating objects and reflections are identified and eliminated) to obtain water surface image data corresponding to a plurality of monitoring and collection sites.

[0144] Specifically, Figure 7 FIG. 1 shows a structural block diagram of the video image processing unit 102 in the system provided by an embodiment of the present invention.

[0145] In a preferred embodiment of the present invention, the video image processing unit 102 specifically includes:

[0146] The video frame extraction module 1021 is used to extract video frames from the image monitoring data to obtain a plurality of frame-by-frame image data;

[0147] An enhancement processing module 1022 is configured to perform enhancement processing on the plurality of frame-by-frame image data to obtain a plurality of enhanced image data;

[0148] An offset elimination module 1023 is configured to perform offset elimination on the plurality of enhanced image data to obtain a plurality of offset eliminated image data;

[0149] A basic water surface data extraction module 1024 is configured to extract a plurality of basic water surface data from the plurality of optimized image data;

[0150] The interference removal module 1025 is used to remove interference from the multiple basic water surface data to obtain the water surface image data of the multiple monitoring and collection sites.

[0151] Furthermore, the traffic monitoring system based on video image analysis and processing also includes:

[0152] The pixel flow rate calculation unit 103 is used to perform feature point detection and displacement analysis on the plurality of water surface image data, and calculate the pixel flow rates of the plurality of monitoring and collection sites.

[0153] In an embodiment of the present invention, the pixel flow rate calculation unit 103 determines multiple water surface feature points (including feature points such as water surface bright spots and texture features) by performing feature point detection on multiple water surface image data, and then uses the optical flow method or the particle tracking method to perform feature matching of adjacent frame images for the multiple water surface feature points, records the displacement of multiple feature points, and obtains the inter-frame interval time of multiple water surface image data, and then calculates the site pixel flow rate of multiple monitoring and collection sites according to the multiple feature point displacements and the corresponding inter-frame interval time.

[0154] Specifically, Figure 8 FIG. 1 shows a structural block diagram of the pixel flow rate calculation unit 103 in the system provided by an embodiment of the present invention.

[0155] In a preferred embodiment of the present invention, the pixel flow rate calculation unit 103 specifically includes:

[0156] A feature point detection module 1031 is configured to perform feature point detection on the plurality of water surface image data to determine a plurality of water surface feature points;

[0157] A feature matching module 1032 is configured to perform feature matching of adjacent frames of the water surface feature points using an optical flow method or a particle tracking method, and record displacements of the feature points.

[0158] An interval time acquisition module 1033 is used to acquire the interval time between frames of the plurality of water surface image data;

[0159] The pixel flow rate calculation module 1034 is used to calculate the site pixel flow rates of the multiple monitoring and collection sites according to the multiple feature point displacements and the corresponding inter-frame interval time.

[0160] Furthermore, the traffic monitoring system based on video image analysis and processing also includes:

[0161] The flow rate fitting correction unit 104 is used to construct a spatial projection model, convert the pixel flow rates of the plurality of stations into water surface flow rates of a plurality of stations through the spatial projection model, and perform fitting correction to obtain a plurality of corrected water surface flow rates.

[0162] In an embodiment of the present invention, the flow velocity fitting and correction unit 104 constructs a spatial projection model from image coordinates to world coordinates, and then converts the pixel flow velocities of multiple sites into water surface flow velocities of multiple sites through the spatial projection model, and then fits and corrects the water surface flow velocities of multiple sites through buoy measurement data to obtain multiple corrected water surface flow velocities.

[0163] The flow data display unit 105 is used to combine the multiple corrected water surface flow velocities and the multiple water level monitoring data to calculate the site flow data of the multiple monitoring and collection sites and perform visual display.

[0164] In an embodiment of the present invention, the flow data display unit 105 combines multiple corrected water surface flow rates and multiple water level monitoring data to calculate the site section flow of multiple monitoring and collection sites, and calculates multiple site time period flows based on the multiple site section flows, and comprehensively organizes the multiple site section flows and the multiple site time period flows to obtain site flow data, and then visualizes the site flow data in a preset visualization platform.

[0165] It should be understood that, although the various steps in the flow chart of each embodiment of the present invention are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in order, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0166] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When executed, the program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0167] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0168] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

[0169] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A flow monitoring method based on video image analysis and processing, characterized in that: The method specifically comprises the following steps: Determine a plurality of monitoring and collection sites, perform communication collection of image monitoring and water level monitoring at the plurality of monitoring and collection sites, and obtain image monitoring data and water level monitoring data; Performing video frame extraction and image preprocessing on the image monitoring data to obtain water surface image data of a plurality of monitoring and collection sites; Performing feature point detection and displacement analysis on the plurality of water surface image data, and calculating the site pixel flow velocities of the plurality of monitoring and collection sites; Constructing a spatial projection model, converting the pixel flow velocities of the plurality of stations into water surface flow velocities of the plurality of stations through the spatial projection model, and performing fitting correction to obtain a plurality of corrected water surface flow velocities; Combining the plurality of corrected water surface flow velocities and the plurality of water level monitoring data, calculating the site flow data of the plurality of monitoring and collection sites and performing a visual display; The video frame extraction and image preprocessing of the image monitoring data to obtain the water surface image data of the plurality of monitoring and collection sites specifically include the following steps: Extracting video frames from the image monitoring data to obtain a plurality of frame-by-frame image data; Performing enhancement processing on the plurality of frame-by-frame image data to obtain a plurality of enhanced image data; performing offset elimination on the plurality of enhanced image data to obtain a plurality of offset eliminated image data; extracting a plurality of basic water surface data from the plurality of offset-eliminated image data; Eliminating interference from the plurality of basic water surface data to obtain water surface image data from the plurality of monitoring and collection sites; Eliminating interference from the plurality of basic water surface data to obtain the water surface image data of the plurality of monitoring and collection sites specifically comprises the following steps: Extracting the HSV color space brightness channel of the enhanced image data to generate a water surface brightness fluctuation map; Applying multi-scale Gabor filtering to the offset-eliminated image data to extract watermark texture features in different directions to construct a multi-directional watermark fingerprint map; performing a multi-frame differential operation on the offset-eliminated image data, marking the moving areas of consecutive frames, and generating a moving area marking map; Based on the water surface brightness fluctuation map and the moving area marking map, a spatiotemporal joint probability model is established, and the spatiotemporal joint probability model is used to calculate the interference confidence to generate an interference heat map; Performing a morphological opening operation on the basic water surface data to segment independent connected domains and form a water surface connected domain segmentation map; Perform feature matching on the multi-directional watermark fingerprint map and the water surface connected domain segmentation map to identify abnormal texture patterns and obtain abnormal texture areas; The interference heat map is Bayesian fused with the texture abnormality area to eliminate the false detection interference area and generate an accurate interference map; Determine and obtain the multi-scale structure of the surrounding clean water surface, extract candidate interference areas based on the precise interference map, and calculate the similarity between the extracted candidate interference areas and the multi-scale structure of the surrounding clean water surface to obtain the true interference area; According to the topological flow direction of the multi-directional watermark fingerprint, a direction-aware repair template is created in the real interference area to generate an anisotropic repair template; Using the texture features of the adjacent clean water surface in the enhanced image data, texture migration and restoration are performed on the real interference area to generate restored water surface data; The anisotropic repair template is optically flow-driven fused with the repaired water surface data to maintain the continuity of water flow and generate motion-consistent repair data. Perform turbulence feature enhancement on the motion-consistent repair data to enhance the visual significance of vortex and jet structures and generate turbulence-enhanced repair data; Fusing the turbulence enhancement repair data with the basic water surface data to generate a complete water surface image, and performing adaptive gamma correction on the complete water surface image to unify the lighting conditions, ultimately obtaining water surface image data from multiple monitoring and collection sites; The performing of feature point detection and displacement analysis on the plurality of water surface image data and calculating the pixel flow rates of the plurality of monitoring and collection sites specifically comprises the following steps: Performing feature point detection on the plurality of water surface image data to determine a plurality of water surface feature points; Using an optical flow method or a particle tracking method, feature matching is performed on the adjacent frame images of the plurality of water surface feature points, and displacements of the plurality of feature points are recorded; Acquire a plurality of inter-frame intervals of the water surface image data; Calculating the pixel flow rates of the plurality of monitoring and collection sites according to the displacements of the plurality of feature points and the corresponding inter-frame intervals; The detecting of feature points on the plurality of water surface image data to determine the plurality of water surface feature points specifically comprises the following steps: Water level monitoring data is height-mapped and overlaid with the fluctuation patterns of the underlying water surface data to generate a water level-texture fusion layer. This layer is then used to intelligently shrink the monitoring area to create an adaptive detection domain. This intelligent shrinkage of the monitoring area involves focusing on the center of the river during high water levels to avoid floating debris on the shore, and expanding to the marginal shallows during low water levels. Determine and obtain the buoy measurement data, analyze the motion trajectory of the buoy measurement data, generate virtual feature points at equal intervals on the motion trajectory, and reversely map the positions of the virtual feature points to lock the actual motion direction of the water surface and obtain the virtual feature point set guided by the buoy; The flow velocity values ​​in the buoy measurement data are parsed, and the spectral channel weights are configured according to the turbidity state to generate a spectral weight configuration table. The spectral weight configuration table is applied to the enhanced image data to perform channel reorganization to obtain a pseudo near-infrared layer. An interference fingerprint library is established based on the basic water surface data, offset-eliminated image data, buoy measurement data, and water level monitoring data. The enhanced image data and the pseudo near-infrared layer are fused to obtain a fused image. The interference fingerprint library is used to detect and eliminate matching areas in the fused image to obtain an interference-resistant multispectral image. Based on the adaptive detection domain, physical feature point monitoring is performed to obtain the detected physical point positions, and the virtual feature point set guided by the buoy is coupled with the detected physical point positions to generate virtual-real fusion feature points; Extract the generated water level-texture fusion layer, construct the river flow field topology map, and use the river flow field topology map to verify the movement direction of the virtual and real fusion feature points, eliminate floating objects on the water surface, and generate a topology verification feature point set; The topology verification feature point set is tracked in the anti-interference multispectral image, and the key points that persist and whose displacement variance is lower than the preset threshold are retained to obtain stable water surface feature points; determining a plurality of water surface feature points based on the stable water surface feature points; Calculating the pixel flow rates of the plurality of monitoring and collection sites according to the plurality of feature point displacements and the corresponding inter-frame intervals specifically includes the following steps: Using the spatial distribution data of the stable water surface feature points, a topological network of feature point motion trajectories is constructed to generate an initial trajectory network; Based on the water surface curvature characteristics of the water level-texture fusion layer, the initial trajectory network is curvature-driven resampling to generate a curvature optimized network; Based on the motion trajectory of the buoy measurement data, the curvature optimization network is corrected for motion consistency to generate a correction trajectory network; The mainstream belt vector field is extracted based on the river flow field topology map, and the correction trajectory network is tensor-fused with the mainstream belt vector field to generate a tensor-fused trajectory; The water ripple flow field is extracted based on the anti-interference multispectral image, and the tensor fusion trajectory is projected into the water ripple flow field for directional filtering to generate directional filtering displacement. According to the confidence weights of the virtual-real fusion feature points, a displacement reliability evaluation matrix is ​​constructed to generate a displacement weight matrix; Verify the motion continuity of the feature point set based on the topology, perform time series smoothing on the directional filter displacement, and generate a smooth displacement sequence; The displacement weight matrix is ​​weightedly fused with the smooth displacement sequence to eliminate low-confidence displacement points and generate a weighted displacement field. The weighted displacement field is converted into the instantaneous velocity field by combining the inter-frame interval time; Based on the adaptive detection domain, a velocity variation coefficient map is established to identify turbulence anomaly areas and generate a turbulence anomaly map; In the instantaneous velocity field, turbulence anomalies are identified and eliminated according to the turbulence anomaly map, the laminar motion component is retained, and the laminar velocity field is generated; Perform spatial correlation correction on the laminar velocity field and the reference velocity of the virtual feature point set guided by the buoy to generate a corrected velocity field; Perform spatial integration on the corrected velocity field, calculate the average pixel velocity of each monitoring and collection site, and obtain the preliminary site pixel velocity; Performing spectral sensitivity compensation on the preliminary site pixel flow velocity based on the spectral weight configuration table, and finally calculating the site pixel flow velocity of the plurality of monitoring and collection sites; The step of constructing a spatial projection model, converting the pixel flow velocities of the plurality of stations into the water surface flow velocities of the plurality of stations through the spatial projection model, and performing fitting correction to obtain the plurality of corrected water surface flow velocities specifically comprises the following steps: Construct a spatial projection model from image coordinates to world coordinates; Converting the pixel flow velocities of the plurality of stations into the water surface flow velocities of the plurality of stations through the spatial projection model; Performing fitting correction on the water surface flow velocities at the plurality of stations using the buoy measurement data to obtain a plurality of corrected water surface flow velocities; The step of combining the plurality of corrected water surface flow velocities and the plurality of water level monitoring data to calculate the site flow data of the plurality of monitoring and collection sites and performing a visual display specifically includes the following steps: Calculating the site cross-sectional flow of the plurality of monitoring and collection sites by combining the plurality of corrected water surface flow velocities and the plurality of water level monitoring data; Establishing a timestamp matrix of site section flows according to the plurality of site section flows, and generating a timestamp flow matrix; Extracting a water level change curve from the water level monitoring data, calculating the water level change rate per unit time, and generating a water level change rate curve; Convert the water level change rate curve into a flow change impact factor to generate a water level-flow impact factor; Perform convolution operation on the water level-flow influencing factor and the timestamp flow matrix to generate the water level correction flow matrix; Obtaining the motion acceleration value of the buoy measurement data to obtain a water flow acceleration sequence; Perform dynamic weighted fusion of the water flow acceleration sequence and the water level correction flow matrix to generate the acceleration correction flow; Extract the traffic change patterns of the same period in history from the visualization platform and generate a historical traffic pattern library; Perform similarity matching between the acceleration-corrected flow and the historical flow pattern library to generate pattern matching flow; Based on the spatial distribution of the modified water surface velocity, a flow transfer relationship between sites is established to generate a watershed flow transfer network; The spatial consistency of pattern matching flow is optimized through the basin flow transfer network to generate spatially optimized flow; Analyzing the light intensity change curve of the image monitoring data to generate a circadian rhythm coefficient; applying the circadian rhythm coefficient to adjust the space optimization flow to generate a circadian regulated flow; Extracting thermal infrared channel data from the anti-interference multispectral image to obtain a water surface temperature distribution map; Calculate the evaporation loss compensation value based on the water surface temperature distribution map and the water level-flow influencing factor to generate the evaporation compensation value; Superimpose the diurnal and night-time regulated flow and the evaporation compensation value to calculate the flow at multiple stations during the period; Comprehensively sorting out the cross-sectional flow rates of the plurality of sites and the time period flow rates of the plurality of sites to obtain site flow data; The site traffic data is visualized on a preset visualization platform.

2. The flow monitoring method based on video image analysis and processing according to claim 1 is characterized in that: Determining a plurality of monitoring and collection sites, performing communication collection of image monitoring and water level monitoring at the plurality of monitoring and collection sites, and obtaining image monitoring data and water level monitoring data specifically includes the following steps: Identify multiple monitoring and collection sites; Selecting multiple relay collection sites from the multiple monitoring collection sites; According to a plurality of preset monitoring parameter data, generating and sending monitoring and collection instructions to a plurality of the relay collection sites; Through the plurality of relay collection sites, communication collection and recording of image monitoring are performed at the plurality of monitoring collection sites to obtain image monitoring data; Through the multiple relay collection sites, communication collection and recording of water level monitoring are performed at the multiple monitoring collection sites to obtain water level monitoring data.

3. A traffic monitoring system based on video image analysis and processing, characterized in that: The system applies the traffic monitoring method based on video image analysis and processing as described in any one of claims 1 to 2 above, and the system includes: A site monitoring and collection unit is used to determine multiple monitoring and collection sites, perform communication collection of image monitoring and water level monitoring at the multiple monitoring and collection sites, and obtain image monitoring data and water level monitoring data; A video image processing unit, configured to perform video frame extraction and image preprocessing on the image monitoring data to obtain water surface image data of a plurality of monitoring and collection sites; A pixel flow rate calculation unit, configured to perform feature point detection and displacement analysis on the plurality of water surface image data, and calculate the pixel flow rates of the plurality of monitoring and collection sites; A flow rate fitting and correction unit is used to construct a spatial projection model, convert the pixel flow rates of the plurality of stations into water surface flow rates of the plurality of stations through the spatial projection model, and perform fitting and correction to obtain a plurality of corrected water surface flow rates; The flow data display unit is used to combine the multiple corrected water surface flow velocities and the multiple water level monitoring data to calculate the site flow data of the multiple monitoring and collection sites and perform visual display.

Images