A method for regional monitoring of water surface velocity based on intelligent visual analysis

Abstract

The present invention discloses a method for monitoring the flow velocity area on the surface of water bodies based on intelligent visual analysis, and relates to the technical field of intelligent visual analysis. The specific implementation scheme is as follows: continuous image acquisition is performed on the target water body to obtain a dynamic image of the flow on the surface of the water body, and image preprocessing is performed on the video frames of the dynamic image; the optical flow estimation algorithm of a convolutional neural network is used to directly compare the global feature similarity through feature extraction, feature matching and optical flow optimization, and output the optical flow field data of the flow velocity on the surface of the water body; after obtaining the optical flow field data, the pixel displacement information of the optical flow estimation is converted into a flow velocity field on an actual physical unit, and the regional distribution of the flow velocity on the surface of the water body is analyzed at the same time. The present invention uses the optical flow estimation algorithm of a convolutional neural network to directly compare the global feature similarity and output the flow velocity data on the surface of the water body.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent visual analysis, and in particular to a method for monitoring regional flow velocity on the surface of a water area based on intelligent visual analysis. Background Art

[0002] At present, the traditional methods for monitoring water surface velocity mainly include:

[0003] 1. Point monitoring method: Use a current meter or ADCP (Acoustic Doppler Current Profiler) to obtain velocity data on the surface of the water area or at different depths. This method has high measurement accuracy, but it can only monitor the velocity at a single point and is difficult to reflect the overall velocity distribution on the surface of the water area. This makes it impossible to use it in mountain streams with large water level fluctuations or shallow rivers. The investment cost of its single-station construction is also high, which limits its applicability in large-scale promotion and application.

[0004] 2. Tracer monitoring method: monitor the direction and velocity of water flow by placing floating objects or dye tracers. Although this method can achieve a certain degree of regional monitoring, it has the disadvantages of requiring contact with the water body, complex deployment, and possible pollution to the water body.

[0005] 3. Remote sensing or laser scanning technology: Use remote sensing images or laser scanning to obtain flow rate information. This type of method has certain regional monitoring capabilities, but the equipment cost is high and is significantly affected by environmental conditions (such as light, weather, etc.). For example, the acoustic time difference flow meter can measure the average flow rate of the cross section, which has certain representativeness and accuracy. However, the technical equipment cost is high, and the equipment needs to be installed on both sides of the river. It has high requirements for the installation environment and conditions, and the protection and power supply of the equipment also bring certain difficulties.

[0006] Although the above technologies meet the needs of water surface velocity monitoring to a certain extent, they still have the following limitations:

[0007] 1. Insufficient regional monitoring capabilities: Point-based and tracer methods can only obtain flow velocity information within a limited range and are unable to reflect the overall flow velocity distribution on the water surface.

[0008] 2. Complex deployment: The tracer method requires contact with the water body and is not suitable for monitoring waters with severe pollution or high-risk environments.

[0009] 3. Cost and environmental limitations: Remote sensing and laser scanning technologies are expensive, greatly affected by environmental conditions, and difficult to promote and apply in complex or extreme environments. Summary of the invention

[0010] Based on this, the present invention provides a water surface flow rate regional monitoring method based on intelligent visual analysis to solve the problems of limited monitoring range, complex deployment and high cost in existing water surface flow rate monitoring methods.

[0011] The present invention provides a method for monitoring the regional flow velocity of a water surface based on intelligent visual analysis, comprising:

[0012] Continuously collecting images of the target water area to obtain dynamic images of the flow on the surface of the water area, and performing image preprocessing on the video frames of the dynamic images;

[0013] The optical flow estimation algorithm of convolutional neural network is used to directly compare the global feature similarity through feature extraction, feature matching and optical flow optimization, and output the optical flow field data of the water surface velocity;

[0014] After acquiring the optical flow field data, the pixel displacement information estimated by the optical flow is converted into a flow velocity field in actual physical units, and the regional distribution of the flow velocity on the water surface is analyzed.

[0015] A high-resolution camera is used to continuously capture images of the target water area. During the process of continuously capturing images of the target water area, the image frame interval requirements, image resolution requirements, water ripple continuity requirements and high-resolution camera stability requirements need to be met.

[0016] The image frame interval requirements specifically include:

[0017] Set the high-resolution camera to a constant frame rate The image is collected, and the time interval between two adjacent frames is In order to ensure that the optical flow estimation model can accurately capture the local feature changes of the water surface flow, the frame rate selection needs to meet the following conditions:

[0018] ;

[0019] in, Indicates the maximum velocity of the water surface. Indicates the time interval between two adjacent frames. Indicates the maximum displacement allowed in the image;

[0020] The image resolution requirements specifically include:

[0021] ;

[0022] ;

[0023] in, Indicates the width of the image. represents the height of the image, L represents the actual width or length of the monitored water area, and r represents the actual distance corresponding to each pixel;

[0024] The water pattern continuity requirements specifically include:

[0025] Assume that a set of texture features in the image are located at the pixel positions of frame t and frame t+1 respectively. and , then it is necessary to meet:

[0026] ;

[0027] ;

[0028] Among them, d represents the displacement vector, which is the target output of optical flow estimation, and L represents the actual width or length of the monitored water area;

[0029] The high-resolution camera stability requirements specifically include:

[0030] The high-resolution camera must remain stable during the acquisition process, and the optical axis must be perpendicular to the water surface or have a fixed tilt angle. The position of the high-resolution camera in the three-dimensional coordinate system is set to , then the position change of the high-resolution camera during the acquisition process must satisfy: , that is, the position of the high-resolution camera does not change.

[0031] The operation of performing image preprocessing on the video frame of the dynamic image includes:

[0032] The video frames are subjected to denoising, grayscale conversion, and scale normalization processing in sequence;

[0033] Use Gaussian filter to denoise the image and remove the interference of environmental noise on texture features. The operation of Gaussian filtering can be expressed as:

[0034] ;

[0035] Among them, x and y represent the pixel coordinates in the x-axis direction and y-axis direction of the image, u and v represent the pixel coordinates in the x-axis direction and y-axis direction of the Gaussian kernel, and k represents the x-axis and y-axis resolution of the Gaussian kernel. represents the original image, represents the filtered image, represents the Gaussian kernel function;

[0036] Gaussian kernel function The calculation includes,

[0037] ;

[0038] in, represents the standard deviation of the Gaussian distribution, represents pi;

[0039] The filtered image is grayed out, and the gray value The calculation formula includes:

[0040] ;

[0041] in, , , Respectively represent the pixel values ​​of the red, green, and blue channels in the filtered image;

[0042] Normalize the pixel values ​​of the image that has been denoised and grayscaled to the range , to meet the input requirements of the optical flow estimation algorithm,

[0043] ;

[0044] in, represents the pixel value of the image after scale normalization, and Represent the minimum and maximum pixel values ​​of the image respectively.

[0045] The feature extraction operation includes:

[0046] Input two consecutive frames of images and Downsampled to Three scales are used to extract features at different resolutions. The downsampled feature map is recorded as and ,

[0047] ;

[0048] ;

[0049] in, Indicates the scale level, represents the preprocessed image at time t, represents the preprocessed image at time t+1;

[0050] The downsampled feature map and Normalized respectively.

[0051] ;

[0052] ;

[0053] in, express The L2 norm of express The feature map after L2 normalization, express The L2 norm of express Feature map after L2 normalization.

[0054] The feature matching operation includes:

[0055] Based on the normalized feature map and , obtain a global correlation matrix C, the calculation of the global correlation matrix C includes,

[0056] ;

[0057] Among them, i represents the feature map after normalization The pixel index of , j represents the normalized feature map The pixel index of

[0058] Optimize the global relevance matrix C using a Transformer-based global attention mechanism to obtain an attention weight A;

[0059] ;

[0060] The feature map normalized based on the attention weight A and the normalized time t+1 Get optimized matching features ,

[0061] .

[0062] The optical flow optimization operation includes:

[0063] Generate the initial optical flow field using direct regression method ,

[0064] ;

[0065] Among them, Regression is a fully connected layer responsible for mapping high-dimensional features to two-dimensional optical flow vector fields;

[0066] The optical flow field is refined using a feature recursive optimization module, which consists of multiple refinement processes, each of which contains a feature residual network. In each refinement process, the feature residual network is used to extract the incremental update of the optical flow field output by the previous refinement process, and the incremental update is combined with the optical flow field output by the previous refinement process to obtain the optical flow field of the current refinement process. For the optical flow field output by the k+1th refinement process, , whose expression is:

[0067] ;

[0068] in, represents the feature residual network contained in the k+1th refinement process, represents the optical flow field output by the kth refinement process;

[0069] After multiple refinement processes, the final high-precision optical flow field is obtained.

[0070] include,

[0071] The homography matrix H is used to map points on the image plane to the actual water surface coordinate system. The calculation of the homography matrix H includes:

[0072] Set the internal parameter matrix of the high-resolution camera to K, and the external parameter to be the rotation matrix R and the translation vector Represented, the calculation formula of the homography matrix H is:

[0073] ;

[0074] Where n represents the water surface normal vector, and d represents the straight-line distance from the camera to the water surface;

[0075] ;

[0076] in, , are the focal lengths in the x and y directions, , Represents the x and y pixel coordinates of the optical center on the image plane;

[0077] The high-resolution camera external parameters describe the transformation relationship between the high-resolution camera coordinate system and the world coordinate system, usually composed of the rotation matrix R and the translation vector composition:

[0078] ;

[0079] ;

[0080] R is a 3×3 orthogonal matrix used to describe the rotation of the high-resolution camera coordinate system relative to the world coordinate system;

[0081] is a 3×1 column vector, used to describe the translation of the optical center of the high-resolution camera relative to the world coordinate system;

[0082] , , Respectively x , y , The translational component in the direction;

[0083] Through the homography matrix H, the pixel coordinates on the image are Can be mapped to the actual coordinates of the water surface :

[0084] .

[0085] include,

[0086] Using the pixel displacement information and image frame rate in the optical flow field, the pixel displacement in the image plane is converted into the actual speed of the water surface; the optical flow estimation result is set as the pixel displacement field ,in and Represents the displacement on the x-axis and y-axis respectively, and the time interval between frames is , the calculation formula of the actual physical flow rate is:

[0087] ;

[0088] ;

[0089] ;

[0090] in, Indicates the conversion ratio between pixels and actual coordinates;

[0091] Finally, the velocity vector V of each pixel can be expressed as:

[0092] ;

[0093] in, and Respectively represent the flow velocity vector x Direction and y Directional weight;

[0094] The size of the flow rate Calculated by vector modulus:

[0095] .

[0096] The analysis of the regional distribution of surface velocity of water bodies includes,

[0097] Based on the number of pixels N within the ROI and the pixel index position in the ROI The corresponding flow rate Average flow rate ,

[0098] ;

[0099] Calculate the maximum flow velocity in the area and minimum value ,

[0100] ;

[0101] ;

[0102] Divide the flow rate into K intervals at uniform intervals, and the number of flow rates in each interval It is expressed as:

[0103] ;

[0104] Where N represents the number of pixels in the ROI. represents the pixel index in the ROI, Represents the indicator function. When V satisfies When the condition The value of is 1, otherwise it is 0; , They represent the minimum and maximum values ​​of the kth interval respectively.

[0105] Beneficial effects: The present invention uses the optical flow estimation algorithm of convolutional neural network to directly compare the global feature similarity and output the surface velocity data of the water area. Compared with the traditional monitoring method, the present invention does not need to rely on tracers and can achieve non-contact and regionalized velocity monitoring, effectively overcoming the limitations of the current technology that can only monitor single-point velocity and is difficult to deploy.

[0106] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0107] The accompanying drawings are used to better understand the present invention and do not constitute a limitation of the present invention.

[0108] Figure 1 is a schematic diagram of a process provided according to the present invention;

[0109] Figure 2 It is a flowchart of an optical flow estimation algorithm of a convolutional neural network provided by the present invention;

[0110] Figure 3 It is a schematic diagram of a scene schematic diagram of river flow velocity monitoring by a drone platform provided in Example 1 of the present invention;

[0111] Figure 4a is the river channel image of the UAV platform at time t provided by Example 1 of the present invention;

[0112] Figure 4b is the river channel image of the UAV platform at time t+1 provided by Example 1 of the present invention;

[0113] Figure 4c is a flow velocity value area distribution diagram provided according to Example 1 of the present invention;

[0114] Figure 4d is a flow velocity direction regional distribution diagram provided according to Example 1 of the present invention;

[0115] Figure 5 2 is a schematic diagram of a scenario for monitoring the discharge flow rate of a sluice pump station provided in Embodiment 2 of the present invention;

[0116] Figure 6a is the image of the pump station monitored by the monitoring camera at time t provided by Example 2 of the present invention;

[0117] Figure 6b is the image of the pump station monitored by the camera at time t+1 provided by Example 2 of the present invention;

[0118] Figure 6c is a flow velocity value area distribution diagram provided according to Example 2 of the present invention;

[0119] Figure 6d This is a flow velocity direction regional distribution diagram provided according to Example 2 of the present invention. DETAILED DESCRIPTION

[0120] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0121] like Figure 1 As shown, the present invention provides a method for monitoring the surface velocity of water area based on intelligent visual analysis, comprising:

[0122] S1: Continuously collect images of the target water area, obtain dynamic images of the flow on the surface of the water area, and perform image preprocessing on the video frames of the dynamic images. It should be noted that:

[0123] The target water area is continuously imaged using a high-resolution camera. During the process of continuous image acquisition of the target water area, the image frame interval requirements, image resolution requirements, water ripple continuity requirements, and high-resolution camera stability requirements need to be met. This application ensures the accuracy and robustness of water surface flow velocity monitoring by setting the requirements of image frame interval, image resolution requirements, water ripple continuity, and high-resolution camera stability requirements during the process of continuous image acquisition of the target water area.

[0124] The image frame interval requirements specifically include:

[0125] Set the high-resolution camera to a constant frame rate The image is collected, and the time interval between two adjacent frames is In order to ensure that the optical flow estimation model can accurately capture the local feature changes of the water surface flow, the frame rate selection needs to meet the following conditions:

[0126] ;

[0127] in, Indicates the maximum velocity of the water surface. Indicates the time interval between two adjacent frames. Indicates the maximum displacement allowed in the image;

[0128] Set the frame rate selection to , is to ensure that the optical flow estimation model can accurately capture the local feature changes of the water surface flow. This condition ensures that the changes in the water texture features in two adjacent frames are within the recognizable range of the optical flow estimation algorithm, avoiding excessive pixel displacement that leads to matching failure.

[0129] The image resolution requirements specifically include:

[0130] ;

[0131] ;

[0132] in, Indicates the width of the image. represents the height of the image, L represents the actual width or length of the monitored water area, and r represents the actual distance corresponding to each pixel. The recommended value is m / px to ensure monitoring accuracy.

[0133] Image resolution meets , , which ensures that the detailed texture of the water surface is captured to support feature extraction for optical flow estimation.

[0134] The water pattern continuity requirements specifically include:

[0135] The surface texture features of the water area between two consecutive frames of images must be consistent, that is, the changes in texture features are mainly caused by the flow of water, rather than other random factors (such as illumination, reflection). Suppose a set of texture features in the image has the following pixel positions at frame t and frame t+1: and , then it is necessary to meet:

[0136] ;

[0137] ;

[0138] Among them, d represents the displacement vector, which is the target output of optical flow estimation, and L represents the actual width or length of the monitored water area;

[0139] The high-resolution camera stability requirements specifically include:

[0140] The high-resolution camera must remain stable during the acquisition process, and the optical axis must be perpendicular to the water surface or have a fixed tilt angle. The position of the high-resolution camera in the three-dimensional coordinate system is set to , then the position change of the high-resolution camera during the acquisition process must satisfy: , that is, the position of the high-resolution camera does not change.

[0141] The operation of performing image preprocessing on the video frame of the dynamic image includes:

[0142] The video frames are subjected to denoising, grayscale conversion, and scale normalization processing in sequence to ensure the quality and consistency of the input data.

[0143] Use Gaussian filter to denoise the image and remove the interference of environmental noise on texture features. The operation of Gaussian filtering can be expressed as:

[0144] ;

[0145] Among them, x and y represent the pixel coordinates in the x-axis direction and y-axis direction of the image, u and v represent the pixel coordinates in the x-axis direction and y-axis direction of the Gaussian kernel, and k represents the x-axis and y-axis resolution of the Gaussian kernel. represents the original image, represents the filtered image, represents the Gaussian kernel function;

[0146] Gaussian kernel function The calculation includes,

[0147] ;

[0148] in, Represents the standard deviation of the Gaussian distribution, which affects the density of the Gaussian kernel. The smaller the standard deviation, the sharper the shape of the Gaussian kernel and the less obvious the smoothing effect; the larger the standard deviation, the flatter the shape of the Gaussian kernel and the more significant the smoothing effect. , represents pi;

[0149] The filtered image is grayed to reduce the computational complexity. The calculation formula includes:

[0150] ;

[0151] in, , , Respectively represent the pixel values ​​of the red, green, and blue channels in the filtered image;

[0152] Normalize the pixel values ​​of the image that has been denoised and grayscaled to the range , to meet the input requirements of the optical flow estimation algorithm,

[0153] ;

[0154] in, represents the pixel value of the image after scale normalization, and Represent the minimum and maximum pixel values ​​of the image respectively.

[0155] S2: Using the convolutional neural network optical flow estimation algorithm, through feature extraction, feature matching and optical flow optimization, directly compare the global feature similarity and output the optical flow field data of the water surface velocity, such as Figure 2 It should be noted that:

[0156] The present invention regards the water surface flow as a special optical flow estimation problem, uses the optical flow estimation algorithm of convolutional neural network, directly compares the global feature similarity, and outputs the optical flow field data of the water surface flow velocity. Through the three key steps of feature extraction, feature matching and optical flow optimization, the present invention can efficiently generate the optical flow field data of the water surface, providing reliable input data for flow velocity calculation and regional monitoring.

[0157] The feature extraction operation includes:

[0158] Input two consecutive frames of images and Downsampled to Three scales are used to extract features at different resolutions. The downsampled feature map is recorded as and ,

[0159] ;

[0160] ;

[0161] in, Indicates the scale level, represents the preprocessed image at time t, Represents the preprocessed image at time t+1; Backbone is a backbone feature extraction network that can capture global context information while retaining local texture details. In the present invention, the Vision Transformer network structure is used as the Backbone backbone feature extraction network.

[0162] The downsampled feature map and Normalized respectively.

[0163] ;

[0164] ;

[0165] in, express The L2 norm of express The feature map after L2 normalization, express The L2 norm of express Feature map after L2 normalization.

[0166] The normalized features are mapped to a high-dimensional feature space to facilitate subsequent global matching.

[0167] The feature matching operation includes:

[0168] Feature matching is the core step of optical flow estimation. The present invention constructs a cost volume through global correlation matching to capture all possible feature correspondences between two input image frames.

[0169] Based on the normalized feature map and , obtain a global correlation matrix C, the calculation of the global correlation matrix C includes,

[0170] ;

[0171] Among them, i represents the feature map after normalization The pixel index of , j represents the normalized feature map The pixel index of

[0172] Optimize the global relevance matrix C using a Transformer-based global attention mechanism to obtain an attention weight A;

[0173] ;

[0174] The feature map normalized based on the attention weight A and the normalized time t+1 Get optimized matching features ,

[0175] .

[0176] The matched features are input into a feature recursive optimization module to refine the optical flow estimation results layer by layer and improve the prediction accuracy.

[0177] The optical flow optimization operation includes:

[0178] Generate the initial optical flow field using direct regression method ,

[0179] ;

[0180] Among them, Regression is a fully connected layer responsible for mapping high-dimensional features to two-dimensional optical flow vector fields;

[0181] The optical flow field is refined using a feature recursive optimization module, which consists of multiple refinement processes, each of which contains a feature residual network. In each refinement process, the feature residual network is used to extract the incremental update of the optical flow field output by the previous refinement process, and the incremental update is combined with the optical flow field output by the previous refinement process to obtain the optical flow field of the current refinement process. For the optical flow field output by the k+1th refinement process, , whose expression is:

[0182] ;

[0183] in, represents the feature residual network contained in the k+1th refinement process, represents the optical flow field output by the kth refinement process;

[0184] After multiple refinement processes, the final high-precision optical flow field is obtained.

[0185] S3: After obtaining the optical flow field data, the pixel displacement information estimated by the optical flow is converted into a velocity field in actual physical units, and the regional distribution of the velocity on the surface of the water body is analyzed. It should be noted that:

[0186] After obtaining the optical flow field, the pixel displacement information estimated by the optical flow needs to be converted into a velocity field in actual physical units, and the regional distribution of the velocity on the water surface needs to be analyzed. The core steps include the calculation of the homography matrix, the calculation of the physical unit velocity, and the vectorization analysis of the regional distribution.

[0187] The homography matrix H is used to map points on the image plane to the actual water surface coordinate system. The calculation of the homography matrix H includes:

[0188] Set the internal parameter matrix of the high-resolution camera to K, and the external parameter to be the rotation matrix R and the translation vector Represented, the calculation formula of the homography matrix H is:

[0189] ;

[0190] Where n represents the water surface normal vector, and d represents the straight-line distance from the camera to the water surface;

[0191] ;

[0192] in, , are the focal lengths in the x and y directions, , Represents the x and y pixel coordinates of the optical center on the image plane;

[0193] The high-resolution camera external parameters describe the transformation relationship between the high-resolution camera coordinate system and the world coordinate system, usually composed of the rotation matrix R and the translation vector composition:

[0194] ;

[0195] ;

[0196] R is a 3×3 orthogonal matrix used to describe the rotation of the high-resolution camera coordinate system relative to the world coordinate system;

[0197] is a 3×1 column vector, used to describe the translation of the optical center of the high-resolution camera relative to the world coordinate system;

[0198] , , Respectively x , y , The translational component in the direction;

[0199] Through the homography matrix H, the pixel coordinates on the image are Can be mapped to the actual coordinates of the water surface :

[0200] .

[0201] include,

[0202] Using the pixel displacement information and image frame rate in the optical flow field, the pixel displacement in the image plane is converted into the actual speed of the water surface; the optical flow estimation result is set as the pixel displacement field ,in and Represents the displacement on the x-axis and y-axis respectively, and the time interval between frames is , the calculation formula of the actual physical flow rate is:

[0203] ;

[0204] ;

[0205] ;

[0206] in, Indicates the conversion ratio between pixels and actual coordinates;

[0207] Finally, the velocity vector V of each pixel can be expressed as:

[0208] ;

[0209] in, and Respectively represent the flow velocity vector x Direction and y Directional weight;

[0210] The size of the flow rate Calculated by vector modulus:

[0211] ;

[0212] The analysis of the regional distribution of surface velocity of water bodies includes,

[0213] Based on the number of pixels N within the ROI and the pixel index position in the ROI The corresponding flow rate Average flow rate ,

[0214] .

[0215] Calculate the maximum flow velocity in the area and minimum value ,

[0216] ;

[0217] ;

[0218] Divide the flow rate into K intervals at uniform intervals, and the number of flow rates in each interval It is expressed as:

[0219] ;

[0220] Where N represents the number of pixels in the ROI. represents the pixel index in the ROI, Represents the indicator function. When V satisfies When the condition The value of is 1, otherwise it is 0; , They represent the minimum and maximum values ​​of the kth interval respectively.

[0221] Example 1: UAV platform river flow monitoring

[0222] In this embodiment, a drone is used to vertically photograph the river channel to obtain a sequence of images, and the regional distribution of flow velocity on the water surface is calculated through image processing and optical flow estimation.

[0223] Step 1: Image acquisition and preprocessing:

[0224] Step 1.1: Image acquisition

[0225] In the process of river flow velocity monitoring by UAV platform, in order to meet the requirements of image frame interval, image resolution, water texture feature continuity and camera stability, the parameters for image acquisition are configured as follows:

[0226] Drone flight altitude: about 60 meters above the ground

[0227] Image resolution: 3840 pixels × 2160 pixels, the corresponding monitoring range of the image is approximately 38 meters × 21 meters, that is, 1 pixel width corresponds to approximately 1 centimeter.

[0228] Image acquisition frequency: 25 frames per second, continuous acquisition of image sequence

[0229] Camera angle: When setting the drone platform image acquisition, the camera is vertical to the ground, the pitch angle is 0°, the rotation angle is 0° (that is, the drone does not rotate at all), and the yaw angle of the drone is set to 0°, that is, the upper part of the image points to the geographical north, the lower part points to the geographical south, and the left side points to the geographical west.

[0230] Requirement 1.1.1: Image frame interval requirements

[0231] Set the high-resolution camera to a constant frame rate The image is collected, and the time interval between two adjacent frames is In order to ensure that the optical flow estimation model can accurately capture the local feature changes of the water surface flow, the frame rate selection needs to meet the following conditions:

[0232] ;

[0233] in, Indicates the maximum velocity of the water surface. Indicates the time interval between two adjacent frames. Indicates the maximum displacement allowed in the image;

[0234] In this embodiment, the resolution of the drone camera is 3840 pixels × 2160 pixels, and the minimum feature acquisition ratio of the subsequent optical flow estimation method is 1 / 16, that is, the maximum pixel displacement allowed is 240 pixels, corresponding to a maximum displacement of 2.4 meters. At the same time, the camera collects 25 frames of images per second. It can be inferred that under the frame rate conditions of this embodiment, it can meet the flow rate monitoring of the river scene with a maximum surface flow velocity of less than 60m / s, which can meet the flow rate monitoring requirements of the scene in this embodiment.

[0235] Requirement 1.1.2: Image resolution requirements

[0236] The image resolution requirements specifically include:

[0237] ;

[0238] ;

[0239] in, Indicates the width of the image. represents the height of the image, L represents the actual width or length of the monitored water area (unit: m), and r represents the actual distance corresponding to each pixel (spatial resolution, unit: m / px).

[0240] The recommended value is m / px to ensure monitoring accuracy. In this embodiment, the image taken by the UAV platform is r=0.01m / px, which meets the image resolution requirement.

[0241] Requirement 1.1.3: Water texture feature consistency

[0242] The surface texture features of the water area between two consecutive frames of images must be consistent, that is, the changes in texture features are mainly caused by the flow of water, rather than other random factors (such as illumination, reflection). Suppose a set of texture features in the image has the following pixel positions at frame t and frame t+1: and , then it is necessary to meet:

[0243] ;

[0244] ;

[0245] Among them, d represents the displacement vector, which is the target output of optical flow estimation, and L represents the actual width or length of the monitored water area;

[0246] In this embodiment, the resolution of the drone camera is 3840 pixels × 2160 pixels, and the minimum feature acquisition ratio of the subsequent optical flow estimation method is 1 / 16, that is, the maximum allowed pixel displacement is 240 pixels, and the corresponding maximum displacement is 2.4 meters, that is, the pixel displacement of a set of texture features between two frames of images should be less than 240 pixels, and the physical displacement should be less than 2.4 meters.

[0247] Requirement 1.1.4: Camera Stability

[0248] The high-resolution camera must remain stable during the acquisition process, and the optical axis must be perpendicular to the water surface or have a fixed tilt angle. The position of the high-resolution camera in the three-dimensional coordinate system is set to , then the position change of the high-resolution camera during the acquisition process must satisfy: , that is, the position of the high-resolution camera does not change.

[0249] Step 1.2: Image preprocessing

[0250] The image preprocessing adopts the method described in the technical solution of the present invention, including denoising, grayscale, and scale normalization operations, which will not be described in detail.

[0251] Step 2: Feature extraction and optical flow estimation

[0252] The method described in the technical solution of the present invention is adopted, and the optical flow estimation algorithm of the convolutional neural network is used to directly compare the global feature similarity and output the optical flow field data of the surface velocity of the water area. The three key steps of feature extraction, feature matching and optical flow optimization are described in the technical solution of the present invention. There is no special change in this embodiment and no further description is made.

[0253] Step 3: Flow velocity calculation and regional distribution analysis

[0254] After obtaining the optical flow field, the pixel displacement information estimated by the optical flow needs to be converted into a velocity field in actual physical units, and the regional distribution of the velocity on the water surface needs to be analyzed. The core steps include the calculation of the homography matrix, the calculation of the physical unit velocity, and the vectorization analysis of the regional distribution.

[0255] Step 3.1: Calculation of the homography matrix between the image plane and the water surface

[0256] The homography matrix H is used to map points on the image plane to the actual water surface coordinate system. The calculation of the homography matrix H includes:

[0257] Set the internal parameter matrix of the high-resolution camera to K, and the external parameter to be the rotation matrix R and the translation vector Represented, the calculation formula of the homography matrix H is:

[0258] ;

[0259] Wherein, n represents the water surface normal vector, d represents the straight-line distance from the camera to the water surface, and in this embodiment, the vertical distance from the drone to the water surface is 60 meters;

[0260] ;

[0261] in, , are the focal lengths in the x and y directions, , Represents the x and y pixel coordinates of the optical center on the image plane;

[0262] In this embodiment, the image resolution is 3840 pixels × 2160 pixels, and the internal parameter matrix K can be obtained:

[0263] ;

[0264] The high-resolution camera external parameters describe the transformation relationship between the high-resolution camera coordinate system and the world coordinate system, usually composed of the rotation matrix R and the translation vector composition:

[0265] ;

[0266] ;

[0267] R is a 3×3 orthogonal matrix used to describe the rotation of the high-resolution camera coordinate system relative to the world coordinate system;

[0268] is a 3×1 column vector, used to describe the translation of the optical center of the high-resolution camera relative to the world coordinate system;

[0269] , , Respectively x , y , The translational component in the direction;

[0270] In this embodiment, the UAV platform is stationary relative to the observation area, and the camera pitch angle, yaw angle, and rotation angle are all 0°, and the upper part of the image points to the geographical north, so:

[0271] ;

[0272] ;

[0273] Through the homography matrix H, the pixel coordinates on the image are Can be mapped to the actual coordinates of the water surface :

[0274] .

[0275] Step 3.2: Calculation of actual flow velocity at the pixel position of the velocity field

[0276] Using the pixel displacement information and image frame rate in the optical flow field, the pixel displacement in the image plane is converted into the actual speed of the water surface; the optical flow estimation result is set as the pixel displacement field ,in and Represents the displacement on the x-axis and y-axis respectively, and the time interval between frames is , the calculation formula of the actual physical flow rate is:

[0277] ;

[0278] ;

[0279] ;

[0280] in, Indicates the conversion ratio between pixels and actual coordinates;

[0281] Finally, the velocity vector V of each pixel can be expressed as:

[0282] ;

[0283] in, and Respectively represent the flow velocity vector x Direction and y Directional weight;

[0284] The size of the flow rate Calculated by vector modulus:

[0285] ;

[0286] Step 3.3: Regional distribution analysis of velocity field

[0287] Statistical indicator 3.3.1: Average flow rate

[0288] Based on the number of pixels N within the ROI and the pixel index position in the ROI The corresponding flow rate Average flow rate ,

[0289] .

[0290] Statistical indicator 3.3.2: Calculate the maximum flow velocity in the area and minimum value ,

[0291] ;

[0292] ;

[0293] Statistical indicators 3.3.3: Flow rate histogram statistical indicators

[0294] Divide the flow rate into K intervals at uniform intervals, and the number of flow rates in each interval It is expressed as:

[0295] ;

[0296] Where N represents the number of pixels in the ROI. represents the pixel index in the ROI, Represents the indicator function. When V satisfies When the condition The value of is 1, otherwise it is 0; , They represent the minimum and maximum values ​​of the kth interval respectively.

[0297] In this embodiment, the river is monitored by a drone platform, and the average flow velocity within the complete monitoring range corresponding to the image is , maximum flow rate , minimum flow rate .

[0298] Table 1 Histogram statistics of river flow rate monitoring by UAV platform

[0299]

[0300] Figure 3 This is a schematic diagram of the scene of river flow velocity monitoring by the drone platform in Example 1, which shows the vertical position relationship between the drone and the river in this example. In this example, the camera's visual range should be based on the river water surface, and the river water surface should be at the center of the camera's visual range as much as possible, and the river bank should be at the edge of the camera's visual range; at the same time, the drone should maintain a vertical height of about 60 meters from the river bank when flying.

[0301] Figures 4a to 4d This is an example diagram of the regional distribution analysis of the flow rate of the river flow rate monitored by the drone platform in Example 1. The figure shows a set of image sequences collected by the drone platform through the camera, and shows the regional distribution analysis results of the flow rate obtained in this example. Figure 4a It shows the river channel image of the drone platform at time t. Figure 4b It shows the river channel image of the drone platform at time t+1. Figure 4c The flow velocity value is represented by a grayscale image. The larger the flow velocity value, the darker the pixel, and the smaller the flow velocity value, the whiter the pixel. Figure 4d The direction and magnitude of the flow velocity are represented by a vector field diagram, where the direction of each arrow is the same as the flow velocity direction of the corresponding area. The longer the arrow, the greater the flow velocity value, and the shorter the arrow, the smaller the flow velocity value. Figure 4c When expressing the magnitude of the flow velocity value, the flow velocity corresponding to each pixel is assigned a unique grayscale value, that is, the granularity is a single pixel; Figure 4d When representing the direction and size of the flow velocity value, the pixel points corresponding to each arrow are sampled at intervals, with a specific interval of 80 pixels in width. The length and direction of the arrow are determined by the flow velocity corresponding to the pixel points in the interval selection, that is, the granularity is 80 pixels.

[0302] Example 2: Discharge flow rate monitoring at a sluice pump station

[0303] In this embodiment, fixed surveillance cameras are installed at key locations of the sluice pump station to regularly capture water area image sequences and use intelligent visual analysis technology to calculate water surface velocity. The optical flow in the image space is converted into a velocity vector in the physical space through the internal and external parameters of the camera, thereby achieving real-time monitoring of the released water flow and velocity field analysis.

[0304] Step 1: Image acquisition and preprocessing:

[0305] Step 1.1: Image acquisition

[0306] In the process of monitoring the discharge flow rate of the sluice pump station, in order to meet the requirements of image frame interval, image resolution, water texture feature continuity and camera stability, the parameters for image acquisition are configured as follows:

[0307] The surveillance camera is about 30 meters horizontally from the discharge outlet of the sluice pump station and about 12 meters vertically from the water surface.

[0308] Image resolution: 2560 pixels × 1440 pixels.

[0309] Image acquisition frequency: 25 frames per second, continuous acquisition of image sequence

[0310] Camera Angle: Set the surveillance camera's yaw angle and rotation angle to 0°, and the pitch angle to 25°.

[0311] Requirement 1.1.1: Image frame interval requirements

[0312] The high-resolution camera is set to collect images at a constant frame rate, and the time interval between two adjacent frames is In order to ensure that the optical flow estimation model can accurately capture the local feature changes of the water surface flow, the frame rate selection needs to meet the following conditions:

[0313] ;

[0314] in, Indicates the maximum velocity of the water surface. Indicates the time interval between two adjacent frames. Indicates the maximum displacement allowed in the image;

[0315] In this embodiment, the resolution of the surveillance camera is 2560 pixels × 1440 pixels, and the minimum feature acquisition ratio of the subsequent optical flow estimation method is 1 / 16, that is, the maximum pixel displacement allowed is 160 pixels, corresponding to a maximum displacement of about 1.6 meters. At the same time, the camera collects 25 frames of images per second. It can be inferred that under the frame rate conditions of this embodiment, it can meet the flow velocity monitoring of river scenes with a maximum surface velocity of less than 40m / s.

[0316] Requirement 1.1.2: Image resolution requirements

[0317] The image resolution requirements specifically include:

[0318] ;

[0319] ;

[0320] in, Indicates the width of the image. represents the height of the image, L represents the actual width or length of the monitored water area (unit: m), and r represents the actual distance corresponding to each pixel (spatial resolution, unit: m / px).

[0321] The recommended value is m / px to ensure monitoring accuracy. In this embodiment, the image captured by the monitoring camera is r≈0.01m / px, which meets the image resolution requirement.

[0322] Requirement 1.1.3: Water texture feature consistency

[0323] The surface texture features of the water area between two consecutive frames of images must be consistent, that is, the changes in texture features are mainly caused by the flow of water, rather than other random factors (such as illumination, reflection). Suppose a set of texture features in the image has the following pixel positions at frame t and frame t+1: and , then it is necessary to meet:

[0324] ;

[0325] ;

[0326] Among them, d represents the displacement vector, which is the target output of optical flow estimation, and L represents the actual width or length of the monitored water area;

[0327] In this embodiment, the resolution of the surveillance camera is 2560 pixels × 1440 pixels, and the minimum feature acquisition ratio of the subsequent optical flow estimation method is 1 / 16, that is, the maximum allowed pixel displacement is 160 pixels, and the corresponding maximum displacement is about 1.6 meters, that is, the pixel displacement of a set of texture features between two frames of images should be less than 160 pixels, and the physical displacement should be less than 1.6 meters.

[0328] Requirement 1.1.4: Camera Stability

[0329] The high-resolution camera must remain stable during the acquisition process, and the optical axis must be perpendicular to the water surface or have a fixed tilt angle. The position of the high-resolution camera in the three-dimensional coordinate system is set to , then the position change of the high-resolution camera during the acquisition process must satisfy: , that is, the position of the high-resolution camera does not change.

[0330] Step 1.2: Image preprocessing

[0331] The image preprocessing adopts the method described in the technical solution of the present invention, including denoising, grayscale, and scale normalization operations, which will not be described in detail.

[0332] Step 2: Feature extraction and optical flow estimation

[0333] The method described in the technical solution of the present invention is adopted, and the optical flow estimation algorithm of the convolutional neural network is used to directly compare the global feature similarity and output the optical flow field data of the surface velocity of the water area. The three key steps of feature extraction, feature matching and optical flow optimization are described in the technical solution of the present invention. There is no special change in this embodiment and no further description is made.

[0334] Step 3: Flow velocity calculation and regional distribution analysis

[0335] After obtaining the optical flow field, the pixel displacement information estimated by the optical flow needs to be converted into a velocity field in actual physical units, and the regional distribution of the velocity on the water surface needs to be analyzed. The core steps include the calculation of the homography matrix, the calculation of the physical unit velocity, and the vectorization analysis of the regional distribution.

[0336] Step 3.1: Calculation of the homography matrix between the image plane and the water surface

[0337] The homography matrix H is used to map points on the image plane to the actual water surface coordinate system. The calculation of the homography matrix H includes:

[0338] Set the internal parameter matrix of the high-resolution camera to K, and the external parameter to be the rotation matrix R and the translation vector Represented, the calculation formula of the homography matrix H is:

[0339] ;

[0340] Wherein, n represents the water surface normal vector, d represents the straight-line distance from the camera to the water surface. In this embodiment, the monitoring camera is about 30 meters horizontally from the discharge port of the sluice pump station and about 12 meters vertically from the water surface. Therefore, the straight-line distance from the camera to the target water surface is about 32.3 meters;

[0341] ;

[0342] in, , are the focal lengths in the x and y directions, , Represents the x and y pixel coordinates of the optical center on the image plane;

[0343] In this embodiment, the image resolution is 2560 pixels × 1440 pixels, and the internal parameter matrix K can be obtained:

[0344] ;

[0345] The high-resolution camera external parameters describe the transformation relationship between the high-resolution camera coordinate system and the world coordinate system, usually composed of the rotation matrix R and the translation vector composition:

[0346] ;

[0347] ;

[0348] R is a 3×3 orthogonal matrix used to describe the rotation of the high-resolution camera coordinate system relative to the world coordinate system;

[0349] is a 3×1 column vector, used to describe the translation of the optical center of the high-resolution camera relative to the world coordinate system;

[0350] , , Respectively x , y , The translational component in the direction;

[0351] In this embodiment, the surveillance camera is stationary relative to the observation area, and the camera yaw angle and rotation angle are both 0°, and only the pitch angle is 25°, so:

[0352] ;

[0353] ;

[0354] Through the homography matrix H, the pixel coordinates on the image are Can be mapped to the actual coordinates of the water surface :

[0355] .

[0356] Step 3.2: Calculation of actual flow velocity at the pixel position of the velocity field

[0357] Using the pixel displacement information and image frame rate in the optical flow field, the pixel displacement in the image plane is converted into the actual speed of the water surface; the optical flow estimation result is set as the pixel displacement field ,in and Represents the displacement on the x-axis and y-axis respectively, and the time interval between frames is , the calculation formula of the actual physical flow rate is:

[0358] ;

[0359] ;

[0360] ;

[0361] in, Indicates the conversion ratio between pixels and actual coordinates;

[0362] Finally, the velocity vector V of each pixel can be expressed as:

[0363] ;

[0364] The size of the flow rate Calculated by vector modulus:

[0365] ;

[0366] in, and Respectively represent the flow velocity vector x Direction and y Directional weight;

[0367] Step 3.3: Regional distribution analysis of velocity field

[0368] Statistical indicator 3.3.1: Average flow rate

[0369] Based on the number of pixels N within the ROI and the pixel index position in the ROI The corresponding flow rate Average flow rate ,

[0370] .

[0371] Statistical indicator 3.3.2: Calculate the maximum flow velocity in the area and minimum value ,

[0372] ;

[0373] ;

[0374] Statistical indicators 3.3.3: Flow rate histogram statistical indicators

[0375] Divide the flow rate into K intervals at uniform intervals, and the number of flow rates in each interval It is expressed as:

[0376] ;

[0377] Where N represents the number of pixels in the ROI. represents the pixel index in the ROI, Represents the indicator function. When V satisfies When the condition The value of is 1, otherwise it is 0; , They represent the minimum and maximum values ​​of the kth interval respectively.

[0378] In this embodiment, the discharge flow rate of the sluice pump station is monitored, and the average flow rate within the complete monitoring range corresponding to the image is , maximum flow rate , minimum flow rate .

[0379] Table 2 Statistical table of discharge flow rate monitoring histogram of sluice pump station

[0380]

[0381] Figure 5 This is a schematic diagram of the scene of the discharge flow rate monitoring of the gate pump station in Example 2, which shows the horizontal and vertical position relationship between the camera and the gate pump station discharge port in this embodiment. In this embodiment, the monitoring camera is about 30 meters horizontally away from the gate pump station discharge port and about 12 meters vertically away from the water surface. At the same time, the pitch angle of the monitoring camera is adjusted so that the camera's visual range covers the gate pump station discharge port and the water surface, and the gate pump station discharge port is located at the center of the camera's visual range.

[0382] Figures 6a to 6d This is an example diagram of flow rate regional distribution analysis of discharge flow rate monitoring of a sluice pump station in Example 2. The diagram shows a set of image sequences captured by a monitoring camera and the flow rate regional distribution analysis results obtained in this example.

[0383] in, Figure 6a It shows the image of the pump station from the surveillance camera at time t. Figure 6b It shows the image of the pump station from the surveillance camera at time t+1. Figure 6c The flow velocity value is represented by a grayscale image. The larger the flow velocity value, the darker the pixel, and the smaller the flow velocity value, the whiter the pixel. Figure 6d The direction and magnitude of the flow velocity are represented by a vector field diagram.

[0384] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for monitoring water surface velocity area based on intelligent visual analysis, characterized in that: include: Continuously collecting images of the target water area to obtain dynamic images of the flow on the surface of the water area, and performing image preprocessing on the video frames of the dynamic images; The optical flow estimation algorithm of convolutional neural network is used to directly compare the global feature similarity through feature extraction, feature matching and optical flow optimization, and output the optical flow field data of the water surface velocity; The feature extraction operation includes: Input two consecutive frames of images and Downsampled to Three scales are used to extract features at different resolutions. The downsampled feature map is recorded as and , ； ； in, Indicates the scale level, represents the preprocessed image at time t, represents the preprocessed image at time t+1; The downsampled feature map and Normalized respectively. ； ； in, express The L2 norm of express The feature map after L2 normalization, express The L2 norm of express Feature map after L2 normalization; The feature matching operation includes: Based on the normalized feature map and , obtain a global correlation matrix C, the calculation of the global correlation matrix C includes, ； Among them, i represents the feature map after normalization The pixel index of , j represents the normalized feature map The pixel index of Optimize the global relevance matrix C using a Transformer-based global attention mechanism to obtain an attention weight A; ； The feature map normalized based on the attention weight A and the normalized time t+1 Get optimized matching features , ； The optical flow optimization operation includes: Generate the initial optical flow field using direct regression method , ； Among them, Regression is a fully connected layer responsible for mapping high-dimensional features to two-dimensional optical flow vector fields; The optical flow field is refined using a feature recursive optimization module, which consists of multiple refinement processes, each of which contains a feature residual network. In each refinement process, the feature residual network is used to extract the incremental update of the optical flow field output by the previous refinement process, and the incremental update is combined with the optical flow field output by the previous refinement process to obtain the optical flow field of the current refinement process. For the optical flow field output by the k+1th refinement process, , whose expression is: ； in, represents the feature residual network contained in the k+1th refinement process, represents the optical flow field output by the kth refinement process; After multiple refinement processes, the final high-precision optical flow field is obtained; After acquiring the optical flow field data, the pixel displacement information estimated by the optical flow is converted into a flow velocity field in actual physical units, and the regional distribution of the flow velocity on the water surface is analyzed.

2. The method for monitoring water surface velocity area based on intelligent visual analysis according to claim 1 is characterized in that: A high-resolution camera is used to continuously capture images of the target water area. During the process of continuously capturing images of the target water area, the image frame interval requirements, image resolution requirements, water ripple continuity requirements and high-resolution camera stability requirements need to be met.

3. The method for monitoring the surface velocity of water area based on intelligent visual analysis according to claim 2 is characterized in that: The image frame interval requirements specifically include: Set the high-resolution camera to a constant frame rate The image is collected, and the time interval between two adjacent frames is In order to ensure that the optical flow estimation model can accurately capture the local feature changes of the water surface flow, the frame rate selection needs to meet the following conditions: ； in, Indicates the maximum velocity of the water surface. Indicates the time interval between two adjacent frames. Indicates the maximum displacement allowed in the image; The image resolution requirements specifically include: ； ； in, Indicates the width of the image. Indicates the height of the image. L represents the actual width or length of the monitored water area, and r represents the actual distance corresponding to each pixel; The water pattern continuity requirements specifically include: Assume that a set of texture features in the image are located at the pixel positions of frame t and frame t+1 respectively. and , then it is necessary to meet: ； ； Among them, d represents the displacement vector, which is the target output of optical flow estimation, and L represents the actual width or length of the monitored water area; The high-resolution camera stability requirements specifically include: The high-resolution camera must remain stable during the acquisition process, and the optical axis must be perpendicular to the water surface or have a fixed tilt angle. The position of the high-resolution camera in the three-dimensional coordinate system is set to , then the position change of the high-resolution camera during the acquisition process must satisfy: , that is, the position of the high-resolution camera does not change.

4. A method for monitoring water surface velocity area based on intelligent visual analysis according to claim 1 or 3, characterized in that: The operation of performing image preprocessing on the video frame of the dynamic image includes: The video frames are subjected to denoising, grayscale conversion, and scale normalization processing in sequence; Use Gaussian filter to denoise the image and remove the interference of environmental noise on texture features. The operation of Gaussian filtering can be expressed as: ； Among them, x and y represent the pixel coordinates in the x-axis direction and y-axis direction of the image, u and v represent the pixel coordinates in the x-axis direction and y-axis direction of the Gaussian kernel, and k represents the x-axis and y-axis resolution of the Gaussian kernel. represents the original image, represents the filtered image, represents the Gaussian kernel function; Gaussian kernel function The calculation includes, ； in, represents the standard deviation of the Gaussian distribution, represents pi; The filtered image is grayed out, and the gray value The calculation formula includes: ； in, , , Respectively represent the pixel values ​​of the red, green, and blue channels in the filtered image; Normalize the pixel values ​​of the image that has been denoised and grayscaled to the range , to meet the input requirements of the optical flow estimation algorithm, ； in, represents the pixel value of the image after scale normalization, and Represent the minimum and maximum pixel values ​​of the image respectively.

5. The method for monitoring water surface velocity area based on intelligent visual analysis according to claim 1, characterized in that: include, The homography matrix H is used to map points on the image plane to the actual water surface coordinate system. The calculation of the homography matrix H includes: Set the internal parameter matrix of the high-resolution camera to K, and the external parameter is the rotation matrix R and the translation vector Represented, the calculation formula of the homography matrix H is: ； Where n represents the water surface normal vector, and d represents the straight-line distance from the camera to the water surface; ； in, , are the focal lengths in the x and y directions, , Represents the x and y pixel coordinates of the optical center on the image plane; The high-resolution camera external parameters describe the transformation relationship between the high-resolution camera coordinate system and the world coordinate system, usually composed of the rotation matrix R and the translation vector composition: ； ； R is a 3×3 orthogonal matrix used to describe the rotation of the high-resolution camera coordinate system relative to the world coordinate system; is a 3×1 column vector, used to describe the translation of the optical center of the high-resolution camera relative to the world coordinate system; , , Respectively x , y , The translational component in the direction; Through the homography matrix H, the pixel coordinates on the image are Can be mapped to the actual coordinates of the water surface : 。 6. The method for monitoring water surface velocity area based on intelligent visual analysis according to claim 5, characterized in that: include, The pixel displacement in the image plane is converted into the actual velocity of the water surface by using the pixel displacement information and image frame rate in the optical flow field; Set the optical flow estimation result to pixel displacement field ,in and Represents the displacement on the x-axis and y-axis respectively, and the time interval between frames is , the calculation formula of the actual physical flow rate is: ； ； ； in, Indicates the conversion ratio between pixels and actual coordinates; Finally, the velocity vector V of each pixel can be expressed as: ； in, and Respectively represent the flow velocity vector x Direction and y Directional weight; The size of the flow rate Calculated by vector modulus: 。 7. The method for monitoring water surface velocity area based on intelligent visual analysis according to claim 6, characterized in that: The analysis of the regional distribution of surface velocity of water bodies includes, Based on the number of pixels N within the ROI and the pixel index position in the ROI The corresponding flow rate Average flow rate , ； Calculate the maximum flow velocity in the area and minimum value , ； ； Divide the flow rate into K intervals at uniform intervals, and the number of flow rates in each interval It is expressed as: ； Where N represents the number of pixels in the ROI. represents the pixel index in the ROI, Represents the indicator function. When V satisfies When the condition The value of is 1, otherwise it is 0; , They represent the minimum and maximum values ​​of the kth interval respectively.

Images