Intelligent monitoring method for river surface flow velocity based on video image

Abstract

The present application relates to the technical field of video compression, and particularly relates to a river surface flow rate intelligent monitoring method based on video images, comprising: acquiring a connected domain formed by edge pixel points in adjacent frame river images, obtaining a matching area according to a characteristic value matching relationship between the connected domains, further obtaining a second reliability of the matching area, and obtaining standard pixel points in the river images according to the river images and corresponding sampling images, correcting the river images by using the standard pixel points to obtain new river videos, compressing the river videos in new data packets and transmitting the river videos to a server end, and intelligently detecting the river surface flow rate by using the server. The present application retains important feature points in the river videos, corrects the river videos to obtain new river videos, so that the compression effect of the new river videos is better, and the flow rate is better monitored after decompression, important information is prevented from being lost due to compression, and the video transmission efficiency and flow rate monitoring accuracy are improved.

Description

Technical Field

[0001] This invention relates to the field of video compression technology, and more specifically to a method for intelligent monitoring of river surface flow velocity based on video images. Background Technology

[0002] When using video images to intelligently monitor the flow velocity of a river surface, the first step is to compress the collected video images of the river surface and transmit them to the intelligent flow velocity monitoring server, which then uses the server to obtain the flow velocity of the river surface.

[0003] In existing methods, video images used for river surface velocity monitoring are often compressed directly using the ZIP compression algorithm. However, since ZIP compression is a lossy compression method, the decompressed video image data on the server side may be lost or corrupted, resulting in the loss of some important information that reflects the river surface velocity. In addition, when there is little repetitive information in the video image, the compression time of the ZIP compression algorithm becomes longer, which is not conducive to the real-time compression and transmission of video image data packets to the server, thus reducing monitoring efficiency. Summary of the Invention

[0004] This invention provides an intelligent monitoring method for river surface flow velocity based on video images to solve existing problems.

[0005] The intelligent monitoring method for river surface flow velocity based on video images of the present invention adopts the following technical solution:

[0006] This invention provides a method for intelligent monitoring of river surface flow velocity based on video images, the method comprising the following steps:

[0007] River video is formed by acquiring consecutive frames of river images;

[0008] The connected regions formed by edge pixels in any river image are denoted as edge connected regions. Based on the coordinates of all pixels in any edge connected region, the edge feature values ​​corresponding to any pixel in the edge connected region are obtained. The edge feature values ​​of any two edge connected regions in the throttling images of adjacent frames are matched to obtain several matching regions.

[0009] The edge connected components in the river image that do not belong to the matching region are denoted as non-corresponding regions. The edge connected components in any frame of the river image that belong to the matching region are denoted as sub-matching regions. The second confidence level of the matching region corresponding to the sub-matching region is obtained based on the distance between the sub-matching region and the non-corresponding region in the river image. The river image is sampled to obtain several sampled images. Based on the pixels of the sub-matching regions in the river image and the sampled images, key points to be retained and their corresponding descriptors are obtained. It is determined whether the key points to be retained are key points in the river image and the sampled images, and the attribute sequence of the key points to be retained is obtained. Based on the attribute sequence, the descriptors of the key points to be retained, and the second confidence level of the sub-matching region where the key points to be retained are obtained, the independence of the key points to be retained is obtained. Based on the independence of the key points to be retained and the distance between the pixels in the river image and the key points to be retained, the independence of the pixels in the river image that are not in the matching region is obtained. The pixel values ​​of the pixels in all river images in the river video are corrected using the independence to obtain a new river video.

[0010] The video of the new river is compressed and transmitted to enable monitoring of the river surface flow velocity.

[0011] Furthermore, the connected component formed by edge pixels in an arbitrary river image is denoted as the edge connected component. Based on the coordinates of all pixels within the edge connected component, the edge feature value corresponding to any pixel in the edge connected component is obtained. The specific steps include the following:

[0012] First, the edge image corresponding to any frame of river image in the data packet is obtained by using the Canny edge detection algorithm. Then, the edge image is analyzed by connected component analysis to obtain multiple connected components in the edge image, which are denoted as edge connected components.

[0013] Then, the coordinates of all pixels within any edge connected region are used as input to the principal component analysis algorithm to obtain the feature values ​​of all pixels in the edge connected region, which are denoted as edge feature values.

[0014] Furthermore, the specific steps for matching the edge feature values ​​of any two edge connected components in the throttling images of adjacent frames to obtain several matching regions are as follows:

[0015] First, obtain the minimum edge feature value in any edge connected region. Starting from the pixel corresponding to the minimum edge feature value, obtain the sequence of edge feature values ​​of all pixels in the edge connected region in a clockwise direction. This sequence is denoted as the feature value sequence. Thus, any river image contains several feature value sequences, and any edge connected region in the river image corresponds to one feature value sequence.

[0016] Then, taking the sequence of eigenvalues ​​corresponding to any edge connected component in any frame of the river image as a node, the th... The nodes corresponding to all feature value sequences of the river image in frame 1 together form the left node, and the first... The nodes corresponding to all feature sequences of the river image frame together form the right node. The relationship between any node in the left node and any node in the right node is recorded as an edge. The cosine similarity of the feature value sequences corresponding to any node in the left node and any node in the right node is used as the edge value of the corresponding edge. The KM algorithm is used to match the nodes in the left node and the nodes in the right node. The nodes that match each other are recorded as a matching node pair. The two edge connected regions corresponding to the matching node pair are recorded as the initial matching region.

[0017] Finally, for any initial matching region, obtain a solid mask with the same size and contour as the two edge connected regions of the initial matching region. The pixel value of the pixels in the mask is 1. Extract the internal regions included by the two edge connected regions of the initial matching region in the adjacent frame, and denote them as matching internal regions. Place the two centroids corresponding to the two matching internal regions overlapping. Rotate any matching internal region angle by angle with the centroid as the center point and clockwise. Obtain the feature vector corresponding to the maximum feature value in the edge connected regions of the two matching internal regions, and denote it as the target feature vector. Stop the rotation when the target feature vectors of the two matching internal regions have the same direction. Obtain the sequence formed by the pixel values ​​of all row pixels of the matched internal region after rotation, and denote it as the first grayscale sequence. Obtain the sequence formed by the pixel values ​​of all row pixels of the other matching internal region without rotation, and denote it as the second grayscale sequence. Use the DTW algorithm to obtain the similarity between the first grayscale sequence and the second grayscale sequence, and denote it as the matching similarity. When the matching similarity is greater than the preset matching similarity threshold, the initial matching region corresponding to the two matching internal regions is denoteed as the matching region.

[0018] Furthermore, the specific steps for obtaining the second confidence level of the matching region corresponding to the sub-matching region based on the distance between the sub-matching region and the non-corresponding region in the river image are as follows:

[0019] First, obtain the mean of the Euclidean distances between all pixels in the sub-matching region and all pixels in all non-corresponding regions, and denote it as the distance between the sub-matching region and all non-corresponding regions.

[0020] Then, the distance between any sub-matching region and all non-corresponding regions in any frame of the river image is used as the input of the exponential decay function, and the output of the exponential decay function is recorded as the first confidence of the sub-matching region. The first confidence of each of the two sub-matching regions of the matching region is obtained, and the average of the two first confidences of the two sub-matching regions of the matching region is subtracted from 1 and recorded as the second confidence of the matching region. Thus, the two sub-matching regions of a matching region correspond to the same second confidence.

[0021] Furthermore, the process of sampling the river image to obtain several sampled images, and obtaining retained key points and corresponding descriptors based on the pixels of the sub-matching regions in the river image and the sampled images, includes the following specific steps:

[0022] First, using a size of The window iterates through the mean sampling of any river image multiple times to obtain multiple sampled images. The SIFT algorithm is used to obtain all key points in the sampled images and the descriptors corresponding to the key points. Key points located in the sub-matching region are recorded as retained key points, and several retained key points and their corresponding descriptors are obtained.

[0023] Furthermore, the specific steps involved in determining whether the retained keypoints are keypoints in the river image and the sampled image, obtaining the attribute sequence of the retained keypoints, and obtaining the independence of the retained keypoints based on the attribute sequence, the descriptors of the retained keypoints, and the second confidence level of the sub-matching region where the retained keypoints are located are as follows:

[0024] First, obtain the pixels at the same location in any river image and several corresponding sample images, and denote them as target pixels. When the target pixel is a key point to be preserved in the river image or sample image, use 1 as the attribute label of the target pixel; when the target pixel is not a key point to be preserved in the river image or sample image, use 0 as the attribute label of the target pixel. Thus, the target pixel corresponds to one attribute label in each of the river image and sample images, and the target pixel corresponds to several attribute labels. The sequence formed by all the attribute labels corresponding to the target pixel is denoted as the attribute sequence of the target pixel. Obtain the attribute sequence of all pixels in the river image.

[0025] Then, based on the attribute sequence and the descriptor that retains key points, independent factors and independent parameters are obtained respectively;

[0026] Finally, the specific calculation method for preserving the independence of key points is as follows:

[0027]

[0028] in, This indicates that the independence of key points is preserved. This indicates the second level of confidence in retaining the sub-matching region described by the key point. This indicates the number of attribute labels with a value of 1 in the attribute sequence that retains key points; This indicates the number of elements in the attribute sequence that retains key points; This indicates the sequence of attributes that retains key points, specifically the first... The independent factor corresponding to each attribute label being 1; This indicates the sequence of attributes that retains key points, specifically the first... The independent parameter corresponding to each attribute tag being 1.

[0029] Furthermore, the specific methods for obtaining the independent factors and independent parameters are as follows:

[0030] The river image or sampled image corresponding to any attribute label of 1 in the attribute sequence of any preserved keypoint in the river image is denoted as the target image. The descriptor corresponding to the preserved keypoint in the target image is denoted as the target descriptor. The average cosine similarity between the target descriptor and the descriptors of all other preserved keypoints in the target image is denoted as the independent factor corresponding to the attribute label of the preserved keypoint being 1. The ratio of the ordinal number of any attribute label of 1 in the attribute sequence of any preserved keypoint in the river image to the number of elements in the attribute sequence is denoted as the independent parameter corresponding to the attribute label of the preserved keypoint being 1.

[0031] Furthermore, the specific steps for obtaining the independence of pixels in the river image that are not in the matching region based on the independence of the retained key points and the distance between the pixels in the river image and the retained key points are as follows:

[0032] Pixels not in the matching region are identified as the first special pixel. Keypoints that are kept with the smallest distance between the first special pixel and the keypoints are identified as the second special pixel. The Euclidean distance between the first and second special pixels is identified as the special distance. The special distance is input into the exponential decay function. The output of the exponential decay function is identified as the special parameter of the first special pixel. The special parameter of the first special pixel is multiplied by the independence of the corresponding second special pixel. The product is identified as the independence of the first special pixel. Thus, the independence of all pixels not in the matching region in the river image is obtained.

[0033] Furthermore, the specific steps for correcting the pixel values ​​of all river images in the river video using independence to obtain a new river video are as follows:

[0034] First, key points with independence greater than a preset independence threshold are recorded as standard pixels, and pixels in the river image other than standard pixels are recorded as non-standard pixels.

[0035] Then, the pixel values ​​of the standard pixels are used as input for bilinear interpolation to obtain the interpolated values ​​corresponding to the non-standard pixels. The specific calculation method for the corrected pixel values ​​of any non-standard pixel is as follows:

[0036]

[0037] in, This represents the corrected pixel value corresponding to a non-standard pixel. Indicates the independence of non-standard pixels. This represents the pixel value of a pixel in a river image. This represents the insertion value corresponding to the pixel.

[0038] Finally, obtain the corrected pixel values ​​corresponding to all non-standard pixels in the river image of any frame in the river video, record the river image formed after correcting the pixel values ​​of the pixels as the new river image, and record the video formed by all consecutive frames of the new river images as the new river video.

[0039] Furthermore, the specific steps involved in compressing and transmitting the new river video to achieve river surface velocity monitoring are as follows:

[0040] First, the new data packet containing the new river video is compressed and encoded using the ZIP compression algorithm to obtain a compressed packet;

[0041] Then, the compressed package is transmitted to the server using a 5G network;

[0042] Finally, the compressed package is decompressed using a program on the server side, and a model is obtained using the relevant flow velocity calculation formula in the program. The decompressed river video is then used for intelligent monitoring of the river surface flow velocity to obtain the river surface flow velocity, and the monitoring results are visualized.

[0043] The beneficial effects of the technical solution of this invention are as follows: It uses feature value sequences to match edge connected regions in adjacent frames, avoiding the problem of unsatisfactory matching results caused by directly describing the shape and contour of edge connected regions using sequence of edge pixels; it calculates the confidence level of 1 by measuring the distance between each corresponding region and non-corresponding regions, avoiding the influence of the force of bright white regions on the flow of objects, which often leads to significant distortion in the measured flow velocity; it preserves important feature points in the video that reflect flow velocity and further corrects the video to obtain a new river video, resulting in better compression and better flow velocity monitoring after decompression, thus avoiding the loss of important information while greatly improving transmission efficiency and flow velocity monitoring accuracy. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1This is a flowchart illustrating the steps of the intelligent monitoring method for river surface flow velocity based on video images according to the present invention. Detailed Implementation

[0046] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the intelligent monitoring method for river surface flow velocity based on video images proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0048] The specific scheme of the intelligent monitoring method for river surface flow velocity based on video images provided by the present invention will be described in detail below with reference to the accompanying drawings.

[0049] Please see Figure 1 The diagram illustrates a flowchart of a video image-based intelligent monitoring method for river surface flow velocity according to an embodiment of the present invention. The method includes the following steps:

[0050] Step S001: Collect video of the river.

[0051] Cameras are set up at fixed positions and angles to capture video images of the river surface, which are recorded as river videos. Single frames of the river videos are recorded as river images.

[0052] To facilitate the compressed transmission of river videos, several frames of river video are combined into a single data packet for analysis and compression, resulting in a data packet containing several consecutive frames of river images.

[0053] At this point, the river video was obtained.

[0054] Step S002: Obtain the connected components formed by edge pixels in adjacent frame river images, and obtain the matching region based on the feature value matching relationship between the connected components.

[0055] River surface velocity monitoring is often obtained by observing the movement of feature points on moving objects on the river surface, such as floating objects and buoys. Since the position of the same object may be different in different frames, and there may be multiple objects in the river, it is necessary to first obtain the same object in different frames, and then calculate the river surface velocity based on the movement of its feature points.

[0056] In addition, due to the motion of objects, the same object may exhibit visual deformation in consecutive frames of river images, meaning that the edges of the object change. Directly using the edges to describe the shape and contour of an object cannot achieve good matching for the same object in consecutive frames. However, by obtaining the feature values ​​of the edges, which reflect the projection length of the edges, the error tolerance is greater, and when compared with the object that has undergone visual deformation in the next frame, the probability of a successful match is higher.

[0057] Step (1): First, the edge image corresponding to any frame of river image in the data packet is obtained by using the Canny edge detection algorithm. The edge image is then analyzed by connected component analysis to obtain multiple connected components in the edge image, which are denoted as edge connected components.

[0058] Then, the coordinates of all pixels within any edge connected region are used as input to the principal component analysis algorithm to obtain the feature values ​​of all pixels in the edge connected region, which are denoted as edge feature values.

[0059] Finally, obtain the minimum edge feature value in any edge connected region. Starting from the pixel corresponding to the minimum edge feature value, obtain the sequence of edge feature values ​​of all pixels in the edge connected region in a clockwise direction. This sequence is denoted as the feature value sequence. Thus, any river image contains several feature value sequences, and any edge connected region in the river image corresponds to one feature value sequence.

[0060] The feature value sequence can represent the projected length of an object in different directions, and since it is along the edge direction, it can accurately describe the shape contour of the edge connected region.

[0061] Step (2): First, take the sequence of feature values ​​corresponding to any edge connected component in any frame of the river image as a node, then the th... The nodes corresponding to all feature value sequences of the river image in frame 1 together form the left node, and the first... The nodes corresponding to all feature sequences of the river image frame together form the right node. The relationship between any node in the left node and any node in the right node is recorded as an edge. The cosine similarity between the feature value sequences corresponding to any node in the left node and any node in the right node is used as the edge value of the corresponding edge.

[0062] Then, the KM algorithm is used to match the nodes in the left node with the nodes in the right node. The nodes that match each other are recorded as matching node pairs, and the two edge connected components corresponding to the matching node pairs are recorded as the initial matching regions.

[0063] Finally, for any initial matching region, obtain a solid mask with the same size and contour as the two edge connected regions of the initial matching region. The pixel value of the pixels in the mask is 1. Extract the internal regions included by the two edge connected regions of the initial matching region in the adjacent frame, and denote them as matching internal regions. Place the two centroids corresponding to the two matching internal regions overlapping. Rotate any matching internal region angle by angle with the centroid as the center point and clockwise. Obtain the feature vector corresponding to the maximum feature value in the edge connected regions of the two matching internal regions, and denote it as the target feature vector. Stop the rotation when the target feature vectors of the two matching internal regions have the same direction. Obtain the sequence formed by the pixel values ​​of all row pixels of the matched internal region after rotation, and denote it as the first grayscale sequence. Obtain the sequence formed by the pixel values ​​of all row pixels of the other matching internal region without rotation, and denote it as the second grayscale sequence. Use the DTW algorithm to obtain the similarity between the first grayscale sequence and the second grayscale sequence, and denote it as the matching similarity. When the matching similarity is greater than the preset matching similarity threshold, the initial matching region corresponding to the two matching internal regions is denoteed as the matching region.

[0064] It should be noted that the preset matching similarity threshold is 0.7 based on experience, but it can be adjusted according to specific circumstances. This embodiment does not impose any specific limitations.

[0065] At this point, the matching region has been obtained.

[0066] Step S003: Obtain the second confidence level of the matching region based on the distance between regions, and obtain the standard pixels in the river image based on the river image and the corresponding sampled image. Use the standard pixels to correct the river image to obtain a new river video.

[0067] Since the waves generated by the rushing river appear as bright white connected regions in the image, and because the waves vary greatly and their shapes and outlines are not fixed, there is no corresponding matching region when matching the edge connected regions in step S002. The closer an object is to the region corresponding to the wave, the more its movement is affected not only by the flow velocity but also by the force exerted by the wave, resulting in a large distortion in the measured river surface flow velocity.

[0068] Step (1): First, denote the edge connected regions in the river image that do not belong to the matching region as non-corresponding regions;

[0069] The non-corresponding areas are mostly white connected regions on the river surface caused by waves, etc.

[0070] Then, the edge connected components belonging to the matching region in any frame of the river image are denoted as sub-matching regions; the distance between each sub-matching region and all non-corresponding regions in any frame of the river image is obtained. The specific method is to obtain the mean of the Euclidean distance between all pixels in the sub-matching region and all pixels in all non-corresponding regions, and denote it as the distance between the sub-matching region and all non-corresponding regions.

[0071] The larger the mean, the less the sub-matching region is affected by the non-corresponding region, that is, the less the objects on the river surface are affected by the waves;

[0072] Finally, the distance between any sub-matching region and all non-corresponding regions in any frame of river image is used as the input of the exponential decay function, and the output of the exponential decay function is recorded as the first confidence of the sub-matching region. The first confidence of each of the two sub-matching regions of the matching region is obtained, and the average of the two first confidences of the two sub-matching regions of the matching region is subtracted from 1 and recorded as the second confidence of the matching region. Then, the two sub-matching regions of a matching region correspond to the same second confidence.

[0073] The greater the distance between the sub-matching region and all non-corresponding regions, the smaller the impact of the non-corresponding regions on the sub-matching region, and the greater the credibility of the sub-matching region.

[0074] Step (2): Since this embodiment requires matching by using feature points in adjacent frames of river images, and since the feature points used for matching are often edge pixels, when there are edge pixels in the river image, the descriptors of edge pixels are significantly different from the descriptors of other edge pixels, reflecting that the corresponding edge pixels are highly independent. When matching adjacent frames of river images, using these edge pixels for matching reduces the probability of errors in the matching results, thereby obtaining highly accurate flow velocity monitoring results.

[0075] In addition, the independence refers to the degree of difference between a feature point and other feature points. The greater the independence, the greater the difference between feature points at different scales. The greater the scale, the greater the independence of the feature point.

[0076] First, using a size of The window iterates through the mean sampling of any river image multiple times to obtain multiple sampled images. The SIFT algorithm is used to obtain all key points in the sampled images and the descriptors corresponding to the key points. Key points located in the sub-matching region are recorded as retained key points, and several retained key points and the descriptors corresponding to the retained key points are obtained.

[0077] It should be noted that, in this embodiment, the size of the window used for mean sampling of the river image is preset based on experience. The size can be adjusted according to the actual situation; this embodiment does not impose a specific limitation.

[0078] It should be noted that the SIFT algorithm is an existing algorithm, so this embodiment will not elaborate on it further.

[0079] Then, obtain the pixels at the same location in any river image and several corresponding sampled images, and denote them as target pixels. When the target pixel is a key point to be preserved in the river image or sampled image, use 1 as the attribute label of the target pixel; when the target pixel is not a key point to be preserved in the river image or sampled image, use 0 as the attribute label of the target pixel. Thus, the target pixel corresponds to one attribute label in each of the river image and sampled images, and the target pixel corresponds to several attribute labels. The sequence formed by all the attribute labels corresponding to the target pixel is denoteed as the attribute sequence of the target pixel; obtain the attribute sequence of all pixels in the river image.

[0080] It should be noted that the key points preserved in the river image are a subset of the river image's pixels, and therefore, the key points also have corresponding attribute sequences.

[0081] Finally, the river image or sampled image corresponding to any attribute label of 1 in the attribute sequence of any preserved keypoint in the river image is denoted as the target image. The descriptor corresponding to the preserved keypoint in the target image is denoted as the target descriptor. The average cosine similarity between the target descriptor and the descriptors of all other preserved keypoints in the target image is denoted as the independent factor corresponding to the attribute label of the preserved keypoint being 1. The ratio of the ordinal number in the attribute sequence corresponding to any attribute label of 1 in the attribute sequence of any preserved keypoint in the river image to the number of elements in the attribute sequence is denoted as the independent parameter corresponding to the attribute label of the preserved keypoint being 1.

[0082] Based on the second confidence level of the sub-matching region with preserved key points in the river image, and the attribute sequence of the preserved key points, the independence of any preserved key point is obtained. The specific calculation method is as follows:

[0083]

[0084] in, This indicates that the independence of key points is preserved. This indicates the second level of confidence in retaining the sub-matching region described by the key point. This indicates the number of attribute labels with a value of 1 in the attribute sequence that retains key points; This indicates the number of elements in the attribute sequence that retains key points; This indicates the sequence of attributes that retains key points, specifically the first... The independent factor corresponding to each attribute label being 1; This indicates the sequence of attributes that retains key points, specifically the first... The independent parameter corresponding to each attribute tag being 1.

[0085] In addition, pixels not in the matching region are identified as the first special pixel, and the keypoints that are kept with the smallest distance between the first special pixel and the keypoints are identified as the second special pixel. The Euclidean distance between the first and second special pixels is identified as the special distance. The special distance is input into the exponential decay function, and the output of the exponential decay function is identified as the special parameter of the first special pixel. The special parameter of the first special pixel is multiplied by the independence of the corresponding second special pixel, and the product is identified as the independence of the first special pixel. Thus, the independence of all pixels not in the matching region in the river image is obtained.

[0086] The greater the second confidence of the sub-matching region to which the key point belongs, the greater the independence of the key point. The greater the proportion of elements with attribute label 1 in the attribute sequence corresponding to the key point, the greater the independence of the key point. At the same time, the greater the difference of the key point at a larger scale, the greater its independence.

[0087] At this point, the independence of all pixels in the river image is obtained.

[0088] Step (3): The pixel values ​​of key points with high independence remain unchanged, while for key points with low independence, they are corrected by interpolation, so that the regularity of the pixels in the river video after correction is stronger, which is convenient for data compression.

[0089] First, key points with independence greater than a preset independence threshold are recorded as standard pixels, and pixels in the river image other than standard pixels are recorded as non-standard pixels.

[0090] It should be noted that the independence threshold is an empirical preset value, usually preset to 0.7, which can be adjusted according to the actual situation. This embodiment does not impose any specific limitations.

[0091] Then, the pixel values ​​of the standard pixels are used as input for bilinear interpolation. Bilinear interpolation is then used to obtain the interpolated values ​​for the non-standard pixels. The pixel values ​​of the standard pixels are kept unchanged, while the pixel values ​​of the non-standard pixels are corrected to obtain the corrected pixel values ​​for any non-standard pixel. The specific calculation method is as follows:

[0092]

[0093] in, This represents the corrected pixel value corresponding to a non-standard pixel. Indicates the independence of non-standard pixels. This represents the pixel value of a pixel in a river image. This represents the insertion value corresponding to the pixel.

[0094] Finally, the corrected pixel values ​​corresponding to all non-standard pixels in the river image of any frame in the river video are obtained. After the pixel values ​​of the river image are corrected, the regularity between the pixel values ​​becomes stronger, which facilitates the subsequent compression processing of the river video formed by the continuous frame river images. The river image formed after the pixel values ​​of the corrected pixels are recorded as the new river image, and the video formed by all the new river images of the continuous frames is recorded as the new river video.

[0095] At this point, a new data packet formed from the new river video is obtained.

[0096] Step S004: Compress the river video in the new data packet and transmit it to the server, using the server to intelligently detect the surface flow velocity of the river.

[0097] First, the new data packet containing the new river video is compressed and encoded using the ZIP compression algorithm to obtain a compressed packet;

[0098] Then, the compressed package is transmitted to the server using a 5G network;

[0099] Finally, the compressed package is decompressed using a program on the server side, and a model is obtained using the relevant flow velocity calculation formula in the program. The decompressed river video is then used for intelligent monitoring of the river surface flow velocity to obtain the river surface flow velocity, and the monitoring results are visualized.

[0100] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent monitoring of river surface flow velocity based on video images, characterized in that, The method comprises the following steps: Obtain a river video formed by continuous frames of river images; Obtain a connected domain formed by edge pixel points in any river image, and obtain edge feature values corresponding to any pixel point in the connected domain according to coordinates of all pixel points in the connected domain, including: first, obtain an edge image corresponding to any frame of river image in the data packet by using a Canny edge detection algorithm, and obtain a plurality of connected domains in the edge image by performing connected domain analysis on the edge image, and the connected domains are denoted as edge connected domains; then, input coordinates of all pixel points in any edge connected domain into a principal component analysis algorithm as input, and obtain feature values of all pixel points in the edge connected domain, and the feature values are denoted as edge feature values; match the edge feature values of any two edge connected domains in the throttle images of adjacent frames to obtain a plurality of matching regions; Denote edge connected domains in the river image that do not belong to the matching regions as non-corresponding regions, and denote edge connected domains in any frame of river image that belong to the matching regions as sub-matching regions, and obtain a second confidence degree of the sub-matching regions corresponding to the matching regions according to distances between the sub-matching regions and the non-corresponding regions in the river image; Sample the river image to obtain a plurality of sampling images, obtain retained key points and corresponding descriptors according to pixel points of the sub-matching regions in the river image and the sampling images, determine whether the retained key points are key points in the river image and the sampling images, obtain an attribute sequence of the retained key points, obtain independence of the retained key points according to the attribute sequence, the descriptors of the retained key points and the second confidence degree of the sub-matching regions where the retained key points are located, and obtain independence of pixel points in the river image that are not in the matching regions according to the independence of the retained key points and distances between the pixel points and the retained key points, including: obtaining a pixel point that is not in the matching regions as a first special pixel point, obtaining a retained key point with the smallest distance between the first special pixel point and the retained key point as a second special pixel point, inputting a Euclidean distance corresponding to the first special pixel point and the second special pixel point into an exponential decay function, inputting an output of the exponential decay function into a special parameter of the first special pixel point, multiplying the special parameter of the first special pixel point and the independence of the corresponding second special pixel point, and obtaining the independence of the first special pixel point as a product result, and then obtaining the independence of all pixel points in the river image that are not in the matching regions; Correct pixel values of the pixel points in all river images in the river video by using the independence to obtain a new river video; Compress and transmit the new river video to realize river surface flow speed monitoring.

2. The video image based intelligent monitoring method of river surface flow velocity according to claim 1, characterized in that, The matching of the edge feature values of any two edge connected domains in the throttle images of adjacent frames to obtain a plurality of matching regions comprises the following specific steps: Firstly, the minimum edge eigenvalue in any edge connected domain is obtained, and a sequence formed by the edge eigenvalues of all pixel points in the edge connected domain in the clockwise direction is obtained from the pixel point corresponding to the minimum edge eigenvalue, and the sequence is denoted as an eigenvalue sequence. Any river image contains a plurality of eigenvalue sequences, and any edge connected domain in the river image corresponds to an eigenvalue sequence. Then, taking the feature value sequence corresponding to any edge connected domain in any frame river image as a node, taking the node corresponding to the feature value sequence of the river image of the first frame as a left side node, and taking the node corresponding to the feature value sequence of the river image of the second frame as a right side node, the relationship between any node in the left side node and any node in the right side node is recorded as an edge, and the cosine similarity of the feature value sequences corresponding to any node in the left side node and any node in the right side node is recorded as the edge value of the corresponding edge. Then, taking the feature value sequence corresponding to any edge connected domain in any frame river image as a node, taking the node corresponding to the feature value sequence of the river image of the first frame as a left side node, and taking the node corresponding to the feature value sequence of the river image of the second frame as a right side node, the relationship between any node in the left side node and any node in the right side node is recorded as an edge, and the cosine similarity of the feature value sequences corresponding to any node in the left side node and any node in the right side node is recorded as the edge value of the corresponding edge. Then, taking the feature value sequence corresponding to any edge connected domain in any frame river image as a node, taking the node corresponding to the feature value sequence of the river image of the first frame as a left side node, and taking the node corresponding to the feature value sequence of the river image of the second frame as a right side node, the relationship between any node in the left side node and any node in the right side node is recorded as an edge, and the cosine similarity of the feature value sequences corresponding to any node in the left side node and any node in the right side node is recorded as the edge value of the corresponding edge. Finally, for any primary matching region, a solid mask with the same size and contour as the two edge connected domains of the primary matching region is obtained, and the pixel value of the pixel point in the mask is 1. The internal regions of the two edge connected domains of the primary matching region in the adjacent frame are extracted, and the internal regions are denoted as matching internal regions. The two centroids corresponding to the two matching internal regions are placed in an overlapping manner, and any one of the matching internal regions is rotated at an angle with the centroid as the center point and the clockwise direction as the direction. The feature vector corresponding to the maximum eigenvalue in the edge connected domains corresponding to the two matching internal regions is obtained, and the feature vector is denoted as a target feature vector. When the directions corresponding to the target feature vectors of the two matching internal regions are the same during the rotation, the rotation is stopped. A sequence formed by the pixel values of all row pixel points of the matching internal region after rotation is denoted as a first gray sequence, and a sequence formed by the pixel values of all row pixel points of the other matching internal region without rotation is denoted as a second gray sequence. The similarity between the first gray sequence and the second gray sequence is obtained by using the DTW algorithm, and the similarity is denoted as a matching similarity. When the matching similarity is greater than a preset matching similarity threshold, the primary matching region corresponding to the two matching internal regions is denoted as a matching region.

3. The video image based intelligent monitoring method of river surface flow velocity according to claim 1, characterized in that, According to the distances between the sub-matching regions and the non-corresponding regions in the river image, the second credibility of the sub-matching region corresponding to the matching region is obtained, and the specific steps include the following: Firstly, the mean value of the Euclidean distances between all pixel points in the sub-matching region and all pixel points in all non-corresponding regions is obtained, and the mean value is denoted as the distance between the sub-matching region and all non-corresponding regions. Then, the distance between any sub-matching region and all non-corresponding regions in any frame of the river image is taken as the input of the exponential decay function, and the output of the exponential decay function is denoted as the first credibility of the sub-matching region. The first credibilities corresponding to the two sub-matching regions of the matching region are obtained, and the mean value of the two first credibilities of the two sub-matching regions of the matching region is subtracted by 1 to obtain the second credibility of the matching region. The two sub-matching regions of one matching region correspond to the same second credibility.

4. The video image based intelligent monitoring method of river surface flow velocity according to claim 1, characterized in that, The river image is sampled to obtain a plurality of sampling images, and the reserved key points and the corresponding descriptors are obtained according to the pixel points of the sub-matching regions in the river image and the sampling images, and the specific steps include the following: First, the mean value sampling is performed on the arbitrary river image multiple times using a window with a size of , and a plurality of sampling images are obtained. The SIFT algorithm is used to obtain all key points in the sampling images and descriptors corresponding to the key points. Key points located on a sub-matching region are recorded as reserved key points, and a plurality of reserved key points and descriptors corresponding to the reserved key points are obtained.

5. The video image based intelligent monitoring method of river surface flow velocity according to claim 1, characterized in that, The attribute sequence of the reserved key point is obtained by judging whether the reserved key point is a key point in the river image and the sampling image. The independence of the reserved key point is obtained according to the attribute sequence, the descriptor of the reserved key point, and the second credibility of the sub-matching region where the reserved key point is located, and the specific steps include the following: Firstly, the pixel points of the same position in any river image and corresponding sampling images are obtained, denoted as target pixel points, when the target pixel points are reserved key points in the river image or the sampling image, 1 is taken as the attribute label of the target pixel points, when the target pixel points are not reserved key points in the river image or the sampling image, 0 is taken as the attribute label of the target pixel points, each of the target pixel points in the river image or the sampling image corresponds to an attribute label, the target pixel points correspond to a plurality of attribute labels, the sequence formed by all the attribute labels corresponding to the target pixel points is denoted as the attribute sequence of the target pixel points, and the attribute sequences of all the pixel points in the river image are obtained; Then, the independent factor and the independent parameter are obtained according to the attribute sequence and the descriptor of the reserved key point respectively; Finally, the specific calculation method of the independence of the reserved key point is as follows: wherein, denotes the independence of the keypoint being retained, denotes the second confidence of the keypoint sub-matching region being retained, denotes the number of attribute labels being 1 in the attribute sequence of the keypoint being retained; denotes the number of elements in the attribute sequence of the keypoint being retained; denotes the independent factor corresponding to the attribute label being 1 in the attribute sequence of the keypoint being retained; denotes the independent factor corresponding to the attribute label being 1 in the attribute sequence of the keypoint being retained; denotes the independent factor corresponding to the attribute label being 1 in the attribute sequence of the keypoint being retained; denotes the independent factor corresponding to the attribute label being 1 in the attribute sequence of the keypoint being retained.

6. The video image based intelligent monitoring method of river surface flow velocity according to claim 5, characterized in that, The specific acquisition method of the independent factor and the independent parameter is as follows: When any attribute label in the attribute sequence of any reserved key point in the river image is 1, the corresponding river image or sampling image is denoted as a target image, the descriptor corresponding to the reserved key point in the target image is denoted as a target descriptor, the average value of the cosine similarity between the target descriptor and the descriptors of all other reserved key points in the target image is denoted as the independent factor corresponding to the reserved key point when the attribute label is 1, and the ratio of the ordinal number corresponding to the attribute sequence in the attribute sequence to the number of elements in the attribute sequence is denoted as the independent parameter corresponding to the reserved key point when the attribute label is 1.

7. The video image based intelligent monitoring method of river surface flow velocity according to claim 1, characterized in that, The pixel values of the pixel points in all river images in the river video are corrected by using the independence, and a new river video is obtained, including the following specific steps: Firstly, the reserved key points with the independence greater than the preset independence threshold are denoted as standard pixel points, and the pixel points other than the standard pixel points in the river image are denoted as non-standard pixel points; Then, the pixel value of the standard pixel point is taken as the input of the bilinear interpolation method, and the bilinear interpolation method is used to obtain the inserted value corresponding to the non-standard pixel point; the specific calculation method of the corrected pixel value of any non-standard pixel point is as follows: wherein, represents the corrected pixel value corresponding to the non-standard pixel point, represents the independence of the non-standard pixel point, represents the pixel value of the pixel point in the river image, represents the interpolation value corresponding to the pixel point; Finally, the corrected pixel values corresponding to all the non-standard pixel points in the river image of any frame in the river video are obtained, the pixel value of the pixel point after correction forms a new river image, and the video formed by all the continuous frames of the new river image is denoted as a new river video.

8. The video image based intelligent monitoring method of river surface flow velocity according to claim 1, characterized in that, The new river video is compressed and transmitted to realize river surface flow speed monitoring, including the following specific steps: Firstly, the new data packet containing the new river video is compressed and encoded by using the ZIP compression algorithm to obtain a compressed packet; Then, the compressed packet is transmitted to the server side by using the 5G network; Finally, the compressed packet is decompressed by using the program in the server side, and a model is obtained by using the related flow speed calculation formula in the program to intelligently monitor the river surface flow speed of the decompressed river video, the river surface flow speed is obtained, and the monitoring result is visualized.

Images