River surface flow velocity intelligent monitoring method based on video image

Abstract

The invention relates to the technical field of video compression, in particular to an intelligent river surface flow velocity monitoring method based on video images, which comprises the following steps: acquiring connected domains formed by edge pixel points in adjacent frames of river images, acquiring a matching area according to a characteristic value matching relationship between the connected domains, and further acquiring second credibility of the matching area, and obtaining standard pixel points in the river image according to the river image and the corresponding sampling image, correcting the river image by using the standard pixel points to obtain a new river video, compressing the river video in the new data packet and transmitting the compressed river video to a server side, and intelligently detecting the flow velocity of the surface of the river by using the server. According to the method, important feature points in the river video are reserved, and the river video is corrected to obtain the new river video, so that the compression effect of the new river video is better, the flow velocity is better monitored after decompression, and the video transmission efficiency and the flow velocity monitoring accuracy are improved while important information is prevented from being compressed and lost.

Description

Technical Field

[0001] This invention relates to the field of video compression technology, and more specifically to a method for intelligent monitoring of river surface flow velocity based on video images. Background Technology

[0002] When using video images to intelligently monitor the flow velocity of a river surface, the first step is to compress the collected video images of the river surface and transmit them to the intelligent flow velocity monitoring server, which then uses the server to obtain the flow velocity of the river surface.

[0003] In existing methods, video images used for river surface velocity monitoring are often compressed directly using the ZIP compression algorithm. However, since ZIP compression is a lossy compression method, the decompressed video image data on the server side may be lost or corrupted, resulting in the loss of some important information that reflects the river surface velocity. In addition, when there is little repetitive information in the video image, the compression time of the ZIP compression algorithm becomes longer, which is not conducive to the real-time compression and transmission of video image data packets to the server, thus reducing monitoring efficiency. Summary of the Invention

[0004] This invention provides an intelligent monitoring method for river surface flow velocity based on video images to solve existing problems.

[0005] The intelligent monitoring method for river surface flow velocity based on video images of the present invention adopts the following technical solution: This invention provides a method for intelligent monitoring of river surface flow velocity based on video images, the method comprising the following steps: River video is formed by acquiring consecutive frames of river images; The connected regions formed by edge pixels in any river image are denoted as edge connected regions. Based on the coordinates of all pixels in any edge connected region, the edge feature values ​​corresponding to any pixel in the edge connected region are obtained. The edge feature values ​​of any two edge connected regions in the throttling images of adjacent frames are matched to obtain several matching regions. The edge connected components in the river image that do not belong to the matching region are denoted as non-corresponding regions. The edge connected components in any frame of the river image that belong to the matching region are denoted as sub-matching regions. The second confidence level of the matching region corresponding to the sub-matching region is obtained based on the distance between the sub-matching region and the non-corresponding region in the river image. The river image is sampled to obtain several sampled images. Based on the pixels of the sub-matching regions in the river image and the sampled images, key points to be retained and their corresponding descriptors are obtained. It is determined whether the key points to be retained are key points in the river image and the sampled images, and the attribute sequence of the key points to be retained is obtained. Based on the attribute sequence, the descriptors of the key points to be retained, and the second confidence level of the sub-matching region where the key points to be retained are obtained, the independence of the key points to be retained is obtained. Based on the independence of the key points to be retained and the distance between the pixels in the river image and the key points to be retained, the independence of the pixels in the river image that are not in the matching region is obtained. The pixel values ​​of the pixels in all river images in the river video are corrected using the independence to obtain a new river video. The video of the new river is compressed and transmitted to enable monitoring of the river's surface flow velocity.

[0006] Furthermore, the connected component formed by edge pixels in an arbitrary river image is denoted as the edge connected component. Based on the coordinates of all pixels within the edge connected component, the edge feature value corresponding to any pixel in the edge connected component is obtained. The specific steps include the following: First, the edge image corresponding to any frame of river image in the data packet is obtained by using the Canny edge detection algorithm. Then, the edge image is analyzed by connected component analysis to obtain multiple connected components in the edge image, which are denoted as edge connected components. Then, the coordinates of all pixels within any edge connected region are used as input to the principal component analysis algorithm to obtain the feature values ​​of all pixels in the edge connected region, which are denoted as edge feature values.

[0007] Furthermore, the specific steps for matching the edge feature values ​​of any two edge connected components in the throttling images of adjacent frames to obtain several matching regions are as follows: First, obtain the minimum edge feature value in any edge connected region. Starting from the pixel corresponding to the minimum edge feature value, obtain the sequence of edge feature values ​​of all pixels in the edge connected region in a clockwise direction. This sequence is denoted as the feature value sequence. Thus, any river image contains several feature value sequences, and any edge connected region in the river image corresponds to one feature value sequence. Then, taking the sequence of eigenvalues ​​corresponding to any edge connected component in any frame of the river image as a node, the th... The nodes corresponding to all feature value sequences of the river image in frame 1 together form the left node, and the first... The nodes corresponding to all feature sequences of the river image frame together form the right node. The relationship between any node in the left node and any node in the right node is recorded as an edge. The cosine similarity of the feature value sequences corresponding to any node in the left node and any node in the right node is used as the edge value of the corresponding edge. The KM algorithm is used to match the nodes in the left node and the nodes in the right node. The nodes that match each other are recorded as a matching node pair. The two edge connected regions corresponding to the matching node pair are recorded as the initial matching region. Finally, for any initial matching region, obtain a solid mask with the same size and contour as the two edge connected regions of the initial matching region. The pixel value of the pixels in the mask is 1. Extract the internal regions included by the two edge connected regions of the initial matching region in the adjacent frame, and denote them as matching internal regions. Place the two centroids corresponding to the two matching internal regions overlapping. Rotate any matching internal region angle by angle with the centroid as the center point and clockwise. Obtain the feature vector corresponding to the maximum feature value in the edge connected regions of the two matching internal regions, and denote it as the target feature vector. Stop the rotation when the target feature vectors of the two matching internal regions have the same direction. Obtain the sequence formed by the pixel values ​​of all row pixels of the matched internal region after rotation, and denote it as the first grayscale sequence. Obtain the sequence formed by the pixel values ​​of all row pixels of the other matching internal region without rotation, and denote it as the second grayscale sequence. Use the DTW algorithm to obtain the similarity between the first grayscale sequence and the second grayscale sequence, and denote it as the matching similarity. When the matching similarity is greater than the preset matching similarity threshold, the initial matching region corresponding to the two matching internal regions is denoteed as the matching region.

[0008] Furthermore, the specific steps for obtaining the second confidence level of the matching region corresponding to the sub-matching region based on the distance between the sub-matching region and the non-corresponding region in the river image are as follows: First, obtain the mean of the Euclidean distances between all pixels in the sub-matching region and all pixels in all non-corresponding regions, and denote it as the distance between the sub-matching region and all non-corresponding regions. Then, the distance between any sub-matching region and all non-corresponding regions in any frame of the river image is used as the input of the exponential decay function, and the output of the exponential decay function is recorded as the first confidence of the sub-matching region. The first confidence of each of the two sub-matching regions of the matching region is obtained, and the average of the two first confidences of the two sub-matching regions of the matching region is subtracted from 1 and recorded as the second confidence of the matching region. Thus, the two sub-matching regions of a matching region correspond to the same second confidence.

[0009] Furthermore, the process of sampling the river image to obtain several sampled images, and obtaining retained key points and corresponding descriptors based on the pixels of the sub-matching regions in the river image and the sampled images, includes the following specific steps: First, using a size of The window iterates through the mean sampling of any river image multiple times to obtain multiple sampled images. The SIFT algorithm is used to obtain all key points in the sampled images and the descriptors corresponding to the key points. Key points located in the sub-matching region are recorded as retained key points, and several retained key points and their corresponding descriptors are obtained.

[0010] Furthermore, the specific steps involved in determining whether the retained keypoints are keypoints in the river image and the sampled image, obtaining the attribute sequence of the retained keypoints, and obtaining the independence of the retained keypoints based on the attribute sequence, the descriptors of the retained keypoints, and the second confidence level of the sub-matching region where the retained keypoints are located are as follows: First, obtain the pixels at the same location in any river image and several corresponding sample images, and denote them as target pixels. When the target pixel is a key point to be preserved in the river image or sample image, use 1 as the attribute label of the target pixel; when the target pixel is not a key point to be preserved in the river image or sample image, use 0 as the attribute label of the target pixel. Thus, the target pixel corresponds to one attribute label in each of the river image and sample images, and the target pixel corresponds to several attribute labels. The sequence formed by all the attribute labels corresponding to the target pixel is denoted as the attribute sequence of the target pixel. Obtain the attribute sequence of all pixels in the river image. Then, based on the attribute sequence and the descriptor that retains the key points, independent factors and independent parameters are obtained respectively; Finally, the specific calculation method for preserving the independence of key points is as follows: in, This indicates that the independence of key points is preserved. This indicates the second level of confidence in retaining the sub-matching region described by the key point. This indicates the number of attribute labels with a value of 1 in the attribute sequence that retains key points; This indicates the number of elements in the attribute sequence that retains key points; This indicates the sequence of attributes that retains key points, specifically the first... The independent factor corresponding to each attribute label being 1; This indicates the sequence of attributes that retains key points, specifically the first... The independent parameter corresponding to each attribute tag being 1.

[0011] Furthermore, the specific methods for obtaining the independent factors and independent parameters are as follows: The river image or sampled image corresponding to any attribute label of 1 in the attribute sequence of any preserved keypoint in the river image is denoted as the target image. The descriptor corresponding to the preserved keypoint in the target image is denoted as the target descriptor. The average cosine similarity between the target descriptor and the descriptors of all other preserved keypoints in the target image is denoted as the independent factor corresponding to the attribute label of the preserved keypoint being 1. The ratio of the ordinal number of any attribute label of 1 in the attribute sequence of any preserved keypoint in the river image to the number of elements in the attribute sequence is denoted as the independent parameter corresponding to the attribute label of the preserved keypoint being 1.

[0012] Furthermore, the specific steps for obtaining the independence of pixels in the river image that are not in the matching region based on the independence of the retained key points and the distance between the pixels in the river image and the retained key points are as follows: Pixels not in the matching region are identified as the first special pixel. Keypoints that are kept with the smallest distance between the first special pixel and the keypoints are identified as the second special pixel. The Euclidean distance between the first and second special pixels is identified as the special distance. The special distance is input into the exponential decay function. The output of the exponential decay function is identified as the special parameter of the first special pixel. The special parameter of the first special pixel is multiplied by the independence of the corresponding second special pixel. The product is identified as the independence of the first special pixel. Thus, the independence of all pixels not in the matching region in the river image is obtained.

[0013] Furthermore, the specific steps for correcting the pixel values ​​of all river images in the river video using independence to obtain a new river video are as follows: First, key points with independence greater than a preset independence threshold are recorded as standard pixels, and pixels in the river image other than standard pixels are recorded as non-standard pixels. Then, the pixel values ​​of the standard pixels are used as input for bilinear interpolation to obtain the interpolated values ​​corresponding to the non-standard pixels. The specific calculation method for the corrected pixel values ​​of any non-standard pixel is as follows: in, This represents the corrected pixel value corresponding to a non-standard pixel. Indicates the independence of non-standard pixels. This represents the pixel value of a pixel in a river image. This represents the insertion value corresponding to the pixel. Finally, obtain the corrected pixel values ​​corresponding to all non-standard pixels in the river image of any frame in the river video, record the river image formed after correcting the pixel values ​​of the pixels as the new river image, and record the video formed by all consecutive frames of the new river images as the new river video.

[0014] Furthermore, the specific steps involved in compressing and transmitting the new river video to achieve river surface velocity monitoring are as follows: First, the new data packet containing the new river video is compressed and encoded using the ZIP compression algorithm to obtain a compressed packet; Then, the compressed package is transmitted to the server using a 5G network; Finally, the compressed package is decompressed using a program on the server side, and a model is obtained using the relevant flow velocity calculation formula in the program. The decompressed river video is then used for intelligent monitoring of the river surface flow velocity to obtain the river surface flow velocity, and the monitoring results are visualized.

[0015] The beneficial effects of the technical solution of this invention are as follows: It uses feature value sequences to match edge connected regions in adjacent frames, avoiding the problem of unsatisfactory matching results caused by directly describing the shape and contour of edge connected regions using sequence of edge pixels; it calculates the confidence level of 1 by measuring the distance between each corresponding region and non-corresponding regions, avoiding the influence of the force of bright white regions on the flow of objects, which often leads to significant distortion in the measured flow velocity; it preserves important feature points in the video that reflect flow velocity and further corrects the video to obtain a new river video, resulting in better compression and better flow velocity monitoring after decompression, thus avoiding the loss of important information while greatly improving transmission efficiency and flow velocity monitoring accuracy. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the steps of the intelligent monitoring method for river surface flow velocity based on video images according to the present invention. Detailed Implementation

[0018] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the intelligent monitoring method for river surface flow velocity based on video images proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0020] The specific scheme of the intelligent monitoring method for river surface flow velocity based on video images provided by the present invention will be described in detail below with reference to the accompanying drawings.

[0021] Please see Figure 1 The diagram illustrates a flowchart of a video image-based intelligent monitoring method for river surface flow velocity according to an embodiment of the present invention. The method includes the following steps: Step S001: Collect video of the river.

[0022] Cameras are set up at fixed positions and angles to capture video images of the river surface, which are recorded as river videos. Single frames of the river videos are recorded as river images. To facilitate the compressed transmission of river videos, several frames of river video are combined into a single data packet for analysis and compression, resulting in a data packet containing several consecutive frames of river images.

[0023] At this point, the river video was obtained.

[0024] Step S002: Obtain the connected components formed by edge pixels in adjacent frame river images, and obtain the matching region based on the feature value matching relationship between the connected components.

[0025] River surface velocity monitoring is often obtained by observing the movement of feature points on moving objects on the river surface, such as floating objects and buoys. Since the position of the same object may be different in different frames, and there may be multiple objects in the river, it is necessary to first obtain the same object in different frames, and then calculate the river surface velocity based on the movement of its feature points.

[0026] In addition, due to the motion of objects, the same object may exhibit visual deformation in consecutive frames of river images, meaning that the edges of the object change. Directly using the edges to describe the shape and contour of an object cannot achieve good matching for the same object in consecutive frames. However, by obtaining the feature values ​​of the edges, which reflect the projection length of the edges, the error tolerance is greater, and when compared with the object that has undergone visual deformation in the next frame, the probability of a successful match is higher.

[0027] Step (1): First, the edge image corresponding to any frame of river image in the data packet is obtained by using the Canny edge detection algorithm. The edge image is then analyzed by connected component analysis to obtain multiple connected components in the edge image, which are denoted as edge connected components. Then, the coordinates of all pixels within any edge connected region are used as input to the principal component analysis algorithm to obtain the feature values ​​of all pixels in the edge connected region, which are denoted as edge feature values. Finally, obtain the minimum edge feature value in any edge connected region. Starting from the pixel corresponding to the minimum edge feature value, obtain the sequence of edge feature values ​​of all pixels in the edge connected region in a clockwise direction. This sequence is denoted as the feature value sequence. Thus, any river image contains several feature value sequences, and any edge connected region in the river image corresponds to one feature value sequence.

[0028] The feature value sequence can represent the projected length of an object in different directions, and since it is along the edge direction, it can accurately describe the shape contour of the edge connected region.

[0029] Step (2): First, take the sequence of feature values ​​corresponding to any edge connected component in any frame of the river image as a node, then the th... The nodes corresponding to all feature value sequences of the river image in frame 1 together form the left node, and the first... The nodes corresponding to all feature sequences of the river image frame together form the right node. The relationship between any node in the left node and any node in the right node is recorded as an edge. The cosine similarity between the feature value sequences corresponding to any node in the left node and any node in the right node is used as the edge value of the corresponding edge. Then, the KM algorithm is used to match the nodes in the left node with the nodes in the right node. The nodes that match each other are recorded as matching node pairs, and the two edge connected components corresponding to the matching node pairs are recorded as the initial matching regions. Finally, for any initial matching region, obtain a solid mask with the same size and contour as the two edge connected regions of the initial matching region. The pixel value of the pixels in the mask is 1. Extract the internal regions included by the two edge connected regions of the initial matching region in the adjacent frame, and denote them as matching internal regions. Place the two centroids corresponding to the two matching internal regions overlapping. Rotate any matching internal region angle by angle with the centroid as the center point and clockwise. Obtain the feature vector corresponding to the maximum feature value in the edge connected regions of the two matching internal regions, and denote it as the target feature vector. Stop the rotation when the target feature vectors of the two matching internal regions have the same direction. Obtain the sequence formed by the pixel values ​​of all row pixels of the matched internal region after rotation, and denote it as the first grayscale sequence. Obtain the sequence formed by the pixel values ​​of all row pixels of the other matching internal region without rotation, and denote it as the second grayscale sequence. Use the DTW algorithm to obtain the similarity between the first grayscale sequence and the second grayscale sequence, and denote it as the matching similarity. When the matching similarity is greater than the preset matching similarity threshold, the initial matching region corresponding to the two matching internal regions is denoteed as the matching region.

[0030] It should be noted that the preset matching similarity threshold is 0.7 based on experience, but it can be adjusted according to specific circumstances. This embodiment does not impose any specific limitations.

[0031] At this point, the matching region has been obtained.

[0032] Step S003: Obtain the second confidence level of the matching region based on the distance between regions, and obtain the standard pixels in the river image based on the river image and the corresponding sampled image. Use the standard pixels to correct the river image to obtain a new river video.

[0033] Since the waves generated by the rushing river appear as bright white connected regions in the image, and because the waves vary greatly and their shapes and outlines are not fixed, there is no corresponding matching region when matching the edge connected regions in step S002. The closer an object is to the region corresponding to the wave, the more its movement is affected not only by the flow velocity but also by the force exerted by the wave, resulting in a large distortion in the measured river surface flow velocity.

[0034] Step (1): First, denote the edge connected regions in the river image that do not belong to the matching region as non-corresponding regions; The non-corresponding areas are mostly white connected regions on the river surface caused by waves, etc.

[0035] Then, the edge connected components belonging to the matching region in any frame of the river image are denoted as sub-matching regions; the distance between each sub-matching region and all non-corresponding regions in any frame of the river image is obtained. The specific method is to obtain the mean of the Euclidean distance between all pixels in the sub-matching region and all pixels in all non-corresponding regions, and denote it as the distance between the sub-matching region and all non-corresponding regions. The larger the mean, the less the sub-matching region is affected by the non-corresponding region, that is, the less the objects on the river surface are affected by the waves; Finally, the distance between any sub-matching region and all non-corresponding regions in any frame of river image is used as the input of the exponential decay function, and the output of the exponential decay function is recorded as the first confidence of the sub-matching region. The first confidence of each of the two sub-matching regions of the matching region is obtained, and the average of the two first confidences of the two sub-matching regions of the matching region is subtracted from 1 and recorded as the second confidence of the matching region. Then, the two sub-matching regions of a matching region correspond to the same second confidence. The greater the distance between the sub-matching region and all non-corresponding regions, the smaller the impact of the non-corresponding regions on the sub-matching region, and the greater the credibility of the sub-matching region.

[0036] Step (2): Since this embodiment requires matching by using feature points in adjacent frames of river images, and since the feature points used for matching are often edge pixels, when there are edge pixels in the river image, the descriptors of edge pixels are significantly different from the descriptors of other edge pixels, reflecting that the corresponding edge pixels are highly independent. When matching adjacent frames of river images, using these edge pixels for matching reduces the probability of errors in the matching results, thereby obtaining highly accurate flow velocity monitoring results. In addition, the independence refers to the degree of difference between a feature point and other feature points. The greater the independence, the greater the difference between feature points at different scales. The greater the scale, the greater the independence of the feature point.

[0037] First, using a size of The window iterates through the mean sampling of any river image multiple times to obtain multiple sampled images. The SIFT algorithm is used to obtain all key points in the sampled images and the descriptors corresponding to the key points. Key points located in the sub-matching region are recorded as retained key points, and several retained key points and the descriptors corresponding to the retained key points are obtained. It should be noted that, in this embodiment, the size of the window used for mean sampling of the river image is preset based on experience. The size can be adjusted according to the actual situation; this embodiment does not impose a specific limitation. It should be noted that the SIFT algorithm is an existing algorithm, so this embodiment will not elaborate on it further.

[0038] Then, obtain the pixels at the same location in any river image and several corresponding sampled images, and denote them as target pixels. When the target pixel is a key point to be preserved in the river image or sampled image, use 1 as the attribute label of the target pixel; when the target pixel is not a key point to be preserved in the river image or sampled image, use 0 as the attribute label of the target pixel. Thus, the target pixel corresponds to one attribute label in each of the river image and sampled images, and the target pixel corresponds to several attribute labels. The sequence formed by all the attribute labels corresponding to the target pixel is denoteed as the attribute sequence of the target pixel; obtain the attribute sequence of all pixels in the river image. It should be noted that the key points preserved in the river image are a subset of the river image's pixels, and therefore, the key points also have corresponding attribute sequences.

[0039] Finally, the river image or sampled image corresponding to any attribute label of 1 in the attribute sequence of any preserved keypoint in the river image is denoted as the target image. The descriptor corresponding to the preserved keypoint in the target image is denoted as the target descriptor. The average cosine similarity between the target descriptor and the descriptors of all other preserved keypoints in the target image is denoted as the independent factor corresponding to the attribute label of the preserved keypoint being 1. The ratio of the ordinal number in the attribute sequence corresponding to any attribute label of 1 in the attribute sequence of any preserved keypoint in the river image to the number of elements in the attribute sequence is denoted as the independent parameter corresponding to the attribute label of the preserved keypoint being 1. Based on the second confidence level of the sub-matching region with preserved key points in the river image, and the attribute sequence of the preserved key points, the independence of any preserved key point is obtained. The specific calculation method is as follows: in, This indicates that the independence of key points is preserved. This indicates the second level of confidence in retaining the sub-matching region described by the key point. This indicates the number of attribute labels with a value of 1 in the attribute sequence that retains key points; This indicates the number of elements in the attribute sequence that retains key points; This indicates the sequence of attributes that retains key points, specifically the first... The independent factor corresponding to each attribute label being 1; This indicates the sequence of attributes that retains key points, specifically the first... The independent parameter corresponding to each attribute tag being 1.

[0040] In addition, pixels not in the matching region are identified as the first special pixel, and the keypoints that are kept with the smallest distance between the first special pixel and the keypoints are identified as the second special pixel. The Euclidean distance between the first and second special pixels is identified as the special distance. The special distance is input into the exponential decay function, and the output of the exponential decay function is identified as the special parameter of the first special pixel. The special parameter of the first special pixel is multiplied by the independence of the corresponding second special pixel, and the product is identified as the independence of the first special pixel. Thus, the independence of all pixels not in the matching region in the river image is obtained.

[0041] The greater the second confidence of the sub-matching region to which the key point belongs, the greater the independence of the key point. The greater the proportion of elements with attribute label 1 in the attribute sequence corresponding to the key point, the greater the independence of the key point. At the same time, the greater the difference of the key point at a larger scale, the greater its independence.

[0042] At this point, the independence of all pixels in the river image is obtained.

[0043] Step (3): The pixel values ​​of key points with high independence remain unchanged, while for key points with low independence, they are corrected by interpolation, so that the regularity of the pixels in the river video after correction is stronger, which is convenient for data compression.

[0044] First, key points with independence greater than a preset independence threshold are recorded as standard pixels, and pixels in the river image other than standard pixels are recorded as non-standard pixels. It should be noted that the independence threshold is an empirical preset value, usually preset to 0.7, which can be adjusted according to the actual situation. This embodiment does not impose any specific limitations.

[0045] Then, the pixel values ​​of the standard pixels are used as input for bilinear interpolation. Bilinear interpolation is then used to obtain the interpolated values ​​for the non-standard pixels. The pixel values ​​of the standard pixels are kept unchanged, while the pixel values ​​of the non-standard pixels are corrected to obtain the corrected pixel values ​​for any non-standard pixel. The specific calculation method is as follows: in, This represents the corrected pixel value corresponding to a non-standard pixel. Indicates the independence of non-standard pixels. This represents the pixel value of a pixel in a river image. This represents the insertion value corresponding to the pixel.

[0046] Finally, the corrected pixel values ​​corresponding to all non-standard pixels in the river image of any frame in the river video are obtained. After the pixel values ​​of the river image are corrected, the regularity between the pixel values ​​becomes stronger, which facilitates the subsequent compression processing of the river video formed by the continuous frame river images. The river image formed after the pixel values ​​of the corrected pixels are recorded as the new river image, and the video formed by all the new river images of the continuous frames is recorded as the new river video.

[0047] At this point, a new data packet formed from the new river video is obtained.

[0048] Step S004: Compress the river video in the new data packet and transmit it to the server, using the server to intelligently detect the surface flow velocity of the river.

[0049] First, the new data packet containing the new river video is compressed and encoded using the ZIP compression algorithm to obtain a compressed packet; Then, the compressed package is transmitted to the server using a 5G network; Finally, the compressed package is decompressed using a program on the server side, and a model is obtained using the relevant flow velocity calculation formula in the program. The decompressed river video is then used for intelligent monitoring of the river surface flow velocity to obtain the river surface flow velocity, and the monitoring results are visualized.

[0050] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent monitoring of river surface flow velocity based on video images, characterized in that, The method includes the following steps: River video is formed by acquiring consecutive frames of river images; The connected regions formed by edge pixels in any river image are denoted as edge connected regions. Based on the coordinates of all pixels in any edge connected region, the edge feature values ​​corresponding to any pixel in the edge connected region are obtained. The edge feature values ​​of any two edge connected regions in the throttling images of adjacent frames are matched to obtain several matching regions. The edge connected components in the river image that do not belong to the matching region are denoted as non-corresponding regions. The edge connected components in any frame of the river image that belong to the matching region are denoted as sub-matching regions. The second confidence level of the matching region corresponding to the sub-matching region is obtained based on the distance between the sub-matching region and the non-corresponding region in the river image. The river image is sampled to obtain several sampled images. Based on the pixels of the sub-matching regions in the river image and the sampled images, key points to be retained and their corresponding descriptors are obtained. It is determined whether the key points to be retained are key points in the river image and the sampled images, and the attribute sequence of the key points to be retained is obtained. Based on the attribute sequence, the descriptors of the key points to be retained, and the second confidence level of the sub-matching region where the key points to be retained are obtained, the independence of the key points to be retained is obtained. Based on the independence of the key points to be retained and the distance between the pixels in the river image and the key points to be retained, the independence of the pixels in the river image that are not in the matching region is obtained. The pixel values ​​of the pixels in all river images in the river video are corrected using the independence to obtain a new river video. The video of the new river is compressed and transmitted to enable monitoring of the river surface flow velocity.

2. The intelligent monitoring method for river surface flow velocity based on video images according to claim 1, characterized in that, The process of obtaining the connected component formed by edge pixels in an arbitrary river image is denoted as the edge connected component. Based on the coordinates of all pixels within the edge connected component, the edge feature value corresponding to any pixel in the edge connected component is obtained. The specific steps include the following: First, the edge image corresponding to any frame of river image in the data packet is obtained by using the Canny edge detection algorithm. Then, the edge image is analyzed by connected component analysis to obtain multiple connected components in the edge image, which are denoted as edge connected components. Then, the coordinates of all pixels within any edge connected region are used as input to the principal component analysis algorithm to obtain the feature values ​​of all pixels in the edge connected region, which are denoted as edge feature values.

3. The intelligent monitoring method for river surface flow velocity based on video images according to claim 1, characterized in that, The specific steps for matching the edge feature values ​​of any two edge connected components in the throttling images of adjacent frames to obtain several matching regions are as follows: First, obtain the minimum edge feature value in any edge connected region. Starting from the pixel corresponding to the minimum edge feature value, obtain the sequence of edge feature values ​​of all pixels in the edge connected region in a clockwise direction. This sequence is denoted as the feature value sequence. Thus, any river image contains several feature value sequences, and any edge connected region in the river image corresponds to one feature value sequence. Then, taking the sequence of eigenvalues ​​corresponding to any edge connected component in any frame of the river image as a node, the th... The nodes corresponding to all feature value sequences of the river image in frame 1 together form the left node, and the first... The nodes corresponding to all feature sequences of the river image frame together form the right node. The relationship between any node in the left node and any node in the right node is recorded as an edge. The cosine similarity of the feature value sequences corresponding to any node in the left node and any node in the right node is used as the edge value of the corresponding edge. The KM algorithm is used to match the nodes in the left node and the nodes in the right node. The nodes that match each other are recorded as a matching node pair. The two edge connected regions corresponding to the matching node pair are recorded as the initial matching region. Finally, for any initial matching region, obtain a solid mask with the same size and contour as the two edge connected regions of the initial matching region. The pixel value of the pixels in the mask is 1. Extract the internal regions included by the two edge connected regions of the initial matching region in the adjacent frame, and denote them as matching internal regions. Place the two centroids corresponding to the two matching internal regions overlapping. Rotate any matching internal region angle by angle with the centroid as the center point and clockwise. Obtain the feature vector corresponding to the maximum feature value in the edge connected regions of the two matching internal regions, and denote it as the target feature vector. Stop the rotation when the target feature vectors of the two matching internal regions have the same direction. Obtain the sequence formed by the pixel values ​​of all row pixels of the matched internal region after rotation, and denote it as the first grayscale sequence. Obtain the sequence formed by the pixel values ​​of all row pixels of the other matching internal region without rotation, and denote it as the second grayscale sequence. Use the DTW algorithm to obtain the similarity between the first grayscale sequence and the second grayscale sequence, and denote it as the matching similarity. When the matching similarity is greater than the preset matching similarity threshold, the initial matching region corresponding to the two matching internal regions is denoteed as the matching region.

4. The intelligent monitoring method for river surface flow velocity based on video images according to claim 1, characterized in that, The specific steps for obtaining the second confidence level of the matching region corresponding to the sub-matching region based on the distance between the sub-matching region and the non-corresponding region in the river image are as follows: First, obtain the mean of the Euclidean distances between all pixels in the sub-matching region and all pixels in all non-corresponding regions, and denote it as the distance between the sub-matching region and all non-corresponding regions. Then, the distance between any sub-matching region and all non-corresponding regions in any frame of the river image is used as the input of the exponential decay function, and the output of the exponential decay function is recorded as the first confidence of the sub-matching region. The first confidence of each of the two sub-matching regions of the matching region is obtained, and the average of the two first confidences of the two sub-matching regions of the matching region is subtracted from 1 and recorded as the second confidence of the matching region. Thus, the two sub-matching regions of a matching region correspond to the same second confidence.

5. The intelligent monitoring method for river surface flow velocity based on video images according to claim 1, characterized in that, The specific steps involved in sampling the river image to obtain several sampled images, and obtaining retained key points and corresponding descriptors based on the pixels of the sub-matching regions in the river image and the sampled images, are as follows: First, using a size of The window iterates through the mean sampling of any river image multiple times to obtain multiple sampled images. The SIFT algorithm is used to obtain all key points in the sampled images and the descriptors corresponding to the key points. Key points located in the sub-matching region are recorded as retained key points, and several retained key points and their corresponding descriptors are obtained.

6. The intelligent monitoring method for river surface flow velocity based on video images according to claim 1, characterized in that, The process of determining whether a retained keypoint is a keypoint in the river image and the sampled image, obtaining the attribute sequence of the retained keypoint, and obtaining the independence of the retained keypoint based on the attribute sequence, the descriptor of the retained keypoint, and the second confidence level of the sub-matching region where the retained keypoint is located, includes the following specific steps: First, obtain the pixels at the same location in any river image and several corresponding sample images, and denote them as target pixels. When the target pixel is a key point to be preserved in the river image or sample image, use 1 as the attribute label of the target pixel; when the target pixel is not a key point to be preserved in the river image or sample image, use 0 as the attribute label of the target pixel. Thus, the target pixel corresponds to one attribute label in each of the river image and sample images, and the target pixel corresponds to several attribute labels. The sequence formed by all the attribute labels corresponding to the target pixel is denoted as the attribute sequence of the target pixel. Obtain the attribute sequence of all pixels in the river image. Then, based on the attribute sequence and the descriptor that retains the key points, independent factors and independent parameters are obtained respectively; Finally, the specific calculation method for preserving the independence of key points is as follows: in, This indicates that the independence of key points is preserved. This indicates the second level of confidence in retaining the sub-matching region described by the key point. This indicates the number of attribute labels with a value of 1 in the attribute sequence that retains key points; This indicates the number of elements in the attribute sequence that retains key points; This indicates the sequence of attributes that retains key points, specifically the first... The independent factor corresponding to each attribute label being 1; This indicates the sequence of attributes that retains key points, specifically the first... The independent parameter corresponding to each attribute tag being 1.

7. The intelligent monitoring method for river surface flow velocity based on video images according to claim 6, characterized in that, The specific methods for obtaining the independent factors and independent parameters are as follows: The river image or sampled image corresponding to any attribute label of 1 in the attribute sequence of any preserved keypoint in the river image is denoted as the target image. The descriptor corresponding to the preserved keypoint in the target image is denoted as the target descriptor. The average cosine similarity between the target descriptor and the descriptors of all other preserved keypoints in the target image is denoted as the independent factor corresponding to the attribute label of the preserved keypoint being 1. The ratio of the ordinal number of any attribute label of 1 in the attribute sequence of any preserved keypoint in the river image to the number of elements in the attribute sequence is denoted as the independent parameter corresponding to the attribute label of the preserved keypoint being 1.

8. The intelligent monitoring method for river surface flow velocity based on video images according to claim 1, characterized in that, The specific steps for obtaining the independence of pixels not in the matching region in the river image based on the independence of the retained key points and the distance between the pixels in the river image and the retained key points are as follows: Pixels not in the matching region are identified as the first special pixel. Keypoints that are kept with the smallest distance between the first special pixel and the keypoints are identified as the second special pixel. The Euclidean distance between the first and second special pixels is identified as the special distance. The special distance is input into the exponential decay function. The output of the exponential decay function is identified as the special parameter of the first special pixel. The special parameter of the first special pixel is multiplied by the independence of the corresponding second special pixel. The product is identified as the independence of the first special pixel. Thus, the independence of all pixels not in the matching region in the river image is obtained.

9. The intelligent monitoring method for river surface flow velocity based on video images according to claim 1, characterized in that, The specific steps involved in correcting the pixel values ​​of all river images in the river video using independence to obtain a new river video are as follows: First, key points with independence greater than a preset independence threshold are recorded as standard pixels, and pixels in the river image other than standard pixels are recorded as non-standard pixels. Then, the pixel values ​​of the standard pixels are used as input for bilinear interpolation to obtain the interpolated values ​​corresponding to the non-standard pixels. The specific calculation method for the corrected pixel values ​​of any non-standard pixel is as follows: in, This represents the corrected pixel value corresponding to a non-standard pixel. Indicates the independence of non-standard pixels. This represents the pixel value of a pixel in a river image. This represents the insertion value corresponding to the pixel. Finally, obtain the corrected pixel values ​​corresponding to all non-standard pixels in the river image of any frame in the river video, record the river image formed after correcting the pixel values ​​of the pixels as the new river image, and record the video formed by all consecutive frames of the new river images as the new river video.

10. The intelligent monitoring method for river surface flow velocity based on video images according to claim 1, characterized in that, The specific steps involved in compressing and transmitting new river videos to monitor river surface velocity are as follows: First, the new data packet containing the new river video is compressed and encoded using the ZIP compression algorithm to obtain a compressed packet; Then, the compressed package is transmitted to the server using a 5G network; Finally, the compressed package is decompressed using a program on the server side, and a model is obtained using the relevant flow velocity calculation formula in the program. The decompressed river video is then used for intelligent monitoring of the river surface flow velocity to obtain the river surface flow velocity, and the monitoring results are visualized.

Images