A method for back-end simple identification of relative historical water levels and water level warnings

Through semantic segmentation technology, the water area recognition model is trained, combined with feature matching and template matching, the problems of on-site measurement and equipment installation in water level monitoring and early warning are solved, and rapid and accurate water level changes and early warning supervision are achieved.

CN118674934BActive Publication Date: 2024-11-12HANGZHOU DINGCHUAN INFORMATION TECH CO LTD
View PDF 11 Cites 0 Cited by

Patent Information

Application Number
CN202411134800.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2024-11-12
Estimated Expiration
2044-08-19

AI Technical Summary

Technical Problem

The prior art requires a large number of on-site measurement and installation equipment in water level monitoring and early warning, which makes it difficult to implement and cannot quickly achieve refined supervision of water level changes and early warnings. Moreover, changes in camera scene locations lead to incorrect identification results.

Method used

The semantic segmentation technology is used to train the water area recognition model, connect the back-end video system to collect images through the existing camera, set the highest relative water level point and early warning point, calculate the spatial position relationship between the point and the water area, and realize the relative historical water level and water level warning, no need for on-site measurement calibration and installation of equipment, and check the changes in scene position through feature matching and template matching.

Benefits of technology

It realizes water level monitoring and early warning without on-site measurement and installation of equipment, simplifies manual review work, improves the accuracy and availability of identification results, and can quickly respond to water level changes and early warnings.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118674934B_ABST
    Figure CN118674934B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for simply identifying relative historical water levels and water level warnings at the back end. The method uses an existing camera to access a back end video system, collects images related to the water surface, and uses semantic segmentation technology for pre-labeling and training to obtain a water surface recognition model. The highest relative water level point and the relative water level warning point are set on the image, the water surface is identified by the water surface recognition model, the spatial position relationship between the point and the water surface is calculated, and the relative historical water level and water level warning information are obtained. The method does not require on-site measurement calibration and installation equipment, and is easy to operate. At the same time, a scene position change check is added, and the current image and the historical image are compared to determine whether the scene position has changed. Abnormal events such as camera angle movement, focal length change, and image abnormality can be well excluded. At the same time, abnormal recognition data and warning data are automatically generated for manual review, which simplifies the workload of manual review and ensures the accuracy and availability of the recognition results.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical fields of semantic segmentation image recognition model, back-end recognition, relative historical water level monitoring and water level early warning, and in particular to a method for back-end simple recognition of relative historical water level and water level early warning. Background Art

[0002] The semantic segmentation image recognition model is a deep learning-based model that is mainly used to identify and segment different regions or objects in an image. The model is built on a convolutional neural network (CNN) and is trained to learn and understand features in an image, thereby achieving pixel-level classification.

[0003] Back-end recognition refers to the collection of front-end camera image data through the video system to the server, which is further processed and recognized by the powerful computing resources and professional algorithms on the server. Compared with front-end recognition, it does not require hardware modification of the front-end camera and has the advantages of unified management, high performance, high reliability and high security.

[0004] Relative historical water level monitoring is a method of evaluating water level changes by comparing current water levels with historical water level data. This monitoring method helps to understand the relative position of the current water level in the historical water level, the trend and speed of water level changes, and possible influencing factors, which is of great significance to water resources management, flood warning, environmental protection and other fields.

[0005] Water level warning refers to a water level threshold set in the hydrological monitoring system to warn of possible disasters such as floods and rainstorms in advance. This mechanism aims to detect potential disaster risks in advance and provide accurate information and guidance to the public so that they can take appropriate actions to deal with possible disasters.

[0006] At present, water level monitoring is mainly based on radar, pressure, resistance and other sensors. There are also front-end recognition methods such as image recognition of water gauge scales or virtual scales. The current monitoring and early warning of water level changes based on image recognition requires on-site measurement or installation of front-end edge computing equipment within the camera scene. Due to the large area of ​​water supervision and the large number of existing cameras, a large amount of on-site measurement and installation are required to quickly grasp the relative water level data and water level warnings. The workload is huge in the early stage. At the same time, due to changes in the position of the camera scene in the later stage, the water level recognition results will be wrong. For a large number of existing supervision cameras in rivers, ponds, reservoirs, lakes, etc., it is very inconvenient to reuse them, and the implementation is difficult, and it is impossible to quickly achieve refined supervision of water level changes and early warnings. Summary of the invention

[0007] The purpose of the present invention is to overcome the defects of the prior art and provide a method for the back-end to simply identify the relative historical water level and water level warning. The method uses the existing camera to access the back-end video system, collects water surface related images, and uses semantic segmentation technology for pre-labeling and training to obtain a water surface recognition model. The relative water level highest point and the relative water level warning point are set on the image, the water surface is identified by the water surface recognition model, the spatial position relationship between the point and the water surface is calculated, and the relative historical water level and water level warning information are obtained. The method does not require on-site measurement calibration and installation equipment, and is easy to operate. At the same time, the scene position change check is added, the current image and the historical image are compared to determine whether the scene position has changed, which can better exclude abnormal events such as camera angle movement, focal length change, image abnormality, etc., and automatically generate recognition abnormal data and warning data for manual review, which simplifies the workload of manual review and ensures the accuracy and availability of the recognition results.

[0008] The technical solution adopted by the present invention is:

[0009] A method for simply identifying relative historical water levels and water level warnings at the back end includes the following steps:

[0010] S1, access video: access the camera video stream through the backend video system.

[0011] S2, determine the scene location: determine the scene location for identifying the relative historical water level and water level warning;

[0012] Furthermore, the scene location needs to include a water surface, and the water level of the water surface has the same reference object within the range of variation for comparative observation.

[0013] S3, water surface recognition model training: collect data sets of image data of scene locations that meet the conditions, mark water surfaces, and perform training to obtain a water surface recognition model W;

[0014] The data set contains no less than 50,000 images, covering different scene locations, time periods and weather conditions, including rivers, ponds, reservoirs, lakes, and rivers, with no less than 3,000 scenes. The water surface annotation is performed by first using a large semantic segmentation model to perform preliminary water surface pre-annotation, and then manually annotating the pre-annotation results.

[0015] S4, parameter setting: set parameters for the scene position of the access video, extract an image, set the relative water level highest point A and the relative water level warning point B in the image, and set the frame extraction time and frequency of the camera. After the parameters are set, save a historical image P1 immediately;

[0016] The relative water level highest point A is the top position of the wall or dam body which the water level cannot reach; the relative water level warning point B is located below the relative water level highest point A and is the point that triggers the water level warning; the line connecting the relative water level highest point A and the relative water level warning point B is perpendicular to the ground or water surface.

[0017] S5, verifying the scene position: comparing the image N1 at the current frame extraction time with the historical image P1 by using feature matching or template matching. If the verified scene position is normal, execute step S6; if the verified scene position is abnormal, record it as abnormal data of the verified scene position, terminate the identification, and then execute step S8;

[0018] Furthermore, the feature matching is to sequentially perform graying, feature point extraction, feature point matching, feature point screening, feature matching initial judgment, and image block feature point matching on the image N1 at the current frame extraction time and the historical image P1;

[0019] Grayscale is to convert a color image into a grayscale image;

[0020] Feature point extraction uses SIFT algorithm to extract feature points and feature point descriptors;

[0021] The feature point matching uses the BF brute force feature matching method and the k nearest neighbor matching strategy to match the extracted feature points and the descriptors of the feature points to obtain the matching result array Array;

[0022] Feature point screening is to traverse the matching result array Array, and the candidate distance error value is calculated using the Euclidean distance. When the first candidate distance error value is less than 85% of the second candidate distance error value, the current matching feature point pair is retained, otherwise, the matching feature point pair is deleted;

[0023] The initial feature matching judgment is that when the number of matching feature point pairs is less than or equal to the matching feature point pair number threshold Z, the scene position abnormality is checked; when the number of matching feature point pairs is greater than the matching feature point pair number threshold Z, image block feature point matching is performed;

[0024] The calculation method of the matching feature point pair number threshold Z is based on the image resolution of 1920*1080 as the standard, and the matching feature point pair number threshold Z is 20; the calculation of the matching feature point pair number threshold Z of different image resolutions X is shown in formula (1):

[0025] Formula (1)

[0026] Where:

[0027] X represents the image resolution;

[0028] Z represents the number of matching feature point pairs;

[0029] Image block feature point matching is to divide the grayscale image of the current frame extraction time image N1 and the grayscale image of the historical image P1 into image blocks of equal proportions, and perform feature point extraction, feature point matching, and feature point screening on the image blocks in order of position to obtain matching feature point pairs, and to determine whether the image blocks have moved by calculating the pixel deviation of the matching feature point pairs; if the number of image blocks that have moved is greater than the number of image blocks that have not moved, it is considered that the camera has moved, and the scene position is abnormal; if the number of image blocks that have moved is less than or equal to the number of image blocks that have not moved, it is considered that the camera has not moved, and the scene position is normal;

[0030] Furthermore, the template matching is adopted when more than half of the images at the scene position have leaf disturbance or when the verification scene position is found to be wrong after manual review in step S8. The template matching is to select a fixed target as the template area roiA in the historical image P1, and grayscale, template area matching, and template pixel deviation calculation are performed on the image N1 at the current frame extraction time and the historical image P1 in sequence;

[0031] The template region matching adopts the standard correlation matching algorithm, taking the template region roiA as the standard, and searches for the best matching position roiB in the grayscale image of the image N1 at the current frame extraction time;

[0032] The template pixel deviation calculation is to calculate the pixel deviation between the template area roiA and the matching position roiB, and determine whether the verification scene position is abnormal based on the calculated pixel deviation.

[0033] S6, calculate the relative pixel reverse water level: use the water surface recognition model W to recognize the water surface of the image N1 at the current frame extraction time. If the water surface is not recognized, it is recorded as abnormal water surface recognition data, the recognition is terminated, and then step S8 is executed; if the water surface is recognized, the water surface is obtained. The pixel coordinate point set is used to calculate the spatial position relationship between the relative water level highest point A, the relative water level warning point B and the water area PS by using the ray method to obtain the water level warning data and the relative water level intersection R; and the relative water level highest point A The relative pixel reverse water level rpwl of the current frame extraction time is obtained by using the pixel distance between the pixels, and the relative pixel reverse water level rpwl of the current frame extraction time is added to the historical relative pixel reverse water level record set R1;

[0034] Further, the ray method is used to calculate the spatial position relationship between the relative water level highest point A, the relative water level warning point B and the water surface PS, specifically, the ray method is used to determine whether the relative water level highest point A or the relative water level warning point B is inside the water surface PS; if the relative water level warning point B is inside the water surface PS, it is recorded as water level warning data;

[0035] The relative water level intersection point A The highest point of relative water level The first intersection point of the ray L2 toward the relative water level warning point B and the water surface PS; if the relative water level highest point A is inside the water surface PS, the relative water level intersection point R is the relative water level highest point A; if there is no relative water level intersection point R, it is recorded as abnormal data for water surface identification;

[0036] The relative pixel reverse water level rpwl of the current frame extraction time is calculated as shown in formula (2):

[0037] Formula (2)

[0038] Where:

[0039] a x Indicates the horizontal position of the highest relative water level point A in the image;

[0040] a y Indicates the vertical position of the highest relative water level point A in the image;

[0041] r x Indicates the horizontal position of the relative water level intersection point R in the image;

[0042] r y Indicates the position of the relative water level intersection point R in the vertical direction of the image;

[0043] rpwl represents the relative pixel reverse water level.

[0044] S7, calculate the relative historical water level: when the historical relative pixel reverse water level record set R1 has accumulated more than two records, the relative pixel reverse water level rpwl in the historical relative pixel reverse water level record set R1 is calculated to obtain all the relative historical water levels rhwl. The calculation is shown in formula (3):

[0045] Formula (3)

[0046] Where:

[0047] rpwl represents the relative pixel reverse water level;

[0048] hpwl represents the maximum pixel reverse water level;

[0049] lpwl represents the minimum pixel reverse water level;

[0050] rhwl represents the relative historical water level;

[0051] Furthermore, the relative historical water level rhwl ranges from 0 to 100; if (hpwl-lpwl) is 0, it means that the water level has not changed and is not involved in the calculation.

[0052] S8, review and identify abnormal and warning data: manually review the abnormal data of the verification scene location, the abnormal data of the water surface identification, and the water level warning data to determine whether they are erroneous data and take appropriate measures.

[0053] The verification scene position abnormality data, water surface recognition abnormality data, and water level warning data are manually reviewed. Specifically, if the verification scene position abnormality data is found, the scene position has not changed, but the verification abnormality is caused by the disturbance of leaves, then the scene position matching method is set, and if the scene position changes, the scene position is corrected; the scene position matching method setting is to change the feature matching to template matching or reset the template area in the template matching; the scene position correction is to control the camera to achieve the normal verification scene position; if it is water surface recognition abnormality data, it is determined whether it is a recognition error or an image quality problem. If it is an image quality problem, the camera is operated and maintained; image quality problems include: image packet loss, occlusion, and blur problems;

[0054] If the water surface is identified incorrectly, the image data is reflowed and step S3 is repeated to annotate and train the incremental data;

[0055] If it is water level warning data, the warning data will be issued after manual review.

[0056] The beneficial effects of the present invention are:

[0057] 1. The water surface recognition model training proposed in the present invention uses a large semantic segmentation model for pre-labeling, which greatly reduces the workload and difficulty of manual labeling.

[0058] 2. The present invention can identify relative historical water levels and water level warnings based on the existing camera backend, without the need for on-site measurement calibration and installation of other equipment. It is easy to implement and can quickly realize water level warnings and relative historical water level monitoring functions.

[0059] 3. The present invention adopts two methods, feature matching and template matching, to verify the camera scene position, ensure the consistency of the recognition scene position, and further ensure the availability of the recognition result.

[0060] 4. The present invention can automatically generate recognition anomaly and warning data, greatly simplifying the workload of manual review and ensuring the accuracy of recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 is a flow chart of the present invention;

[0062] Figure 2 It is a loss function curve diagram of the water surface recognition model training stage of the present invention;

[0063] Figure 3 It is a schematic diagram of the parameter setting method of the present invention;

[0064] Figure 4 It is a schematic diagram of the position change of the template area of ​​the template matching of the present invention.

[0065] In the figure: 1. Camera, 2. Water surface, 3. Dam body, 4. Formwork area, 5. Building, 6. Stone pier, 7. Fixing rod. DETAILED DESCRIPTION

[0066] The present invention will be described in further detail below in conjunction with the embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0067] The basic principle of the present invention is to access a large number of existing video points through the back-end video system, select video points related to water surface 2 such as rivers, ponds, reservoirs, lakes, rivers, etc., and collect a large number of images related to water surface 2. Pre-marking is performed using a large model, and further manual review and training are performed to obtain a water surface recognition model. By marking the highest relative water level point and the relative water level warning point on the image, the water surface recognition model is used to identify the water surface 2, calculate the spatial position relationship between the current point and the water surface 2, obtain the relative pixel reverse water level and determine whether the water level is warned, and further obtain the relative historical water level through the relative pixel reverse water level of the historical image.

[0068] The present invention is simple and convenient to operate. Users can configure the recognition of relative historical water levels and water level warnings by themselves, without complicated operations such as on-site measurement and calibration. At the same time, a scene change verification function is added. By comparing the scene changes between the current image and the historical image, it is determined whether the scene position of camera 1 has been changed, and a method for scene position correction is provided. It can better exclude abnormal events such as camera 1 angle movement, focal length change, image abnormality, etc., thereby further ensuring the accuracy of the recognition results. Automatically generating recognition abnormality and warning data for manual review also greatly simplifies the workload of manual review and ensures the accuracy of the recognition results. Finally, through operation and maintenance and warning disposal, the disposal of recognition abnormality and warning data is realized.

[0069] The present invention provides a specific implementation method of a back-end simple method for identifying relative historical water levels and water level warnings, such as Figure 1 As shown, the following steps are included:

[0070] S1, access video: use national standard 28181 protocol, camera rtsp streaming agent, third-party protocol and other methods to access video; at the same time, record the GPS information of the current video point.

[0071] S2, determine the scene position: by adjusting the direction angle of camera 1, determine the scene position for identifying relative historical water levels and water level warnings; the scene position needs to include water surface 2, and at the same time, the water level of water surface 2 has the same reference object within the range of variation for comparative observation; because it is necessary to consider the highest water level and the lowest water level within the range of water level variation, the reference object can also play a role in comparative observation, so the selected reference objects are walls, dam bodies 3, poles, columns, etc.

[0072] S3, water surface recognition model training:

[0073] S3.1, data set collection: collect image data of scene locations that meet the conditions, where the number of data sets is no less than 50,000 image data, and the image data covers different scene locations, time periods and weather conditions, where the scene locations include rivers, ponds, reservoirs, lakes, and rivers, with no less than 3,000 scenes, to obtain data set D.

[0074] S3.2, data pre-labeling: For the collected data set D, use the Grounded-SAM large model to pre-label the water surface 2 to obtain the pre-labeled data set P.

[0075] S3.3, manual review: perform manual review on the pre-labeled dataset P, manually label the incorrectly labeled images, and finally obtain the manually reviewed labeled dataset R.

[0076] S3.4, semantic segmentation training: Use BiSeNet V2 to perform semantic segmentation training on the data set R, and finally obtain the water surface recognition model W. The loss function curve of the water surface recognition model training stage is shown in Figure 2 shown.

[0077] S4, parameter setting: set the parameters of the scene position of the accessed video, extract an image, set the relative water level highest point A and the relative water level warning point B in the image, and set the frame extraction recognition time and frequency of camera 1. After the parameters are set, save a historical image P1 immediately; set the scene position verification method, including feature matching and template matching, and the default setting is feature matching;

[0078] The relative water level highest point A is the top position of the wall or dam body 3 that the water level cannot reach; the relative water level warning point B is located below the relative water level highest point A and is the point that triggers the water level warning; the line connecting the relative water level highest point A and the relative water level warning point B is perpendicular to the ground or water surface 2; the relative water level highest point A and the relative water level warning point B select a close view, such as Figure 3 shown.

[0079] S5, verifying the scene position: after the parameters are set, the frame is extracted according to the set time, and the image N1 of the current frame is obtained after the frame extraction, and it is recorded on the file server to obtain the image address and time of the current frame extraction; the image N1 of the current frame extraction time is compared with the historical image P1, and the feature matching method is used by default for comparison. If more than half of the images at the scene position have leaf disturbance, template matching is used; if the verification scene position is normal, step S6 is executed; if the verification scene position is abnormal, it is recorded as the verification scene position abnormal data, the identification is terminated, and then step S8 is executed;

[0080] S5.1, feature matching: grayscale conversion, feature point extraction, feature point matching, feature point screening, feature matching initial judgment, and image block feature point matching are performed on the image N1 at the current frame extraction time and the historical image P1 in sequence;

[0081] The grayscale conversion is to convert a color image into a grayscale image;

[0082] The feature point extraction is to extract the feature points and the descriptors of the feature points using SIFT algorithm;

[0083] The feature point matching uses the BF brute force feature matching method and the k nearest neighbor matching strategy to match the extracted feature points and the descriptors of the feature points to obtain a matching result array Array;

[0084] The feature point screening is to traverse the matching result array Array, and the candidate distance error value is calculated using the Euclidean distance. When the first candidate distance error value is less than 85% of the second candidate distance error value, the current matching feature point pair is retained, otherwise, the matching feature point pair is deleted;

[0085] The initial feature matching judgment is that when the number of matching feature point pairs is less than or equal to the matching feature point pair number threshold Z, the scene position abnormality is checked; when the number of matching feature point pairs is greater than the matching feature point pair number threshold Z, image block feature point matching is performed;

[0086] Furthermore, by matching feature points at 6000 scene locations, there are two types of images with image resolutions of 1920*1080 and 2560*1440. The threshold Z of the number of matching feature point pairs for different image resolutions X is shown in formula (1):

[0087] Formula (1)

[0088] The threshold Z of the number of matching feature point pairs for 1920*1080 is calculated to be 20, and the threshold Z of the number of matching feature point pairs for 2560*1440 is 36. The matching results of 6000 scene position feature points are as follows:

[0089]

[0090] The images of the two scene positions whose number of matching feature point pairs is less than or equal to the matching feature point pair number threshold Z are recorded as the verification scene position abnormal data, and the images of the other 5998 scene positions whose number of matching feature point pairs is greater than the matching feature point pair number threshold Z are used for image block feature point matching.

[0091] The image block feature point matching is to divide the grayscale image of the current frame extraction time image N1 and the grayscale image of the historical image P1 into image blocks of equal proportions, extract feature points, match feature points, and screen feature points in sequence according to the position order to obtain matching feature point pairs, and determine whether the image blocks are moved by calculating the pixel deviation of the matching feature point pairs;

[0092] The image resolution is 1920*1080. The resolution of the divided image block is 1920 / 5*1080 / 5. If the number of matching feature point pairs of the image block is less than 10, the image block does not participate in the calculation; if the number of matching feature point pairs is greater than or equal to 10, the pixel deviation of the matching feature point pairs is calculated. When calculating the pixel deviation of the matching feature point pairs, in order to avoid extreme value interference, the maximum value of 10% and the minimum value of 10% of the pixel deviation are removed, and then the average deviation is calculated. If the average pixel deviation of the image block is greater than 10 pixels, the image block is considered to have moved; among all image blocks, if the number of image blocks that move is greater than the number of image blocks that do not move, it is considered that camera 1 has moved, and the verification scene position is abnormal; if the number of image blocks that move is less than or equal to the number of image blocks that do not move, it is considered that camera 1 has not moved, and the verification scene position is normal.

[0093] The result of calculating the amount of scene position movement is as follows:

[0094]

[0095] The images with 35 scene positions moved are recorded as verification scene position abnormality data, and step S6 is performed on the images with 5963 scene positions not moved.

[0096] S5.2, template matching: when more than half of the images at the scene position have leaf disturbance or when the verification scene position is found to be wrong after manual review in step S8, template matching is to select a fixed target as the template area 4roiA in the historical image P1, and the template area 4roiA selects a fixed target, such as a building 5, a stone pier 6, a fixed pole 7, etc. Figure 4 As shown; and grayscale, template area matching, and template pixel deviation calculation are performed on the image N1 at the current frame extraction time and the historical image P1 in sequence;

[0097] The template region matching adopts the standard correlation matching algorithm, taking the template region 4roiA as the standard, and searches for the best matching position roiB in the grayscale image of the image N1 at the current frame extraction time;

[0098] The template pixel deviation calculation is to calculate the pixel deviation between the template area 4roiA and the matching position roiB. If the pixel deviation is greater than 10 pixels, it is considered that the camera 1 has moved and recorded as the verification scene position abnormality data.

[0099] Furthermore, after manually reviewing the abnormal data of 37 verification scene positions among the 6000 scene positions in step S8, it was found that 3 of them were due to feature matching errors caused by leaf disturbance, and the verification scene positions were normal after modification to template matching; 11 were caused by image packet loss, occlusion, and blur; and 23 were caused by scene position changes. The feature matching and template matching are performed by comparing the current frame image N1 with the historical image P1, so when the current frame image N1 has abnormal images such as image packet loss, occlusion, and blur, there will be a small number of matching feature point pairs or a large pixel deviation, resulting in an abnormal verification scene position, thereby discovering abnormal images such as image packet loss, occlusion, and blur.

[0100] S6, calculate the relative pixel reverse water level: calculate the relative pixel reverse water level for the 5963 normal scene positions, use the water surface recognition model W to recognize the water surface of the image N1 at the current frame extraction time, if the water surface 2 is not recognized, it is recorded as abnormal water surface recognition data, the recognition is terminated, and then step S8 is executed; if the water surface 2 is recognized, the water surface recognition model W is obtained. The pixel coordinate point set is used to calculate the spatial position relationship between the relative water level highest point A, the relative water level warning point B and the water area PS by using the ray method to obtain the water level warning data and the relative water level intersection R; and the highest relative water level The relative pixel reverse water level rpwl of the current frame extraction time is obtained by using the pixel distance between the pixels, and the relative pixel reverse water level rpwl of the current frame extraction time is added to the historical relative pixel reverse water level record set R1;

[0101] The ray method is used to calculate the spatial position relationship between the relative water level highest point A, the relative water level warning point B and the water surface PS, specifically, the ray method is used to determine whether the relative water level highest point A or the relative water level warning point B is inside the water surface PS; if the relative water level warning point B is inside the water surface PS, it is recorded as water level warning data; the ray method calculation is to draw a ray from a point in any direction, and count the number of intersections of the ray and the water surface PS. If the number of intersections is an even number, the point is outside the water surface PS, and if the number of intersections is an odd number, the point is inside the water surface PS;

[0102] The relative water level intersection The highest point of relative water level The first intersection point of the ray L2 toward the relative water level warning point B and the water surface PS; if the relative water level highest point A is inside the water surface PS, the relative water level intersection point R is the relative water level highest point A; if there is no relative water level intersection point R, it is recorded as abnormal data for water surface identification;

[0103] The relative pixel reverse water level rpwl of the current frame extraction time is calculated as shown in formula (2):

[0104] Formula (2)

[0105] Where:

[0106] a x Indicates the horizontal position of the highest relative water level point A in the image;

[0107] a y Indicates the vertical position of the highest relative water level point A in the image;

[0108] r x Indicates the horizontal position of the relative water level intersection point R in the image;

[0109] r y Indicates the position of the relative water level intersection point R in the vertical direction of the image;

[0110] rpwl represents the relative pixel reverse water level.

[0111] Furthermore, among the calculation results of the relative pixel reverse water level of the 5963 normal scene positions in the verification scene positions, there are 87 water surface identification abnormal data and 31 water level warning data; excluding the 87 water surface identification abnormal data, 5876 scene positions execute step S7, and record the value of the image relative pixel reverse water level at the current frame extraction time, the image address of the current frame extraction, the frame extraction time, and the frame extraction date; the values ​​of some normal relative pixel reverse water levels are as follows: 123, 60, 59.

[0112] S7, calculate the relative historical water level: calculate the relative historical water level for 5876 scene positions. When the historical relative pixel reverse water level record set R1 of the scene position has accumulated more than two records, calculate the relative pixel reverse water level rpwl in the historical relative pixel reverse water level record set R1 to obtain all the relative historical water levels rhwl. The calculation is shown in formula (3):

[0113] Formula (3)

[0114] Furthermore, the relative historical water level rhwl ranges from 0 to 100; if (hpwl-lpwl) is 0, it means that the water level has not changed and is not included in the calculation.

[0115] Calculating the relative historical water level can help us understand the position of the current water level in the historical water level and help analyze the high and low of the current water level in the historical water level.

[0116] The distribution of the relative historical water level results for 5876 scenarios is as follows:

[0117]

[0118] S8, review and identify abnormal and warning data:

[0119] S8.1, automatically generate identification anomaly and warning data: The identification anomaly data types include verification scene location anomaly data, water surface identification anomaly data, and water level warning data.

[0120] The abnormality and warning data automatically generated for 6,000 scene recognitions are as follows:

[0121]

[0122] S8.2, manual review of abnormal identification and warning data: If the scene position is abnormal data, the scene position has not changed, but the disturbance of leaves has caused the abnormal verification, then the scene position matching method is set. If the scene position has changed, S8.3 scene position correction is performed; the scene position matching method setting is to change feature matching to template matching or reset the template area 4roiA in template matching;

[0123] If it is water surface recognition abnormal data, determine whether it is a recognition error or an image quality problem. If it is an image quality problem, perform S8.4 operation and maintenance and early warning treatment on camera 1; if it is a water surface recognition error, return the image data, repeat S3 steps, and annotate and train the incremental data;

[0124] If it is water level warning data, after manual review, S8.4 operation and maintenance and warning disposal are carried out.

[0125] S8.3, scene position correction: scene position correction is to perform pan / tilt control on camera 1, and obtain pixel deviation by calculating template pixel deviation using template matching method. The pixel deviation must be less than or equal to 2, so as to verify that the scene position is normal.

[0126] S8.4, Operation and maintenance and early warning disposal: For image quality issues including image packet loss, occlusion, and blur, camera 1 operation and maintenance processing is carried out; for water level warning data, after manual review, the warning data is issued.

[0127] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for back-end simple identification of relative historical water level and water level warning, characterized in that: The following steps are involved: S1, access the camera video stream through the backend video system; S2, determine the scene location for identifying relative historical water levels and water level warnings; S3, collecting data sets, marking water areas, and training image data of scene locations that meet the conditions to obtain a water area recognition model W; S4, set parameters for the scene position of the access video, extract an image, set the relative water level highest point A and the relative water level warning point B in the image, and set the frame extraction time and frequency of the camera. After the parameters are set, save a historical image P1 immediately; S5, compare the image N1 of the current frame extraction time with the historical image P1 by using feature matching or template matching. If the verification scene position is normal, execute step S6; if the verification scene position is abnormal, record it as verification scene position abnormal data, terminate the identification, and then execute step S8; S6, using the water surface recognition model W to recognize the water surface of the image N1 at the current frame extraction time, if the water surface is not recognized, it is recorded as abnormal water surface recognition data, the recognition is terminated, and then step S8 is executed; if the water surface is recognized, the water surface is obtained. The pixel coordinate point set is used to calculate the spatial position relationship between the relative water level highest point A, the relative water level warning point B and the water area PS by using the ray method to obtain the water level warning data and the relative water level intersection R; and the highest relative water level The relative pixel reverse water level rpwl of the current frame extraction time is obtained by using the pixel distance between the pixels, and the relative pixel reverse water level rpwl of the current frame extraction time is added to the historical relative pixel reverse water level record set R1; S7, when the historical relative pixel reverse water level record set R1 has accumulated more than two records, the relative pixel reverse water level rpwl in the historical relative pixel reverse water level record set R1 is calculated to obtain all relative historical water levels rhwl; S8, manually review the abnormal data of the verification scene location, the abnormal data of the water surface identification, and the water level warning data to determine whether they are erroneous data and take appropriate measures; In step S4, the relative water level highest point A is the top position of the wall or dam body that the water level cannot reach; the relative water level warning point B is located below the relative water level highest point A and is the point that triggers the water level warning; the line connecting the relative water level highest point A and the relative water level warning point B is perpendicular to the ground or water surface; In the step S6, the ray method is used to calculate the spatial position relationship between the relative water level highest point A, the relative water level warning point B and the water surface PS, specifically, the ray method is used to determine whether the relative water level highest point A or the relative water level warning point B is inside the water surface PS; if the relative water level warning point B is inside the water surface PS, it is recorded as water level warning data; The relative water level intersection The highest point of relative water level The first intersection point of the ray L2 toward the relative water level warning point B and the water surface PS; if the relative water level highest point A is inside the water surface PS, the relative water level intersection point R is the relative water level highest point A; if there is no relative water level intersection point R, it is recorded as abnormal data for water surface identification; The relative pixel reverse water level rpwl of the current frame extraction time is calculated as shown in formula (2): Formula (2) Where: a x Indicates the horizontal position of the highest relative water level point A in the image; a y Indicates the vertical position of the highest relative water level point A in the image; r x Indicates the horizontal position of the relative water level intersection point R in the image; r y Indicates the position of the relative water level intersection point R in the vertical direction of the image; rpwl represents the relative pixel reverse water level; In step S7, the relative historical water level rhwl is calculated as shown in formula (3): Formula (3) Where: rpwl represents the relative pixel reverse water level; hpwl represents the maximum pixel reverse water level; lpwl represents the minimum pixel reverse water level; rhwl represents the relative historical water level; The relative historical water level rhwl ranges from 0 to 100. If (hpwl-lpwl) is 0, it means that the water level has not changed and is not included in the calculation.

2. The method for back-end simple identification of relative historical water level and water level warning according to claim 1 is characterized by: In step S2, the scene position needs to include a water surface, and the water level of the water surface has the same reference object within the variation range for comparison and observation.

3. The method for back-end simple identification of relative historical water level and water level warning according to claim 1 is characterized by: In step S3, the data set contains no less than 50,000 images, and the image data covers different scene locations, time periods and weather conditions, wherein the scene locations include rivers, ponds, reservoirs, lakes, and rivers, with a number of no less than 3,000; the water area annotation is to first use a large semantic segmentation model to perform preliminary water area pre-annotation, and then manually annotate the pre-annotation results.

4. The method for back-end simple identification of relative historical water level and water level warning according to claim 1 is characterized by: In step S5, the feature matching is to sequentially perform graying, feature point extraction, feature point matching, feature point screening, feature matching initial judgment, and image block feature point matching on the image N1 at the current frame extraction time and the historical image P1; Grayscale is to convert a color image into a grayscale image; Feature point extraction uses SIFT algorithm to extract feature points and feature point descriptors; The feature point matching uses the BF brute force feature matching method and the k nearest neighbor matching strategy to match the extracted feature points and the descriptors of the feature points to obtain the matching result array Array; Feature point screening is to traverse the matching result array Array, and the candidate distance error value is calculated using the Euclidean distance. When the first candidate distance error value is less than 85% of the second candidate distance error value, the current matching feature point pair is retained, otherwise, the matching feature point pair is deleted; The initial feature matching judgment is that when the number of matching feature point pairs is less than or equal to the matching feature point pair number threshold Z, the scene position is checked to be abnormal; When the number of matching feature point pairs is greater than the matching feature point pair number threshold Z, image block feature point matching is performed; Image block feature point matching is to divide the grayscale image of image N1 at the current frame extraction time and the grayscale image of historical image P1 into image blocks of equal proportion, and perform feature point extraction, feature point matching, and feature point screening on the image blocks in order of position to obtain matching feature point pairs, and calculate the pixel deviation of the matching feature point pairs to determine whether the image blocks have moved; if the number of image blocks that have moved is greater than the number of image blocks that have not moved, it is considered that the camera has moved and the scene position is abnormal; if the number of image blocks that have moved is less than or equal to the number of image blocks that have not moved, it is considered that the camera has not moved and the scene position is normal.

5. The method for back-end simple identification of relative historical water level and water level warning according to claim 4 is characterized in that: The calculation method of the matching feature point pair number threshold Z is based on the image resolution of 1920*1080 as the standard, and the matching feature point pair number threshold Z is 20; the calculation of the matching feature point pair number threshold Z of different image resolutions X is shown in formula (1): Formula (1) Where: X represents the image resolution; Z represents the number of matching feature point pairs.

6. The method for back-end simple identification of relative historical water level and water level warning according to claim 1 is characterized by: In step S5, the template matching is adopted when more than half of the images at the scene position have leaf disturbance or when the verification scene position is found to be wrong after manual review in step S8. The template matching is to select a fixed target as the template area roiA in the historical image P1, and grayscale, template area matching, and template pixel deviation calculation are performed on the image N1 at the current frame extraction time and the historical image P1 in sequence; The template region matching adopts the standard correlation matching algorithm, taking the template region roiA as the standard, and searches for the best matching position roiB in the grayscale image of the image N1 at the current frame extraction time; The template pixel deviation calculation is to calculate the pixel deviation between the template area roiA and the matching position roiB, and determine whether the verification scene position is abnormal based on the calculated pixel deviation.

7. The method for back-end simple identification of relative historical water level and water level warning according to claim 1 is characterized by: In the step S8, the verification scene position abnormality data, water surface identification abnormality data, and water level warning data are manually reviewed. Specifically, if the verification scene position abnormality data is manually reviewed, the scene position has not changed, but the verification abnormality is caused by the disturbance of leaves, then the scene position matching method is set, and if the scene position changes, the scene position is corrected; the scene position matching method setting is to change the feature matching to template matching or reset the template area in the template matching; the scene position correction is to control the camera to achieve the normal verification scene position; If the water surface is identified with abnormal data, determine whether it is an identification error or an image quality problem. If it is an image quality problem, perform maintenance on the camera. Image quality issues include: image packet loss, occlusion, and blur. If there is an error in water surface recognition, the image data will be reflowed and step S3 will be repeated to annotate and train the incremental data. If it is water level warning data, the warning data will be issued after manual review.

Citation Information

Patent Citations

  • Water control system and method for water management

    CA3004686A1

  • Water level monitoring method, storage medium, network device and water level monitoring system

    CN110443243A

  • Remote image intelligent identification method for hydraulic engineering deformation

    CN111416964A

  • Embedded terminal reservoir water level early warning method based on semantic segmentation

    CN112163496A

  • Water level height monitoring method and device, electronic equipment and storage medium

    CN114140518A