Abnormal event detection method, system and equipment for reservoir supervision

Abstract

The invention discloses an abnormal event detection method, system and device for reservoir supervision. The method comprises the steps of obtaining video image data shot by cameras installed around a reservoir; inputting the video image data into a pre-trained reservoir abnormal event detection and tracking model for prediction and identification; wherein the reservoir abnormal event detection and tracking model is built by a Yolov10 model which is subjected to iteration training for multiple times and integrates an Ultra algorithm and a Botport tracking algorithm, a detection head network of the Yolov10 model is composed of a one-to-one network and a one-to-many network and is used for generating one-to-one prediction and one-to-many prediction for different targets, and the detection head network of the Yolov10 model is used for generating one-to-one prediction and one-to-many prediction for different targets. The learning accuracy of the detection head network is improved, and meanwhile, only one-to-one prediction is carried out in the reasoning stage, so that the efficiency is improved; and outputting the identified abnormal event type. The efficiency of finding abnormal events in the daily supervision process of the reservoir can be improved, and reservoir personnel can conveniently find the abnormal events in time and take further actions.

Description

Technical Field

[0001] The present application relates to the technical field of reservoir supervision, and in particular to an abnormal event detection method, system and equipment for reservoir supervision. Background Art

[0002] With the promotion of the concept of smart water conservancy and its vigorous promotion in various places, reservoirs, as an important part of water conservancy, have a great need to improve their intelligent supervision and operation. Traditional reservoir operation supervision mainly relies on manual inspections. The on-duty personnel conduct inspections at specific times and along specific routes to achieve the purpose of reservoir supervision, but this solution still has some shortcomings. First, it consumes a lot of manpower and material resources. It depends on the inspections of on-duty personnel, which requires the formulation of a daily duty schedule. At the same time, according to the regional division of the reservoir, personnel are coordinated to conduct inspections. Second, it cannot be inspected in real time. The on-duty personnel of the reservoir conduct inspections according to the duty schedule, so there are loopholes. The time interval between two inspections is relatively long. Once an abnormal event that endangers the reservoir occurs, there is a problem of untimely discovery, which may cause serious harm to the operation of the reservoir.

[0003] Combined with the current development of computer artificial intelligence technology, computer vision methods can promptly detect abnormal events occurring in reservoirs, making it easier for on-duty personnel to detect and take action in a timely manner, minimizing equipment and property losses in the reservoir. This also reduces the inspection pressure on on-duty personnel and effectively improves the efficiency of detecting abnormal events. However, there is currently a lack of a scientific and reasonable abnormal event detection method for reservoir supervision. Summary of the Invention

[0004] The present application provides a method, system and device for detecting abnormal events for reservoir supervision to solve the above problems.

[0005] In one aspect, the present application provides a method for detecting abnormal events for reservoir supervision, the method comprising the following steps:

[0006] Step S1: Obtain video image data captured by cameras installed around the reservoir;

[0007] Step S2: Inputting the video image data into a pre-trained reservoir abnormal event detection and tracking model for prediction and identification; wherein the reservoir abnormal event detection and tracking model is built using a Yolov10 model that has undergone multiple iterations of training and integrates the Ultralytics algorithm and the Botsort tracking algorithm. The detection head network of the Yolov10 model consists of a one-to-one network and a one-to-many network, which are used to generate one-to-one predictions and one-to-many predictions for different targets to improve the learning accuracy of the detection head network. At the same time, only one-to-one predictions are performed in the inference stage to improve efficiency.

[0008] Step S3: Output the identified abnormal event type.

[0009] In one implementation of the present application, the abnormal event types include: fireworks, gathering of people, lingering of people, and climbing over a fence.

[0010] In one implementation of the present application, the Yolov10 model network structure consists of a backbone network, a feature aggregation network, and a detection head network. The backbone network adopts CSPDarknet53, which is composed of Conv, C2f, and SCDown alternately. The last layer of the backbone network is composed of SPPF and PSA. The C2f network in the 5th and 7th layers of the backbone network and the PSA network in the last layer output feature maps of three resolutions, denoted as P3, P4, and P5. The feature aggregation network consists of Upsample, C2f, Conv, and Concat. The resolution of P3, P4, and P5 is from high to low. Therefore, the feature aggregation network mainly uses upsampling, convolution, and channel splicing to fuse the other two feature maps into another feature map, thereby generating a fused feature map, which is denoted as P6, P7, and P8 respectively.

[0011] The obtained P6, P7, and P8 feature maps are respectively input into the final detection head network. After processing by the detection head network, the predicted coordinates, categories, and confidence levels are obtained. After corresponding post-processing, the prediction results are obtained.

[0012] In one implementation of the present application, the abnormal event detection and tracking model uses Deepstream to build a method for pulling and pushing live video streams, and integrates abnormal event detection and tracking models to form a complete set of abnormal event judgment models, analyze the video stream in real time, and obtain the tracking box, category, and tracking ID of the detection target.

[0013] In one implementation of the present application, the judgment of abnormal fireworks events is specifically as follows: Opencv is used to implement image grayscale and Otsu's method to obtain the highlighted area of the fireworks; the area value of the fireworks area in the frame is calculated, and the distance between the center points is obtained using the Euclidean formula. The area value and the distance value are compared using frame-by-frame information. When the area value changes in size and the distance value is less than a certain threshold, the threshold is read from the configuration file, and the area value and distance value of the current frame are compared with the next frame.

[0014] In one implementation of the present application, the judgment of abnormal events of personnel gathering is specifically as follows: a personnel tracking frame and tracking ID are generated through a reservoir abnormal event detection and tracking model, the center point coordinates of each tracking ID and the number of people in the entire area are calculated, and the distance between the center points of each tracking ID is judged using frame-by-frame information. When the distance is less than a threshold and the number of people in the area exceeds a specified threshold, an alarm information of an abnormal event of personnel gathering in the area is generated.

[0015] In one implementation of the present application, the judgment of a personnel stay event is specifically as follows: a personnel tracking frame and tracking ID are generated through a reservoir abnormal event detection and tracking model, the coordinates of the personnel center point of the tracking ID and the current frame time are calculated, and the cumulative appearance time of the ID in the current area and the distance from the initial center point are recorded frame by frame. When the distance is always less than the range of the current area and the appearance time exceeds a certain threshold, it can be judged that a stay event has occurred for the person corresponding to the ID. The area range is confirmed based on the scene captured by the on-site camera.

[0016] In one implementation of the present application, the judgment of a fence climbing over event is specifically as follows: continuously tracking the person outside the fence and recording the tracking ID and tracking frame coordinates, using the tracking frame to capture the target image in real time, using the pre-set fence coordinates, and using the vector cross product method to determine whether the person is on the same side of the fence. When the same ID is judged to exist on both sides of the fence and the similarity of the target image between the previous and next frames is greater than the specified threshold, it can be determined that a fence climbing over event has occurred, and the alarm information is promptly drawn into the image and pushed for display in the video stream.

[0017] The present application also provides an abnormal event detection system for reservoir supervision, which includes: a video stream reading module, which is used to use DeepStream to read the reservoir site video stream and decode each frame of image; an abnormal event judgment module, which is used to load each frame of image for target identification and tracking, and load configuration file parameters for use in abnormal events, adopt corresponding post-processing methods according to different target categories, filter whether there are abnormal events in the target, and return alarm information if there are abnormal events; a video stream push module, which is used to receive each frame of image and alarm information, complete the drawing of the alarm information on the corresponding frame, save the frame as an alarm image, and use the push stream module to push each frame of image to the specified stream address for real-time display.

[0018] The present application also provides an abnormal event detection device for reservoir supervision, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can complete the aforementioned abnormal event detection method for reservoir supervision.

[0019] The present application provides a method, system, and device for detecting abnormal events for reservoir supervision, which have the following beneficial effects:

[0020] (1) Computer vision can be used to promptly detect abnormal events in the reservoir, making it easier for on-duty personnel to detect and take action in a timely manner, thereby minimizing equipment and property losses in the reservoir;

[0021] (2) It reduces the inspection pressure of on-duty personnel and can effectively improve the efficiency of discovering abnormal events;

[0022] (3) The Yolov10 model is a significantly improved model in the Yolo series. It accelerates the model inference speed by removing non-maximum suppression and improves the efficiency of video stream AI analysis;

[0023] (4) The model is built using the Ultralytics library, and the tracking method uses the Botsort model, which further enhances the accuracy of the recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0025] Figure 1 A flow chart of an abnormal event detection method for reservoir supervision provided in an embodiment of the present application;

[0026] Figure 2 A diagram showing the composition of an abnormal event detection system for reservoir supervision provided in an embodiment of the present application;

[0027] Figure 3 A schematic diagram of an abnormal event detection device for reservoir supervision provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0029] The embodiments of the present application provide a method, system, and device for detecting abnormal events for reservoir supervision. The technical solutions proposed in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0030] Figure 1 This is a flow chart of an abnormal event detection method for reservoir supervision provided in an embodiment of the present application. Figure 1 As shown, the method mainly includes the following steps:

[0031] Step S1: Obtain video image data captured by cameras installed around the reservoir;

[0032] Step S2: Inputting the video image data into a pre-trained reservoir abnormal event detection and tracking model for prediction and identification; wherein the reservoir abnormal event detection and tracking model is built using a Yolov10 model that has undergone multiple iterations of training and integrates the Ultralytics algorithm and the Botsort tracking algorithm. The detection head network of the Yolov10 model consists of a one-to-one network and a one-to-many network, which are used to generate one-to-one predictions and one-to-many predictions for different targets to improve the learning accuracy of the detection head network. At the same time, only one-to-one predictions are performed in the inference stage to improve efficiency.

[0033] Step S3: Output the identified abnormal event type.

[0034] In the embodiment of the present application, first, according to the division of the reservoir supervision area, the image data captured by the on-site camera is collected, and the open source data set is used as a supplement. The categories of abnormal events are divided into fireworks, gathering of people, people staying, and climbing over the fence. The annotation categories are divided into fireworks and people. The anomaly detection dataset is produced with reference to the annotation categories and is divided into a training set and a validation set in a ratio of 9:1.

[0035] Furthermore, the Yolov10 model was built using Pytorch and the Ultralytics library, and the model was trained using anomaly detection dataset. An abnormal event detection model was obtained through multiple iterations. The tracking method adopted the Botsort model, which was integrated to output continuous target tracking boxes, categories, and tracking IDs.

[0036] Specifically, the anomaly detection dataset consists of reservoir on-site image data and open-source image data. This dataset is used to construct a dataset that covers a wide range of features and has balanced samples. Considering that fireworks in real environments often have color features, the fireworks categories are subdivided. The model in this embodiment is built using the Ultralytics library. The Yolov10 model is a significantly improved model in the Yolo series. By removing non-maximum suppression, it accelerates the model inference speed and improves the efficiency of video stream AI analysis.

[0037] Furthermore, the Yolov10 model consists of a backbone network, a feature aggregation network, and a detection head network. The backbone network adopts CSPDarknet53, which is composed of Conv, C2f, and SCDown alternately. The last layer of the backbone network is composed of SPPF and PSA. The C2f network in the 5th and 7th layers of the backbone network and the PSA network in the last layer output feature maps of three resolutions, denoted as P3, P4, and P5. The feature aggregation network is composed of Upsample, C2f, Conv, and Concat. The resolution of P3, P4, and P5 is from high to low. Therefore, the feature aggregation network mainly uses upsampling, convolution, and channel splicing to fuse the other two feature maps into another feature map, thereby generating a fused feature map, which is denoted as P6, P7, and P8 respectively.

[0038] The resulting P6, P7, and P8 feature maps are fed into the final detection head network. The detection head network consists of a one-to-one network and a one-to-many network, generating one-to-one and one-to-many predictions for different targets. The two networks share parameters to improve the learning accuracy of the detection head network. During the inference phase, only one-to-one predictions are performed to increase efficiency. The detection head network processes the predicted coordinates, category, and confidence level, and after appropriate post-processing, the prediction result is obtained.

[0039] The loss functions used by the Yolov10 model are CIou Loss, Distribution Focal Loss, and BCELoss. The first two are regression losses, which are used to calculate the loss between the predicted coordinate frame and the true coordinate frame. The latter is classification loss, which is used to calculate the loss between the predicted category and the true category. The loss function formula is as follows:

[0040]

[0041] The formula is CIou loss, ρ 2 (b,b gt ) is the square of the Euclidean distance between the center point of the predicted box and the true box, c 2 is the square of the diagonal length of the minimum bounding rectangle containing the predicted box and the true box, θ and θ gt are the arctangent of the aspect ratios of the predicted box and the ground-truth box, respectively, and α is a dynamic weight used to balance the contributions of scale invariance and aspect ratio consistency.

[0042]

[0043] The formula is Distribution Focal Loss, where q(t) is the distribution predicted by the model, p(t) is the true distribution, and t is the coordinate of the predicted box.

[0044] BCE(p,y)=-ylog(p)-(1-y)log(1-p)

[0045] The formula is cross entropy loss, p is the confidence of the prediction, and y is the true label category.

[0046] The tracking algorithm used in the embodiment of the present application is Botsort. This method uses the output of the target detection model as input, and uses an improved Kalman filter and re-identification technology to associate the detection results with historical trajectories to improve the accuracy of target tracking. It also adds camera motion compensation to the coordinate frame prediction to solve the problem of inaccurate coordinate frame prediction in the moving state. The Kalman filter combines the system's dynamic model and observation data to estimate the system's state while reducing the impact of noise. It mainly completes the system estimation by predicting and updating two states. The formulas of the state equation and observation equation are as follows:

[0047] x k =F k x k-1 +B k u k +w k

[0048] where x k is the state vector, the aspect ratio in the improved state vector is width to height, F k is the state transfer matrix, B k is the control input matrix, u k is the control input vector, w k is the process noise, which is usually assumed to be Gaussian white noise with zero mean.

[0049] z k =H k x k +v k

[0050] where z k is the observation vector, H k is the observation matrix, v k For observation noise, it is usually assumed to be Gaussian white noise with zero mean.

[0051] Furthermore, Deepstream is used to build a video stream pulling and pushing system, and the built abnormal event detection and tracking model is integrated into the system. By reading the video stream in real time, the tracking frames and tracking IDs of different targets are obtained, and this information is used to judge subsequent abnormal events.

[0052] For identified fireworks targets, the image is clipped using a tracking frame to obtain the fireworks area, which is then grayscaled and processed with Canny edge detection to obtain the fireworks highlight area. The area and center coordinates of this area are calculated, and whether it is fireworks is determined by judging whether there is any change in the area and the distance to the center coordinates frame by frame.

[0053] In this embodiment, the methods used to judge abnormal fireworks events include image grayscale, Otsu's method, and distance calculation. Opencv is used to implement image grayscale and Otsu's method. The highlighted area of the fireworks is obtained using this method, and the area value of the fireworks area in the frame is calculated. The distance calculation is obtained using the Euclidean formula. The area value and distance value are compared using frame-by-frame information. When the area value changes in size and the distance value is less than a certain threshold, the threshold is usually very small and can be pre-specified. The area value and distance value of the current frame are compared with the next frame. The formula is as follows:

[0054] x c =(x1+x2) / 2

[0055] y c =(y1+y2) / 2

[0056] The formula is the center point coordinate calculation formula, x c and y c is the target center coordinate, x i and y i are the coordinates of the upper left and lower right corners of the coordinate box,

[0057]

[0058] The formula is the Euclidean distance calculation formula, d represents the distance between two points, x1, x2, y1, y2 represent two center points, and the Otsu method is an automatic threshold calculation method that uses a binary image to calculate the inter-class variance at a certain threshold. The threshold at which the inter-class variance is the largest is recorded as the threshold with the best segmentation effect. The calculation formula for the inter-class variance is as follows:

[0059]

[0060] in the formula is the between-class variance, w0 and w1 represent the proportion of pixels below and above the t threshold, and μ0 and μ1 represent the average grayscale values of the previous two sets.

[0061] To identify crowd gatherings, the obtained tracking frames and tracking IDs are used to calculate the coordinates of the center point of each tracking ID and the number of people within the entire area. The distance between the center points of each tracking ID is determined frame by frame. When the distance is less than a threshold and the number of people within the area exceeds a specified threshold, an alarm is generated for an abnormal crowd gathering event in that area. The distance calculation formula used in this embodiment is the same as that used for fireworks recognition. The number of people threshold is configured based on the on-site situation, generally estimated based on the area size.

[0062] To determine a person's lingering event, the tracking frame and tracking ID are obtained, and the tracking ID, the coordinates of the person's center point, and the current frame time are calculated. The cumulative appearance time of the ID in the current area and the distance from the initial center point are recorded frame by frame. When the distance is consistently less than the current area range and the appearance time exceeds a certain threshold, the person corresponding to the ID is considered to have lingered. In this embodiment, the area range is determined by the resolution of the on-site camera. If a person remains within the area, they are considered to have not left the area. The time threshold needs to be configured in advance and is generally set to half an hour. These parameters are all written in the configuration file. The distance calculation formula uses the Euclidean formula, the same as for fireworks recognition.

[0063] The judgment of fence climbing events is applicable to the monitoring of the fences between the reservoir and the outside world and the fences of important facilities in the reservoir. The personnel outside the fence are continuously tracked and the tracking ID and tracking frame coordinates are recorded. The tracking frame is used to capture the target image in real time. The pre-set fence coordinates are used to use the vector cross product method to determine whether the personnel are on the same side of the fence. When the same ID is judged to exist on both sides of the fence and the similarity of the target images between the previous and next frames is greater than the specified threshold, it can be judged that a fence climbing event has occurred, and the alarm information is promptly drawn into the image and pushed to be displayed in the video stream. In this embodiment, the fence climbing event is judged by the tracking ID and the target cutout. The fence coordinates are generally linear areas. The vector cross product method is used to determine whether the center coordinates between frames are on the same side of the linear area. If the center point coordinates of the current tracking ID appear on both sides and the target cutout similarity exceeds the specified threshold, it can be judged that the person corresponding to the ID has climbed over the fence. The similarity threshold is generally set to 0.9. If it exceeds 0.9, it is considered that the two are the same person. The current alarm information is displayed in the video stream. The similarity judgment adopts the cosine similarity method. The vector cross product method and the similarity calculation method are as follows.

[0064] P i A=(x A -x p ,y A -y p )

[0065] P i B=(x B -x p,y B -y p )

[0066]

[0067] Where A and B represent the coordinates of the center point of the person in the previous frame and the current frame, P i 、P i+1 is the endpoint of the line, P i A.P. i B.P i P i+1 Point P i Vector to point A, point P i Vector to point B, point P i To point P i+1 The vector, P i A×P i P i+1 、P i B×P i P i+1 are vectors P i A and P i B and vector P i P i+1 The cross product of x i 、y i are the coordinates of the corresponding points,

[0068]

[0069] Where A·B represents the dot product of vector A and vector B, ||A||·||B|| represents the norm product of vector A and vector B, and the cosine similarity ranges from [-1, 1]. Values close to 1 indicate similarity, otherwise they are dissimilar.

[0070] The above is an abnormal event detection method for reservoir supervision provided by an embodiment of the present application. Based on the same inventive concept, an abnormal event detection system for reservoir supervision is also provided in an embodiment of the present application. Figure 2 The composition diagram of an abnormal event detection system for reservoir supervision provided in an embodiment of the present application is as follows: Figure 2As shown, the system mainly includes: a video stream reading module 201, which is used to use DeepStream to read the reservoir site video stream and decode each frame of image; an abnormal event judgment module 202, which is used to load each frame of image for target recognition and tracking, and load configuration file parameters for use in abnormal events, adopt corresponding post-processing methods according to different target categories, filter whether there are abnormal events in the target, and return alarm information if there are abnormal events; a video stream pushing module 203, which is used to receive each frame of image and alarm information, complete the drawing of the alarm information on the corresponding frame, save the frame as an alarm image, and use the push stream module to push each frame of image to the specified stream address for real-time display.

[0071] The above is an abnormal event detection system for reservoir supervision provided by the embodiment of the present application. Based on the same inventive concept, the embodiment of the present application also provides an abnormal event detection device for reservoir supervision. Figure 3 A schematic diagram of an abnormal event detection device for reservoir supervision provided in an embodiment of the present application is shown as follows: Figure 3 As shown, the device mainly includes: at least one processor 301; and a memory 302 communicatively connected to the at least one processor; wherein the memory 302 stores instructions that can be executed by the at least one processor 301, and the instructions are executed by the at least one processor 301 so that the at least one processor 301 can complete the aforementioned abnormal event detection method for reservoir supervision.

[0072] Two test examples of this application are specifically shown below.

[0073] Example 1 Detection of abnormal events of fireworks in reservoirs

[0074] Application Scenario: This system was deployed at a large reservoir to monitor in real time whether illegal fireworks were occurring around the reservoir area. The area was densely covered with vegetation, and illegal picnics were common among tourists, posing a fire hazard. The system needed to quickly identify and issue an alert at the initial stage of fireworks to prevent them from spreading.

[0075] Specific process: (1) Hardware deployment: The camera model is DS-2CD3T86FWDV3-I8 4K high-definition camera (resolution 3840×2160, frame rate 30fps), installed at 10 commanding heights around the reservoir, covering the entire area. The processing equipment uses the NVIDIA Jetson AGX Orin edge computing platform (64GB memory, equipped with a 256-core GPU), deployed in the monitoring center. Network architecture: Video streams are transmitted via optical fiber with a delay of less than 50ms.

[0076] (2) Model training and deployment: Dataset: 5,000 images of fireworks at the reservoir site (including daytime, nighttime, and foggy scenes) were collected, supplemented by the public dataset FireNet (20,000 images), and labeled as "fireworks". Training parameters: Yolov10 model input size: 640×640; training cycle: 300 epochs, batch size = 32; optimizer: AdamW, initial learning rate 0.001, cosine annealing strategy.

[0077] The model performance is shown in Table 1 below:

[0078] Table 1 Fireworks scene model performance

[0079] index training set Validation set Precision 98.5％ 95.2％ Recall 97.8％ 93.6％

[0080] The fire detection logic is as follows:

[0081] Image preprocessing: (1) Grayscale conversion: Use OpenCV's cv2.cvtColor to convert the RGB image to grayscale. (2) Otsu's binarization method: Determine the threshold by maximizing the inter-class variance and segment the highlight area.

[0082] Dynamic analysis: (1) Area change: Calculate the pixel area of the fireworks area in consecutive frames. If the area growth rate is greater than 10% (formula: ΔS=(S t+1 -S t ) / St×100%), it is determined that the fire is spreading. (2) Center point distance: Calculate the Euclidean distance between the center points of adjacent frames. If the distance is less than 5 pixels (actually about 0.5 meters), it is determined that they are from the same fire source.

[0083] The test data and results are as follows: (1) Test scene: simulated picnic fireworks (lasting 10 minutes, a total of 18,000 frames of video).

[0084] (2) The test results of the fireworks scene are shown in Table 2 below.

[0085] Table 2 Fireworks scene test results

[0086] Frame Range Fireworks detection count False positives Response time 1-6000(initial burning) 5985 15 35ms 6001-12000 5992 8 32ms 12001-18000 5998 2 30ms

[0087] Alarm triggering: The system issues an alarm 3 seconds after the initial combustion (90th frame), and the sprinkler system is activated.

[0088] The calculation example is as follows: (1) Area change judgment: The area of the fireworks area in the tth frame S t =1200S t =1200 pixels, t+1 frame S t+1 =1380S t+1=1380 pixels, ΔS=(1380-1200) / 1200*100%=15%, 15%>10%, triggering an alarm.

[0089] (2) Center point distance calculation: The center point of the t-th frame is (x1, y1) = (500, 300) (x1, y1) = (500, 300), and the t+1-th frame is (x2, y2) = (502, 302) (x2, y2) = (502, 302).

[0090] If the number of pixels is less than 5, it is determined to be the same fire source.

[0091] Example 2: Detection of Reservoir Fence Climbing Events

[0092] Application Scenario: A city has a 2.5-meter-high fence around its drinking water reservoir, but the risk of illegal intrusion still exists. The system needs to monitor human activity on both sides of the fence in real time and accurately identify any attempts to cross it.

[0093] Implementation steps: (1) Hardware deployment: The camera uses DH-IPC-HFW5849H-AS-PV 8-megapixel starlight-level camera (supports thermal imaging), and one is deployed every 50 meters along the fence, for a total of 40 cameras. The edge device uses Atlas 500Pro (equipped with Ascend 910B AI chip) to achieve local processing.

[0094] (2) The model optimization process is as follows: 100 hours of video footage of the fenced area (including scenes of climbing over, wandering, and normal traffic) is collected and labeled as "climbing" as the dataset. Tracking algorithm: Botsort parameters are adjusted to: Kalman filter noise covariance: Q = 0.1, R = 0.5, re-identification threshold: cosine similarity > 0.85. Model performance: climbing detection accuracy rate is 96.3%, and recall rate is 94.7%.

[0095] (3) Logic judgment for climbing over, fence coordinate calibration: Use a calibration tool to delineate the fence line (e.g., coordinates of two points (100, 200) to (500, 200)). Vector cross product method: Calculate the positional relationship between the person's center point and the fence line. If the same ID switches from the outside to the inside of the line in adjacent frames, and the image similarity is greater than 0.9, it is determined to be a climbing over. Similarity calculation: Use ResNet-18 to extract target features and calculate cosine similarity.

[0096] Test data and results:

[0097] Test scenario: simulate 10 climbing actions (5 successful, 5 interference actions such as approaching the fence).

[0098] The test results are shown in the following table:

[0099] Event Type Detection number False positives Response delay Real Crossing 5 0 40ms Interference action 0 1 -

[0100] Alarm effect: The system triggered an alarm 0.5 seconds before the climbing action was completed (15th frame), and security personnel intercepted it in time.

[0101] The calculation example is as follows: fence endpoints P1 (100, 200), P2 (500, 200), person center point A (150, 180) (outside), next frame B (150, 220) (inside).

[0102] Compute the cross product:

[0103] Cross A =(150-100)(200-200)-(180-200)(500-100)=0-(-20×400)=8000>0;

[0104] Cross B =(150-100)(200-200)-(220-200)(500-100)=0-(20×400)=-8000<0.

[0105] The sign of the cross product changes, and the determination position switches.

[0106] Image similarity:

[0107] Extract feature vector V t =[0.2,0.8,-0.1],V t+1 =[0.18,0.79,-0.09],

[0108]

[0109] If they are judged to be the same target, a climbing alarm will be triggered.

[0110] This application provides an abnormal event detection method, system, and equipment for reservoir supervision. By using computer vision methods, abnormal events occurring in the reservoir can be detected in a timely manner, making it easier for on-duty personnel to detect and take action in a timely manner, thereby minimizing equipment and property losses in the reservoir. This reduces the inspection pressure on on-duty personnel and effectively improves the efficiency of detecting abnormal events. The Yolov10 model is a significantly improved model in the Yolo series. By removing non-maximum suppression, the model inference speed is accelerated, thereby improving the efficiency of video stream AI analysis. The model is built using the Ultralytics library, and the tracking method uses the Botsort model, which further enhances the accuracy of the recognition results.

[0111] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0112] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0113] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0114] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0115] The various embodiments in this application are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the partial description of the method embodiments.

[0116] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0117] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for detecting abnormal events for reservoir supervision, characterized in that: The method comprises the following steps: Step S1: Obtain video image data captured by cameras installed around the reservoir; Step S2: Inputting the video image data into a pre-trained reservoir abnormal event detection and tracking model for prediction and identification; wherein the reservoir abnormal event detection and tracking model is built using a Yolov10 model that has undergone multiple iterations of training and integrates the Ultralytics algorithm and the Botsort tracking algorithm. The Yolov10 model's detection head network consists of a one-to-one network and a one-to-many network, which are used to generate one-to-one predictions and one-to-many predictions for different targets. At the same time, only one-to-one predictions are performed during the inference phase. Step S3: Output the identified abnormal event type.

2. The abnormal event detection method for reservoir supervision according to claim 1, characterized in that: The types of abnormal events include: fireworks, gatherings of people, people staying, and climbing over fences.

3. The abnormal event detection method for reservoir supervision according to claim 1, characterized in that: The network structure of the Yolov10 model consists of a backbone network, a feature aggregation network, and a detection head network. The backbone network adopts CSPDarknet53, which is composed of Conv, C2f, and SCDown alternately. The last layer of the backbone network is composed of SPPF and PSA. The C2f network in the 5th and 7th layers of the backbone network and the PSA network in the last layer output feature maps of three resolutions, which are recorded as P3, P4, and P5. The feature aggregation network consists of Upsample, C2f, Conv, and Concat. The resolution of P3, P4, and P5 is from high to low. Therefore, the feature aggregation network mainly uses upsampling, convolution, and channel splicing to fuse the other two feature maps into another feature map, thereby generating fused feature maps, which are recorded as P6, P7, and P8 respectively. The obtained P6, P7, and P8 feature maps are respectively input into the final detection head network. After processing by the detection head network, the predicted coordinates, categories, and confidence levels are obtained. After corresponding post-processing, the prediction results are obtained.

4. The abnormal event detection method for reservoir supervision according to claim 1, characterized in that: The abnormal event detection and tracking model uses Deepstream to build a method for pulling and pushing live video streams, and integrates abnormal event detection and tracking models to form a complete abnormal event judgment model, which analyzes the video stream in real time to obtain the tracking box, category, and tracking ID of the detection target.

5. The abnormal event detection method for reservoir supervision according to any one of claims 1 or 2, characterized in that: The judgment of abnormal fireworks events is specifically as follows: OpenCV is used to implement image grayscale and Otsu's method to obtain the highlighted area of the fireworks; the area value of the fireworks area in the frame is calculated, and the distance between the center points is obtained using the Euclidean formula. The area value and distance value are compared using frame-by-frame information. When the area value changes in size and the distance value is less than a certain threshold, the threshold is read from the configuration file, and the area value and distance value of the current frame are compared with the next frame.

6. The abnormal event detection method for reservoir supervision according to any one of claims 1 or 2, characterized in that: The judgment of abnormal events of personnel gathering is specifically as follows: the personnel tracking frame and tracking ID are generated through the reservoir abnormal event detection and tracking model, the center point coordinates of each tracking ID and the number of people in the entire area are calculated, and the distance between the center points of each tracking ID is judged using frame-by-frame information. When the distance is less than the threshold and the number of people in the area exceeds the specified threshold, an alarm information of abnormal event of personnel gathering in the area is generated.

7. The abnormal event detection method for reservoir supervision according to any one of claims 1 or 2, characterized in that: The judgment of personnel stay events is specifically as follows: through the reservoir abnormal event detection and tracking model, the personnel tracking frame and tracking ID are generated, the coordinates of the personnel center point of the tracking ID and the current frame time are calculated, and the cumulative appearance time of the ID in the current area and the distance from the initial center point are recorded frame by frame. When the distance is always less than the range of the current area and the appearance time exceeds a certain threshold, it can be judged that the person corresponding to the ID has a stay event. The area range is confirmed based on the scene captured by the on-site camera.

8. The abnormal event detection method for reservoir supervision according to any one of claims 1 or 2, characterized in that: Fence-crossing events are determined by continuously tracking people outside the fence and recording their tracking IDs and tracking frame coordinates. The tracking frame is used to capture the target image in real time. Using the pre-set fence coordinates, a vector cross product method is used to determine whether people are on the same side of the fence. If the same ID is determined to exist on both sides of the fence and the similarity of the target images between the previous and next frames exceeds a specified threshold, a fence-crossing event is detected. Alarm information is then promptly drawn into the image and displayed in the video stream.

9. An abnormal event detection system for reservoir supervision, characterized in that: The system comprises: The video stream reading module is used to use DeepStream to read the reservoir site video stream and decode each frame of the image; The abnormal event judgment module is used to load each frame of image for target recognition and tracking, and load configuration file parameters for abnormal event use. It uses corresponding post-processing methods according to different target categories to filter whether there are abnormal events on the target. If there are abnormal events, it will return an alarm information; The video stream push module is used to receive each frame of image and alarm information, complete the drawing of the alarm information on the corresponding frame, save the frame as the alarm image, and use the push stream module to push each frame of image to the specified stream address for real-time display.

10. An abnormal event detection device for reservoir supervision, characterized in that: The device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can complete the abnormal event detection method for reservoir supervision described in any one of claims 1-8.

Images