Debris Flow Video Recognition Model Sample Set and Model Construction Method, Edge Monitoring System

Abstract

The present invention discloses a sample set and model construction method for a debris flow video recognition model and an edge monitoring system. Aiming at the defects of high computational overhead and dependence on hardware performance in the prior art, the present invention provides a debris flow edge monitoring solution. The sample set construction method of the debris flow video recognition model uses grayscale images to represent the frame image information of the monitoring video. The sample data takes into account three objectives: including disaster dynamics, removing environmental interference, and simplifying image feature expression, while meeting the training requirements of the recognition model and adapting to the performance of edge devices. The debris flow video recognition model construction method provides an improved Tiny Darknet model for training. The debris flow edge monitoring system is an automatic monitoring and early warning system with low cost, stable operation, and excellent performance, significantly reducing the dependence of intelligent monitoring on the communication network, enabling the disaster monitoring and early warning to be completely advanced to the original field monitoring position, facilitating the implementation of multi-point field layout, and enhancing systematic monitoring and early warning.

Description

Technical Field

[0001] The present invention relates to a monitoring technology based on big data and machine learning, and particularly to a debris flow video recognition and monitoring model implemented based on video image recognition, as well as a debris flow automatic monitoring system adapted to edge devices, belonging to the technical fields of data recognition, electrical digital data processing, geological disaster monitoring and early warning calculation models and systems. Background Art

[0002] The debris flow intelligent early warning system automatically and real-timely collects and calculates the debris flow disaster environment signals, and outputs the disaster precursor and process analysis results, providing timely data information support for scientific decision-making such as disaster occurrence early warning, disaster process monitoring, and disaster emergency response. In such a system, the video signal collected by video monitoring is an important original disaster situation information. As a signal type recording geological disaster environment information, the monitoring video has many advantages such as rich and intuitive information, clear spatial information, strong multi-target tracking ability, and support for image recognition and understanding, but also faces disadvantages such as huge data volume, high analysis cost, strong computing power requirement, and high equipment construction cost.

[0003] Since the video signal analysis has high requirements for both analysis technology and equipment performance, in the traditional debris flow monitoring system, the video signal collected by the video monitoring terminal needs to be transmitted to the system center / core computing unit for calculation and analysis. Since the video monitoring terminal for debris flow occurrence prediction is usually arranged in the upstream formation area of the gully, while the center / core computing unit is usually arranged in the downstream outlet area of the gully with relatively better economic and transportation conditions, the early warning system needs to bear a huge video signal transmission cost. With the rise and development of edge computing technology, the prior art has developed a solution for realizing debris flow video recognition and early warning by using edge computing.

[0004] The prior art "Fluid Disaster Detection and Early Warning Platform Based on Attention Mechanism Flow Network" (CN 117935487 A) provides a method for monitoring and recognizing debris flow video streams, which uses a combination of deep learning algorithms and traditional opencv algorithms to detect the boundaries of fluid disasters such as debris flows and mountain floods in the video stream, and then compares with the early warning boundary threshold line to determine whether to trigger a disaster early warning. Among them, the algorithm includes two steps. Algorithm one is an attention-based flow network algorithm, and algorithm two is the overall algorithm for video stream detection. Algorithm one uses the attention mechanism to calculate the correlation between the features of two frames of images, improves the dynamic change situation between the two frames of images, inputs the two consecutive frames of images into this network, and finally outputs the motion changes on these two consecutive frames of images through this algorithm. On this basis, algorithm two uses skip calculation to filter non-mountain flood and other background factors to improve the detection effect of fluids such as mountain floods.

[0005] The above-mentioned existing technologies have two defects: First, the analysis and processing of debris flow videos still use the traditional method of extracting image features or flattening image features and inputting them into the algorithm process to output analysis results, and the computational complexity is still huge. Because the first algorithm adopts this technical concept, the flow network of the simple attention mechanism can detect any significant changes in two frames of images, which brings serious interference to the detection of mountain flood fluids. Therefore, it is necessary to superimpose the second algorithm to eliminate the redundant features extracted by the first algorithm from interfering with the detection and recognition of mountain flood fluid targets. Second, although edge computing devices are introduced into the debris flow monitoring system, the overall technical concept of the early warning plan does not show adjustments suitable for edge computing. In other words, extracting flattened features, repeatedly iterating and calculating around image features, eliminating redundant features, etc. in this algorithm all require relatively large computational overhead; the technical solution can be implemented on edge devices mainly relying on current high-performance edge computing devices, rather than reducing the computing power requirements brought by improving the computing logic and refining the algorithm in video recognition, and the cost of the technical solution is relatively high. Summary of the Invention

[0006] The purpose of the present invention is to address the deficiencies of the existing technologies, provide a new logic for analyzing and calculating debris flow monitoring videos, thereby efficiently constructing a sample set for a debris flow video recognition model; and further provide a method for constructing a debris flow video recognition model, as well as a method for constructing a debris flow intelligent monitoring system on low-cost edge computing devices.

[0007] To achieve the above purpose, the present invention first provides the following method for constructing a sample set of a debris flow video recognition model.

[0008] A method for constructing a sample set of a debris flow video recognition model,

[0009] Obtain a monitoring video V and video-related parameters;

[0010] Divide the monitoring video V into positive sample videos containing debris flow activity content PV and negative sample videos without debris flow activity content NV ;

[0011] Construct a positive sample set PS , including:

[0012] Step S100, read the positive sample video PV , mark the first frame as the background frame Fr 0;

[0013] Step S200, set a cache frame queue of size n QC , load the queue QC , the first frame of the queue QC is denoted as Fr 1, the last frame is denoted asFr e ,

[0014] Step S300, read the positive sample video frame by frame PV to the current frame Fr i ,

[0015] Step S310, analyze the first current frame Fr i,i=1 , and enter Step S330;

[0016] Step S320, starting from the second current frame Fr i,i>1 for analysis, execute:

[0017] Step S321, determine whether the previous frame Fr i-1 is a debris flow frame PPF , if so, enter Step S322, if not, enter Step S330, where the debris flow frame PPF is a frame image containing debris flow targets;

[0018] Step S322, calculate the histograms of the current frame Fr i and the tail frame Fr e , calculate the percentage difference between the two histograms s ; if s > threshold S, cache the current frame Fr i to the queue QC as the tail frame Fr e , enter Step S330, if s ≤ threshold S, update Fr i to Fr i+1 and enter Step S320;

[0019] Step S330 includes:

[0020] Step S331, calculate the frame difference image between the current frame Fr i and the background frame Fr 0 and perform binarization processing to obtain the current long frame difference binary map LBI i ′, calculate the white proportion of the current long frame difference binary map LBI i ′, calculate w i , calculate w i and wi-1 Difference Δw i , where the w i-1 is the result value of the previous calculation w i ;

[0021] If Δw i > 0, save the current long frame difference binary image LBI i ' as the long frame difference binary image LBI i , and enter S332,

[0022] If Δw i ≤ 0, discard the current long frame difference binary image LBI i ' and update the current frame Fr i , count the number of discarded times k,

[0023] If k < threshold K, enter step S331,

[0024] If k ≥ threshold K, update the current background frame Fr 0 to the current first frame Fr 1, and enter step S320;

[0025] Step S332, calculate the frame difference image between the current frame Fr i and the first frame Fr 1 and perform binarization processing to obtain the current short frame difference binary image SBI i , and enter step S350;

[0026] Step S340 includes:

[0027] Step S341, perform an opening operation on the long frame difference binary image LBI i and save the result image OLBI i , perform an opening operation on the short frame difference binary image SBI i and save the result image OSBI i into the binary image queue QB , count the size m of the queue QB , if m < threshold M, repeatedly perform an opening operation on the short frame difference binary image SBI i and save the result image OSBI i into the queue QB until m = threshold M, and enter step S342;

[0028] Step S342, read the queue QB , and for the result image OSBI i Map the white pixels to the result image through M masks OLBI i to generate a multi-valued grayscale image MGI i and save it to the positive sample set PS , and update Fr i and enter Step S320; for any j th mask mapping , j = 1, 2, 3, … M, execute:

[0029] Map the queue QB the 1st to j th result images OSBI i to the result image through mask mapping OLBI i and set the number of grayscale values to j +2, where the 1st is 255, the j +2nd is 0, and the rest decrease in arithmetic progression according to the gradient d ;

[0030] Process all positive sample videos PV , and obtain the positive sample set PS ;

[0031] Process all negative sample videos NV , and construct the negative sample set in the same way as constructing the positive sample set PS ; [NS ;

[0032] The positive sample set PS and the negative sample set NS form the sample set of the debris flow video recognition model S .

[0033] The basic idea of the above method for constructing the sample set of the debris flow video recognition model is to use a grayscale image to represent the frame image information of the monitoring video. The key is to scientifically quantify the debris flow target, various dynamic and static elements in the complex environment, and the debris flow movement characteristics in the monitoring video with different grayscale values, so as to obtain a sample image with rich grayscale value information, so that the information required for debris flow video recognition can be expressed limitedly and fully, and become the basis for refined machine learning operations.

[0034] The above method for constructing the sample set of the debris flow video recognition model can be optimized in the following aspects. Each optimization scheme can be implemented independently or simultaneously on the premise of no conflict.

[0035] Optimization 1. Queue QC The size n is determined according to Equation 1. Among them, FPS max —The maximum frame rate of the camera (unit: fps), D —The distance between the camera and the monitoring target (unit: m).

[0036] Equation 1

[0037] Optimization 2. The value of the threshold K is the maximum frame rate of the camera FPS max .

[0038] Optimization 3. Threshold M and gradient d Satisfy the relationship of Equation 2.

[0039] Equation 2.

[0040] Optimization 4. Respectively, all multi-valued grayscale images in the positive sample set PS and the negative sample set NS are MGI image-enhanced and then combined into a sample set S .

[0041] Using the above method for constructing the sample set of the debris flow video recognition model, the present invention also provides the following method for constructing the debris flow video recognition model.

[0042] A method for constructing a debris flow video recognition model, using the above method for constructing the sample set of the debris flow video recognition model to construct a sample set S , and adopting machine learning to train the debris flow video recognition model.

[0043] For the above method for constructing the debris flow video recognition model, to adapt to the performance of edge computing devices, machine learning can select the Tiny Darknet model as the basis and make improvements.

[0044] The present invention also provides the following debris flow edge monitoring system.

[0045] A debris flow edge monitoring system, the hardware system takes the edge device as the core device, and the core device is connected to the debris flow monitoring camera; the software system includes a video acquisition module, a sample set construction module, a model training module, and a result output module; the video acquisition module obtains the debris flow monitoring video from the debris flow monitoring camera V , the sample set construction module executes the above method for constructing the sample set of the debris flow video recognition model to construct a sample set S , the model training module installs the above debris flow video recognition model, and the result output module outputs the debris flow recognition result.

[0046] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The present invention discloses a technical concept for analyzing and identifying debris flow monitoring videos to achieve dynamic monitoring of debris flows, aiming to solve the ultimate technical problem of providing a low-cost edge monitoring solution applicable to the field of debris flow channels. To solve the ultimate technical problem, the technical concept of the present invention designs two technical improvement routes. One is to design the final solution as an edge computing system, that is, to select an edge computing device with certain computing capabilities but low cost as the core component of the system, and ensure the computing efficiency and stable operation of the system, so as to be able to promote and apply in the field of debris flow channels in the wild where the economy, transportation, and infrastructure conditions are relatively backward. The other is to support the aforementioned improvement route, abandon the concept of directly extracting feature analysis from video data in the prior art, transform the video frame image feature analysis problem into a grayscale feature analysis problem of grayscale images, make full use of the 256 grayscale values and their corresponding image meanings in the grayscale images, and reduce the huge and complex information of the debris flow and its environment captured by the monitoring video to a sufficient and limited range, thereby greatly reducing the computing overhead and adapting to the low-cost system resources with edge computing devices as the core. (2) To provide the final solution, the method for constructing a sample set of the debris flow video recognition model of the present invention provides a method for extracting image sample data from debris flow monitoring videos. Each image sample is a multi-value grayscale image containing the dynamic information of the debris flow within a time window, and different grayscale values in the image represent the debris flow morphology and environmental elements at different moments. Thus, the sample data used for machine learning training takes into account three objectives: including disaster dynamic information, removing interference from harsh environments, and simplifying the form of image feature expression, and can simultaneously meet the requirements of the recognition model training reaching the standard and the model running on edge computing devices. This technical concept has never been discussed in the existing debris flow automatic monitoring and early warning technologies. (3) Based on the results of previous research, the method for constructing a debris flow video recognition model of the present invention provides an improved Tiny Darknet model as a machine learning training model. The improvement of the Tiny Darknet model is based on two aspects: theoretically, the requirement for the grayscale calculation amount of sample data; in the actual debris flow prevention and control project, the derivative information of the monitoring video lens perspective in pattern recognition. Therefore, the number of channels and the number of convolutional layers are reduced to balance the computing amount and the model performance requirements. (4) The debris flow edge monitoring system of the present invention is a low-cost, stable-running, and excellent-performance debris flow disaster automatic monitoring and early warning system. By arranging all the image analysis and calculations of the debris flow videos collected by the monitoring terminal on the edge computing device, it significantly reduces the dependence of intelligent monitoring on the communication network, enables the debris flow disaster monitoring and early warning to be "moved forward" to the field of debris flow channels, and effectively avoids the situation where the data transmission is interfered by the harsh environment during the disaster, restricting the function of the central computing unit, and causing large-scale automatic monitoring and early warning to be delayed, disconnected, or even ineffective.Moreover, since the debris flow edge monitoring system of the present invention has low investment and operation costs, it can be installed in more places in the debris flow hazard area, or installed specifically in the high-risk area of the debris flow channel to achieve multi-point control of the entire basin and enhance the systematic intelligent monitoring and early warning ability of disasters. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is to use positive sample videos PV to construct a positive sample set PS Schematic diagram of the main steps of the process.

[0048] Figure 2 is a certain positive sample video PV 100 Frame image in a certain analysis.

[0049] Figure 3 is a certain positive sample video PV 100 Binary image in a certain analysis.

[0050] Figure 4 is a certain positive sample video PV 100 Composite multi-value grayscale image in a certain analysis MGI i .

[0051] Figure 5 are the ROC curve and PR curve of the classifier.

[0052] Figure 6 is the monitoring video V test Frame image of.

[0053] Figure 7 is the monitoring video V test Grayscale image in recognition.

[0054] Figure 8 is a schematic diagram of the software system framework of the debris flow edge monitoring system.

[0055] Figure 9 is a simple diagram of the field layout of the debris flow edge monitoring system.

[0056] The numerical marks in the drawings are respectively:

[0057] 100 Video acquisition module; 200 Sample set construction module; 300 Model training module; 400 Result output module; 500 Data transmission module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0058] The preferred embodiments of the present invention will be further described below with reference to the accompanying drawings. Example 1

[0059] As Figures 1 - 4 shown, the debris flow video recognition model sample set is constructed by the method of the present invention.

[0060] 1. Obtain debris flow monitoring videos V

[0061] Obtain debris flow monitoring videos V , and the monitoring videos V include positive sample videos PV and negative sample videos NV . The content of the positive sample videos PV records debris flow activities, and the content of the negative sample videos NV does not record debris flow activities.

[0062] There are 21 positive sample videos in total, and the video duration ranges from 19s to 137s. The monitoring scenarios include debris flow flume simulation experiments and field sites of debris flow gullies. There are 18 negative sample videos PV in total, and the video duration ranges from 23s to 109s. The monitoring scenario is the field site of the debris flow gully and events or phenomena that interfere with debris flow recognition occur, such as rain, flowing / rising water in the gully, and wind shaking the surrounding plants. NV

[0063] At the same time, obtain the relevant parameters of each video, including the maximum frame rate of the camera FPS max (unit: fps) and the distance between the camera and the monitoring target D (unit: m).

[0064] 2. Use the positive sample videos PV to construct the positive sample set PS

[0065] Figure 1 is a schematic diagram of the main steps of the process of using the positive sample videos PV to construct the positive sample set PS .[[]]

[0066] Step S100, read the positive sample video PV , and mark the first frame as the background frame Fr 0.

[0067] When marking the background frame for the first time Fr 0, by default Fr 0 is a non-debris flow frame NPF , that is, a frame image that does not contain debris flow targets.

[0068] Step S200, set a cache frame queue of size n QC , load the queue QC , queue​​QC The first frame is denoted as Fr 1. The last frame is denoted as Fr e .

[0069] In this embodiment, the value of n is determined according to Equation 1. Taking a certain positive sample video PV 100 as an example for analysis, there is: FPS max = 24 fps, D = 376 m, n = 24 / 2 = 12.

[0070] Step S300, read the positive sample video frame by frame PV to the current frame Fr i , and perform the current frame analysis:

[0071] Step S310, the first current frame Fr i,i=1 analysis, enter step S330;

[0072] Fr i,i=1 During the analysis, since the first marked Fr 0 is NPF , so just directly enter step S330.

[0073] Step S320, the current frame analysis starting from the second current frame Fr i,i>1 onwards, execute:

[0074] Step S321, judge whether the previous frame Fr i-1 is a debris flow frame PPF (a debris flow frame PPF refers to a frame image containing a debris flow target), if so, enter step S322, if not, enter step S330;

[0075] Step S322, perform queue QC similarity analysis, including: calculate the histograms of the current frame Fr i and the last frame Fr e respectively, calculate the difference percentage of the two histograms s , and make a conditional judgment:

[0076] If s > threshold S, cache the current frame Fr i to the queue QC as the last frame Fr e , enter step S330, ifs ≤ threshold S, update Fr i as Fr i+1 and enter step S320.

[0077] In this embodiment, the threshold S takes a value of 1%.

[0078] Figure 2 is a certain positive sample video PV 100 a frame image in a certain analysis, where (a) background frame Fr 0, (b) current frame Fr i .

[0079] Step S330, perform queue QC frame difference processing, including:

[0080] Step S331, perform long frame difference processing, including:

[0081] Calculate the current frame Fr i and the background frame Fr 0 to obtain a frame difference image and perform binarization processing to obtain the current long frame difference binary image LBI i ′, calculate the white proportion of the current long frame difference binary image LBI i ′, calculate w i , calculate w i and w i-1 ( w i-1 refers to the result value of the previous calculation w i of, if it is the first calculation w i of, then w i-1 = 0) difference Δw i , make a conditional judgment;

[0082] If Δw i > 0, save the current long frame difference binary image LBI i ′ as the long frame difference binary image LBI i , enter S332,

[0083] If Δw i ≤ 0, discard the current long frame difference binary image LBI i' and update the current frame Fr i (that is, read the next frame to the current frame), count the number of discarded times k, and make a conditional judgment:

[0084] If k < threshold K, go to step S331.

[0085] If k ≥ threshold K, set the current background frame Fr 0 as the current first frame Fr 1 (that is, update the queue QC ), and enter step S320.

[0086] In this example, the threshold K takes the value FPS max = 24.

[0087] Step S332, perform short frame difference processing, including:

[0088] Calculate the frame difference image between the current frame Fr i and the first frame Fr 1 and perform binarization to obtain the current short frame difference binary image SBI i , and enter step S340.

[0089] Figure 3 is the binary image in a certain positive sample video PV 100 in a certain analysis, where (a) is the long frame difference binary image LBI i , (b) is the short frame difference binary image SBI i .

[0090] Step S340, synthesize a multi-value grayscale image, including:

[0091] Step S341, perform opening operation, including: perform opening operation on the long frame difference binary image LBI i , save the result image OLBI i , perform opening operation on the short frame difference binary image SBI i and store the result image OSBI i into the binary image queue QB , count the size m of the queue QB If m < threshold M, repeatedly perform opening operation on the short frame difference binary image SBI i and store the result image OSBI i into the queue QB until m = threshold M, and enter step S342.

[0092] In this embodiment, the long frame difference binary image LBI i and the short frame difference binary image SBI i are respectively processed by an opening operation with a 3×3 rectangular convolution kernel to remove minute image noises. Meanwhile, the value of M is determined according to Equation 2, and we have: d = 10, M = 24.

[0093] Step S342, perform image synthesis, including: reading the queue QB , and mapping the white pixels of the result image OSBI i to the result image OLBI i through M times of masking to generate a multi-value grayscale image MGI i and save it to the positive sample set PS , and update Fr i and enter Step S320.

[0094] During the masking, for any j -th ( j = 1, 2, 3, … M ) masking, execute: map the queue QB the 1st to j th result images OSBI i to the result image OLBI i and set the number of gray values of the masked result image to j + 2, where the 1st one is 255, the j + 2-th one is 0, and the rest decrease in arithmetic progression according to the gradient d , that is, the gray values of the result image are [255, (255 - d ), (255 - 2 d ), …, (255 -jd ), 0].

[0095] In this embodiment, the result image of the j = 1-time masking has 3 gray values: [255, 245, 0], and the result image of the j = 2-time masking has 4 gray values: [255, 245, 235, 0]. The result image of the j = M = 24-time masking has 26 gray values: [255, 245, 235, ..., 15, 0]. A multi-value grayscale image with 26 gray values is generated through 24-time masking MGI i and saved to the positive sample set PS .

[0096] Figure 4 is a certain positive sample video PV 100 The synthesized multi - valued grayscale image in a certain analysis MGI i 。

[0097] Process all positive sample videos PV ,obtain the positive sample set PS 。

[0098] In this embodiment, 21 positive sample videos PV are all processed by extracting frame images at 10 frames / s, and a total of 738 multi - valued grayscale images are obtained MGI ,forming the positive sample set PS 。

[0099] 3. Use negative sample videos NV to construct the negative sample set NS

[0100] In accordance with the same method as constructing the positive sample set PS , process each negative sample video one by one NV ,obtain the negative sample set NS 。

[0101] In this embodiment, 18 negative sample videos NV are all processed by extracting frame images at 10 frames / s, and a total of 692 multi - valued grayscale images are obtained MGI ,forming the negative sample set NS 。

[0102] 4. Obtain the sample set S

[0103] Merge the positive sample set PS and the negative sample set NS , shuffle the image order, and form the sample set of the debris - flow video recognition model S 。 Example 2

[0104] Use the method of the present invention to construct the sample set of the debris - flow video recognition model. The same parts as in Example 1 will not be repeated. The differences are that, to expand the sample set, pre - process the multi - valued grayscale images MGI 。

[0105] The pre - processing specifically uses image enhancement processing: the 738 multi - valued grayscale images obtained from 21 positive sample videos in Example 1 PV and the 692 multi - valued grayscale images obtained from 18 negative sample videos MGI and 18 negative sample videos NV obtained MGI ​​, perform random rotation (rotation angle 0° - 5°), enlargement, reduction, and addition of image noise (addition probability 5%) in sequence.

[0106] Obtain 4428 positive sample images to form a positive sample set PS ; obtain 4152 negative sample images to form a negative sample set NS . Combine them to form a sample set S , which contains 8580 multi-valued grayscale images MGI . Example 3

[0107] As Figures 5 - 7 shown, use the sample set S constructed in Example 2 to further construct a debris flow video recognition model using the method of the present invention.

[0108] 1. Divide the training set, test set, and validation set

[0109] Randomly divide the sample set S into a training set, a test set, and a validation set according to the ratio of 7:2:1.

[0110] 2. Train a binary classifier

[0111] The machine learning model is based on the Tiny Darknet model (Reference: Shi Lulu, et al. Journal of Nanjing University of Posts and Telecommunications (Natural Science Edition), 2018, 38(4): 89 - 95. DOI: 10.14132 / j.cnki.1673 - 5439.2018.04.013) and is improved to effectively reduce the data transmission volume and the data calculation and storage overhead.

[0112] The improvements to the Tiny Darknet model include: ① Reducing the number of channels: Since the images in the sample set are grayscale images with 1 color channel, the number of image channels in the original model is reduced to 1. The computational amount of the model in the convolutional layer is significantly reduced. Especially when the image size remains unchanged, the calculation is reduced by about 2 / 3, greatly improving the operation efficiency. ② Reducing the number of convolutional layers: Since there is a special design for the debris flow monitoring perspective and image target detection and positioning are not required in image analysis, the relevant convolutional layers for performing detection tasks are deleted; and since the redundant features of the grayscale image samples are few, the number of convolutional layers can be appropriately reduced. After comprehensive testing, it is most appropriate to reduce the 16 convolutional layers of the original model to 8.

[0113] Use the sample set S to train a binary classifier.

[0114] 3. Test the effect of the classifier

[0115] The obtained confusion matrix after training is shown in Table 1. In the table, TP is the number of samples correctly classified in the positive class, TN is the number of samples correctly classified in the negative class, FP is the number of samples in the negative class misclassified as the positive class, FN is the number of samples in the positive class misclassified as the negative class, and the number of samples N = 8580. The classifier has good balance in classifying positive and negative samples (as shown in Table 2).

[0116] Table 1 Confusion Matrix of Training Results

[0117] Predicted positive class Predicted negative class Actual positive class TP = 3823 FN = 482 Actual negative class FP = 452 TN = 3700

[0118] Table 2 Classifier Performance

[0119] Index Overall accuracy Ac Model precision Pr Recall Re F1 score Specificity Sp Score (TP + TN)) / N = 87.6% TP / (TP + FP) = 89.4% TP / (TP + FN)) = 88.9% 2Pr*Re / (Pr + Re) = 89.0% TN / (TN + FP) = 89.1%

[0120] The ROC and PR curves show that the AUC values of the classifier are both approximately 0.9, indicating that the classifier has good discrimination ability. Figure 5 They are the ROC curve and PR curve of the classifier.

[0121] 4. Example Verification

[0122] Use a debris flow monitoring video in Example 1 V other than the debris flow monitoring video V test to conduct an example verification on the classifier.

[0123] When reading and analyzing the 11th frame image, after classification by the classifier, the probability of identifying the image as a debris flow is 87.3%. The classifier performs excellently.

[0124] Figure 6 It is the monitoring video V test of the frame images, where (a) is the 1st frame Fr 1, (b) is the 5th frame Fr 5, (c) is the 11th frame Fr 11 ; Figure 7 It is the grayscale image in the monitoring video recognition V test where (a) is the long frame difference binary image LBI , (b) is the short frame difference binary image SBI , (c) is the multi-value grayscale image MGI .

[0125] The obtained classifier model after training is the constructed debris flow video recognition model. Example 4

[0126] As Figures 8 - 9 shown, use the debris flow video recognition model constructed in Example 3 to further develop a debris flow edge monitoring system.

[0127] The debris flow edge monitoring system includes a hardware system and a software system.

[0128] The hardware system takes the edge device as the core device. The core device is connected to the debris flow monitoring camera and also includes necessary input / output peripherals, power supply devices, and communication lines. In this embodiment, the edge device selects Raspberry Pi 4B, equipped with a 1.5G processor and 4G memory. The USB interface is connected to the debris flow monitoring camera (1280*720 pixels), the 5V 3A interface is connected to the solar power supply device, and the gigabit Ethernet card is connected to the 4G communication module communication line (such as SIM7600CE).

[0129] Figure 8 It is a schematic diagram of the software system framework of the debris flow edge monitoring system.

[0130] The software system includes a video acquisition module 100, a sample set construction module 200, a model training module 300, a result output module 400, and a necessary data transmission module 500. The video acquisition module 100 uses the data transmission module 500 to obtain the debris flow monitoring video from the debris flow monitoring camera V , and the sample set construction module 200 executes the method for constructing the sample set of the debris flow video recognition model of the present invention to construct the sample set S , the model training module 300 runs the debris flow video recognition model constructed in Embodiment 3, and the result output module 400 outputs the debris flow recognition result.

[0131] The software system is written into the edge device. Specifically: on the operating system of Raspberry Pi 4B, compile and install OpenCV in C++ version to implement the functions of obtaining real-time frames of the camera and performing image preprocessing. Subsequently, compile and install the debris flow video recognition model developed in Embodiment 3 onto Raspberry Pi 4B.

[0132] To test the compatibility between the edge device in this monitoring system and the debris flow video recognition model it carries (i.e., the model developed in Embodiment 3), a classification operation compatibility comparison (Table 3) is set. During the process from reading samples to completing classification, the method of the present invention can fully reach the maximum frame rate of the camera, 24fps, for video image recognition on Raspberry Pi 4B, that is, it can perform intelligent recognition of the captured video without delay.

[0133] Table 3 Classification operation compatibility comparison detection

[0134] Darknet method Method of the present invention Edge device Raspberry Pi 4B Raspberry Pi 4B Image sample Directly extract video frame image Multi - value grayscale image MGI Average maximum frame rate from reading the sample to completing classification 1.6 frames / s 4.5 frames / s Classification accuracy 69% 88%

[0135] The above debris flow edge monitoring system was installed in Jiangjiagou, Dongchuan District, Yunnan on June 3, 2024 for field debris flow real-time monitoring. Figure 9 It is a simple diagram of the field layout of the debris flow edge monitoring system.Figure 9 In the figure, the upper - figure camera icon indicates the installation position of the debris - flow ditch of the edge - monitoring system. The complete set of edge - monitoring systems is deployed on the installation column in the lower figure. That is, from the camera shooting to the output of the recognition result, it is all completed at the original site of the camera icon in the upper figure. The system has been running stably for 6 months, and the recognition results of the real - time monitoring video are consistent with the actual records of multiple observations.

Claims

1. A method for constructing a sample set of a debris flow video recognition model, characterized in that: Get surveillance video V And video related parameters; Monitor video V Classified as positive sample videos containing debris flow activities PV Negative sample videos that do not contain debris flow activity content NV ; Construct positive sample set[ PS ], include: Step S100, read the positive sample video PV , mark the first frame as the background frame Fr 0; Step S200, set a cache frame queue of size n QC , load queue QC ,queue QC The first frame is recorded as Fr 1. The last frame is recorded as Fr e , Step S300: Read the positive sample video frame by frame PV To current frame Fr i , Step S310, first current frame Fr i,i=1 Analyze and proceed to step S330; Step S320, from the second current frame Fr i,i>1 From the analysis, execute: Step S321, determine the previous frame Fr i-1 Is it a debris flow frame? PPF If yes, go to step S322, if no, go to step S330, the debris flow frame PPF is a frame image containing debris flow targets; Step S322, respectively calculate the current frame Fr i With end frame Fr e Histogram of , calculate the difference percentage between the two histograms s ;like s >Threshold S, cache current frame Fr i To Queue QC End frame Fr e , go to step S330, if s ≤ threshold S, update Fr i for Fr i+1 And proceed to step S320; Step S330 includes: Step S331, calculate the current frame Fr i With background frame Fr 0 frame difference image and perform binarization processing to obtain the current long frame difference binary image LBI i ′, calculate the current long frame difference binary image LBI i The white proportion of ′ w i ,calculate w i and w i-1 The difference Δw i , w i-1 The last calculation w i The result value of like Δw i >0, the current long frame difference binary image LBI i 'Save as a long frame difference binary image LBI i , enter S332, like Δw i ≤0, discard the current long frame difference binary image LBI i ′ and update the current frame Fr i , that is, read the next frame to the current frame, and count the number of discards k, If k<threshold K, go to step S331, If k ≥ threshold K, the current background frame Fr 0 is updated to the current first frame Fr 1, proceed to step S320; Step S332, calculate the current frame Fr i With the first frame Fr 1 frame difference image and perform binarization processing to obtain the current short frame difference binary image SBI i , proceed to step S340; Step S340 includes: Step S341: convert the long frame difference binary image LBI i Perform an open operation and save the result graph OLBI i , the short frame difference binary image SBI i Perform the opening operation and display the result graph OSBI i Store in binary image queue QB , count queue QB Size m, if m < threshold M, repeat the short frame difference binary image SBI i Perform the opening operation and display the result graph OSBI i Store in queue QB Until m=threshold M, go to step S342; Step S342, read queue QB , the result graph OSBI i White pixels are mapped to the result image through M masks OLBI i Generate a multi-value grayscale image MGI i And save it to the positive sample set [ PS ],renew Fr i And proceed to step S320; for any j Secondary Mask Mapping , j = 1, 2, 3, ... M, implement: Queue QB 1st~ j Results OSBI i Mask mapping to result map OLBI i And set the number of gray values ​​to j +2, of which the first is 255, the j +2 are 0, and the rest are based on the gradient d Arithmetic decrease; Process all positive sample videos PV , get the positive sample set[ PS ]; Process all negative sample videos NV , according to and construct the positive sample set [ PS ]The same method is used to construct negative sample sets[ NS ]; Positive sample set[ PS ] and negative sample set[ NS ]The sample set that constitutes the debris flow video recognition model[ S ].

2. The method for constructing a debris flow video recognition model sample set according to claim 1, characterized in that: The queue QC The size n is determined according to formula 1, Formula 1 In the formula, FPS max —Maximum frame rate of the camera, in fps, D —The distance between the camera and the monitoring target, in meters.

3. The method for constructing a debris flow video recognition model sample set according to claim 1, characterized in that: The threshold K is the maximum frame rate of the camera. FPS max .

4. The method for constructing a debris flow video recognition model sample set according to claim 1, characterized in that: The threshold M and the gradient d Satisfying formula 2, Formula 2.

5. The method for constructing a debris flow video recognition model sample set according to claim 1, characterized in that: The positive sample set [ PS ] and the negative sample set [ NS ] All multi-value grayscale images MGI After image enhancement, the sample set is formed [ S ].

6. The method for constructing a debris flow video recognition model sample set according to claim 5, characterized in that: The image enhancement includes: MGI Randomly rotate 0°~5°, zoom in, zoom out, and add 5% image noise.

7. A method for constructing a debris flow video recognition model, characterized in that: The sample set is constructed by using the method for constructing a debris flow video recognition model sample set according to any one of claims 1 to 6. S ], using machine learning to train a debris flow video recognition model.

8. The method for constructing a debris flow video recognition model according to claim 7, characterized in that: The machine learning adopts an improved Tiny Darknet model, and the improvements of the Tiny Darknet model include: setting the number of image channels to 1 and setting the number of convolutional layers to 8.

9. Debris flow edge monitoring system, characterized by: The hardware system uses an edge device as a core device, and the core device is connected to a debris flow monitoring camera; the software system includes a video acquisition module (100), a sample set construction module (200), a model training module (300), and a result output module (400); the video acquisition module (100) obtains debris flow monitoring video from the debris flow monitoring camera V The sample set construction module (200) executes the debris flow video recognition model sample set construction method according to any one of claims 1 to 6 to construct a sample set [ S ], the model training module (300) installs the debris flow video recognition model constructed according to claim 7 or 8, and the result output module (400) outputs the debris flow recognition result.

10. The debris flow edge monitoring system according to claim 9, characterized in that: The edge device is a Raspberry Pi, on which the C++ version of OpenCV and the debris flow video recognition model are compiled and installed.

Images