Debris flow disaster early warning method based on image recognition and related device

Abstract

The invention provides a debris flow disaster early-warning method based on image recognition, and the method comprises the steps: firstly obtaining two groups of comparison video stream data of a multi-view camera in different time periods, carrying out the mountain feature registration of the two groups of comparison video stream data to guarantee the consistency of space coordinates, filtering and removing invalid interference information through an interference region mask, and obtaining a pre-warning result; and then determining a mountain sliding association difference pixel point set according to the filtered video stream data, and finally constructing a three-dimensional mountain sliding monitoring model in combination with the set and the registered video stream data, thereby determining a target monitoring area of the detection equipment and carrying out monitoring and early warning on the target monitoring area. Spatial position deviation of video stream data in different time periods is eliminated through mountain feature registration, various interference factors in the natural environment are effectively eliminated in combination with interference area mask filtering, mountain sliding can be accurately recognized, the debris flow disaster early warning precision is remarkably improved, the early warning signal false alarm condition is reduced, and the early warning efficiency is improved. And potential safety hazards caused by false alarms are reduced.

Description

Technical Field

[0001] This application relates to the field of geological disaster early warning technology, specifically to a debris flow disaster early warning method and related device based on image recognition. Background Technology

[0002] Debris flows, as a highly destructive natural disaster, pose a serious threat to the lives and property of residents in mountainous areas and to infrastructure. Therefore, efficient and accurate early warning technologies have always been a key research focus in the field of disaster prevention and mitigation. Currently, existing debris flow early warning methods mainly include sensor-based monitoring methods and image recognition-based methods. Among them, image recognition-based early warning methods typically deploy cameras to collect video stream data of mountain areas, use computer vision technology to analyze the changing characteristics of the mountain surface in the video, such as identifying potential landslide signs by comparing differences in images from different time periods, and then issuing early warning signals in conjunction with monitoring data from monitoring equipment (such as radar, rain gauges, etc.).

[0003] However, due to various interference factors in the natural environment, such as changes in sky light and shadow, reflections from man-made buildings, instantaneous dynamic objects such as birds or falling rocks, and periodic dynamic changes such as wind-induced vegetation swaying, these interferences can easily be misinterpreted as landslide signals, leading to a decrease in the accuracy of debris flow disaster warnings and potentially causing false alarms of debris flow warning signals, thus posing safety hazards. Summary of the Invention

[0004] This application provides an image recognition-based debris flow disaster early warning method and related device, which can accurately identify landslide characteristics, significantly improve the early warning accuracy of debris flow disasters, reduce false alarms of early warning signals, and reduce safety hazards caused by false alarms.

[0005] A first aspect of this application provides an image recognition-based method for early warning of debris flow disasters, the method comprising: Acquire the first comparative video stream data of the multi-camera in the first time period and the second comparative video stream data of the multi-camera in the second time period; Mountain feature registration is performed on the first and second comparison video stream data to obtain registered first comparison video stream data and registered second comparison video stream data. Interference region masking is applied to the registered first comparison video stream data and the registered second comparison video stream data to obtain masked first comparison video stream data and masked second comparison video stream data. Based on the first comparison video stream data after mask filtering and the second comparison video stream data after mask filtering, determine the set of differential pixels associated with mountain sliding. Based on the set of differential pixels associated with landslides, the first and second comparison video streams, a three-dimensional landslide monitoring model is constructed. The target monitoring area of ​​the detection equipment is determined, and the detection equipment is controlled to monitor the target monitoring area for debris flow and issue a debris flow early warning signal.

[0006] In one possible implementation, the step of performing mountain feature registration on the first and second comparison video stream data to obtain registered first and second comparison video stream data includes: Extract stable feature points of the mountain from each frame of the first and second comparison video stream data; The stable feature points of the mountains in each frame of the first comparison video stream data are matched to determine the correspondence between the feature points of the first comparison video stream data. The spatial position of each frame of the first comparison video stream data is corrected according to the correspondence between the feature points of the first frame to obtain the first comparison video stream data with inter-frame registration. The stable feature points of the mountains in each frame of the second comparison video stream data are matched to determine the correspondence between the feature points of the frames in the second comparison video stream data. The spatial position of each frame of the second comparison video stream data is corrected according to the correspondence between the feature points of the frames to obtain the second comparison video stream data with inter-frame registration. Select any frame image from the first comparison video stream data of the inter-frame registration as the first reference frame, and perform matching calculations between the stable feature points of the mountains in each frame image of the second comparison video stream data of the inter-frame registration and the stable feature points of the mountains in the first reference frame to obtain the spatial transformation matrix. Based on the spatial transformation matrix, coordinate transformation is performed on each frame of the second comparison video stream data of the inter-frame registration to obtain registered second comparison video stream data with spatial coordinates consistent with the first comparison video stream data. The first comparison video stream data of the inter-frame registration is used as the first comparison video stream data for registration.

[0007] In one possible implementation, extracting stable feature points of the mountain from each frame of the first and second comparison video stream data includes: Preprocess each frame of the first and second comparison video stream data, performing grayscale conversion and Gaussian filtering noise reduction sequentially to obtain a denoised grayscale image. Feature point detection is performed on the denoised grayscale image to obtain an initial feature point set; Obtain the initial mountain calibration range of the multi-view camera and the corresponding GIS terrain data; Based on the initial mountain calibration range and GIS terrain data, the sky area, artificial building area and dense vegetation area are removed, and the static mountain mask corresponding to the pure mountain area in the denoised grayscale image is determined. The initial feature point set is filtered based on the static mountain mask to remove non-mountain feature points outside the static mountain mask, resulting in a candidate feature point set. The gray-level gradient stability of each feature point in the candidate feature point set is calculated to obtain the gray-level stability calculation result of the feature point; Based on the grayscale stability calculation results of the feature points, the stable feature points of the mountain that meet the preset conditions in the candidate feature point set are determined, and the stable feature points of the mountain are obtained.

[0008] In one possible implementation, calculating the gray-level gradient stability of each feature point in the candidate feature point set to obtain the feature point gray-level stability calculation result includes: For each feature point in the candidate feature point set, a pre-defined neighborhood region is selected in the denoised grayscale image, centered on the candidate feature point. Calculate the grayscale gradient value of each pixel in the horizontal and vertical directions within the neighborhood, and determine the horizontal gradient matrix and vertical gradient matrix of all pixels in the neighborhood. Based on the horizontal and vertical gradient matrices, determine the overall grayscale gradient value of each pixel; Based on the comprehensive gray-level gradient value, the standard deviation of the comprehensive gray-level gradient values ​​of all pixels in the neighborhood is determined, and the standard deviation is used as the gray-level gradient stability calculation result of the corresponding candidate feature point.

[0009] In one possible implementation, the step of performing interference region masking filtering on the registered first comparison video stream data and the registered second comparison video stream data to obtain masked registered first comparison video stream data and masked registered second comparison video stream data includes: Acquire multi-camera calibration parameters and GIS terrain data; Based on each frame of the registered first comparison video stream data and the registered second comparison video stream data, and combined with the multi-camera calibration parameters and GIS terrain data, a static interference area mask is constructed. Inter-frame difference calculation is performed on multiple consecutive frames of images at the same spatial location in the first and second registration comparison video stream data to obtain an inter-frame difference image; Based on the inter-frame difference image, the instantaneous dynamic interference region and the periodic dynamic interference region are determined; Based on the static interference region mask, the instantaneous dynamic interference region, and the periodic dynamic interference region, a comprehensive interference region mask is constructed. The frames of the first and second comparison video streams are filtered according to the integrated interference region mask to obtain the first and second comparison video streams after mask filtering.

[0010] In one possible implementation, the step of constructing a static interference area mask based on each frame of the registered first comparison video stream data and the registered second comparison video stream data, combined with the multi-camera calibration parameters and GIS terrain data, includes: Extract intrinsic and extrinsic parameters from the multi-view camera calibration parameters; Based on the intrinsic and extrinsic parameters, the two-dimensional pixel coordinates of each frame of the registered first comparison video stream data and the registered second comparison video stream data are transformed into three-dimensional world coordinates; Extract real GIS terrain data that matches the monitoring area from the GIS terrain data, wherein the real GIS terrain data includes terrain type data, altitude data, and boundary coordinates of the monitoring target mountain; The converted 3D world coordinates are spatially overlaid with the real GIS terrain data, and the non-monitored target mountain areas are removed from the 3D world coordinates to obtain the 3D world coordinates of the static interference area. The three-dimensional world coordinates of the static interference region are converted inversely into two-dimensional pixel coordinates of each frame of the first and second comparison video stream data, and the regions corresponding to the two-dimensional pixel coordinates are marked to obtain the static interference region mask.

[0011] In one possible implementation, the step of performing inter-frame difference calculation on multiple consecutive frames of images at the same spatial location in the registered first comparison video stream data and the registered second comparison video stream data to obtain an inter-frame difference image includes: From the first and second registration comparison video stream data, N consecutive frames of images within the same monitoring period are extracted respectively, wherein the sequence of the N consecutive frames of images covers the same spatial monitoring area; The pixel grayscale values ​​of each frame in the captured N consecutive frames are uniformly mapped to a preset range to obtain the preprocessed N consecutive frames. The first frame in the preprocessed N consecutive frames of images is taken as the reference frame, and the pixel grayscale difference between the i-th frame and the reference frame is calculated sequentially in time order. The pixel grayscale difference of all pixels is binarized to distinguish the pixels into dynamically changing pixels and static pixels, thus obtaining a single-frame difference image; Perform an inter-frame logical AND operation on N to 1 consecutive single-frame difference images to obtain an inter-frame difference image.

[0012] This example provides an image recognition-based debris flow disaster early warning method. First, two sets of comparative video stream data from multiple cameras at different time periods are acquired. Mountain feature registration is performed on the two sets of comparative video stream data to ensure spatial coordinate consistency. Then, invalid interference information is removed through interference area masking. Next, a set of differential pixels associated with landslides is determined based on the filtered video stream data. Finally, a three-dimensional landslide monitoring model is constructed by combining this set with the registered video stream data, thereby determining the target monitoring area of ​​the detection equipment and performing debris flow disaster monitoring and early warning in the target monitoring area. This method can eliminate spatial positional deviations in video stream data across different time periods through mountain feature registration, and effectively eliminate various interference factors in the natural environment through interference area masking. It can accurately identify landslide characteristics, significantly improve the early warning accuracy of debris flow disasters, reduce false alarms, and lower the safety hazards caused by false alarms.

[0013] A second aspect of this application provides a debris flow disaster early warning device based on image recognition, the device comprising: The acquisition unit is used to acquire the first comparison video stream data of the multi-camera in a first time period and the second comparison video stream data of the multi-camera in a second time period. The first processing unit is used to perform mountain feature registration on the first comparison video stream data and the second comparison video stream data to obtain registered first comparison video stream data and registered second comparison video stream data. The second processing unit is used to perform interference region masking filtering on the registered first comparison video stream data and the registered second comparison video stream data to obtain masked registered first comparison video stream data and masked registered second comparison video stream data. The third processing unit is used to determine the set of mountain sliding related difference pixels based on the masked registration first comparison video stream data and the masked registration second comparison video stream data. The early warning unit is used to construct a three-dimensional landslide monitoring model based on the set of differential pixels associated with landslides, the first comparison video stream data, and the second comparison video stream data, determine the target monitoring area of ​​the detection equipment, control the detection equipment to monitor the target monitoring area for debris flow, and issue a debris flow early warning signal.

[0014] In one possible implementation, in the aspect of performing mountain feature registration on the first and second comparison video stream data to obtain registered first comparison video stream data and registered second comparison video stream data, the first processing unit is configured to: Extract stable feature points of the mountain from each frame of the first and second comparison video stream data; The stable feature points of the mountains in each frame of the first comparison video stream data are matched to determine the correspondence between the feature points of the first comparison video stream data. The spatial position of each frame of the first comparison video stream data is corrected according to the correspondence between the feature points of the first frame to obtain the first comparison video stream data with inter-frame registration. The stable feature points of the mountains in each frame of the second comparison video stream data are matched to determine the correspondence between the feature points of the frames in the second comparison video stream data. The spatial position of each frame of the second comparison video stream data is corrected according to the correspondence between the feature points of the frames to obtain the second comparison video stream data with inter-frame registration. Select any frame image from the first comparison video stream data of the inter-frame registration as the first reference frame, and perform matching calculations between the stable feature points of the mountains in each frame image of the second comparison video stream data of the inter-frame registration and the stable feature points of the mountains in the first reference frame to obtain the spatial transformation matrix. Based on the spatial transformation matrix, coordinate transformation is performed on each frame of the second comparison video stream data of the inter-frame registration to obtain registered second comparison video stream data with spatial coordinates consistent with the first comparison video stream data. The first comparison video stream data of the inter-frame registration is used as the first comparison video stream data for registration.

[0015] In one possible implementation, regarding the extraction of mountain stability feature points from each frame of the first and second comparison video stream data, the first processing unit is configured to: Preprocess each frame of the first and second comparison video stream data, performing grayscale conversion and Gaussian filtering noise reduction sequentially to obtain a denoised grayscale image. Feature point detection is performed on the denoised grayscale image to obtain an initial feature point set; Obtain the initial mountain calibration range of the multi-view camera and the corresponding GIS terrain data; Based on the initial mountain calibration range and GIS terrain data, the sky area, artificial building area and dense vegetation area are removed, and the static mountain mask corresponding to the pure mountain area in the denoised grayscale image is determined. The initial feature point set is filtered based on the static mountain mask to remove non-mountain feature points outside the static mountain mask, resulting in a candidate feature point set. The gray-level gradient stability of each feature point in the candidate feature point set is calculated to obtain the gray-level stability calculation result of the feature point; Based on the grayscale stability calculation results of the feature points, the stable feature points of the mountain that meet the preset conditions in the candidate feature point set are determined, and the stable feature points of the mountain are obtained.

[0016] In one possible implementation, in the aspect of calculating the gray-level gradient stability of each feature point in the candidate feature point set to obtain the feature point gray-level stability calculation result, the first processing unit is configured to: For each feature point in the candidate feature point set, a pre-defined neighborhood region is selected in the denoised grayscale image, centered on the candidate feature point. Calculate the grayscale gradient value of each pixel in the horizontal and vertical directions within the neighborhood, and determine the horizontal gradient matrix and vertical gradient matrix of all pixels in the neighborhood. Based on the horizontal and vertical gradient matrices, determine the overall grayscale gradient value of each pixel; Based on the comprehensive gray-level gradient value, the standard deviation of the comprehensive gray-level gradient values ​​of all pixels in the neighborhood is determined, and the standard deviation is used as the gray-level gradient stability calculation result of the corresponding candidate feature point.

[0017] In one possible implementation, regarding the step of performing interference region masking filtering on the registered first comparison video stream data and the registered second comparison video stream data to obtain masked registered first comparison video stream data and masked registered second comparison video stream data, the second processing unit is configured to: Acquire multi-camera calibration parameters and GIS terrain data; Based on each frame of the registered first comparison video stream data and the registered second comparison video stream data, and combined with the multi-camera calibration parameters and GIS terrain data, a static interference area mask is constructed. Inter-frame difference calculation is performed on multiple consecutive frames of images at the same spatial location in the first and second registration comparison video stream data to obtain an inter-frame difference image; Based on the inter-frame difference image, the instantaneous dynamic interference region and the periodic dynamic interference region are determined; Based on the static interference region mask, the instantaneous dynamic interference region, and the periodic dynamic interference region, a comprehensive interference region mask is constructed. The frames of the first and second comparison video streams are filtered according to the integrated interference region mask to obtain the first and second comparison video streams after mask filtering.

[0018] In one possible implementation, in the aspect of constructing a static interference area mask based on each frame of the registered first comparison video stream data and the registered second comparison video stream data, combined with the multi-camera calibration parameters and GIS terrain data, the second processing unit is configured to: Extract intrinsic and extrinsic parameters from the multi-view camera calibration parameters; Based on the intrinsic and extrinsic parameters, the two-dimensional pixel coordinates of each frame of the registered first comparison video stream data and the registered second comparison video stream data are transformed into three-dimensional world coordinates; Extract real GIS terrain data that matches the monitoring area from the GIS terrain data, wherein the real GIS terrain data includes terrain type data, altitude data, and boundary coordinates of the monitoring target mountain; The converted 3D world coordinates are spatially overlaid with the real GIS terrain data, and the non-monitored target mountain areas are removed from the 3D world coordinates to obtain the 3D world coordinates of the static interference area. The three-dimensional world coordinates of the static interference region are converted inversely into two-dimensional pixel coordinates of each frame of the first and second comparison video stream data, and the regions corresponding to the two-dimensional pixel coordinates are marked to obtain the static interference region mask.

[0019] In one possible implementation, in the aspect of performing inter-frame difference calculation on multiple consecutive frames of images at the same spatial location in the registered first comparison video stream data and the registered second comparison video stream data to obtain an inter-frame difference image, the second processing unit is configured to: From the first and second registration comparison video stream data, N consecutive frames of images within the same monitoring period are extracted respectively, wherein the sequence of the N consecutive frames of images covers the same spatial monitoring area; The pixel grayscale values ​​of each frame in the captured N consecutive frames are uniformly mapped to a preset range to obtain the preprocessed N consecutive frames. The first frame in the preprocessed N consecutive frames of images is taken as the reference frame, and the pixel grayscale difference between the i-th frame and the reference frame is calculated sequentially in time order. The pixel grayscale difference of all pixels is binarized to distinguish the pixels into dynamically changing pixels and static pixels, thus obtaining a single-frame difference image; Perform an inter-frame logical AND operation on N to 1 consecutive single-frame difference images to obtain an inter-frame difference image.

[0020] A third aspect of this application provides a terminal including a processor, an input device, an output device, and a memory, wherein the processor, input device, output device, and memory are interconnected, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to execute the step instructions as described in the image recognition-based debris flow disaster early warning method in the first aspect of this application.

[0021] A fourth aspect of this application provides a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in the image recognition-based debris flow disaster early warning method of the first aspect of this application.

[0022] A fifth aspect of this application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the image recognition-based debris flow disaster early warning method of the first aspect of this application. The computer program product may be a software installation package. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This application provides a schematic diagram of an image recognition-based debris flow disaster early warning method. Figure 2 This application provides a schematic diagram of the overall structure of a debris flow disaster early warning device based on image recognition. Figure 3 This application provides a schematic diagram of the structure of a terminal. Figure label: Acquisition Unit-1, First Processing Unit-2, Second Processing Unit-3, Third Processing Unit-4, Early Warning Unit-5. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0026] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0027] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0028] To better understand the image recognition-based debris flow disaster early warning method provided in this application embodiment, the following is a brief introduction to the scenarios in which this image recognition-based debris flow disaster early warning method is applied. In southwestern my country, mountainous areas cover a large area, with steep slopes and uneven vegetation cover in some areas. During the rainy season in summer, debris flows are highly likely to occur, posing a direct threat to houses, farmland, and access roads in villages at the foot of the mountains. To ensure the safety of villagers, early warning systems are deployed in mountain monitoring areas. These systems collect video streams of the mountains through cameras and combine them with monitoring data from detection equipment to generate early warnings. However, in actual operation, the system may experience false alarms. During the day, the changing angle of the sun creates moving light spots on the mountain surface, which the system may mistakenly interpret as a landslide. Reflections from houses at the foot of the mountain, birds flying past the camera in the distance, and swaying bushes caused by gusts of wind can also trigger debris flow disaster early warning signals. These false alarms may cause villagers to distrust the early warning system, and may even lead to frequent deployments of disaster prevention personnel to verify whether a mudslide disaster has actually occurred, wasting a lot of manpower and resources. If a mudslide disaster actually occurs and the response is delayed due to the false alarms, it will directly threaten the lives and property of villagers and pose a safety hazard.

[0029] The image recognition-based debris flow disaster early warning method is applied to an image recognition-based debris flow disaster early warning device. Figure 1 A schematic diagram of the overall process for a debris flow disaster early warning method based on image recognition is shown. Figure 1 As shown, it includes: S1. Obtain the first comparison video stream data of the multi-camera in the first time period and the second comparison video stream data of the multi-camera in the second time period.

[0030] The multi-view camera system comprises two or more camera units positioned at different angles to capture images of the monitored mountain from multiple perspectives, enabling subsequent 3D modeling of the mountain. In this example, video stream data is collected over two time periods. The first comparison video stream contains multiple frames stored in chronological order. Assuming the multi-view camera system has three camera units, the first comparison video stream contains multiple frames from the first, second, and third camera units during the first time period. The second comparison video stream is similar. The acquisition duration for both the first and second time periods is the same, and the specific acquisition duration can be any time between 3 and 5 minutes.

[0031] S2. Perform mountain feature registration on the first comparison video stream data and the second comparison video stream data to obtain registered first comparison video stream data and registered second comparison video stream data.

[0032] First, stable feature points of the mountains are extracted from each frame of the first and second comparison video streams to provide a precise reference for subsequent registration. Next, for the first comparison video stream, the stable feature points of the mountains in each frame are matched to determine the correspondence between feature points between frames. Then, based on this correspondence, spatial position correction is performed on each frame of the first comparison video stream to eliminate positional deviations caused by slight camera shake within the same time period, resulting in the first comparison video stream data for inter-frame registration. Similarly, stable feature points of the mountains in each frame of the second comparison video stream are matched and spatial position corrected. The process involves obtaining second-frame-registered comparison video stream data; then, selecting one frame from the first-frame-registered comparison video stream data as the first reference frame, matching the stable feature points of the mountains in each frame of the second-frame-registered comparison video stream data with the stable feature points of the mountains in the first reference frame to obtain a spatial transformation matrix that enables coordinate transformation; and performing coordinate transformation on each frame of the second-frame-registered comparison video stream data based on this spatial transformation matrix to ensure that the spatial coordinates of the second-frame-registered comparison video stream data are consistent with those of the first-frame-registered comparison video stream data, thus obtaining registered second-frame-registered comparison video stream data, while simultaneously using the first-frame-registered comparison video stream data directly as registered first-frame-registered comparison video stream data.

[0033] In this example, the spatial deviation between the first and second comparison video stream data can be effectively eliminated through phased inter-frame registration and cross-time period registration. This ensures that the same pixel in the video stream data of different time periods can accurately correspond to the same physical location of the mountain, avoiding "false difference" information caused by positional misalignment during subsequent image difference comparison, and improving the accuracy of subsequent accurate identification of the difference pixels associated with mountain sliding.

[0034] S3. Perform interference region masking on the registered first comparison video stream data and the registered second comparison video stream data to obtain masked first comparison video stream data and masked second comparison video stream data.

[0035] First, the calibration parameters of the multi-view camera and GIS terrain data matching the monitoring area are acquired to provide data support for accurate identification of interference areas. Then, combining the calibration parameters of the multi-view camera, GIS terrain data, and frames from the first and second registration comparison video streams, a static interference area mask is constructed through spatial coordinate transformation and geographic information matching. This mask marks fixed, unchanging non-monitoring target areas such as sky areas and artificial building areas. Next, for consecutive frames at the same spatial location in the first and second registration comparison video streams, inter-frame difference calculations are performed to obtain inter-frame difference images. Based on the variation characteristics of pixel grayscale values ​​in the differential image, instantaneous dynamic interference regions and periodic dynamic interference regions are distinguished and determined. Then, the static interference region mask, the instantaneous dynamic interference region, and the periodic dynamic interference region are integrated to generate a comprehensive interference region mask that covers all invalid interference regions. Finally, based on this comprehensive interference region mask, each frame of the first and second registration comparison video stream data is filtered to remove interference pixel data within the mask and retain the pure mountain monitoring area pixel data outside the mask. The final results are the first and second registration comparison video stream data after mask filtering.

[0036] In this example, by performing several processing steps on the video stream data, including static masking to eliminate fixed interference, dynamic differential identification of real-time interference, and comprehensive masking and filtering, various interference factors in the natural environment, such as sky light and shadow, reflections from artificial buildings, falling rocks, and wind-induced vegetation swaying, can be eliminated. This ensures that the data used for subsequent difference analysis focuses only on the actual mountain monitoring area, effectively preventing interference signals from being misjudged as landslide features, and improving the accuracy of subsequent identification of differential pixels related to landslides from the data source.

[0037] S4. Based on the first comparison video stream data after mask filtering and the second comparison video stream data after mask filtering, determine the set of differential pixels associated with mountain sliding.

[0038] First, multi-image averaging processing is performed on the masked and filtered first and second comparison video streams. By calculating the average grayscale value of corresponding pixels in each frame of the two video streams, random interference such as electronic noise is eliminated, resulting in a first baseline average image and a second average image corresponding to the masked and filtered first and second comparison video streams, respectively. Then, the first baseline average image and the second average image are converted into first and second grayscale images, respectively, simplifying the computational workload of subsequent image difference analysis while preserving the grayscale variation characteristics of the mountain surface. Subsequently, n-layer pyramid images are constructed for the first and second grayscale images, forming a first image set and a second image set. The image size gradually decreases from the bottom layer to the top layer, capturing the grayscale changes of the mountain at different scales and avoiding missing the differences caused by local minor slippage. Then, the corresponding layers of the first image set and the second image set are subtracted pixel by pixel to obtain a set of difference images containing information on differences at each scale. Next, each image in the set of difference images is subjected to low-pass filtering and magnification, and all the processed difference images are superimposed to generate a reference image with the same resolution as the first baseline average image. Finally, a pixel grayscale threshold is set, and pixels in the reference image with a grayscale value greater than the threshold are selected. These pixels are the areas with significant grayscale differences after registration and interference filtering, thus obtaining the pixels directly related to mountain slippage. Then, the pixels directly related to mountain slippage are integrated to obtain a set of difference pixels associated with mountain slippage.

[0039] S5. Construct a three-dimensional landslide monitoring model based on the set of differential pixels associated with landslide, the first comparison video stream data, and the second comparison video stream data, determine the target monitoring area of ​​the detection equipment, and control the detection equipment to monitor the target monitoring area for debris flow and issue a debris flow early warning signal.

[0040] First, a three-dimensional landslide monitoring model is constructed using registered first and second comparison video streams. Based on the intrinsic and extrinsic parameters of the multi-view camera, the two-dimensional pixel coordinates of each frame in the registered video streams are converted into three-dimensional world coordinates. Combining the complementarity of multi-view data, the three-dimensional reconstruction of the mountain is completed, generating a first three-dimensional image corresponding to the first time period and a second three-dimensional image corresponding to the second time period. Then, depth values ​​corresponding to the set of differential pixels associated with landslide movement are extracted from the first and second three-dimensional images, forming a first depth value set (corresponding to the first three-dimensional image) and a second depth value set (corresponding to the second three-dimensional image). This is then analyzed using a formula... in, to For the first depth value set element, to For elements of the second depth value set, The landslide distance is calculated based on the number of differing pixels. The 3D image, depth value set, and landslide distance are integrated to construct a 3D landslide monitoring model that reflects the landslide status in real time. Then, the target monitoring area for the detection equipment is determined: the 3D coordinates of the abscissa, ordinate, and depth value of the differing pixels associated with the landslide are extracted from the 3D landslide monitoring model. Combined with the installation position parameters of the multi-view camera, the relative position between the landslide area and the multi-view camera is determined. Next, the preset relative positional relationship between the detection equipment and the multi-view camera is obtained. A coordinate transformation algorithm is used to transform the coordinates of the landslide area from the multi-view camera coordinate system to the detection equipment coordinate system. The transformed area is the target monitoring area that the detection equipment needs to focus on monitoring. Finally, the detection equipment is controlled to monitor the target area for debris flow and issue early warning signals. Specifically, the coordinates of the target area are sent to the detection equipment, which is then controlled to focus on this key area rather than monitoring the entire area. Real-time dynamic data of the mountain within the target area is collected, and the collected dynamic data is combined with the previously calculated mountain sliding distance for comprehensive judgment. If the mountain sliding distance exceeds the preset distance threshold, or if the detection equipment detects that the speed of rock movement exceeds the preset speed threshold, it is determined that there is a risk of debris flow outbreak, and a debris flow early warning signal is immediately issued to the surrounding area through the early warning system.

[0041] This example provides an image recognition-based debris flow disaster early warning method. First, two sets of comparative video stream data from multiple cameras at different time periods are acquired. Mountain feature registration is performed on the two sets of comparative video stream data to ensure spatial coordinate consistency. Then, invalid interference information is removed through interference area masking. Next, a set of differential pixels associated with landslides is determined based on the filtered video stream data. Finally, a three-dimensional landslide monitoring model is constructed by combining this set with the registered video stream data, thereby determining the target monitoring area of ​​the detection equipment and performing debris flow disaster monitoring and early warning in the target monitoring area. This method can eliminate spatial positional deviations in video stream data across different time periods through mountain feature registration, and effectively eliminate various interference factors in the natural environment through interference area masking. It can accurately identify landslide characteristics, significantly improve the early warning accuracy of debris flow disasters, reduce false alarms, and lower the safety hazards caused by false alarms.

[0042] In one possible implementation, the step of performing mountain feature registration on the first and second comparison video stream data to obtain registered first and second comparison video stream data includes: S201. Extract stable feature points of the mountain from each frame of the first and second comparison video stream data.

[0043] Specifically, step S201 includes the following steps: S2011. Preprocess each frame of the first and second comparison video stream data by performing grayscale conversion and Gaussian filtering noise reduction in sequence to obtain a denoised grayscale image.

[0044] First, grayscale conversion is performed on each frame of color image in the first and second comparison video streams. The RGB three-channel pixel values ​​can be converted into single-channel grayscale values ​​using a weighted average method, thereby removing the interference of color information on subsequent feature extraction and reducing the amount of data processing. Then, Gaussian filtering is performed on the grayscale image for noise reduction. A 5x5 Gaussian convolution kernel can be used, where the standard deviation δ of the Gaussian function is set to 0.8-1.2. The image is smoothed through convolution operation, so that the grayscale value of each pixel is replaced by the weighted average of the grayscale values ​​of its neighboring pixels, thereby suppressing high-frequency noise in the image. Finally, a denoised grayscale image with intact edges and low noise is obtained.

[0045] S2012. Perform feature point detection on the denoised grayscale image to obtain an initial feature point set.

[0046] Specifically, a scale-invariant feature transformation algorithm or an accelerated robust feature transformation algorithm is used to detect feature points in the denoised grayscale image. First, a Gaussian difference pyramid is constructed in a multi-scale space. Potential extreme points are found by comparing pixel value changes at different scales. Then, these extreme points are located, and low-contrast points and points with weak edge responses are eliminated, retaining candidate points with stable grayscale changes. Next, the gradient direction of each candidate point is calculated. Specifically, the gradient direction of each candidate point is calculated by first calculating the gradient components, using the following formula: in, The gradient is in the horizontal direction. The gradient is in the vertical direction. For pixels The grayscale value.

[0047] Furthermore, the gradient magnitude is calculated using the following formula: in, This represents the gradient magnitude of a pixel, a non-negative value. A larger value indicates a more drastic change in grayscale. The gradient is in the horizontal direction. This represents the gradient in the vertical direction.

[0048] Furthermore, the gradient direction is calculated using the following formula: in, For pixels gradient direction, For the arctangent function in the fourth quadrant, according to and The sign determines the specific quadrant, ensuring that the direction angle is unique.

[0049] Furthermore, based on the neighborhood regions surrounding the candidate feature points, a gradient direction histogram is calculated for all pixels. For example, the 0°~360° range is divided into 8 bins, and then the gradient direction of each pixel is accumulated according to its bin, with the accumulated weight being the gradient magnitude of that pixel. The direction corresponding to the peak value in the histogram is the principal direction of the candidate feature point, thus obtaining the principal gradient direction of each candidate point.

[0050] Then, descriptors for the feature points are constructed based on the main direction, so that the feature points have rotation invariance and scale invariance. Finally, all the selected feature points, including coordinates, scale, direction and descriptor information, are integrated into the initial feature point set.

[0051] In this example, a feature point detection algorithm with scale and rotation invariance is used to ensure that key feature points in the image can be stably extracted under different shooting conditions, providing sufficient initial samples for the subsequent screening of stable feature points of the mountain.

[0052] S2013. Obtain the initial mountain calibration range of the multi-camera system and the corresponding GIS terrain data.

[0053] The initial mountain calibration range of the multi-camera system is a monitoring area boundary pre-set during the equipment installation phase. It is calculated by combining the physical installation angle of the camera with the lens focal length and is represented by a rectangular area in the image coordinate system, including the pixel coordinates of the upper left and lower right corners. This range covers the main mountain area that needs to be monitored, excluding non-monitoring areas at the edge of the lens's field of view. The GIS topographic data corresponding to the initial mountain calibration range is collected in advance or retrieved from a geographic information database, including topographic vector data matching the calibration range, altitude data, terrain type classification data, and the precise boundary coordinates of the monitored target mountain.

[0054] This example, by acquiring the initial calibration range and GIS terrain data, provides a geospatial reference for subsequently distinguishing between mountainous areas and non-mountainous interference areas, ensuring the spatial accuracy of feature point selection.

[0055] S2014. Based on the initial mountain calibration range and GIS terrain data, remove the sky area, artificial building area and dense vegetation area, and determine the static mountain mask corresponding to the pure mountain area in the denoised grayscale image.

[0056] First, the image coordinates of the initial mountain calibration range are spatially mapped to the mountain boundary coordinates in the GIS terrain data: using the installation position and attitude angular parameters of the multi-camera, the image pixel coordinates are converted into geographic coordinates, and then matched with the mountain boundary in the GIS data to determine the approximate spatial range of the mountain in the image. Then, interference areas are removed based on grayscale and texture features: the sky area can be filtered by setting a grayscale threshold, which usually shows high grayscale values ​​and gradual changes; artificial building areas can be identified by edge detection and shape analysis, which have regular geometric shapes and continuous edges; and dense vegetation areas can be identified by combining vegetation type labels in the GIS terrain data, which show dense texture features and specific spectral responses. Finally, the remaining areas after removing interference areas are marked as pure mountain areas, and a corresponding static mountain mask is generated—this mask is a binary matrix with the same size as the denoised grayscale image, where the pixel position of the pure mountain area is marked as 1, and the non-mountain interference areas (sky, buildings, dense vegetation) are marked as 0.

[0057] This example demonstrates how a combination of spatial mapping and feature analysis can accurately delineate a purely mountainous region, providing spatial constraints for the targeted selection of feature points in the future.

[0058] S2015. Based on the static mountain mask, the initial feature point set is filtered to remove non-mountain feature points other than the static mountain mask, thus obtaining a candidate feature point set.

[0059] The process involves iterating through each feature point in the initial feature point set and extracting its pixel coordinates (x, y) in the denoised grayscale image. Then, the corresponding coordinate (x, y) value in the static mountain mask is retrieved. If the value is 1 (indicating a pure mountain area), the feature point is retained; if the value is 0 (indicating a non-mountain interference area), it is removed from the initial feature point set. After this filtering, the remaining feature points are all located within pure mountain areas. These feature points are then re-integrated to form a candidate feature point set. This step effectively excludes feature points from non-mountain areas such as the sky, man-made structures, and densely vegetated areas through the spatial constraints of the static mountain mask. This ensures that subsequent feature point stability analysis focuses only on the true mountain areas, reducing unnecessary computation and avoiding interference from non-mountain feature points on the accuracy of subsequent mountain registration.

[0060] S2016. Calculate the gray-level gradient stability of each feature point in the candidate feature point set to obtain the feature point gray-level stability calculation result.

[0061] Specifically, step S2016 includes: S20161. For each feature point in the candidate feature point set, select a pre-sized neighborhood region in the denoised grayscale image with the candidate feature point as the center.

[0062] First, the preset size of the neighborhood region is determined. Based on the texture density of the mountain surface, a square neighborhood of 3x3 pixels or 5x5 pixels is selected. When the mountain texture is generally dense, a 3x3 pixel neighborhood is used; when the mountain texture is generally sparse, a 5x5 pixel neighborhood is selected. Then, the pixel coordinates of the candidate feature points are used... Centered on a point, determine the pixel coordinate range of the neighborhood region. Specifically, for a 3×3 neighborhood, the pixel coordinate range is... When the neighborhood is 5×5, the range is If the neighborhood exceeds the boundary of the denoised grayscale image, edge completion is used to ensure the integrity of the neighborhood region.

[0063] S20162. Calculate the grayscale gradient value of each pixel in the horizontal and vertical directions within the neighborhood, and determine the horizontal gradient matrix and vertical gradient matrix of all pixels in the neighborhood.

[0064] The Sobel operator can be used to calculate the horizontal gradient of each pixel in the neighborhood. and vertical gradient For any pixel in the neighborhood Horizontal gradient and vertical gradient The formula for calculating the grayscale change rate is as follows: in, This represents the grayscale value of a pixel within its neighborhood in the denoised grayscale image. The horizontal gradient of a pixel. is the vertical gradient of the pixel.

[0065] Furthermore, based on the spatial location of pixels within their neighborhood, all pixels are... Arranged in row and column order, forming a horizontal gradient matrix with the same size as the neighborhood. Similarly, all pixels Arranged to form a vertical gradient matrix .

[0066] S20163. Determine the comprehensive grayscale gradient value of each pixel based on the horizontal gradient matrix and the vertical gradient matrix.

[0067] Among them, the Euclidean distance method can be used to calculate the horizontal gradient matrix. and vertical gradient matrix The gradient values ​​at corresponding positions are combined to obtain the comprehensive grayscale gradient value of each pixel. The calculation formula is: .

[0068] S20164. Based on the comprehensive gray-level gradient value, determine the standard deviation of the comprehensive gray-level gradient values ​​of all pixels in the neighborhood region, and use the standard deviation as the gray-level gradient stability calculation result of the corresponding candidate feature point.

[0069] First, the mean of the comprehensive gray-level gradient within the neighborhood is calculated. It can count the total number of pixels N in the neighborhood area, and then calculate the combined grayscale gradient value of all pixels. The arithmetic mean is given by the formula: ,in, Let N be the mean of the combined gray-level gradients within the neighborhood, and N be the total number of pixels in the neighborhood. Let be the combined gray-level gradient value of the i-th pixel in the neighborhood, i = 1, 2, ..., N.

[0070] Furthermore, based on the mean of the comprehensive gray-scale gradient Calculate the deviation of the combined grayscale gradient value of all pixels from the mean value. The calculation formula is as follows: ,in This represents the standard deviation of the overall gray-level gradient values ​​within the neighborhood, used to reflect the uniformity of gray-level changes within the neighborhood. The deviation of the overall grayscale gradient value of the i-th pixel from the mean is then calculated, and the standard deviation is... The standard deviation is the result of the gray-level gradient stability calculation for the corresponding candidate feature points. The smaller the value, the more uniform the grayscale variation intensity of each pixel in the neighborhood, and the more stable the grayscale distribution around the feature point. For example, the grayscale variation around the corner of a rock is gentle and consistent; conversely, a larger value indicates a more uniform grayscale variation. The larger the value, the greater the difference in grayscale within the neighborhood. The feature point may be located in an unstable region, such as the edge of vegetation, where some pixels change drastically and others change gradually.

[0071] S2027. Based on the grayscale stability calculation results of the feature points, determine the stable feature points of the mountain that meet the preset conditions in the candidate feature point set, and obtain the stable feature points of the mountain.

[0072] The grayscale gradient stability threshold can be set and calibrated based on the mountain type and historical environmental data of the monitoring area. The calibration method is as follows: Historical video stream data of the monitoring area with no landslides in the past 12 months are collected, covering different weather conditions such as sunny, cloudy, and windy days. Then, the historical data is classified into rocky mountains and soil mountains, and candidate feature points are extracted for pure mountain areas. The standard deviation of gray-level gradient stability for each candidate point is calculated. This forms two types of mountains. Distribution histogram; then take the points corresponding to 95% of the candidate points in the histogram. The maximum value is used as the threshold for this type of mountain. For example, rocky mountains The distribution is concentrated between 2 and 7, so we take 7, which corresponds to the 95th percentile, as the quantile. Soil and mountain The distribution is concentrated between 3 and 9, so we take 9, which corresponds to the 95th percentile, as the quantile. Additionally, it can be recalibrated quarterly based on newly collected no-slip data. Adapt to seasonal changes in the texture of the mountain surface.

[0073] Specifically, if the monitoring target is a rocky mountain, the threshold is... It can be set to 5-8, if the monitoring target is the soil / mountain threshold. The threshold can be set to 8-10. After setting the gray-level gradient stability threshold, calculate the corresponding standard deviation of gray-level gradient stability for each feature point in the candidate feature point set, and then compare the standard deviation of gray-level gradient stability with the threshold. For comparison, select a value where the standard deviation of grayscale gradient stability is less than the threshold. The characteristic points form a set of stable characteristic points of the mountain.

[0074] S202. Match the stable feature points of the mountains in each frame of the first comparison video stream data to determine the correspondence between the feature points of the first comparison video stream data, and perform spatial position correction on each frame of the first comparison video stream data according to the correspondence between the feature points of the first frame to obtain the first comparison video stream data with inter-frame registration.

[0075] First, inter-frame feature point matching is performed on the first comparison video stream. The frame images arranged in chronological order in the first comparison video stream can be used as the object. Stable feature points corresponding to each frame image are extracted from the set of stable feature points of the mountain. Then, the FLANN matcher can be used to perform inter-frame feature point matching. Adjacent frames are grouped together. The feature point descriptors of the previous frame are compared with the feature point descriptors of the next frame. Matching pairs with similarity higher than the preset matching threshold are retained to form the inter-frame feature point correspondence between each pair of adjacent frames. Then, spatial position correction is performed based on the correspondence.

[0076] Furthermore, for the correspondence of feature points in each group of adjacent frames, the positional offset between the two frames is calculated. This can be done by using the coordinate difference statistics of the matching pairs. If the offset exceeds the preset jitter threshold, the homography matrix is ​​used to perform spatial transformation on the image of the next frame so that the stable feature points of the next frame are aligned with the corresponding feature points of the previous frame in pixel coordinates. After the correction is completed frame by frame in chronological order, the spatial positional deviation of all frames in the first comparison video stream is eliminated, and the first comparison video stream data with inter-frame registration is obtained.

[0077] S203. Match the stable feature points of the mountains in each frame of the second comparison video stream data to determine the correspondence between the feature points of the frames in the second comparison video stream data, and perform spatial position correction on each frame of the second comparison video stream data according to the correspondence between the feature points of the frames to obtain the second comparison video stream data with inter-frame registration.

[0078] The processing of the second comparison video stream data for inter-frame registration can be obtained in the same way as step S202.

[0079] S204. Select any frame image from the first comparison video stream data of the inter-frame registration as the first reference frame, and perform matching calculations on the stable feature points of the mountains in each frame image of the second comparison video stream data of the inter-frame registration with the stable feature points of the mountains in the first reference frame to obtain the spatial transformation matrix.

[0080] First, a first reference frame is selected. This can be any frame from the first comparison video stream data of the inter-frame registration, either an intermediate frame or the first frame. The stable feature points of the mountain in this frame are extracted to form a reference feature point set.

[0081] Furthermore, for each frame of the second comparison video stream data for inter-frame registration, its stable feature points are extracted and denoted as the set of feature points to be matched. Then, the FLANN matcher can be used to match the set of feature points to be matched with the reference feature point set of the first reference frame. In order to avoid false matches caused by changes in the natural environment, the RANSAC algorithm is used to remove outliers in the matching pairs to retain valid matching pairs that satisfy spatial consistency.

[0082] Furthermore, based on the coordinate information of the effective matching pairs, the least squares method can be used to solve the 3×3 spatial transformation matrix. Through matrix operations, the coordinates of any pixel in the second video stream can be converted to coordinates consistent with the first reference frame, thus achieving spatial alignment of the two time-segment video streams.

[0083] S205. Perform coordinate transformation on each frame of the second comparison video stream data of the inter-frame registration according to the spatial transformation matrix to obtain the registered second comparison video stream data with the same spatial coordinates as the first comparison video stream data.

[0084] Specifically, for each frame of the second comparison video stream data for inter-frame registration, a coordinate transformation is performed pixel-by-pixel, converting the original two-dimensional pixel coordinates of each pixel in the image. Substitute the spatial transformation matrix and obtain the new coordinates through matrix multiplication. During the conversion process, if the new coordinates are not integers, bilinear interpolation is used to supplement the pixel grayscale values. After the conversion is completed, the pixel coordinates of each frame in the second comparison video stream data of the inter-frame registration are consistent with the spatial coordinate system of the first reference frame.

[0085] S206. Use the first comparison video stream data of the inter-frame registration as the first comparison video stream data for registration.

[0086] Since the first comparison video stream data for inter-frame registration has completed the internal inter-frame spatial position correction through step S202, and its coordinate system is the reference standard for the subsequent coordinate transformation of the second video stream, no additional coordinate adjustment is required, and it is directly determined as the first comparison video stream data for registration.

[0087] In one possible implementation, the step of performing interference region masking filtering on the registered first comparison video stream data and the registered second comparison video stream data to obtain masked registered first comparison video stream data and masked registered second comparison video stream data includes: S301. Obtain calibration parameters for multi-camera systems and GIS terrain data.

[0088] First, the calibration parameters of the multi-camera system are obtained, including intrinsic and extrinsic parameters. Intrinsic parameters are the inherent parameters of the camera itself, including the focal length, pixel size, and principal point coordinates of each camera unit. Extrinsic parameters are the position and attitude parameters of the camera after installation, including the overall installation position, horizontal rotation angle, and pitch angle of the multi-camera system. Then, GIS terrain data can be collected through drone aerial photography or satellite remote sensing. GIS terrain data includes digital elevation model data, terrain type classification vector data, and the boundary coordinates of the monitored target mountain, with labels for mountains, sky, man-made structures, water bodies, and vegetation categories within the labeled area.

[0089] S302. Based on each frame of the registered first comparison video stream data and the registered second comparison video stream data, and combined with the multi-camera calibration parameters and GIS terrain data, construct a static interference area mask.

[0090] Specifically, step S302 includes the following steps: S3021. Extract intrinsic and extrinsic parameters from the calibration parameters of the multi-camera.

[0091] S3022. Based on the intrinsic and extrinsic parameters, the two-dimensional pixel coordinates of each frame of the registered first comparison video stream data and the registered second comparison video stream data are converted into three-dimensional world coordinates.

[0092] Wherein, the two-dimensional pixel coordinates of a certain pixel in the registered video stream image are: First, the two-dimensional pixel coordinates are converted into three-dimensional coordinates in the camera coordinate system. Then, by combining the extrinsic parameter rotation matrix and translation vector, the camera coordinates are transformed into three-dimensional coordinates in the world coordinate system. .

[0093] S3023. Extract real GIS terrain data matching the monitoring area from the GIS terrain data, wherein the real GIS terrain data includes terrain type data, altitude data, and boundary coordinates of the monitoring target mountain.

[0094] Specifically, based on three-dimensional world coordinates, the geographical range covered by the registered video stream is statistically analyzed, and then the terrain type vector data of the above range is cropped from the original GIS data, including mountains, sky, man-made buildings, water bodies, and vegetation, with each category corresponding to a unique attribute label.

[0095] Furthermore, the digital elevation model (DEM) data of the aforementioned geographical area is cropped, and the polygonal boundary coordinates of the target mountain are extracted from the mountain vector layer of the GIS topographic data.

[0096] S3024. Spatially overlay the converted three-dimensional world coordinates with the real GIS terrain data, and remove the non-monitoring target mountain areas from the three-dimensional world coordinates to obtain the three-dimensional world coordinates of the static interference area.

[0097] Among them, the transformed three-dimensional world coordinates can be Spatial inclusion determination is performed using the boundary coordinates of the monitored mountain. If a certain coordinate... If the coordinates fall outside the polygon enclosed by the boundary coordinates, the coordinates are determined to correspond to a non-target mountain area and marked as a static interference candidate; if they fall within the boundary, proceed to the next step of screening.

[0098] Furthermore, the elevation range within the target mountain boundary is extracted from the DEM data. If a certain coordinate within the boundary... < If it is, then it is determined to be a low-ground interference area. > If it is determined to be a sky interference area, then it is considered a sky interference area. < If so, proceed to the next screening step.

[0099] Furthermore, query the terrain type labels corresponding to the three-dimensional coordinates within the boundary and altitude range. If the label is a man-made building or a water body, it is directly determined as a static interference area; if the label is a densely vegetated area, it is directly determined as a static interference area; if the label is a rocky mountain or a soil mountain, it is directly determined as a pure mountain area and is retained without being marked.

[0100] Furthermore, the three-dimensional world coordinates of the non-target mountain area, lowland / sky area, and artificial building / water body / dense vegetation area selected in the above three steps are integrated to obtain the three-dimensional world coordinates of the static interference area.

[0101] S3025. Convert the three-dimensional world coordinates of the static interference region into two-dimensional pixel coordinates of each frame of the first and second comparison video stream data, mark the region corresponding to the two-dimensional pixel coordinates, and obtain the static interference region mask.

[0102] Among these, the three-dimensional world coordinates of the static disturbance region can be determined based on the rotation matrix and translation vector of the extrinsic parameters. Convert to camera coordinates Then, using the intrinsic parameter matrix K, the camera coordinates are... Convert to 2D pixel coordinates of the image Then, a binary matrix with the exact same size as the registered video stream image can be constructed as a mask for the static interference region. This mask is used for the converted static interference pixel coordinates. In the mask, the corresponding position is marked as 1, indicating that the pixel belongs to the static interference area; the other unmarked pixel positions are marked as 0, indicating that the pixel belongs to the pure mountain monitoring area.

[0103] In this example, the final generated static interference region mask can be directly used for static interference filtering in subsequent registered video streams. Furthermore, through inverse coordinate transformation, the mapping from real geographic interference regions to image pixel interference regions is completed, enabling the removal of static interference to have an executable image-level template.

[0104] S303. Perform inter-frame difference calculation on multiple consecutive frames of images at the same spatial position in the registered first comparison video stream data and the registered second comparison video stream data to obtain inter-frame difference images.

[0105] Specifically, step S303 includes the following steps: S3031. From the first and second registration comparison video stream data, extract N consecutive frames of images within the same monitoring period, wherein the sequence of the N consecutive frames of images covers the same spatial monitoring area.

[0106] S3032. Map the pixel grayscale values ​​of each frame in the captured N consecutive frames to a preset range to obtain the preprocessed N consecutive frames.

[0107] Since the conventional range of image grayscale values ​​is considered, the preset interval can be uniformly set to 0-255. Then, a linear mapping can be performed on the grayscale value of each pixel in each frame of the image, using the following formula: ,in, pixels in a frame image The original grayscale value, This represents the minimum pixel grayscale value among all pixels in this frame of the image. It is the maximum pixel grayscale value of all pixels in this frame of the image. These are the normalized grayscale values ​​after mapping. If a frame of an image... = If the frame is not found, the frame is directly removed and an adjacent frame is added from the registered video stream.

[0108] S3033. Take the first frame of the preprocessed N consecutive frames as the reference frame, and calculate the pixel grayscale difference between the i-th frame and the reference frame in chronological order.

[0109] Here, the first frame of the N consecutive preprocessed images is taken as the reference frame, denoted as . Frames 2 to N (denoted as N) are processed in chronological order. i) Perform inter-frame grayscale difference calculation frame by frame, for each frame Each pixel in Calculate the pixel's relationship with the reference frame. same coordinate The absolute value of the pixel grayscale difference is taken to eliminate pixels with a grayscale value higher than that of the reference frame. or The directional difference in grayscale above the baseline frame is expressed by the formula: ,in, The normalized grayscale value of the pixel in the reference frame. Let be the normalized grayscale value of the pixel in the i-th frame. For the i-th frame and the reference frame in pixels After calculating the grayscale difference at each position, N-1 inter-frame grayscale difference images are obtained. The size of each difference image is the same as the original frame image, and the pixel value is the grayscale difference at the corresponding position. .

[0110] S3034. Binarize the pixel gray - level differences of all pixels to distinguish the pixels into dynamically changing pixels and static pixels, and obtain a single - frame difference image.

[0111] Among them, by binarizing the inter - frame gray - level difference image, the continuous gray - level differences are converted into black - and - white dynamic / static markers. First, a difference threshold T can be set. The threshold T needs to be dynamically adjusted according to the noise level of the monitoring environment. The conventional value range is 15 - 30. If the noise in the monitoring area is strong, T takes 25 - 30. If the noise is weak, T takes 15 - 20. Then, binary judgment is performed on each pixel of each difference image: If >T, it is determined that the pixel is a dynamically changing pixel, and its gray - level value is set to 255 (white), indicating that there is a significant dynamic change at this position. If <T, it is determined that the pixel is a static pixel, and its gray - level value is set to 0 (black), indicating that there is no obvious dynamic change at this position.

[0112] Among them, after each difference image is binarized, a single - frame difference image is obtained. The white area in the image intuitively reflects the dynamic change area compared with the reference frame, and the black area is the static area.

[0113] S3035. Perform an inter - frame logical AND operation on N to 1 consecutive single - frame difference images to obtain an inter - frame difference image.

[0114] Among them, for all N - 1 single - frame difference images, a per - pixel logical AND operation is performed on the same pixel coordinates. Only when the pixel is 255 in all N - 1 single - frame difference images, the final result is set to 255. If the pixel is 0 in any one of the N - 1 single - frame difference images, the final result is set to 0. For example, if a certain pixel is 255 in the single - frame difference image of, indicating dynamic, but is 0 in the single - frame difference image of, indicating static, it means that this dynamic only appears in and may be a bird flying across the lens. After the logical AND operation, the final value of this pixel is 0 and is thus excluded. If a certain pixel is 255 in all N - 1 single - frame difference images, it means that this dynamic persists, and 255 is finally retained. After the operation, an inter - frame difference image is obtained. The white area in the image is the continuously stable dynamic interference area, and the black area is the static area or accidental dynamic area.

[0115] S304. Determine the instantaneous dynamic interference area and the periodic dynamic interference area according to the inter - frame difference image.

[0116] First, the core thresholds for dynamic interference determination are set, including the dynamic gray-scale difference threshold T, the instantaneous interference time threshold t1, and the periodic interference period threshold t2. Then, dynamic interference is classified for inter-frame difference images.

[0117] Furthermore, if the pixel grayscale difference in a certain region of the differential image continues to exceed T, but the duration is less than t1, and there is no recurring pattern of change in that region, it is determined to be an instantaneous dynamic interference region, such as falling rocks or birds flying across the lens. The characteristics of this type of interference are short-term suddenness and no periodicity, and it can be eliminated in one go.

[0118] Furthermore, if the pixel grayscale difference in a certain area of ​​the differential image shows repeated changes exceeding T to below T, and the change period is less than t2, it is determined to be a periodic dynamic interference area, such as wind-induced swaying of bushes or fluttering of flags. The characteristics of this type of interference are long-term repetition and fixed period, which require continuous filtering.

[0119] Finally, the pixel coordinates of the two types of dynamic interference regions are marked to obtain the instantaneous dynamic interference mask region and the periodic dynamic interference mask region.

[0120] S305. Construct a comprehensive interference region mask based on the static interference region mask, the instantaneous dynamic interference region, and the periodic dynamic interference region.

[0121] First, the instantaneous dynamic interference region and the periodic dynamic interference region are integrated, and the pixel coordinates of the two types of dynamic interference regions are merged to generate a joint mask for the dynamic interference region. Then, the joint mask for the dynamic interference region is integrated with the mask for the static interference region to completely cover the pixel coordinates of the static interference region and the dynamic interference region, generating the final comprehensive interference region mask. This mask is a binary matrix with the same size as the registered video stream image, where 1 indicates that the pixel belongs to static interference / dynamic interference, and 0 indicates that the pixel belongs to the pure mountain monitoring area.

[0122] S306. Filter each frame of the registered first comparison video stream data and the registered second comparison video stream data according to the comprehensive interference region mask to obtain the mask-filtered registered first comparison video stream data and the mask-filtered registered second comparison video stream data.

[0123] Specifically, for each frame of the registered first comparison video stream data and the registered second comparison video stream data, the marker value of the pixel in the comprehensive interference area mask is first queried pixel by pixel. If the marker value is 1 (interference area), the gray value of the pixel is set to 0 (or the data record of the pixel is directly removed); if the marker value is 0 (pure mountain monitoring area), the original gray value of the pixel is retained.

[0124] Furthermore, after the filtering process is completed, the first registered comparison video stream data is converted into the first registered comparison video stream data after mask filtering, and the second registered comparison video stream data is converted into the second registered comparison video stream data after mask filtering.

[0125] This example uses a pixel-by-pixel mask matching filtering method to ensure that the identification of differential pixels related to landslides in step S4 is based solely on valid landslide data, completely avoiding the impact of interference signals on the differential analysis and improving the accuracy of subsequent early warning analysis from the data source.

[0126] For those consistent with the above, please refer to Figure 2 , Figure 2 This application provides a schematic diagram of the structure of a debris flow disaster early warning device based on image recognition. For example... Figure 2 As shown, the device includes: Acquisition unit 1 is used to acquire the first comparison video stream data of the multi-camera in the first time period and the second comparison video stream data of the multi-camera in the second time period; The first processing unit 2 is used to perform mountain feature registration on the first comparison video stream data and the second comparison video stream data to obtain registered first comparison video stream data and registered second comparison video stream data. The second processing unit 3 is used to perform interference region masking filtering on the registered first comparison video stream data and the registered second comparison video stream data to obtain masked registered first comparison video stream data and masked registered second comparison video stream data. The third processing unit 4 is used to determine the set of mountain sliding related difference pixels based on the masked registration first comparison video stream data and the masked registration second comparison video stream data. The early warning unit 5 is used to construct a three-dimensional landslide monitoring model based on the set of differential pixels associated with landslide, the first comparison video stream data, and the second comparison video stream data, determine the target monitoring area of ​​the detection equipment, control the detection equipment to monitor the target monitoring area for debris flow, and issue a debris flow early warning signal.

[0127] In one possible implementation, in the aspect of performing mountain feature registration on the first and second comparison video stream data to obtain registered first comparison video stream data and registered second comparison video stream data, the first processing unit 2 is configured to: Extract stable feature points of the mountain from each frame of the first and second comparison video stream data; The stable feature points of the mountains in each frame of the first comparison video stream data are matched to determine the correspondence between the feature points of the first comparison video stream data. The spatial position of each frame of the first comparison video stream data is corrected according to the correspondence between the feature points of the first frame to obtain the first comparison video stream data with inter-frame registration. The stable feature points of the mountains in each frame of the second comparison video stream data are matched to determine the correspondence between the feature points of the frames in the second comparison video stream data. The spatial position of each frame of the second comparison video stream data is corrected according to the correspondence between the feature points of the frames to obtain the second comparison video stream data with inter-frame registration. Select any frame image from the first comparison video stream data of the inter-frame registration as the first reference frame, and perform matching calculations between the stable feature points of the mountains in each frame image of the second comparison video stream data of the inter-frame registration and the stable feature points of the mountains in the first reference frame to obtain the spatial transformation matrix. Based on the spatial transformation matrix, coordinate transformation is performed on each frame of the second comparison video stream data of the inter-frame registration to obtain registered second comparison video stream data with spatial coordinates consistent with the first comparison video stream data. The first comparison video stream data of the inter-frame registration is used as the first comparison video stream data for registration.

[0128] In one possible implementation, regarding the extraction of mountain stability feature points from each frame of the first and second comparison video stream data, the first processing unit 2 is configured to: Preprocess each frame of the first and second comparison video stream data, performing grayscale conversion and Gaussian filtering noise reduction sequentially to obtain a denoised grayscale image. Feature point detection is performed on the denoised grayscale image to obtain an initial feature point set; Obtain the initial mountain calibration range of the multi-view camera and the corresponding GIS terrain data; Based on the initial mountain calibration range and GIS terrain data, the sky area, artificial building area and dense vegetation area are removed, and the static mountain mask corresponding to the pure mountain area in the denoised grayscale image is determined. The initial feature point set is filtered based on the static mountain mask to remove non-mountain feature points outside the static mountain mask, resulting in a candidate feature point set. The gray-level gradient stability of each feature point in the candidate feature point set is calculated to obtain the gray-level stability calculation result of the feature point; Based on the grayscale stability calculation results of the feature points, the stable feature points of the mountain that meet the preset conditions in the candidate feature point set are determined, and the stable feature points of the mountain are obtained.

[0129] In one possible implementation, in the aspect of calculating the gray-level gradient stability of each feature point in the candidate feature point set to obtain the feature point gray-level stability calculation result, the first processing unit 2 is configured to: For each feature point in the candidate feature point set, a pre-defined neighborhood region is selected in the denoised grayscale image, centered on the candidate feature point. Calculate the grayscale gradient value of each pixel in the horizontal and vertical directions within the neighborhood, and determine the horizontal gradient matrix and vertical gradient matrix of all pixels in the neighborhood. Based on the horizontal and vertical gradient matrices, determine the overall grayscale gradient value of each pixel; Based on the comprehensive gray-level gradient value, the standard deviation of the comprehensive gray-level gradient values ​​of all pixels in the neighborhood is determined, and the standard deviation is used as the gray-level gradient stability calculation result of the corresponding candidate feature point.

[0130] In one possible implementation, regarding the interference region masking of the registered first comparison video stream data and the registered second comparison video stream data to obtain masked registered first comparison video stream data and masked registered second comparison video stream data, the second processing unit 3 is configured to: Acquire multi-camera calibration parameters and GIS terrain data; Based on each frame of the registered first comparison video stream data and the registered second comparison video stream data, and combined with the multi-camera calibration parameters and GIS terrain data, a static interference area mask is constructed. Inter-frame difference calculation is performed on multiple consecutive frames of images at the same spatial location in the first and second registration comparison video stream data to obtain an inter-frame difference image; Based on the inter-frame difference image, the instantaneous dynamic interference region and the periodic dynamic interference region are determined; Based on the static interference region mask, the instantaneous dynamic interference region, and the periodic dynamic interference region, a comprehensive interference region mask is constructed. The frames of the first and second comparison video streams are filtered according to the integrated interference region mask to obtain the first and second comparison video streams after mask filtering.

[0131] In one possible implementation, in the aspect of constructing a static interference area mask based on each frame of the registered first comparison video stream data and the registered second comparison video stream data, combined with the multi-camera calibration parameters and GIS terrain data, the second processing unit 3 is used to: Extract intrinsic and extrinsic parameters from the multi-view camera calibration parameters; Based on the intrinsic and extrinsic parameters, the two-dimensional pixel coordinates of each frame of the registered first comparison video stream data and the registered second comparison video stream data are transformed into three-dimensional world coordinates; Extract real GIS terrain data that matches the monitoring area from the GIS terrain data, wherein the real GIS terrain data includes terrain type data, altitude data, and boundary coordinates of the monitoring target mountain; The converted 3D world coordinates are spatially overlaid with the real GIS terrain data, and the non-monitored target mountain areas are removed from the 3D world coordinates to obtain the 3D world coordinates of the static interference area. The three-dimensional world coordinates of the static interference region are converted inversely into two-dimensional pixel coordinates of each frame of the first and second comparison video stream data, and the regions corresponding to the two-dimensional pixel coordinates are marked to obtain the static interference region mask.

[0132] In one possible implementation, in the aspect of performing inter-frame difference calculation on multiple consecutive frames of images at the same spatial location in the registered first comparison video stream data and the registered second comparison video stream data to obtain an inter-frame difference image, the second processing unit 3 is configured to: From the first and second registration comparison video stream data, N consecutive frames of images within the same monitoring period are extracted respectively, wherein the sequence of the N consecutive frames of images covers the same spatial monitoring area; The pixel grayscale values ​​of each frame in the captured N consecutive frames are uniformly mapped to a preset range to obtain the preprocessed N consecutive frames. The first frame in the preprocessed N consecutive frames of images is taken as the reference frame, and the pixel grayscale difference between the i-th frame and the reference frame is calculated sequentially in time order. The pixel grayscale difference of all pixels is binarized to distinguish the pixels into dynamically changing pixels and static pixels, thus obtaining a single-frame difference image; Perform an inter-frame logical AND operation on N to 1 consecutive single-frame difference images to obtain an inter-frame difference image.

[0133] For examples consistent with the above embodiments, please refer to... Figure 3 , Figure 3 A schematic diagram of a terminal structure provided in an embodiment of this application is shown in the figure. It includes a processor, an input device, an output device, and a memory. The processor, input device, output device, and memory are interconnected. The memory is used to store a computer program, which includes program instructions. The processor is configured to call the program instructions. The program includes instructions for performing the following steps. Acquire the first comparative video stream data of the multi-camera in the first time period and the second comparative video stream data of the multi-camera in the second time period; Mountain feature registration is performed on the first and second comparison video stream data to obtain registered first comparison video stream data and registered second comparison video stream data. Interference region masking is applied to the registered first comparison video stream data and the registered second comparison video stream data to obtain masked first comparison video stream data and masked second comparison video stream data. Based on the first comparison video stream data after mask filtering and the second comparison video stream data after mask filtering, determine the set of differential pixels associated with mountain sliding. Based on the set of differential pixels associated with landslides, the first and second comparison video streams, a three-dimensional landslide monitoring model is constructed. The target monitoring area of ​​the detection equipment is determined, and the detection equipment is controlled to monitor the target monitoring area for debris flow and issue a debris flow early warning signal.

[0134] This example provides an image recognition-based debris flow disaster early warning method. First, two sets of comparative video stream data from multiple cameras at different time periods are acquired. Mountain feature registration is performed on the two sets of comparative video stream data to ensure spatial coordinate consistency. Then, invalid interference information is removed through interference area masking. Next, a set of differential pixels associated with landslides is determined based on the filtered video stream data. Finally, a three-dimensional landslide monitoring model is constructed by combining this set with the registered video stream data, thereby determining the target monitoring area of ​​the detection equipment and performing debris flow disaster monitoring and early warning in the target monitoring area. This method can eliminate spatial positional deviations in video stream data across different time periods through mountain feature registration, and effectively eliminate various interference factors in the natural environment through interference area masking. It can accurately identify landslide characteristics, significantly improve the early warning accuracy of debris flow disasters, reduce false alarms, and lower the safety hazards caused by false alarms.

[0135] The above mainly describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, the terminal includes the corresponding hardware structure and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments provided herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0136] This application embodiment can divide the terminal into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0137] This application also provides a computer storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the image recognition-based debris flow disaster early warning methods described in the above method embodiments.

[0138] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program that causes a computer to perform some or all of the steps of any of the image recognition-based debris flow disaster early warning methods described in the above method embodiments.

[0139] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0140] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0141] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0142] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0143] Furthermore, the functional units in the various embodiments of the application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software program module.

[0144] If the integrated unit is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0145] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage device, which may include: a flash drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc.

[0146] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A debris flow disaster early warning method based on image recognition, characterized in that, include: Acquire the first comparative video stream data of the multi-camera in the first time period and the second comparative video stream data of the multi-camera in the second time period; Mountain feature registration is performed on the first and second comparison video stream data to obtain registered first comparison video stream data and registered second comparison video stream data. Interference region masking is applied to the registered first comparison video stream data and the registered second comparison video stream data to obtain masked first comparison video stream data and masked second comparison video stream data. Based on the first comparison video stream data after mask filtering and the second comparison video stream data after mask filtering, determine the set of differential pixels associated with mountain sliding. Based on the set of differential pixels associated with landslides, the first and second comparison video streams, a three-dimensional landslide monitoring model is constructed. The target monitoring area of ​​the detection equipment is determined, and the detection equipment is controlled to monitor the target monitoring area for debris flow and issue a debris flow early warning signal.

2. The debris flow disaster early warning method based on image recognition according to claim 1, characterized in that, The step of registering mountain features in the first and second comparison video stream data to obtain registered first and second comparison video stream data includes: From each frame of the first and second comparison video stream data, extract stable feature points of the mountain; The stable feature points of the mountains in each frame of the first comparison video stream data are matched to determine the correspondence between the feature points of the first comparison video stream data. The spatial position of each frame of the first comparison video stream data is corrected according to the correspondence between the feature points of the first frame to obtain the first comparison video stream data with inter-frame registration. The stable feature points of the mountains in each frame of the second comparison video stream data are matched to determine the correspondence between the feature points of the frames in the second comparison video stream data. The spatial position of each frame of the second comparison video stream data is corrected according to the correspondence between the feature points of the frames to obtain the second comparison video stream data with inter-frame registration. Select any frame image from the first comparison video stream data of the inter-frame registration as the first reference frame, and perform matching calculations between the stable feature points of the mountains in each frame image of the second comparison video stream data of the inter-frame registration and the stable feature points of the mountains in the first reference frame to obtain the spatial transformation matrix. Based on the spatial transformation matrix, coordinate transformation is performed on each frame of the second comparison video stream data of the inter-frame registration to obtain registered second comparison video stream data with spatial coordinates consistent with the first comparison video stream data. The first comparison video stream data of the inter-frame registration is used as the first comparison video stream data for registration.

3. The debris flow disaster early warning method based on image recognition according to claim 2, characterized in that, Extracting stable feature points of the mountain from each frame of the first and second comparison video stream data includes: Preprocess each frame of the first and second comparison video stream data, performing grayscale conversion and Gaussian filtering noise reduction sequentially to obtain a denoised grayscale image. Feature point detection is performed on the denoised grayscale image to obtain an initial feature point set; Obtain the initial mountain calibration range of the multi-view camera and the corresponding GIS terrain data; Based on the initial mountain calibration range and GIS terrain data, the sky area, artificial building area and dense vegetation area are removed, and the static mountain mask corresponding to the pure mountain area in the denoised grayscale image is determined. The initial feature point set is filtered based on the static mountain mask to remove non-mountain feature points outside the static mountain mask, resulting in a candidate feature point set. The gray-level gradient stability of each feature point in the candidate feature point set is calculated to obtain the gray-level stability calculation result of the feature point; Based on the grayscale stability calculation results of the feature points, the stable feature points of the mountain that meet the preset conditions in the candidate feature point set are determined, and the stable feature points of the mountain are obtained.

4. The debris flow disaster early warning method based on image recognition according to claim 3, characterized in that, The calculation of the gray-level gradient stability of each feature point in the candidate feature point set, to obtain the feature point gray-level stability calculation result, includes: For each feature point in the candidate feature point set, a pre-defined neighborhood region is selected in the denoised grayscale image, centered on the candidate feature point. Calculate the grayscale gradient value of each pixel in the horizontal and vertical directions within the neighborhood, and determine the horizontal gradient matrix and vertical gradient matrix of all pixels in the neighborhood. Based on the horizontal and vertical gradient matrices, determine the overall grayscale gradient value of each pixel; Based on the comprehensive gray-level gradient value, the standard deviation of the comprehensive gray-level gradient values ​​of all pixels in the neighborhood is determined, and the standard deviation is used as the gray-level gradient stability calculation result of the corresponding candidate feature point.

5. The debris flow disaster early warning method based on image recognition according to claim 1, characterized in that, The step of performing interference region masking filtering on the registered first comparison video stream data and the registered second comparison video stream data to obtain masked registered first comparison video stream data and masked registered second comparison video stream data includes: Acquire multi-camera calibration parameters and GIS terrain data; Based on each frame of the registered first comparison video stream data and the registered second comparison video stream data, and combined with the multi-camera calibration parameters and GIS terrain data, a static interference area mask is constructed. Inter-frame difference calculation is performed on multiple consecutive frames of images at the same spatial location in the first and second registration comparison video stream data to obtain an inter-frame difference image; Based on the inter-frame difference image, the instantaneous dynamic interference region and the periodic dynamic interference region are determined; Based on the static interference region mask, the instantaneous dynamic interference region, and the periodic dynamic interference region, a comprehensive interference region mask is constructed. The frames of the first and second comparison video streams are filtered according to the comprehensive interference region mask to obtain the first and second comparison video streams after mask filtering.

6. The debris flow disaster early warning method based on image recognition according to claim 5, characterized in that, The step of constructing a static interference area mask based on each frame of the registered first comparison video stream data and the registered second comparison video stream data, combined with the multi-view camera calibration parameters and GIS terrain data, includes: Extract intrinsic and extrinsic parameters from the multi-view camera calibration parameters; Based on the intrinsic and extrinsic parameters, the two-dimensional pixel coordinates of each frame of the registered first comparison video stream data and the registered second comparison video stream data are transformed into three-dimensional world coordinates; Extract real GIS terrain data that matches the monitoring area from the GIS terrain data, wherein the real GIS terrain data includes terrain type data, altitude data, and boundary coordinates of the monitoring target mountain; The converted 3D world coordinates are spatially overlaid with the real GIS terrain data, and the non-monitored target mountain areas are removed from the 3D world coordinates to obtain the 3D world coordinates of the static interference area. The three-dimensional world coordinates of the static interference region are converted inversely into two-dimensional pixel coordinates of each frame of the first and second comparison video stream data, and the regions corresponding to the two-dimensional pixel coordinates are marked to obtain the static interference region mask.

7. The debris flow disaster early warning method based on image recognition according to claim 5, characterized in that, The step of performing inter-frame difference calculation on multiple consecutive frames of images at the same spatial location in the registered first comparison video stream data and the registered second comparison video stream data to obtain an inter-frame difference image includes: From the first and second registration comparison video stream data, N consecutive frames of images within the same monitoring period are extracted respectively, wherein the sequence of the N consecutive frames of images covers the same spatial monitoring area; The pixel grayscale values ​​of each frame in the captured N consecutive frames are uniformly mapped to a preset range to obtain the preprocessed N consecutive frames. The first frame in the preprocessed N consecutive frames of images is taken as the reference frame, and the pixel grayscale difference between the i-th frame and the reference frame is calculated sequentially in time order. The pixel grayscale difference of all pixels is binarized to distinguish the pixels into dynamically changing pixels and static pixels, thus obtaining a single-frame difference image; Perform an inter-frame logical AND operation on N to 1 consecutive single-frame difference images to obtain an inter-frame difference image.

8. A debris flow disaster early warning device based on image recognition, characterized in that, include: The acquisition unit is used to acquire the first comparison video stream data of the multi-camera in a first time period and the second comparison video stream data of the multi-camera in a second time period. The first processing unit is used to perform mountain feature registration on the first comparison video stream data and the second comparison video stream data to obtain registered first comparison video stream data and registered second comparison video stream data. The second processing unit is used to perform interference region masking filtering on the registered first comparison video stream data and the registered second comparison video stream data to obtain masked registered first comparison video stream data and masked registered second comparison video stream data. The third processing unit is used to determine the set of mountain sliding related difference pixels based on the masked registration first comparison video stream data and the masked registration second comparison video stream data. The early warning unit is used to construct a three-dimensional landslide monitoring model based on the set of differential pixels associated with landslides, the first comparison video stream data, and the second comparison video stream data, determine the target monitoring area of ​​the detection equipment, control the detection equipment to monitor the target monitoring area for debris flow, and issue a debris flow early warning signal.

9. A terminal, characterized in that, The system includes a processor, an input device, an output device, and a memory, which are interconnected. The memory stores a computer program, which includes program instructions. The processor is configured to invoke the program instructions to execute the image recognition-based debris flow disaster early warning method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the image recognition-based debris flow disaster early warning method as described in any one of claims 1-7.

Images