Dual-video debris flow early warning method, device and system
By combining binocular cameras and radar, and using image differences to calculate the distance of landslides, the problem of low accuracy and high cost of existing debris flow early warning technologies has been solved, achieving low-cost and high-precision debris flow early warning.
Patent Information
- Application Number
- CN202511152032.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-12
- Publication Date
- 2025-11-28
AI Technical Summary
Existing debris flow early warning technologies have low accuracy and high cost, especially when monitoring a wide area, as they require a large number of instruments that are susceptible to interference.
The system uses binocular cameras to capture images of the mountain from different angles, obtaining video streams from two time periods. The sliding distance of the mountain is calculated by analyzing the differences between the images, and radar is used to monitor the target area and send out early warning signals.
This improved the accuracy of early warning and reduced costs, avoiding the need for deploying a large number of high-precision radars and enabling targeted monitoring of potential debris flow areas.
Smart Images
Figure CN121034022A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of geological disaster early warning, and in particular to a double-video debris flow early warning method, device and system. BACKGROUND
[0002] Debris flow is a geological disaster that often occurs in mountainous areas, and has the characteristics of suddenness and strong destructive power. In order to avoid a large amount of property and life loss, it is necessary to monitor and timely early warn the debris flow disaster. The devices for monitoring and early warning of debris flow at present include mud flow meters, mud level monitors, etc. However, these instruments need to be installed on the mountain slope where debris flow is prone to occur and are in contact with the monitored area, and thus are easily disturbed by the contact area, resulting in inaccurate early warning. Since the range monitored by a single instrument is small, when the monitored area is wide, a large number of instruments are needed, and thus the cost is high. SUMMARY
[0003] In view of this, the embodiments of the present application provide a double-video debris flow early warning method, device and system, which are used to solve the technical problem of low accuracy of the existing debris flow early warning technology.
[0004] The technical solution adopted by the present application is as follows: In a first aspect, the present application provides a double-video debris flow early warning method, which comprises the following steps: obtaining a first reference video stream obtained by a first camera unit of a binocular camera shooting a mountain body and a second reference video stream obtained by a second camera unit of the binocular camera shooting the mountain body in a first time period; obtaining a first video stream obtained by the first camera unit of the binocular camera shooting the mountain body and a second video stream obtained by the second camera unit of the binocular camera shooting the mountain body in a second time period after the first time period; obtaining a mountain body sliding distance and a target area monitored by a radar according to the image difference between the first reference video stream, the first video stream, the second reference video stream and the second video stream; controlling the radar to monitor the target area, and sending a debris flow early warning signal according to the mountain body sliding distance and / or the monitoring result; The first time period and the second time period are equal.
[0005] Preferably, in the step of controlling the radar to monitor the target area and sending a debris flow early warning signal according to the mountain body sliding distance and / or the monitoring result, the early warning signal is sent when the mountain body sliding distance exceeds a preset distance.
[0006] Preferably, the speed of the mountain body sliding is calculated according to the distance of the mountain body sliding and the interval time between the first time period and the second time period.
[0007] Preferably, the binocular camera simultaneously captures the mountain from different angles.
[0008] Preferably, the radar is an x-band radar.
[0009] Preferably, the obtaining the mountain sliding distance and the target area monitored by the radar according to the image difference between the first reference video stream, the first video stream, the second reference video stream and the second video stream further comprises the following steps: analyzing whether the change of the mountain satisfies a preset condition according to the first reference video stream and the first video stream; if yes, obtaining the image difference between the first reference video stream and the first video stream; obtaining the mountain sliding distance and the target area monitored by the radar according to the image difference, the first reference video stream, the first video stream, the second reference video stream and the second video stream; the analyzing whether the change of the mountain satisfies a preset condition according to the first reference video stream and the first video stream further comprises the following steps: obtaining an average image of the first reference video stream as a first average image; obtaining an average image of the first video stream as a second average image; obtaining a cross-correlation coefficient of the first average image and the second average image and a first threshold value; if the cross-correlation coefficient is greater than the first threshold value, the change of the mountain satisfies the preset condition, otherwise, the change of the mountain does not satisfy the preset condition.
[0010] Preferably, the cross-correlation coefficient is e, the first threshold value is E, if e>E, it indicates that the change of the mountain satisfies the preset condition, otherwise, the change of the mountain does not satisfy the preset condition, wherein E=0.95.
[0011] In a second aspect, the present application further provides a debris flow early warning device based on video images and radar, the device comprising: a reference video stream acquisition module, the video stream acquisition module is used for obtaining a first reference video stream obtained by a first camera unit in a binocular camera capturing a mountain in a first time period and a second reference video stream obtained by a second camera unit capturing the mountain; a video stream acquisition module, the video stream acquisition module is used for obtaining a first video stream obtained by the first camera unit in the binocular camera capturing the mountain in a second time period after the first time period and a second video stream obtained by the second camera unit capturing the mountain; a video stream analysis module, the video stream analysis module is used for obtaining a mountain sliding distance and a target area monitored by a radar according to an image difference between the first reference video stream, the first video stream, the second reference video stream and the second video stream; A radar monitoring module is configured to control the radar to monitor the target area and send a debris flow early warning signal according to the monitoring result.
[0012] Preferably, the video stream analysis module further comprises a condition checking sub-module configured to check whether the change of the mountain in the first reference video stream and the first video stream satisfies a preset condition. An image difference acquisition sub-module is configured to acquire the image difference between the first reference video stream and the first video stream if the change of the mountain in the first reference video stream and the first video stream satisfies the preset condition. A sliding distance and target area acquisition sub-module is configured to acquire the sliding distance of the mountain and the target area for radar monitoring according to the image difference, the first reference video stream, the first video stream, the second reference video stream and the second video stream.
[0013] In a third aspect, the present application further provides a debris flow early warning system based on video images and radar, which comprises a binocular camera, a radar, at least one processor, at least one memory and computer program instructions stored in the memory, the binocular camera and the radar are electrically connected with the processor, and the computer program instructions are executed by the processor to realize the method of the first aspect.
[0014] Beneficial effects: The double-video debris flow early warning method, device, system and medium of the present application utilize the analysis of video streams in two time periods to acquire the tiny image difference between the two time periods, which can reflect the tiny sliding condition of the photographed mountain before the debris flow outbreak. The present application calculates the sliding distance of the mountain according to the image difference and finds the area where the mountain slides, and then utilizes the radar to closely monitor the area, so that the potential area of the debris flow outbreak can be monitored in a targeted manner, without the need to arrange a large number of high-precision radars to cover all areas of the mountain, thus having the characteristics of low cost, low energy consumption and high precision. Since the binocular camera and the radar can monitor the mountain in a non-contact manner, they are not easily affected by the monitored mountain itself, and the accuracy of the early warning can be further improved. BRIEF DESCRIPTION OF DRAWINGS
[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments of the present application will be briefly introduced below. For those skilled in the art, other drawings can also be obtained on the basis of these drawings without creative labor, and these are within the protection scope of the present application.
[0016] Figure 1 The flowchart of the double-video debris flow early warning method of the present application is shown in the figure. Figure 2 This is a flowchart illustrating the method for obtaining the mountain sliding distance according to the present invention; Figure 3 This is a flowchart illustrating the method for analyzing mountain changes according to the present invention; Figure 4 This is a flowchart illustrating the method for acquiring difference images according to the present invention; Figure 5 This is a flowchart illustrating the method for calculating the sliding distance of a mountain in this invention; Figure 6 This is a flowchart illustrating the method for obtaining the first set of depth values according to the present invention. Figure 7 This is a flowchart illustrating the method for acquiring the target area monitored by radar according to the present invention. Figure 8 This is a schematic diagram of the debris flow early warning device based on video images and radar according to the present invention; Figure 9 This is a structural block diagram of the debris flow early warning system based on binocular imaging and radar of the present invention. Figure 10 This is a schematic diagram illustrating the process of dividing a first grayscale image into several uniform rectangular regions according to the present invention. Figure 11 This is a schematic diagram illustrating the acquisition of the abnormal pixel closest to the center of the rectangular region according to the present invention. Detailed Implementation
[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. In the description of the present invention, it should be understood that the terms "center," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Where there is no conflict, the various features in the embodiments and examples of this invention can be combined with each other, all of which are within the scope of protection of this invention.
[0018] Example 1 like Figure 1 As shown, this embodiment provides a dual-video debris flow early warning method, which includes the following steps: S1: Acquire the first reference video stream C1 obtained by the first camera unit of the binocular camera capturing the mountain and the second reference video stream C2 obtained by the second camera unit capturing the mountain during the first time period; The binocular camera has two camera units positioned at different locations, allowing for the capture of images of the monitored mountain from two different angles. The binocular camera has a baseline length of L and a resolution of W*H. To avoid interference from accidental factors such as wind or falling rocks, this step involves acquiring a video stream over a specific period. This acquired video stream includes multiple frames stored in chronological order. The length of the first time period can be set empirically.
[0019] S2: Obtain the first video stream C11 obtained by the first camera unit of the binocular camera capturing the mountain and the second video stream C22 obtained by the second camera unit capturing the mountain during the second time period after the first time period; This step involves using a binocular camera to capture images of the mountain at regular intervals, obtaining a video stream over a given period. The acquired video stream includes multiple frames stored in chronological order. The intervals and the length of the first time period can be set empirically. As an optional but advantageous implementation, the first and second time periods are equal.
[0020] S3: Obtain the landslide distance and the target area monitored by radar based on the image differences between the first reference video stream, the second reference video stream, and the second video stream; Before a mudslide, minor landslides often occur. This step utilizes the analysis of video streams from two time periods to capture minute image differences between them. These differences reflect subtle landslides captured on camera. Therefore, this step can calculate the distance of the landslide and locate the affected area based on these image differences. This area is then closely monitored using radar, providing early warning before a sudden mudslide erupts.
[0021] S4: Control the radar to monitor the target area and send a debris flow warning signal based on the landslide distance and / or monitoring results.
[0022] This embodiment utilizes a binocular camera to continuously photograph the mountainside. Once an area exhibiting minor landslides is identified, the radar is controlled to focus on monitoring that area. When a significant change in the mountain's shape is detected within a short period, a warning signal is generated and transmitted. This avoids deploying numerous high-precision radars to monitor the entire mountain area, thus saving energy and reducing costs while ensuring accurate and timely warnings. The radar can be an X-band radar to monitor the movement of rocks within the target area. A warning signal is sent when the detected rock movement speed exceeds a preset speed. Alternatively, this embodiment can send a warning signal when the calculated landslide distance exceeds a preset distance. Furthermore, this embodiment can calculate the landslide speed based on the landslide distance and the interval T between the first and second time periods. Let the landslide distance be S, then the landslide speed v = T / S. A warning signal can also be sent when the landslide speed exceeds an allowable speed.
[0023] like Figure 2 As shown, as an optional but advantageous implementation, in this embodiment, step S3: obtaining the landslide distance and the target area monitored by radar based on the image differences between the first reference video stream, the second reference video stream, and the second video stream, further includes the following steps: S31: Analyze whether the changes in the mountain meet the preset conditions based on the first reference video stream and the first video stream; This step determines whether a small landslide occurred before a debris flow occurred. If the landslide is large, it indicates that a debris flow may have already occurred. Since this embodiment is mainly used for early warning before a debris flow occurs, it is mainly applicable to the analysis and processing of small landslides. In order to provide the accuracy of the analysis, this step first analyzes the degree of change of the mountain and selects the cases where the degree of change of the mountain meets the requirements for analysis and processing.
[0024] like Figure 3 As shown, as an optional but advantageous implementation, in this embodiment, S31: analyzing whether the changes in the mountain meet the preset conditions based on the first reference video stream and the first video stream further includes the following steps: S311: Obtain the average image of the first reference video stream as the first average image; This step can use a multi-image averaging method to process multiple frames of the first reference video stream to obtain an average image, thereby reducing noise caused by external interference factors such as falling rocks and wind.
[0025] S312: Obtain the average image of the first video stream as the second average image; This step can use the multi-image averaging method to process multiple frames of the first video stream to obtain an average image, thereby reducing noise caused by external interference factors such as falling rocks and wind.
[0026] S313: Obtain the cross-correlation coefficient of the first average image and the second average image, and the first threshold.
[0027] The first threshold is the maximum value that the cross-correlation coefficient cannot exceed, and this value can be determined based on experience.
[0028] S314: If the cross-correlation coefficient is greater than the first threshold, the change of the mountain meets the preset condition; otherwise, the preset condition is not met.
[0029] For example, if the cross-correlation coefficient is e and the first threshold is E, then if e > E, it indicates that the landslide belongs to the category of minor landslides and the change in the mountain meets the preset conditions; otherwise, it indicates that the landslide does not belong to the category of minor landslides and the change in the mountain does not meet the aforementioned preset conditions. As an optional but advantageous implementation method, E = 0.95.
[0030] S32: If so, obtain the image difference between the first reference video stream and the first video stream; This step identifies image differences between video streams captured at two different times, assuming the landslide is minor. Image differences refer to pixels in the image that have changed significantly.
[0031] like Figure 4As shown, as an optional but advantageous implementation, in this embodiment, step S32: if so, obtaining the image difference between the first reference video stream and the first video stream further includes the following steps: S321: Obtain the first grayscale image obtained by converting the first average image and the second grayscale image obtained by converting the second average image; S322: The first image set is obtained by constructing an n-layer pyramid image based on the first grayscale image; Where n is a positive integer greater than 1, and as an optional but advantageous implementation, n is 5. The n-layer pyramid image constructed from the first grayscale image has a total of n images, and the set of these n images is the first image set.
[0032] S323: The second image set is obtained by constructing an n-layer pyramid image based on the second grayscale image; The n-layer pyramid image constructed from the second grayscale image has a total of n images, and the set of these n images is the second image set.
[0033] The n-layer pyramid image is composed of a series of images, with the bottom image being the largest and the top image being the smallest. S324: Subtract the first image set from the second image set to obtain the difference image set; This step involves subtracting the first layer image from the first image set to obtain the difference image of the first layer, subtracting the second layer image from the first image set to obtain the difference image of the second layer, and so on, until the nth layer image from the first image set is obtained. All difference images from layers 1 to n are then combined into a difference image set. Subtracting two images means subtracting the pixel values of corresponding pixels (pixels with the same horizontal and vertical coordinates in both images) and taking the absolute value. This absolute value is used as the pixel value of the corresponding pixels (pixels with the same horizontal and vertical coordinates) in the difference image.
[0034] S325: Perform low-pass filtering and magnification on each image in the differential image set; This step can process each image in the difference set separately, first by performing low-pass filtering and then by magnification.
[0035] S326: The images after low-pass filtering and magnification are superimposed to obtain a reference image with the same resolution as the first average image; After low-pass filtering and magnification of all n images in the difference image set, the n images are restored to their original size before the pyramid image was constructed. Then, these n processed images are superimposed to restore the original resolution before the pyramid image was constructed.
[0036] S327: Obtain a set of several pixels in the reference image whose pixel values are greater than a set threshold as the set of difference pixels.
[0037] The threshold value can be determined empirically. As an optional implementation, the threshold value can be any real number between 15 and 45. For pixels in the reference image whose pixel values are greater than the threshold value, the horizontal and vertical coordinates of the pixel in the reference image can be obtained. Pixels in the first grayscale image and the second grayscale image with the same horizontal and vertical coordinates are respectively the difference pixels in the first grayscale image and the difference pixels in the second grayscale image. S33: Obtain the mountain sliding distance and the target area monitored by radar based on the image differences, the first reference video stream, the first video stream, the second reference video stream, and the second video stream.
[0038] like Figure 5 As shown, the method for obtaining the mountain sliding distance mainly includes the following steps: S331: Obtain the three-dimensional reconstructed image of the first grayscale image based on the first reference video stream and the second reference video stream as the first three-dimensional image; Since this embodiment uses a binocular camera to simultaneously capture images of the mountain from different angles, a three-dimensional image of the mountain can be created using the first and second reference video streams captured by the two camera units of the binocular camera. To avoid noise interference, a grayscale image obtained by converting the average image of the first and second reference video streams can be used to create the three-dimensional image. The method for creating a three-dimensional image using images captured by the binocular camera can employ existing technology.
[0039] S332: Obtain the three-dimensional reconstructed image of the second grayscale image based on the first video stream and the second video stream as the second three-dimensional image; Similarly, in this embodiment, two video streams captured by a binocular camera in the second time period are used to create a three-dimensional image. The method is the same as the previous step and will not be repeated here.
[0040] S333: Based on the first three-dimensional image and the set of difference pixels, obtain the set of depth values corresponding to the set of difference pixels as the first depth value set (za1, za2, ..., za(m-1), zam); This step uses the first 3D image to obtain the depth value of each difference pixel in the difference pixel set in the first 3D image, and combines these depth values into a first depth value set. An element in the set represents the depth value corresponding to a difference pixel. For example, za1 represents the depth value of the first difference pixel, za2 represents the depth value of the second difference pixel, za(m-1) represents the depth value of the (m-1)th difference pixel, and zam represents the depth value of the mth difference pixel.
[0041] like Figure 6 As shown, in a preferred embodiment, S333 in this embodiment, which involves obtaining a set of depth values corresponding to the set of difference pixels as a first depth value set (za1, za2, ..., za(m-1), zam), further includes the following steps: S3331: Obtain several uniform target pixels in the first grayscale image based on the set of difference pixels; S3332: In the first grayscale image, obtain the corresponding first target region centered on each uniform target pixel; In the first grayscale image, a region with a width of w and a height of h is selected centered on each target pixel, where the width is the length in the x-direction of the first grayscale image and the height is the length in the y-direction of the first grayscale image; S3333: Obtain the second target region corresponding to each first target region in the first three-dimensional image; In the first 3D image, a region with a width of w and a height of h is selected, centered on the target pixel. This region is the second target region corresponding to the first target region in the previous step. The width is the length in the x-direction of the first 3D image, and the height is the length in the y-direction of the first 3D image. S3334: For each first target region, obtain the depth value of each pixel in the first target region according to the corresponding second target region; Input the two-dimensional coordinates of all pixels in the first target region into the three-dimensional image, and obtain the third coordinate corresponding to the two-dimensional coordinates in the three-dimensional image. The third coordinate is the depth value of the pixel.
[0042] For example, the coordinates of the i-th pixel in the first target region are (xi, yi), where xi is the horizontal coordinate and yi is the vertical coordinate. That is, the i-th pixel is located in the xi-th column and yi-th row of the first grayscale image. In the first 3D image, the 3D coordinates of the pixel with horizontal coordinate xi and vertical coordinate yi are (xi, yi, zi), where zi is the depth value of the i-th pixel in the first target region. Since each difference pixel corresponds to a first target region, the aforementioned operation can be performed on each of the first target regions one by one.
[0043] S3335: For each first target region, obtain the three-dimensional coordinates (xi, yi, zi) of each pixel in the first target region, where the xi coordinate and yi coordinate are the horizontal and vertical coordinates of the pixel in the first grayscale image, respectively, and the zi coordinate is the depth value of the pixel.
[0044] S3336: For each first target region, perform surface fitting on all pixels in the first target region based on the three-dimensional coordinates of the pixel to obtain the surface equation. This step involves creating a surface for all pixels in the first target region based on their 3D coordinates. Since there are multiple first target regions, the aforementioned operation can be performed on each of these regions individually to obtain the surface equation corresponding to each first target region.
[0045] S3337: Obtain the depth value of each target pixel based on its horizontal and vertical coordinates in the first grayscale image and the corresponding surface equation. The set of depth values of all target pixels is used as the first depth value set.
[0046] This step substitutes the x-coordinate and y-coordinate of the target pixel in the first grayscale image into the surface equation corresponding to the first target region where the target pixel is located, to obtain the depth value of the target pixel, and uses all the obtained depth values of the target pixels as the first depth value set.
[0047] S334: Based on the second three-dimensional image and the set of difference pixels, obtain the set of depth values corresponding to the set of difference pixels as the second depth value set (zb1, zb2, ..., zb(m-1), zbm); Similarly, this step uses the second 3D image to obtain the depth value of each difference pixel in the difference pixel set in the second 3D image, and combines these depth values into a second depth value set. An element in the set represents the depth value corresponding to a difference pixel. For example, zb1 represents the depth value of the first difference pixel, zb2 represents the depth value of the second difference pixel, zb(m-1) represents the depth value of the (m-1)th difference pixel, and zbm represents the depth value of the mth difference pixel.
[0048] As a preferred embodiment, in this embodiment, S334: obtaining the set of depth values corresponding to the set of difference pixels as the second depth value set (zb1, zb2, ..., zb(m-1), zbm) based on the second three-dimensional image and the set of difference pixels further includes the following steps: S3341: Obtain several uniform target pixels in the second grayscale image based on the set of difference pixels; S3342: In the second grayscale image, obtain the corresponding first target region centered on each uniform target pixel; in the second grayscale image, select a region with width w and height h centered on each target pixel, where the width is the length in the x direction of the second grayscale image and the height is the length in the y direction of the second grayscale image. S3343: Obtain the second target region corresponding to each first target region in the second three-dimensional image; In the second 3D image, a region with a width of w and a height of h is selected, centered on the target pixel. This region is the second target region corresponding to the first target region in the previous step. The width is the length in the x-direction of the second 3D image, and the height is the length in the y-direction of the second 3D image. S3344: For each first target region, obtain the depth value of each pixel in the first target region according to the corresponding second target region; input the two-dimensional coordinates of all pixels in the first target region into the three-dimensional image, and obtain the third coordinate corresponding to the two-dimensional coordinates in the three-dimensional image. The third coordinate is the depth value of the pixel.
[0049] For example, the coordinates of the i-th pixel in the first target region are (xi, yi), where xi is the horizontal coordinate and yi is the vertical coordinate. That is, the i-th pixel is located in the xi-th column and yi-th row of the second grayscale image. In the first 3D image, the 3D coordinates of the pixel with horizontal coordinate xi and vertical coordinate yi are (xi, yi, zi), where zi is the depth value of the i-th pixel in the first target region. Since each difference pixel corresponds to a first target region, the aforementioned operation can be performed on each of the first target regions one by one.
[0050] S3345: For each first target region, obtain the three-dimensional coordinates (xi, yi, zi) of each pixel in the first target region, where the xi coordinate and yi coordinate are the horizontal and vertical coordinates of the pixel in the second grayscale image, respectively, and the zi coordinate is the depth value of the pixel.
[0051] S3346: For each first target region, perform surface fitting on all pixels in the first target region based on the three-dimensional coordinates of the pixels to obtain the surface equation; this step involves performing surface fitting on all pixels in the first target region based on the three-dimensional coordinates of all pixels in the first target region. Since there are multiple first target regions, the aforementioned operation can be performed on each of these first target regions one by one to obtain the surface equation corresponding to each first target region.
[0052] S3347: Obtain the depth value of each target pixel based on its x-coordinate, y-coordinate, and the corresponding surface equation in the first grayscale image. The set of depth values for all target pixels is then used as the second depth value set. This step substitutes the x-coordinate and y-coordinate of the target pixel in the first grayscale image into the surface equation corresponding to the first target region where the target pixel is located to obtain its depth value. All obtained depth values of the target pixels are then used as the second depth value set.
[0053] like Figure 10 As shown, the aforementioned uniform target pixel refers to the difference pixel closest to the center of each region after the first grayscale image or the second grayscale image is divided into multiple regions. This can be achieved by first uniformly dividing the first grayscale image or the second grayscale image into k rectangular regions of equal length and width, and then obtaining the center coordinates of each rectangular region. Figure 10 The small circles in the diagram represent the center coordinates of each region, while the large hollow circles represent abnormal pixels. For example... Figure 11 As shown, the closest difference pixel to the center coordinate within the rectangular area is then selected as the uniform target pixel. In other words, the search can be performed with the center of the rectangular area as the center, and the closest difference pixel to the center can be found from all pixels in the set of difference pixels as the uniform target pixel for that area. Figure 11 The solid circle represents a uniform target pixel selected from the difference pixels. Since the reference image, the first grayscale image, and the second grayscale image have the same resolution, the x and y coordinates of the difference pixels in the reference image can be used as the x and y coordinates of the difference pixels in the first and second grayscale images (i.e., difference pixels can be represented by pixels in the reference image, the first grayscale image, and the second grayscale image, and the same difference pixel has the same x and y coordinates in the aforementioned three images), to obtain the difference pixel in the first or second grayscale image that is closest to the divided distance region as the target pixel. The aforementioned operation is performed on each divided rectangular region to obtain the same number of target pixels as the divided rectangular region.
[0054] In addition to obtaining the depth value of the target pixel using the aforementioned method, a radar point cloud map of the mountain surface can also be generated using radar. Then, the target pixel is registered with the radar point cloud map, and the depth value of the target pixel is obtained by the position of the registered target pixel in the radar point cloud map.
[0055] S335: Calculate the landslide distance S based on the first depth value set and the second depth value set, where S = (zb1 - za1 + zb2 - za2 + zb(m-1) - za(m-1) + zbm - zam) / m, where m is a positive integer greater than 1. In this embodiment, S33: Obtaining the landslide distance and the target area monitored by radar based on the image differences, the first reference video stream, the first video stream, the second reference video stream, and the second video stream further includes the following steps: S336: Obtain the three-dimensional coordinates of each target pixel, wherein the three-dimensional coordinates of the target pixel include the horizontal coordinate, vertical coordinate and depth value of the target pixel in the second grayscale image; S337: The relative position between the landslide area and the binocular camera is obtained based on the 3D coordinates of each target pixel. After the binocular camera is installed at the monitoring location, its parameters are calibrated, and a 3D image is reconstructed using the binocular camera. The 3D coordinates of each target pixel in the camera coordinate system are obtained using the 2D coordinates of the target pixels and the reconstructed 3D image. That is, the 2D coordinates of the target pixels are substituted into the 3D image to obtain the 3D coordinates of the target pixels. The landslide area corresponding to the target pixel is taken as the landslide area. The relative position between the landslide area corresponding to the target pixel and the binocular camera can be found using the aforementioned 3D coordinates.
[0056] S338: Obtain the relative position between the radar and the binocular camera; after both the radar and the binocular camera are installed at the designated location, the relative positional relationship between the two is determined.
[0057] S339: Obtain the target area monitored by the radar based on the relative position between the landslide area and the binocular camera, and the relative position between the radar and the binocular camera. This step can be achieved by transforming the landslide area from the camera coordinate system of the binocular camera to the radar coordinate system, thereby obtaining the coordinates of the landslide area in the radar coordinate system.
[0058] Example 2 Please see Figure 8 This embodiment provides a debris flow early warning device based on video images and radar. The device includes: A reference video stream acquisition module is used to acquire a first reference video stream obtained by the first camera unit of the binocular camera capturing the mountain and a second reference video stream obtained by the second camera unit capturing the mountain within a first time period. The video stream acquisition module is used to acquire the first video stream obtained by the first camera unit of the binocular camera capturing the mountain and the second video stream obtained by the second camera unit capturing the mountain during a second time period after the first time period. The video stream analysis module is used to obtain the landslide distance and the target area monitored by radar based on the image differences between the first reference video stream, the second reference video stream, and the second video stream. A radar monitoring module is included, which controls the radar to monitor the target area and sends a debris flow early warning signal based on the monitoring results. The module for obtaining the direction of travel and pedestrian location also includes: The video stream analysis module further includes: a condition verification submodule, which is used to analyze whether the changes in the mountain meet preset conditions based on the first reference video stream and the first video stream; an image difference acquisition submodule, which is used to acquire the image difference between the first reference video stream and the first video stream if the conditions are met; and a sliding distance and target area acquisition submodule, which is used to acquire the mountain sliding distance and the radar-monitored target area based on the image difference, the first reference video stream, the first video stream, the second reference video stream, and the second video stream.
[0059] Example 3 In addition, combined Figure 9 The dual-video debris flow early warning method described in the foregoing embodiments of the present invention can be implemented by the smart pole of this embodiment. Figure 9 A schematic diagram of a debris flow early warning system based on video images and radar provided in an embodiment of the present invention is shown.
[0060] The smart pole in this embodiment may include a processing circuit 401, a binocular camera 404, a radar 405, and a memory 402 storing computer program instructions.
[0061] Specifically, the processing circuit 401 described above may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of the present invention.
[0062] Memory 402 may include mass storage for data or instructions. For example, and not limitingly, memory 402 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 402 may include removable or non-removable (or fixed) media. Where appropriate, memory 402 may be internal or external to a data processing device. In a particular embodiment, memory 402 is a non-volatile solid-state memory. In a particular embodiment, memory 402 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.
[0063] The processor 401 reads and executes computer program instructions stored in the memory 402 to implement any of the regional random smart pole data addressing methods in the above embodiments.
[0064] In one example, the smart pole of this embodiment may also include a communication interface 403 and a bus 410. Wherein, as... Figure 9 As shown, the processing circuit 401, memory 402, communication interface 403, binocular camera 404, and radar 405 are connected through bus 410 and complete mutual communication.
[0065] The communication interface 403 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of the present invention.
[0066] Bus 410 includes hardware, software, or both, that couples the various components used for the smart pole together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 410 may include one or more buses. While specific buses are described and illustrated in embodiments of the invention, the invention contemplates any suitable bus or interconnect.
[0067] Example 4 Furthermore, in conjunction with the dual-video debris flow early warning method in the above embodiments, this invention can be implemented using a computer-readable storage medium. This computer-readable storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any one of the dual-video debris flow early warning methods in the above embodiments.
[0068] The above is a detailed description of the dual-video debris flow early warning method, device, equipment, and storage medium provided in the embodiments of the present invention.
[0069] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.
[0070] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the required tasks. The programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.
[0071] It should also be noted that the exemplary embodiments mentioned in this invention describe methods or systems based on a series of steps or apparatus. However, this invention is not limited to the order of the steps described above; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.
[0072] The above description is merely a specific embodiment of the present invention. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the protection scope of the present invention.
Claims
1. A method for double video debris flow early warning, the method comprising the steps of: obtaining a first reference video stream captured by a first camera unit of a binocular camera and a second reference video stream captured by a second camera unit of the binocular camera in a first time period; obtaining a first video stream captured by the first camera unit and a second video stream captured by the second camera unit in a second time period after the first time period; obtaining a target area of a radar monitoring and a sliding distance of a mountain according to image differences between the first reference video stream, the first video stream, the second reference video stream and the second video stream; controlling the radar to monitor the target area and sending a debris flow early warning signal according to the sliding distance of the mountain and / or the monitoring result; the first time period is equal to the second time period.
2. The dual video debris flow early warning method according to claim 1, characterized in that, In the step of controlling the radar to monitor the target area and sending a debris flow early warning signal according to the sliding distance of the mountain and / or the monitoring result, the early warning signal is sent when the sliding distance of the mountain exceeds a preset distance.
3. The dual video debris flow early warning method according to claim 2, characterized in that, The speed of the mountain sliding is calculated according to the sliding distance of the mountain and an interval time between the first time period and the second time period.
4. The dual video debris flow early warning method according to claim 1, characterized in that, The binocular camera captures the mountain from different angles at the same time.
5. The dual video debris flow early warning method according to claim 1, characterized in that, The radar is an x-band radar.
6. The dual video debris flow early warning method according to claim 1, characterized in that, The step of obtaining the target area of the radar monitoring and the sliding distance of the mountain according to the image differences between the first reference video stream, the first video stream, the second reference video stream and the second video stream further comprises the steps of: analyzing whether a change of the mountain meets a preset condition according to the first reference video stream and the first video stream; if yes, obtaining an image difference between the first reference video stream and the first video stream; obtaining the target area of the radar monitoring and the sliding distance of the mountain according to the image difference, the first reference video stream, the first video stream, the second reference video stream and the second video stream. The step of analyzing whether the change of the mountain meets the preset condition according to the first reference video stream and the first video stream further comprises the steps of: obtaining an average image of the first reference video stream as a first average image; obtaining an average image of the first video stream as a second average image; obtaining a cross-correlation coefficient of the first average image and the second average image and a first threshold value; if the cross-correlation coefficient is greater than the first threshold value, the change of the mountain meets the preset condition, otherwise, the change of the mountain does not meet the preset condition.
7. The dual video debris flow early warning method according to claim 1, wherein a The cross-correlation coefficient is e, the first threshold value is E, if e>E, it indicates that the change of the mountain meets the preset condition, otherwise, the change of the mountain does not meet the preset condition, wherein E=0.
95.
8. A device for early warning of mudslides based on video images and radar, characterized in that, The apparatus comprises: a reference video stream obtaining module, the video stream obtaining module being configured to obtain a first reference video stream captured by a first camera unit of a binocular camera and a second reference video stream captured by a second camera unit of the binocular camera in a first time period; a video stream obtaining module, the video stream obtaining module being configured to obtain a first video stream captured by the first camera unit and a second video stream captured by the second camera unit in a second time period after the first time period; The video stream analysis module is configured to acquire the landslide distance and the target area for radar monitoring according to image differences between the first reference video stream, the first video stream, the second reference video stream and the second video stream. The radar monitoring module is configured to control the radar to monitor the target area and send a debris flow early warning signal according to a monitoring result.
9. The apparatus of claim 8, wherein: The video stream analysis module further includes a condition checking submodule configured to check whether a change of the mountain meets a preset condition according to the first reference video stream and the first video stream. An image difference acquisition submodule is configured to acquire image differences between the first reference video stream and the first video stream if the change of the mountain meets the preset condition. A landslide distance and target area acquisition submodule is configured to acquire the landslide distance and the target area for radar monitoring according to the image differences, the first reference video stream, the first video stream, the second reference video stream and the second video stream.
10. A video image and radar based debris flow warning system, characterized in that, The device comprises: a binocular camera, a radar and at least one processor, at least one memory and computer program instructions stored in the memory, the binocular camera and the radar being electrically connected to the processor, and the computer program instructions being executed by the processor to implement the method according to any one of claims 1-7.