Landslide detection method, device, equipment and storage medium

Abstract

The present application discloses a landslide detection method, apparatus, device and storage medium, which relate to the field of image processing technology. The present application obtains a current frame image of a roadside slope area by real-time shooting; based on the current frame image, detects through a preset target classification model to obtain potential landslide targets, and achieves preliminary landslide target classification; tracks the potential landslide targets through a preset target tracking model to obtain the motion trajectory of the potential landslide targets; based on the motion trajectory, determines whether there are signs of landslide in the roadside slope area, thereby timely discovering signs of landslide and solving the problem of low reliability of landslide detection.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to landslide detection methods, devices, equipment and storage media. Background Art

[0002] Traditionally, landslide detection on roadside slopes is typically performed through periodic remote sensing (drone / satellite) photography. However, this method has a low detection frequency. Since landslides are often sudden and uncertain, delayed information may result in missing the optimal emergency response time. Alternatively, detection is performed through sensors (such as inclinometers and strain gauges), which can only provide data at local points and cannot fully cover the entire slope area, affecting the accuracy of landslide detection.

[0003] Therefore, it is currently impossible to accurately detect landslides on road side slopes in real time, that is, the reliability of landslide detection is low. Summary of the Invention

[0004] The main purpose of this application is to provide a landslide detection method, device, equipment and storage medium, aiming to solve the technical problem of low reliability of landslide detection.

[0005] To achieve the above objectives, the present application proposes a landslide detection method, which comprises:

[0006] Obtaining the current frame image of the slope area on the side of the road;

[0007] Based on the current frame image, detecting through a preset target classification model to obtain potential landslide targets;

[0008] Tracking the potential landslide target through a preset target tracking model to obtain a motion trajectory of the potential landslide target;

[0009] Based on the motion trajectory, it is determined whether there is a sign of landslide in the slope area on the side of the road.

[0010] In one embodiment, the step of detecting a potential landslide target based on the current frame image using a preset target classification model includes:

[0011] Separating the dynamic target and the static target in the current frame image by using a preset background separation model to obtain a foreground image containing the dynamic target;

[0012] Calculating the foreground image based on a preset target recognition algorithm to obtain a candidate moving target;

[0013] The candidate moving targets are classified by a preset target classification model to obtain potential landslide targets.

[0014] In one embodiment, the preset target classification model includes a first target classification sub-model and a second target classification sub-model. The step of classifying the candidate moving targets using the preset target classification model to obtain potential landslide targets includes:

[0015] Classifying the candidate moving targets by using a first target classification sub-model, screening interference targets among the candidate moving targets, and obtaining candidate landslide targets;

[0016] Based on the area of ​​the candidate landslide target and a preset area threshold, determining whether the candidate landslide target has the possibility of landslide;

[0017] If yes, identifying the motion characteristics of the candidate landslide target through a second target classification sub-model;

[0018] If the motion feature is a landslide feature, the candidate landslide target is determined to be a potential landslide target.

[0019] In one embodiment, the step of determining whether the candidate landslide target has the possibility of landslide based on the area of ​​the candidate landslide target and a preset area threshold includes:

[0020] Determining whether the area of ​​the candidate landslide target is greater than or equal to a preset area threshold;

[0021] If the area of ​​the candidate landslide target is smaller than a preset area threshold, it is determined that the candidate landslide target does not have the possibility of landslide;

[0022] After the step of determining whether the candidate landslide target has the possibility of landslide based on the area of ​​the candidate landslide target and the preset area threshold, the method further includes:

[0023] If not, determining whether the number of candidate landslide targets with an area smaller than a preset area threshold is greater than a preset number threshold;

[0024] If it is greater than, the second target classification sub-model is used to determine whether the area with the largest number of candidate landslide targets whose areas are smaller than the preset area threshold has landslide characteristics.

[0025] In one embodiment, the preset target recognition algorithm includes a multi-frame difference method and a dense optical flow algorithm, and the step of calculating the foreground image based on the preset target recognition algorithm to obtain a candidate moving target includes:

[0026] comparing differences between foreground images including a current frame and a first preset frame preceding the current frame based on a multi-frame difference method, and determining an initial motion region based on the differences;

[0027] Calculating the optimal motion vector for each pixel in the initial motion area based on a dense optical flow algorithm;

[0028] A candidate moving target is determined based on the optimal motion vector, a preset motion speed threshold, and a preset motion direction.

[0029] In one embodiment, the step of tracking the trajectory of the potential landslide target by using a preset target tracking model to obtain the motion trajectory of the potential landslide target includes:

[0030] Tracking the potential landslide target in the foreground image including the current frame and the second preset frame before the current frame by using a preset target tracking model to obtain the motion trajectory of the potential landslide target;

[0031] The step of determining whether there is a sign of landslide in the slope area on the side of the road based on the motion trajectory includes:

[0032] If the movement trajectory is downward movement, and the area change of the potential landslide target is increasing over time, it is determined that there are signs of landslide in the slope area on the side of the road;

[0033] After the step of determining whether there is a landslide sign in the slope area on the side of the road based on the motion trajectory, the method further includes:

[0034] If there are signs of landslide, an early warning is issued based on the movement trajectory.

[0035] In one embodiment, the step of separating the dynamic target and the static target in the current frame image by using a preset background separation model to obtain a foreground image containing the dynamic target includes:

[0036] Separating the dynamic objects and static objects in the current frame and the third consecutive preset frame image thereof by a preset background separation model to obtain a foreground image containing the dynamic objects;

[0037] After the step of separating the dynamic objects and the static objects in the current frame image by using a preset background separation model to obtain a foreground image containing the dynamic objects, the method further includes:

[0038] The model parameters of the preset background separation model are updated to separate the dynamic objects and static objects in the next frame image and the previous preset frame image based on the updated background separation model.

[0039] In addition, to achieve the above-mentioned purpose, the present application also proposes a landslide detection device, which includes:

[0040] An acquisition module is used to acquire a current frame image of a slope area on the side of the road;

[0041] A detection module is used to detect potential landslide targets based on the current frame image using a preset target classification model;

[0042] A tracking module, configured to track the potential landslide target using a preset target tracking model to obtain a motion trajectory of the potential landslide target;

[0043] A judgment module is used to judge whether there is a sign of landslide in the slope area on the side of the road based on the motion trajectory.

[0044] In addition, to achieve the above objectives, the present application also proposes a landslide detection device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the landslide detection method described above.

[0045] In addition, to achieve the above objectives, the present application also proposes a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the steps of the landslide detection method described above are implemented.

[0046] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the landslide detection method described above are implemented.

[0047] One or more technical solutions proposed in this application have at least the following technical effects:

[0048] The present application captures and obtains a current frame image of a road-side slope area in real time; based on the current frame image, a preset target classification model is used for detection to obtain a potential landslide target, thereby achieving preliminary landslide target classification, thereby enabling more targeted detection of whether the potential landslide target has changed. Specifically, a preset target tracking model can be used to track the trajectory of the potential landslide target, thereby obtaining a motion trajectory of the potential landslide target and dynamically tracking the motion trajectory of the potential landslide target; based on the motion trajectory, it is determined whether the road-side slope area has signs of landslide; by dynamically tracking the potential landslide target in real time, the motion trajectory obtained can reflect whether the road-side slope area has overall signs of landslide.

[0049] It can be understood that the present application detects potential landslide targets in real time through target classification technology, and analyzes dynamic changes in slope areas through target tracking technology, thereby promptly discovering signs of landslides and improving the reliability of landslide detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present application and, together with the description, serve to explain the principles of the present application.

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0052] Figure 1 A schematic diagram of a flow chart provided for the first embodiment of the landslide detection method of the present application;

[0053] Figure 2 A schematic diagram of a scene provided for the first embodiment of the landslide detection method of this application;

[0054] Figure 3 A schematic diagram of a simplified flow chart of a landslide detection method provided in Example 2 of the present application;

[0055] Figure 4 A flow chart of the second embodiment of the landslide detection method of the present application is provided;

[0056] Figure 5 This is a schematic diagram of the module structure of the landslide detection device according to an embodiment of the present application;

[0057] Figure 6 Schematic diagram of the equipment structure of the hardware operating environment involved in the landslide detection method in the embodiment of the present application.

[0058] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0059] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0060] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0061] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication, and program execution capabilities, such as a tablet computer, personal computer, or mobile phone, or an electronic device or landslide detection device capable of performing the aforementioned functions. This embodiment and the following embodiments will be described using a landslide detection device as an example.

[0062] Based on this, the present application embodiment provides a landslide detection method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the landslide detection method of the present application.

[0063] In this embodiment, the landslide detection method includes steps S10 to S40:

[0064] Step S10, obtaining a current frame image of the slope area on the side of the road;

[0065] It should be noted that traditional landslide detection on roadside slopes is usually carried out through remote sensing detection (drones / satellites) with periodic (for example, daily or weekly) photography. However, this method has a low detection frequency. At the same time, this method is greatly affected by environmental factors such as weather and lighting, and cannot work effectively at night or in bad weather. Moreover, since the occurrence of slope landslides is usually sudden and uncertain, delayed information may result in missing the optimal emergency response time (unable to capture sudden landslides in real time); traditional landslide detection on roadside slopes also includes detection through physical sensors (such as inclinometers, strain gauges, etc.). This method can only provide data at local points, has high deployment and maintenance costs, and is difficult to fully cover the entire slope area. Moreover, this method can only monitor specific points and cannot perceive the landslide status of the entire slope area, affecting the accuracy of landslide detection.

[0066] Therefore, it is currently impossible to accurately detect landslides on road side slopes in real time, that is, the reliability of landslide detection is low.

[0067] In order to solve the above problems, this embodiment deploys surveillance cameras at a preset density in the slope area on the side of the road, without deploying complex sensors. Figure 2 Specifically, the deployment density of cameras can be increased in areas with complex terrain (such as areas with steep slopes, fragile rock formations, etc.) (for example, one camera can be deployed every 50 meters); the deployment density can be reduced in areas with flat terrain, for example, one camera can be deployed every 100 meters, balancing costs and monitoring needs.

[0068] The minimum configuration of the camera can be 1080P resolution, 30fps frame rate, and support for wide dynamic range (WDR) to adapt to backlit scenes; in addition, it can be equipped with infrared function to achieve night monitoring.

[0069] Based on the above method, it is possible to obtain the image or video of the slope area on the side of the road. Figure 3Specifically, in order to detect whether there is a landslide in the slope area in real time, this embodiment can obtain the current frame image of the slope area on the side of the road in real time, so as to facilitate subsequent processing of the current frame image and improve the real-time detection.

[0070] Step S20, based on the current frame image, detecting using a preset target classification model to obtain potential landslide targets;

[0071] It should be noted that before processing the current frame image, image stabilization processing can be performed first; refer to Figure 3 Specifically, a real-time anti-shake technology based on feature point matching can be used to respectively extract SIFT feature points (Scale-invariant feature transform, a local area in the current frame image) of the current frame image and the previous frame image corresponding to the current frame, and based on multiple pairs of the best matching points, calculate an affine transformation matrix (used to describe the relative motion between the two frames, including translation, rotation, and scaling, etc.).

[0072] Based on the affine transformation matrix, the Kalman filter is used to predict the jitter of the next frame image corresponding to the current frame; according to the jitter of the next frame image, the current frame image is reversely displaced to offset the expected jitter; thereby reducing false detections caused by mechanical vibration of the camera, and the processing delay is less than 10ms, that is, the calculation and compensation process can be completed in a very short time, ensuring that the output video stream is as smooth as possible and without obvious delay.

[0073] Furthermore, based on the current frame image, detection is performed using a preset target classification model, that is, each target in the image is classified and identified, thereby identifying potential landslide targets.

[0074] Specifically, the preset target classification model can be based on a convolutional neural network model or a support vector machine (SVM) training model; potential landslide targets can be newly appeared surface cracks, displaced soil, rocks, or accumulated sediments.

[0075] Step S30, tracking the trajectory of the potential landslide target by using a preset target tracking model to obtain the motion trajectory of the potential landslide target;

[0076] Since detection through a preset target classification model can only achieve preliminary landslide target classification but cannot indicate whether a landslide is currently occurring, this embodiment uses a preset target tracking model to track the trajectory of the potential landslide target, that is, to specifically detect whether the potential landslide target has changed, thereby achieving dynamic analysis of the movement state of the landslide target.

[0077] Among them, the preset target tracking model can be obtained based on convolutional neural network training, or it can be a model based on the optical flow method; the motion trajectory of the potential landslide target can be a downward irregular motion trajectory, a downward straight motion trajectory, a stationary motion trajectory, or a displacement trajectory to the left or right, etc.

[0078] Step S40: Based on the motion trajectory, determine whether there is any sign of landslide in the slope area on the side of the road.

[0079] It can be understood that based on the real-time analysis of the motion trajectory of the potential landslide target, it can be determined whether a landslide is about to occur, that is, whether there are signs of landslide in the slope area on the side of the road.

[0080] Furthermore, after the step of determining whether there is a landslide sign in the slope area on the side of the road based on the motion trajectory, the method further includes:

[0081] If there are signs of landslide, an early warning is issued based on the movement trajectory.

[0082] Specifically, once a landslide is confirmed, an early warning can be issued based on the movement trajectory so that timely measures can be taken to reduce possible risks. The early warning based on the movement trajectory can be based on the location of the landslide or the implementation of road closures based on the movement trajectory.

[0083] Among them, early warnings can be divided into multiple levels of warnings, such as potential risks (level one warning), confirmed warnings (level two warnings), etc.

[0084] This implementation deploys cameras to capture real-time images of the slope area and employs multi-level image processing in conjunction with a deep learning model to achieve a fully automated closed-loop process from data acquisition to early warning. This addresses the current issues of low monitoring frequency and limited real-time detection, providing a pure algorithm-based, real-time landslide detection method that reduces hardware costs and improves the timeliness of early warnings.

[0085] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 4 , step S20 includes steps S01 to S03:

[0086] Step S01, separating dynamic objects and static objects in the current frame image using a preset background separation model to obtain a foreground image containing dynamic objects;

[0087] In order to reduce the amount of calculation and improve the speed of landslide detection: the dynamic targets and static targets in the current frame image are separated by a preset background separation model to obtain a foreground image containing the dynamic targets; in the subsequent processing process, only the part containing dynamic information (i.e., the foreground image) needs to be processed; at the same time, removing the static information helps to remove unnecessary background interference, making the landslide target identification in the foreground image more accurate.

[0088] For example, in the monitoring of the slope area on the side of the road, the background may include static elements such as vegetation and buildings, while the soil movement caused by landslides is a dynamic target that requires more attention. Effective background separation can ensure that dynamic targets will not be ignored or misjudged as part of the background during the subsequent identification of landslide targets. It can also ensure that static targets, such as vegetation moved by the wind, will not be misjudged as landslide targets.

[0089] Among them, dynamic targets include soil, rocks, people, vehicles, animals, etc., and static targets include trees, bushes, etc.

[0090] Specifically, the implementation of separating the dynamic target and the static target in the current frame image by using a preset background separation model to obtain a foreground image containing the dynamic target may be:

[0091] The dynamic targets and static targets in the current frame and the third consecutive preset frame image thereof are separated by a preset background separation model to obtain a foreground image containing the dynamic targets.

[0092] Among them, the preset background separation model can be a mixture Gaussian model (GMM) or a frame difference method. By analyzing continuous multiple frame images through the mixture Gaussian model, static targets and dynamic targets can be distinguished and separated more accurately.

[0093] Furthermore, after the step of separating the dynamic targets and static targets in the current frame image through the preset background separation model to obtain a foreground image containing dynamic targets, it is also possible to: update the model parameters of the preset background separation model to separate the dynamic targets and static targets in the next frame image and its previous preset frame image based on the updated background separation model.

[0094] Specifically, since the analysis of multiple consecutive frames of images by models such as the mixed Gaussian model is based on the color distribution of pixels, the illumination gradient (sunrise and sunset, etc.) will affect the analysis accuracy of the pixels of each frame of images by the mixed Gaussian model during the background separation process. Therefore, in this embodiment, after the step of separating the dynamic targets and static targets in the current frame image through the preset background separation model to obtain a foreground image containing dynamic targets, the model parameters of the preset background separation model are updated (for example, the Gaussian distribution of the mixed Gaussian model is adjusted, or the difference threshold in the frame difference method is adjusted), so that the dynamic targets and static targets in the next frame image and the previous preset frame image are separated based on the updated background separation model, and the deviation caused by the illumination gradient can be adjusted.

[0095] Due to the gradual change of illumination, the impact on the pixels between two consecutive frames or five consecutive frames is relatively small. In this embodiment, after separating the dynamic targets and static targets in the current frame image through a preset background separation model to obtain a foreground image containing dynamic targets, the model parameters of the preset background separation model can be updated every preset frame.

[0096] Considering that the weather is different and the speed of illumination change is different, the preset frame interval required for each update of the model parameters of the preset background separation model can be determined based on the current weather.

[0097] S02, calculating the foreground image based on a preset target recognition algorithm to obtain a candidate moving target;

[0098] Furthermore, the foreground image only includes objects that may be moving (ie, objects with motion properties). In order to detect landslides in real time, it is possible to further detect whether there are objects that are moving in the current frame image.

[0099] Specifically, the preset target recognition algorithm includes a multi-frame difference method and a dense optical flow algorithm. The specific implementation method of calculating the foreground image based on the preset target recognition algorithm to obtain the candidate moving target may be:

[0100] Based on the multi-frame difference method, the differences between the foreground images of the current frame and the first preset frame before it are compared, and the initial motion area is determined based on the differences; the optimal motion vector of each pixel in the initial motion area is calculated based on the dense optical flow algorithm; and the candidate motion target is determined based on the optimal motion vector, a preset motion speed threshold and a preset motion direction.

[0101] Specifically, a multi-frame difference method is used to detect moving objects by comparing the difference between the current frame and the first preset frame before it (such as 3 consecutive frames or 5 consecutive frames); and the difference in pixel values ​​between each pair of adjacent frames is calculated; if the brightness of a pixel in the consecutive frames changes significantly, then this pixel may belong to a moving object, and the changed pixel points are marked to form a preliminary motion area (that is, the initial motion area is determined based on the difference).

[0102] Based on the dense optical flow algorithm, the optimal motion vector of the pixel points in the initial motion area is calculated, which contains information about the speed and direction of how the pixel moves from one frame to the next. By setting a reasonable speed range and direction as a threshold, it filters out those moving targets that are too slow (possibly noise or minor changes that are not of interest) and have deviated from the direction of movement. The coordinates and motion parameters (speed, direction, area) of the candidate moving targets are obtained.

[0103] For example, the screening targets have a speed greater than 0.15m / s and a movement direction that conforms to the gravity trend (downward deflection greater than 30°).

[0104] It should be noted that before the foreground image is calculated based on the preset target recognition algorithm to obtain the candidate moving target, noise suppression and target feature enhancement processing can also be performed. Specifically, it can work by replacing each pixel value in the foreground image with the median of its neighborhood pixel values. This can effectively eliminate small-sized discrete noise points such as rain, snow, and flying insects without affecting important information such as image edges; further, those short-term motion areas lasting less than 0.5 seconds are marked and treated as noise for elimination. For example, the changes caused by the slight shaking of plants due to a breeze usually do not last for a long time, so these interference factors can be identified and removed in this way.

[0105] Furthermore, the local contrast of the image is enhanced by the local contrast enhancement (CLAHE) algorithm to highlight the texture features of targets such as soil and rocks. Figure 3 .

[0106] Step S03: classify the candidate moving targets using a preset target classification model to obtain potential landslide targets.

[0107] Furthermore, since the candidate moving targets may still contain some dynamic targets that are not related to landslides (such as pedestrians, vehicles, leaves blown by the wind, etc.), the foreground image can be calculated based on a preset target recognition algorithm to obtain candidate moving targets, thereby removing noise and irrelevant areas, screening out candidate moving targets that may be landslide targets, and reducing the complexity of subsequent processing.

[0108] Among them, the preset target classification algorithm can be a lightweight MobileNetV3 network, which is used to quickly identify and distinguish "natural interference objects (birds, branches, pedestrians and vehicles, etc.)" from "potential landslide targets", referring to Figure 3 .

[0109] In this embodiment, the above-mentioned method can accurately distinguish interference objects and identify potential landslide targets, thereby contributing to the accuracy of subsequent landslide detection.

[0110] Based on the first and second embodiments of the present application, in the third embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to above and will not be described in detail. On this basis, in a feasible implementation, the preset target classification model includes a first target classification sub-model and a second target classification sub-model; the specific implementation method of classifying the candidate moving targets using the preset target classification model to obtain potential landslide targets can be:

[0111] The candidate moving targets are classified by the first target classification sub-model, and interference targets among the candidate moving targets are screened to obtain candidate landslide targets; based on the area of ​​the candidate landslide target and a preset area threshold, it is determined whether the candidate landslide target has the possibility of landslide; if so, the motion characteristics of the candidate landslide target are identified by the second target classification sub-model; if the motion characteristics are landslide characteristics, the candidate landslide target is determined to be a potential landslide target.

[0112] Specifically, the candidate moving targets may be classified by the first target classification sub-model to distinguish interference targets (such as pedestrians, vehicles, animals, etc.) from candidate landslide targets that may be related to landslides among the candidate moving targets.

[0113] Furthermore, landslides usually involve large-scale soil movement, so area is an important screening condition; if the area of ​​a candidate landslide target is too small (such as a single pedestrian or a small object), the possibility of it being a landslide can be ruled out. Therefore, based on the area of ​​the candidate landslide target and the preset area threshold, it can be judged whether the candidate landslide target has the possibility of landslide, referring to Figure 3 .

[0114] Among them, the area of ​​the candidate landslide target can be calculated based on the dense optical flow algorithm, or can be calculated by the target detection box identified by the first target classification sub-model; the preset area threshold can be greater than 500 or 1000 pixels, etc.

[0115] In a feasible implementation manner, the determining whether the candidate landslide target has the possibility of landslide based on the area of ​​the candidate landslide target and the preset area threshold may also be:

[0116] It is determined whether the area of ​​the candidate landslide target is greater than or equal to a preset area threshold; if the area of ​​the candidate landslide target is less than the preset area threshold, it is determined that the candidate landslide target has no possibility of landslide.

[0117] Specifically, if the area of ​​a candidate landslide target is smaller than the preset area threshold, it is considered unlikely to be a landslide target and is eliminated; otherwise, the motion characteristics of the candidate landslide target are further identified through the second target classification sub-model.

[0118] The second target classification sub-model can be a ResNet101 V2 network, which identifies motion features of candidate landslide targets. Motion features typically contain multiple levels of information, such as texture changes, shape changes, and displacement patterns. These features may be distributed across different levels of abstraction: low-level features such as edges, corners, and color changes; mid-level features such as local shape and texture changes; and high-level features such as overall trajectory consistency and directional regularity. The deep structure of ResNet101 V2 can gradually extract features from low to high levels, capturing richer motion information and providing comprehensive input for subsequent classification.

[0119] Specifically, the target classification model to be trained can be trained by superimposing virtual interference objects (such as a 3D model of a flying bird) on the real slope video, combined with a negative sample library (containing 20 types of common interference scenes (animal activities, vehicles, etc.)), etc., to obtain the above-mentioned preset target classification model and improve the generalization ability of the model.

[0120] In a feasible implementation manner, after the step of determining whether the candidate landslide target has the possibility of landslide based on the area of ​​the candidate landslide target and the preset area threshold, the method further includes:

[0121] If not, determine whether the number of candidate landslide targets with an area smaller than a preset area threshold is greater than a preset number threshold; if greater, determine whether the area with the largest number of candidate landslide targets with an area smaller than the preset area threshold has landslide characteristics through the second target classification sub-model.

[0122] It is understandable that in the early stages of a landslide, there may not be large-scale soil movement, but rather a series of small-scale cracks, loose soil, and other phenomena. These phenomena appear as multiple small-area targets in the image. Even if a single small-area change is not enough to be identified as a landslide, if they are concentrated in a specific area, it may be a precursor to a larger-scale landslide. Figure 3Therefore, it is necessary to pay special attention to these small-area but densely distributed changes. Therefore, when the candidate landslide target does not have the possibility of landslide, in order to avoid missed detection, it can be further determined whether the number of candidate landslide targets with an area smaller than a preset area threshold is greater than a preset number threshold;

[0123] Furthermore, if the number of candidate landslide targets with an area smaller than a preset area threshold is greater than a preset number threshold, the second target classification sub-model is used to determine whether the area with the most candidate landslide targets with an area smaller than the preset area threshold has landslide characteristics; it can be understood that once an area with a large number of small-area candidate landslide targets is determined, using the second target classification sub-model to conduct a detailed analysis of the area can help identify more subtle landslide characteristics (such as small displacements, gradual deformation, etc.). Such detailed analysis helps to capture early signs of landslides and improve the timeliness and efficiency of detection. Figure 3 .

[0124] At the same time, in order to improve the accuracy of early warning, the conditions for the above-mentioned first-level early warning can be that a landslide feature is detected in a single frame and the area of ​​the landslide target is greater than 1 square meter; the action of the first-level early warning can be to mark the area and start high-frequency sampling (increased to 60 times per second); the condition for the second-level early warning can be that the landslide feature is detected in 60 consecutive frames (with an interval of 1 second); the action of the second-level early warning can be to trigger an audible and visual alarm and push the coordinate information to the maintenance platform, refer to Figure 3 .

[0125] The specific implementation of tracking the trajectory of the potential landslide target by using the preset target tracking model to obtain the motion trajectory of the potential landslide target may be:

[0126] By using a preset target tracking model, the trajectory of a potential landslide target in a foreground image including a current frame and a second preset frame before the current frame is tracked to obtain a motion trajectory of the potential landslide target.

[0127] In order to accurately analyze the motion trajectory of potential landslide targets, a preset target tracking model can be used to track the trajectory of potential landslide targets in the foreground image including the current frame and the second preset frame before it (for example, the foreground image of the current frame and the five frames before it). Based on the position changes of the tracked potential landslide targets, the motion trajectory of the potential landslide targets can be calculated.

[0128] The preset target tracking model may be a SORT (Simple Online and Realtime Tracking) algorithm or a Deep SORT algorithm.

[0129] Specifically, according to the characteristics of the landslide phenomenon, the implementation method of judging whether there are signs of landslide in the slope area on the side of the road based on the motion trajectory can be: if the motion trajectory is downward movement and the area change of the potential landslide target is increasing over time, it is determined that there are signs of landslide in the slope area on the side of the road.

[0130] If the movement trajectory is downward, it matches the slope gradient. If the area change of the potential landslide target increases over time, it reflects the process of soil and rock accumulation. When the above conditions are met at the same time, it can be determined that there are signs of landslide in the slope area on the side of the road.

[0131] Specifically, in the above-mentioned process of detecting landslide targets based on area or speed, the preset area threshold and the preset speed threshold can be dynamically adjusted based on time or weather, so as to avoid misjudgment caused by reduced visibility at night or increased object movement speed due to rainy or snowy weather; specifically, the preset area threshold can be automatically relaxed at night, or the preset speed threshold can be increased in rainy and snowy weather (to avoid misjudgment of raindrops).

[0132] In this embodiment, by combining multiple noise suppression and deep learning algorithms, the problem of false alarms caused by complex environmental interference is solved; a multi-level early warning mechanism is used to achieve risk-graded response to avoid excessive alarms.

[0133] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the landslide detection method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0134] This application also provides a landslide detection device, please refer to Figure 5 , the landslide detection device comprises:

[0135] An acquisition module 10 is used to acquire a current frame image of a slope area on the side of the road;

[0136] A detection module 20 is configured to detect a potential landslide target based on the current frame image using a preset target classification model;

[0137] A tracking module 30 is configured to track the potential landslide target using a preset target tracking model to obtain a motion trajectory of the potential landslide target;

[0138] The judgment module 40 is configured to judge whether there is any sign of landslide in the slope area on the side of the road based on the motion trajectory.

[0139] In one embodiment, the detection module 20 includes:

[0140] A separation processing submodule is used to separate the dynamic target and the static target in the current frame image by using a preset background separation model to obtain a foreground image containing the dynamic target;

[0141] A calculation submodule, configured to calculate the foreground image based on a preset target recognition algorithm to obtain a candidate moving target;

[0142] The classification submodule is used to classify the candidate moving targets through a preset target classification model to obtain potential landslide targets.

[0143] In one embodiment, the preset target classification model includes a first target classification sub-model and a second target classification sub-model, and the classification sub-module includes:

[0144] a classification unit, configured to classify the candidate moving targets by using a first target classification sub-model, filter interference targets among the candidate moving targets, and obtain candidate landslide targets;

[0145] A first judging unit is configured to judge whether the candidate landslide target has the possibility of landslide based on the area of ​​the candidate landslide target and a preset area threshold;

[0146] a second judgment unit, configured to identify the motion characteristics of the candidate landslide target by using a second target classification sub-model if the candidate landslide target exists;

[0147] The first determining unit is configured to determine that the candidate landslide target is a potential landslide target if the motion feature is a landslide feature.

[0148] In one embodiment, the first determining unit includes:

[0149] A judging subunit, configured to judge whether the area of ​​the candidate landslide target is greater than or equal to a preset area threshold;

[0150] a determination subunit, configured to determine that the candidate landslide target does not have the possibility of landslide if the area of ​​the candidate landslide target is smaller than a preset area threshold;

[0151] Wherein, after the step of determining whether the candidate landslide target has the possibility of landslide based on the area of ​​the candidate landslide target and the preset area threshold, the classification submodule further includes:

[0152] a third judging unit, configured to, if not present, judge whether the number of candidate landslide targets having an area smaller than a preset area threshold is greater than a preset number threshold;

[0153] The second determining unit is configured to determine, if greater than, whether the region having the most candidate landslide targets with an area smaller than a preset area threshold has landslide characteristics through a second target classification sub-model.

[0154] In one embodiment, the preset target recognition algorithm includes a multi-frame difference method and a dense optical flow algorithm, and the calculation submodule includes:

[0155] a comparing unit, configured to compare differences between foreground images including a current frame and a first preset frame preceding the current frame based on a multi-frame difference method, and determine an initial motion region based on the differences;

[0156] A calculation unit, configured to calculate an optimal motion vector for each pixel in the initial motion area based on a dense optical flow algorithm;

[0157] The third determining unit is configured to determine a candidate moving target based on the optimal motion vector, a preset motion speed threshold, and a preset motion direction.

[0158] In one embodiment, the tracking module 30 includes:

[0159] The trajectory tracking submodule is used to track the potential landslide target in the foreground image including the current frame and the second preset frame before the current frame by using a preset target tracking model to obtain the motion trajectory of the potential landslide target;

[0160] The judgment module 40 includes:

[0161] a determination submodule, configured to determine that there is a sign of landslide in the slope area on the side of the road if the movement trajectory is downward movement and the area change of the potential landslide target is increasing over time;

[0162] Wherein, after the step of determining whether there is a landslide sign in the slope area on the side of the road based on the motion trajectory, the landslide detection device further includes:

[0163] The early warning module is used to issue an early warning based on the motion trajectory if there are signs of landslide.

[0164] In one embodiment, the separation processing submodule includes:

[0165] a separation processing unit, configured to separate dynamic objects and static objects in the current frame and the third consecutive preset frame image thereof using a preset background separation model to obtain a foreground image containing dynamic objects;

[0166] Wherein, after the step of separating the dynamic targets and the static targets in the current frame image by using a preset background separation model to obtain a foreground image containing the dynamic targets, the separation processing submodule further includes:

[0167] An updating unit is used to update the model parameters of the preset background separation model to separate the dynamic objects and static objects in the next frame image and the previous preset frame image based on the updated background separation model.

[0168] The landslide detection device provided in this application utilizes the landslide detection method described in the aforementioned embodiments, resolving the technical issue of low landslide detection reliability. Compared to the prior art, the landslide detection device provided in this application achieves the same beneficial effects as the landslide detection method described in the aforementioned embodiments. Other technical features of the landslide detection device are the same as those disclosed in the aforementioned embodiments and are not further detailed here.

[0169] The present application provides a landslide detection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the landslide detection method of the above-mentioned embodiment 1.

[0170] Reference below Figure 6 , which shows a schematic diagram of the structure of a landslide detection device suitable for implementing embodiments of the present application. The landslide detection device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, tablet computers, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6 The landslide detection device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0171] like Figure 6As shown, the landslide detection device may include a processing device 1001 (e.g., a central processing unit, graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the landslide detection device. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to I / O interface 1006: input devices 1007, such as a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008, such as a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 1003, such as a magnetic tape or hard disk; and communication devices 1009. The communication device 1009 can allow the landslide detection device to communicate with other devices wirelessly or by wire to exchange data. Although the figures show a landslide detection device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or provided instead.

[0172] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0173] The landslide detection device provided in this application utilizes the landslide detection method described in the aforementioned embodiment, resolving the technical issue of low landslide detection reliability. Compared to the prior art, the landslide detection device provided in this application achieves the same beneficial effects as the landslide detection method described in the aforementioned embodiment. Other technical features of the landslide detection device are the same as those disclosed in the aforementioned embodiment and are not further detailed here.

[0174] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0175] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0176] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, and the computer-readable program instructions are used to execute the landslide detection method in the above embodiment.

[0177] The computer-readable storage medium provided herein may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including, but not limited to, wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0178] The computer-readable storage medium may be included in the landslide detection device, or may exist independently without being assembled into the landslide detection device.

[0179] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the landslide detection device, the landslide detection device is enabled to execute the landslide detection method.

[0180] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0181] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to the various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the prescribed logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0182] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0183] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned landslide detection method. This computer-readable storage medium can address the technical issue of low landslide detection reliability. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are similar to those of the landslide detection method provided in the aforementioned embodiments and are not further elaborated here.

[0184] The present application also provides a computer program product, comprising a computer program, which implements the steps of the above-mentioned landslide detection method when executed by a processor.

[0185] The computer program product provided in this application can solve the technical problem of low reliability of landslide detection. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the landslide detection method provided in the above embodiment, which will not be repeated here.

[0186] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A landslide detection method, characterized in that: The method includes: Obtaining the current frame image of the slope area on the side of the road; Separating the dynamic target and the static target in the current frame image by a preset background separation model to obtain a foreground image containing the target with motion attributes; Calculating the foreground image based on a preset target recognition algorithm to obtain a candidate moving target; The preset target classification model is used to distinguish natural interference objects from potential landslide targets, and the preset target classification model includes a first target classification sub-model and a second target classification sub-model; Classifying the candidate moving targets by using a first target classification sub-model, screening interference targets among the candidate moving targets, and obtaining candidate landslide targets; Determining whether the area of ​​the candidate landslide target is greater than or equal to a preset area threshold; If the area of ​​the candidate landslide target is smaller than a preset area threshold, it is determined that the candidate landslide target does not have the possibility of landslide; If so, the motion features of the candidate landslide target are identified by a second target classification sub-model, wherein the motion features include texture changes, shape changes, and displacement patterns, and the motion features are distributed at different levels of abstraction. The second target classification sub-model includes a ResNet101 V2 network. If the motion feature is a landslide feature, determining the candidate landslide target as a potential landslide target; Tracking the potential landslide target through a preset target tracking model to obtain a motion trajectory of the potential landslide target; Based on the motion trajectory, determining whether there is a landslide sign in the slope area on the side of the road; The step of determining whether there is a sign of landslide in the slope area on the side of the road based on the motion trajectory includes: If the movement trajectory is downward movement, and the area change of the potential landslide target is increasing over time, it is determined that there are signs of landslide in the slope area on the side of the road.

2. The method according to claim 1, wherein After the step of determining that the candidate landslide target does not have the possibility of landslide if the area of ​​the candidate landslide target is smaller than the preset area threshold, the method further includes: If not, determining whether the number of candidate landslide targets with an area smaller than a preset area threshold is greater than a preset number threshold; If it is greater than, the second target classification sub-model is used to determine whether the area with the largest number of candidate landslide targets whose areas are smaller than the preset area threshold has landslide characteristics.

3. The method according to claim 1, wherein The preset target recognition algorithm includes a multi-frame difference method and a dense optical flow algorithm. The step of calculating the foreground image based on the preset target recognition algorithm to obtain a candidate moving target includes: comparing differences between foreground images including a current frame and a first preset frame preceding the current frame based on a multi-frame difference method, and determining an initial motion region based on the differences; Calculating the optimal motion vector for each pixel in the initial motion area based on a dense optical flow algorithm; A candidate moving target is determined based on the optimal motion vector, a preset motion speed threshold, and a preset motion direction.

4. The method according to claim 1, wherein The step of tracking the trajectory of the potential landslide target by using a preset target tracking model to obtain the motion trajectory of the potential landslide target includes: Tracking the potential landslide target in the foreground image including the current frame and the second preset frame before the current frame by using a preset target tracking model to obtain the motion trajectory of the potential landslide target; After the step of determining whether there is a landslide sign in the slope area on the side of the road based on the motion trajectory, the method further includes: If there are signs of landslide, an early warning is issued based on the movement trajectory.

5. The method according to claim 1, wherein The step of separating the dynamic targets and the static targets in the current frame image by using a preset background separation model to obtain a foreground image containing targets with motion attributes includes: Separating dynamic objects and static objects in the current frame and the third consecutive preset frame image thereof by a preset background separation model to obtain a foreground image containing objects with motion attributes; After the step of separating the dynamic objects and the static objects in the current frame image by using a preset background separation model to obtain a foreground image containing objects with motion attributes, the method further includes: The model parameters of the preset background separation model are updated to separate the dynamic objects and static objects in the next frame image and the previous preset frame image based on the updated background separation model.

6. A landslide detection device, characterized in that: The device comprises: An acquisition module is used to acquire a current frame image of a slope area on the side of the road; A detection module, the detection module comprising: A separation processing submodule is used to separate dynamic targets and static targets in the current frame image by using a preset background separation model to obtain a foreground image containing targets with motion attributes; A calculation submodule, configured to calculate the foreground image based on a preset target recognition algorithm to obtain a candidate moving target; The preset target classification model is used to distinguish natural interference objects from potential landslide targets, and the preset target classification model includes a first target classification sub-model and a second target classification sub-model; A classification submodule, the classification submodule comprising: a classification unit, configured to classify the candidate moving targets by using a first target classification sub-model, filter interference targets among the candidate moving targets, and obtain candidate landslide targets; A first judgment unit, wherein the first judgment unit includes: A judging subunit, configured to judge whether the area of ​​the candidate landslide target is greater than or equal to a preset area threshold; a determination subunit, configured to determine that the candidate landslide target does not have the possibility of landslide if the area of ​​the candidate landslide target is smaller than a preset area threshold; a second judgment unit, configured to, if present, identify motion features of the candidate landslide target through a second target classification sub-model, wherein the motion features include texture changes, shape changes, and displacement patterns, and the motion features are distributed at different levels of abstraction, and the second target classification sub-model includes a ResNet101 V2 network; a first determining unit, configured to determine the candidate landslide target as a potential landslide target if the motion feature is a landslide feature; A tracking module, configured to track the potential landslide target using a preset target tracking model to obtain a motion trajectory of the potential landslide target; a judgment module, configured to judge whether there is a sign of landslide in the slope area on the side of the road based on the motion trajectory; The judgment module includes: The determination submodule is configured to determine that there are signs of landslide in the slope area on the side of the road if the movement trajectory is downward movement and the area change of the potential landslide target is increasing over time.

7. A landslide detection device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the landslide detection method according to any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the landslide detection method according to any one of claims 1 to 5 are implemented.

Images