Disaster Monitoring Method and Device, Storage Medium, and Electronic Device for Road Slopes Based on Deep Learning

Abstract

The present invention provides a method and device for disaster monitoring of road slopes based on deep learning, a storage medium, and an electronic device. The method includes: obtaining a two-dimensional image of a target road slope area at the current time; converting the two-dimensional image into a current depth image by using a pre-trained deep learning model; obtaining a historical depth image of the target road slope area at a historical time; locating a landslide area in the two-dimensional image according to the current depth image and the historical depth image; calculating topographic parameters of the landslide area, and calculating a risk value of a geological disaster occurring in the landslide area according to the topographic parameters. It solves the technical problem of low accuracy in predicting geological disasters occurring on road slopes in the related art, improves the accuracy and robustness of the depth estimation algorithm in road slope detection, and improves road safety.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to a method and device for disaster monitoring of road slopes based on deep learning, a storage medium, and an electronic device. Background Art

[0002] Building an accurate and efficient road slope monitoring model and providing real-time slope safety assessment and early warning services are crucial for enhancing the resilience of road infrastructure and reducing disaster losses.

[0003] In related technologies, road slope monitoring methods mainly rely on manual surveys or ground radar measurements. These methods have problems such as high cost, limited coverage, and insufficient real-time performance, and it is difficult to meet the long-term monitoring needs of large-scale road slopes. In recent years, the rapid development of computer vision technology has provided new technical means for road slope monitoring. Using depth estimation algorithms, it is possible to reconstruct three-dimensional terrain information based on monocular or multiocular visual images, thereby achieving non-contact slope detection. However, existing depth estimation algorithms have certain limitations in slope detection applications, such as being greatly affected by light changes and having insufficient adaptability to the terrain of road slopes. Therefore, how to improve the accuracy and robustness of depth estimation algorithms in road slope detection has become a key issue in current research.

[0004] In view of the above problems existing in related technologies, no efficient and accurate solution has been found yet. Summary of the Invention

[0005] The present invention provides a method and device for disaster monitoring of road slopes based on deep learning, a storage medium, and an electronic device to solve the technical problems in related technologies.

[0006] According to an embodiment of the present invention, a method for disaster monitoring of a road slope based on deep learning is provided, including: obtaining a two-dimensional image of a target road slope area at the current time; converting the two-dimensional image into a current depth image by using a pre-trained deep learning model; obtaining a historical depth image of the target road slope area at a historical time; locating a landslide area in the two-dimensional image according to the current depth image and the historical depth image; calculating topographic parameters of the landslide area, and calculating a risk value of a geological disaster occurring in the landslide area according to the topographic parameters.

[0007] Optionally, converting the two-dimensional image into the current depth image using a pre-trained deep learning model includes: extracting the spatial information of the two-dimensional image respectively at multiple network layers of the pre-trained deep learning model to obtain multi-scale features, where each network layer corresponds to a feature extraction scale, and the multi-scale features include local texture features and global semantic features; predicting the current depth image of the two-dimensional image through the multi-scale features in the deep learning model.

[0008] Optionally, before converting the two-dimensional image into the current depth image using a pre-trained deep learning model, the method further includes: constructing the following spatial structure loss function :

[0009] , where is the predicted depth value, is the true depth value, represents the gradient, is the structural similarity loss, , , is the weighting coefficient, is the total number of pixels; constructing the following self-supervised loss function : ; where is the sample depth image, is the predicted depth image generated based on the sample two-dimensional image corresponding to the sample depth image; constructing a depth estimation optimization loss function using the following formula , where is the hyperparameter corresponding to the weight; self-supervised training the initial model using the depth estimation optimization loss function to obtain the deep learning model.

[0010] Optionally, locating the landslide area in the two-dimensional image according to the current depth image and the historical depth image includes: for each pixel point in the current depth image, calculating a first depth change value between the current depth value of the pixel point and the historical depth value of the corresponding pixel point in the historical depth image; determining whether the depth change value is greater than a preset safety threshold; if the depth change value is greater than the preset safety threshold, obtaining the neighborhood pixels of the pixel point; calculating a second depth change value between the current depth value of the neighborhood pixels and the historical depth value of the corresponding pixel point in the historical depth image; determining whether the second depth change value is greater than the preset safety threshold; if the second depth change value is greater than the preset safety threshold, clustering the pixel point and the neighborhood pixels into the landslide area in the two-dimensional image.

[0011] Optionally, calculating the terrain parameters of the landslide area and calculating the risk value of geological disasters occurring in the landslide area based on the terrain parameters includes: calculating the area, maximum depth change amount, and terrain slope of the landslide area; calculating the risk value of geological disasters occurring in the landslide area based on the area, the maximum depth change amount, and the terrain slope.

[0012] Optionally, calculating the risk value of geological disasters occurring in the landslide area based on the area, the maximum depth change amount, and the terrain slope includes: calculating the risk value R of geological disasters occurring in the landslide area using the following formula: ; where is the disaster risk score, is the area of the area, is the maximum depth change amount, is the terrain slope, , , are the corresponding weight coefficients.

[0013] Optionally, after locating the landslide area in the two-dimensional image according to the current depth image and the historical depth image, the method further includes: extracting a three-dimensional point cloud set of the landslide area; performing point cloud reconstruction on the three-dimensional point cloud set using the Alpha Shape algorithm to generate multiple candidate boundaries of the landslide area; calculating the distances from each pixel point in the landslide area to the multiple candidate boundaries respectively, and selecting a target boundary with the minimum total distance from all pixel points in the landslide area to the boundary among the multiple candidate boundaries; updating the landslide area with the area enclosed by the target boundary.

[0014] According to another embodiment of the present invention, there is provided a disaster monitoring device for a road slope based on deep learning, including: a first acquisition module for acquiring a two-dimensional image of a target road slope area at the current time; a conversion module for converting the two-dimensional image into a current depth image using a pre-trained deep learning model; a second acquisition module for acquiring a historical depth image of the target road slope area at a historical time; a positioning module for locating a landslide area in the two-dimensional image according to the current depth image and the historical depth image; a calculation module for calculating the terrain parameters of the landslide area and calculating the risk value of geological disasters occurring in the landslide area based on the terrain parameters.

[0015] Optionally, the conversion module includes: an extraction unit configured to extract spatial information of the two-dimensional image at multiple network layers of a pre-trained deep learning model to obtain multi-scale features, where each network layer corresponds to a feature extraction scale, and the multi-scale features include local texture features and global semantic features; and a prediction unit configured to predict a current depth image of the two-dimensional image in the deep learning model through the multi-scale features.

[0016] Optionally, the apparatus further includes: a first construction module configured to construct the following spatial structure loss function before the conversion module converts the two-dimensional image into a current depth image by using a pre-trained deep learning model :

[0017] , where is the predicted depth value, is the true depth value, represents the gradient, is the structural similarity loss, , , is the weighting coefficient, is the total number of pixels; a second construction module configured to construct the following self-supervised loss function : ; where is the sample depth image, is the predicted depth image generated based on the sample two-dimensional image corresponding to the sample depth image; a third construction module configured to construct a depth estimation optimization loss function by using the following formula , where is the hyperparameter corresponding to the weight; a training module configured to self-supervisedly train the initial model by using the depth estimation optimization loss function to obtain the deep learning model.

[0018] Optionally, the positioning module includes: a first calculation unit configured to calculate a first depth change value between the current depth value of each pixel point in the current depth image and the historical depth value of the corresponding pixel point in the historical depth image; a first determination unit configured to determine whether the depth change value is greater than a preset safety threshold; an acquisition unit configured to, if the depth change value is greater than the preset safety threshold, acquire the neighborhood pixels of the pixel point; a second calculation unit configured to calculate a second depth change value between the current depth value of the neighborhood pixels and the historical depth value of the corresponding pixel point in the historical depth image; a second determination unit configured to determine whether the second depth change value is greater than the preset safety threshold; and a clustering unit configured to, if the second depth change value is greater than the preset safety threshold, cluster the pixel point and the neighborhood pixels into a landslide area in the two-dimensional image.

[0019] Optionally, the computing module includes: a first computing unit configured to compute the area of the landslide area, the maximum depth change amount, and the terrain slope; a second computing unit configured to compute the risk value of a geological disaster occurring in the landslide area based on the area, the maximum depth change amount, and the terrain slope.

[0020] Optionally, the second computing unit includes: a computing subunit configured to compute the risk value R of a geological disaster occurring in the landslide area by using the following formula: ; where is the disaster risk score, is the area of the area, is the maximum depth change amount, is the terrain slope, , , are the corresponding weight coefficients.

[0021] Optionally, the apparatus further includes: an extraction module configured to extract a three-dimensional point cloud set of the landslide area after the positioning module locates the landslide area in the two-dimensional image according to the current depth image and the historical depth image; a reconstruction module configured to perform point cloud reconstruction on the three-dimensional point cloud set by using the Alpha Shape algorithm to generate a plurality of candidate boundaries of the landslide area; a selection module configured to compute the distances from each pixel point in the landslide area to the plurality of candidate boundaries respectively, and select a target boundary with the minimum total distance from all pixel points in the landslide area to the boundary among the plurality of candidate boundaries; an update module configured to update the landslide area by using the area enclosed by the target boundary.

[0022] According to another aspect of the embodiments of the present application, there is also provided a storage medium, which includes a stored program that, when running, executes the above steps.

[0023] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory communicate with each other through the communication bus; where: the memory is used for storing a computer program; the processor is used for executing the steps in the above method by running the program stored on the memory.

[0024] According to yet another embodiment of the present invention, there is also provided a storage medium, where a computer program is stored in the storage medium, and where the computer program is configured to execute the steps in any one of the above apparatus embodiments when running.

[0025] According to the embodiments of the present invention, a two-dimensional image of a target road slope area at the current time is obtained; a pre-trained deep learning model is used to convert the two-dimensional image into a current depth image; a historical depth image of the target road slope area at a historical time is obtained; a landslide area in the two-dimensional image is located according to the current depth image and the historical depth image; topographic parameters of the landslide area are calculated, and a risk value of a geological disaster occurring in the landslide area is calculated according to the topographic parameters. By converting the two-dimensional image into a three-dimensional accurate image to locate the landslide area and calculating the risk value of a geological disaster occurring through the topographic parameters, the technical problem of low accuracy in predicting geological disasters on road slopes in the related art is solved, the accuracy and robustness of the depth estimation algorithm in road slope detection are improved, and road safety is enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of the present invention, and constitute a part of this application. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0027] Figure 1 is a block diagram of the hardware structure of a computer according to an embodiment of the present invention;

[0028] Figure 2 is a flowchart of a method for monitoring disasters of a road slope based on deep learning according to an embodiment of the present invention;

[0029] Figure 3 is a schematic diagram of a landslide area in a target road slope area according to an embodiment of the present invention;

[0030] Figure 4 is a flowchart of road slope monitoring based on a depth estimation algorithm according to an embodiment of the present invention;

[0031] Figure 5 is a block diagram of a device for monitoring disasters of a road slope based on deep learning according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] In order to enable those skilled in the art of the present technology to better understand the solutions of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application. It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other.

[0033] It should be noted that the terms "first", "second", etc. in the description, claims and above-mentioned drawings of this application are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0034] Embodiment 1

[0035] The method embodiment provided by the first embodiment of this application can be executed on a server, a security device, a fire warning device, a computer, a mobile phone, or a similar computing device. Taking running on a computer as an example, Figure 1 is a hardware structure block diagram of a computer according to an embodiment of this application. As Figure 1 shown, the computer may include one or more ( Figure 1 only one is shown in the figure) processors 102 (the processors 102 may include, but are not limited to, processing devices such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the above computer may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that, Figure 1 the structure shown is only schematic and does not limit the structure of the above computer. For example, the computer may further include more or fewer components than Figure 1 shown in the figure, or have a different configuration from Figure 1 shown in the figure.

[0036] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to a method for disaster monitoring of road slopes based on deep learning in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories may be connected to the computer through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0037] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by a communication provider of a computer. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, abbreviated as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0038] In this embodiment, a disaster monitoring method for road slopes based on deep learning is provided. Figure 2 It is a flowchart of a disaster monitoring method for road slopes based on deep learning according to an embodiment of the present invention, as Figure 2 shown, and the process includes the following steps:

[0039] Step S202, obtaining a two-dimensional image of the target road slope area at the current time;

[0040] Optionally, the two-dimensional image can be a visual image of the road slope obtained from scenarios such as drones and road monitoring systems, and can also be preprocessed such as denoising and color correction to obtain a two-dimensional image of the road slope. .

[0041] Step S204, converting the two-dimensional image into a current depth image by using a pre-trained deep learning model;

[0042] Using the deep neural network of the deep learning model to perform depth prediction on the input two-dimensional image to generate a depth image : ;

[0043] wherein, is the predicted depth image, are the parameters of the neural network model. By establishing a deep learning neural model, the depth information in the two-dimensional image of the road slope can be predicted, providing basic data for subsequent disaster detection.

[0044] Step S206, obtaining a historical depth image of the target road slope area at a historical time;

[0045] Obtain the two-dimensional image of the target road slope area according to a period and convert it into a depth image. The history can be the time of the previous period of the current time.

[0046] Step S208: Locate the landslide area in the two-dimensional image according to the current depth image and the historical depth image;

[0047] Step S210: Calculate the terrain parameters of the landslide area, and calculate the risk value of geological disasters occurring in the landslide area according to the terrain parameters.

[0048] Optionally, the terrain parameters include the area of the landslide area, the maximum depth change amount, and the terrain slope.

[0049] In addition to landslide disasters, geological disasters can also be disasters such as collapses, rockfalls, debris flows, and avalanches.

[0050] Through the above steps, a two-dimensional image of the target road slope area at the current time is obtained; the two-dimensional image is converted into a current depth image by using a pre-trained deep learning model; a historical depth image of the target road slope area at a historical time is obtained; the landslide area in the two-dimensional image is located according to the current depth image and the historical depth image; the terrain parameters of the landslide area are calculated, and the risk value of geological disasters occurring in the landslide area is calculated according to the terrain parameters. By converting the two-dimensional image into a three-dimensional precision image to locate the landslide area and calculating the risk value of geological disasters through terrain parameters, the technical problem of low accuracy in predicting geological disasters on road slopes in the related art is solved, the accuracy and robustness of the depth estimation algorithm in road slope detection are improved, and road safety is enhanced.

[0051] In an implementation manner of this embodiment, converting the two-dimensional image into the current depth image by using a pre-trained deep learning model includes: extracting the spatial information of the two-dimensional image respectively in multiple network layers of the pre-trained deep learning model to obtain multi-scale features, where each network layer corresponds to a feature extraction scale, and the multi-scale features include local texture features and global semantic features; predicting the current depth image of the two-dimensional image through the multi-scale features in the deep learning model.

[0052] The multi-scale features include the feature information of the two-dimensional image at multiple resolutions and multiple scales, such as local texture features, global semantic features, deep features, shallow features, etc.

[0053] To improve the accuracy of depth estimation, the neural network extracts multi-scale features, that is, spatial information at different levels, from the input two-dimensional image. This process can be expressed as , where represents the feature extraction network for the deep learning model, including multiple network layers, represents the feature map of the th layer, is the total number of layers of the neural network. Through multi-scale feature extraction, the deep neural network is used to extract different levels of spatial information in the two-dimensional image, combine the low-level texture details with the high-level global semantics, and achieve accurate prediction of the depth image.

[0054] Generate a high-precision depth image based on the depth estimation result, so that it can intuitively present the three-dimensional morphological information of the road slope. The depth value of the depth image is obtained by the depth estimation algorithm.

[0055]

[0056] Among them, and are the horizontal pixel coordinate and vertical pixel coordinate in the image respectively, is the pixel position at which the depth value is located, is the deep learning model, is the input two-dimensional image, are the parameters of the deep neural network. When the deep learning model is trained, by optimizing the depth estimation error is minimized.

[0057]

[0058] Among them, are the optimized network parameters. During the training process, the neural network of the deep learning model continuously adjusts by the gradient descent algorithm to minimize the depth estimation error and improve the depth estimation accuracy of the model for the road slope.

[0059] In this embodiment, before using the pre-trained deep learning model to convert the two-dimensional image into the current depth image, it further includes: constructing the following spatial structure loss function :

[0060] Among them, is the predicted depth value, is the true depth value, represents the gradient, is the structural similarity loss, , , are the weighting coefficients, is the total number of pixels; constructing the following self-supervised loss function : ; Among them, is the sample depth image, is the predicted depth image generated based on the sample two-dimensional image corresponding to the sample depth image; constructing the depth estimation optimization loss function using the following formula , where is the hyperparameter corresponding to the weight; the initial model is self-supervised trained using the depth estimation optimization loss function to obtain the deep learning model.

[0061] In this embodiment, when training the deep learning model, two loss functions are constructed, the spatial structure loss function is composed of depth error, gradient consistency and structural similarity loss. By optimizing the loss function, the depth estimation error can be reduced. Using the self-supervised learning method, the model is optimized through disparity consistency, so that the model can still effectively learn depth information without real data. The self-supervised loss calculates the error between the original image and the image generated by depth estimation. The final depth estimation optimization loss function , where is the hyperparameter used to control the weights of the supervised loss and the self-supervised loss.

[0062] In an implementation manner of this embodiment, locating the landslide area in the two-dimensional image according to the current depth image and the historical depth image includes: for each pixel point in the current depth image, calculating a first depth change value between the current depth value of the pixel point and the historical depth value of the corresponding pixel point in the historical depth image; determining whether the depth change value is greater than a preset safety threshold; if the depth change value is greater than the preset safety threshold, obtaining the neighborhood pixels of the pixel point; calculating a second depth change value between the current depth value of the neighborhood pixels and the historical depth value of the corresponding pixel point in the historical depth image; determining whether the second depth change value is greater than the preset safety threshold; if the second depth change value is greater than the preset safety threshold, clustering the pixel point and the neighborhood pixels into the landslide area in the two-dimensional image.

[0063] Optionally, when the number of neighborhood pixels of any pixel point i in the current depth image is multiple, it is determined whether the proportion of the second depth change values greater than the preset safety threshold among the multiple neighborhood pixels exceeds a preset proportion (such as 1 / 4). If it exceeds the preset proportion, the pixel point and all corresponding neighborhood pixels are clustered into the landslide area in the two-dimensional image. If it does not exceed the preset proportion, or the second depth change value of a neighborhood pixel of pixel point i is less than or equal to the preset safety threshold, then the pixel point is deleted.

[0064] After traversing all pixel points in the current depth image, one or more landslide areas can be obtained.

[0065] This implementation manner uses the time series analysis method. By comparing the current depth image with the historical depth image, the depth change amount is calculated to detect geological disaster areas such as landslides and collapses.

[0066]

[0067] Among them, and are the depth values at the current moment and the previous moment respectively. The landslide area is defined as a set of connected pixels that meet the conditions .

[0068]

[0069] Among them, is the preset safety threshold for the set depth change, represents the neighborhood of pixel . If multiple pixel points in the neighborhood satisfy that the depth change is greater than the threshold, that is , where represents the number of pixel points in the neighborhood whose depth change is greater than the preset safety threshold, represents the total number of pixels in the neighborhood, then this area is clustered into a landslide area with potential geological hazard risks.

[0070] In an implementation scenario of this embodiment, after locating the landslide area in the two-dimensional image according to the current depth image and the historical depth image, it further includes: extracting the three-dimensional point cloud set of the landslide area; performing point cloud reconstruction on the three-dimensional point cloud set by using the Alpha Shape algorithm to generate multiple candidate boundaries of the landslide area; calculating the distances from each pixel point in the landslide area to the multiple candidate boundaries respectively, and selecting, from the multiple candidate boundaries, the target boundary with the minimum total distance from all pixel points in the landslide area to the boundary; updating the landslide area with the area enclosed by the target boundary.

[0071] The Alpha Shape algorithm of this embodiment is a computational geometry tool used to extract boundary and shape features from a three-dimensional point cloud set. By introducing an adjustable parameter α (Alpha value) to control the complexity and details of the shape, a geometric contour with topological consistency can be generated from the point set. By adjusting the size of the parameter α, multiple candidate boundaries can be generated for the same three-dimensional point cloud set.

[0072] In one example, the Alpha Shape algorithm is used to perform point cloud reconstruction on the three-dimensional point cloud set to generate multiple candidate boundaries of the landslide area, including: dividing the three-dimensional point cloud set into a triangle set, where no other points are included inside the circumcircle of each triangle in the triangle set; configuring multiple α values, and for each α value, determining whether the circumradius of each triangle in the triangle set is greater than α, filtering out the noise triangles in the triangle set whose circumradius is greater than α to obtain multiple groups of target triangle sets, the number of target triangle sets being the same as the number of α values; for each group of target triangle sets, extracting the non-overlapping triangular edges in the target triangle set to obtain the candidate boundary corresponding to the target triangle set.

[0073] Generate a three-dimensional point cloud for the landslide area , where is the point cloud data set of the three-dimensional point cloud set, is the detected landslide area, obtained by optimizing and extracting the pixel point set . Subsequently, boundary fitting is performed on the geological disaster area.

[0074]

[0075] Among them, is the candidate boundary of the geological disaster area such as landslide or collapse, obtained by extracting the boundary based on using the Alpha Shape algorithm. is the minimum distance from each pixel point in the landslide area to the boundary , represents finding the optimal target boundary such that the total distance from all points in the landslide area to the boundary is minimized, thereby obtaining the optimal boundary shape, which can reduce the error caused by model prediction, making the boundary of the updated final landslide area more consistent with the naturally formed terrain boundary, and thus can intuitively display the boundary and change trend of the disaster area.

[0076] In one example, calculating the terrain parameters of the landslide area and calculating the risk value of the geological disaster occurring in the landslide area based on the terrain parameters includes: calculating the area, maximum depth change amount, and terrain slope of the landslide area; calculating the risk value of the geological disaster occurring in the landslide area based on the area, the maximum depth change amount, and the terrain slope.

[0077] Optionally, calculating the risk value of the geological disaster occurring in the landslide area based on the area, the maximum depth change amount, and the terrain slope includes: calculating the risk value R of the geological disaster occurring in the landslide area using the following formula: ; among them, is the disaster risk score, is the area of the said region, is the maximum depth change amount, is the terrain slope, , , is the corresponding weight coefficient.

[0078] Among them, the maximum depth change amount is the difference between the maximum depth value and the minimum depth value in the landslide area.

[0079] According to the terrain parameters such as the area of the landslide area monitored by the intelligent monitoring algorithm, a scoring mechanism is used to evaluate the disaster risk level and quantify the risk value of the landslide area .

[0080]

[0081] Among them, is the risk value of the disaster risk score, is the area of the landslide area, is the maximum depth change amount of the landslide area, is the terrain slope of the landslide area, , , is the weight coefficient.

[0082] Optionally, the disaster risk level classification of the landslide area can also be carried out according to the obtained risk value of the disaster risk score .

[0083]

[0084] Among them, and are the risk level division thresholds. By providing timely geological disaster risk information, road safety can be ensured.

[0085] Figure 3 is a schematic diagram of the landslide area in the target road slope area in the embodiment of the present invention, including three landslide areas, and the corresponding risk levels are identified by different color depths.

[0086] In order to solve the problems of high cost, limited coverage, and insufficient real-time performance in the traditional method during the monitoring of road slopes, this embodiment proposes a road slope monitoring model based on the depth estimation algorithm. This model uses deep learning to generate high-precision depth images and combines change detection, object detection, and spatial clustering methods to realize the automatic identification and early warning of disasters such as landslides, collapses, and rockfalls. It includes three main parts: dataset preparation, depth image generation and disaster detection, and disaster early warning and visualization.

[0087] Figure 4 It is a flowchart of road slope monitoring based on depth estimation algorithm in an embodiment of the present invention, including: obtaining road slope monitoring images or UAV aerial images to construct a dataset; constructing a depth estimation algorithm based on deep learning; inputting data into the depth estimation algorithm to generate a high-precision depth image, comparing the current depth with historical depth images to detect geological disaster areas; calculating landslide risk scores, evaluating the severity of disasters, and displaying the disaster areas based on 3D visualization technology, and generating early warning information to be sent to the road management department.

[0088] The solution of this embodiment uses computer vision and deep learning technologies to generate high-precision depth images from two-dimensional images of road monitoring or UAV aerial photography, and combines intelligent detection algorithms to identify disasters such as landslides, collapses, and rockfalls, realizing automatic monitoring and early warning of road slopes. The depth estimation algorithm is used to generate high-precision depth images to accurately represent the morphology of road slopes. Combining time series analysis and spatial clustering methods, automatic identification of geological disaster areas such as landslides and collapses is realized. Real-time monitoring is carried out using 3D visualization technology, intuitively displaying slope changes, assisting road safety management, and improving the road disaster response ability.

[0089] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that makes contributions to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc), including several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0090] Embodiment 2

[0091] In this embodiment, a disaster monitoring device for road slopes based on deep learning is also provided. This device is used to implement the above embodiments and preferred implementation methods, and those that have been described will not be repeated. As used below, the term "module" can be a combination of software and hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementations in hardware, or combinations of software and hardware, can also be conceived.

[0092] Figure 5 It is a structural block diagram of a disaster monitoring device for road slopes based on deep learning in an embodiment of the present invention, as Figure 5 shown, including:

[0093] The first acquisition module 50 is configured to acquire a two-dimensional image of the target road slope area at the current time;

[0094] The conversion module 52 is configured to convert the two-dimensional image into a current depth image by using a pre-trained deep learning model;

[0095] The second acquisition module 54 is configured to acquire a historical depth image of the target road slope area at a historical time;

[0096] The positioning module 56 is configured to locate a landslide area in the two-dimensional image according to the current depth image and the historical depth image;

[0097] The calculation module 58 is configured to calculate topographic parameters of the landslide area, and calculate a risk value of a geological disaster occurring in the landslide area according to the topographic parameters.

[0098] Optionally, the conversion module includes: an extraction unit configured to extract spatial information of the two-dimensional image at multiple network layers of the pre-trained deep learning model to obtain multi-scale features, where each network layer corresponds to a feature extraction scale, and the multi-scale features include local texture features and global semantic features; a prediction unit configured to predict a current depth image of the two-dimensional image in the deep learning model by using the multi-scale features.

[0099] Optionally, the apparatus further includes: a first construction module configured to construct the following spatial structure loss function before the conversion module converts the two-dimensional image into a current depth image by using a pre-trained deep learning model :

[0100] , where is the predicted depth value, is the true depth value, represents the gradient, is the structural similarity loss, , , is the weighting coefficient, is the total number of pixels; a second construction module configured to construct the following self-supervised loss function : ; where is the sample depth image, is the predicted depth image generated based on the sample two-dimensional image corresponding to the sample depth image; a third construction module configured to construct a depth estimation optimization loss function by using the following formula , where is a hyperparameter corresponding to the weight; a training module for self-supervised training of an initial model using the depth estimation optimization loss function to obtain the deep learning model.

[0101] Optionally, the positioning module includes: a first calculation unit for calculating, for each pixel point in the current depth image, a first depth change value between the current depth value of the pixel point and the historical depth value of the corresponding pixel point in the historical depth image; a first judgment unit for judging whether the depth change value is greater than a preset safety threshold; an acquisition unit for, if the depth change value is greater than the preset safety threshold, acquiring the neighborhood pixels of the pixel point; a second calculation unit for calculating a second depth change value between the current depth value of the neighborhood pixels and the historical depth value of the corresponding pixel point in the historical depth image; a second judgment unit for judging whether the second depth change value is greater than the preset safety threshold; a clustering unit for, if the second depth change value is greater than the preset safety threshold, clustering the pixel point and the neighborhood pixels into a landslide area in the two-dimensional image.

[0102] Optionally, the calculation module includes: a first calculation unit for calculating the area, the maximum depth change amount, and the terrain slope of the landslide area; a second calculation unit for calculating a risk value of a geological disaster occurring in the landslide area according to the area, the maximum depth change amount, and the terrain slope.

[0103] Optionally, the second calculation unit includes: a calculation subunit for calculating a risk value R of a geological disaster occurring in the landslide area using the following formula: ; where is the disaster risk score, is the area of the region, is the maximum depth change amount, is the terrain slope, , , are the corresponding weight coefficients.

[0104] Optionally, the device further includes: an extraction module for extracting a three-dimensional point cloud set of the landslide area after the positioning module locates the landslide area in the two-dimensional image according to the current depth image and the historical depth image; a reconstruction module for performing point cloud reconstruction on the three-dimensional point cloud set using the Alpha Shape algorithm to generate multiple candidate boundaries of the landslide area; a selection module for calculating the distances from each pixel point in the landslide area to the multiple candidate boundaries respectively, and selecting a target boundary with the minimum total distance from all pixel points in the landslide area to the boundary among the multiple candidate boundaries; an update module for updating the landslide area with the area enclosed by the target boundary.

[0105] It should be noted that the above-mentioned various modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: all the above-mentioned modules are located in the same processor; or, the above-mentioned various modules are located in different processors in any combination form.

[0106] Embodiment 3

[0107] The embodiment of the present invention also provides a storage medium, in which a computer program is stored, and the computer program is set to execute the steps in any one of the above method embodiments when running.

[0108] Optionally, in this embodiment, the above storage medium can be set to store a computer program for execution:

[0109] S1, obtain a two-dimensional image of the target road slope area at the current time;

[0110] S2, use a pre-trained deep learning model to convert the two-dimensional image into a current depth image;

[0111] S3, obtain a historical depth image of the target road slope area at a historical time;

[0112] S4, locate the landslide area in the two-dimensional image according to the current depth image and the historical depth image;

[0113] S5, calculate the terrain parameters of the landslide area, and calculate the risk value of geological disasters occurring in the landslide area according to the terrain parameters.

[0114] Optionally, in this embodiment, the above storage medium may include but is not limited to: various media such as USB flash drives, read-only memories (ROM for short), random access memories (RAM for short), mobile hard disks, magnetic disks or optical discs that can store computer programs.

[0115] The embodiment of the present invention also provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is set to run the computer program to execute the steps in any one of the above method embodiments.

[0116] Optionally, the above electronic device may further include a transmission device and an input / output device, where the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0117] Optionally, in this embodiment, the above processor can be set to execute the following steps through a computer program:

[0118] S1, Obtain a two-dimensional image of the target road slope area at the current time;

[0119] S2, Use a pre-trained deep learning model to convert the two-dimensional image into a current depth image;

[0120] S3, Obtain a historical depth image of the target road slope area at a historical time;

[0121] S4, Locate the landslide area in the two-dimensional image according to the current depth image and the historical depth image;

[0122] S5, Calculate the terrain parameters of the landslide area, and calculate the risk value of geological disasters occurring in the landslide area according to the terrain parameters.

[0123] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and will not be elaborated herein.

[0124] The serial numbers of the above embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0125] In the above embodiments of the present application, the descriptions of the respective embodiments have their own emphases. For parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0126] In several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, and the indirect coupling or communication connection of units or modules may be in an electrical or other form.

[0127] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0128] In addition, each functional unit in various embodiments of the present application may be integrated into one processing unit, may exist physically alone for each unit, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0129] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a controller, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs that can store program codes.

[0130] The above are only the preferred embodiments of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A road slope disaster monitoring method based on deep learning, characterized in that: include: Obtain a two-dimensional image of the target road slope area at the current time; Converting the two-dimensional image into a current depth image using a pre-trained deep learning model; Acquire a historical depth image of the target road slope area at a historical time; Locating a landslide area in the two-dimensional image according to the current depth image and the historical depth image; Calculating the terrain parameters of the landslide area, and calculating the risk value of geological disasters occurring in the landslide area according to the terrain parameters; Before converting the two-dimensional image into the current depth image using the pre-trained deep learning model, the method further includes: constructing the following spatial structure loss function: : ,in, is the predicted depth value, is the true depth value, represents the gradient, is the structural similarity loss, , , is the weighting coefficient, is the total number of pixels; construct the following self-supervised loss function : ;in, is the sample depth image, The predicted depth image is generated based on the sample two-dimensional image corresponding to the sample depth image; the depth estimation optimization loss function is constructed using the following formula ,in is a hyperparameter corresponding to the weight; the initial model is trained by self-supervision using the depth estimation optimization loss function to obtain the deep learning model; Among them, locating the landslide area in the two-dimensional image according to the current depth image and the historical depth image includes: for each pixel point in the current depth image, calculating a first depth change value between the current depth value of the pixel point and the historical depth value of the corresponding pixel point in the historical depth image; judging whether the depth change value is greater than a preset safety threshold; if the depth change value is greater than the preset safety threshold, obtaining the neighborhood pixels of the pixel point; calculating a second depth change value between the current depth value of the neighborhood pixel and the historical depth value of the corresponding pixel point in the historical depth image; judging whether the second depth change value is greater than the preset safety threshold; if the second depth change value is greater than the preset safety threshold, clustering the pixel point and the neighborhood pixels into a landslide area in the two-dimensional image.

2. The method according to claim 1, characterized in that Using a pre-trained deep learning model to convert the two-dimensional image into a current depth image includes: Extracting spatial information of the two-dimensional image in multiple network layers of a pre-trained deep learning model to obtain multi-scale features, wherein each network layer corresponds to a feature extraction scale, and the multi-scale features include local texture features and global semantic features; A current depth image of the two-dimensional image is predicted in the deep learning model using the multi-scale features.

3. The method according to claim 1, characterized in that Calculating the terrain parameters of the landslide area, and calculating the risk value of geological disasters occurring in the landslide area according to the terrain parameters includes: Calculating the area, maximum depth change, and terrain slope of the landslide area; The risk value of geological disasters occurring in the landslide area is calculated according to the area of ​​the area, the maximum depth change, and the terrain slope.

4. The method according to claim 3, characterized in that Calculating the risk value of geological disasters occurring in the landslide area according to the area, the maximum depth change, and the terrain slope includes: The risk value R of geological disasters in the landslide area is calculated using the following formula: ； in, is the disaster risk score, is the area of ​​the region, is the maximum depth variation, is the terrain slope, , , is the corresponding weight coefficient.

5. The method according to claim 1, characterized in that After locating the landslide area in the two-dimensional image according to the current depth image and the historical depth image, the method further includes: Extracting a three-dimensional point cloud set of the landslide area; Reconstructing the three-dimensional point cloud set using an Alpha Shape algorithm to generate multiple candidate boundaries of the landslide area; Calculating the distances from each pixel point in the landslide area to the multiple boundaries to be selected, and selecting a target boundary with the minimum total distance from all pixels in the landslide area to the boundaries from the multiple boundaries to be selected; The landslide area is updated using the area enclosed by the target boundary.

6. A road slope disaster monitoring device based on deep learning, characterized in that: include: A first acquisition module is used to acquire a two-dimensional image of the target road slope area at the current time; A conversion module, configured to convert the two-dimensional image into a current depth image using a pre-trained deep learning model; A second acquisition module is used to acquire a historical depth image of the target road slope area at a historical time; A positioning module, used for locating the landslide area in the two-dimensional image according to the current depth image and the historical depth image; A calculation module, used to calculate the terrain parameters of the landslide area, and calculate the risk value of geological disasters occurring in the landslide area according to the terrain parameters; The device further includes: a first construction module for constructing the following spatial structure loss function before the conversion module uses the pre-trained deep learning model to convert the two-dimensional image into the current depth image: : ,in, is the predicted depth value, is the true depth value, represents the gradient, is the structural similarity loss, , , is the weighting coefficient, is the total number of pixels; the second building block is used to construct the following self-supervised loss function : ;in, is the sample depth image, is a predicted depth image generated based on the sample two-dimensional image corresponding to the sample depth image; a third construction module is used to construct a depth estimation optimization loss function using the following formula ,in is a hyperparameter corresponding to the weight; a training module, used to self-supervise the training of the initial model using the depth estimation optimization loss function to obtain the deep learning model; Among them, the positioning module includes: a first calculation unit, used to calculate, for each pixel point in the current depth image, a first depth change value between the current depth value of the pixel point and the historical depth value of the corresponding pixel point in the historical depth image; a first judgment unit, used to judge whether the depth change value is greater than a preset safety threshold; an acquisition unit, used to acquire the neighborhood pixels of the pixel point if the depth change value is greater than the preset safety threshold; a second calculation unit, used to calculate the second depth change value between the current depth value of the neighborhood pixel and the historical depth value of the corresponding pixel point in the historical depth image; a second judgment unit, used to judge whether the second depth change value is greater than the preset safety threshold; a clustering unit, used to cluster the pixel point and the neighborhood pixels into a landslide area in the two-dimensional image if the second depth change value is greater than the preset safety threshold.

7. A storage medium, characterized in that: A computer program is stored in the storage medium, wherein the computer program is configured to execute the steps of the road slope disaster monitoring method based on deep learning in any one of claims 1 to 5 when running.

8. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; wherein: Memory, used to store computer programs; A processor, configured to execute the steps of the road slope disaster monitoring method based on deep learning according to any one of claims 1 to 5 by running a program stored in a memory.

Images