Deep learning-based road slope disaster monitoring method and device, storage medium and electronic equipment

Abstract

The invention provides a deep learning-based road slope disaster monitoring method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining a two-dimensional image of a target road slope region at the current time; converting the two-dimensional image into a current depth image by adopting a pre-trained deep learning model; acquiring a historical depth image of the target road slope area in historical time; positioning a landslide area in the two-dimensional image according to the current depth image and the historical depth image; and calculating topographic parameters of the landslide area, and calculating a risk value of geological disasters in the landslide area according to the topographic parameters. According to the invention, the technical problem of low precision of predicting the geological disaster of the road slope in the prior art is solved, the precision and robustness of the depth estimation algorithm in the road slope detection are improved, and the road safety is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a road slope disaster monitoring method and device based on deep learning, a storage medium, and an electronic device. Background Art

[0002] Building an accurate and efficient road slope monitoring model and providing real-time slope safety assessment and early warning services are crucial to improving the resilience of road infrastructure and reducing disaster losses.

[0003] In the related technologies, road slope monitoring methods mainly rely on manual surveys or ground radar measurements. These methods have problems such as high cost, limited coverage, and lack of real-time performance, and are difficult to meet the long-term monitoring needs of large-scale road slopes. In recent years, the rapid development of computer vision technology has provided new technical means for road slope monitoring. Using depth estimation algorithms, three-dimensional terrain information can be reconstructed based on monocular or multi-eye visual images, thereby realizing non-contact slope detection. However, existing depth estimation algorithms have certain limitations in slope detection applications, such as being greatly affected by changes in illumination and insufficient adaptability to road slope terrain. Therefore, how to improve the accuracy and robustness of depth estimation algorithms in road slope detection has become a key issue in current research.

[0004] With respect to the above-mentioned problems existing in the related technologies, no efficient and accurate solutions have been found yet. Summary of the invention

[0005] The present invention provides a road slope disaster monitoring method and device based on deep learning, a storage medium, and an electronic device to solve technical problems in related technologies.

[0006] According to one embodiment of the present invention, a road slope disaster monitoring method based on deep learning is provided, comprising: obtaining a two-dimensional image of a target road slope area at a current time; converting the two-dimensional image into a current depth image using a pre-trained deep learning model; obtaining a historical depth image of the target road slope area at a historical time; locating a landslide area in the two-dimensional image according to the current depth image and the historical depth image; calculating terrain parameters of the landslide area, and calculating a risk value of a geological disaster occurring in the landslide area according to the terrain parameters.

[0007] Optionally, using a pre-trained deep learning model to convert the two-dimensional image into a current depth image includes: extracting spatial information of the two-dimensional image in multiple network layers of the pre-trained deep learning model to obtain multi-scale features, wherein each network layer corresponds to a feature extraction scale, and the multi-scale features include local texture features and global semantic features; and predicting the current depth image of the two-dimensional image through the multi-scale features in the deep learning model.

[0008] Optionally, before converting the two-dimensional image into the current depth image using the pre-trained deep learning model, the method further includes: constructing the following spatial structure loss function : ,

[0009] in, is the predicted depth value, is the true depth value, represents the gradient, is the structural similarity loss, , , is the weighting coefficient, is the total number of pixels; construct the following self-supervised loss function : ;in, is the sample depth image, The predicted depth image is generated based on the sample two-dimensional image corresponding to the sample depth image; the depth estimation optimization loss function is constructed using the following formula ,in is the hyperparameter corresponding to the weight; the depth estimation optimization loss function is used to self-supervise the training of the initial model to obtain the deep learning model.

[0010] Optionally, locating the landslide area in the two-dimensional image according to the current depth image and the historical depth image includes: for each pixel point in the current depth image, calculating a first depth change value between a current depth value of the pixel point and a historical depth value of a corresponding pixel point in the historical depth image; judging whether the depth change value is greater than a preset safety threshold; if the depth change value is greater than the preset safety threshold, obtaining neighborhood pixels of the pixel point; calculating a second depth change value between a current depth value of the neighborhood pixel and a historical depth value of a corresponding pixel point in the historical depth image; judging whether the second depth change value is greater than the preset safety threshold; if the second depth change value is greater than the preset safety threshold, clustering the pixel point and the neighborhood pixels into a landslide area in the two-dimensional image.

[0011] Optionally, calculating the terrain parameters of the landslide area, and calculating the risk value of geological disasters occurring in the landslide area based on the terrain parameters includes: calculating the regional area, maximum depth change, and terrain slope of the landslide area; calculating the risk value of geological disasters occurring in the landslide area based on the regional area, the maximum depth change, and the terrain slope.

[0012] Optionally, calculating the risk value of geological disasters occurring in the landslide area according to the area, the maximum depth change, and the terrain slope includes: calculating the risk value R of geological disasters occurring in the landslide area using the following formula: ;in, is the disaster risk score, is the area of ​​the region, is the maximum depth variation, is the terrain slope, , , is the corresponding weight coefficient.

[0013] Optionally, after locating the landslide area in the two-dimensional image according to the current depth image and the historical depth image, the method also includes: extracting a three-dimensional point cloud set of the landslide area; reconstructing the three-dimensional point cloud set using the Alpha Shape algorithm to generate multiple candidate boundaries of the landslide area; calculating the distance from each pixel point in the landslide area to the multiple candidate boundaries, and selecting a target boundary from the multiple candidate boundaries with the smallest total distance from all pixels in the landslide area to the boundary; and updating the landslide area using the area enclosed by the target boundary.

[0014] According to another embodiment of the present invention, a road slope disaster monitoring device based on deep learning is provided, including: a first acquisition module, used to acquire a two-dimensional image of a target road slope area at a current time; a conversion module, used to convert the two-dimensional image into a current depth image using a pre-trained deep learning model; a second acquisition module, used to acquire a historical depth image of the target road slope area at a historical time; a positioning module, used to locate a landslide area in the two-dimensional image according to the current depth image and the historical depth image; a calculation module, used to calculate the terrain parameters of the landslide area, and calculate the risk value of geological disasters occurring in the landslide area according to the terrain parameters.

[0015] Optionally, the conversion module includes: an extraction unit, used to extract the spatial information of the two-dimensional image in multiple network layers of a pre-trained deep learning model to obtain multi-scale features, wherein each network layer corresponds to a feature extraction scale, and the multi-scale features include local texture features and global semantic features; a prediction unit, used to predict the current depth image of the two-dimensional image through the multi-scale features in the deep learning model.

[0016] Optionally, the device further includes: a first construction module, configured to construct the following spatial structure loss function before the conversion module uses a pre-trained deep learning model to convert the two-dimensional image into a current depth image: :

[0017] ,in, is the predicted depth value, is the true depth value, represents the gradient, is the structural similarity loss, , , is the weighting coefficient, is the total number of pixels; the second building block is used to construct the following self-supervised loss function : ;in, is the sample depth image, is a predicted depth image generated based on the sample two-dimensional image corresponding to the sample depth image; a third construction module is used to construct a depth estimation optimization loss function using the following formula ,in is a hyperparameter corresponding to the weight; a training module is used to self-supervise the training of the initial model using the depth estimation optimization loss function to obtain the deep learning model.

[0018] Optionally, the positioning module includes: a first calculation unit, used to calculate, for each pixel point in the current depth image, a first depth change value between the current depth value of the pixel point and the historical depth value of the corresponding pixel point in the historical depth image; a first judgment unit, used to judge whether the depth change value is greater than a preset safety threshold; an acquisition unit, used to acquire the neighborhood pixels of the pixel point if the depth change value is greater than the preset safety threshold; a second calculation unit, used to calculate the second depth change value between the current depth value of the neighborhood pixel and the historical depth value of the corresponding pixel point in the historical depth image; a second judgment unit, used to judge whether the second depth change value is greater than the preset safety threshold; a clustering unit, used to cluster the pixel point and the neighborhood pixels into a landslide area in the two-dimensional image if the second depth change value is greater than the preset safety threshold.

[0019] Optionally, the calculation module includes: a first calculation unit, used to calculate the regional area, maximum depth change, and terrain slope of the landslide area; a second calculation unit, used to calculate the risk value of geological disasters occurring in the landslide area based on the regional area, the maximum depth change, and the terrain slope.

[0020] Optionally, the second calculation unit includes: a calculation subunit, configured to calculate the risk value R of geological disasters occurring in the landslide area by using the following formula: ;in, is the disaster risk score, is the area of ​​the region, is the maximum depth variation, is the terrain slope, , , is the corresponding weight coefficient.

[0021] Optionally, the device also includes: an extraction module, used to extract a three-dimensional point cloud set of the landslide area after the positioning module locates the landslide area in the two-dimensional image according to the current depth image and the historical depth image; a reconstruction module, used to reconstruct the three-dimensional point cloud set using the Alpha Shape algorithm to generate multiple candidate boundaries of the landslide area; a selection module, used to calculate the distance from each pixel point in the landslide area to the multiple candidate boundaries, and select a target boundary with the smallest total distance from all pixels in the landslide area to the boundary from the multiple candidate boundaries; an update module, used to update the landslide area using the area enclosed by the target boundary.

[0022] According to another aspect of an embodiment of the present application, a storage medium is further provided, which includes a stored program, and the above steps are executed when the program is run.

[0023] According to another aspect of an embodiment of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; wherein: the memory is used to store computer programs; and the processor is used to execute the steps in the above method by running the program stored in the memory.

[0024] According to yet another embodiment of the present invention, a storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned device embodiments when running.

[0025] Through the embodiments of the present invention, a two-dimensional image of a target road slope area at a current time is obtained; a pre-trained deep learning model is used to convert the two-dimensional image into a current depth image; a historical depth image of the target road slope area at a historical time is obtained; a landslide area in the two-dimensional image is located according to the current depth image and the historical depth image; terrain parameters of the landslide area are calculated, and a risk value of a geological disaster occurring in the landslide area is calculated according to the terrain parameters. The landslide area is located by converting the two-dimensional image into a three-dimensional precision image, and the risk value of a geological disaster occurring is calculated according to the terrain parameters. The technical problem of low accuracy in predicting the occurrence of geological disasters on road slopes in the related art is solved, the accuracy and robustness of the depth estimation algorithm in road slope detection are improved, and road safety is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 is a hardware structure block diagram of a computer according to an embodiment of the present invention; Figure 2 is a flow chart of a road slope disaster monitoring method based on deep learning according to an embodiment of the present invention; Figure 3 is a schematic diagram of a landslide area in a target road slope area in an embodiment of the present invention; Figure 4 is a flow chart of road slope monitoring based on a depth estimation algorithm in an embodiment of the present invention; Figure 5 It is a structural block diagram of a road slope disaster monitoring device based on deep learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] Example 1 The method embodiment provided in the first embodiment of the present application can be executed in a server, security equipment, fire warning equipment, computer, mobile phone, or similar computing device. Taking running on a computer as an example, Figure 1 is a hardware structure block diagram of a computer in an embodiment of the present application. Figure 1 As shown, the computer may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the computer may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0030] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a road slope disaster monitoring method based on deep learning in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0031] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0032] In this embodiment, a road slope disaster monitoring method based on deep learning is provided. Figure 2 is a flow chart of a road slope disaster monitoring method based on deep learning according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps: Step S202, obtaining a two-dimensional image of the target road slope area at the current time; Optionally, the two-dimensional image can be a visual image of the road slope obtained from a scene such as a drone or a road monitoring system, and pre-processing operations such as denoising and color correction can be performed to obtain a two-dimensional image of the road slope. .

[0033] Step S204, converting the two-dimensional image into a current depth image using a pre-trained deep learning model; Deep Neural Networks using Deep Learning Models For the input two-dimensional image Perform depth prediction and generate depth image : ; in, is the predicted depth image, is the parameter of the neural network model. By establishing a deep learning neural model, the depth information in the two-dimensional image of the road slope can be predicted, providing basic data for subsequent disaster detection.

[0034] Step S206, obtaining a historical depth image of the target road slope area at a historical time; A two-dimensional image of the target road slope area is obtained periodically and converted into a depth image, and the history can be the time of a cycle before the current time.

[0035] Step S208, locating the landslide area in the two-dimensional image according to the current depth image and the historical depth image; Step S210, calculating the terrain parameters of the landslide area, and calculating the risk value of geological disasters occurring in the landslide area according to the terrain parameters.

[0036] Optionally, the terrain parameters include the area of ​​the landslide region, the maximum depth change, and the terrain slope.

[0037] In addition to landslides, geological disasters may also include collapse, rockfall, mud-rock flow, avalanche and other disasters.

[0038] Through the above steps, a two-dimensional image of the target road slope area at the current time is obtained; the pre-trained deep learning model is used to convert the two-dimensional image into a current depth image; the historical depth image of the target road slope area at the historical time is obtained; the landslide area in the two-dimensional image is located according to the current depth image and the historical depth image; the terrain parameters of the landslide area are calculated, and the risk value of geological disasters occurring in the landslide area is calculated according to the terrain parameters. The landslide area is located by converting the two-dimensional image into a three-dimensional precision image, and the risk value of geological disasters occurring is calculated according to the terrain parameters. The technical problem of low accuracy in predicting geological disasters occurring on road slopes in related technologies is solved, the accuracy and robustness of the depth estimation algorithm in road slope detection are improved, and road safety is improved.

[0039] In one implementation of the present embodiment, using a pre-trained deep learning model to convert the two-dimensional image into a current depth image includes: extracting spatial information of the two-dimensional image in multiple network layers of the pre-trained deep learning model to obtain multi-scale features, wherein each network layer corresponds to a feature extraction scale, and the multi-scale features include local texture features and global semantic features; and predicting the current depth image of the two-dimensional image using the multi-scale features in the deep learning model.

[0040] Multi-scale features include feature information of two-dimensional images at multiple resolutions and multiple scales, such as local texture features and global semantic features, deep features, shallow features, etc.

[0041] In order to improve the accuracy of depth estimation, the neural network will input a two-dimensional image Extract multi-scale features, that is, spatial information at different levels. This process can be expressed as ,in Represents a feature extraction network for deep learning models, including multiple network layers. Indicates The feature map of the layer, is the total number of layers of the neural network. Through multi-scale feature extraction, a deep neural network is used to extract spatial information at different levels in a two-dimensional image, combining low-level texture details with high-level global semantics to achieve accurate prediction of deep images.

[0042] Based on the depth estimation results, a high-precision depth image is generated, which can intuitively present the three-dimensional morphological information of the road slope. Obtained by the depth estimation algorithm.

[0043]

[0044] in, and are the horizontal pixel coordinates and vertical pixel coordinates in the image, respectively. is the pixel position The depth value at is a deep learning model, is the input 2D image, are the parameters of the deep neural network. The deep learning model is trained by optimizing Minimize the depth estimation error.

[0045]

[0046] in, is the optimized network parameter. During the training process, the neural network of the deep learning model is continuously adjusted through the gradient descent algorithm. , in order to minimize the depth estimation error and improve the model's depth estimation accuracy for road slopes.

[0047] In this embodiment, before using the pre-trained deep learning model to convert the two-dimensional image into the current depth image, it also includes: constructing the following spatial structure loss function :

[0048] ,in, is the predicted depth value, is the true depth value, represents the gradient, is the structural similarity loss, , , is the weighting coefficient, is the total number of pixels; construct the following self-supervised loss function : ;in, is the sample depth image, The predicted depth image is generated based on the sample two-dimensional image corresponding to the sample depth image; the depth estimation optimization loss function is constructed using the following formula ,in is the hyperparameter corresponding to the weight; the depth estimation optimization loss function is used to self-supervise the training of the initial model to obtain the deep learning model.

[0049] In this embodiment, when training a deep learning model, two loss functions are constructed: the spatial structure loss function It consists of depth error, gradient consistency and structural similarity loss. By optimizing the loss function, the depth estimation error can be reduced. The self-supervised learning method is used to optimize the model through parallax consistency, so that the model can still effectively learn depth information without real data. Calculate the error between the original image and the image generated by the depth estimation. The final depth estimation optimization loss function ,in is a hyperparameter that controls the weights of the supervised loss and the self-supervised loss.

[0050] In one implementation of the present embodiment, locating the landslide area in the two-dimensional image according to the current depth image and the historical depth image includes: for each pixel point in the current depth image, calculating a first depth change value between the current depth value of the pixel point and the historical depth value of the corresponding pixel point in the historical depth image; judging whether the depth change value is greater than a preset safety threshold; if the depth change value is greater than the preset safety threshold, obtaining the neighborhood pixels of the pixel point; calculating a second depth change value between the current depth value of the neighborhood pixel and the historical depth value of the corresponding pixel point in the historical depth image; judging whether the second depth change value is greater than the preset safety threshold; if the second depth change value is greater than the preset safety threshold, clustering the pixel point and the neighborhood pixels into a landslide area in the two-dimensional image.

[0051] Optionally, when the number of neighboring pixels of any pixel point i in the current depth image is multiple, it is determined whether the proportion of the second depth change values ​​of the multiple neighboring pixels that is greater than a preset safety threshold exceeds a preset proportion (such as 1 / 4); if it exceeds the preset proportion, the pixel point and all corresponding neighboring pixels are clustered as a landslide area in the two-dimensional image; if it does not exceed the preset proportion, or the second depth change value of a neighboring pixel of the pixel point i is less than or equal to the preset safety threshold, the pixel point is deleted.

[0052] After traversing all the pixel points in the current depth image, one or more landslide areas may be obtained.

[0053] This embodiment uses a time series analysis method to calculate the depth change by comparing the current depth image with the historical depth image. , detect geological disaster areas such as landslides and collapses.

[0054]

[0055] in, and are the depth values ​​of the current moment and the previous moment respectively. The landslide area is defined as a set of connected pixels that meet the conditions .

[0056]

[0057] in, is the preset safety threshold for depth change, Represents pixels If multiple pixels in the neighborhood satisfy the depth change greater than the threshold, that is ,in Indicates the number of points in the neighborhood whose depth change is greater than the preset safety threshold. represents the total number of pixels in the neighborhood, the area is clustered into a landslide area with potential risk of geological hazards.

[0058] In an implementation scenario of this embodiment, after locating the landslide area in the two-dimensional image according to the current depth image and the historical depth image, it also includes: extracting a three-dimensional point cloud set of the landslide area; using the Alpha Shape algorithm to reconstruct the three-dimensional point cloud set to generate multiple candidate boundaries of the landslide area; calculating the distance from each pixel point in the landslide area to the multiple candidate boundaries, and selecting a target boundary from the multiple candidate boundaries with the minimum total distance from all pixels in the landslide area to the boundary; and using the area enclosed by the target boundary to update the landslide area.

[0059] The Alpha Shape algorithm of this embodiment is a computational geometry tool for extracting boundaries and shape features from a three-dimensional point cloud set. It controls the complexity and details of the shape by introducing an adjustable parameter α (Alpha value), thereby generating a topologically consistent geometric contour from a point set. By adjusting the size of the parameter α, multiple candidate boundaries can be generated for the same three-dimensional point cloud set.

[0060] In one example, the Alpha Shape algorithm is used to reconstruct the three-dimensional point cloud set to generate multiple candidate boundaries of the landslide area, including: dividing the three-dimensional point cloud set into a triangle set, wherein the circumscribed circle of each triangle in the triangle set does not contain other points; configuring multiple α values, and for each α value, determining whether the circumscribed circle radius of each triangle in the triangle set is greater than α, filtering out noise triangles in the triangle set whose circumscribed circle radius is greater than α, and obtaining multiple groups of target triangle sets, the number of target triangle sets is the same as the number of α values; for each group of target triangle sets, extracting non-overlapping triangle edges in the target triangle set to obtain candidate boundaries corresponding to the target triangle set.

[0061] Generate 3D point cloud of landslide area ,in is a point cloud dataset of a three-dimensional point cloud collection, is the detected landslide area, which is composed of a set of pixel points After optimization and extraction, the boundary of the geological disaster area is fitted.

[0062]

[0063] in, It is the candidate boundary of the geological disaster area such as landslide or collapse, based on The boundary is extracted using the Alpha Shape algorithm. is each pixel in the landslide area To the border The minimum distance Represents finding the optimal target boundary , so that the total distance from all points in the landslide area to the boundary is minimized, thereby obtaining the optimal boundary shape, which can reduce the error caused by model prediction and make the boundary of the final updated landslide area more consistent with the naturally formed terrain boundary, thereby intuitively displaying the boundary and change trend of the disaster area.

[0064] In one example, calculating the terrain parameters of the landslide area, and calculating the risk value of geological disasters occurring in the landslide area based on the terrain parameters includes: calculating the regional area, the maximum depth change, and the terrain slope of the landslide area; calculating the risk value of geological disasters occurring in the landslide area based on the regional area, the maximum depth change, and the terrain slope.

[0065] Optionally, calculating the risk value of geological disasters occurring in the landslide area according to the area, the maximum depth change, and the terrain slope includes: calculating the risk value R of geological disasters occurring in the landslide area using the following formula: ;in, is the disaster risk score, is the area of ​​the region, is the maximum depth variation, is the terrain slope, , , is the corresponding weight coefficient.

[0066] The maximum depth change is the difference between the maximum depth value and the minimum depth value in the landslide area.

[0067] Based on the terrain parameters such as the area of ​​the landslide area monitored by the intelligent monitoring algorithm, a scoring mechanism is used to assess the disaster risk level and quantify the risk value of the landslide area. .

[0068]

[0069] in, is the risk value of the disaster risk score, is the area of ​​the landslide region, is the maximum depth change in the landslide area, is the topographic slope of the landslide area, , , is the weight coefficient.

[0070] Optionally, the risk value of the disaster risk score can be obtained , classify the disaster risk level of landslide areas.

[0071]

[0072] in, and It is the threshold for risk classification. It ensures road safety by providing timely geological disaster risk information.

[0073] Figure 3is a schematic diagram of a landslide area in a target road slope area in an embodiment of the present invention, including three landslide areas, and corresponding risk levels are identified by different color depths.

[0074] In order to solve the problems of high cost, limited coverage, and lack of real-time performance of traditional methods in road slope monitoring, this embodiment proposes a road slope monitoring model based on a depth estimation algorithm. The model uses deep learning to generate high-precision depth images, and combines change detection, target detection, and spatial clustering methods to achieve automatic identification and early warning of disasters such as landslides, collapses, and rockfalls. It includes three parts: data set preparation, depth image generation and disaster detection, and disaster early warning and visualization.

[0075] Figure 4 It is a flow chart of road slope monitoring based on a depth estimation algorithm in an embodiment of the present invention, including: obtaining road slope monitoring images or drone aerial images to construct a data set; constructing a depth estimation algorithm based on deep learning; inputting data into the depth estimation algorithm to generate a high-precision depth image, comparing the current depth with the historical depth image, and detecting geological disaster areas; calculating landslide risk scores, evaluating the severity of the disaster, and displaying the disaster area based on three-dimensional visualization technology, and generating early warning information and sending it to the road management department.

[0076] The solution of this embodiment uses computer vision and deep learning technology to generate high-precision depth images from road monitoring images or two-dimensional images taken by drones, and combines intelligent detection algorithms to identify disasters such as landslides, collapses, and rolling stones, thereby realizing automatic monitoring and early warning of road slopes. A depth estimation algorithm is used to generate high-precision depth images to accurately characterize the shape of road slopes. Combined with time series analysis and spatial clustering methods, automatic identification of geological disaster areas such as landslides and collapses is achieved. Three-dimensional visualization technology is used for real-time monitoring, intuitive display of slope changes, assistance in road safety management, and improvement of road disaster response capabilities.

[0077] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0078] Example 2 In this embodiment, a road slope disaster monitoring device based on deep learning is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, can also be conceived.

[0079] Figure 5 is a structural block diagram of a road slope disaster monitoring device based on deep learning according to an embodiment of the present invention. Figure 5 As shown, including: A first acquisition module 50 is used to acquire a two-dimensional image of the target road slope area at the current time; A conversion module 52, configured to convert the two-dimensional image into a current depth image using a pre-trained deep learning model; A second acquisition module 54 is used to acquire a historical depth image of the target road slope area at a historical time; A positioning module 56, configured to locate a landslide area in the two-dimensional image according to the current depth image and the historical depth image; The calculation module 58 is used to calculate the terrain parameters of the landslide area, and calculate the risk value of geological disasters occurring in the landslide area according to the terrain parameters.

[0080] Optionally, the conversion module includes: an extraction unit, used to extract the spatial information of the two-dimensional image in multiple network layers of a pre-trained deep learning model to obtain multi-scale features, wherein each network layer corresponds to a feature extraction scale, and the multi-scale features include local texture features and global semantic features; a prediction unit, used to predict the current depth image of the two-dimensional image through the multi-scale features in the deep learning model.

[0081] Optionally, the device further includes: a first construction module, configured to construct the following spatial structure loss function before the conversion module uses a pre-trained deep learning model to convert the two-dimensional image into a current depth image: :

[0082] ,in, is the predicted depth value, is the true depth value, represents the gradient, is the structural similarity loss, , , is the weighting coefficient, is the total number of pixels; the second building block is used to construct the following self-supervised loss function : ;in, is the sample depth image, is a predicted depth image generated based on the sample two-dimensional image corresponding to the sample depth image; a third construction module is used to construct a depth estimation optimization loss function using the following formula ,in is a hyperparameter corresponding to the weight; a training module is used to self-supervise the training of the initial model using the depth estimation optimization loss function to obtain the deep learning model.

[0083] Optionally, the positioning module includes: a first calculation unit, used to calculate, for each pixel point in the current depth image, a first depth change value between the current depth value of the pixel point and the historical depth value of the corresponding pixel point in the historical depth image; a first judgment unit, used to judge whether the depth change value is greater than a preset safety threshold; an acquisition unit, used to acquire the neighborhood pixels of the pixel point if the depth change value is greater than the preset safety threshold; a second calculation unit, used to calculate the second depth change value between the current depth value of the neighborhood pixel and the historical depth value of the corresponding pixel point in the historical depth image; a second judgment unit, used to judge whether the second depth change value is greater than the preset safety threshold; a clustering unit, used to cluster the pixel point and the neighborhood pixels into a landslide area in the two-dimensional image if the second depth change value is greater than the preset safety threshold.

[0084] Optionally, the calculation module includes: a first calculation unit, used to calculate the regional area, maximum depth change, and terrain slope of the landslide area; a second calculation unit, used to calculate the risk value of geological disasters occurring in the landslide area based on the regional area, the maximum depth change, and the terrain slope.

[0085] Optionally, the second calculation unit includes: a calculation subunit, configured to calculate the risk value R of geological disasters occurring in the landslide area by using the following formula: ;in, is the disaster risk score, is the area of ​​the region, is the maximum depth variation, is the terrain slope, , , is the corresponding weight coefficient.

[0086] Optionally, the device also includes: an extraction module, used to extract a three-dimensional point cloud set of the landslide area after the positioning module locates the landslide area in the two-dimensional image according to the current depth image and the historical depth image; a reconstruction module, used to reconstruct the three-dimensional point cloud set using the Alpha Shape algorithm to generate multiple candidate boundaries of the landslide area; a selection module, used to calculate the distance from each pixel point in the landslide area to the multiple candidate boundaries, and select a target boundary with the smallest total distance from all pixels in the landslide area to the boundary from the multiple candidate boundaries; an update module, used to update the landslide area using the area enclosed by the target boundary.

[0087] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0088] Example 3 An embodiment of the present invention further provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0089] Optionally, in this embodiment, the above storage medium may be configured to store a computer program for executing: S1, obtaining a two-dimensional image of the target road slope area at the current time; S2, converting the two-dimensional image into a current depth image using a pre-trained deep learning model; S3, obtaining a historical depth image of the target road slope area at a historical time; S4, locating the landslide area in the two-dimensional image according to the current depth image and the historical depth image; S5, calculating the terrain parameters of the landslide area, and calculating the risk value of geological disasters occurring in the landslide area according to the terrain parameters.

[0090] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0091] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0092] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0093] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program: S1, obtaining a two-dimensional image of the target road slope area at the current time; S2, converting the two-dimensional image into a current depth image using a pre-trained deep learning model; S3, obtaining a historical depth image of the target road slope area at a historical time; S4, locating the landslide area in the two-dimensional image according to the current depth image and the historical depth image; S5, calculating the terrain parameters of the landslide area, and calculating the risk value of geological disasters occurring in the landslide area according to the terrain parameters.

[0094] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0095] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0096] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0097] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0098] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0099] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0100] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, controller or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc., various media that can store program codes.

[0101] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A road slope disaster monitoring method based on deep learning, characterized in that: include: Obtain a two-dimensional image of the target road slope area at the current time; Converting the two-dimensional image into a current depth image using a pre-trained deep learning model; Acquire a historical depth image of the target road slope area at a historical time; Locating a landslide area in the two-dimensional image according to the current depth image and the historical depth image; The terrain parameters of the landslide area are calculated, and the risk value of geological disasters occurring in the landslide area is calculated based on the terrain parameters.

2. The method according to claim 1, characterized in that Using a pre-trained deep learning model to convert the two-dimensional image into a current depth image includes: Extracting spatial information of the two-dimensional image in multiple network layers of a pre-trained deep learning model to obtain multi-scale features, wherein each network layer corresponds to a feature extraction scale, and the multi-scale features include local texture features and global semantic features; A current depth image of the two-dimensional image is predicted in the deep learning model using the multi-scale features.

3. The method according to claim 1, characterized in that Before converting the two-dimensional image into a current depth image using a pre-trained deep learning model, the method further includes: Construct the following spatial structure loss function : ， in, is the predicted depth value, is the true depth value, represents the gradient, is the structural similarity loss, , , is the weighting coefficient, is the total number of pixels; Construct the following self-supervised loss function : ;in, is the sample depth image, A predicted depth image generated based on a sample two-dimensional image corresponding to the sample depth image; The depth estimation optimization loss function is constructed using the following formula: ,in is the hyperparameter corresponding to the weight; The depth estimation optimization loss function is used to self-supervise the training of the initial model to obtain the deep learning model.

4. The method according to claim 1, characterized in that: Locating the landslide area in the two-dimensional image according to the current depth image and the historical depth image includes: For each pixel in the current depth image, calculating a first depth change value between a current depth value of the pixel and a historical depth value of a corresponding pixel in the historical depth image; Determine whether the depth change value is greater than a preset safety threshold; If the depth change value is greater than a preset safety threshold, obtaining neighboring pixels of the pixel point; Calculating a second depth change value between a current depth value of the neighborhood pixel and a historical depth value of a corresponding pixel point in the historical depth image; Determining whether the second depth change value is greater than the preset safety threshold; If the second depth change value is greater than a preset safety threshold, the pixel point and the neighborhood pixels are clustered into a landslide area in the two-dimensional image.

5. The method according to claim 1, characterized in that: Calculating the terrain parameters of the landslide area, and calculating the risk value of geological disasters occurring in the landslide area according to the terrain parameters includes: Calculating the area, maximum depth change, and terrain slope of the landslide area; The risk value of geological disasters occurring in the landslide area is calculated according to the area of ​​the area, the maximum depth change, and the terrain slope.

6. The method according to claim 5, characterized in that Calculating the risk value of geological disasters occurring in the landslide area according to the area, the maximum depth change, and the terrain slope includes: The risk value R of geological disasters in the landslide area is calculated using the following formula: ； in, is the disaster risk score, is the area of ​​the region, is the maximum depth variation, is the terrain slope, , , is the corresponding weight coefficient.

7. The method according to claim 1, characterized in that After locating the landslide area in the two-dimensional image according to the current depth image and the historical depth image, the method further includes: Extracting a three-dimensional point cloud set of the landslide area; Reconstructing the three-dimensional point cloud set using an Alpha Shape algorithm to generate multiple candidate boundaries of the landslide area; Calculating the distances from each pixel point in the landslide area to the multiple boundaries to be selected, and selecting a target boundary with the minimum total distance from all pixels in the landslide area to the boundaries from the multiple boundaries to be selected; The landslide area is updated using the area enclosed by the target boundary.

8. A road slope disaster monitoring device based on deep learning, characterized in that: include: A first acquisition module is used to acquire a two-dimensional image of the target road slope area at the current time; A conversion module, configured to convert the two-dimensional image into a current depth image using a pre-trained deep learning model; A second acquisition module is used to acquire a historical depth image of the target road slope area at a historical time; A positioning module, used for locating the landslide area in the two-dimensional image according to the current depth image and the historical depth image; The calculation module is used to calculate the terrain parameters of the landslide area and calculate the risk value of geological disasters occurring in the landslide area according to the terrain parameters.

9. A storage medium, characterized in that: A computer program is stored in the storage medium, wherein the computer program is configured to execute the steps of the road slope disaster monitoring method based on deep learning in any one of claims 1 to 7 when running.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; wherein: Memory, used to store computer programs; A processor, configured to execute the steps of the road slope disaster monitoring method based on deep learning according to any one of claims 1 to 7 by running a program stored in a memory.

Images