Mountain state real-time monitoring and identification method and system based on deep learning

Through multi-angle image acquisition and analysis using deep learning technology, combined with dual-target detection and multi-level alarm mechanisms, the problems of blind spots and insufficient early warning in existing mountain monitoring systems have been solved, achieving efficient, real-time monitoring and early warning of mountain disasters.

CN120766202APending Publication Date: 2025-10-10CHONGQING YIER PUBLIC SAFETY EMERGENCY IND DEV CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202510770008.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-06-10
Publication Date
2025-10-10

AI Technical Summary

Technical Problem

Existing real-time monitoring methods and systems for mountain conditions are difficult to fully cover mountain areas, cannot provide multi-angle and all-round real-time monitoring data, lack the ability to efficiently identify and track multiple targets and complex environments, and lack the ability to predict motion trajectories, affecting the accuracy and timeliness of the early warning system.

Method used

A deep learning-based method is used to achieve efficient, real-time monitoring and early warning of mountain disasters through multi-angle image acquisition, dual-target detection, rockfall-crack detection model, rockfall trajectory prediction and crack analysis, combined with a multi-level alarm mechanism.

Benefits of technology

It realizes all-round and multi-dimensional mountain monitoring, accurately identifies falling rocks and cracks, precisely tracks the movement trajectory of falling rocks, extracts the changing characteristics of cracks with high precision, integrates a multi-level alarm mechanism, improves monitoring accuracy and safety, and prevents disasters.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120766202A_ABST
    Figure CN120766202A_ABST
Patent Text Reader

Abstract

The invention discloses a mountain state real-time monitoring and identification method and system based on deep learning. The method comprises the following steps: performing multi-angle dynamic image acquisition through a PTZ camera; training a dual-target detection network based on a YOLOv7 framework, constructing a rockfall-crack detection model, and synchronously outputting bounding box coordinates of a rockfall target and position information of a crack region; constructing a rockfall trajectory prediction model through a DeepSORT multi-target tracking algorithm, and obtaining rockfall movement trajectory data; pixel-level semantic segmentation is carried out on the crack area, a crack mask graph is generated, and a crack main direction vector is extracted; carrying out registration and difference calculation on the image to obtain a crack width variation and an extension length, identifying a newly-added crack region and marking spatial distribution of the newly-added crack region; and constructing a multistage alarm mechanism which comprises multistage alarm signals of an on-site audible and visual alarm device and a cloud platform. The mountain disaster monitoring and early warning system has the advantages that efficient and real-time monitoring and early warning of mountain disasters are achieved through rockfall detection and tracking and crack analysis and prediction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to mountain state monitoring, and in particular to a method and system for real-time monitoring and identification of mountain states based on deep learning. Background Art

[0002] With the impact of climate change and human activities, mountain stability is increasingly threatened. Therefore, accurate monitoring and early warning are crucial. Real-time mountain status monitoring and identification methods, primarily based on modern sensing technology, big data analysis, and artificial intelligence, aim to obtain real-time information on mountain dynamics to prevent natural disasters such as landslides and collapses.

[0003] Current commercially available real-time mountain condition monitoring and identification methods and systems rely on static cameras or single-angle image acquisition, making them incapable of comprehensive mountain coverage. This leads to blind spots and an inability to provide multi-angle, all-encompassing real-time monitoring data. Furthermore, many systems are limited to single-target detection, such as fallen rocks or cracks, lacking the ability to efficiently identify and track multiple targets in complex environments, making them unable to cope with the dynamic complexity of mountain changes. Furthermore, many existing methods rely solely on simple image segmentation or edge detection techniques, making it difficult to accurately extract pixel-level features of cracks. This results in inadequate crack analysis and can easily lead to missed detections or misjudgment of crack trends. Furthermore, existing monitoring methods generally lack the ability to predict motion trajectories, making it impossible to accurately assess risks based on the early detection of potential hazards, thus compromising the accuracy and timeliness of early warning systems. While some systems incorporate alert mechanisms, most employ relatively simple strategies, lacking intelligent judgment based on multidimensional data, and are unable to effectively avoid excessive or inadequate early warning responses. Summary of the Invention

[0004] In order to improve existing methods and systems, a real-time monitoring and identification method and system for mountain conditions based on deep learning is provided. This method realizes efficient, real-time monitoring and early warning of mountain disasters through multi-angle image acquisition, precise target detection and tracking, crack analysis and prediction, combined with a multi-level alarm mechanism, significantly improving monitoring accuracy and safety.

[0005] In order to achieve the above objects, the technical solution adopted by the present invention is:

[0006] The real-time monitoring and identification method of mountain conditions based on deep learning includes:

[0007] PTZ cameras deployed in the mountain monitoring area are used to capture dynamic images from multiple angles, generating time-series images and real-time video streams.

[0008] A dual-target detection network is trained based on the YOLOv7 framework to build a rockfall-crack detection model, which simultaneously outputs the bounding box coordinates of the rockfall target and the location information of the crack area.

[0009] Based on the identified falling rocks, a falling rock trajectory prediction model is constructed through a DeepSORT multi-target tracking algorithm, falling rock motion trajectory data is obtained, and the trajectory is rendered as a path marked with different colors;

[0010] Based on the identified crack area, pixel-level semantic segmentation is performed to generate a crack mask image, and crack main direction features are extracted and direction vectors are generated;

[0011] Based on the multi-angle time series images, registration and difference calculation are performed to obtain crack width variation and extension length, and by comparing historical mask images with the current frame, new crack areas are identified and their spatial distribution is marked.

[0012] Based on the falling rock trajectory prediction results and crack variation data, a multi-level alarm mechanism is constructed, including on-site sound and light alarm devices and multi-level alarm signals on the cloud platform.

[0013] Preferably, the double-target detection network is trained based on the YOLOv7 framework to construct a falling rock-crack detection model, which synchronously outputs the bounding box coordinates of the falling rock target and the position information of the crack area, specifically including:

[0014] Based on the obtained multi-angle dynamic images of the mountain monitoring area, a dataset is constructed and divided into a training set, a validation set and a test set;

[0015] Based on the multi-class target detection of YOLOv7, the falling rock and crack are detected, the double-target detection network is trained based on the obtained dataset, and the trained falling rock-crack detection model is obtained.

[0016] Based on the output results of the falling rock-crack detection model, the image pixel coordinates of the falling rock target bounding box and the image pixel coordinates of the crack area rectangular box are obtained.

[0017] Preferably, based on the identified falling rocks, a falling rock trajectory prediction model is constructed through a DeepSORT multi-target tracking algorithm, falling rock motion trajectory data is obtained, and the trajectory is rendered as a path marked with different colors, specifically including:

[0018] Based on the falling rock target identified by the falling rock-crack detection model, the bounding box of the falling rock target in each frame of the real-time video stream and its confidence are obtained.

[0019] A falling rock trajectory prediction model is constructed, the bounding box and confidence of each falling rock target in each frame are taken as inputs of the DeepSORT algorithm, and appearance features of each falling rock target are extracted;

[0020] The state of each rockfall target is predicted based on the Kalman filter inside DeepSORT, which predicts the next position of the target based on its historical state and the current detected bounding box position;

[0021] Based on the bounding box position data of each rockfall target in each frame, a motion trajectory is generated and fused with the predicted trajectory;

[0022] Based on the trajectories of different rockfall targets, different colors are used for rendering, and solid lines represent the motion trajectory obtained by image acquisition, and dashed lines represent the predicted trajectory.

[0023] Preferably, the pixel-level semantic segmentation based on the identified crack region generates a crack mask image and extracts crack main direction features and generates a direction vector, specifically including:

[0024] Based on the crack image data identified by the rockfall-crack detection model, the crack region is labeled, and pixel-level semantic segmentation is performed through U-Net segmentation network training;

[0025] Based on the trained model, the crack image is inferred, and each pixel in the image is classified as a crack or non-crack, generating a crack mask image;

[0026] Based on the crack mask image, the crack region contour is obtained, and by fitting the crack region contour points to a straight line using the least squares method, the main direction of the crack region is obtained, and a direction vector consistent with the main direction of the crack is generated.

[0027] Preferably, the registration and difference calculation based on multi-angle time series images are used to obtain the crack width change and extension length, and by comparing the historical mask image with the current frame, new crack regions are identified and their spatial distribution is marked, specifically including:

[0028] Based on multi-angle time series images, the historical image and the current image are aligned through a feature point-based registration method;

[0029] By aligning the historical image and the current image, the crack region between the two images is extracted;

[0030] Based on the crack contour in the current image and the historical mask image, the crack width change and the crack extension length are calculated and obtained;

[0031] Based on the part that appears in the current frame but does not appear in the historical mask image, it is marked as a new crack region, and based on the crack region change, it is marked with different colors to obtain the spatial distribution of the crack region.

[0032] Preferably, the rockfall trajectory prediction result and the crack change amount data are used to construct a multi-level alarm mechanism, which includes a multi-level alarm signal of a local sound and light alarm device and a cloud platform, and specifically includes:

[0033] Based on the rockfall trajectory prediction result and the crack change amount data, an alarm threshold is set, and a multi-level alarm mechanism is constructed.

[0034] Based on the predicted trajectory, if the trajectory prediction rockfall is about to enter a dangerous area or pose a threat to the crowd, equipment, etc., an alarm is triggered.

[0035] Based on the crack change amount threshold, if the change amount exceeds the predetermined safety range or if the newly detected crack area growth trend is obvious, an alarm is triggered.

[0036] The multi-level alarm mechanism includes primary, secondary, and tertiary alarms, and the alarm signal is adjusted in real time according to the alarm level.

[0037] Multi-party alarm is performed through the multi-level alarm signal of the local sound and light alarm device and the cloud platform.

[0038] Further, a mountain state real-time monitoring and identification system based on deep learning is proposed, which includes:

[0039] Image acquisition module: the image acquisition module realizes multi-angle dynamic image and real-time video stream acquisition of the mountain monitoring area by deploying multiple PTZ cameras.

[0040] Target detection module: the target detection module uses a rockfall-crack detection model based on YOLOv7 to locate and classify rockfall and crack targets in images.

[0041] Crack segmentation and direction extraction module: the crack segmentation and direction extraction module uses a U-Net network to perform pixel-level semantic segmentation on crack regions and extract the main direction of the crack to generate a direction vector.

[0042] Rockfall trajectory prediction module: the rockfall trajectory prediction module uses the DeepSORT algorithm to track multiple rockfall targets and predict their motion trajectories.

[0043] Crack change module: the crack change module uses multi-temporal image registration and difference analysis to extract crack width change, extension length, and identify new crack areas.

[0044] Alarm module: the alarm module sets multi-level alarm thresholds based on rockfall trajectory prediction and crack change trend, and triggers sound and light and cloud alarm response.

[0045] Processor: the processor is used to process the calculation process of each formula and the construction calculation process of each model.

[0046] Compared with the prior art, the advantages of the present invention are:

[0047] By capturing time-series images and real-time video streams from multi-angle PTZ cameras, this method enables comprehensive, multi-dimensional mountain monitoring. Utilizing a dual-target detection network within the YOLOv7 framework, this method accurately identifies falling rocks and cracks and outputs their location information in real time, providing a crucial basis for early warning of mountain hazards. Furthermore, combined with the DeepSORT algorithm, it can precisely track and predict the trajectory of falling rocks, rendering them with color-coded markers to help monitors more intuitively identify potential hazards. Furthermore, pixel-level semantic segmentation of cracks based on a U-Net network accurately extracts the main direction and variation of cracks, providing reliable data for analyzing their evolution trends. Through multi-angle image registration and differential calculations, the system can also accurately identify changes in crack width and newly added crack areas, enabling real-time monitoring of mountain stability changes. Finally, an integrated multi-level alarm mechanism triggers both on-site and cloud-based audible and visual alarms based on predicted trajectories and crack variation, effectively preventing disasters. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of the method proposed in the present invention;

[0049] Figure 2 Schematic diagram of the rockfall-crack detection model proposed in the present invention;

[0050] Figure 3 Schematic diagram of the rockfall trajectory prediction model proposed in the present invention;

[0051] Figure 4 This is a schematic diagram of the main direction vector of the crack proposed in the present invention;

[0052] Figure 5 This is a schematic diagram of image registration and differential calculation proposed by the present invention;

[0053] Figure 6 This is a schematic diagram of the multi-level alarm proposed by the present invention;

[0054] Figure 7 This is a diagram of the architecture of the electronic equipment in this solution;

[0055] Figure 8 This is a schematic diagram of the computer-readable storage medium structure in this solution. DETAILED DESCRIPTION

[0056] The following description is intended to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are merely examples, and those skilled in the art may conceive of other obvious variations.

[0057] The real-time mountain status monitoring and identification system based on deep learning includes:

[0058] Image acquisition module: The image acquisition module realizes multi-angle dynamic image and real-time video stream acquisition of the mountain monitoring area by deploying multiple PTZ cameras;

[0059] Target detection module: The target detection module is based on the rockfall and crack detection model built by YOLOv7 to locate and classify rockfall and crack targets in the image;

[0060] Crack segmentation and direction extraction module: The crack segmentation and direction extraction module uses the U-Net network to perform pixel-level semantic segmentation on the crack area and extract the main direction of the crack to generate a direction vector;

[0061] Rockfall trajectory prediction module: The rockfall trajectory prediction module uses the DeepSORT algorithm to perform multi-target tracking of rockfall targets and predict their motion trajectories;

[0062] Crack change module: Based on multi-temporal image registration and differential analysis, the crack change module extracts crack width changes, extension lengths, and identifies newly added crack areas.

[0063] Alarm module: Based on rockfall trajectory prediction and crack change trends, the alarm module sets multi-level alarm thresholds and triggers sound, light, and cloud alarm responses;

[0064] Processor: The processor is used to process the calculation process of each formula and the construction calculation process of each model.

[0065] See Figure 1 As shown in FIG, the real-time monitoring and identification method of mountain conditions based on deep learning includes:

[0066] Step 1: Use PTZ cameras deployed in the mountain monitoring area to capture dynamic images from multiple angles, generating time-series images and real-time video streams.

[0067] Step 2: Train a dual-target detection network based on the YOLOv7 framework, build a rockfall-crack detection model, and simultaneously output the bounding box coordinates of the rockfall target and the location information of the crack area;

[0068] Step 3: Based on the identified rockfalls, a rockfall trajectory prediction model is constructed using the DeepSORT multi-target tracking algorithm to obtain rockfall trajectory data and render the trajectory as a path marked with different colors;

[0069] Step 4: Perform pixel-level semantic segmentation based on the identified crack area to generate a crack mask map, extract the main direction features of the crack and generate a direction vector;

[0070] Step 5: Perform registration and differential calculation based on multi-angle time series images to obtain the change in crack width and extension length. By comparing the historical mask image with the current frame, identify the newly added crack area and mark its spatial distribution;

[0071] Step 6: Based on the rockfall trajectory prediction results and crack change data, a multi-level alarm mechanism is constructed, including on-site sound and light alarm devices and multi-level alarm signals on the cloud platform.

[0072] See Figure 2 As shown in the figure, a dual-target detection network is trained based on the YOLOv7 framework to build a rockfall-crack detection model. The bounding box coordinates of the rockfall target and the location information of the crack area are output simultaneously. Specifically,

[0073] A dataset is constructed based on the multi-angle dynamic images of the mountain monitoring area and divided into training set, validation set and test set;

[0074] Rockfalls and cracks are detected using YOLOv7's multi-category target detection. The dual-target detection network is trained using the acquired dataset to obtain a trained rockfall-crack detection model.

[0075] Based on the output results of the rockfall-crack detection model, the image pixel coordinates of the bounding box of the rockfall target and the image pixel coordinates of the rectangular box of the crack area are obtained.

[0076] Specifically, the YOLOv7 network consists of multiple parts: the backbone network is responsible for extracting high-dimensional features of the image, usually using convolutional layers; the neck network performs feature fusion by merging features at different levels; and the detection head outputs the category, bounding box coordinates, and confidence level of each object;

[0077] The loss function of YOLOv7 mainly consists of three parts: positioning loss is used to measure the gap between the predicted bounding box and the actual bounding box, confidence loss is used to predict whether the box contains the target, and category loss is used to calculate the cross entropy loss of the category for each box.

[0078] Get the image pixel coordinates of the bounding box through the output of the model.

[0079] See Figure 3 As shown in the figure, based on the identified falling rocks, a rockfall trajectory prediction model is constructed using the DeepSORT multi-target tracking algorithm to obtain the rockfall motion trajectory data and render the trajectory as a path marked with different colors. Specifically,

[0080] Based on the rockfall targets identified by the rockfall-crack detection model, the bounding box and confidence score of the rockfall targets in each frame of the real-time video stream are obtained;

[0081] A rockfall trajectory prediction model is constructed, and the bounding box and confidence of each rockfall target in each frame are taken as inputs of the DeepSORT algorithm, and the appearance features of each rockfall target are extracted;

[0082] The state of each rockfall target is predicted based on the Kalman filter inside DeepSORT, which predicts the next position of the target based on the historical state of the target and the current detected bounding box position;

[0083] Based on the bounding box position data of each rockfall target in each frame, a motion trajectory is generated and fused with the predicted trajectory;

[0084] Based on the trajectories of different rockfall targets, different colors are used for rendering, and solid lines are used to represent the motion trajectories obtained by image acquisition, and dashed lines are used to represent the predicted trajectories.

[0085] Specifically, the appearance features of each detected rockfall target need to be extracted and represented as a vector. The core of DeepSORT is the Kalman filter, which is used to predict the motion state of the target. The Kalman filter predicts the possible position of the target in the next frame based on the historical state of the target and the current detected bounding box. The prediction model of the Kalman filter is:

[0086]

[0087] where, is the predicted position, is the current position, is the state transition matrix, representing the state transition from one time to the next, is the control matrix, representing the influence of external input, is the control vector, is the process noise;

[0088] Through the update step of the Kalman filter, the predicted target position and velocity are calculated;

[0089] In multiple frames of video, DeepSORT generates a trajectory based on the motion information of the target. The trajectory of each target is composed of multiple position points, each point corresponding to the position of the target in a certain frame;

[0090] The real trajectory and the predicted trajectory of the target are used for trajectory visualization. Different colors are used for rendering according to the trajectories of different targets. The real trajectory is represented by a solid line, and the predicted trajectory is represented by a dashed line. For example, assume that the trajectory of target 1 is represented by red, and the trajectory of target 2 is represented by green. The real trajectory is represented by a solid line, and the predicted trajectory is represented by a dashed line.

[0091] Referring to Figure 4As shown, based on the identified crack area, pixel-level semantic segmentation is performed, a crack mask map is generated, and crack main direction features are extracted and a direction vector is generated, which specifically includes:

[0092] Based on the crack image data identified by the rockfall-crack detection model, the crack area is labeled, and pixel-level semantic segmentation is performed through U-Net segmentation network training;

[0093] Based on the trained model, the crack image is inferred, each pixel in the image is classified as a crack or non-crack, and a crack mask map is generated;

[0094] Based on the crack mask map, the contour of the crack area is obtained, the contour points of the crack area are fitted into a straight line by the least square method, the main direction of the crack area is obtained, and a direction vector consistent with the main direction of the crack is generated.

[0095] Specifically, the crack image is pixel-level labeled, each pixel is labeled as "crack" or "non-crack", and the U-Net model is used for segmentation network training. The network structure of U-Net includes: the encoder is composed of multiple convolutional layers, which is used to extract the features of the image; the bottleneck layer connects the encoder and the decoder part, and contains convolution operation to extract higher-level features; the decoder restores the spatial resolution through transposed convolution; the low-level features in the encoder are connected with the up-sampling results in the decoder through the jump connection, which helps to restore the image details;

[0096] The trained U-Net model is used to infer new images to obtain the classification probability of each pixel, and the probability of each pixel is converted to 0 or 1. The generated mask map is the labeling result of the crack area;

[0097] Based on the crack mask map, the boundary of the crack area is obtained by binarizing the connected region of the image, and the edge of the crack is found using the contour extraction algorithm. The obtained boundary is the contour point set of the crack. The least square method is used to fit the straight line of the crack contour points to obtain the main direction of the crack. The goal of the least square method is to minimize the sum of squared residuals. By taking the partial derivative of the slope and intercept and setting it to zero, the optimal slope and intercept are obtained.

[0098] The direction vector of the main direction can be represented as:

[0099]

[0100] Wherein, is the direction vector, is the slope of the fitted straight line;

[0101] The normalized direction vector is the main direction of the crack area.

[0102] Referring to Figure 5 As shown, based on multi-angle time series images, registration and difference calculation are performed to obtain the crack width change and the extension length. By comparing the historical mask image with the current frame, the newly added crack region is identified and its spatial distribution is marked. Specifically, the method comprises the following steps:

[0103] Based on multi-angle time series images, a feature point-based registration method is used to align the historical image with the current image;

[0104] By aligning the historical image with the current image, the crack region between the two images is extracted;

[0105] Based on the crack contours in the current image and the historical mask image, the change in the crack region width and the extension length of the crack are calculated and obtained;

[0106] Based on the part that appears in the current frame but does not appear in the historical mask image, the newly added crack region is marked, and the spatial distribution of the crack region is obtained based on the crack region change and marked with different colors.

[0107] Specifically, feature points in the image are extracted by a feature point detection algorithm. By matching the feature points in the historical image and the current image, outliers are removed, and the transformation matrix between the two images is estimated. The transformation matrix makes the feature points of the historical image correspond to the feature points in the current image.

[0108] Once the historical image and the current image are aligned, the crack region is extracted by image difference. The crack region that appears in the current image but does not appear in the historical image is marked as a newly added crack. According to the change in the crack region, the change in the crack region is marked with different colors.

[0109] Referring to Figure 6 As shown, based on the rockfall trajectory prediction result and the crack change data, a multi-level alarm mechanism is constructed, including a multi-level alarm signal of a field sound and light alarm device and a cloud platform. Specifically, the method comprises the following steps:

[0110] Based on the rockfall trajectory prediction result and the crack change data, an alarm threshold is set, and a multi-level alarm mechanism is constructed;

[0111] Based on the predicted trajectory, if the trajectory prediction rockfall is about to enter a dangerous area or pose a threat to the crowd, equipment, etc., an alarm is triggered;

[0112] Based on the crack change data, if the change exceeds the predetermined safety range or if the newly added crack region is detected to have a significant growth trend, an alarm is triggered;

[0113] The multi-level alarm mechanism includes primary alarm, secondary alarm and tertiary alarm. The alarm signal is adjusted in real time according to the alarm level;

[0114] The multi-level alarm signal of the on-site acousto-optic alarm device and the cloud platform is used for multi-party alarm.

[0115] Specifically, according to the predicted rockfall trajectory, a dangerous area is set, i.e., an area where the rockfall may pose a threat to the crowd, equipment, etc. For the crack change amount, the change amplitude of the crack is defined.

[0116] The multi-level alarm mechanism usually includes three levels:

[0117] Level 1 alarm: indicating a potential emergency situation that needs immediate attention, such as the rockfall entering the dangerous area soon, or the crack change amount exceeding a large threshold;

[0118] Level 2 alarm: indicating a moderate danger or risk that may affect the environment or personnel, which requires vigilance but does not require immediate response;

[0119] Level 3 alarm: indicating a general warning, which needs monitoring but does not require emergency response;

[0120] Whenever the alarm condition is met, the system will generate corresponding signals according to different alarm levels, and perform multi-party alarm through the acousto-optic alarm device and the cloud platform.

[0121] Further, the method according to the embodiments of the present application can also be implemented by means of Figure 7 The architecture of the electronic device as shown. As Figure 7 shown, the electronic device 500 can include a bus 501, one or more CPUs 502, a read-only memory (ROM) 503, a random access memory (RAM) 504, a communication port 505 connected to a network, an input / output component 506, a hard disk 507, etc. The storage device in the electronic device 500, such as the ROM 503 or the hard disk 507, can store the deep learning-based mountain state real-time monitoring and identification method and system provided by the present application. The electronic device 500 can also include a terminal interface 508. Of course, Figure 7 The architecture shown is only exemplary, and when implementing different devices, one or more components of the electronic device shown can be omitted according to actual needs. Figure 7

[0122] Figure 8 is a structural diagram of a computer readable storage medium provided by an embodiment of the present application. As Figure 8 ​As shown, a computer-readable storage medium 600 according to one embodiment of the present application is shown. Computer-readable instructions are stored on the computer-readable storage medium 600. When the computer-readable instructions are executed by the processor, the method and system for real-time monitoring and identification of mountain conditions based on deep learning according to the embodiment of the present application described with reference to the above figures can be executed. The storage medium 600 includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0123] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. Furthermore, the foregoing descriptions of specific embodiments of this specification are provided. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential sequence shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0124] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0125] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A real-time monitoring and identification method for mountain conditions based on deep learning, characterized in that: include: PTZ cameras deployed in the mountain monitoring area are used to capture dynamic images from multiple angles, generating time-series images and real-time video streams. A dual-target detection network is trained based on the YOLOv7 framework to build a rockfall-crack detection model, which simultaneously outputs the bounding box coordinates of the rockfall target and the location information of the crack area. Based on the identified falling rocks, a rockfall trajectory prediction model is constructed using the DeepSORT multi-target tracking algorithm to obtain rockfall motion trajectory data and render the trajectory as a path marked with different colors. Perform pixel-level semantic segmentation based on the identified crack area to generate a crack mask map, extract the main direction features of the crack and generate a direction vector; Based on multi-angle time series images, registration and difference calculation are performed to obtain the change in crack width and extension length. By comparing the historical mask image with the current frame, the newly added crack area is identified and its spatial distribution is marked. Based on the rockfall trajectory prediction results and crack change data, a multi-level alarm mechanism is constructed, including on-site sound and light alarm devices and multi-level alarm signals on the cloud platform.

2. The method for real-time monitoring and identification of mountain conditions based on deep learning according to claim 1 is characterized in that: The dual-target detection network is trained based on the YOLOv7 framework, a rockfall-crack detection model is constructed, and the bounding box coordinates of the rockfall target and the location information of the crack area are outputted synchronously. Specifically, A dataset is constructed based on the multi-angle dynamic images of the mountain monitoring area and divided into training set, validation set and test set; Rockfalls and cracks are detected using YOLOv7's multi-category target detection. The dual-target detection network is trained using the acquired dataset to obtain a trained rockfall-crack detection model. Based on the output results of the rockfall-crack detection model, the image pixel coordinates of the bounding box of the rockfall target and the image pixel coordinates of the rectangular box of the crack area are obtained.

3. The method for real-time monitoring and identification of mountain conditions based on deep learning according to claim 1 is characterized in that: The method of constructing a rockfall trajectory prediction model based on the identified rockfall using the DeepSORT multi-target tracking algorithm, obtaining rockfall motion trajectory data, and rendering the trajectory as paths marked with different colors specifically includes: Based on the rockfall targets identified by the rockfall-crack detection model, the bounding box and confidence score of the rockfall targets in each frame of the real-time video stream are obtained; A rockfall trajectory prediction model was constructed based on the DeepSORT algorithm. The bounding box and confidence score of each rockfall target in each frame were used as inputs to the DeepSORT algorithm, and appearance features of each rockfall target were extracted. The state of each rockfall target is predicted based on the Kalman filter within DeepSORT. The Kalman filter predicts its next position based on the target's historical state and the current detected bounding box position. Generate a motion trajectory based on the bounding box position data of each rockfall target in each frame and fuse it with the predicted trajectory; The trajectories of different rockfall targets are rendered in different colors, and the solid lines represent the motion trajectories obtained by image acquisition, while the dotted lines represent the predicted trajectories.

4. The method for real-time monitoring and identification of mountain conditions based on deep learning according to claim 1, characterized in that: The pixel-level semantic segmentation based on the identified crack area, generating a crack mask map, extracting the main direction features of the crack and generating a direction vector specifically includes: Based on the crack image data identified by the rockfall-crack detection model, the crack area is annotated and pixel-level semantic segmentation is performed through U-Net segmentation network training; Based on the trained model, the crack image is inferred, each pixel in the image is classified as a crack or non-crack, and a crack mask is generated; The outline of the crack area is obtained based on the crack mask image. The main direction of the crack area is obtained by fitting the contour points of the crack area into a straight line using the least squares method, and a direction vector consistent with the main direction of the crack is generated.

5. The method for real-time monitoring and identification of mountain conditions based on deep learning according to claim 1 is characterized in that: The registration and difference calculation based on the multi-angle time series images are performed to obtain the crack width change and extension length, and the newly added crack areas are identified and their spatial distribution is marked by comparing the historical mask image with the current frame. Specifically, the following steps are performed: Based on multi-angle time series images, historical images are aligned with current images through a feature point-based registration method; By aligning the historical image with the current image, the crack area between the two images is extracted; Based on the crack outlines in the current image and the historical mask image, the change in the crack area width and the extension length of the crack are calculated; Based on the parts that appear in the current frame but do not appear in the historical mask image, they are marked as new crack areas, and marked with different colors based on the change in the crack area to obtain the spatial distribution of the crack area.

6. The method for real-time monitoring and identification of mountain conditions based on deep learning according to claim 1, characterized in that: The multi-level alarm mechanism is constructed based on the rockfall trajectory prediction results and crack change data, including the multi-level alarm signals of the on-site sound and light alarm device and the cloud platform, specifically including: Based on the rockfall trajectory prediction results and crack change data, the alarm threshold is set and a multi-level alarm mechanism is established; Based on the predicted trajectory, if the trajectory predicts that the rockfall is about to enter a dangerous area or pose a threat to people or equipment, an alarm is triggered; A threshold is set based on the amount of crack change. If the change exceeds a predetermined safety range or if a significant growth trend in the newly detected crack area is detected, an alarm is triggered. The multi-level alarm mechanism includes a high-level alarm, a secondary alarm, and a tertiary alarm, and adjusts the alarm signal in real time according to the alarm level; Multi-party alarms are issued through multi-level alarm signals from on-site sound and light alarm devices and cloud platforms.

7. A system for real-time monitoring and identification of mountain conditions based on deep learning, for implementing the method for real-time monitoring and identification of mountain conditions based on deep learning according to any one of claims 1 to 6, characterized in that: include: Image acquisition module: The image acquisition module realizes multi-angle dynamic image and real-time video stream acquisition of the mountain monitoring area by deploying multiple PTZ cameras; Target detection module: The target detection module is based on the rockfall and crack detection model built by YOLOv7 to locate and classify rockfall and crack targets in the image; Crack segmentation and direction extraction module: The crack segmentation and direction extraction module uses the U-Net network to perform pixel-level semantic segmentation on the crack area and extract the main direction of the crack to generate a direction vector; Rockfall trajectory prediction module: The rockfall trajectory prediction module uses the DeepSORT algorithm to perform multi-target tracking of rockfall targets and predict their motion trajectories; Crack change module: Based on multi-temporal image registration and differential analysis, the crack change module extracts crack width changes, extension lengths, and identifies newly added crack areas. Alarm module: Based on rockfall trajectory prediction and crack change trends, the alarm module sets multi-level alarm thresholds and triggers sound, light, and cloud alarm responses; Processor: The processor is used to process the calculation process of each formula and the construction calculation process of each model.

8. An electronic device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the real-time monitoring and identification method of mountain conditions based on deep learning as described in any one of claims 1-6.

9. A computer-readable storage medium storing computer-readable instructions, characterized in that: When the computer-readable instructions are executed by a processor, the method for real-time monitoring and identification of mountain conditions based on deep learning according to any one of claims 1 to 6 is implemented.

Citation Information

Cited By

  • Road bridge engineering concrete crack detection method and system based on image recognition

    CN121280427A

  • Real-time monitoring and early warning method and system for mountain rockfall in complex terrain based on multi-modal space-time fusion and edge adaptive reasoning

    CN122223941A