Intelligent visual monitoring method and system for small-scale deformation body of side slope

Abstract

The invention discloses an intelligent visual monitoring method and system for a small-scale deformation body of a side slope, and relates to the technical field of side slope monitoring, and the method comprises the steps: searching targets in all view fields of a side slope surface, carrying out the initial deployment of the targets, carrying out the target detection of the targets, and determining whether the targets exist or not; after the existence of the target is confirmed, pixel-level positioning is firstly carried out on the edge of the target, then sub-pixel-level positioning is carried out on the edge of the target, the ellipse circle center coordinate of the target ring is obtained, and the displacement of the target is calculated in combination with the historical position of the target; and judging the structural deformation of the slope according to the target displacement, and judging the landslide risk of the slope by using the structural deformation. According to the invention, high-precision displacement monitoring is carried out on a plurality of point positions of the whole slope surface through vision, planar coverage monitoring of the monitored slope surface is realized, the construction deployment difficulty and the monitoring cost of monitoring equipment are greatly reduced, and the monitoring efficiency of a small-scale deformation body of the slope is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of slope monitoring, and more specifically, to an intelligent visual monitoring method and system for small-scale deformation bodies of slopes. Background Art

[0002] Cutting slopes to build houses refers to a way of building houses in mountainous rural areas due to site restrictions, often cutting the slopes to expand the space for building houses. This way of building houses is mainly concentrated in mountainous and hilly areas in terms of geographical distribution. Since cutting slopes to build houses changes the original terrain and destroys the natural balance of the mountain; excessive excavation forms high and steep slopes; excavation of slopes causes the surface soil of the slopes to become loose, and a large amount of rainwater infiltrates during heavy rainfall, which can easily cause geological disasters such as landslides and collapses. In order to reduce the landslide disasters caused by cutting slopes to build houses, it is necessary to establish a set of early warning systems for cutting slopes and landslide disasters to monitor the cut slopes in real time and give early warnings for possible landslide disasters.

[0003] Traditional monitoring methods include manual inspections, displacement monitoring, rainfall monitoring, groundwater level monitoring, etc. GNSS surface displacement meters, rain gauges, crack meters, inclinometers and other equipment are usually deployed in landslide areas to monitor environmental factors and slope deformation. Traditional monitoring methods generally have disadvantages such as high cost, inconvenient installation, and difficult later maintenance. Moreover, these devices are deployed in a point-like manner and cannot form a surface coverage of the area. At the same time, after the event is detected, it is impossible to report the on-site pictures, and a separate monitoring system needs to be installed. Visual intelligent deformation monitoring is different from the traditional contact measurement method. It is a monitoring method based on machine vision and combined with traditional photogrammetry, digital image processing, artificial intelligence and other disciplines. Therefore, how to carry out low-cost and high-density surface coverage monitoring of the slope surface, and solve the difficulties in the deployment of existing monitoring equipment, high later maintenance costs, and the inability to report on-site situation images are urgent problems that need to be solved. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes an intelligent visual monitoring method and system for small-scale deformation bodies of slopes, which monitors the tiny displacement of each target through visual measurement, thereby monitoring the changes in the slope structure to which the target is attached, and predicting landslide disasters based on the displacement change data of the slope.

[0005] A first aspect of the present invention provides an intelligent visual monitoring method for small-scale deformation of a slope, comprising the following steps:

[0006] Search for targets within all fields of view on the slope surface, perform target initialization deployment, and perform target detection to determine whether the target exists;

[0007] After confirming the existence of the target, first perform pixel-level positioning on the edge of the target, then perform sub-pixel-level positioning on the edge of the target, obtain the coordinates of the ellipse center of the target ring, and calculate the target displacement based on the historical position of the target;

[0008] The structural deformation of the slope is determined based on the target displacement, and the landslide risk of the slope is determined based on the structural deformation.

[0009] In this solution, target detection is performed based on a method combining machine learning and deep learning to detect and search for all targets, specifically:

[0010] Use a machine learning algorithm for template matching to perform preliminary and rapid detection on all targets that have completed initial deployment in the field of view, obtain a preset target template image, build a sliding window based on the target template image, and use the sliding window to perform matching operations on the image to be detected;

[0011] The similarity R between the sub-image in the sliding window and the target template image is calculated according to the matching operation. ccoeff , the matching operation formula is:

[0012]

[0013] Wherein, T represents the target template image, I represents the sub-image to be matched in the sliding window, (x, y) represents the element coordinates of the upper left corner of the current sliding window in the matrix corresponding to the sub-image to be matched, and (x′, y′) represents the element coordinates of the target template image and the current sliding window in the matrix corresponding to the sub-image to be matched;

[0014] After traversing the image to be detected, the similarity distribution in the image to be detected is obtained, and a position greater than a preset similarity threshold is found according to the similarity distribution. The position is used as the target position, and the target position is used to determine whether the target exists, and the target is re-detected based on the deep learning algorithm.

[0015] In this solution, the target is tested again based on the deep learning algorithm, specifically:

[0016] Based on data retrieval, a number of target images of slope cutting scenes are obtained, the target images of slope cutting scenes are annotated, and a target target data set is established. All images in the target target data set are segmented according to a preset sub-image size and a coverage ratio between sub-images, and the sub-image data sets obtained after segmentation are divided into a training set and a test set according to a preset ratio.

[0017] The YOLO v5 network is selected to build a target detection model, and the training set is used for model training. The MobileNet unit and the CBAM unit are introduced into the backbone network of the YOLO v5. The MobileNet unit adjusts the number of channels through a 1*1 convolution operation, and uses a depth-separable convolution to reduce the amount of calculation. A three-channel dimension connection is adopted, and a CBAM unit is added to one of the channels. Combined with the position information of the network layer, the deep and shallow feature maps of the target are extracted through channel and spatial attention processing;

[0018] The processed feature map is concatenated with the feature maps of the other two branches to obtain output features, the output features are introduced into a weighted bidirectional feature pyramid network for multi-scale feature fusion, and the fused feature map is sent to a detection head for target detection;

[0019] The detection head is decoupled and split into an independent classification branch and a positioning branch. In the classification branch, the detection target is set as a target category. An attention mechanism is set in the detection head to adaptively weight the fused feature map to improve task adaptability.

[0020] The weighted fusion feature map is reduced in dimension, the classification task and the positioning task are completed in the classification branch and the positioning branch respectively, and the target detection model is tested using the test set after iterative training. When the model performance meets the preset standard, the trained target detection model is output;

[0021] Use the trained target detection model to perform a second detection on the image to be detected to confirm whether the target exists and obtain the pixel coordinates of the target. If both detections confirm that the target is lost, the target loss alarm is triggered.

[0022] In this scheme, after confirming the existence of the target, the target edge is first located at the pixel level, and then the target edge is located at the sub-pixel level, specifically:

[0023] Acquire the image to be detected of the slope surface, calibrate according to the camera parameters to obtain the distortion correction parameters, use the distortion correction parameters to correct the distortion of the image to be detected, and crop the image according to the confirmation information of the target;

[0024] The target in the cropped image to be detected is located according to the pixel coordinates of the target, the pixel-level edge of the target is extracted using the Sobel operator, and then the sub-pixel coordinates of the current pixel edge point are calculated based on the local area pixels centered on the pixel edge point.

[0025] In this scheme, the Sobel operator is used to calculate the gradient of the target pixel-level edge in the width and height directions to obtain the current pixel edge point (x i ,y i) The gradient dx in the width direction and the gradient dy in the height direction are used to calculate the gradient direction α, α = arctan (dy / dx);

[0026] Take the current pixel edge point (x i ,y i ) as the center, expand n points along the gradient direction on both sides, use bilinear interpolation to calculate the gradient values ​​of n points, perform quadratic curve interpolation on the gradient values, and obtain the vertex coordinates of the parabola;

[0027] The sub-pixel edge point of the image is calculated based on the pixel coordinates of the current pixel edge point, the gradient direction and the vertex coordinates of the parabola The calculation formula of the sub-pixel edge point is:

[0028]

[0029] Among them, x d ,y d They represent the horizontal and vertical coordinates of the vertices of the parabola respectively.

[0030] In this solution, after obtaining the sub-pixel edge coordinates of the target, an ellipse is fitted to the edge to obtain the ellipse center coordinates of the target ring, and the target displacement is calculated in combination with the historical position of the target, specifically:

[0031] Using the extracted sub-pixel edge point coordinates, the least squares fitting method is used to obtain the coordinates of the ellipse center of the target ring. The calculation formula is:

[0032]

[0033] Among them, A, B, C, D, and E are all constants;

[0034] The current position of the target is obtained according to the coordinates of the center of the ellipse of the target ring, the historical position of the target in the previous monitoring cycle of the current target is retrieved, and the target displacement of the current target is obtained through the current position of the target and the historical position of the target;

[0035] The displacement of the slope is calculated based on the average displacement of all targets in the field of view of the slope. Based on the displacement, an abnormal displacement alarm is generated and an analysis is performed on whether there is a landslide risk. When the landslide risk is higher than the preset risk value, a landslide warning is generated, and the on-site monitoring screen is captured and uploaded in real time for visual display.

[0036] In this solution, when monitoring the slope surface, if the ambient temperature changes significantly, the current position of the acquired target is compensated for temperature drift to compensate for the positioning accuracy.

[0037] The second aspect of the present invention provides an intelligent visual monitoring system for small-scale deformation bodies of slopes, comprising: a target deployment unit, a target detection unit, a target positioning unit, a displacement warning unit and a monitoring visualization unit;

[0038] The target deployment unit is responsible for searching for targets within all fields of view on the slope surface and performing target initialization deployment;

[0039] The target detection unit is responsible for performing target detection based on a method combining machine learning and deep learning, detecting and searching all targets, and triggering a target loss alarm when a target is lost;

[0040] The target positioning unit is responsible for positioning the target at the pixel level, and then positioning the target edge at the sub-pixel level. The coordinates of the ellipse center of the target ring are obtained by using the least squares fitting method using the extracted sub-pixel edge point coordinates to generate the current position of the target.

[0041] The displacement warning unit is responsible for obtaining the slope displacement according to the current position of the target and the historical position of the target, determining whether a displacement abnormality alarm is generated and analyzing whether there is a landslide risk. When the landslide risk is higher than a preset risk value, a landslide warning is generated;

[0042] The monitoring visualization unit is responsible for visually displaying the alarm information and early warning information, and when a landslide warning occurs, it captures the on-site monitoring picture and uploads it in real time to realize remote real-time viewing of the on-site picture.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] The present invention monitors the tiny displacement of each target through visual measurement, thereby monitoring the changes in the slope structure to which the target is attached, and performs high-precision displacement monitoring on multiple points on the entire slope, thereby achieving surface coverage monitoring of the monitored slope, and predicting landslide disasters based on the displacement change data of the slope. At the same time, the present invention solves the problems of difficult deployment and high subsequent maintenance costs of existing monitoring equipment, greatly reduces the construction deployment difficulty and monitoring costs of monitoring equipment, and improves the monitoring efficiency of small-scale deformed bodies on the slope. In addition, when a landslide disaster occurs, the on-site monitoring screen image can be transmitted back, allowing staff to review the on-site situation online, solving the problem that the existing equipment cannot report on-site situation images after problems occur on-site. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments or exemplary embodiments of the present invention, the drawings required for use in the embodiments or exemplary descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained according to the drawings without paying creative work.

[0046] Figure 1 A flow chart of an intelligent visual monitoring method for small-scale deformation bodies of slopes is shown;

[0047] Figure 2 A flowchart of an intelligent visual monitoring method for small-scale deformation of slopes is shown;

[0048] Figure 3 A flowchart of secondary detection of a target based on a deep learning algorithm is shown;

[0049] Figure 4 A schematic diagram of the process of extracting sub-pixel edges of a target is shown;

[0050] Figure 5 A block diagram of an intelligent visual monitoring system for small-scale deformation bodies on slopes is shown. DETAILED DESCRIPTION

[0051] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0052] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0053] Figure 1 , 2 The flowchart and the block diagram of the intelligent visual monitoring method for small-scale deformation bodies of slopes are shown respectively.

[0054] like Figure 1 As shown, in a first embodiment of the present invention, an intelligent visual monitoring method for small-scale deformation bodies of slopes is provided, comprising:

[0055] S102, searching for targets within all fields of view on the slope surface, initializing the target deployment, and performing target detection on the target to determine whether the target exists;

[0056] It should be noted that visual measurement technology is introduced into the monitoring and early warning of landslide disasters for slope cutting and house building. Several visual targets are installed and deployed on the slope surface, and the tiny displacement of each target is monitored through visual measurement, so as to monitor the changes in the slope structure to which the target is attached. Use all targets in the field of view of the visual device to initialize the target deployment, and adjust the equipment temperature so that the working temperature is at a stable target temperature to avoid temperature drift errors caused by temperature changes. When monitoring the slope surface, if the ambient temperature changes significantly, the current position of the acquired target is compensated for temperature drift to compensate for positioning accuracy. If the working temperature is not adjusted, multiple target displacement measurements are required.

[0057] During the deployment of target search and monitoring, target detection is required to confirm whether the target exists. Target monitoring is achieved by combining machine learning and deep learning. The rapid initial target detection uses a machine learning algorithm of template matching. When the target is found to be missing, in order to enhance the detection accuracy and reduce false positives, the deep learning model is started to perform a secondary detection on the target, confirm the target is missing, and then trigger the alarm.

[0058] Use the template matching machine learning algorithm to perform preliminary and rapid detection on all the targets that have completed initial deployment in the field of view, obtain a preset target template image, build a sliding window based on the target template image, and use the sliding window to perform a matching operation on the image to be detected; calculate the similarity R between the sub-image in the sliding window and the target template image based on the matching operation ccoeff , the matching operation formula is:

[0059]

[0060] Wherein, T represents the target template image, I represents the sub-image to be matched in the sliding window, (x, y) represents the element coordinates of the upper left corner of the current sliding window in the matrix corresponding to the sub-image to be matched, and (x′, y′) represents the element coordinates of the target template image and the current sliding window in the matrix corresponding to the sub-image to be matched;

[0061] After traversing the image to be detected, the similarity distribution in the image to be detected is obtained, and a position greater than a preset similarity threshold is found according to the similarity distribution. The position is used as the target position, and the target position is used to determine whether the target exists, and the target is re-detected based on the deep learning algorithm.

[0062] Secondary detection of targets based on deep learning algorithms, such as Figure 3As shown. Based on data retrieval, several target images of slope cutting scenes are obtained, the target images of slope cutting scenes are annotated, and a target target data set is established. All images in the target target data set are segmented according to the preset sub-image size and inter-sub-image coverage. The segmented sub-images contain part of the information in the original image. Since the inter-sub-image coverage is set, the same target may appear in multiple sub-images, which is conducive to sufficient coverage of the target. Each sub-image is independently used for target target detection, which improves the accuracy of target detection. The sub-image data set obtained after segmentation is divided into a training set and a test set according to a preset ratio. Sub-image segmentation enhances and expands the training set to a certain extent, helping the subsequent target detection model to better capture small targets and improve the performance of the model.

[0063] The YOLO v5 network is selected to build a target detection model, and the training set is used for model training. The MobileNet unit and the CBAM unit are introduced into the backbone network of YOLO v5. The MobileNet unit adjusts the number of channels through a 1*1 convolution operation, and uses a depthwise separable convolution to reduce the amount of calculation. The introduction of the MobileNet unit can effectively improve the feature extraction capability and detection accuracy of the model.

[0064] The MobileNet unit in the backbone network is connected in three channels. A CBAM unit is added to one of the channels. Combined with the location information of the network layer, the deep and shallow feature maps of the target are extracted through channel and spatial attention processing, and higher weight information is set for the typical features of the target area. The processed feature map is spliced ​​with the feature maps of the other two branches to obtain the output features, which improves the model's attention to the different state features of the target and its learning and expression capabilities.

[0065] The feature pyramid network in the original YOLO v5 network is replaced by a weighted bidirectional feature pyramid network, and the output features are imported into the weighted bidirectional feature pyramid network for multi-scale feature fusion. The weighted bidirectional feature pyramid network introduces a learnable weight factor to characterize the importance of different input features in the fusion feature map, and the target detection model is structurally optimized using top-down and bottom-up multi-scale feature fusion. The bottom-level features are passed to the top layer to retain the multi-scale features of the target, and the shallow feature map is fully utilized to improve the detection accuracy.

[0066] The fused feature map is sent to the detection head for target detection. The detection head is decoupled in the detection head and split into independent classification branches and positioning branches. The detection target is set as a target category in the classification branch. An attention mechanism is set in the detection head. The attention mechanism consists of average pooling, two fully connected layers and a normalization layer. The fused feature map is adaptively weighted, and the switching of different feature map channels is dynamically controlled according to different tasks to improve task adaptability. One-dimensional convolution is used to reduce the dimension of the weighted fused feature map, and convolution is used in the classification branch to perform classification operations to detect and identify the target. Convolution is used in the positioning branch to perform positioning and confidence operations respectively to predict the anchor frame position and confidence of the target to complete the positioning task.

[0067] Set the hyperparameters of the target detection model. After completing the preset maximum number of training rounds, use the test set to test the target detection model. When the model performance meets the preset standards, output the trained target detection model. Otherwise, continue training. Use the trained target detection model to perform a second detection on the image to be detected to confirm whether the target exists and obtain the pixel coordinates of the center of the target anchor frame. If both detections confirm that the target is lost, a target loss alarm is triggered.

[0068] S104, after confirming the existence of the target, first perform pixel-level positioning on the edge of the target, then perform sub-pixel-level positioning on the edge of the target, obtain the coordinates of the center of the ellipse of the target ring, and calculate the target displacement in combination with the historical position of the target.

[0069] It should be noted that the image to be detected of the slope surface is obtained, and the distortion correction parameters are obtained by calibration according to the camera parameters. The camera parameters are an offline step, and the camera is calibrated in advance. The distortion correction parameters are used to correct the distortion of the image to be detected, and the image is cropped according to the confirmation information of the target; the target in the cropped image to be detected is located according to the pixel coordinates of the target, and the pixel-level edge of the target is extracted using the Sobel operator, and then the sub-pixel coordinates of the current pixel edge point are calculated based on the local area pixels centered on the pixel edge point.

[0070] like Figure 4 As shown in the figure, the sub-pixel edge of the target is extracted as follows: the Sobel operator is used to calculate the gradient of the pixel-level edge of the target in the width and height directions, and the edge point of the current pixel (x i ,y i ) The gradient dx in the width direction and the gradient dy in the height direction are used to calculate the gradient direction α, α = arctan (dy / dx);

[0071] Take the current pixel edge point (x i ,y i) as the center, expand n points along the gradient direction on both sides, use bilinear interpolation to calculate the gradient values ​​of n points, perform quadratic curve interpolation on the gradient values, and obtain the vertex coordinates of the parabola;

[0072] The sub-pixel edge point of the image is calculated based on the pixel coordinates of the current pixel edge point, the gradient direction and the vertex coordinates of the parabola The calculation formula of the sub-pixel edge point is:

[0073]

[0074] Among them, x d ,y d They represent the horizontal and vertical coordinates of the vertices of the parabola respectively.

[0075] After obtaining the sub-pixel edge coordinates of the target, perform ellipse fitting on the edge to obtain the ellipse center coordinates of the target ring, and calculate the target displacement based on the historical position of the target. Specifically:

[0076] Using the extracted sub-pixel edge point coordinates, the least squares fitting method is used to obtain the coordinates of the ellipse center of the target ring. The calculation formula is:

[0077]

[0078] Among them, A, B, C, D, and E are all constants;

[0079] The current position of the target is obtained according to the coordinates of the center of the ellipse of the target ring, the historical position of the target in the previous monitoring cycle of the current target is retrieved, and the target displacement of the current target is obtained through the current position of the target and the historical position of the target.

[0080] S106, determining the structural deformation of the slope according to the target displacement, and further determining the landslide risk of the slope using the structural deformation.

[0081] It should be noted that the displacement of the slope is calculated based on the average displacement of all targets in the field of view of the slope, and a displacement abnormality alarm is generated based on the displacement. When the displacement is greater than the preset displacement threshold, a displacement abnormality alarm is generated. The preset displacement threshold is obtained by integrating and analyzing the historical displacement of the slope, obtaining the displacement change rate or the average displacement change value based on the historical displacement sequence, and predicting the displacement of the current monitoring period based on the displacement change rate or the average displacement change value as the preset displacement threshold.

[0082] Use big data to obtain a large number of slope cutting landslide instances, perform attribution processing in the slope cutting landslide instances, obtain the causal data of the landslide event, embed the causal data, use mRMR correlation analysis to extract landslide influencing factors, set weight information according to the occurrence frequency of the landslide influencing factors, and extract corresponding indicator parameter sequences from the slope cutting landslide instances based on the landslide influencing factors; pre-screen slope cutting landslide instances that meet the similarity standard according to the geographical characteristics and structural characteristics of the target slope, obtain multi-source characteristic parameters of the target slope within a preset time period based on the landslide influencing factors, and compare the multi-source characteristic parameters with the pre-screened slope cutting landslide instances. The index parameter sequence corresponding to the slope instance is matched using a dynamic time warping algorithm to obtain the average DTW (dynamic time warping) distance of the landslide influencing factor index, the average DTW distance obtained is normalized, the score of the landslide influencing factor index is obtained using the normalized average DTW distance and weight information, the scores of all landslide influencing factor indexes are integrated to obtain the landslide risk score, and the existence of landslide risk is analyzed according to the landslide risk score. When the landslide risk score is higher than the preset risk value, a landslide warning is generated, the on-site monitoring screen is captured and uploaded in real time, and the on-site screen and warning information are visualized. The landslide influencing factors include but are not limited to rainfall, temperature and humidity, and soil moisture content, etc., which can be extended to access rain gauges, thermometers and moisture meters, and moisture meters, and integrate and analyze multidimensional data, so as to obtain the deformation information of the slope behind the house in real time, continuously and accurately. In another preferred embodiment of the present invention, by installing observation targets at both ends of the crack, the position, length, width, etc. of the crack can be measured by visual monitoring to monitor the development of the slope crack.

[0083] Figure 5 A block diagram of an intelligent visual monitoring system for small-scale deformation bodies on slopes is shown.

[0084] The second embodiment of the present invention provides an intelligent visual monitoring system 5 for small-scale deformation bodies of slopes, comprising: a target deployment unit 501, a target detection unit 502, a target positioning unit 503, a displacement warning unit 504 and a monitoring visualization unit 505;

[0085] The target deployment unit is responsible for searching for targets within all fields of view on the slope surface and performing target initialization deployment;

[0086] The target detection unit is responsible for performing target detection based on a method combining machine learning and deep learning, detecting and searching all targets, and triggering a target loss alarm when a target is lost;

[0087] The target positioning unit is responsible for positioning the target at the pixel level, and then positioning the target edge at the sub-pixel level. The coordinates of the ellipse center of the target ring are obtained by using the least squares fitting method using the extracted sub-pixel edge point coordinates to generate the current position of the target.

[0088] The displacement warning unit is responsible for obtaining the slope displacement according to the current position of the target and the historical position of the target, determining whether a displacement abnormality alarm is generated and analyzing whether there is a landslide risk. When the landslide risk is higher than a preset risk value, a landslide warning is generated;

[0089] The monitoring visualization unit is responsible for visually displaying the alarm information and early warning information, and when a landslide warning occurs, it captures the on-site monitoring picture and uploads it in real time to realize remote real-time viewing of the on-site picture.

[0090] In addition, the monitoring system can be expanded to connect rain gauges, thermometers, hygrometers, and moisture meters, and integrate and analyze multi-dimensional data, thereby obtaining real-time, continuous, and accurate information on the deformation of the slope behind the house.

[0091] In the several embodiments provided in the present application, it should be understood that the disclosed method can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms. In addition, the functional units in the various embodiments of the present invention can be all integrated into one processing unit, or each unit can be used as a unit separately, or two or more units can be integrated into one unit; the above-mentioned integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0092] If the above-mentioned integrated unit of the present invention is implemented in the form of a software functional unit and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0093] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An intelligent visual monitoring method for small-scale deformation of slopes, characterized in that: The following steps are involved: Search for targets within all fields of view on the slope surface, perform target initialization deployment, and perform target detection to determine whether the target exists; After confirming the existence of the target, first perform pixel-level positioning on the edge of the target, then perform sub-pixel-level positioning on the edge of the target, obtain the coordinates of the ellipse center of the target ring, and calculate the target displacement based on the historical position of the target; The structural deformation of the slope is determined based on the target displacement, and the landslide risk of the slope is determined based on the structural deformation.

2. The intelligent visual monitoring method for small-scale deformation of slope according to claim 1 is characterized in that: Target detection is performed based on a method that combines machine learning and deep learning to detect and search for all targets, specifically: Use a machine learning algorithm for template matching to perform preliminary and rapid detection on all targets that have completed initial deployment in the field of view, obtain a preset target template image, build a sliding window based on the target template image, and use the sliding window to perform matching operations on the image to be detected; The similarity R between the sub-image in the sliding window and the target template image is calculated according to the matching operation. ccoeff , the matching operation formula is: Wherein, T represents the target template image, I represents the sub-image to be matched in the sliding window, (x, y) represents the element coordinates of the upper left corner of the current sliding window in the matrix corresponding to the sub-image to be matched, and (x′, y′) represents the element coordinates of the target template image and the current sliding window in the matrix corresponding to the sub-image to be matched; After traversing the image to be detected, the similarity distribution in the image to be detected is obtained, and a position greater than a preset similarity threshold is found according to the similarity distribution. The position is used as the target position, and the target position is used to determine whether the target exists, and the target is re-detected based on the deep learning algorithm.

3. The intelligent visual monitoring method for small-scale deformation of slope according to claim 2 is characterized in that: The target is tested again based on the deep learning algorithm, specifically: Based on data retrieval, a number of target images of slope cutting scenes are obtained, the target images of slope cutting scenes are annotated, and a target target data set is established. All images in the target target data set are segmented according to a preset sub-image size and a coverage ratio between sub-images, and the sub-image data sets obtained after segmentation are divided into a training set and a test set according to a preset ratio. The YOLO v5 network is selected to build a target detection model, and the training set is used for model training. The MobileNet unit and the CBAM unit are introduced into the backbone network of the YOLO v5. The MobileNet unit adjusts the number of channels through a 1*1 convolution operation, and uses a depth-separable convolution to reduce the amount of calculation. A three-channel dimension connection is adopted, and a CBAM unit is added to one of the channels. Combined with the position information of the network layer, the deep and shallow feature maps of the target are extracted through channel and spatial attention processing; The processed feature map is concatenated with the feature maps of the other two branches to obtain output features, the output features are introduced into a weighted bidirectional feature pyramid network for multi-scale feature fusion, and the fused feature map is sent to a detection head for target detection; The detection head is decoupled and split into an independent classification branch and a positioning branch. In the classification branch, the detection target is set as a target category. An attention mechanism is set in the detection head to adaptively weight the fused feature map to improve task adaptability. The weighted fusion feature map is reduced in dimension, the classification task and the positioning task are completed in the classification branch and the positioning branch respectively, and the target detection model is tested using the test set after iterative training. When the model performance meets the preset standard, the trained target detection model is output; Use the trained target detection model to perform a second detection on the image to be detected to confirm whether the target exists and obtain the pixel coordinates of the target. If both detections confirm that the target is lost, the target loss alarm is triggered.

4. The intelligent visual monitoring method for small-scale deformation of slope according to claim 1 is characterized in that: After confirming the existence of the target, first perform pixel-level positioning on the edge of the target, and then perform sub-pixel positioning on the edge of the target, specifically: Acquire the image to be detected of the slope surface, calibrate according to the camera parameters to obtain the distortion correction parameters, use the distortion correction parameters to correct the distortion of the image to be detected, and crop the image according to the confirmation information of the target; The target in the cropped image to be detected is located according to the pixel coordinates of the target, the pixel-level edge of the target is extracted using the Sobel operator, and then the sub-pixel coordinates of the current pixel edge point are calculated based on the local area pixels centered on the pixel edge point.

5. The intelligent visual monitoring method for small-scale deformation of slope according to claim 4 is characterized in that: The Sobel operator is used to calculate the gradient of the target pixel-level edge in the width and height directions to obtain the current pixel edge point (x i ,y i ) The gradient dx in the width direction and the gradient dy in the height direction are used to calculate the gradient direction α, α = arctan (dy / dx); Take the current pixel edge point (x i ,y i ) as the center, expand n points along the gradient direction on both sides, use bilinear interpolation to calculate the gradient values ​​of n points, perform quadratic curve interpolation on the gradient values, and obtain the vertex coordinates of the parabola; The sub-pixel edge point of the image is calculated based on the pixel coordinates of the current pixel edge point, the gradient direction and the vertex coordinates of the parabola The calculation formula of the sub-pixel edge point is: Among them, x d ,y d They represent the horizontal and vertical coordinates of the vertices of the parabola respectively.

6. The intelligent visual monitoring method for small-scale deformation of slope according to claim 1 is characterized in that: After obtaining the sub-pixel edge coordinates of the target, perform ellipse fitting on the edge to obtain the ellipse center coordinates of the target ring, and calculate the target displacement based on the historical position of the target. Specifically: Using the extracted sub-pixel edge point coordinates, the least squares fitting method is used to obtain the coordinates of the ellipse center of the target ring. The calculation formula is: Among them, A, B, C, D, and E are all constants; The current position of the target is obtained according to the coordinates of the center of the ellipse of the target ring, the historical position of the target in the previous monitoring cycle of the current target is retrieved, and the target displacement of the current target is obtained through the current position of the target and the historical position of the target; The displacement of the slope is calculated based on the average displacement of all targets in the field of view of the slope. Based on the displacement, an abnormal displacement alarm is generated and an analysis is performed on whether there is a landslide risk. When the landslide risk is higher than the preset risk value, a landslide warning is generated, and the on-site monitoring screen is captured and uploaded in real time for visual display.

7. The intelligent visual monitoring method for small-scale deformation of slope according to claim 1 is characterized in that: When monitoring the slope surface, if the ambient temperature changes significantly, the current position of the acquired target is compensated for temperature drift to compensate for positioning accuracy.

8. An intelligent visual monitoring system for small-scale deformation of slopes, characterized in that: The intelligent visual monitoring method for small-scale deformation bodies of slopes according to any one of claims 1 to 7 comprises: a target deployment unit, a target detection unit, a target positioning unit, a displacement warning unit and a monitoring visualization unit; The target deployment unit is responsible for searching for targets within all fields of view on the slope surface and performing target initialization deployment; The target detection unit is responsible for performing target detection based on a method combining machine learning and deep learning, detecting and searching all targets, and triggering a target loss alarm when a target is lost; The target positioning unit is responsible for positioning the target at the pixel level, and then positioning the target edge at the sub-pixel level. The coordinates of the ellipse center of the target ring are obtained by using the least squares fitting method using the extracted sub-pixel edge point coordinates to generate the current position of the target. The displacement warning unit is responsible for obtaining the slope displacement according to the current position of the target and the historical position of the target, determining whether a displacement abnormality alarm is generated and analyzing whether there is a landslide risk. When the landslide risk is higher than a preset risk value, a landslide warning is generated; The monitoring visualization unit is responsible for visually displaying the alarm information and early warning information, and when a landslide warning occurs, it captures the on-site monitoring picture and uploads it in real time to realize remote real-time viewing of the on-site picture.

Images