Slope creep monitoring method, device, equipment, system and storage medium

Abstract

The present application provides a slope creep monitoring method, device, equipment, system and storage medium, which relate to the field of geological disaster monitoring. The slope creep monitoring method includes using a sliding window method to calculate the similarity between a reference image and a window image in a first search area preset in a first acquisition image, and determining a first matching image with the highest similarity and greater than a preset first threshold; setting a second search area in the first acquisition image based on the coordinate range of the first matching image; obtaining the range of the first coordinates corresponding to the window image with the highest similarity to the reference image in the second search area based on a sliding window method with a step size of less than 1 pixel; and calculating and outputting a first slope displacement value based on the first coordinate range and the coordinate range of the monitoring area. By using a sliding window with a step size of less than 1 pixel to perform recognition on a finer-grained image, a more accurate image is determined for displacement calculation, so that the calculation result is more accurate, and the conclusions and alarms obtained thereby are more reliable.

Description

Technical Field

[0001] The present invention relates to the field of geological disaster monitoring, and in particular to a slope creep monitoring method, a slope creep monitoring device, a slope creep monitoring equipment, a non-volatile storage medium and a slope creep monitoring system. Background Art

[0002] Under the influence of internal and external forces of geological structures, slopes are prone to varying degrees of creep deformation over time. When creep deformation accumulates to a certain extent, it can lead to slope instability and collapse, resulting in serious consequences. Therefore, it is necessary to monitor slopes and calculate their displacement to determine their creep deformation. This allows for proactive protective measures to minimize losses caused by disasters.

[0003] Currently, the most common method for slope monitoring is to use satellite positioning, manually collect images to record geological conditions, and then calculate slope displacement by processing a large number of images. Using image processing and recognition technology to acquire target parameters is not limited by measurement range, is non-contact, and can dynamically reflect the entire deformation of the rock and soil.

[0004] However, existing technologies for calculating displacement from image processing often suffer from the effects of GPS positioning errors and the camera angle of mobile devices. The resulting images have perspective deviations from previous images, significantly impacting subsequent displacement calculations. Standardizing these images does not completely eliminate this effect, as image transformations are based on a plane, and the actual monitoring and positioning areas are often irregular in shape. This causes distortion in the observation and positioning areas after image transformation, introducing errors into subsequent displacement calculations. Consequently, the calculated results are less accurate, and significant deviations can occur when calculating displacement creep for targets at different distances, rendering existing image processing unable to meet precision requirements. Summary of the Invention

[0005] In view of this, the present invention aims to provide a slope creep monitoring method, which addresses the problem of low accuracy of image processing used in slope creep monitoring in the existing technology, improves the precision and accuracy of displacement calculation in geological monitoring, and improves the accuracy of geological disaster warning, so that relevant departments or personnel can prepare for defense in advance.

[0006] In order to achieve the above objectives, the embodiments of the present application are implemented in the following manner:

[0007] In a first aspect, an embodiment of the present application provides a slope creep monitoring method, the method comprising: setting a first search area in a first acquisition map based on a coordinate range of a monitoring area; determining the similarity between each window image of the first search area and a reference image using a sliding window method, wherein the reference image is an image of the monitoring area captured from a pre-acquisition map based on the coordinate range of the monitoring area; determining the window image in the window image with the highest similarity to the reference image and greater than a first threshold as a first matching image; setting a second search area in the first acquisition map based on the coordinate range of the first matching image, the range of the second search area being smaller than the range of the first search area; obtaining a first coordinate range corresponding to the window image with the highest similarity to the reference image in the second search area based on a sliding window method with a step size of less than 1 pixel; calculating and outputting a first slope displacement value based on the first coordinate range and the coordinate range of the monitoring area, the slope displacement value being used to characterize the slope creep condition.

[0008] In the embodiment of the present application, the sliding window method is used twice to identify the image, find the most similar image, and calculate the displacement by comparing the image with the highest similarity with the reference image, so that the calculated displacement value has higher reliability. Among them, the first sliding window method finds the image that meets the preset threshold requirements, and the second sliding window method uses a sub-pixel range search with a step size of less than 1 pixel to find the window image with higher similarity. The displacement is then calculated, which can further improve the accuracy of the calculation result. The results calculated and analyzed using more accurate displacement values ​​are also more reliable, thereby improving the accuracy of disaster warning. Using the pre-collected image as a reference and comparing it with the image collected later, the differences and changes in the monitoring areas before and after are clearly recorded, providing a basis for calculating the displacement. The two sliding window methods are used to set a search area for image recognition, which directly excludes other images far away from the monitoring location; the second search area is set according to the window image coordinates in the first search area, and a smaller search range is set, thereby improving the efficiency of image recognition.

[0009] In one embodiment, the first acquired image is a close-up image.

[0010] In one embodiment, the slope creep monitoring method further includes: when it is determined that the first matching image does not exist in the first acquisition image, setting a third search area in the second acquisition image based on the coordinate range of the preset monitoring area, and the second acquisition image is a distant image; using a sliding window method to determine the similarity between each window image in the third search area and a distant reference image, wherein the distant reference image is a window image obtained from the pre-collected image and has the highest similarity with the reference image; determining the window image with the highest similarity with the reference image and greater than a second threshold among the window images as the second matching image; setting a fourth search area in the second acquisition image based on the coordinate range of the second matching image, the range of the fourth search area being smaller than the range of the third search area; obtaining a second coordinate range corresponding to the window image with the highest similarity with the distant reference image in the fourth search area based on a sliding window method with a step size of less than 1 pixel; and calculating and outputting a second slope displacement value based on the second coordinate range and the coordinate range of the monitoring area.

[0011] In the embodiment of the present application, if a window image that meets the first threshold requirement cannot be found in the first acquisition image, the search range is expanded to obtain a larger perspective image, namely the second acquisition image. The image that meets the requirement is found in the second acquisition image, thereby improving the probability of finding the window image with the highest similarity. Furthermore, a two-sliding window search method (the latter sliding window search is a sub-pixel range search) is also used in the perspective image search, resulting in more precise search and calculated results, and more accurate output alarm results.

[0012] In one embodiment, when it is determined that the second matching image does not exist in the second acquisition image, a search result is output, where the search result indicates that the monitoring area is not found in either the first acquisition image or the second acquisition image.

[0013] In this embodiment, when no image with a similarity reaching the first threshold or the second threshold is found, the search result of no monitoring area is output, which makes it easier for relevant personnel to confirm whether the problem is caused by excessive slope creep, image offset or image recognition, and to take various measures for adjustment or warning.

[0014] In one embodiment, before setting the first search area in the first acquisition map based on the coordinate range of the preset monitoring area, the slope creep monitoring method further includes: obtaining the pre-acquisition map, the pre-acquisition map including a close-up map and a distant map; intercepting the reference image in the close-up map based on the coordinate range of the monitoring area; and obtaining the distant reference map in the distant map based on a sliding window method and the reference image.

[0015] In one embodiment, the method of obtaining the distant reference image from the distant image based on the sliding window method and the reference image includes: determining a target reduction ratio from a preset reduction ratio range one by one within a preset number of times, the reduction ratio range being pre-set based on the scaling ratio of the near-view image and the distant image; reducing the reference image according to the target reduction ratio; calculating the similarity between the reduced reference image and the window image in the distant image based on the sliding window method; recording the window image with the highest similarity to the reduced reference image as the distant reference image, and recording the corresponding reduction ratio, the reduction ratio being the target reduction ratio when obtaining the window image with the highest similarity.

[0016] In this embodiment, an image of the monitoring area is captured from a pre-collected close-up image as a reference image, and then a window image with the highest similarity to the reference image is obtained from a pre-collected long-range image through a sliding window method as a long-range reference image. This ensures that when the displacement value is subsequently calculated, whether the collected image is a long-range image or a close-up image, there is a control group, which makes it easier to clarify the changes in the front and rear slopes.

[0017] In one embodiment, after calculating and outputting the second slope displacement value, the slope creep monitoring method further includes: based on the reduction ratio corresponding to the reference image and the distant reference image, converting the second slope displacement value into a slope displacement value adapted to the near view image and then outputting the converted value.

[0018] In this embodiment, the displacement value calculated using the distant view image can be converted into a displacement value output that is consistent with the proportion in the near view image, so that relevant personnel can directly obtain the results and make judgments based on the results in order to take early warning and prevention measures.

[0019] In a second aspect, an embodiment of the present application provides a slope creep monitoring device, comprising: a setting module for setting a first search area in a first acquisition map based on a coordinate range of a monitoring area; a determination module for determining the similarity between each window image of the first search area and a reference image using a sliding window method, wherein the reference image is an image of the monitoring area captured from a pre-acquisition map based on the coordinate range of the monitoring area; the determination module is further used to determine the window image in the window image with the highest similarity to the reference image and greater than a first threshold as a first matching image; the setting module is further used to set a second search area in the first acquisition map based on the coordinate range of the first matching image, the range of the second search area being smaller than the range of the first search area; the determination module is further used to obtain, based on a sliding window method with a step size of less than 1 pixel, a first coordinate range corresponding to the window image with the highest similarity to the reference image in the second search area; the determination module is further used to calculate and output a first slope displacement value based on the first coordinate range and the coordinate range of the monitoring area, and the slope displacement value is used to characterize the slope creep situation.

[0020] In a third aspect, an embodiment of the present application provides a slope creep monitoring device, comprising a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the slope creep monitoring method as described in any one of the first aspects, or realizes the function of the slope creep monitoring device as described in the second aspect, by calling the computer-readable instructions.

[0021] In a fourth aspect, an embodiment of the present application provides a non-volatile storage medium storing computer-readable instructions. The processor executes the slope creep monitoring method as described in any one of the first aspects, or realizes the function of the slope creep monitoring device as described in the second aspect, by calling the computer-readable instructions.

[0022] In the fifth aspect, an embodiment of the present application provides a slope creep monitoring system, comprising the slope creep monitoring device provided in the third aspect; and a camera device for acquiring the pre-collected map, the first collection map and the second collection map based on the control instructions of the slope creep monitoring device.

[0023] Other features and advantages of the present disclosure will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by practicing the above-mentioned technology of the present disclosure.

[0024] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments of the present invention are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A flow chart of the first case of the slope creep monitoring method provided in an embodiment of the present application;

[0027] Figure 2 A flow chart of the second case of the slope creep monitoring method provided in an embodiment of the present application;

[0028] Figure 3 A structural block diagram of a slope creep monitoring device provided in an embodiment of the present application;

[0029] Figure 4 This is a structural block diagram of the slope creep monitoring system provided in an embodiment of the present application.

[0030] Icons: slope creep monitoring device 300; setting module 310; determination module 320; slope creep monitoring system 400; camera device 410; slope creep monitoring system 440; display module 450; disaster warning module 460. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0032] Before introducing the specific implementation methods of the present application in detail, a brief introduction to the sliding window method involved in the present application is first given.

[0033] When processing an image using the sliding window method, the process can be described as follows: within an image of size W×H, a window of size w×h is moved according to a certain pattern (W>>w, H>>h). A series of operations are performed on the pixel values ​​within the window after each slide. After the operation, the window is slid rightward or downward by a certain step size (for example, 1 pixel, 2 pixels, etc.), and so on, until the entire image is processed. For example, the similarity between the window image of a reference image and the original image is calculated. If the reference image is 5×5 pixels and the original image is 15×15 pixels, the window image in the original image has the same size as the reference image, namely 5×5 pixels. Starting from a set position in the original image, the similarity between the window image in the original image and the reference image is calculated using feature extraction and comparison. The window is moved 1 pixel in the same direction. The calculation is repeated a number of times (15-5+1)×(15-5+1). The image with the highest similarity is determined as the desired image.

[0034] Next, the specific implementation methods of this application are introduced in detail.

[0035] Please refer to Figure 1 , Figure 1 This is a flowchart of a first embodiment of a slope creep monitoring method provided in an embodiment of the present application. The method may include the following steps:

[0036] Before monitoring the slope creep, images are collected in advance as comparison images, and reference images of each monitoring area are intercepted from the comparison images according to the different monitoring areas.

[0037] In one embodiment, the reference image may be obtained in the following manner.

[0038] First, a pre-collected image is obtained using a camera device. The pre-collected image may include a near-view image and a far-view image. It is understood that the pre-collected image may be stored in a memory and, when needed, directly retrieved from the memory. Alternatively, the pre-collected image may be acquired using the camera device when needed.

[0039] Next, a reference image is captured from the close-up image in the pre-collected image.

[0040] In this embodiment, the coordinate range of the monitoring area is set in the near-view image, and the monitoring area is cut out from the near-view image to form the reference image. For example, the monitoring area is represented as A(x, y, w, h), where x and y are the coordinates of the monitoring area at the upper left corner of the image, w and h are the width and height of the monitoring area, r is the monitoring range in pixels, t is the threshold, b is the near-view image, and c is the distant view image. The monitoring area is set and cut out from the near-view image as the reference image, denoted as Ab.

[0041] Based on the sliding window method and the reference image, a distant view reference image is obtained from the distant view image in the pre-collected image.

[0042] In one embodiment, obtaining a distant view reference image includes: determining a target reduction ratio from a preset reduction ratio range one by one within a preset number of times, where the reduction ratio range is set based on the ratio of a near view image to a distant view image in a pre-collected image; reducing the reference image according to the target reduction ratio; calculating the similarity between the reduced reference image and the window image in the distant view image based on a sliding window method; recording the window image with the highest similarity to the reduced reference image as the distant view reference image, and recording the reduction ratio and the image position coordinates at which the reduction ratio is obtained, where the reduction ratio is the target reduction ratio when obtaining the window image with the highest similarity.

[0043] In this embodiment, in the process of obtaining the distant view reference image using the sliding window method, in order to improve the speed and avoid searching the entire image, an area can be drawn on the distant view image, which includes the image Ab, and a sliding window search is performed in the area to improve efficiency.

[0044] In this embodiment, the similarity between the reference image and the distant image is calculated using a sliding window method, and the image with the highest similarity is obtained as the distant reference image. Exemplarily, the distant reference image is obtained as follows: given a ratio m between the near-view image b and the distant image c, where m is between (0-1), the reduction ratio range can be set to [m-0.1, m+0.1]. It should be noted that m here is an approximate ratio that can be roughly judged by the human eye or roughly estimated by the ratio value scaled by the camera device. The specific ratio is subject to the result of the subsequent multi-scale sliding window calculation. Similarly, [m-0.1, m+0.1] is also an example and is not necessarily 0.1, but can also be 0.2 or 0.3, depending on the performance requirements. No specific limitation is made here. A target reduction ratio is selected within the reduction ratio range, and the reference image Ab is reduced according to the target reduction ratio. The reduced reference image Ab is applied to the distant image c using a sliding window method, and the similarity between Ab and each window image is calculated. Different target reduction ratios are selected within the reduction ratio range to calculate the similarity between the reduced reference image Ab and the window images of the distant image c. The number of repetitions can be set based on performance or accuracy requirements. The window image with the highest similarity is designated as the distant reference image Ac, and the target reduction ratio for obtaining the window image with the highest similarity is recorded as the reduction ratio Ratiom.

[0045] S111 , setting a first search area in a first acquisition map based on a coordinate range of the monitoring area.

[0046] In one embodiment, when the camera device obtains a pre-collected image, the preset points are called to adjust the settings of the camera device.

[0047] In this embodiment, the pre-collected images are divided into a close-up image and a long-range image of the monitored area. When the camera device obtains the pre-collected images, the camera calls preset points for shooting. The preset point information is parameters pre-stored in the camera, including parameters such as shooting angle and focal length. The user can save different preset points for the close-up image and the long-range image, respectively, to facilitate the camera to call corresponding preset points when capturing different images. For example, the camera information for capturing the close-up image is set to the first preset point, and the camera information for capturing the long-range image is set to the second preset point. When capturing the close-up image or the long-range image, the camera calls the first preset point or the second preset point, respectively, to adjust the camera settings and capture the required image. Thus, the camera can directly call the angle, focal length, and other parameters used when capturing the image, so that the images captured before and after will not differ due to factors such as shooting angle and camera settings. The only difference is due to changes in slope or natural scenery. The calculated displacement value is more accurate.

[0048] In one embodiment, the camera adjusts the camera settings according to the preset points. At regular intervals, the camera uses the same preset points as when capturing the close-up image to capture a close-up image a, i.e., a first captured image. Exemplarily, the camera calls the first preset point to obtain the first captured image.

[0049] In one embodiment, the first search area can be set to (xr, yr, w+r*2, h+r*2). The letters in the first search area also represent the horizontal and vertical coordinates, width, height and range of the upper left vertex of the image, which will not be repeated later. By setting the first search area, the search range can be narrowed. The range of the first search area needs to be set larger than the range of the monitoring area. For example, it can be set to twice the range of the monitoring area or other sizes. Different sizes are set according to specific circumstances to facilitate image recognition and search for the monitoring area. The maximum and minimum ranges that can be set for the monitoring area are determined by the range r.

[0050] S112: Determine the similarity between each window image in the first search area and the reference image using a sliding window method.

[0051] In one embodiment, in the first search area, a window image is extracted and compared with a reference image in a sliding window manner. The window size of the sliding window is equal to the width and height of the reference image Ab, and the sliding step is 1 pixel. The similarity between the window image and the reference image Ab is calculated. Exemplarily, the sliding step in this step should be greater than or equal to 1 pixel, that is, it can be 1, 2, 3 or other lengths, which is set according to requirements and device performance.

[0052] In this embodiment, the window image may be compared with the reference image Ab in the following manner to determine the similarity between the window image and the reference image Ab.

[0053] In one embodiment, the present invention further provides a model for calculating image similarity. This model is pre-trained and can be used directly to calculate the input image feature vector. A similarity result can be obtained by calculating the mean square difference between the feature vectors of two images. The similarity result is the mean square difference between the feature vectors. A smaller mean square difference indicates a higher similarity between the two images.

[0054] In this embodiment, an image similarity calculation model is provided. The network architecture constructed by the image similarity calculation model includes: an input layer, a convolution layer, a batch normalization layer, a pooling layer, a Dropout layer, and a global average pooling layer. The specific number of layers and input size can be increased or decreased according to actual conditions.

[0055] In this embodiment, the similarity calculation process first feeds all images to be calculated into the model to obtain the feature vectors of each image. The feature vector of the reference image is then compared with the feature vectors of each sliding window to calculate the mean square error (MSE), i.e., the similarity. The image with the smallest MSE is the most similar image, and the position coordinates of the most similar image are then obtained. It is understood that the image similarity calculation model is used to calculate the similarity of at least two input images by extracting the feature vectors of the input images, comparing them pairwise, obtaining the mean square error between the two feature vectors, and outputting the corresponding similarity result. Exemplarily, the reference image and any window image captured in the first search area set in the first acquisition image are input into the image similarity calculation model. The image similarity calculation model outputs the feature vectors between the reference image and the input window image, and outputs the similarity result by calculating the mean square error of the feature vectors.

[0056] In this embodiment, the input image can be multiple images. The image similarity calculation model calculates and obtains the feature vectors of each input image. The similarity between the input image and the comparison image is calculated by calculating the mean squared difference between each input image feature vector and the feature vector of the image being compared. The image with the highest similarity is then output. For example, features of a reference image are extracted, and all window images in a first search region are input into the image similarity calculation model. The model extracts and calculates feature vectors for each window image, compares them with the feature vector of the reference image, and calculates the mean squared difference. A smaller mean squared difference indicates a higher similarity. Finally, all similarities are compared, and the highest similarity value and the window image are output.

[0057] In one embodiment, the present invention provides a method for training an image similarity calculation model, including: collecting images of all preset points in each time period; and training an independent model for each preset point.

[0058] In this embodiment, the collected data includes images of all preset points. The collection method can be that the same camera adjusts its own parameters based on the preset points to obtain images, or different cameras collect images based on their own stored preset points and send them to the upper device.

[0059] In this embodiment, the process of training an independent model for each preset point includes: selecting any preset point and obtaining a negative sample set; obtaining a positive sample set; combining the negative sample set and the positive sample set as training samples of the preset point and inputting the model into the model to train the model; repeating the above steps to train the preset point until the training is completed using all the collected images of the preset point, and then it is determined that a complete training is completed; repeating the complete training multiple times, and calculating the loss value during each training until the calculated loss value stops decreasing continuously, and using the model at this time as the image similarity calculation model.

[0060] In this embodiment, different images obtained at different positions of the same captured image at the same preset point using the sliding window method are negative samples; conversely, images at the same position in different captured images obtained using the sliding window method are positive samples. Exemplarily, the process of obtaining a negative sample set includes: selecting an image data of any preset point; extracting an image using the sliding window method within the preset search area (xr, yr, w+r*2, h+r*2), wherein the window width and height of the sliding window are equal to the width and height of the monitoring area, and the sliding step size is 1, so that m=(2r+1) 2 -1 different image; a portion of the acquired images is selected to form a negative sample set Tn. The process of acquiring the positive sample set includes: the preset point used to acquire the positive sample set is the same as the preset point used to acquire the negative sample set, selecting the position coordinates of any image at the preset point, selecting multiple different acquired images at the preset point, and cutting out images from the acquired images with the same coordinates and size. The obtained images are the positive sample set Tp. The positive sample set and the negative sample set are combined and used as a batch input for model training to train the model. A batch consists of several positive samples and several negative samples. Specifically, the negative sample set only requires a random extraction of a portion from m images, which is related to the batch_size of the model training. For example, there is a ratio between positive samples and negative samples, such as 5:45, that is, the number of positive samples input each time is 5, the number of negative samples input is 45, and the batch_size is 50. When all the acquired images of the preset point are input into the model for training in the same way, it is an epoch, and the model is considered to have completed one training. Therefore, while ensuring that image features at different positions are distinguished, it also ensures that image features at the same position are as similar as possible, thereby ensuring the effectiveness of subsequent similarity calculations.

[0061] In this embodiment, the output result of the model is calculated by the Loss function to determine whether the model training is completed. Specifically, the Loss function design includes: the feature vector set of the positive sample set Tp is Vp, Vp={P1,P 2, P3...P n}, similarly, the feature vector set of the negative sample set Tn is Vn, Vn={N1,N 2, N 3. ..N m}. Each eigenvector in Vp and Vn can be represented as a floating point vector (v1, v2, v3...v k ), where k is consistent with the model output. For ease of calculation, both positive and negative samples are compared with P1 to calculate the mean square error. At the same time, to ensure that the farther the distance between the negative sample and P1, the smaller the loss value, the loss value of the negative sample is expressed as the inverse of the mean square error. Therefore, the loss value formula for the positive sample is:

[0062]

[0063] The loss value formula for negative samples is:

[0064]

[0065] In this embodiment, through a customized loss function and training method, samples of the same and different classes are put into the same batch for training. The loss design increases the feature differences of samples of different classes and reduces the feature differences of samples of the same class, so that the feature vectors extracted by the model are more adaptable to the requirements of subsequent similarity calculation.

[0066] S113 , determining a window image among the window images that has the highest similarity with the reference image and is greater than a first threshold as a first matching image.

[0067] In one embodiment, the first threshold is manually selected and set based on demand or device performance.

[0068] In this embodiment, the window image with the highest similarity obtained by the sliding window method, that is, the first matching image, can be recorded as Aa, the coordinate position of the first matching image is (x', y', w', h'), the similarity is s, and the similarity s requires a preset first threshold t1. When s is greater than t1, the target can be found within the first search range, that is, the monitoring area, and S114 is executed; if the first matching image does not exist, that is, the similarity of the window image with the highest similarity is less than the preset first threshold, S120 is executed.

[0069] S114: Set a second search area in the first acquisition image based on the coordinate range of the first matching image.

[0070] In this embodiment, the range of the second search area is smaller than that of the first search area. Exemplarily, the second search area can be set to (x'-1, y'-1, w'+2, h'+2). It is understandable that the coordinate range of the second search area is set based on the coordinates of the first matching image, and the range of the second search area is much smaller than the range of the first search area. It can be based on the coordinates of the first matching image, with the horizontal and vertical coordinates expanded by 1 pixel, and the width and height expanded by two pixels. It can be set according to actual conditions or device performance. The second search area can be a part of the first search area, or it can partially overlap with the first search area. When the first matching image is an edge window image, the setting part of the second search area is outside the first search area. This is only to illustrate the difference between the first search area and the second search area, without specific limitation.

[0071] S115 , based on a sliding window method with a step size of less than 1 pixel, obtaining a first coordinate range corresponding to a window image having the highest similarity to the reference image in the second search area.

[0072] In this embodiment, when a similar monitoring area has been found, the target is further searched in the sub-pixel range to narrow the range of the window image and improve the image accuracy. The sub-pixel means that the pixel is less than 1.

[0073] In this embodiment, the sub-pixel image is extracted by, but not limited to, obtaining it through the getRectSubPix interface of OpenCV.

[0074] In this embodiment, a sliding window method with a sliding step size of less than 1 pixel is used in the second search area to calculate the similarity between the reference image Ab and the window image in the second search area. The specific process is the same as the sliding window method described above and will not be repeated here. It should be noted that the step size of the sliding window method here is a decimal less than 1 pixel. The specific setting value can be 0.1, 0.01, or other values, selected based on accuracy and performance requirements. The sub-pixel range search method is used here to obtain a more accurate image, improve the measurement accuracy when calculating displacement, and obtain more precise results.

[0075] In this embodiment, the window image Aa1 with the highest similarity is obtained by a sliding window method with a sliding step size of less than 1 pixel as the required target image, and the coordinates of its corresponding position are (x'', y'', w'', h''), that is, the first coordinate range, and step S116 is executed.

[0076] S116: Calculate and output a first slope displacement value based on the first coordinate range and the coordinate range of the monitoring area.

[0077] In one embodiment, the first slope displacement value can be calculated by subtracting the coordinate range A(x, y, w, h) of the detection area from the first coordinate range Aa1 to obtain the first slope displacement value. For example, the lateral displacement of the monitoring area is: x''-x, and the longitudinal displacement is: y''-y. The slope displacement value can assist relevant technical personnel in analyzing the slope creep situation, obtaining monitoring results, and taking preventive measures in advance to reduce losses caused by disasters. Please refer to Figure 2 , Figure 2 This is a flow chart of a second case of a slope creep monitoring method provided in an embodiment of the present application. The slope creep monitoring method may include the following steps:

[0078] S120: Determine whether a first matching image does not exist in the first acquisition image.

[0079] In S113 , if no first matching image is found in the first search area, that is, if no first matching image exists (the similarity of the window image with the highest similarity is less than a preset first threshold), S120 is executed.

[0080] S121: Set a third search area in the second acquisition map based on the coordinate range of the preset monitoring area.

[0081] In this embodiment, the second captured image is a distant view image.

[0082] In one embodiment, camera settings are adjusted based on the preset point for acquiring a distant view image, and a distant view image d, i.e., the second acquisition image, is acquired. It is understood that the first acquisition image is a close-up image, while the second acquisition image is a distant view image. The window image position with the highest similarity to the distant view reference image Ad is searched in the second acquisition image d. The specific process is similar to searching for the reference image Aa in the first acquisition image a, and can be seen in step S110. This description will not be repeated here, and only the differences will be discussed.

[0083] In this embodiment, the third search area can be set to (xr, yr, w+r*2, h+r*2). It should be noted that the basis for setting the third search area is to obtain the coordinate position of the reduced magnification image, that is, the x, y, w, and h here are the position coordinates of the reference image obtained when the reduction ratio was previously confirmed on the distant image.

[0084] S122: Determine the similarity between each window image in the third search area and the distant view reference image using a sliding window method.

[0085] In this embodiment, in the third search area, the similarity between the window image and the distant reference image Ac is calculated in a sliding window manner, and the sliding step size may be the same as the sliding step size in step S112.

[0086] S123: Determine, among the window images, a window image that has the highest similarity with the reference image and is greater than a second threshold as a second matching image.

[0087] In this embodiment, the second threshold can be set to the same value as the first threshold, or it can be set based on the similarity between the reference image and the perspective reference image. This value is determined by multiplying the similarity between reference image Ab and perspective reference image Ac by a percentage. The specific threshold value is selected based on requirements or device performance. For example, if the similarity between reference image Ab and perspective reference image Ac is 90%, and the percentage is set to 80%, the second threshold value is 72%.

[0088] In this embodiment, a second matching image with the highest similarity and greater than a second threshold t2 is obtained. The second threshold t2 may be the same as or different from the first threshold t1 and is set according to needs and accuracy requirements. After the second matching image is obtained, S124 is executed. If the second matching image does not exist in the second acquisition image, S130 is executed.

[0089] S124: Set a fourth search area in the second acquisition image based on the coordinate range of the second matching image.

[0090] In this embodiment, the fourth search area is smaller than the third search area.

[0091] S125 , based on a sliding window method with a step size of less than 1 pixel, obtain a second coordinate range corresponding to a window image having the highest similarity to the distant reference image in the fourth search area.

[0092] In one embodiment, a sliding window method with a step size of less than 1 pixel is used to further search for a window image similar to the distant reference image Ac in the set fourth search area. The specific process is similar to step S115 and will not be repeated here. Only the differences are explained.

[0093] In this embodiment, since the near view image and the far view image are different, the far view reference image Ac is used when calculating the similarity of the second acquired image. Finally, the image Ad1 with the highest similarity is determined, and the position coordinates of Ad1 are in the second coordinate range, and step S126 is executed.

[0094] S126: Calculate and output a second slope displacement value based on the second coordinate range and the coordinate range of the monitoring area.

[0095] In one embodiment, the second slope displacement value is calculated by subtracting the coordinate range A (x, y, w, h) of the detection area from the second coordinate range Ad1. The second slope displacement value is the displacement value in the perspective image and can also be used by relevant personnel to analyze geological creep.

[0096] After S126, S127 (not shown in the figure) may be executed to calculate and output the second slope displacement value, and then, based on the reduction ratio of the reference image and the distant reference image, convert the second slope displacement value into a slope displacement value that matches the first near-view image and output it.

[0097] In one embodiment, when there is no second matching image in the second acquisition image, that is, the similarity of the window image with the highest similarity is less than a preset first threshold, S130 is executed.

[0098] In this embodiment, the displacement values ​​in the distant view image are converted into displacement values ​​in the near view image before being output, and the output is scaled uniformly, so as to facilitate statistical analysis of monitoring results of different areas.

[0099] S130: Determine whether the second matching image does not exist in the second acquisition image.

[0100] S131, output the search results.

[0101] In this embodiment, when it is determined that the second matching image does not exist in the second acquisition image, a search result is output, where the search result indicates that the monitoring area is not found in either the first acquisition image or the second acquisition image.

[0102] In this embodiment, when image recognition fails to find a monitoring area, a result indicating that no monitoring area was found is output. For example, this may be r+1, i.e., the scale of the distant view image is converted to the scale of the near view image and then added by 1. The specific form of the output result is set according to user needs and is not specifically limited in this application. This facilitates timely troubleshooting and resolution by relevant technical personnel.

[0103] In the slope creep monitoring method provided in the embodiment of the present application, the process of obtaining the reference image and the distant reference image can be obtained by the method provided in the embodiment of the present application, or the reference image can be directly obtained by other methods or devices.

[0104] Therefore, the embodiment of the present application can calculate more accurate slope displacement values ​​for technical personnel to analyze, making the analysis results more reliable, thereby improving the accuracy of disaster warnings.

[0105] See also Figure 3 Based on the same inventive concept, an embodiment of the present application provides a slope creep monitoring device 300 . The slope creep monitoring device 300 includes a setting module 310 and a determining module 320 .

[0106] In one embodiment, the setting module 310 is configured to set a first search area in the first acquisition map based on a coordinate range of the monitoring area.

[0107] In one embodiment, the determination module 320 is configured to determine the similarity between each window image of the first search area and a reference image using a sliding window method, wherein the reference image is an image of the monitoring area captured from a pre-collected image based on a coordinate range of the monitoring area.

[0108] The determination module 320 is further configured to determine, among the window images, a window image having the highest similarity with the reference image and greater than a first threshold, as a first matching image.

[0109] The setting module 310 is further configured to set a second search area in the first acquisition image based on the coordinate range of the first matching image, where the range of the second search area is smaller than the range of the first search area.

[0110] The determination module 320 is further configured to obtain, based on a sliding window method with a step size of less than 1 pixel, a first coordinate range corresponding to a window image having the highest similarity to the reference image in the second search area;

[0111] The determination module 320 is further configured to calculate and output a first slope displacement value based on the first coordinate range and the coordinate range of the monitoring area, where the slope displacement value is used to characterize the slope creep condition.

[0112] Based on the same inventive concept, one embodiment of the present application provides a slope creep monitoring device. The slope creep monitoring device includes a memory and a processor. The memory stores computer-readable instructions, and the processor invokes the computer-readable instructions to execute any of the aforementioned slope creep monitoring methods or implement the functions of the aforementioned slope creep monitoring device.

[0113] Based on the same inventive concept, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the computer executes any of the aforementioned slope creep monitoring methods.

[0114] In the embodiments provided herein, it should be understood that the disclosed methods and devices may also be implemented in other ways. The device embodiments described above are merely illustrative. The functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0115] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0116] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

[0117] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0118] See also Figure 4 Based on the same inventive concept, an embodiment of the present application provides a slope creep monitoring system 400 . The slope creep monitoring system 400 includes a camera 410 , a slope creep monitoring device 440 , a display module 450 , and a disaster warning module 460 .

[0119] In one embodiment, the camera device 410 is used to obtain the pre-collected images, the first collected images, and the second collected images based on the control instructions of the slope creep monitoring device.

[0120] In this embodiment, the camera device 410 can be a camera that supports the pan-tilt function and has the function of calling preset points when shooting. In subsequent use, the pan-tilt can call the preset points to adjust the camera parameter settings, for example, adjust the shooting direction, focus, focus or other methods to adjust the settings; it can also be connected to the video platform via the network to upload the captured images.

[0121] In one embodiment, the slope creep monitoring device 440 is configured to obtain a first slope displacement value and a second slope displacement value based on the first acquisition image and the second acquisition image, a reference image, and a distant reference image.

[0122] In one embodiment, the slope creep monitoring device 440 is further used to obtain a reference image and a distant reference image based on a preset acquisition image; the slope creep monitoring device 440 is further used to convert the second slope displacement value into a slope displacement value adapted to the first near-view image and output it.

[0123] In this embodiment, the slope creep monitoring device 440 is used to analyze the collected images, calculate the similarity of the images, and perform image processing processes such as capturing window images and monitoring area images in the images. It is also used to calculate the displacement of the monitoring area based on the results of image processing.

[0124] In one embodiment, the slope creep monitoring device 440 is also used to connect to cameras of different models from different manufacturers, and provide an interface for the data acquisition module to collect data and control the camera pan / tilt platform.

[0125] In this embodiment, the slope creep monitoring device 440 may have the function of connecting to different cameras, or may have this function indirectly by connecting to other devices or modules.

[0126] In one embodiment, the slope creep monitoring device 440 is further used to obtain data collected by a camera device and adjust the camera device in real time according to the needs of data analysis.

[0127] In this embodiment, the slope creep monitoring device 440 obtains data collected by the camera device and adjusts the camera device by directly connecting to the camera device or indirectly connecting to the cloud or other devices to achieve data collection and camera adjustment.

[0128] In one embodiment, the display module 450 is connected to the slope creep monitoring device 440 and is used to present the collected data and the results of the analysis by the processing module to the user. The presentation method can be different methods such as charts and texts.

[0129] In one embodiment, the disaster warning module 460 is connected to the slope creep monitoring device 440 and is used to trigger a warning based on pre-set warning conditions and data analysis results.

[0130] In this embodiment, the disaster warning module 460 can receive the analysis results of the images and data of the slope creep monitoring device 440, trigger an alarm according to pre-set warning conditions, and send an alarm message to relevant personnel.

Claims

1. A slope creep monitoring method, characterized in that: include: Setting a first search area in the first acquisition map based on the coordinate range of the monitoring area; Determining the similarity between each window image of the first search area and a reference image using a sliding window method, wherein the reference image is an image of the monitoring area captured from a pre-collected image based on a coordinate range of the monitoring area; Determine, among the window images, a window image that has the highest similarity with the reference image and is greater than a first threshold as a first matching image; Setting a second search area in the first acquisition image based on the coordinate range of the first matching image, wherein the range of the second search area is smaller than the range of the first search area; Obtaining, based on a sliding window method with a step size of less than 1 pixel, a first coordinate range corresponding to a window image in the second search area having the highest similarity to the reference image; Calculating and outputting a first slope displacement value based on the first coordinate range and the coordinate range of the monitoring area, wherein the slope displacement value is used to characterize the slope creep condition; The first acquisition image is a close-up image, and the camera is set to a first preset point when acquiring the first acquisition image; The slope creep monitoring method further includes: when it is determined that the first matching image does not exist in the first acquisition image, setting a third search area in the second acquisition image based on the coordinate range of the monitoring area, wherein the second acquisition image is a distant view image; and when acquiring the second acquisition image, setting the camera to a second preset point; Determining the similarity between each window image in the third search area and a distant reference image using a sliding window method, wherein the distant reference image is a window image obtained from the pre-collected image and having the highest similarity to the reference image; Determine, among the window images, a window image that has the highest similarity with the reference image and is greater than a second threshold as a second matching image; Setting a fourth search area in the second acquisition image based on the coordinate range of the second matching image, wherein the range of the fourth search area is smaller than the range of the third search area; Obtaining, based on a sliding window method with a step size of less than 1 pixel, a second coordinate range corresponding to a window image in the fourth search area having the highest similarity to the distant reference image; Based on the second coordinate range and the coordinate range of the monitoring area, a second slope displacement value is calculated and output.

2. The method according to claim 1, characterized in that The slope creep monitoring method further includes: when it is determined that the second matching image does not exist in the second acquisition image, outputting a search result, wherein the search result indicates that the monitoring area is not found in either the first acquisition image or the second acquisition image.

3. The method according to claim 1, characterized in that Before setting the first search area in the first acquisition map based on the coordinate range of the preset monitoring area, the slope creep monitoring method further includes: Acquire the pre-collected image, wherein the pre-collected image includes a near-view image and a far-view image; intercepting the reference image in the close-up image based on the coordinate range of the monitoring area; Based on a sliding window method and the reference image, the distant view reference image is obtained in the distant view image.

4. The method according to claim 3, characterized in that The acquiring the distant view reference image from the distant view image based on the sliding window method and the reference image includes: within a preset number of times, determining a target reduction ratio from a preset reduction ratio range, wherein the reduction ratio range is pre-set based on a scaling ratio between the near view image and the distant view image; Reducing the reference image according to the target reduction ratio; Based on the sliding window method, calculating the similarity between the reduced reference image and the window image in the perspective image; Recording the window image with the highest similarity to the reduced reference image as the distant reference image, and recording the corresponding reduction ratio, where the reduction ratio is the target reduction ratio when obtaining the window image with the highest similarity; After calculating and outputting the second side slope displacement value, the method further includes: based on the reduction ratios corresponding to the reference image and the distant reference image, converting the second side slope displacement value into a side slope displacement value adapted to the near view image and then outputting the converted side slope displacement value.

5. A slope creep monitoring device, characterized in that: include: A setting module, configured to set a first search area in the first acquisition map based on a coordinate range of the monitoring area; a determination module, configured to determine, using a sliding window method, a similarity between each window image of the first search area and a reference image, wherein the reference image is an image of the monitoring area captured from a pre-collected image based on a coordinate range of the monitoring area; The determining module is further configured to determine, among the window images, a window image having the highest similarity with the reference image and greater than a first threshold value as a first matching image; The setting module is further configured to set a second search area in the first acquisition image based on a coordinate range of the first matching image, wherein the range of the second search area is smaller than the range of the first search area; The determining module is further configured to obtain, based on a sliding window method with a step size of less than 1 pixel, a first coordinate range corresponding to a window image in the second search area having the highest similarity to the reference image; The determining module is further configured to calculate and output a first slope displacement value based on the first coordinate range and the coordinate range of the monitoring area, wherein the slope displacement value is used to characterize the slope creep condition; The first acquisition image is a close-up image, and the camera is set to a first preset point when acquiring the first acquisition image; The setting module is further configured to, when determining that the first matching image does not exist in the first acquisition image, set a third search area in the second acquisition image based on the coordinate range of the monitoring area, wherein the second acquisition image is a distant view image; and when acquiring the second acquisition image, the camera is set to a second preset point; The determination module is further configured to determine, using a sliding window method, a similarity between each window image in the third search area and a distant reference image, wherein the distant reference image is a window image obtained from the pre-collected image and having the highest similarity to the reference image; and determine, among the window images, a window image having the highest similarity to the reference image and greater than a second threshold value as a second matching image; The setting module is further configured to set a fourth search area in the second acquisition image based on a coordinate range of the second matching image, wherein the range of the fourth search area is smaller than the range of the third search area; The determination module is further configured to obtain, based on a sliding window method with a step size of less than 1 pixel, a second coordinate range corresponding to a window image in the fourth search area having the highest similarity to the distant reference image; and calculate and output a second slope displacement value based on the second coordinate range and the coordinate range of the monitoring area.

6. A slope creep monitoring device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the slope creep monitoring method according to any one of claims 1 to 4 by calling the computer-readable instructions.

7. A non-volatile storage medium, characterized in that: Computer-readable instructions are stored therein, and the processor executes the slope creep monitoring method according to any one of claims 1 to 4 by calling the computer-readable instructions.

8. A slope creep monitoring system, characterized in that: include: The slope creep monitoring device according to claim 6; The camera device is used to obtain the pre-collected images, the first collected image and the second collected image based on the control instructions of the slope creep monitoring equipment.

Images