Slope anomaly monitoring method, device and equipment and storage medium

Abstract

The invention discloses a slope anomaly monitoring method, device and equipment, a storage medium and a vehicle. The method comprises the following steps: acquiring camera shooting data of a plurality of cameras; performing image stitching on the shooting data of each camera based on the time information to obtain a panoramic stitching video; according to the invention, the abnormal monitoring is carried out based on the preset Gaussian mixture model and the panoramic stitching video to obtain the abnormal monitoring result, and the abnormal monitoring is carried out on the panoramic stitching video through the Gaussian mixture model, so that the comprehensive abnormal monitoring of the whole side slope is realized, the influence of weather on the abnormal monitoring is avoided, and the accuracy of the abnormal monitoring of the side slope is improved.

Description

Technical Field

[0001] The present invention relates to the field of safety detection technology, and in particular to a slope anomaly monitoring method, device, equipment and storage medium. Background Art

[0002] With the continuous expansion of mining scale and the improvement of safety requirements, timely and accurate monitoring of mine slope deformation and identification and early warning of rockfall and landslide events have become particularly important.

[0003] At present, the technologies for surface mine slope safety monitoring mainly include GNSS-based contact displacement monitoring technology, synthetic aperture radar imaging technology and monocular visual deformation monitoring technology. GNSS-based contact displacement monitoring technology is monitored by a base station plus multiple measuring stations plus data analysis equipment. The base station and the measuring station receive satellite signals regularly to obtain the three-dimensional coordinates of each point. Finally, the coordinate change before and after the measuring point is solved by the data analysis equipment to obtain the displacement deformation of the monitored mountain. Synthetic aperture radar imaging technology is a method of imaging and monitoring the address slope using radar beams. The radar transmits microwave signals, receives echo signals to obtain information on the surface of the geological slope, generates high-resolution slope images, and analyzes and processes the images to obtain information such as the topography, displacement and deformation of the slope surface. Monocular visual deformation monitoring technology installs artificial targets on the monitoring slope, collects video data of the monitoring area in real time or regularly through the camera, and transmits it to the image processing server. The image processing server calculates the coordinate changes of the frames before and after the target in the video through image processing technologies such as template matching and feature point matching to monitor the displacement deformation state of the mountain.

[0004] In the above methods, the GNSS monitoring system can only perform single "point" level monitoring, with large blind spots, large errors, poor timeliness, and relatively high costs. Synthetic aperture radar imaging technology is limited by meteorological conditions and may not work properly; data processing and analysis are difficult and have limited accuracy. Monocular visual deformation monitoring technology has poor generalization ability and is greatly affected by external weather and lighting changes. In addition, in mountain monitoring scenarios, the target is easily blocked by plants and animals, resulting in monitoring failures and high subsequent maintenance costs. Summary of the invention

[0005] The present invention provides a slope anomaly monitoring method, device, equipment and storage medium to solve the technical problem of low accuracy in slope anomaly monitoring and improve the accuracy of slope anomaly monitoring.

[0006] According to one aspect of the present invention, a slope anomaly monitoring method is provided, wherein a plurality of cameras are arranged opposite to the slope, and the method comprises:

[0007] Get camera shooting data of multiple cameras;

[0008] Based on the time information, the image data shot by each camera is stitched together to obtain a panoramic stitching video;

[0009] Anomaly monitoring is performed based on the preset mixed Gaussian model and panoramic stitching video to obtain anomaly monitoring results.

[0010] Optionally, based on the above solution, anomaly monitoring is performed based on a preset mixed Gaussian model and panoramic stitching video to obtain anomaly monitoring results, including:

[0011] Remove vegetation from the image frames in the panoramic stitching video to obtain the target monitoring video;

[0012] Based on the preset mixed Gaussian model, the target monitoring video is monitored for abnormalities and the abnormal monitoring results are obtained.

[0013] Optionally, based on the above solution, the target monitoring video is monitored for abnormalities based on a preset mixed Gaussian model to obtain abnormal monitoring results, including:

[0014] According to the time sequence of the image frames in the target monitoring video, the image frames in the target monitoring video are sequentially used as the current image frames;

[0015] For any pixel point in the current image frame, a pixel value of the pixel point is obtained, and a target Gaussian model matching the pixel point among multiple benchmark Gaussian models is determined according to the pixel value of the pixel point and the model parameters of the benchmark Gaussian model, wherein the model parameters of the benchmark Gaussian model are initialization model parameters or model parameters determined according to an image frame previous to the current image frame;

[0016] The background modeling of the image frames in the target monitoring video is performed according to the target Gaussian model matching each pixel point, and anomaly monitoring is performed according to the background modeling corresponding to any adjacent image frames to obtain anomaly monitoring results.

[0017] Optionally, based on the above solution, a target Gaussian model matching the pixel point among multiple reference Gaussian models is determined according to the pixel value of the pixel point and the model parameters of the reference Gaussian model, including:

[0018] For any reference Gaussian model, judging whether the reference Gaussian model matches the pixel point according to the pixel value of the pixel point and the model parameters of the reference Gaussian model;

[0019] If there is any reference Gaussian model that matches the pixel point, the reference Gaussian model is used as the target Gaussian model that matches the pixel point;

[0020] If any benchmark Gaussian model does not match the pixel point, the weight and model parameters of the benchmark Gaussian model are adjusted, and the pixel point is matched based on the adjusted benchmark Gaussian model to obtain a target Gaussian model that matches the pixel point.

[0021] Optionally, based on the above solution, the slope includes a plurality of transverse sections, each transverse section is provided with at least one target, the camera shooting data of each camera includes target position information of at least one target, and the method further includes:

[0022] Determine the reference displacement parameter of the monitoring target corresponding to each camera according to the camera shooting data of each camera;

[0023] Parameter fusion is performed based on the reference displacement parameters of each monitoring target to obtain the target displacement parameters of each monitoring target, and target displacement monitoring is performed based on each target displacement parameter.

[0024] Optionally, based on the above solution, the reference displacement parameters of the monitoring targets corresponding to each camera are determined according to the camera shooting data of each camera, including:

[0025] For each camera, obtain the initial shooting data and real-time shooting data of the camera;

[0026] Based on the initial shooting data, feature point matching and contour extraction are performed to obtain the initial position information of the monitoring target;

[0027] Based on the real-time shooting data, feature point matching and contour extraction are performed to obtain the real-time position information of the monitoring target;

[0028] The reference displacement parameter of the monitoring target is determined according to the initial position information and real-time position information of the monitoring target.

[0029] Optionally, on the basis of the above solution, parameter fusion is performed based on the reference displacement parameters of each monitoring target to obtain the target displacement parameters of each monitoring target, including:

[0030] The smallest reference displacement parameter among the reference displacement parameters of each monitoring target is used as the calibration displacement parameter;

[0031] The reference displacement parameter of each monitoring target is calibrated based on the calibration unique parameter to obtain the target displacement parameter of each monitoring target.

[0032] According to another aspect of the present invention, there is provided a slope abnormality monitoring device, comprising:

[0033] A shooting data acquisition module is used to acquire camera shooting data of multiple cameras;

[0034] A panoramic image stitching module is used to stitch the images captured by each camera based on time information to obtain a panoramic stitching video;

[0035] The anomaly monitoring module is used to perform anomaly monitoring based on a preset mixed Gaussian model and panoramic stitching video to obtain anomaly monitoring results.

[0036] According to another aspect of the present invention, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the slope abnormality monitoring method provided by any embodiment of the present invention.

[0037] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions, and the computer instructions are used to enable a processor to implement the slope anomaly monitoring method provided by any embodiment of the present invention when executed.

[0038] The technical solution of the embodiment of the present invention is to set up multiple cameras on the slope object to obtain camera shooting data of multiple cameras; perform image stitching on the shooting data of each camera based on time information to obtain a panoramic stitching video; perform anomaly monitoring based on a preset mixed Gaussian model and the panoramic stitching video to obtain anomaly monitoring results, and perform anomaly monitoring on the panoramic stitching video through the mixed Gaussian model, thereby realizing comprehensive anomaly monitoring of the entire slope, avoiding the influence of weather on anomaly monitoring, and improving the accuracy of slope anomaly monitoring.

[0039] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0041] Figure 1a is a flow chart of a slope anomaly monitoring method provided in Embodiment 1 of the present invention;

[0042] Figure 1b is a schematic diagram of image stitching provided by Embodiment 1 of the present invention;

[0043] Figure 1cThis is a schematic diagram of abnormal monitoring provided by Embodiment 1 of the present invention;

[0044] Figure 1d is a schematic diagram of a cross-section of a mine slope provided in Embodiment 1 of the present invention;

[0045] Figure 2a is a flow chart of a slope anomaly monitoring method provided by Embodiment 2 of the present invention;

[0046] Figure 2b It is a schematic diagram of a flow chart of detecting an abnormal deformation event of an entire slope provided by the second embodiment of the present invention;

[0047] Figure 2c It is a schematic diagram of a process of target displacement monitoring provided by the second embodiment of the present invention;

[0048] Figure 2d It is a schematic diagram of another process of target displacement monitoring provided by the second embodiment of the present invention;

[0049] Figure 2e is a flow chart of a data fusion calibration process provided by the second embodiment of the present invention;

[0050] Figure 3 It is a structural schematic diagram of a slope anomaly monitoring device provided in Embodiment 3 of the present invention;

[0051] Figure 4 It is a structural schematic diagram of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0053] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0054] Embodiment 1

[0055] Figure 1a This is a flow chart of a slope anomaly monitoring method provided in the first embodiment of the present invention. This embodiment is applicable to the case of slope anomaly monitoring, and is particularly applicable to the case of real-time anomaly monitoring of a mine slope including multiple transverse sections. The method can be executed by a slope anomaly monitoring device, which can be implemented in the form of hardware and / or software, and can be configured in a slope anomaly monitoring device. Figure 1a As shown, the method includes:

[0056] S110: Obtain camera shooting data of multiple cameras.

[0057] In this embodiment, multiple cameras are set at the object to be monitored so that the camera's shooting area covers the entire area of ​​the mine slope that needs to be monitored. In order to prevent the data captured by a single camera from being affected by factors such as camera parameters, which affects the accuracy of the captured data, the fields of view of adjacent cameras can be set to have a certain degree of overlap, and the captured data of the overlapping areas of adjacent cameras are fused to obtain accurate captured data of the overlapping areas. Optionally, the overlap of the fields of view of adjacent cameras can be set to be no less than one-third of the camera's field of view to ensure the accuracy of the camera captured data.

[0058] Optionally, the camera shooting data of all cameras within a set time period can be obtained. In order to improve the synchronization of data collection, all cameras can be set to be controlled by the same edge computing board, and the edge computing board can send streaming instructions to multiple cameras respectively to obtain the camera shooting data sent by the camera, and the camera shooting data of all cameras at the same time point are sequentially constructed into data packets, and the data packets are provided to the slope anomaly monitoring device, so that the slope anomaly monitoring device can perform anomaly monitoring according to the data packets.

[0059] S120 , stitching the images of the camera shooting data based on the time information to obtain a panoramic stitching video.

[0060] In this embodiment, in order to achieve comprehensive abnormality monitoring of the slope, the camera shooting data of each camera is stitched to obtain panoramic stitching data of the slope, and the abnormality monitoring of the slope is performed through the panoramic stitching video within a certain period of time.

[0061] Optionally, the camera shooting data acquired at the same time (such as the above-mentioned data packets) can be sequentially numbered according to the position of the camera to which the camera shooting data belongs, and the camera shooting data can be spliced ​​based on the sequential number of the camera shooting data to finally obtain a panoramic stitching image corresponding to the time point. After obtaining the panoramic stitching images corresponding to different time points in the above manner, the panoramic stitching images can be connected according to the connection sequence of the time points to obtain a panoramic stitching video. Among them, the stitching of the camera shooting data can adopt the existing stitching method, which is not limited here. For example, the sticher algorithm can be used to complete the panoramic stitching of the camera shooting data at the same time point.

[0062] Figure 1b FIG. 1 is a schematic diagram of image stitching provided by the first embodiment of the present invention. Figure 1b As shown, the upper and lower images on the left correspond to the camera shooting data of two adjacent cameras at the same time point. The circular target area in the image is the target. There is an overlapping area in the two images. The upper right target in the upper image coincides with the upper left target in the lower image. After splicing the two images, the spliced ​​image on the right can be obtained.

[0063] S130: Perform abnormality monitoring based on a preset mixed Gaussian model and the panoramic stitching video to obtain an abnormality monitoring result.

[0064] In this embodiment, the slope abnormality monitoring is performed by panoramic stitching video, which can achieve comprehensive monitoring of the slope. Optionally, modeling can be performed based on the video frames in the panoramic stitching video, and abnormality monitoring can be performed based on the modeling comparison of adjacent video frames.

[0065] In the process of visual imaging, the change of a single pixel under the influence of illumination conforms to the Gaussian distribution. However, the Gaussian distribution of each area in the same field of view is different, and bimodal and trimodal distributions are possible. Even in some complex scenes, the distribution difference of pixel changes is greater. Based on this, this embodiment provides a method based on a mixed Gaussian model to perform background modeling on the monitoring area, perform anomaly monitoring through background modeling, and improve the accuracy of slope anomaly monitoring by improving the accuracy of background modeling.

[0066] In general, multiple Gaussian models can be pre-set, and for different pixels, matching Gaussian models can be selected according to their Gaussian transformation distribution characteristics affected by light. Different pixels can be modeled using different Gaussian models to obtain more accurate modeling results.

[0067] In one embodiment of the present invention, anomaly monitoring is performed based on a preset mixed Gaussian model and a panoramic stitching video to obtain anomaly monitoring results, including:

[0068] Remove vegetation from the image frames in the panoramic stitching video to obtain the target monitoring video;

[0069] Based on the preset mixed Gaussian model, the target monitoring video is monitored for abnormalities and the abnormal monitoring results are obtained.

[0070] Vegetation may affect the accuracy of abnormal event monitoring. In order to improve the accuracy of slope abnormal event monitoring, before modeling, the vegetation in the panoramic stitching video can be removed to obtain the target monitoring video, and then the target monitoring video can be modeled based on the mixed Gaussian model for abnormal monitoring. Optionally, vegetation removal can be performed by histogram mapping. Histogram mapping can extract the effect of the difference part based on the histogram features of the two images. The slope is mainly composed of vegetation and rocks, and there is a large difference in the distribution of histogram features of vegetation and rock sand. Through the histogram feature direction mapping, the area where the vegetation is located can be better extracted, and then the vegetation can be preliminarily removed. Specifically, the image frames of the panoramic stitching video can be mapped based on the extracted histogram features of the vegetation in the image frames in the panoramic stitching video, such as the histogram features of green plants and dead grass, according to the histogram features of the vegetation, to obtain the image frames after the vegetation is removed, and the continuous image frames after the vegetation is removed constitute the target monitoring video.

[0071] Based on the above scheme, the target monitoring video is monitored for abnormalities based on the preset mixed Gaussian model to obtain abnormal monitoring results, including:

[0072] According to the time sequence of the image frames in the target monitoring video, the image frames in the target monitoring video are sequentially used as the current image frames;

[0073] For any pixel point in the current image frame, a pixel value of the pixel point is obtained, and a target Gaussian model matching the pixel point among multiple benchmark Gaussian models is determined according to the pixel value of the pixel point and the model parameters of the benchmark Gaussian model, wherein the model parameters of the benchmark Gaussian model are initialization model parameters or model parameters determined according to an image frame previous to the current image frame;

[0074] The background modeling of the image frames in the target monitoring video is performed according to the target Gaussian model matching each pixel point, and anomaly monitoring is performed according to the background modeling corresponding to any adjacent image frames to obtain anomaly monitoring results.

[0075] In this embodiment, abnormal monitoring can be performed through background modeling of each image frame in the target monitoring video, such as obtaining background modeling of a set number of adjacent image frames, comparing the differences in background modeling of different image frames, and obtaining abnormal monitoring results. Exemplarily, the background modeling corresponding to two adjacent image frames can be obtained, and the background modeling corresponding to the latter image frame and the background modeling corresponding to the previous image frame can be used for "image subtraction" processing to obtain abnormal events existing in the area corresponding to the image frame. Figure 1c is a schematic diagram of abnormal monitoring provided by the first embodiment of the present invention, such as Figure 1c As shown, it schematically shows the result of "image subtraction" processing between two adjacent image frames, where the target in the target box is the foreground target that appears in the latter image frame compared to the previous image frame. The foreground target may be fallen rocks, indicating that fallen rocks and landslides may occur on the slope. Timely alarms are issued through abnormal monitoring results to avoid safety accidents and property losses.

[0076] In the above process, in order to make the background modeling more accurate, the corresponding target Gaussian model can be used for different pixels in different image frames for background modeling. Optionally, multiple initial Gaussian models can be pre-set, and the target Gaussian model corresponding to the pixel in the image frame can be determined in turn for the image frame in the target monitoring video, so that the image frame can be modeled using a mixed Gaussian model.

[0077] Assuming that the current image frame is the first image frame in the target monitoring video, the benchmark Gaussian model is a pre-set initial Gaussian model; assuming that the current image frame is not the first image frame in the target monitoring video, the benchmark Gaussian model may be the initial Gaussian model, or it may be the initial Gaussian model that has been matched with the image frame before the current image frame.

[0078] Assuming that the number of pre-set initial Gaussian models is M, that is, M initial Gaussian models are randomly generated first, the first N frames of the target monitoring video can be taken as the current image frames one by one. For each current image frame, the attribution of each pixel in the current image frame is determined, and the target Gaussian model corresponding to each pixel is determined. The model parameters and weights of each Gaussian model are gradually updated to construct a mixed Gaussian model.

[0079] Optionally, determining a target Gaussian model matching the pixel point among multiple reference Gaussian models according to the pixel value of the pixel point and the model parameters of the reference Gaussian model includes:

[0080] For any reference Gaussian model, judging whether the reference Gaussian model matches the pixel point according to the pixel value of the pixel point and the model parameters of the reference Gaussian model;

[0081] If there is any reference Gaussian model that matches the pixel point, the reference Gaussian model is used as the target Gaussian model that matches the pixel point;

[0082] If any benchmark Gaussian model does not match the pixel point, the weight and model parameters of the benchmark Gaussian model are adjusted, and the pixel point is matched based on the adjusted benchmark Gaussian model to obtain a target Gaussian model that matches the pixel point.

[0083] You can select the corresponding principle according to actual needs to determine the attribution of pixels and determine whether the pixels match the reference Gaussian model. For example, you can determine the attribution of pixels based on the σ principle, 2σ principle, 3σ principle, etc. Taking the attribution determination of pixels based on the 2.5σ principle as an example, you can use the following formula to determine whether the pixels match the reference Gaussian model:

[0084] ;

[0085] Where I(k) is the pixel value of a pixel in the kth image frame (current image frame), μ is the mean parameter in the benchmark Gaussian model, and σ is the variance parameter in the benchmark Gaussian model. If the pixel value of the pixel in the current image frame satisfies the above formula, then the benchmark Gaussian model is the target Gaussian model corresponding to the pixel; otherwise, the benchmark Gaussian model does not match the pixel, and it is necessary to determine whether other benchmark Gaussian models match the pixel.

[0086] Assuming that all benchmark Gaussian models do not match the pixel point, the model parameters and weights of the benchmark Gaussian model can be adjusted, and the matching judgment can be re-performed until the adjusted benchmark Gaussian model matches the pixel point. The adjusted benchmark Gaussian model is used as the target Gaussian model corresponding to the pixel point, and the adjusted benchmark Gaussian model is used as the benchmark Gaussian model for the next matching judgment.

[0087] In one implementation, the weight of the baseline Gaussian model may be adjusted using the following formula:

[0088]

[0089] Among them, w k，t is the weight of the k-th Gaussian model at time t, w k，t-1 is the weight of the k-th Gaussian model at time t-1, α is the learning rate, M k,t is a binary state value, which can be set based on the matching degree between the current pixel and the k-th Gaussian model. For example, if the current pixel matches the k-th Gaussian model, M can be set. k,t It can be set to 1, otherwise it can be set to 0.

[0090] And, the model parameters of the benchmark Gaussian model are adjusted by the following formula:

[0091] ;

[0092] ;

[0093] ;

[0094] in, is the Gaussian model, α is the learning rate, is the mean parameter of the t-th Gaussian model, is the standard deviation parameter of the t-th Gaussian model, X t is the pixel value at time t, ρ is the parameter update weight, is the variance.

[0095] ;

[0096] in, is the covariance matrix, , I is the three-dimensional unit matrix, X t is the pixel value at time t, are the mean and standard deviation of the t-th Gaussian model respectively.

[0097] After determining the target Gaussian model corresponding to different pixels, the background modeling of the corresponding pixels is performed based on the corresponding target Gaussian model.

[0098] By continuously adjusting the model parameters and weights of the Gaussian model, the model parameters and weights of the Gaussian model are made more suitable for each image frame in the target monitoring video, making the background modeling more accurate, and thus making the abnormality monitoring more accurate.

[0099] The technical solution of the embodiment of the present invention is to set up multiple cameras on the slope object to obtain camera shooting data of multiple cameras; perform image stitching on the shooting data of each camera based on time information to obtain a panoramic stitching video; perform anomaly monitoring based on a preset mixed Gaussian model and the panoramic stitching video to obtain anomaly monitoring results, and perform anomaly monitoring on the panoramic stitching video through the mixed Gaussian model, thereby realizing comprehensive anomaly monitoring of the entire slope, avoiding the influence of weather on anomaly monitoring, and improving the accuracy of slope anomaly monitoring.

[0100] Optionally, based on the above solution, the slope includes a plurality of transverse sections, each transverse section is provided with at least one target, the camera shooting data of each camera includes target position information of at least one target, and the method further includes:

[0101] Determine the reference displacement parameter of the monitoring target corresponding to each camera according to the camera shooting data of each camera;

[0102] Parameter fusion is performed based on the reference displacement parameters of each monitoring target to obtain the target displacement parameters of each monitoring target, and target displacement monitoring is performed based on each target displacement parameter.

[0103] The above abnormal monitoring mainly monitors abnormal events, such as rockfall, landslide and other events, and realizes comprehensive abnormal event monitoring of the slope. In addition, abnormal monitoring can also be performed by setting targets on the slope.

[0104] Figure 1d Schematic diagram of a cross section of a mine slope provided by the first embodiment of the present invention. Figure 1d As shown, mining excavation is generally carried out in layers horizontally, so that the slope forms multiple transverse sections. In order to monitor abnormal events in the transverse sections in real time, targets can be arranged on the longitudinal surface of each transverse section. The distance between the targets can be set according to actual needs. For example, the distance between adjacent targets can be 20 meters.

[0105] The camera data of each camera contains the location information of the monitoring target, and the reference displacement parameters of the monitoring target can be determined based on the location information of the same monitoring target at different time points. Among them, since the field of view of the camera is overlapping, the location information of the same monitoring target may be determined by the camera data of the same camera, or by the camera data of different cameras. When determined by the camera data of different cameras, the location information of the monitoring target can be determined by taking the characteristic value (mean, etc.).

[0106] After determining the baseline displacement parameters of each monitoring target, considering that the monitoring targets may have undergone certain position adjustments due to other reasons, in this embodiment, parameter fusion correction is performed on the baseline displacement parameters of the monitoring targets to obtain corrected target displacement parameters, and target displacement monitoring and alarm are performed based on the corrected target displacement parameters.

[0107] Wherein, determining the reference displacement parameters of the monitoring targets corresponding to each camera according to the camera shooting data of each camera includes:

[0108] For each camera, obtain the initial shooting data and real-time shooting data of the camera;

[0109] Based on the initial shooting data, feature point matching and contour extraction are performed to obtain the initial position information of the monitoring target;

[0110] Based on the real-time shooting data, feature point matching and contour extraction are performed to obtain the real-time position information of the monitoring target;

[0111] The reference displacement parameter of the monitoring target is determined according to the initial position information and real-time position information of the monitoring target.

[0112] Optionally, the initial shooting data of the camera can be the shooting data of the camera at the set start monitoring time point, or it can be the shooting data corresponding to the previous image frame at the current moment; the real-time shooting data of the camera can be the shooting data of the camera at the current moment. For the initial shooting data and real-time shooting data, feature point matching and contour extraction can be used to obtain the initial position information and real-time position information of each monitoring target within the camera's field of view, respectively. Different monitoring targets can be set with different feature points to distinguish different monitoring targets within the camera's field of view. For example, the feature points of monitoring target 1 can be set as M feature points in the first direction, and the feature points of monitoring target 2 can be set as N feature points in the second direction. The corresponding monitoring targets are determined by different feature points.

[0113] The initial position information of the monitoring target may be a two-dimensional coordinate or a three-dimensional coordinate, which may be determined by the camera coordinates of the monitoring target and the camera transformation parameters. Similarly, the real-time position information of the monitoring target may be a two-dimensional coordinate or a three-dimensional coordinate. The reference displacement parameter of the monitoring target may be determined according to the coordinate difference between the initial position information and the real-time position information.

[0114] In one embodiment of the present invention, parameter fusion is performed based on the reference displacement parameters of each monitoring target to obtain the target displacement parameters of each monitoring target, including:

[0115] The smallest reference displacement parameter among the reference displacement parameters of each monitoring target is used as the calibration displacement parameter;

[0116] The reference displacement parameter of each monitoring target is calibrated based on the calibration unique parameter to obtain the target displacement parameter of each monitoring target.

[0117] Optionally, the monitoring target with the smallest base displacement parameter can be used as a calibration target, and the base displacement parameters of other monitoring targets can be calibrated based on the smallest base displacement parameter, so as to remove environmental interference and improve the accuracy of target monitoring.

[0118] Exemplarily, it is assumed that camera 1 corresponds to monitoring target 1, monitoring target 2 and monitoring target 3, camera 2 corresponds to monitoring target 4, monitoring target 5 and monitoring target 6, and camera 3 corresponds to monitoring target 7, monitoring target 8 and monitoring target 9. Assume that at time T, the reference displacement parameters of monitoring target 1, monitoring target 2, monitoring target 3, monitoring target 4, monitoring target 5, monitoring target 6, monitoring target 7, monitoring target 8 and monitoring target 9 are v1, v2, v3, v4, v5, v6, v7, v8, v9 respectively. Then min (v1, v2, v3, v4, v5, v6, v7, v8) can be taken as the calibration displacement parameter, and the reference displacement parameter of each monitoring target is calibrated by the calibration displacement parameter to obtain the target displacement parameter of each monitoring target.

[0119] Embodiment 2

[0120] Figure 2a It is a flow chart of a slope anomaly monitoring method provided in the second embodiment of the present invention. This embodiment provides a preferred embodiment based on the above embodiment.

[0121] In general, the embodiment of the present invention arranges a plurality of monitoring targets on the transverse section of the target monitoring slope, and sets a plurality of cameras on the object of the target monitoring slope, covering all monitoring areas of the target monitoring slope.

[0122] like Figure 2a As shown, the method provided in this embodiment includes:

[0123] S210, data collection and registration fusion.

[0124] Get the camera shooting data of each camera at the same time and build a data package.

[0125] For each data packet, the images corresponding to the data packet are sequentially numbered, and panoramic stitching is performed based on the sequential numbers to obtain a panoramic stitching image corresponding to each time point. The specific stitching method can refer to the above embodiment and will not be repeated here.

[0126] S220, monitoring of overall abnormal deformation events of slopes.

[0127] Figure 2b FIG. 2 is a flow chart of monitoring an abnormal deformation event of a slope as a whole provided by the second embodiment of the present invention. Figure 2b As shown, after the vegetation is removed from the panoramic stitching image, background modeling is performed based on the mixed Gaussian model, foreground extraction is performed through the background modeling of different image frames, and abnormal event recognition and alarm are performed based on the foreground extraction results. During the background modeling process, the model parameters and weights of the background model (mixed Gaussian model) are adjusted in real time. The specific vegetation removal method and background modeling method can refer to the above embodiments, which will not be repeated here.

[0128] S230, target displacement monitoring.

[0129] Figure 2c FIG. 1 is a flow chart of target displacement monitoring provided by Embodiment 2 of the present invention. Figure 2c As shown in the figure, each monitoring camera is aimed at multiple monitoring targets. First, the pixel-level position information of the monitoring targets is obtained from the full image, and different targets are identified. The displacement of the target is monitored based on the position information of the same target at different time points. It should be pointed out that in order to better distinguish different targets and let the entire system focus more on high-precision displacement detection, each target is marked, which can easily detect and distinguish different monitoring targets.

[0130] Figure 2d FIG. 2 is another flow chart of target displacement monitoring provided by the second embodiment of the present invention. Figure 2d As shown, for the initial frame (initial shooting data) and the real-time monitoring frame (real-time shooting data), the open source lightweight target detection algorithm picodevice is used to quickly complete the detection and identification of the target. Specifically, the image ORB and sift feature points are extracted, the SIFT descriptor is extracted to determine the target features and thus the monitoring target, and the target edge contour information is extracted to determine the geometric center and thus the position information. The initial position information and real-time position information of the monitoring target are obtained in the above manner, and the deformation variable of the monitoring target can be calculated based on the initial position information and the real-time position information to obtain the reference displacement parameters of the monitoring target.

[0131] S240, data fusion analysis and early warning.

[0132] Figure 2e FIG. 1 is a flow chart of a data fusion calibration process provided by Embodiment 2 of the present invention. Figure 2e As shown in the figure, the above part has completed the background modeling and foreground extraction of the panoramic view of the mine slope, which can not only detect and identify potential abnormal events (such as rockfall and landslide) on the mine slope in a timely manner from a global level and give early warnings in a timely manner. In addition, it also provides foreground difference information within the field of view of each monitoring camera. In order to link the information of each independent monitoring device, the isolated and local monitoring is changed to global, linked and overall monitoring. The foreground difference information of each monitoring target can be calculated separately, and then the mean value is calculated, and then the foreground difference mean of each camera is sorted. In order to effectively remove the monitoring errors caused by environmental changes, especially illumination changes, the linkage fusion of the data of each monitoring target of each monitoring camera is completed in a reverse calibration method, and finally the deformation information of the entire mine is obtained.

[0133] This embodiment realizes the real-time and accuracy of slope monitoring by monitoring abnormal events based on a mixed Gaussian model and performing correction monitoring of displacement monitoring through reference displacement information of each monitoring target.

[0134] Embodiment 3

[0135] Figure 3 Schematic diagram of the structure of a slope anomaly monitoring device provided by Embodiment 3 of the present invention. Figure 3 As shown, the device comprises:

[0136] A shooting data acquisition module 310 is used to acquire camera shooting data of multiple cameras;

[0137] A panoramic image stitching module 320 is used to stitch the images of the camera shooting data based on the time information to obtain a panoramic stitching video;

[0138] The anomaly monitoring module 330 is used to perform anomaly monitoring based on a preset mixed Gaussian model and the panoramic stitching video to obtain anomaly monitoring results.

[0139] The technical solution of the embodiment of the present invention is to set up multiple cameras on the slope object to obtain camera shooting data of multiple cameras; perform image stitching on the shooting data of each camera based on time information to obtain a panoramic stitching video; perform anomaly monitoring based on a preset mixed Gaussian model and the panoramic stitching video to obtain anomaly monitoring results, and perform anomaly monitoring on the panoramic stitching video through the mixed Gaussian model, thereby realizing comprehensive anomaly monitoring of the entire slope, avoiding the influence of weather on anomaly monitoring, and improving the accuracy of slope anomaly monitoring.

[0140] Optionally, based on the above solution, the abnormality monitoring module 330 is specifically used for:

[0141] Remove vegetation from the image frames in the panoramic stitching video to obtain the target monitoring video;

[0142] Based on the preset mixed Gaussian model, the target monitoring video is monitored for abnormalities and the abnormal monitoring results are obtained.

[0143] Optionally, based on the above solution, the abnormality monitoring module 330 is specifically used for:

[0144] According to the time sequence of the image frames in the target monitoring video, the image frames in the target monitoring video are sequentially used as the current image frames;

[0145] For any pixel point in the current image frame, a pixel value of the pixel point is obtained, and a target Gaussian model matching the pixel point among multiple benchmark Gaussian models is determined according to the pixel value of the pixel point and the model parameters of the benchmark Gaussian model, wherein the model parameters of the benchmark Gaussian model are initialization model parameters or model parameters determined according to an image frame previous to the current image frame;

[0146] The background modeling of the image frames in the target monitoring video is performed according to the target Gaussian model matching each pixel point, and anomaly monitoring is performed according to the background modeling corresponding to any adjacent image frames to obtain anomaly monitoring results.

[0147] Optionally, based on the above solution, the abnormality monitoring module 330 is specifically used for:

[0148] For any reference Gaussian model, judging whether the reference Gaussian model matches the pixel point according to the pixel value of the pixel point and the model parameters of the reference Gaussian model;

[0149] If there is any reference Gaussian model that matches the pixel point, the reference Gaussian model is used as the target Gaussian model that matches the pixel point;

[0150] If any benchmark Gaussian model does not match the pixel point, the weight and model parameters of the benchmark Gaussian model are adjusted, and the pixel point is matched based on the adjusted benchmark Gaussian model to obtain a target Gaussian model that matches the pixel point.

[0151] Optionally, based on the above solution, the slope includes a plurality of transverse sections, each transverse section is provided with at least one target, the camera shooting data of each camera includes target position information of at least one target, and the device further includes a target displacement monitoring module for:

[0152] Determine the reference displacement parameter of the monitoring target corresponding to each camera according to the camera shooting data of each camera;

[0153] Parameter fusion is performed based on the reference displacement parameters of each monitoring target to obtain the target displacement parameters of each monitoring target, and target displacement monitoring is performed based on each target displacement parameter.

[0154] Optionally, based on the above solution, the target displacement monitoring module is specifically used for:

[0155] For each camera, obtain the initial shooting data and real-time shooting data of the camera;

[0156] Based on the initial shooting data, feature point matching and contour extraction are performed to obtain the initial position information of the monitoring target;

[0157] Based on the real-time shooting data, feature point matching and contour extraction are performed to obtain the real-time position information of the monitoring target;

[0158] The reference displacement parameter of the monitoring target is determined according to the initial position information and real-time position information of the monitoring target.

[0159] Optionally, based on the above solution, the target displacement monitoring module is specifically used for:

[0160] The smallest reference displacement parameter among the reference displacement parameters of each monitoring target is used as the calibration displacement parameter;

[0161] The reference displacement parameter of each monitoring target is calibrated based on the calibration unique parameter to obtain the target displacement parameter of each monitoring target.

[0162] The slope anomaly monitoring device provided in the embodiment of the present invention can execute the slope anomaly monitoring method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0163] Embodiment 4

[0164] Figure 4 It is a structural schematic diagram of an electronic device provided in Embodiment 4 of the present invention. Figure 4A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0165] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0166] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0167] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a slope anomaly monitoring method.

[0168] In some embodiments, the slope anomaly monitoring method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the slope anomaly monitoring method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the slope anomaly monitoring method in any other appropriate manner (e.g., by means of firmware).

[0169] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0170] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0171] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0172] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0173] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0174] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0175] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0176] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A slope anomaly monitoring method, characterized in that: A plurality of cameras are arranged opposite to the slope, and the method comprises: Acquire camera shooting data of a plurality of cameras; Performing image stitching on the camera shooting data based on the time information to obtain a panoramic stitching video; Anomaly monitoring is performed based on a preset mixed Gaussian model and the panoramic stitching video to obtain anomaly monitoring results.

2. The method according to claim 1, characterized in that The abnormality monitoring is performed based on the preset mixed Gaussian model and the panoramic stitching video to obtain the abnormality monitoring result, including: Performing vegetation removal on image frames in the panoramic stitching video to obtain a target monitoring video; Based on a preset mixed Gaussian model, the target monitoring video is monitored for abnormalities to obtain abnormal monitoring results.

3. The method according to claim 2, characterized in that The abnormality monitoring of the target monitoring video based on the preset mixed Gaussian model to obtain the abnormality monitoring result includes: According to the time sequence of the image frames in the target monitoring video, the image frames in the target monitoring video are sequentially used as current image frames; For any pixel point in the current image frame, a pixel value of the pixel point is obtained, and a target Gaussian model matching the pixel point among multiple benchmark Gaussian models is determined according to the pixel value of the pixel point and the model parameters of the benchmark Gaussian model, wherein the model parameters of the benchmark Gaussian model are initialization model parameters or model parameters determined according to an image frame previous to the current image frame; The background modeling is performed on the image frames in the target monitoring video according to the target Gaussian model matched with each pixel point, and the abnormality monitoring is performed according to the background modeling corresponding to any adjacent image frames to obtain the abnormality monitoring result.

4. The method according to claim 3, characterized in that The step of determining a target Gaussian model matching the pixel point among a plurality of reference Gaussian models according to the pixel value of the pixel point and the model parameters of the reference Gaussian model comprises: For any reference Gaussian model, judging whether the reference Gaussian model matches the pixel point according to the pixel value of the pixel point and the model parameters of the reference Gaussian model; If there is any reference Gaussian model that matches the pixel point, the reference Gaussian model is used as the target Gaussian model that matches the pixel point; If any benchmark Gaussian model does not match the pixel point, the weight and model parameters of the benchmark Gaussian model are adjusted, and the pixel point is matched based on the adjusted benchmark Gaussian model to obtain a target Gaussian model that matches the pixel point.

5. The method according to claim 1, characterized in that The slope includes a plurality of transverse sections, each of the transverse sections is provided with at least one target, the camera shooting data of each camera includes target position information of at least one target, and the method further includes: Determine the reference displacement parameter of the monitoring target corresponding to each camera according to the camera shooting data of each camera; Parameter fusion is performed based on the reference displacement parameters of each monitoring target to obtain the target displacement parameters of each monitoring target, and target displacement monitoring is performed based on each target displacement parameter.

6. The method according to claim 5, characterized in that The method of determining the reference displacement parameter of the monitoring target corresponding to each camera according to the camera shooting data of each camera includes: For each camera, obtaining initial shooting data and real-time shooting data of the camera; Performing feature point matching and contour extraction based on the initial shooting data to obtain initial position information of the monitoring target; Performing feature point matching and contour extraction based on the real-time shooting data to obtain real-time position information of the monitoring target; The reference displacement parameter of the monitoring target is determined according to the initial position information and the real-time position information of the monitoring target.

7. The method according to claim 5, characterized in that The step of performing parameter fusion based on the reference displacement parameters of each monitoring target to obtain the target displacement parameters of each monitoring target includes: The smallest reference displacement parameter among the reference displacement parameters of each monitoring target is used as the calibration displacement parameter; The reference displacement parameter of each monitoring target is calibrated based on the calibration unique parameter to obtain the target displacement parameter of each monitoring target.

8. A slope abnormality monitoring device, characterized in that: include: A shooting data acquisition module is used to acquire camera shooting data of multiple cameras; A panoramic image stitching module, used for stitching the image data shot by each camera based on time information to obtain a panoramic stitching video; The anomaly monitoring module is used to perform anomaly monitoring based on a preset mixed Gaussian model and the panoramic stitching video to obtain anomaly monitoring results.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the slope anomaly monitoring method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the slope anomaly monitoring method according to any one of claims 1 to 7 when executed.

Images