A dangerous rock displacement detection method and system, storage medium, and electronic equipment

Abstract

The present invention belongs to the field of disaster prevention technology and specifically relates to a dangerous rock displacement detection method and system, storage medium, and electronic device. The method comprises fixing markers on the dangerous rock to be observed and collecting images of the dangerous rock to be observed; sampling the images of the dangerous rock to be observed, detecting markers in the sample images using a YOLO-Pose network, and recording the positions of the diagonal intersections of all markers; calculating the sub-pixel precision displacement of the positions of the diagonal intersections of all markers using a 2D-DIC algorithm; plotting a displacement curve based on the sub-pixel precision displacement; determining, based on the displacement curve, whether the relative displacement of the markers over a certain period of time shows a fixed trend; if so, triggering a dangerous rock sliding trend warning; and determining, based on the displacement curve, whether the cumulative displacement of the markers over a certain period of time exceeds a warning value; if so, triggering a dangerous rock fall warning at that location. The present invention can achieve all-weather accurate monitoring and early warning of dangerous rock movement without the need for power supply or subsequent human maintenance.

Description

Technical Field

[0001] The present invention belongs to the technical field of disaster prevention, and in particular relates to a dangerous rock displacement detection method and system, a storage medium, and an electronic device. Background Art

[0002] Monitoring the displacement of dangerous rock masses can help detect hazards early and implement preventive measures, providing a reference for feedback on prevention and control construction designs, guiding construction, and verifying prevention and control effectiveness. Currently, commonly used methods for monitoring dangerous rock deformation include displacement sensor measurement, 3D laser scanning monitoring, and localized crack deformation monitoring. Displacement sensor measurement requires fixing the sensor to the dangerous rock mass, requiring power, making deployment complex and costly. 3D laser scanning monitoring and total stations can accurately and comprehensively record the dangerous rock mass's shape with high precision, but both methods are costly and lack real-time performance. Localized crack deformation monitoring can monitor the deformation trends of specific cracks in real time and provide early warnings, but requires sensors to be placed on both sides of the crack. This type of monitoring only reflects changes in the crack locally, not the entire dangerous rock mass. Alternatively, video surveillance can be used to monitor dangerous rock geological hazards. Video surveillance offers the advantages of non-contact, low cost, high real-time performance, and a large monitoring area. However, video monitoring requires strict on-site conditions, cannot operate at night, and has low accuracy, often limiting its use to trace the source of the event. Summary of the Invention

[0003] Aiming at the deficiencies of the prior art, the present invention invents a dangerous rock displacement detection method and system, a storage medium, and an electronic device.

[0004] The present invention is achieved through the following technical solutions:

[0005] In a first aspect, a method for detecting displacement of dangerous rocks comprises:

[0006] Fix markers on the dangerous rocks to be observed;

[0007] Arrange cameras based on the locations of the dangerous rocks to be observed, and collect images of the dangerous rocks to be observed;

[0008] The collected images of dangerous rocks to be observed are sampled, and the landmarks in the sample images are detected using the YOLO-Pose network, and the positions of the diagonal intersections of all the landmarks are recorded;

[0009] Calculate the sub-pixel precision displacement of the diagonal intersections of all markers in the k-th frame sample image relative to the diagonal intersections of all markers in the k-1-th frame sample image using a 2D-DIC algorithm;

[0010] Drawing a displacement curve based on the sub-pixel precision displacement, and recording the displacement change trends in the u direction and the v direction;

[0011] Determining whether the relative displacement of one or more markers over a certain period of time presents a fixed trend based on the displacement curve, and if so, triggering a dangerous rock sliding trend warning;

[0012] Based on the displacement curve, it is determined whether the cumulative displacement of one or more markers in a certain period of time exceeds the warning value. If so, a dangerous rock fall warning at the point is triggered.

[0013] In some embodiments, a marker is painted with a long-lasting luminescent material, and the marker is in the shape of a planar geometric figure containing at least two intersecting diagonal lines.

[0014] In some embodiments, the size of the marker in the image of the dangerous rock to be observed is greater than or equal to Pixels.

[0015] In some embodiments, the YOLO-Pose network is used to locate the positions of the intersections of the diagonals of all the markers in the kth frame, the k-1th frame, and the kcth frame of the sample image.

[0016] In some embodiments, the parameters of the 2D-DIC algorithm include:

[0017] Image acquisition window size matching the size of the landmark;

[0018] Normalized product correlation coefficient quantifies the matching degree of the landmark areas in two adjacent frames;

[0019] The initial displacement value is obtained by rounding the coordinate difference of the key points detected by YOLO-Pose;

[0020] The Newton-Raphson algorithm is combined with the bilinear interpolation algorithm to calculate the sub-pixel displacement.

[0021] In some embodiments, the interpolation accuracy of the bilinear interpolation algorithm is 0.1 pixel.

[0022] In some embodiments, the kc-th frame sample image is used to calculate the relative displacement of the marker in the k-th frame image compared to the kc-th frame image using a 2D-DIC algorithm after every c calculation cycles as a reference for subsequent calculations.

[0023] In a second aspect, a dangerous rock displacement detection system includes:

[0024] The dangerous rock unit to be observed is used to fix markers on the dangerous rock;

[0025] An image acquisition unit is used to arrange cameras according to the location of the dangerous rock to be observed and to acquire images of the dangerous rock to be observed;

[0026] The landmark positioning unit is used to sample the collected dangerous rock images to be observed, detect the landmarks in the sample images through the YOLO-Pose network, and record the positions of the diagonal intersections of all landmarks;

[0027] a displacement calculation unit, configured to calculate, by a 2D-DIC algorithm, a sub-pixel precision displacement of the positions of the intersections of the diagonals of all the markers in the k-th frame sample image relative to the positions of the intersections of the diagonals of all the markers in the k-1-th frame sample image;

[0028] A curve drawing unit, configured to draw a displacement curve according to the sub-pixel precision displacement and record displacement change trends in the u and v directions;

[0029] a sliding trend warning unit, configured to determine, based on the displacement curve, whether the relative displacement of one or more markers over a certain period of time presents a fixed trend, and if so, trigger a dangerous rock sliding trend warning;

[0030] The fall warning unit is used to determine whether the cumulative displacement of one or more markers in a certain period of time exceeds the warning value according to the displacement curve. If so, it triggers the dangerous rock fall warning at the point.

[0031] In a third aspect, a computer-readable storage medium is provided, characterized in that one or more programs are stored therein, and when the one or more programs are executed, the above method can be implemented.

[0032] In a fourth aspect, a device includes a processor, a communication interface, a memory and a communication bus; the processor, the communication interface and the memory communicate with each other through the communication bus; it is characterized in that the processor is used to execute the program stored in the above-mentioned computer-readable storage medium.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] 1. Through markers and cameras, accurate monitoring and early warning of dangerous rock movement can be achieved around the clock, without human intervention, saving costs;

[0035] 2. After being installed on dangerous rock masses, markers coated with long-lasting luminescent materials do not require power supply or subsequent human maintenance and can provide clear and reliable features for video monitoring both during the day and at night;

[0036] 3. It can provide early warning for the cumulative displacement and relative displacement trend of dangerous rock masses, including early warning for dangerous rock falls and dangerous rock sliding trends, to better support advance warning of dangerous rock mass observations.

[0037] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing this application information. The purposes and other advantages of the present invention can be realized and obtained by the structures indicated in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 Schematic diagram of a dangerous rock displacement detection method according to this embodiment.

[0040] Figure 2 Schematic diagram of the dangerous rock marker in this embodiment.

[0041] Figure 3 Schematic diagram of the pixel positions of the YOLO-Pose calculated landmarks in this embodiment.

[0042] Figure 4 2D-DIC is a schematic diagram of the relative displacement of the markers calculated in this embodiment.

[0043] Figure 5 FIG. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting the present invention. To better illustrate the present embodiment, some components in the accompanying drawings may be omitted, enlarged, or reduced in size, and do not represent actual product dimensions. Those skilled in the art will appreciate that some well-known structures and their descriptions may be omitted from the accompanying drawings. The positional relationships depicted in the accompanying drawings are for illustrative purposes only and are not to be construed as limiting the present invention.

[0045] In some embodiments, as Figure 1 As shown, a dangerous rock displacement detection method includes:

[0046] Fix markers on the dangerous rocks to be observed;

[0047] Arrange cameras based on the locations of the dangerous rocks to be observed, and collect images of the dangerous rocks to be observed;

[0048] The collected images of dangerous rocks to be observed are sampled, and the landmarks in the sample images are detected using the YOLO-Pose network, and the positions of the diagonal intersections of all the landmarks are recorded;

[0049] Calculate the sub-pixel precision displacement of the diagonal intersections of all markers in the k-th frame sample image relative to the diagonal intersections of all markers in the k-1-th frame sample image using a 2D-DIC algorithm;

[0050] Drawing a displacement curve based on the sub-pixel precision displacement, and recording the displacement change trends in the u direction and the v direction;

[0051] Determining whether the relative displacement of one or more markers over a certain period of time presents a fixed trend based on the displacement curve, and if so, triggering a dangerous rock sliding trend warning;

[0052] Based on the displacement curve, it is determined whether the cumulative displacement of one or more markers in a certain period of time exceeds the warning value. If so, a dangerous rock fall warning at the point is triggered.

[0053] Specifically, the markers are Figure 2 As shown, a marker is coated with a long afterglow luminescent material. The marker is in the shape of a rectangle with intersecting diagonals or a plane geometric figure containing at least two intersecting diagonals. It is then fixed on the dangerous rock. It can absorb sunlight energy during the day and emit afterglow at night. One or more markers can be fixed on the dangerous rock body to be observed according to actual needs.

[0054] Furthermore, a surveillance camera is installed at a suitable distance in front of the dangerous rock mass to be observed, continuously collecting images of the dangerous rock mass to be observed, and ensuring that the size of the marker in the image is not less than Pixels.

[0055] Further, such as Figure 3 As shown, the collected dangerous rock images to be observed are sampled, and the k-th frame image, k-1-th frame image and kc-th frame image of the sample image are input into the YOLO-Pose network. The YOLO-Pose network locates the positions of the diagonal intersections of all markers in the k-th frame, k-1-th frame and kc-th frame of the sample image, and calculates the initial displacement of the diagonal intersection positions of all markers on the dangerous rock in the k-th frame image relative to the diagonal intersection positions of the markers on the dangerous rock in the previous c-frame images based on the k-th frame and kc-th frame images. The initial displacement of the diagonal intersection positions of the markers on the dangerous rock relative to the diagonal intersection positions of the markers on the dangerous rock in the previous 1-frame image is also calculated based on the k-th frame image and k-1-th frame image. In addition, the accumulated error is eliminated by taking every c frames as a calculation cycle.

[0056] Furthermore, the displacement of the intersection points of the diagonals of all the markers in the k-th frame sample image relative to the intersection points of the diagonals of all the markers in the k-1-th frame sample image is calculated using the 2D-DIC algorithm.

[0057] Specifically, the size of the marker is used as the matching image acquisition window, and the normalized product correlation coefficient (ZNCC) is used to quantify the degree of matching of the marker area in two adjacent frames of images. The outliers are eliminated, and then the coordinate difference of the intersection of the diagonal lines of the marker monitored by the YOLO-Pose network is rounded as the initial displacement value in the u and v directions. The Newton-Raphson (NR iteration) algorithm is combined with the bilinear interpolation algorithm to calculate the sub-pixel displacement, where the interpolation accuracy of the bilinear interpolation algorithm is 0.1 pixel.

[0058] Furthermore, the sub-pixel precision displacement of all markers in the current frame sampling image relative to the previous frame sampling image is recorded and a displacement curve is drawn. The displacement change trend in the u and v directions is recorded. Every c calculation cycles, the 2D-DIC algorithm is used to calculate the relative displacement of the markers in the k-th frame image compared to the kc-th frame image as the benchmark for subsequent calculations to eliminate the cumulative displacement error of the calculation, where c can be taken as 100.

[0059] Specifically, such as Figure 4 As shown, the initial displacement of the marker, the k-th frame image, and the ki-th frame image are input into 2D-DIC for calculation. If the precise relative displacement of one or more markers in the k-th frame sampling image relative to the ki-th frame sampling image shows a fixed trend over a certain period of time, it indicates that the dangerous rock at that location has a tendency to slide, triggering a dangerous rock displacement trend warning. If the cumulative displacement obtained by superimposing the relative displacements over a certain period of time exceeds the warning value, a dangerous rock fall warning for that location is triggered.

[0060] In some embodiments, a dangerous rock displacement detection system includes:

[0061] The dangerous rock unit to be observed is used to fix markers on the dangerous rock;

[0062] An image acquisition unit is used to arrange cameras according to the location of the dangerous rock to be observed and to acquire images of the dangerous rock to be observed;

[0063] The landmark positioning unit is used to sample the collected dangerous rock images to be observed, detect the landmarks in the sample images through the YOLO-Pose network, and record the positions of the diagonal intersections of all landmarks;

[0064] a displacement calculation unit, configured to calculate, by a 2D-DIC algorithm, a sub-pixel precision displacement of the positions of the intersections of the diagonals of all the markers in the k-th frame sample image relative to the positions of the intersections of the diagonals of all the markers in the k-1-th frame sample image;

[0065] A curve drawing unit, configured to draw a displacement curve according to the sub-pixel precision displacement and record displacement change trends in the u and v directions;

[0066] a sliding trend warning unit, configured to determine, based on the displacement curve, whether the relative displacement of one or more markers over a certain period of time presents a fixed trend, and if so, trigger a dangerous rock sliding trend warning;

[0067] The fall warning unit is used to determine whether the cumulative displacement of one or more markers in a certain period of time exceeds the warning value according to the displacement curve. If so, it triggers the dangerous rock fall warning at the point.

[0068] The embodiment of the present disclosure also provides an electronic device, such as Figure 5 As shown, the system includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus.

[0069] The memory is a computer-readable storage medium for storing one or more programs.

[0070] The processor is configured to execute a program stored in a computer-readable storage medium.

[0071] The computer-readable storage medium may be included in the device / apparatus described in the above embodiments; or may exist independently without being assembled into the device / apparatus.

[0072] Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting displacement of dangerous rocks, characterized in that: include: Fix markers on the dangerous rocks to be observed; Using a long-lasting luminescent material to paint a marker, wherein the marker is in the shape of a plane geometric figure containing at least two intersecting diagonal lines; Arrange cameras based on the locations of the dangerous rocks to be observed, and collect images of the dangerous rocks to be observed; The collected images of dangerous rocks to be observed are sampled, and the landmarks in the sample images are detected using the YOLO-Pose network, and the positions of the diagonal intersections of all the landmarks are recorded; The YOLO-Pose network is used to locate the positions of the diagonal intersections of all markers in the kth, k-1th, and kcth frames of the sample image. Based on the kth and kcth frames, the initial displacements of the diagonal intersections of all markers on the dangerous rock in the kth frame image relative to the diagonal intersections of the markers on the dangerous rock in the previous c frames are calculated. The initial displacements of the diagonal intersections of the markers on the dangerous rock in the kth frame image relative to the diagonal intersections of the markers on the dangerous rock in the previous frame image are also calculated based on the kth and k-1th frames. In addition, the cumulative error is eliminated by taking every c frames as a calculation cycle. Calculate the sub-pixel precision displacement of the diagonal intersections of all markers in the k-th frame sample image relative to the diagonal intersections of all markers in the k-1-th frame sample image using a 2D-DIC algorithm; The parameters of the 2D-DIC algorithm include: Image acquisition window size matching the size of the landmark; Normalized product correlation coefficient quantifies the matching degree of the landmark areas in two adjacent frames; The initial displacement value is obtained by rounding the coordinate difference of the key points detected by YOLO-Pose; The Newton-Raphson algorithm is combined with the bilinear interpolation algorithm to calculate the displacement with sub-pixel accuracy; Drawing a displacement curve based on the sub-pixel precision displacement, and recording the displacement change trends in the u direction and the v direction; Determining whether the relative displacement of one or more markers over a certain period of time presents a fixed trend based on the displacement curve, and if so, triggering a dangerous rock sliding trend warning; Based on the displacement curve, it is determined whether the cumulative displacement of one or more markers in a certain period of time exceeds a warning value. If so, a dangerous rock fall warning is triggered.

2. A dangerous rock displacement detection method according to claim 1, characterized in that: The size of the marker in the dangerous rock image to be observed is greater than or equal to 50*50 pixels.

3. A dangerous rock displacement detection method according to claim 1, characterized in that: The interpolation accuracy of the bilinear interpolation algorithm is 0.1 pixel.

4. A dangerous rock displacement detection method according to claim 1, characterized in that: The kc-th frame sample image is used to calculate the relative displacement of the marker in the k-th frame image compared to the kc-th frame image using the 2D-DIC algorithm after every c calculation cycles as a benchmark for subsequent calculations.

5. A dangerous rock displacement detection system, characterized in that: include: The dangerous rock unit to be observed is used to fix a marker on the dangerous rock, and the marker is coated with a long-afterglow luminescent material, wherein the marker is in the shape of a plane geometric figure containing at least two intersecting diagonals; An image acquisition unit is used to arrange cameras according to the location of the dangerous rock to be observed and to acquire images of the dangerous rock to be observed; a landmark positioning unit, configured to perform frame sampling on the collected dangerous rock images to be observed, detect landmarks in the sample images using a YOLO-Pose network, record the positions of the diagonal intersections of all landmarks, locate the positions of the diagonal intersections of all landmarks in the k-th frame, the k-1-th frame, and the kc-th frame of the sample images using the YOLO-Pose network, calculate the initial displacement of the positions of the diagonal intersections of all landmarks on the dangerous rock in the k-th frame image relative to the positions of the diagonal intersections of the landmarks on the dangerous rock in the previous c-th frame images based on the k-th frame and the kc-th frame images, and calculate the initial displacement of the positions of the diagonal intersections of the landmarks on the dangerous rock relative to the positions of the diagonal intersections of the landmarks on the dangerous rock in the previous 1-frame image based on the k-th frame and the k-1-th frame images; and eliminate the accumulated error by calculating every c frames as one cycle; A displacement calculation unit is configured to calculate the sub-pixel precision displacement of the positions of the intersections of the diagonals of all markers in the k-th frame sample image relative to the positions of the intersections of the diagonals of all markers in the k-1-th frame sample image using a 2D-DIC algorithm, wherein the parameters of the 2D-DIC algorithm include: Image acquisition window size matching the size of the landmark; Normalized product correlation coefficient quantifies the matching degree of the landmark areas in two adjacent frames; The initial displacement value is obtained by rounding the coordinate difference of the key points detected by YOLO-Pose; The Newton-Raphson algorithm is combined with the bilinear interpolation algorithm to calculate the displacement with sub-pixel accuracy; A curve drawing unit, configured to draw a displacement curve according to the sub-pixel precision displacement and record displacement change trends in the u and v directions; a sliding trend warning unit, configured to determine, based on the displacement curve, whether the relative displacement of one or more markers over a certain period of time presents a fixed trend, and if so, trigger a dangerous rock sliding trend warning; The fall warning unit is used to determine whether the cumulative displacement of one or more markers in a certain period of time exceeds the warning value according to the displacement curve, and if so, trigger the dangerous rock fall warning.

6. A computer-readable storage medium, characterized in that One or more programs are stored, and when the one or more programs are executed, the method according to any one of claims 1 to 4 can be implemented.

7. A device comprising a processor, a communication interface, a memory, and a communication bus; the processor, the communication interface, and the memory communicate with each other via the communication bus; characterized in that: The processor is configured to execute the program stored in the computer-readable storage medium according to claim 6.

Images