Binocular vision road slope displacement monitoring method and system, electronic equipment and medium

Abstract

The invention provides a binocular vision road slope displacement monitoring method and system, electronic equipment and a medium, and belongs to the field of geological disaster monitoring, and the method comprises the steps: calibrating two cameras disposed on one side of a road, and obtaining calibration parameters; the two cameras form a binocular camera, a preset distance is kept, and optical axes cover a plurality of ArUco targets arranged on the slope protection structure; collecting a monitored area image based on the calibrated binocular camera, and extracting a unique code and a corner pixel coordinate of each ArUco target from the monitored area image; based on the calibration parameters and the pixel coordinates, reconstructing angular point three-dimensional coordinates, and performing center fitting to obtain fitted three-dimensional coordinates as monitoring points; and monitoring the slope displacement according to the three-dimensional coordinate change of the monitoring point at the previous and later moments. According to the invention, convenient, efficient and reliable slope displacement monitoring is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of geological disaster monitoring, in particular to a binocular vision highway slope displacement monitoring method, system, electronic device and medium. BACKGROUND

[0002] The development of computer vision technology has given birth to binocular displacement monitoring methods, which reconstruct the three-dimensional coordinates of the target by simulating the principle of human eye parallax, providing a non-contact and low-cost solution for slope monitoring. However, existing binocular technology is mostly verified in controlled environments and has not yet broken through the application barriers of large-scale outdoor scenes.

[0003] The core defects of existing binocular technology and quantized images are: camera pose offset caused by outdoor vibration and temperature change, static calibration parameters (rotation matrix R, translation vector T) error up to ±0.5° within 48 hours, causing depth calculation deviation >10%; overexposure of the corner point positioning offset caused by the overexposed area of the target under strong light (error up to 3-5 pixels on sunny days), and rain and fog weather leading to a traditional texture matching false detection rate of over 40%; the baseline distance of existing systems is mostly less than 2 meters (requiring special supports), resulting in a depth error of >20mm for a 200-meter monitoring point; and no active anti-shake mechanism, wind vibration introduces ±2mm displacement noise.

[0004] Existing methods cannot reconcile the three contradictions: calibration accuracy persistence, environmental interference robustness, and engineering cost control, forming an irreconcilable three paradoxes. The core pain points are: 1. Parameter drift destroys measurement continuity; 2. Light sensitivity leads to matching failure; 3. Special hardware limits large-scale deployment. To break out of this dilemma, dynamic external parameter optimization, anti-interference decoding algorithm, and embedded integration of monitoring facilities are needed. SUMMARY

[0005] Therefore, it is necessary to provide a binocular vision highway slope displacement monitoring method, system, electronic device and medium to realize convenient, efficient and reliable slope displacement monitoring and improve the ability of highway slope safety monitoring and geological disaster early warning.

[0006] To solve the above technical problems, in a first aspect, the present application provides a binocular vision highway slope displacement monitoring method, comprising: Calibrating the camera parameters of two cameras arranged on one side of the highway to obtain calibration parameters; the two cameras together form a binocular camera, the two cameras maintain a predetermined distance, and the optical axes of the two cameras cover a slope monitoring area where a plurality of ArUco targets are located, and the ArUco targets are arranged on the highway slope protection structure; Based on the calibrated binocular camera, images of a slope monitoring area containing multiple ArUco targets are synchronously collected, and the unique code of each ArUco target and the pixel coordinates of the corner points of each ArUco target are extracted from the monitoring area images; Based on the calibration parameters and the pixel coordinates, the three-dimensional coordinates of the corner points of each ArUco target are reconstructed, and the three-dimensional coordinates of the corner points of each ArUco target are fitted to obtain fitted three-dimensional coordinates, and the points corresponding to the fitted three-dimensional coordinates are taken as monitoring points; According to the change of the three-dimensional coordinates of the monitoring points at the front and rear moments, the slope displacement is monitored.

[0007] In a possible implementation, the camera parameters of the two cameras arranged on one side of the highway are calibrated to obtain calibration parameters, including: The calibration board arranged on the opposite side of the highway is taken as a reference target to calibrate the internal parameters and external parameters of the binocular camera, the shooting area of the binocular camera completely covers the overlapping field of view where the calibration board is located, and the calibration board is a checkerboard calibration board arranged on the slope protection structure of the highway; The internal parameters include the focal length, principal point and distortion coefficient of a single camera, and the external parameters include the rotation matrix and translation vector of the binocular camera.

[0008] In a possible implementation, the binocular camera after calibration synchronously collects images of a slope monitoring area containing multiple ArUco targets, and the unique code of each ArUco target and the pixel coordinates of the corner points of each ArUco target are extracted from the monitoring area images, including: Two images of the left and right of the slope detection area shot by the binocular camera are collected, and the collected images are preprocessed; The quadrilateral regions of multiple ArUco targets in the two preprocessed images are extracted, and the quadrilateral regions are projected into squares; The squares are divided into a plurality of grids, each grid is encoded based on the percentage of black and white pixels in the grid, a corresponding binary symbol is generated, all binary symbols are arranged in order, and the unique code of each ArUco target is obtained; The pixel coordinates of the corner points of each ArUco target are obtained in a counterclockwise direction.

[0009] In a possible implementation, the collected images are preprocessed, including: The collected images are denoised and enhanced; Based on the distortion coefficient of the binocular camera, the denoised and enhanced images are corrected for distortion; Based on the external parameters of the binocular camera, the images after distortion correction are corrected for epipolar line; The imaging planes of the left and right images after the epipolar rectification are coplanar and the corresponding epipolar lines are aligned.

[0010] In a possible implementation, the preset interval is a baseline distance of the two cameras; and the reconstructing of the three-dimensional coordinates of the ArUco target corner points based on the calibration parameters and the pixel coordinates comprises: determining a parallax between the binocular cameras based on the left camera coordinate system, determining depths of the ArUco target corner points based on the parallax, the focal length and the baseline distance, wherein the depth is a Z-axis coordinate value of the ArUco target corner point in the left camera coordinate system, the optical center of the left camera is the coordinate origin, and the optical axis direction is the positive direction of the Z axis; determining X and Y coordinates of the ArUco target corner points based on the focal length, the principal point, the pixel coordinates and the depth, and combining the X, Y and Z coordinates into a three-dimensional vector as the three-dimensional coordinates of the ArUco target corner points in the left camera coordinate system.

[0011] In a possible implementation, the monitoring of the slope displacement based on the changes of the three-dimensional coordinates of the monitoring points at the previous and next time points comprises: determining a difference vector of the three-dimensional coordinates at the two time points based on the three-dimensional coordinates of the monitoring points collected at the previous and next time points; determining the displacement of the monitoring point by using a three-dimensional Euclidean distance formula based on the difference vector.

[0012] In a second aspect, the application further provides a binocular vision highway slope displacement early warning method, comprising: determining the displacement of the slope monitoring point based on the binocular vision highway slope displacement monitoring method in any of the above possible implementations; setting a corresponding displacement early warning threshold for the slope with different geology; triggering an audible and light alarm when the displacement of the slope exceeds the early warning threshold.

[0013] In a third aspect, the application further provides a binocular vision highway slope displacement monitoring system, comprising: a camera parameter calibration unit configured to calibrate camera parameters of two cameras arranged on one side of the highway to obtain calibration parameters; the two cameras together form a binocular camera, the two cameras are kept at a preset interval, and the optical axes of the two cameras cover a slope monitoring area where the ArUco target is arranged; a pixel coordinate extraction unit configured to extract unique codes of each ArUco target and pixel coordinates of each ArUco target corner point from the collected images based on the calibrated binocular camera to synchronously collect slope monitoring area images containing multiple ArUco targets; a three-dimensional coordinate reconstruction unit, configured to reconstruct respective three-dimensional coordinates of respective corner points of the ArUco target based on the calibration parameters and the pixel coordinates, and to perform center fitting on the three-dimensional coordinates of the respective corner points of the ArUco target to obtain fitted three-dimensional coordinates, and to take a point corresponding to the fitted three-dimensional coordinates as a monitoring point; a slope displacement monitoring unit, configured to monitor a slope displacement according to a change in the three-dimensional coordinates of the monitoring point at different time points.

[0014] In a fourth aspect, the present application also provides an electronic device, comprising a memory and a processor, wherein, the memory is configured to store a program; the processor, coupled to the memory, is configured to execute the program stored in the memory to implement the steps of the binocular vision-based slope displacement monitoring method and / or the binocular vision-based slope displacement early warning method.

[0015] In a fifth aspect, the present application also provides a computer-readable storage medium, which stores one or more programs, and the one or more programs are executable by one or more processors to implement the steps of the binocular vision-based slope displacement monitoring method and / or the binocular vision-based slope displacement early warning method.

[0016] The binocular vision-based slope displacement monitoring method provided by the present application firstly calibrates camera parameters of two cameras arranged on one side of a road to obtain calibration parameters, and the two cameras jointly form a binocular camera and maintain a preset interval, wherein maintaining the preset interval between the two cameras can greatly reduce the calculation error of the depth in the three-dimensional coordinates of the monitoring point, thereby improving the accuracy of displacement detection; then, the binocular camera after calibration synchronously collects images of a slope monitoring area containing a plurality of ArUco targets, extracts unique codes of each ArUco target and pixel coordinates of corner points of each ArUco target from the collected images, wherein the ArUco target is composed of an internal binary matrix, and the decoding of the unique code can be realized; further, based on the calibration parameters and the pixel coordinates, the three-dimensional coordinates of the respective corner points of the ArUco target are reconstructed, and center fitting is performed on the three-dimensional coordinates of the respective corner points of the ArUco target to obtain fitted three-dimensional coordinates, and a point corresponding to the fitted three-dimensional coordinates is taken as a monitoring point, wherein the center fitting of the corner points can resist single-point detection jitter; finally, the slope displacement is monitored according to the change in the three-dimensional coordinates of the monitoring point at different time points, the displacement amount is calculated through the difference in the three-dimensional space coordinates, the spatial vector change of the slope displacement can be accurately reflected, compared with the traditional single-point measurement method, the displacement misjudgment caused by the lack of measurement dimension is effectively avoided, and the accuracy and reliability of geological disaster warning are greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0018] Figure 1 An embodiment flowchart of a double-vision highway slope displacement monitoring method provided by the present application is shown in the figure. Figure 2 An embodiment schematic diagram of target point arrangement provided by the present application is shown in the figure. Figure 3 An embodiment flowchart of a double-vision highway slope displacement monitoring method provided by the present application is shown in the figure. Figure 1 An embodiment flowchart of S101 in the method is shown in the figure. Figure 4 An embodiment schematic diagram of coordinate system conversion provided by the present application is shown in the figure. Figure 5 An embodiment schematic diagram of camera calibration calculation formula provided by the present application is shown in the figure. Figure 6 An embodiment flowchart of S102 in the method is shown in the figure. Figure 1 An embodiment flowchart of S102 in the method is shown in the figure. Figure 7 An embodiment flowchart of image preprocessing provided by the present application is shown in the figure. Figure 8 An embodiment schematic diagram of epipolar line correction provided by the present application is shown in the figure. Figure 9 An embodiment flowchart of three-dimensional coordinate reconstruction of ArUco target corners provided by the present application is shown in the figure. Figure 10 An embodiment schematic diagram of three-dimensional displacement calculation provided by the present application is shown in the figure. Figure 11 An embodiment flowchart of S104 in the method is shown in the figure. Figure 1 An embodiment flowchart of S104 in the method is shown in the figure. Figure 12 An embodiment flowchart of a double-vision highway slope displacement pre-warning method provided by the present application is shown in the figure. Figure 13 An embodiment flowchart of double-vision highway slope displacement monitoring and pre-warning provided by the present application is shown in the figure. Figure 14 An embodiment schematic diagram of a double-vision camera and a marker highway slope installation structure provided by the present application is shown in the figure. Figure 15An embodiment structure schematic diagram of a binocular vision highway slope displacement monitoring system provided by the present application is shown in the figure. Figure 16 An embodiment structure schematic diagram of an electronic device provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present application will be clearly and completely described in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person skilled in the art without creative work fall within the scope of protection of the present application.

[0020] In the description of the embodiments of the present application, unless otherwise specified, the meaning of “a plurality of” is two or more. The association relationship of the associated objects is described by “and / or”, which means that there can be three relationships, for example: A and / or B can represent the three cases of A alone, A and B together, and B alone.

[0021] The “first”, “second” and the like described in the embodiments of the present application are only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the technical features limited by “first” and “second” can explicitly or implicitly include at least one of the features.

[0022] In this document, the term “embodiment” means that the specific features, structures or characteristics described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor is it independent or alternative to other embodiments. The person skilled in the art explicitly and implicitly understands that the embodiments described herein can be combined with other embodiments.

[0023] Before the embodiments are displayed, the following terms are explained.

[0024] ArUco target: ArUco marker, a two-dimensional barcode or marker system used in computer vision and augmented reality fields, developed by the Computer Vision Center (CVC) of the University of Zaragoza, Spain. It consists of a binary pattern composed of black and white squares, each marker has a unique identifier, which can be detected and recognized by computer vision algorithms, and provides the position, orientation and size information of the marker. In computer vision applications, ArUco targets are often used as calibration boards for camera calibration, pose estimation and virtual object positioning in augmented reality scenes, etc.

[0025] Gimbal: A stabilizing device that counteracts device vibrations through mechanical or electronic compensation, ensuring image acquisition stability.

[0026] GPS / PPS synchronization device: A time synchronization module based on satellite second pulse signals, enabling millisecond-level coordination among multiple devices.

[0027] Road monitoring column: A support structure along the road for mounting monitoring devices, typically made of concrete or steel.

[0028] Epipolar geometry constraint: A mathematical principle in stereo vision that describes the correspondence between points in two views, used for camera pose calculation.

[0029] Frequency domain phase correlation method: An algorithm that converts images to the frequency domain to analyze phase information, achieving illumination-robust matching.

[0030] Radial line correction: A preprocessing method that transforms images so that corresponding points in binocular images lie on the same horizontal line.

[0031] Three-dimensional Euclidean distance: The straight-line distance between two points in three-dimensional space, representing the actual displacement.

[0032] Common view area: The overlapping area of the fields of view of binocular cameras, a necessary condition for stereo matching.

[0033] The present application provides a binocular vision highway slope displacement monitoring method, system, electronic device and medium, which are described below.

[0034] Figure 1 An embodiment flowchart of the binocular vision highway slope displacement monitoring method provided by the present application is shown in Figure 1 The binocular vision highway slope displacement monitoring method includes: S101, calibrate the camera parameters of two cameras arranged on one side of the highway to obtain calibration parameters; the two cameras together form a binocular camera, and the optical axes of the two cameras cover a slope monitoring area where a plurality of ArUco targets are arranged on the highway slope protection structure; In some embodiments of the present application, ArUco targets (markers) are arranged at certain distances on the target slope, and a horizontal section line is arranged on the slope structure with a horizontal interval of 30-50m, and 3 ArUco targets are arranged on each section line to provide obvious and easily identifiable monitoring points; in some embodiments of the present application, the ArUco target is a high-reflectivity coded ArUco target that can actively emit light.

[0035] In some embodiments of the present application, the binocular camera is two high-resolution, low-distortion cameras, which are installed on a highway monitoring column at a height of 8-12 m; the preset distance between the binocular cameras is 10-15 m, and the lenses are aligned with the monitoring area of the slope and the highway to obtain synchronous and clear slope images; preferably, a stabilizing holder is installed between the camera and the column to reduce the influence of wind vibration; preferably, a GPS / PPS synchronization device is added to the binocular camera to ensure that the time error of the two cameras is less than 1 ms, to avoid inter-frame motion blur, and the frame rate of the camera is preferably 30-60 Hz. .

[0036] It should be noted that the preset distance between the binocular cameras is tens of meters, which can reduce the monitoring error.

[0037] In some embodiments of the present application, an embodiment of the arrangement of the target points is shown in Figure 2 .

[0038] S102, synchronously collecting the slope monitoring area images containing a plurality of ArUco targets based on the calibrated binocular camera, extracting the unique code of each ArUco target and the pixel coordinates of each ArUco target corner from the monitoring area images; S103, reconstructing the three-dimensional coordinates of each corner of the ArUco target based on the calibration parameters and the pixel coordinates, and performing center fitting on the three-dimensional coordinates of each corner of the ArUco target to obtain fitted three-dimensional coordinates, and taking the point corresponding to the fitted three-dimensional coordinates as the monitoring point; S104, monitoring the displacement of the slope according to the change of the three-dimensional coordinates of the monitoring point at the previous and subsequent time.

[0039] In some embodiments of the present application, the time interval of the monitoring point at the previous and subsequent time can be set to one day.

[0040] In summary, the binocular vision highway slope displacement monitoring method provided by the application firstly calibrates camera parameters of two cameras arranged on one side of the highway to obtain calibration parameters, the two cameras jointly form a binocular camera, and a preset interval is maintained between the two cameras, wherein maintaining the preset interval between the two cameras can greatly reduce the calculation error of the depth in the three-dimensional coordinates of the subsequent monitoring points, thereby improving the accuracy of displacement detection; then, based on the calibrated binocular camera, an image of a slope monitoring area containing a plurality of ArUco targets is synchronously collected, the unique code of each ArUco target and the pixel coordinates of each ArUco target corner point are extracted from the collected image, wherein since the ArUco target is composed of an internal binary matrix, the decoding of the unique code can be realized; further, based on the calibration parameters and the pixel coordinates, the three-dimensional coordinates of each corner point of the ArUco target are reconstructed, the three-dimensional coordinates of each corner point of the ArUco target are center fitted to obtain fitted three-dimensional coordinates, and the points corresponding to the fitted three-dimensional coordinates are taken as monitoring points, wherein the center fitting of the corner points can resist single-point detection jitter; finally, the slope displacement is monitored according to the change of the three-dimensional coordinates of the monitoring points at the previous and subsequent time, the displacement amount is calculated through the three-dimensional space coordinate difference, the spatial vector change of the slope displacement can be accurately reflected, compared with the traditional single-point measurement method, the displacement misjudgment caused by the missing of the measurement dimension is effectively avoided, and the accuracy and reliability of the geological disaster warning are greatly improved.

[0041] In some embodiments of the application, as shown in Figure 3 , Figure 1 The step S101 comprises: S301, taking the calibration board arranged on the opposite side of the highway as a reference target, calibrating the internal parameters and external parameters of the binocular camera, the shooting area of the binocular camera completely covers the overlapping field of view where the calibration board is located, and the calibration board is a chessboard calibration board arranged on the slope protection structure of the highway slope; S302, the internal parameters include the focal length, principal point and distortion coefficient of a single camera, and the external parameters include the rotation matrix and translation vector of the binocular camera.

[0042] It should be noted that when the binocular camera is calibrated and the monitoring point coordinates are obtained, the conversion of four coordinate systems is involved, wherein is a pixel coordinate system; is an image coordinate system; is a camera coordinate system; is a world coordinate system, the pixel coordinate system is a pixel unit, and the remaining coordinate systems are length units, and the point is a real coordinate point in the world coordinate system, is a pixel coordinate point in the corresponding pixel coordinate system.

[0043] It should be noted that the conversion from the world coordinate system to the camera coordinate system is a rigid body transformation, that is, the object will not be deformed, and only rotation ) and translation ( ), which involves the extrinsic matrix of the camera; the conversion from the camera coordinate system to the image coordinate system is called perspective projection, which is a mapping from three-dimensional space to two-dimensional plane, and is realized by the principle of similar triangles, and the numerical value involves the focal length of the camera internal parameter; the conversion from the image coordinate system to the pixel coordinate system needs to consider the discretization of the pixel, and the numerical value involves the physical size of the image pixel and the pixel coordinate value of the origin of the image coordinate system in the pixel coordinate system.

[0044] In some embodiments of the application, the schematic diagram of coordinate system conversion is as shown in Figure 4 .

[0045] In some embodiments of the application, the schematic diagram of camera calibration calculation formula is as shown in Figure 5 .

[0046] In some embodiments of the application, Zhang Zhengyou calibration method is used to obtain the internal parameter matrix (focal length , principal point , radial distortion coefficient and tangential distortion coefficient ) of the two cameras respectively, and the calibration reprojection error is the pixel difference between the image point and the three-dimensional point cloud parameter reprojected to the two-dimensional image, so as to evaluate the accuracy of the internal parameter calibration.

[0047] In some embodiments of the application, as shown in Figure 6 , Figure 1 , step S102 comprises: S601, collecting left and right two images of the slope detection area shot by the binocular camera, and pre-processing the collected images; S602, extracting the quadrilateral region of a plurality of ArUco targets in the two pre-processed images, and projecting the quadrilateral region into a square; S603, dividing the square into a plurality of grids, encoding each grid based on the percentage of black and white pixels in the different grids, generating corresponding binary symbols, and arranging all the binary symbols in order to obtain the unique code of each ArUco target; S604, obtaining the pixel coordinates of each corner point of each ArUco target in the counterclockwise direction.

[0048] It should be noted that: in order to overcome the sensitivity of ArUco to strong light conditions in the daytime, a phase correlation method based on frequency domain is used to realize the displacement change independent of light.

[0049] In some embodiments of the application, the unique code of ArUco is calculated by an internal binary matrix, and the decoding and checking of the unique code are realized by Hamming code.

[0050] In some embodiments of the present application, the collected images are preprocessed, including: S701, denoising and enhancing the collected images; S702, based on the distortion coefficient of the binocular camera, correcting the distortion of the denoised and enhanced images; S703, based on the extrinsic parameter of the binocular camera, performing epipolar rectification on the distortion-corrected images; Wherein, the imaging planes of the left and right two images after epipolar rectification are coplanar, and the corresponding epipolar lines are aligned.

[0051] In some embodiments of the present application, median filtering is used for image denoising to remove random noise and salt and pepper noise in the image; image histogram equalization is used for image enhancement to improve the overall contrast and visual effect of the image.

[0052] In some embodiments of the present application, the schematic diagram of epipolar rectification is as shown in Figure 8 .

[0053] In some embodiments of the present application, the preset distance maintained between the two cameras is the baseline distance set value between the binocular cameras; as shown in Figure 8 , based on the calibrated parameters, the three-dimensional coordinates of the ArUco target corner points are reconstructed, including: S901, taking the left camera coordinate system as the reference, determining the parallax between the binocular cameras, based on the parallax, focal length and baseline distance, determining the depth of each corner point of the ArUco target, the depth being the Z-axis coordinate value of the ArUco target corner point in the left camera coordinate system, wherein the left camera optical center is the coordinate origin and the optical axis direction is the positive direction of the Z-axis; S902, based on the focal length, principal point, pixel coordinates and depth, determining the X and Y coordinates of each corner point of the ArUco target, combining the X, Y and Z coordinates into a three-dimensional vector as the three-dimensional coordinates of each corner point of the ArUco target in the left camera coordinate system.

[0054] In some embodiments of the present application, the schematic diagram of three-dimensional displacement calculation is as shown in Figure 10 .

[0055] In some embodiments of the present application, for a point (corner point of the ArUco target) in space, the image point coordinates on the left camera imaging plane are , the image point coordinates on the right camera imaging plane are , the focal length of the camera is , the baseline distance of the two cameras is , then according to the parallax , the calculation formula of the depth of the object can be derived as By using the above parallax principle, the subsequent three-dimensional space coordinate reconstruction calculation is as follows Figure 6 According to the triangular principle, the three-dimensional coordinate calculation formula is , , Combined with the binocular parallax principle and the target three-dimensional geometric constraint (depth information and encoding direction), the left and right camera two-dimensional coordinates (x, y) are converted into spatial displacement . .

[0056] In some embodiments of the present application, as shown in Figure 11 , Figure 1 , the S104 step comprises: S1101, determining the difference vector of the three-dimensional coordinates of the monitoring points collected at the previous time and the next time based on the three-dimensional coordinates of the monitoring points collected at the previous time and the next time; S1102, determining the displacement of the monitoring point by using the three-dimensional Euclidean distance formula according to the difference vector.

[0057] In some embodiments of the present application, based on the difference vector , the displacement is calculated as , is a scalar, representing the linear displacement distance (unit: millimeter) of the monitoring point in the actual three-dimensional space.

[0058] Since the traditional slope monitoring mainly relies on manual regular inspection or post-analysis, it cannot avoid the lag of disaster occurrence. The present application can realize 7*24 hours all-weather automatic monitoring, and the purpose of monitoring is to avoid disaster occurrence and immediate response, so as to gain valuable golden time for personnel evacuation and engineering intervention, and fundamentally change the backward situation of passive response of the traditional way. Therefore, the present application also provides a visual highway slope displacement early warning method, comprising: S1201, determining the displacement of the slope monitoring point based on the binocular vision highway slope displacement monitoring method; S1202, setting corresponding displacement threshold values for slopes of different geologies; S1203, triggering an audible and light alarm when the slope displacement exceeds the early warning threshold.

[0059] In some embodiments of the present application, according to the Engineering Rock Mass Classification Standard GB / T 50218, the threshold values are set as follows: the first early warning of the soil slope (height <20m) is 3mm / day, and the second early warning of the rock slope (height >=20m) is 5mm / day. When the displacement value detected on the same day exceeds the set warning value, the environment is more dangerous, otherwise it is safer. Finally, the slope displacement value is displayed on the screen and the alarm is triggered; at the same time, a three-dimensional displacement vector cloud chart is generated in the cloud.

[0060] In some embodiments of the present application, by studying the relationship between the slope image plane two-dimensional pixel coordinates and the three-dimensional coordinates, the means of machine vision technology are introduced, the whole system program is developed by using MATLAB 2024a and Python 3.10 mixed programming language, and the NVIDIA Jetson AGX Xavier is used to process two-way 1080P@30fps in real time, so as to realize dynamic real-time slope displacement evaluation.

[0061] In some embodiments of the present application, the flow chart of binocular vision highway slope displacement monitoring and early warning is as shown in Figure 13

[0062] In some embodiments of the present application, the embodiment schematic diagram of the installation structure of the binocular camera and the marker on the highway slope is as shown in Figure 14

[0063] In order to better implement the binocular vision highway slope displacement monitoring method in the embodiments of the present application, on the basis of the binocular vision highway slope displacement monitoring method, as shown in Figure 15 the present application also provides a binocular vision highway slope displacement monitoring system, the binocular vision highway slope displacement monitoring system 1500 comprises: A camera parameter calibration unit 1501 is configured to calibrate the camera parameters of two cameras arranged on one side of the highway to obtain calibration parameters; the two cameras jointly form a binocular camera, the two cameras maintain a preset distance, and the optical axes of the two cameras cover a slope monitoring area where a plurality of targets are located, and ArUco targets are arranged on the highway slope protection structure; A pixel coordinate extraction unit 1502 is configured to synchronously collect slope monitoring area images containing a plurality of ArUco targets based on the calibrated binocular camera, and extract unique codes of each ArUco target and pixel coordinates of each ArUco target corner from the collected images; A three-dimensional coordinate reconstruction unit 1503 is configured to reconstruct each three-dimensional coordinate of each corner of the ArUco target based on the calibration parameters and the pixel coordinates, and perform center fitting on the three-dimensional coordinates of each corner of the ArUco target to obtain fitted three-dimensional coordinates, and take the points corresponding to the fitted three-dimensional coordinates as monitoring points; A slope displacement monitoring unit 1504 is configured to monitor the slope displacement according to the changes of the three-dimensional coordinates of the monitoring points at different time points.

[0064] The binocular vision highway slope displacement monitoring system 1500 provided by the above embodiments can realize the technical solutions described in the above binocular vision highway slope displacement monitoring method embodiments, and the principles of the specific implementation of the above modules or units can be referred to the corresponding contents in the above binocular vision highway slope displacement monitoring method embodiments, which will not be described here.

[0065] ​​like Figure 16 As shown, the present invention also provides an electronic device 1600. The electronic device 1600 includes a processor 1601, a memory 1602, and a display 1603. Figure 16 Only some components of the electronic device 1600 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0066] In some embodiments, processor 1601 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 1602 or process data, such as the binocular vision highway slope displacement monitoring method and / or the binocular vision highway slope displacement early warning method of the present invention.

[0067] In some embodiments, processor 1601 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 1601 may be local or remote. In some embodiments, processor 1601 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, inter-cloud, multi-cloud, or any combination thereof.

[0068] In some embodiments, memory 1602 may be an internal storage unit of electronic device 1600, such as a hard disk or memory of electronic device 1600. In other embodiments, memory 1602 may also be an external storage device of electronic device 1600, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 1600.

[0069] Furthermore, the memory 1602 may include both internal storage units of the electronic device 1600 and external storage devices. The memory 1602 is used to store application software and various types of data installed on the electronic device 1600.

[0070] In some embodiments, display 1603 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 1603 is used to display information from electronic device 1600 and to display a visual user interface. Components 1601-1603 of electronic device 1600 communicate with each other via a system bus.

[0071] In an embodiment, when the processor 1601 executes the binocular vision highway slope displacement monitoring program in the memory 1602, the following steps can be implemented: Calibrate the camera parameters of two cameras arranged on one side of the highway to obtain calibration parameters; the two cameras together form a binocular camera, the two cameras maintain a preset distance, and the optical axes of the two cameras cover a slope monitoring area where a plurality of ArUco targets are located, and the ArUco targets are arranged on the highway slope protection structure; Based on the calibrated binocular camera, the slope monitoring area image containing a plurality of ArUco targets is synchronously collected, and the unique code of each ArUco target and the pixel coordinates of each ArUco target corner point are extracted from the collected image; Based on the calibration parameters and the pixel coordinates, the three-dimensional coordinates of each corner point of the ArUco target are reconstructed, and the three-dimensional coordinates of each corner point of the ArUco target are fitted to obtain fitted three-dimensional coordinates, and the points corresponding to the fitted three-dimensional coordinates are taken as monitoring points; According to the change of the three-dimensional coordinates of the monitoring points at the front and rear moments, the slope displacement is monitored.

[0072] In an embodiment, when the processor 1601 executes the binocular vision highway slope displacement monitoring program in the memory 1602, the following steps can be implemented: Based on the binocular vision highway slope displacement monitoring method, the displacement of the slope monitoring point is determined; Set the corresponding displacement warning threshold for slopes of different geology; When the slope displacement exceeds the warning threshold, trigger the sound and light alarm.

[0073] It should be understood that: the processor 1601 in executing the binocular vision highway slope displacement monitoring program, and / or, the binocular vision highway slope displacement warning program in the memory 1602, in addition to the above functions, can also realize other functions, which can be referred to the description of the previous method embodiment.

[0074] Further, the type of the electronic device 1600 is not limited, and the electronic device 1600 can be a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop, or the like. Exemplary embodiments of the portable electronic device include, but are not limited to, a portable electronic device running an IOS, an android, a microsoft, or other operating system. The portable electronic device can also be another portable electronic device, such as a laptop having a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present application, the electronic device 1600 can not be a portable electronic device, but a desktop computer having a touch-sensitive surface (e.g., a touch panel). Correspondingly, the embodiments of the present application also provide a computer readable storage medium for storing computer readable programs or instructions, which are executed by a processor to implement the steps or functions in the binocular vision highway slope displacement monitoring method and / or the binocular vision highway slope displacement early warning method provided by the above method embodiments.

[0075] Those skilled in the art can understand that all or part of the processes of the above-mentioned embodiments can be completed by a computer program instructing related hardware (such as a processor, a controller, etc.) to complete, and the computer program can be stored in a computer readable storage medium. The computer readable storage medium is a magnetic disk, an optical disk, a read-only memory, or a random access memory, etc.

[0076] The binocular vision highway slope displacement monitoring method, system, electronic device, and medium provided by the present application are described in detail above, and specific examples are applied to explain the principles and implementation modes of the present application. The above description of the embodiments is only used to help understand the method and its core idea of the present application. For those skilled in the art, according to the idea of the present application, the specific implementation mode and application range can be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for monitoring displacement of a highway slope based on binocular vision, characterized in that, The application relates to a method for monitoring slope displacement based on ArUco targets. The method comprises the following steps: Camera parameters of two cameras arranged on one side of a road are calibrated to obtain calibration parameters; The two cameras jointly form a binocular camera, a preset interval is kept between the two cameras, and optical axes of the two cameras cover a slope monitoring area where a plurality of ArUco targets are arranged on a slope protection structure of the road; Based on the calibrated binocular camera, images of the slope monitoring area containing the plurality of ArUco targets are synchronously collected, unique codes of the ArUco targets and pixel coordinates of corner points of the ArUco targets are extracted from the monitoring area images; Based on the calibration parameters and the pixel coordinates, three-dimensional coordinates of the corner points of the ArUco targets are reconstructed, and the three-dimensional coordinates of the corner points of the ArUco targets are centrally fitted to obtain fitted three-dimensional coordinates, and points corresponding to the fitted three-dimensional coordinates are taken as monitoring points; 2. The method of claim 1, wherein, Slope displacement is monitored according to changes of the three-dimensional coordinates of the monitoring points at previous and subsequent time points. The calibration of the camera parameters of the two cameras arranged on one side of the road to obtain the calibration parameters comprises the following steps: A calibration board arranged on the opposite side of the road is taken as a reference target to calibrate internal parameters and external parameters of the binocular camera, a shooting area of the binocular camera completely covers an overlapping field of view where the calibration board is arranged, and the calibration board is a checkerboard calibration board arranged on the slope protection structure of the road; 3. The method of claim 1, wherein, The internal parameters include a focal length, a principal point and a distortion coefficient of a single camera, and the external parameters include a rotation matrix and a translation vector of the binocular camera. The synchronous collection of the images of the slope monitoring area containing the plurality of ArUco targets based on the calibrated binocular camera and the extraction of the unique codes of the ArUco targets and the pixel coordinates of the corner points of the ArUco targets from the monitoring area images comprise the following steps: Left and right images of the slope monitoring area shot by the binocular camera are collected, and the collected images are preprocessed; A quadrilateral region of the plurality of ArUco targets in the two preprocessed images is extracted, and the quadrilateral region is projected as a square; The square is divided into a plurality of grids, each grid is encoded based on a percentage of black and white pixels in the grid, corresponding binary symbols are generated, and the unique codes of the ArUco targets are obtained by sequentially arranging all the binary symbols; 4. The method of claim 3, wherein, Pixel coordinates of the corner points of the ArUco targets are obtained in a counterclockwise direction. The preprocessing of the collected images comprises the following steps: The collected images are denoised and enhanced; Based on the distortion coefficient of the binocular camera, the denoised and enhanced images are distortion-corrected; Based on the external parameters of the binocular camera, the distortion-corrected images are rectified by epipolar lines; 5. The method of claim 1, wherein, The imaging planes of the rectified left and right images are coplanar, and corresponding epipolar lines are aligned. With the left camera coordinate system as the reference, disparity between the binocular cameras is determined, and based on the disparity, focal length and baseline distance, depth of each corner point of the ArUco target is determined, the depth being the depth of the ArUco target corner point in the left camera coordinate system Z axis coordinate value, wherein the optical center of the left camera is the coordinate origin, and the optical axis direction is the positive direction of the Z axis. Based on the focal length, principal point, pixel coordinates and depth, determine the 3D coordinates of each corner of the ArUco target in the left camera coordinate system. X Y X , Y and Z combine the coordinates into a 3D vector as the 3D coordinates of each corner of the ArUco target in the left camera coordinate system.​​ 6. The method of claim 1, wherein, The preset interval is a baseline distance of the two cameras; and the reconstruction of the three-dimensional coordinates of the corner points of the ArUco targets based on the calibration parameters and the pixel coordinates comprises the following steps: The monitoring of the slope displacement according to the changes of the three-dimensional coordinates of the monitoring points at the previous and subsequent time points comprises the following steps: Determine a difference vector of the three-dimensional coordinates of the monitoring points collected at the previous time and the next time; According to the difference vector, determine the displacement of the monitoring points by using a three-dimensional Euclidean distance formula.

7. A binocular vision highway slope displacement early warning method, comprising: Based on the binocular vision highway slope displacement monitoring method of any one of claims 1-6, determine the displacement of the slope monitoring points; Set corresponding displacement early warning thresholds for slopes of different geologies; When the slope displacement exceeds the early warning threshold, trigger an audible and visual alarm.

8. A binocular vision highway slope displacement monitoring system, characterized in that, Comprise: A camera parameter calibration unit for calibrating the camera parameters of two cameras arranged on one side of the highway to obtain calibration parameters; The two cameras together form a binocular camera, the two cameras maintain a predetermined distance, and the optical axes of the two cameras cover a slope monitoring area where a plurality of targets are located, and the ArUco targets are arranged on the slope protection structure of the highway; A pixel coordinate extraction unit for synchronously collecting slope monitoring area images containing a plurality of ArUco targets based on the calibrated binocular camera, and extracting the unique code of each ArUco target and the pixel coordinates of the corners of each ArUco target from the collected images; A three-dimensional coordinate reconstruction unit for reconstructing the three-dimensional coordinates of each corner of the ArUco target based on the calibration parameters and the pixel coordinates, and performing center fitting on the three-dimensional coordinates of each corner of the ArUco target to obtain fitted three-dimensional coordinates, and taking the points corresponding to the fitted three-dimensional coordinates as monitoring points; A slope displacement monitoring unit for monitoring the slope displacement according to the changes of the three-dimensional coordinates of the monitoring points at the previous and next times.

9. An electronic device, characterized by comprising: Comprise a memory and a processor, wherein, The memory is used to store programs; The processor is coupled with the memory and is used to execute the programs stored in the memory to implement the binocular vision highway slope displacement monitoring method of any one of claims 1-6, and / or the steps of the binocular vision highway slope displacement early warning method of claim 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores one or more programs that can be executed by one or more processors to implement the binocular vision highway slope displacement monitoring method of any one of claims 1-6, and / or the steps of the binocular vision highway slope displacement early warning method of claim 7.

Images