Power equipment safety distance monitoring method based on SGBM and binocular distance measurement

Abstract

The invention provides a power equipment safety distance monitoring method based on SGBM and binocular distance measurement. The method comprises the following steps: calibrating a binocular camera to obtain an internal and external parameter matrix; collecting left and right images in a power construction scene by using a binocular camera and performing stereo correction on the left and right images; during stereo correction, the correction matrix is used for carrying out reprojection on the pixels, so that the problem of image distortion caused by radial distortion and tangential distortion is solved, and the precision of subsequent size measurement is improved; generating a disparity map by using OpenCV and SGBM, and then calculating the distance from each object in the power construction scene to the binocular camera; the distance between the power equipment and the worker and the binocular camera in the power construction scene is utilized, and the three-dimensional coordinate of the object in the world coordinate system is calculated through the depth calculation formula, so that the distance between the power equipment and the worker is accurately measured in real time, and potential safety hazards in construction site operation are effectively prevented.

Description

Technical Field

[0001] The invention belongs to the technical field of electric power safety, and in particular relates to a method for monitoring the safe distance of electric power equipment based on SGBM and binocular ranging. Background Art

[0002] In the power construction industry, ensuring the personal safety of workers is both the most important and the most basic requirement for workers. Power construction activities are one of the typical high-risk production activities. Accidents often occur due to workers' negligence, improper assembly of tools, and irregular behavior. Especially in the maintenance and overhaul of power equipment, ensuring a safe distance between workers and equipment is crucial to preventing accidents. There are a large number of power construction sites and the occurrence of construction accidents is inevitable, but the distance between workers and equipment can be supervised and checked on the construction site to effectively reduce the incidence of accidents. Therefore, accurate distance measurement methods are very necessary.

[0003] Traditional distance measurement methods for power equipment include direct contact measurement and laser ranging, but these methods have limitations, such as the need for manual operation and insufficient ability to monitor in real time in complex environments. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides a method for monitoring the safe distance of power equipment based on SGBM and binocular ranging. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0005] A method for monitoring the safe distance of electric power equipment based on SGBM and binocular ranging includes:

[0006] S100, calibrating a binocular camera to obtain an internal and external parameter matrix of the binocular camera;

[0007] S200, using a binocular camera to collect left and right images in a power construction scene, and using the internal and external parameter matrix to perform stereo correction on the left and right images so as to adjust the left and right images to a coplanar state;

[0008] S300, performing stereo matching on the corrected left and right images using the SGBM algorithm to generate a disparity map;

[0009] S400, calculating the distance from each object in the electric power construction scene to the binocular camera according to the disparity map;

[0010] S500, using the distance from each object in the electric power construction scene to the binocular camera, calculate the three-dimensional coordinates of the object in the world coordinate system;

[0011] S600, calculating the actual distance between the worker and the power equipment using the three-dimensional coordinates of the worker and the power equipment in the world coordinate system;

[0012] S700 determines whether to issue a safety alarm according to the actual distance between the worker and the power equipment.

[0013] Beneficial effects:

[0014] The present invention provides a method for monitoring the safe distance of electric power equipment based on SGBM and binocular ranging, comprising: calibrating a binocular camera to obtain an internal and external parameter matrix; using the binocular camera to collect left and right images in an electric power construction scene and performing stereo correction on them; using the correction matrix to reproject pixels during stereo correction, thereby solving the image distortion problem caused by radial distortion and tangential distortion and improving the accuracy of subsequent size measurement; using OpenCV and SGBM to generate a disparity map, and then calculating the distance from each object in the electric power construction scene to the binocular camera; using the distance from the electric power equipment and the worker to the binocular camera in the electric power construction scene, calculating the three-dimensional coordinates of the object in the world coordinate system through a depth calculation formula, thereby measuring the distance between the electric power equipment and the worker in real time and accurately, thereby effectively preventing safety hazards in construction site operations.

[0015] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The present invention provides a flow chart of a method for monitoring safe distance of electric power equipment based on SGBM and binocular ranging.

[0017] Figure 2 is a flow chart of the SGBM algorithm used in stereo matching of the present invention;

[0018] Figure 3 2 are the depth map (left) and grayscale map (right) obtained by the present invention. DETAILED DESCRIPTION

[0019] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0020] like Figure 1 As shown, the present invention provides a method for monitoring the safe distance of electric power equipment based on SGBM and binocular ranging, including:

[0021] S100, calibrating a binocular camera to obtain an internal and external parameter matrix of the binocular camera;

[0022] With the continuous development of computer vision technology in recent years, binocular ranging technology has shown great potential in this field. Binocular ranging technology can obtain left and right images through binocular cameras without direct contact with the object to be measured, and use binocular stereo matching algorithm to accurately calculate the distance between the object and the camera. The core principle of this technology is to use the position difference of binocular cameras to shoot the same target object through two lenses respectively, and calculate the depth information of the object by analyzing the position difference of the same point in the two views (i.e. parallax), combined with mathematical models, and provide non-contact, high-precision, real-time three-dimensional perception capabilities for safety detection. Using binocular cameras to complete binocular ranging tasks has many advantages, such as non-contact, high-precision, and good real-time performance.

[0023] Before performing binocular vision ranging, the present invention needs to calibrate the binocular camera to obtain the internal and external parameters of the camera, so as to solve the problem of image distortion. The binocular calibration is mainly to obtain the internal and external parameter matrix and distortion coefficient of the binocular camera.

[0024] As an optional implementation manner of the present invention, S100 includes:

[0025] S110, calibrating the internal parameters of the binocular camera to obtain an internal parameter matrix;

[0026] The purpose of intrinsic calibration is to determine the intrinsic properties of the camera, such as focal length and principal point coordinates. The intrinsic parameter matrix is ​​expressed as:

[0027]

[0028] In the formula, f x , f y Respectively represent the focal length of the camera in the x and y directions, in pixels; c x , c y represents the coordinates of the principal point in the image coordinate system, where the principal point is the intersection of the optical axis and the image plane; the intrinsic parameter matrix includes the intrinsic parameter matrix K1 of the left camera and the intrinsic parameter matrix K2 of the right camera;

[0029] S120, calibrating the binocular camera using a radial distortion correction formula and a tangential distortion correction formula to obtain a distortion coefficient;

[0030] Camera distortion is usually divided into radial distortion and tangential distortion, where the radial distortion correction formula is:

[0031]

[0032] In the formula, x corrected ,y corrected represents the coordinates after radial distortion correction; r represents the radial distance; k1, k2, k3 represent the radial distortion coefficients respectively;

[0033] The tangential distortion correction formula is:

[0034]

[0035] In the formula, x corrected ,y corrected represents the coordinates after tangential distortion correction; p1, p2 represent the tangential distortion coefficients;

[0036] S130, calibrating the extrinsic parameters of the binocular camera to obtain an extrinsic parameter matrix; the extrinsic parameter matrix includes a rotation matrix R and a translation vector T.

[0037] The rotation matrix R and the translation vector T are used to describe the relative position relationship between the binocular camera coordinate systems, which can be expressed as:

[0038] [X c ,Y c ,X c ] T =R[X w ,Y w ,Z w ] T +T# Formula (4)

[0039] In the formula, [X c ,Y c ,Z c ] T Represents the point coordinates in the camera coordinate system; [X w ,Y w ,Z w ] T represents the point coordinates in the world coordinate system; the rotation matrix R includes the intrinsic parameter matrix R1 of the left camera and the intrinsic parameter matrix R2 of the right camera, and the translation vector T includes the intrinsic parameter matrix T1 of the left camera and the intrinsic parameter matrix T2 of the right camera.

[0040] S200, using a binocular camera to collect left and right images in a power construction scene, and using the internal and external parameter matrix to perform stereo correction on the left and right images so as to adjust the left and right images to a coplanar state;

[0041] This step uses the internal and external parameter matrix of the binocular camera to perform stereo correction on the images obtained by the left and right cameras. The purpose of stereo correction is to adjust the images taken by the left and right cameras to a coplanar state and make them have the same row alignment, thereby simplifying the stereo matching process.

[0042] As an optional implementation manner of the present invention, S200 includes:

[0043] S210, calculating the optical center position of the binocular camera using the internal and external parameter matrix; wherein the calculated optical center position of the binocular camera is expressed as:

[0044]

[0045] S220, using the optical center position to calculate new rotation matrix parameters of the binocular camera; wherein the new rotation moment parameters are expressed as:

[0046]

[0047] S230, using the new rotation matrix parameters to update the rotation matrix of the binocular camera to obtain an updated rotation matrix; wherein the updated rotation matrix is ​​expressed as:

[0048]

[0049] S240, updating the intrinsic parameter matrix of the binocular camera to obtain an updated intrinsic parameter matrix; wherein the updated intrinsic parameter matrix is ​​expressed as:

[0050]

[0051] S250, calculating the stereo correction matrix of the binocular camera by using the updated rotation matrix and the updated intrinsic parameter matrix; wherein the stereo correction matrix is ​​expressed as:

[0052]

[0053] S260: Perform stereoscopic correction on the left and right images using the stereoscopic correction matrix, so that the epipolar lines of the left and right images are completely parallel and the corresponding pixels are on the same row.

[0054] S300, performing stereo matching on the corrected left and right images using the SGBM algorithm to generate a disparity map;

[0055] This step uses the SGBM (Semi-Global Block Matching) algorithm to perform stereo matching on the rectified left and right images to generate a disparity map. The disparity d is the horizontal pixel displacement of the same object point in the left and right images. Figure 2 , Figure 2 This is the matching flow chart of the SGBM algorithm. Figure 2 The matching process generates a disparity map of the left and right images.

[0056] As an optional implementation manner of the present invention, S300 includes:

[0057] S310, using the SGBM algorithm to construct an energy function for stereo matching of the corrected left and right images, wherein the energy function is expressed as:

[0058]

[0059] Where E(D) is the energy function, which represents the total cost of a certain disparity configuration D; D p , D q represents the disparity of pixel points p and q; C(p,D p ) represents the matching cost; P1, P2 represent the smoothing penalty coefficients; N(p) is the neighborhood function, which represents the neighborhood of pixel p;

[0060] S320 , minimizing the energy function to determine the best disparity configuration, thereby generating a disparity map of the left and right images.

[0061] S400, calculating the distance from each object in the electric power construction scene to the binocular camera according to the disparity map;

[0062] In this step, the disparity map is substituted into the depth calculation formula to obtain the distance from the power equipment to the binocular camera. The depth calculation formula is expressed as:

[0063]

[0064] Where Z represents the distance from the object in the left and right images to the binocular camera; f represents the focal length of the camera, in pixels; b represents the baseline of the binocular camera, which is the distance between the optical centers of the left and right cameras; and d represents the parallax.

[0065] S500, using the distance from each object in the electric power construction scene to the binocular camera, calculate the three-dimensional coordinates of the object in the world coordinate system;

[0066] In this step, the three-dimensional coordinates of each object in the power construction scene in the world coordinate system are expressed as:

[0067]

[0068] In the formula, (X, Y, Z) represents the three-dimensional coordinates of the power object in the world coordinate system; Z represents the distance from the power equipment to the binocular camera; c x , c y represents the coordinates of the principal point of the image; f x , f y Represent the focal length of the stereo camera on the x-axis and y-axis respectively.

[0069] S600, calculating the actual distance between the worker and the power equipment using the three-dimensional coordinates of the worker and the power equipment in the world coordinate system;

[0070] This step calculates the actual distance between the worker and the power equipment based on the three-dimensional coordinates of the worker and the power equipment in the world coordinate system and using the Euclidean distance calculation formula; the Euclidean distance calculation formula is expressed as:

[0071]

[0072] Where (X1, Y1, Z1) and (X2, Y2, Z2) represent the three-dimensional coordinates of the worker and the power equipment respectively; D represents the actual distance between the worker and the power equipment.

[0073] S700 determines whether to issue a safety alarm according to the actual distance between the worker and the power equipment.

[0074] This step determines whether the actual distance between the worker and the power equipment is less than the safety distance threshold. If so, a safety alarm is issued, thereby eliminating potential safety hazards in advance. Safety distance threshold D safe It is a threshold set according to national or industry safety standards.

[0075] Through steps S100 to S700, the present invention can monitor the distance between workers and power equipment in real time, help construction site managers to promptly discover and deal with possible safety issues, and improve the safety of construction site operations.

[0076] In the actual environment, the safety distance between the worker and the power equipment is set to 0.8m. The depth obtained based on the SGBM algorithm and the binocular vision ranging model, the distance between the worker and the power equipment, and the depth generated by the experiment are shown in Tables 1 and 2. The generated grayscale image is referenced Figure 3 .

[0077] Table 1

[0078]

[0079] Table 2

[0080]

[0081]

[0082] From Table 1, Table 2 and Figure 3 It can be seen that the present invention realizes real-time monitoring of the safe distance between power equipment and workers through binocular stereo vision ranging technology. The system not only has the characteristics of high precision and high efficiency, but also shows strong adaptability in the complex environment of the power construction site.

[0083] In the present invention, the safe distance monitoring method for power equipment based on SGBM and binocular ranging uses the Fusiello algorithm in step 2 to perform stereo correction on the distortion-corrected on-site image to adjust the polar lines of the left and right images to the same horizontal line, and uses the SGBM algorithm to generate a disparity map, and obtains the optimal disparity configuration by minimizing the energy function, and further determines the actual distance between the worker and the power equipment. The results show that compared with the prior art, the binocular stereo vision ranging method based on the SGBM algorithm shows significant advantages in measuring the distance between power equipment and workers. This method can accurately calculate the three-dimensional distance between the equipment and the worker under various working conditions, and has high measurement accuracy. Within the calibration distance, the average ranging error is 2.88%, which is much better than the traditional ranging method. At the same time, the average calculation time of this method is 0.3 seconds, which can respond in real time and effectively ensure the safety of the power construction site.

[0084] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality of components or steps.

[0085] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for monitoring the safe distance of electric power equipment based on SGBM and binocular ranging, characterized in that: include: S100, calibrating a binocular camera to obtain an internal and external parameter matrix of the binocular camera; S200, using a binocular camera to collect left and right images in a power construction scene, and using the internal and external parameter matrix to perform stereo correction on the left and right images so as to adjust the left and right images to a coplanar state; S300, performing stereo matching on the corrected left and right images using the SGBM algorithm to generate a disparity map; S400, calculating the distance from each object in the electric power construction scene to the binocular camera according to the disparity map; S500, using the distance from each object in the electric power construction scene to the binocular camera, calculate the three-dimensional coordinates of the object in the world coordinate system; S600, calculating the actual distance between the worker and the power equipment using the three-dimensional coordinates of the worker and the power equipment in the world coordinate system; S700 determines whether to issue a safety alarm according to the actual distance between the worker and the power equipment.

2. The method for monitoring safe distance of electric power equipment based on SGBM and binocular ranging according to claim 1 is characterized in that: S100 includes: S110, calibrating the internal parameters of the binocular camera to obtain an internal parameter matrix; S120, calibrating the binocular camera using a radial distortion correction formula and a tangential distortion correction formula to obtain a distortion coefficient; S130, calibrating the extrinsic parameters of the binocular camera to obtain an extrinsic parameter matrix; the extrinsic parameter matrix includes a rotation matrix R and a translation vector T.

3. The method for monitoring safe distance of electric power equipment based on SGBM and binocular ranging according to claim 1 is characterized in that: The internal parameter matrix in S110 is expressed as: In the formula, f x , f y Respectively represent the focal length of the camera in the x and y directions, in pixels; c x , c y Represents the coordinates of the principal point in the image coordinate system. The principal point is the intersection of the optical axis and the image plane. The intrinsic parameter matrix includes an intrinsic parameter matrix K1 of a left camera and an intrinsic parameter matrix K2 of a right camera; The radial distortion correction formula in S120 is: In the formula, x corrected ,y corrected represents the coordinates after radial distortion correction; r represents the radial distance; k1, k2, k3 represent the radial distortion coefficients respectively; The tangential distortion correction formula in S120 is: In the formula, x corrected ,y corrected represents the coordinates after tangential distortion correction; p1, p2 represent the tangential distortion coefficients; The rotation matrix R and translation vector T in S130 are used to describe the relative position relationship between the binocular camera coordinate systems, and are expressed as: [X c , y c , Z c T = R[X w , Y w , Z w T + T # Formula (4)​​ In the formula, [X c , Y c , Z c ] T Represents the point coordinates in the camera coordinate system; [X w , Y w , Z w ] T represents the point coordinates in the world coordinate system; the rotation matrix R includes the intrinsic parameter matrix R1 of the left camera and the intrinsic parameter matrix R2 of the right camera, and the translation vector T includes the intrinsic parameter matrix T1 of the left camera and the intrinsic parameter matrix T2 of the right camera.

4. The method for monitoring safe distance of electric power equipment based on SGBM and binocular ranging according to claim 3 is characterized in that: S200 includes: S210, calculating the optical center position of the binocular camera using the internal and external parameter matrix; S220, calculating new rotation matrix parameters of the binocular camera using the optical center position; S230, updating the rotation matrix of the binocular camera using the new rotation matrix parameters to obtain an updated rotation matrix; S240, updating the intrinsic parameter matrix of the binocular camera to obtain an updated intrinsic parameter matrix; S250, calculating a stereo correction matrix of the binocular camera by using an updated rotation matrix and an updated intrinsic parameter matrix; S260: Perform stereoscopic correction on the left and right images using the stereoscopic correction matrix, so that the epipolar lines of the left and right images are completely parallel and the corresponding pixels are on the same row.

5. The method for monitoring safe distance of electric power equipment based on SGBM and binocular ranging according to claim 4 is characterized in that: The optical center position of the binocular camera calculated in S210 is expressed as: The new rotation torque parameter in S220 is expressed as: The updated rotation matrix in S230 is expressed as: The updated internal parameter matrix in S240 is expressed as: The stereo correction matrix in S250 is expressed as:

6. The method for monitoring safe distance of electric power equipment based on SGBM and binocular ranging according to claim 1 is characterized in that: S300 includes: S310, using the SGBM algorithm to construct an energy function for stereo matching of the corrected left and right images, wherein the energy function is expressed as: Where E(D) is the energy function, which represents the total cost of a certain disparity configuration D; D p , D q represents the disparity of pixel points p and q; C(p, D p ) represents the matching cost of pixel points p and q; P1, P2 represent the smoothing penalty coefficients; N(p) is the neighborhood function, which represents the neighborhood of pixel point p; S320 , minimizing the energy function to determine the best disparity configuration, thereby generating a disparity map of the left and right images.

7. The method for monitoring safe distance of electric power equipment based on SGBM and binocular ranging according to claim 1 is characterized in that: S400 includes: Substituting the disparity map into the depth calculation formula to obtain the distance from the power equipment to the binocular camera, the depth calculation formula is expressed as: Where Z represents the distance from the object in the left and right images to the binocular camera; f represents the focal length of the camera, in pixels; b represents the baseline of the binocular camera, which is the distance between the optical centers of the left and right cameras; and d represents the parallax.

8. The method for monitoring safe distance of electric power equipment based on SGBM and binocular ranging according to claim 1, characterized in that: The three-dimensional coordinates of the object in the world coordinate system in S500 are expressed as: In the formula, (X, Y, Z) represents the three-dimensional coordinates of the power object in the world coordinate system; Z represents the distance from the power equipment to the binocular camera; c z , c y represents the coordinates of the principal point of the image; f x , f y Represent the focal length of the stereo camera on the x-axis and y-axis respectively.

9. The method for monitoring safe distance of electric power equipment based on SGBM and binocular ranging according to claim 1, characterized in that: S600 includes: According to the three-dimensional coordinates of the worker and the power equipment in the world coordinate system, the actual distance between the worker and the power equipment is calculated using the Euclidean distance calculation formula; the Euclidean distance calculation formula is expressed as: Where (X1, Y1, Z1) and (X2, Y2, Z2) represent the three-dimensional coordinates of the worker and the power equipment respectively; D represents the actual distance between the worker and the power equipment.

10. The method for monitoring safe distance of electric power equipment based on SGBM and binocular ranging according to claim 1, characterized in that: S600 includes: Determine whether the actual distance between the worker and the electrical equipment is less than the safety distance threshold, and if so, issue a safety alarm.

Images