A method for secondary precise alignment of images in a substation inspection robot

Abstract

This invention discloses a method for secondary precise alignment of images by a substation inspection robot, relating to the field of substation operation and maintenance technology. The method includes establishing an error model for the inspection robot to capture and detect images; decoupling a three-level PI alignment strategy based on the error model; and actively repositioning and calibrating the camera. By repositioning the substation inspection robot's pose and the camera, and then performing secondary longitudinal and lateral calibration of the camera pose to compensate for navigation errors and mechanical wear errors, the robot's pose and the camera's pose can be closer to the predetermined pose when the inspection robot stops at inspection points along the inspection route to capture images. This exhibits high accuracy and strong robustness, ensuring that the captured image is consistent with the predetermined image.

Description

Technical Field

[0001] This invention relates to the field of substation operation and maintenance technology, and in particular to a method for secondary precise alignment of images by a substation inspection robot. Background Technology

[0002] When the substation inspection robot performs routine inspections on electrical equipment, it moves along a predetermined inspection route, stops at predetermined inspection points, and then takes pictures of the electrical equipment. In order to facilitate the acquisition of the condition of the electrical equipment, a calibration image is usually set in advance at the inspection point, and the inspection robot needs to capture the inspection image that is consistent with the calibration image.

[0003] However, there are navigation and mechanical errors between the docking position and the calibration position of the inspection robot. Due to mechanical wear, the direction and distance of the inspection robot may deviate from the predetermined direction. This may also cause the gimbal to deviate in adjusting the direction and position of the camera. As a result, there is a discrepancy between the detection image and the calibration image captured by the camera, which increases the amount of data to be analyzed in the detection image and may even lead to misjudgment of the condition of electrical equipment. Summary of the Invention

[0004] To overcome the deficiencies of existing technologies, the purpose of this invention is to provide a method for secondary precise alignment of images of a substation inspection robot. By repositioning the pose of the substation inspection robot and the camera, and then performing secondary longitudinal and lateral calibration of the camera pose to compensate for navigation errors and mechanical wear errors, the robot pose and camera pose can be closer to the predetermined pose when the inspection robot stops at the inspection point to capture images along the inspection route. This demonstrates high precision and strong robustness, ensuring that the captured image is consistent with the predetermined image.

[0005] This invention provides the following technical solution: a method for secondary precise alignment of images of a substation inspection robot, comprising establishing an error model of the images captured and detected by the inspection robot; decoupling a three-level PI alignment strategy based on the error model; and actively repositioning and calibrating the camera.

[0006] As a preferred embodiment of the substation inspection robot image secondary precise alignment method described in this invention, the error model is specifically described as follows: the current position of the camera is regarded as a three-dimensional point X in the three-dimensional model of the camera. c X c The three-dimensional coordinates are X c =(X c Y c Z c ), X c The vector of three-dimensional coordinates is X. c =(X c Y c Zc ) T In the 3D model where the camera is located, that is, in the camera coordinate system Oc-xyz, the expression for the coordinates of the pixel m of the principal point of the plane image captured by the camera is:

[0007] m = (u, v) T

[0008] Where m represents the pixel of the principal point of the planar image captured by the predetermined camera, and (u, v) represent the coordinates of pixel m of the principal point of the planar image captured by the predetermined camera. Calibrating the planar image captured by the camera is equivalent to calibrating the coordinates of the pixel of the principal point of the planar image captured by the predetermined camera, and its calculation expression is:

[0009]

[0010] Where fu and fv represent the scaling factors in the horizontal and vertical directions of the planar image captured by the camera, respectively. When fu = fv, the planar image captured by the camera is a square. (u0, v0) represents the pixel coordinates of the principal point of the current planar image captured by the camera.

[0011] As a preferred embodiment of the substation inspection robot image secondary precise alignment method described in this invention: the predetermined position of the camera is regarded as a three-dimensional point in the three-dimensional model of the substation. The error vector of the actual position is Current chassis position chassis displacement error vector t h and the current camera pose The camera pose in the calibration state can be obtained. and camera pose in detection state The calculation expression is:

[0012]

[0013] Then, camera pose Error T c The calculation expression is:

[0014]

[0015] in, This indicates the camera pose during calibration. This indicates the camera pose during the detection state. This indicates the camera pose in the detection state before adjustment, T c Indicates camera pose The error, R cx R cyR is an orthogonal matrix representing the pitch and rotation angle errors of the camera corresponding to a predetermined shooting scene. x R y These are orthogonal matrices, corresponding to the camera pose P respectively. c The pitch angle θx and rotation angle θy, t hp and t pc This indicates the position parameters of the inspection robot at the inspection point. This indicates the current chassis position. The vector representing the current camera pose, i.e., the current camera pose P. c The vector of pitch angle θx and rotation angle θy, t h t represents the displacement error vector of the chassis. c =(t cx , t cy , t cz ) T It is the camera's displacement error vector, (t cx , t cy , t cz The coordinates of the camera in the 3D model of the substation are the X, Y, and Z axes, which represent the coordinates of the camera's location in the camera's coordinate system.

[0016] As a preferred embodiment of the substation inspection robot image secondary precise alignment method described in this invention: the pixel count of the principal point of the actual planar image captured by the camera is m* = (u*, v*). T , corresponding to X in the camera's 3D model c Its calculation expression is:

[0017]

[0018] In the 3D model map of the substation, the inspection route of the inspection robot can be considered as a plane, and the road surface near the inspection point can be considered as an absolute plane. Therefore, the y-coordinate of the chassis in the chassis coordinate system of the inspection robot is always 0, and t hy =0, we can get:

[0019] t cz =t cy cotθ x

[0020]

[0021] Among them, t h Let Pc represent the chassis displacement error vector, and Pc represent the camera pose in the calibration state. P represents the camera pose during shooting. p Indicates gimbal pose, P h Indicates the chassis pose under calibration conditions, T hpLet T represent the pose transformation matrix of the inspection robot in the calibration state, and T hp = (1, t) hp ) and T pc = (1, t) pc ), where t hp and t pc R represents the position parameters of the inspection robot at the inspection point. x R y The orthogonal matrices in the standard state correspond to the camera pose P. c The pitch angle θx and rotation angle θy, This represents the camera's position before adjustment during shooting, and Tc represents the error of the camera's pose Pc. These are orthogonal matrices in the shooting state, corresponding to the pose in the shooting state, respectively. The pitch angle θx and rotation angle θy, R cx R cy The orthogonal matrix t represents the pitch and rotation angle errors of the camera corresponding to the predetermined shooting image. hp and t pc This indicates the position parameters of the inspection robot at the inspection point. This indicates the chassis position and orientation during filming. This represents the pose transformation matrix of the inspection robot during the shooting process. The vector representing the current camera pose, i.e., the current camera pose P. c The vector of pitch angle θx and rotation angle θy, t c =(t cx , t cy , t cz ) T It is the camera's displacement error vector, (t cx , t cy , t cz The coordinates of the camera in the 3D model of the substation are the X, Y, and Z axes, which represent the coordinates of the camera's location in the camera's coordinate system.

[0022] As a preferred embodiment of the substation inspection robot image secondary precise alignment method described in this invention, the error model decoupling three-level PI control includes:

[0023] Phase 1: PTZ pose correction for camera position:

[0024] First, the calibration image D is located using the PI controller. c and real-time inspection of image D r Matching points between;

[0025] Then, using the classic 5-point algorithm and pose solving algorithm, we can obtain from I c and Ir Calculate R cx R cy ;

[0026] For R cx R cy By decomposing the components, the errors in pitch and rotation angles can be obtained, and their calculation expressions are as follows;

[0027]

[0028] Where R cx R cy Represents a matrix, i th j th The elements in the i-th row and j-th column will be Δθ x and Δθ y By multiplying by the anti-Jacob matrix J θ =diag(ω x ω y Convert ) to execution time T cx and Y cy As shown below;

[0029] [τ cx ,τ cy ] T =J θ [△θ x ,△θ y ] T

[0030] Where ωx and ωy represent the pitch and rotation angular velocities of the gimbal, Δθx and Δθy are the pitch and rotation angular velocities of the camera pose PTZ, and the PI controller is used to generate the execution times Tcx and Ycy, until ||(θ cx ,θ cy )||2≤ε θ This indicates that the camera pose PTZ calibration is complete.

[0031] As a preferred embodiment of the substation inspection robot image secondary precise alignment method described in this invention:

[0032] Phase 2: Correction of camera pose PTZ longitudinal translation;

[0033] Because the pose error of the camera pose PTZ is rectified in stage 1, i.e. R cx R cy =1, the translation error t corresponding to Δv of the camera pose PTZ. cy and t cz , can be taken as a known constant, and the corresponding translation t of the chassis h The calculation expression is:

[0034]

[0035] Among them, arctan(t) hx / t hz )=θ y or θ y +π, pitch angle θx∈(0, π / 2), rotation angle θy∈(0, 2π), therefore, sinθx>0, the direction of chassis translation is determined by θy and Δv. Based on the values ​​of arctan(thx / thz) corresponding to different conditions, we can obtain:

[0036]

[0037] Specifically, when Δv>0, the camera pose PTZ should shift forward, and when Δv<0, it should shift backward.

[0038] This means that at this stage, the camera pose PTZ will be longitudinally translated by the chassis, and the PI controller is also used to generate the translation t. h The camera pose PTZ calibration is completed in the X-axis direction until |Δv|≤εv.

[0039] As a preferred embodiment of the substation inspection robot image secondary precise alignment method described in this invention:

[0040] Phase 3: Camera pose PTZ lateral translation correction, the same as Phase 1, corresponding to the translation th of Δu, its calculation expression is:

[0041]

[0042] Among them, t h The direction is θ y -π / 2 or θ y +π / 2, therefore;

[0043]

[0044] Specifically, when Δu>0, the camera pose PTZ should be shifted to the right, and when Δu<0, the camera pose PTZ should be shifted to the left.

[0045] At this stage, the camera pose PTZ will be translated laterally, and then the PI controller will be applied again to generate translation h until |Δu|≤εu, indicating that the camera pose PTZ has been calibrated in the Y-axis direction.

[0046] The beneficial effects of this invention are as follows: by repositioning the pose of the substation inspection robot and the camera, and then performing secondary longitudinal and lateral calibration of the camera pose to compensate for navigation errors and mechanical wear errors, the pose of the inspection robot and the camera can be closer to the predetermined pose when the inspection robot stops at the inspection point to capture images along the inspection route. This results in high precision and strong robustness, ensuring that the captured image is consistent with the predetermined image. Attached Figure Description

[0047] Figure 1 This is a schematic diagram showing the direction of th corresponding to the longitudinal translation Δy and Δv of the camera pose PTZ of a substation inspection robot image secondary precise alignment method provided in an embodiment of the present invention.

[0048] Figure 2 This is a schematic diagram showing the direction of th corresponding to the Δy and Δv of the lateral translation of the camera pose PTZ in a substation inspection robot image secondary precise alignment method provided in an embodiment of the present invention.

[0049] Figure 3 The image shown is a simulation comparison result of a method for secondary precise alignment of images of a substation inspection robot provided in an embodiment of the present invention. Detailed Implementation

[0050] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0051] Example 1

[0052] Reference Figure 1 and 2 As an embodiment of the present invention, a method for secondary precise alignment of images of a substation inspection robot is provided, comprising:

[0053] S1: Establish an error model for the inspection robot to capture and detect images;

[0054] The inspection robot consists of a camera pose measurement and control (PTZ) unit, a router, and a chassis. The PTZ unit is equipped with an infrared camera and a high-definition camera with up to 32x zoom. The high-definition camera has an image resolution of 2688×1520, a principal point resolution of [1356.09], a focal length of 774.79, and a focal length of 2596.99, respectively. The router is used to upload real-time detection images to the upper-level computer, receive and distribute control commands to the PTZ unit, and distribute control commands to the camera pose measurement and control and control of the chassis.

[0055] The error model is described in detail as follows: The current position of the camera is regarded as a three-dimensional point X in the camera's three-dimensional model. c X cThe three-dimensional coordinates are X c =(X c Y c Z c ), X c The vector of three-dimensional coordinates is X. c =(X c Y c Z c ) T In the 3D model where the camera is located, that is, in the camera coordinate system Oc-xyz, the expression for the coordinates of the pixel m of the principal point of the plane image captured by the camera is:

[0056] m = (u, v) T

[0057] Where m represents the pixel of the principal point of the planar image captured by the predetermined camera, and (u, v) represent the coordinates of pixel m of the principal point of the planar image captured by the predetermined camera. Calibrating the planar image captured by the camera is equivalent to calibrating the coordinates of the pixel of the principal point of the planar image captured by the predetermined camera, and its calculation expression is:

[0058]

[0059] Where fu and fv represent the scaling factors in the horizontal and vertical directions of the planar image captured by the camera, respectively. When fu = fv, the planar image captured by the camera is a square. (u0, v0) represents the pixel coordinates of the principal point of the current planar image captured by the camera.

[0060] Let p represent the pose of the inspection robot. Then, a binary tuple (R, t) can be used to represent the pose p of the inspection robot. Let Tij = (Rij, tij) represent the transformation from Pi to Pj, that is, Pj = TijP. i That is, the transformation of the pose p of the inspection robot from point i to point j in a planar coordinate system.

[0061] The pose of the inspection robot includes the pose of the chassis, the pose of the gimbal, and the pose of the camera.

[0062] The predetermined position of the camera is regarded as a three-dimensional point in the three-dimensional model of the substation. X c The three-dimensional coordinates are X c =(X c Y c Z c The error vector of the actual position of the three-dimensional point is... The pixel value of the principal point in the actual planar image captured by the camera is m* = (u*, v*). T This corresponds to the current position in the camera's 3D model. Its calculation expression is:

[0063]

[0064] In the 3D model map of the substation, the inspection route of the inspection robot can be considered as a plane, and the road surface near the inspection point can be considered as an absolute plane. Therefore, the y-coordinate of the chassis in the chassis coordinate system of the inspection robot is always 0, and t hy =0, we can get:

[0065] t cz =t cy cotθ x

[0066]

[0067] The coordinate vectors Δu and Δv of the error between the center point of the calibration image and the center point of the inspection image are:

[0068]

[0069] In the 3D model map of the substation, the distance Z between the electrical equipment that the inspection robot needs to photograph and the camera is... c Much larger than the chassis translation error t of the inspection robot c That is, simplifying the above expression, we get:

[0070]

[0071] The error Tc of the camera pose Pc of the inspection robot is calculated as follows:

[0072]

[0073] Among them, Z c Z represents the distance between the electrical equipment to be photographed by the inspection robot and the camera; Tc represents the translation error of the inspection robot's chassis; fu and fv represent the scale coefficients in the horizontal and vertical directions of the planar image captured by the camera, respectively; when fu = fv, the planar image captured by the camera is a square. c Δu and Δv are unknown, only fu, fv, and θx are known quantities.

[0074] To obtain Δu and Δv, specifically, the three-dimensional point X in the calibration state. c Choose the optical axis of the camera, i.e., X. c = (0, 0, Zc)T, m = (u0, v0)T, construct a virtual square with side length 2w centered at Xc. Virtual Square A square plane with a side length of 2w is located at a distance Zc = fu from the camera plane. The three-dimensional coordinates of the center point in the camera coordinate system are Xc = [0, 0, Z_c], and the two-dimensional coordinates in the image are m = [u0, v0]T. The three-dimensional coordinates of the four vertices of the square are:

[0075]

[0076] The calibration image corresponds to The coordinates of the pixel are:

[0077]

[0078] The pixel homology matrix H between the calibration image and the inspection image is obtained through image registration. The pixels corresponding to the same source in the positive calibration image are then... Multiplying the coordinates of a pixel by the pixel homology matrix H yields a new square. And the corresponding four vertices, their relationship is as follows:

[0079]

[0080] Based on the planar properties of the same source transformation, calculate the new square. The intersection of the two diagonals gives us Δu and Δv.

[0081] The chassis pose indicates the predetermined stopping position of the inspection robot on the planned inspection route, that is, the stopping position of the inspection point. Generally, the inspection point is an inspection stopping area, slightly larger than the vertical projection area of ​​the inspection robot on the ground. It can be compared with the size relationship between a car and a parking space in life. As long as the inspection robot enters this area, it means that the inspection robot has entered the predetermined stopping position. The gimbal pose indicates the pose of the gimbal on the inspection robot. The gimbal is a support device for mounting and fixing the camera. There are two types: fixed gimbal and motorized gimbal. Fixed gimbal is suitable for situations where the monitoring range is not large. After the camera is mounted on the fixed gimbal, the horizontal and pitch angles of the camera can be adjusted. The camera pose includes the horizontal, vertical and focal length poses of the camera. The horizontal pose indicates the horizontal movement, that is, the rotation of the camera. The vertical pose indicates the vertical movement, that is, the height and pitch of the camera lens. The focal length pose indicates zoom, that is, adjusting the focal length of the camera. High resolution cameras are preferred.

[0082] When the inspection robot is taking pictures, there are two working states: calibration state and inspection state. In calibration state, the transformation between chassis pose and camera pose is represented as:

[0083] P p =T hp P h =(R x R y , thp )

[0084] Among them, P p Indicates gimbal pose, P h Indicates chassis position, T hp Let T represent the pose transformation matrix of the inspection robot, and T hp = (1, t) hp ) and T pc = (1, t) pc ), where t hp and t pc This represents the position parameters of the inspection robot at the inspection point, where P h R is considered a fixed, known parameter. x R y These are orthogonal matrices, corresponding to the camera pose P respectively. c The pitch angle θx and rotation angle θy, the orthogonal matrix is ​​calculated as follows:

[0085]

[0086] It should be noted that θx and θy are known parameters at each preset inspection point, while the camera's focal length, which is an ignored constant in this model, is a function of automatic zoom and focus in current cameras.

[0087] Under inspection, chassis position P h And represents the camera pose, P c Angle deviation from calibration, which means the current chassis pose The calculation expression is:

[0088] P h Indicates chassis position, P p Indicates gimbal pose, P h Indicates the chassis position.

[0089]

[0090] in, This indicates the current chassis pose, which is the current pose of the inspection vehicle. h Let t represent the displacement error vector of the chassis. h =(t hx , t hy , t hz ) T In other words, it is the error vector between the displacement of the inspection trolley from the previous inspection point to the next predetermined inspection point and the actual arrival at the next predetermined inspection point, (t hx , t hy , t hzThese are the coordinate parameters of the X, Y, and Z axes of the inspection trolley's location in the 3D model of the substation, using the inspection trolley as the coordinate system. This indicates the current camera pose.

[0091] Current chassis position chassis displacement error vector t h and the current camera pose The camera pose in the calibration state can be obtained. and camera pose in detection state The calculation expression is:

[0092]

[0093] Then, camera pose Error T c The calculation expression is:

[0094]

[0095] The predetermined position of the camera is regarded as a three-dimensional point in the three-dimensional model of the substation. The error vector of the actual position is Current chassis position chassis displacement error vector t h and the current camera pose The camera pose in the calibration state can be obtained. and camera pose in detection state The calculation expression is:

[0096]

[0097] Then, camera pose Error T c The calculation expression is:

[0098]

[0099] The pixel value of the principal point in the actual planar image captured by the camera is m* = (u*, v*). T , corresponding to X in the camera's 3D model c Its calculation expression is:

[0100]

[0101] In the 3D model map of the substation, the inspection route of the inspection robot can be considered as a plane, and the road surface near the inspection point can be considered as an absolute plane. Therefore, the y-coordinate of the chassis in the chassis coordinate system of the inspection robot is always 0, and t hy =0, we can get:

[0102] t cz =t cy cotθ x

[0103]

[0104] Among them, t h Let Pc represent the chassis displacement error vector, and Pc represent the camera pose in the calibration state. P represents the camera pose during shooting. p Indicates gimbal pose, P h Indicates the chassis pose under calibration conditions, T hp Let T represent the pose transformation matrix of the inspection robot in the calibration state, and T hp = (1, t) hp ) and T pc = (1, t) pc ), where t hp and t pc R represents the position parameters of the inspection robot at the inspection point. x R y The orthogonal matrices in the standard state correspond to the camera pose P. c The pitch angle θx and rotation angle θy, This represents the camera's position before adjustment during shooting, and Tc represents the error of the camera's pose Pc. These are orthogonal matrices in the shooting state, corresponding to the pose in the shooting state, respectively. The pitch angle θx and rotation angle θy, R cx R cy The orthogonal matrix t represents the pitch and rotation angle errors of the camera corresponding to the predetermined shooting image. hp and t pc This indicates the position parameters of the inspection robot at the inspection point. This indicates the chassis position and orientation during filming. This represents the pose transformation matrix of the inspection robot during the shooting process. The vector representing the current camera pose, i.e., the current camera pose P. c The vector of pitch angle θx and rotation angle θy, t c =(t cx , t cy , t cz ) T It is the camera's displacement error vector, (t cx , t cy , t cz The coordinates of the camera in the 3D model of the substation are the X, Y, and Z axes, which represent the coordinates of the camera's location in the camera's coordinate system.

[0105] S2: Based on the error model, the three-level PI alignment strategy is decoupled to actively reposition and calibrate the camera. This is to accurately control the inspection robot to correct the estimated camera pose error T. c A camera pose error control strategy needs to be designed. This strategy decouples the three-level PI alignment strategy, ensuring robustness against mechanical wear and compensating for errors caused by mechanical wear, so that the detected image captured by the camera matches the predetermined detected image.

[0106] In order to solve the problem, based on the spatial structure of the substation, the following reasonable assumptions are made.

[0107] Assumption 1: The road surface near the inspection point is an absolutely flat surface, i.e., t hy =0. (The road surface is not an absolutely flat plane; sometimes the initial translation error of the chassis may not be greater than the Z-axis in the real world.) c Much smaller.

[0108] Assumption 2: The distance between the electrical equipment and the high camera is much greater than the initial translation error of the chassis, i.e., Z c >>|t h ||2.

[0109] Error model decoupling three-level PI control includes:

[0110] Phase 1: PTZ pose correction for camera position:

[0111] First, the calibration image D is found by accelerating robust features. c and real-time inspection of image D r Matching points between;

[0112] Then, using the classic 5-point algorithm and pose solving algorithm, we can obtain from I c and I r Calculate R cx R cy ;

[0113] For R cx R cy By decomposing the components, the errors in pitch and rotation angles can be obtained, and their calculation expressions are as follows;

[0114]

[0115] Where R cx R cy Represents a matrix, i th j th The elements in the i-th row and j-th column will be Δθ x and Δθ y By multiplying by the anti-Jacob matrix J θ =diag(ω x ωy Convert ) to execution time T cx and Y cy As shown below;

[0116] [τ cx ,τ cy ] T =J θ [△θ x ,△θ y ] T

[0117] Where ωx and ωy represent the pitch and rotation angular velocities of the gimbal, and Δθx and Δθy are the pitch and rotation angular velocities of the camera pose PTZ, which will be longitudinally translated by the chassis during this phase. Similar to Phase 1, considering mechanical wear and image registration errors, a PI controller is used to generate execution times Tcx and Ycy until ||(θ... cx θ cy )||2≤ε θ This indicates that the camera pose PTZ calibration is complete.

[0118] Phase 2: Correction of camera pose PTZ longitudinal translation;

[0119] Please see Figure 1 Because the pose error of the camera pose PTZ is rectified in stage 1, i.e. R cx R cy =1, the translation error t corresponding to Δv of the camera pose PTZ. cy and t cz , can be taken as a known constant, and the corresponding translation t of the chassis h The calculation expression is:

[0120]

[0121] Among them, arctan(t) hx / t hz )=θ y or θ y +π, pitch angle θx∈(0, π / 2), rotation angle θy∈(0, 2π), therefore, sinθx>0, the direction of chassis translation is determined by θy and Δv. Based on the values ​​of arctan(thx / thz) corresponding to different conditions, we can obtain:

[0122]

[0123] Specifically, when Δv>0, the camera pose PTZ should shift forward, and when Δv<0, it should shift backward.

[0124] This means that at this stage, the camera pose PTZ will be longitudinally translated by the chassis, and the PI controller is also used to generate the translation t. h The camera pose PTZ calibration is completed in the X-axis direction until |Δv|≤εv.

[0125] Phase 3: Camera pose PTZ lateral translation correction

[0126] Please see Figure 2 Similar to stage 1, the calculation expression for the translation th corresponding to Δu is:

[0127]

[0128] Among them, t h The direction is θ y -π / 2 or θ y +π / 2, therefore;

[0129]

[0130] Specifically, when Δu>0, the camera pose PTZ should be shifted to the right, and when Δu<0, the camera pose PTZ should be shifted to the left.

[0131] At this stage, the camera pose PTZ will be translated laterally, and then the PI controller will be applied again to generate translation h until |Δu|≤εu, indicating that the camera pose PTZ has been calibrated in the Y-axis direction.

[0132] By repositioning the substation inspection robot and its camera, and then performing secondary longitudinal and lateral calibration of the camera pose PTZ, navigation errors and mechanical wear errors are compensated. This allows the inspection robot and its camera pose to be closer to the predetermined pose when the robot stops at the inspection point to capture images along the inspection route. This results in high precision and strong robustness, ensuring that the captured images are consistent with the predetermined images.

[0133] Example 2

[0134] Reference Figure 3 This is another embodiment of the present invention. Unlike the first embodiment, this embodiment provides an experimental verification of a method for secondary precise alignment of images of a substation inspection robot. To verify and explain the technical effects of this method, this embodiment uses a traditional technical solution to compare and test with the method of the present invention, and compares the experimental results using scientific demonstration methods to verify the real effect of this method.

[0135] To quantitatively evaluate the consistency between calibration and inspection images captured by different methods, we propose two metrics: image coverage and average dense optical flow (ADOF). Image coverage is defined as the ratio of the number of overlapping pixels between the inspection and calibration images to the image resolution. ADOF is defined as follows:

[0136] Where M×N=2688×1520 is the image resolution. Represents the dense optical flow calculated by the Farneback algorithm. These are the calibration image and the inspection image after isotope conversion. Obviously, the higher the image coverage and the smaller the ADOF, the more consistent the calibration and inspection images will be.

[0137] Figure 3 The simulation comparison results are shown in the figure. A podolist of (a) displacement error, (b) number of iterations, (c) runtime, (d) image coverage, (e) ADOF, (f) Au, and (g) Au values ​​for the ACR, CRP, and APR methods at 10 calibration points is presented. (a) shows that APR achieves the smallest displacement error, with a median of 8.13 mm, an order of magnitude higher than ACR and CRP. The figure also shows that our APR method has a runtime of less than 60 seconds, faster than ACR and CRP, and achieves better coverage of the target image.

[0138] Although APR appears to require more iterations, its lower runtime is due to the fact that the chassis only needs to rotate twice during the entire APR control process, moving forward or backward in each iteration's translation. In contrast, since the PTZ camera's translation depends on the chassis movement, the ACR and CRP strategies require the chassis to rotate once and then move forward or backward in each iteration's camera vector update, which is time-consuming and reduces translation robustness. Considering that the focus of conventional substation inspection tasks is on repositioning accuracy and image quality, rather than inspection time, APR's runtime performance is acceptable. Furthermore, as... Figure 3 As shown in (d) to (g), the coverage and pixel deviation between the calibration and inspection images captured by APR are better than those of the ACR and CRP methods, meaning that the inspection images obtained by APR are most consistent with the calibration images. Observing the data distribution in each graph, we can find that the APR method is the most accurate and robust integrated method. Simulation results verify that APR can achieve simpler control than other strategies to correct the displacement error of the substation inspection robot.

[0139] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for secondary precise alignment of images in a substation inspection robot, characterized in that, include: Establish an error model for the inspection robot to capture and detect images; Based on the error model, the three-level PI alignment strategy is decoupled, and the camera is actively repositioned and calibrated. The error model decoupling of the three-level PI control includes: Phase 1: PTZ pose correction for camera position: First, the calibration image D is located using the PI controller. c and real-time inspection of image D r Matching points between; Then, using the classic 5-point algorithm and pose solving algorithm, from I c and I r Calculate R cx R cy ; For R cx R cy By decomposing the components, the errors in pitch and rotation angles can be obtained, and their calculation expressions are as follows; Where R cx R cy The orthogonal matrix representing the pitch and rotation angle errors of the camera corresponding to the predetermined shooting scene is Δθ. x and Δθ y By multiplying by the anti-Jacob matrix J θ =diag(ω x ω y To convert to execution time τ cx and τ cy As shown below; [t cx ,t cy ] T =J θ [Dth x ,Dth y ] T Where ωx and ωy represent the pitch and rotation angular velocities of the gimbal, Δθx and Δθy are the pitch and rotation angular velocities of the camera pose (PTZ), and the PI controller is used to generate the execution time τ. cx and τ cy until ||(θ) cx θ cy )||2≤ε θ This indicates that the camera pose PTZ calibration is complete. Phase 2: Correction of camera pose PTZ longitudinal translation; Because the pose error of the camera pose PTZ is rectified in stage 1, i.e. R cx R cy =1, the translation error t corresponding to Δv of the camera pose PTZ. cy and t cz Δv and Δu are known constants, which are the coordinate vectors of the error between the center point of the calibration image and the center point of the inspection image, respectively. The corresponding translation t of the chassis is also a constant. h The calculation expression is: Among them, arctan(t) hx / t hz )=θ y orθ y +π, pitch angle θx∈(0, π / 2), rotation angle θy∈(0, 2π), therefore, sinθx>0, the direction of chassis translation is determined by θy and Δv. Based on the values ​​of arctan(thx / thz) corresponding to different conditions, we can obtain: Specifically, when Δv>0, the camera pose PTZ should shift forward, and when Δv<0, it should shift backward. This means that at this stage, the camera pose PTZ will be longitudinally translated by the chassis, and the PI controller is also used to generate the translation t. h Until |Δv|≤εv, it indicates that the camera pose PTZ has been calibrated in the X-axis direction; Phase 3: Camera pose PTZ lateral translation correction, the same as Phase 1, corresponding to the translation th of Δu, its calculation expression is: Among them, t h The direction is θ y -π / 2 or θ y +π / 2, therefore; Specifically, when Δu>0, the camera pose PTZ should be shifted to the right, and when Δu<0, the camera pose PTZ should be shifted to the left. At this stage, the camera pose PTZ will be translated laterally, and then the PI controller will be applied again to generate translation h until |Δu|≤εu, indicating that the camera pose PTZ has been calibrated in the Y-axis direction.

2. The method for secondary precise alignment of images of a substation inspection robot as described in claim 1, characterized in that, The error model is described in detail as follows: The current position of the camera is regarded as a three-dimensional point X in the camera's three-dimensional model. c X c The three-dimensional coordinates are X c =(X c Y c Z c ), X c The vector of three-dimensional coordinates is X. c =(X c Y c Z c ) T In the 3D model where the camera is located, that is, in the camera coordinate system Oc-xyz, the expression for the coordinates of the pixel m of the principal point of the plane image captured by the camera is: m=(u,v) T Where m represents the pixel of the principal point of the planar image captured by the predetermined camera, and (u, v) represent the coordinates of pixel m of the principal point of the planar image captured by the predetermined camera. Calibrating the planar image captured by the camera is equivalent to calibrating the coordinates of the pixel of the principal point of the planar image captured by the predetermined camera, and its calculation expression is: Where fu and fv represent the scaling factors in the horizontal and vertical directions of the planar image captured by the current camera, respectively. When fu = fv, the planar image captured by the camera is a square. (u0, v0) represents the pixel coordinates of the principal point of the planar image captured by the current camera.

3. The substation inspection robot image secondary precise alignment method as described in claim 2, characterized in that: The predetermined position of the camera is regarded as a three-dimensional point in the three-dimensional model of the substation. The error vector of the actual position is Current chassis position chassis displacement error vector t h and the current camera pose The camera pose in the calibration state can be obtained. and camera pose in detection state The calculation expression is: Then, camera pose Error T c The calculation expression is: in, This indicates the camera pose during calibration. This indicates the camera pose during the detection state. This indicates the camera pose in the detection state before adjustment, T c Indicates camera pose The error, T hp = (1, t) hp ) and T pc = (1, t) pc ), This indicates the chassis pose during calibration. R represents the pose transformation matrix of the inspection robot under shooting conditions. cx R cy R is an orthogonal matrix representing the pitch and rotation angle errors of the camera corresponding to a predetermined shooting scene. x R y These are orthogonal matrices, corresponding to the camera pose P respectively. c The pitch angle θx and rotation angle θy, t hp and t pc This indicates the position parameters of the inspection robot at the inspection point. This indicates the current chassis position. The vector representing the current camera pose, i.e., the current camera pose P. c The vector of pitch angle θx and rotation angle θy, t h t represents the displacement error vector of the chassis. c =(t cx , t cy , t cz ) T It is the camera's displacement error vector, (t cx , t cy , t cz The coordinates of the camera in the 3D model of the substation are the X, Y, and Z axes, which represent the coordinates of the camera's location in the camera's coordinate system.

4. The substation inspection robot image secondary precise alignment method as described in claim 3, characterized in that: The pixel count of the principal point in the actual planar image captured by the camera is m* = (u*, v*). T , corresponding to X in the camera's 3D model c Its calculation expression is: In the 3D model map of the substation, the inspection route of the inspection robot can be considered as a plane, and the road surface near the inspection point can be considered as an absolute plane. Therefore, the y-coordinate of the chassis in the chassis coordinate system of the inspection robot is always 0, and t hy =0, we can get: t cz =t cy cotθ x Among them, t h Let Pc represent the chassis displacement error vector, and Pc represent the camera pose in the calibration state. P represents the camera pose during shooting. p Indicates gimbal pose, P h Indicates the chassis pose under calibration conditions, T hp Let T represent the pose transformation matrix of the inspection robot in the calibration state, and T hp = (1, t) hp ) and T pc = (1, t) pc ), where t hp and t pc R represents the position parameters of the inspection robot at the inspection point. x R y The orthogonal matrices in the standard state correspond to the camera pose P. c The pitch angle θx and rotation angle θy, This represents the camera's position before adjustment during shooting, and Tc represents the error of the camera's pose Pc. These are orthogonal matrices in the shooting state, corresponding to the pose in the shooting state, respectively. The pitch angle θx and rotation angle θy, R cx R cy The orthogonal matrix t represents the pitch and rotation angle errors of the camera corresponding to the predetermined shooting image. hp and t pc This indicates the position parameters of the inspection robot at the inspection point. This indicates the chassis position and orientation during filming. This represents the pose transformation matrix of the inspection robot during the shooting process. The vector representing the current camera pose, i.e., the current camera pose P. c The vector of pitch angle θx and rotation angle θy, t c =(t cx , t cy , t cz ) T It is the camera's displacement error vector, (t cx , t cy , t cz The coordinates of the camera in the 3D model of the substation are the X, Y, and Z axes, which represent the coordinates of the camera's location in the camera's coordinate system.

Images