A vision compensation method for pipeline inspection robots, a pipeline inspection robot, and a storage medium.

Abstract

This invention discloses a visual compensation method for pipeline inspection robots, belonging to the field of pipeline inspection robot technology, aiming to solve the problems of poor adaptability and insufficient accuracy of existing technologies. The method constructs a priori geometric model of the pipeline cylinder and its mapping relationship with multiple coordinate systems through system calibration; acquires sequential images of the pipeline inner wall and divides the region of interest; uses an improved gradient variance weighted function to evaluate the focusing state; combines a focusing depth mapping model with geometric constraint solving to calculate the robot's 6-DOF pose deviation; and achieves coordinated pose optimization and camera focus compensation through dual closed-loop control. This solution does not rely on pipeline texture features, effectively solving the pose drift and image defocusing problems in low-texture, highly symmetrical pipeline environments, improving detection accuracy and image acquisition stability, and is suitable for the inspection of closed or semi-closed pipelines such as municipal drainage and industrial transportation.

Description

Technical Field

[0001] This invention relates to a visual compensation method for pipeline inspection robots. Background Technology

[0002] As a core infrastructure for fluid transportation, pipelines are highly susceptible to defects such as corrosion, cracks, and deposits on their inner walls, which directly impact transportation safety. Pipeline inspection robots have become essential equipment for pipeline operation and maintenance. However, existing technologies for pose control and visual acquisition in pipeline inspection robots suffer from the following problems:

[0003] Existing technologies largely rely on the fusion of multiple sensors, such as inertial measurement units (IMUs), odometry, and lidar, to achieve pose correction, with vision serving only as an auxiliary positioning method. Such solutions suffer from high hardware costs and complex sensor calibration. Furthermore, lidar is susceptible to dust interference inside pipes, and IMUs experience cumulative drift, leading to decreased pose accuracy over long distances. Additionally, the lack of a direct correlation between visual clarity and pose deviation fails to fundamentally resolve the vicious cycle of "pose shift - image defocus."

[0004] Therefore, there is an urgent need for a technical solution that can establish a collaborative relationship between visual compensation and pose optimization to solve the problems of poor adaptability, insufficient accuracy, and low stability of existing technologies in pipeline inspection scenarios. Summary of the Invention

[0005] The main objective of this invention is to provide a visual compensation method for pipeline inspection robots, which aims to solve the aforementioned technical problems.

[0006] To achieve the above objectives, the present invention proposes a vision compensation method for a pipeline inspection robot, which includes S1: obtaining the design parameters of the pipeline under test, constructing a cylindrical geometric prior model of the pipeline, and determining the mapping relationship between the reference coordinate system of the pipeline's central axis and the robot's body coordinate system.

[0007] S2. Acquire a sequence of images of the inner wall of the pipeline continuously collected at a preset frame rate as the pipeline inspection robot moves along the pipeline. Divide the region of interest into regions of interest for each frame of the collected image. Calculate the focus evaluation value of each region of interest using the gradient variance weighted focus evaluation function. Based on the focus evaluation value, identify the out-of-focus areas and corresponding out-of-focus degrees in the image.

[0008] S3. Based on the focus evaluation value of the defocused area and the preset focus depth mapping model, calculate the actual depth value of the inner wall of the pipe corresponding to the defocused area, compare the actual depth value of each area with the depth value of the cylindrical geometry prior model, and obtain the 6-DOF pose deviation of the robot body relative to the reference coordinate system of the central axis of the pipe. The 6-DOF pose deviation includes axial offset, radial offset, pitch angle deviation, yaw angle deviation, and roll angle deviation.

[0009] S4. Based on the obtained 6-DOF pose deviation, confirm the control quantity of the robot pose execution unit and correct the robot's spatial pose; at the same time, based on the pose correction of the robot pose data, adjust the focusing parameters of the industrial camera through an adaptive variable step size focusing algorithm.

[0010] S5. Repeat steps S2 to S4. Based on the pose correction data and focus compensation results of the continuous frame sequence, construct a pose cumulative error compensation model to iteratively correct the robot's global travel pose and update the parameters of the focus depth mapping model.

[0011] In one embodiment, the steps of obtaining the design parameters of the pipe under test, constructing the cylindrical geometric prior model of the pipe, and determining the mapping relationship between the reference coordinate system of the pipe's central axis and the robot's body coordinate system are as follows:

[0012] S11. Obtain the camera's focal length, principal point coordinates, radial distortion coefficient, and tangential distortion coefficient, and establish the mapping relationship between the image pixel coordinate system and the camera coordinate system.

[0013] S12. Obtain the rotation matrix and translation vector of the camera coordinate system relative to the robot body coordinate system, and establish the rigid transformation relationship between the camera coordinate system and the robot body coordinate system.

[0014] S13. Obtain the inner diameter, wall thickness, and design curvature parameters of the pipe to be tested, and construct a cylindrical geometric prior model of the pipe. Establish a reference coordinate system with the central axis of the pipe as the Z-axis, and determine the theoretical coordinates and theoretical depth value of any point on the inner wall of the pipe in the reference coordinate system. The theoretical depth value is the theoretical straight-line distance from the point to the optical center of the camera.

[0015] In one embodiment, the step of dividing the acquired image into regions of interest for each frame specifically involves:

[0016] The image is divided into four quadrants with the image center as the origin. Each quadrant is an independent region of interest. At the same time, the region within a preset radius of the image center is set as the core region of interest, and the weight of the core region of interest is higher than that of the quadrant regions.

[0017] In one embodiment, the gradient variance weighted focusing evaluation function is calculated as follows:

[0018] ;

[0019] Where FM is the focus evaluation value of a single region of interest, and the focus evaluation value is positively correlated with image sharpness; M and N are the pixel width and pixel height of the region of interest, respectively. These are the pixel coordinates within the region of interest; The weighting coefficient of a pixel is positively correlated with the gradient magnitude of that pixel. For pixels The gradient magnitude at a given point can be calculated using the Sobel operator. This represents the average gradient magnitude of all pixels within the region of interest.

[0020] In one embodiment, the weight coefficient of the pixel The calculation formula is:

[0021] ;

[0022] in, Let be the maximum gradient magnitude within the region of interest. The local constant is 10. -8 .

[0023] In one embodiment, the step of obtaining the 6-DOF pose deviation of the robot body relative to the reference coordinate system of the pipe's central axis specifically involves:

[0024] S31. Convert the actual depth values ​​corresponding to each defocused area into a set of actual coordinate points in the pipeline reference coordinate system;

[0025] S32. Based on the least squares method, the actual coordinate point set is fitted and matched with the theoretical cylindrical surface of the prior cylindrical geometric model, and the optimal transformation matrix of the actual coordinate point set relative to the theoretical cylindrical surface is solved. The optimal transformation matrix includes the rotation matrix R and the translation vector T.

[0026] S33. Decompose the optimal transformation matrix into the 6-DOF pose deviation of the robot body relative to the pipeline reference coordinate system;

[0027] Among them, the translation vector , , This is the radial offset. This is the axial offset.

[0028] Rotation matrix Decomposed into pitch angle deviation Yaw angle deviation Roll angle deviation The decomposition formula is:

[0029] ;

[0030] ;

[0031] ;

[0032] in, Let be the element in the i-th row and j-th column of the rotation matrix R.

[0033] In one embodiment, the step of determining the control quantity of the robot pose execution unit based on the obtained 6-DOF pose deviation and correcting the robot's spatial pose specifically includes:

[0034] S41. Construct the objective function for model predictive control, taking the minimization of the robot's 6-DOF pose deviation as the optimization objective:

[0035] ;

[0036] Where J is the objective function value, p is the prediction time domain, and c is the control time domain; Let k be the predicted pose deviation at time k; The pose reference value at time k is 0; Let be the control increment at time k; Q is the weight matrix of the pose deviation; R is the weight matrix of the control increment.

[0037] S42. Under the condition of satisfying the displacement and velocity constraints of the actuator, solve the optimal solution of the objective function to obtain the optimal control quantity sequence in the control time domain, and output the first control quantity to the pose execution unit to drive the actuator to correct the robot's spatial pose.

[0038] In one embodiment, the step of simultaneously adjusting the focus parameters of the industrial camera based on the pose-corrected robot pose data using an adaptive variable step size focusing algorithm specifically includes:

[0039] Based on the robot pose data after pose correction, the theoretical depth value from the camera to the inner wall of the pipe is calculated through a cylindrical geometry prior model. The theoretical depth value is then converted into the initial focus position of the camera. Starting from the initial focus position, the camera's focus travel is traversed using a variable step size hill climbing method. The global focus evaluation value of the image at different focus positions is calculated. The focus position corresponding to the maximum value of the global focus evaluation value is taken as the optimal focus position. The camera is then controlled to adjust to the optimal focus position to complete the focal length compensation.

[0040] When the difference between the global focus evaluation values ​​of two adjacent focus positions is greater than the preset step size threshold, a large step size traversal is used; when the difference is less than or equal to the preset step size threshold, a small step size traversal is used, wherein the large step size is 5 to 10 times the small step size.

[0041] In addition, to achieve the above objectives, the present invention also provides a pipeline inspection robot, which includes a memory, a processor, and a vision compensation program stored in the memory and executable on the processor. When the vision compensation program is executed by the processor, it implements the steps of the vision compensation method for the pipeline inspection robot as described above.

[0042] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a vision compensation program, which, when executed by a processor, implements the steps of the vision compensation method for the pipeline inspection robot as described in any of the preceding claims.

[0043] In the technical solution of this invention, the defocus information is obtained by focusing the evaluation function, and the pose deviation is calculated by combining the prior geometric model of the pipeline cylinder. This solves the technical bottleneck of feature point matching failure in pipeline low texture and high symmetry environment. The pose calculation accuracy is not affected by the pipe wall texture and is suitable for various pipeline inspection scenarios. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0045] Figure 1 This is a schematic diagram of the hardware operating environment of the device involved in the embodiments of the present invention;

[0046] Figure 2 This is a schematic flowchart of the vision compensation method for pipeline inspection robots according to the present invention.

[0047] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0048] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0049] like Figure 1 As shown, Figure 1 This is a schematic diagram of the terminal structure of the pipeline robot involved in the embodiment of the present invention.

[0050] like Figure 1As shown, the terminal may include: a processor 1001, such as a CPU; a communication bus 1002; a user interface 1003; a network interface 1004; and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0051] Optionally, the terminal may also include a camera, RF (Radio Frequency) circuitry, sensors, audio circuitry, a WiFi module, and so on. Sensors may include light sensors, motion sensors, and other sensors. Specifically, light sensors may include ambient light sensors and proximity sensors. The ambient light sensor can adjust the display brightness according to the ambient light level, while the proximity sensor can turn off the display and / or backlight when the mobile terminal is moved to the ear. As a type of motion sensor, a gravity accelerometer can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity, and can be used for applications that identify the mobile terminal's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition functions (such as pedometers, taps), etc. Of course, the mobile terminal may also be equipped with other sensors such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, which will not be elaborated here.

[0052] Those skilled in the art will understand that Figure 1 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0053] like Figure 1 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a data calculation program.

[0054] exist Figure 1 In the terminal shown, network interface 1004 is mainly used to connect to the backend server and communicate data with it; user interface 1003 is mainly used to connect to the client (user terminal) and communicate data with it; while processor 1001 can be used to call the data calculation program stored in memory 1005 and perform the following operations:

[0055] S1. Obtain the design parameters of the pipeline under test, construct the cylindrical geometric prior model of the pipeline, and determine the mapping relationship between the reference coordinate system of the pipeline's central axis and the robot's body coordinate system.

[0056] S2. Acquire a sequence of images of the inner wall of the pipeline continuously collected at a preset frame rate as the pipeline inspection robot moves along the pipeline. Divide the region of interest into regions of interest for each frame of the collected image. Calculate the focus evaluation value of each region of interest using the gradient variance weighted focus evaluation function. Based on the focus evaluation value, identify the out-of-focus areas and corresponding out-of-focus degrees in the image.

[0057] S3. Based on the focus evaluation value of the defocused area and the preset focus depth mapping model, calculate the actual depth value of the inner wall of the pipe corresponding to the defocused area, compare the actual depth value of each area with the depth value of the cylindrical geometry prior model, and obtain the 6-DOF pose deviation of the robot body relative to the reference coordinate system of the central axis of the pipe. The 6-DOF pose deviation includes axial offset, radial offset, pitch angle deviation, yaw angle deviation, and roll angle deviation.

[0058] S4. Based on the obtained 6-DOF pose deviation, confirm the control quantity of the robot pose execution unit and correct the robot's spatial pose; at the same time, based on the pose correction of the robot pose data, adjust the focusing parameters of the industrial camera through an adaptive variable step size focusing algorithm.

[0059] S5. Repeat steps S2 to S4. Based on the pose correction data and focus compensation results of the continuous frame sequence, construct a pose cumulative error compensation model to iteratively correct the robot's global travel pose and update the parameters of the focus depth mapping model.

[0060] Furthermore, the processor 1001 can call the data calculation program stored in the memory 1005 and also perform the following operations:

[0061] S11. Obtain the camera's focal length, principal point coordinates, radial distortion coefficient, and tangential distortion coefficient, and establish the mapping relationship between the image pixel coordinate system and the camera coordinate system.

[0062] S12. Obtain the rotation matrix and translation vector of the camera coordinate system relative to the robot body coordinate system, and establish the rigid transformation relationship between the camera coordinate system and the robot body coordinate system.

[0063] S13. Obtain the inner diameter, wall thickness, and design curvature parameters of the pipe to be tested, and construct a cylindrical geometric prior model of the pipe. Establish a reference coordinate system with the central axis of the pipe as the Z-axis, and determine the theoretical coordinates and theoretical depth value of any point on the inner wall of the pipe in the reference coordinate system. The theoretical depth value is the theoretical straight-line distance from the point to the optical center of the camera.

[0064] Furthermore, the processor 1001 can call the data calculation program stored in the memory 1005 and also perform the following operations:

[0065] The image is divided into four quadrants with the image center as the origin. Each quadrant is an independent region of interest. At the same time, the region within a preset radius of the image center is set as the core region of interest, and the weight of the core region of interest is higher than that of the quadrant regions.

[0066] Furthermore, the processor 1001 can call the data calculation program stored in the memory 1005 and also perform the following operations:

[0067] S31. Convert the actual depth values ​​corresponding to each defocused area into a set of actual coordinate points in the pipeline reference coordinate system;

[0068] S32. Based on the least squares method, the actual coordinate point set is fitted and matched with the theoretical cylindrical surface of the prior cylindrical geometric model, and the optimal transformation matrix of the actual coordinate point set relative to the theoretical cylindrical surface is solved. The optimal transformation matrix includes the rotation matrix R and the translation vector T.

[0069] S33. Decompose the optimal transformation matrix into the 6-DOF pose deviation of the robot body relative to the pipeline reference coordinate system.

[0070] Furthermore, the processor 1001 can call the data calculation program stored in the memory 1005 and also perform the following operations:

[0071] S41. With minimizing the robot's 6-DOF pose deviation as the optimization objective, construct the objective function for model predictive control;

[0072] S42. Under the conditions of satisfying the displacement and velocity constraints of the actuator, solve for the optimal solution of the objective function to obtain the optimal control quantity sequence in the control time domain. Output the first control quantity to the pose execution unit to drive the actuator to correct the robot's spatial pose.

[0073] Furthermore, the processor 1001 can call the data calculation program stored in the memory 1005 and also perform the following operations:

[0074] Based on the robot pose data after pose correction, the theoretical depth value from the camera to the inner wall of the pipe is calculated through a cylindrical geometry prior model. The theoretical depth value is then converted into the initial focus position of the camera. Starting from the initial focus position, the camera's focus travel is traversed using a variable step size hill climbing method. The global focus evaluation value of the image at different focus positions is calculated. The focus position corresponding to the maximum value of the global focus evaluation value is taken as the optimal focus position. The camera is then controlled to adjust to the optimal focus position to complete the focal length compensation.

[0075] When the difference between the global focus evaluation values ​​of two adjacent focus positions is greater than the preset step size threshold, a large step size traversal is used; when the difference is less than or equal to the preset step size threshold, a small step size traversal is used, wherein the large step size is 5 to 10 times the small step size.

[0076] The specific embodiments of the pipeline robot of the present invention are basically the same as the embodiments of the vision compensation method for pipeline inspection robots described below, and will not be repeated here.

[0077] Reference Figure 2 The first embodiment of the present invention provides a visual compensation method for a pipeline inspection robot, the visual compensation method for a pipeline inspection robot includes:

[0078] S1. Obtain the design parameters of the pipeline under test, construct the cylindrical geometric prior model of the pipeline, and determine the mapping relationship between the reference coordinate system of the pipeline's central axis and the robot's body coordinate system.

[0079] Specifically, it includes:

[0080] S11. Obtain the camera's focal length, principal point coordinates, radial distortion coefficient, and tangential distortion coefficient, and establish the mapping relationship between the image pixel coordinate system and the camera coordinate system.

[0081] In this embodiment, the obtained focal length parameter reflects the imaging magnification capability of the camera, the principal point coordinates are the origin of the image coordinate system, usually located near the center of the image, and the radial distortion coefficient is used to correct pixel shifts caused by lens optical distortion, such as barrel distortion and pincushion distortion.

[0082] A pixel coordinate system can be established through intrinsic parameter calibration. With camera coordinate system The mapping relationship is as follows: pixel coordinates are converted into normalized image coordinates, and then mapped to the camera coordinate system through an intrinsic parameter matrix. The formula is:

[0083] ;

[0084] in, The focal lengths are in the x and y directions. The coordinates of the main point.

[0085] S12. Obtain the rotation matrix and translation vector of the camera coordinate system relative to the robot body coordinate system, and establish the rigid transformation relationship between the camera coordinate system and the robot body coordinate system.

[0086] In this embodiment, the robot is controlled to move the camera to multiple different postures, images of the calibration board are acquired, and the rotation matrix of the camera coordinate system relative to the robot's body coordinate system is solved. With translation vector The rigid transformation relationship means that there is no relative motion between the camera and the robot body; therefore, a point in the camera coordinate system at any given time can be obtained through... and The formula for converting to the robot's body coordinate system is:

[0087] ;

[0088] in, Coordinates in the robot's body coordinate system These are the coordinates in the camera coordinate system.

[0089] S13. Obtain the inner diameter, wall thickness, and design curvature parameters of the pipe to be tested, and construct a cylindrical geometric prior model of the pipe. Establish a reference coordinate system with the central axis of the pipe as the Z-axis, and determine the theoretical coordinates and theoretical depth value of any point on the inner wall of the pipe in the reference coordinate system. The theoretical depth value is the theoretical straight-line distance from the point to the optical center of the camera.

[0090] In this embodiment, the design parameters of the pipe under test can be obtained from the pipe design drawings and previous measurement data. The inner diameter parameter determines the radius of the cylindrical model, and the pipe wall thickness is used to distinguish between the inner and outer walls of the pipe to ensure that the depth value calculation is for the inner wall.

[0091] Curvature parameters are designed to adapt to the model construction of non-linear pipes (such as bends). The core of the cylindrical geometry prior model is to abstract the inner wall of the pipe as an ideal cylindrical surface and establish a reference coordinate system with the central axis of the pipe as the Z-axis. This coordinate system provides an absolute reference for the pose deviation calculation.

[0092] The theoretical coordinates of any point on the inner wall of the pipe can be identified by the cylindrical equation, and the theoretical depth value is the value of that point. The Euclidean distance between the optical center of the camera and the coordinates of the camera in the reference coordinate system.

[0093] S2. Acquire a sequence of images of the inner wall of the pipeline continuously collected at a preset frame rate as the pipeline inspection robot moves along the pipeline. Divide the region of interest into regions of interest for each frame of the acquired image. Calculate the focus evaluation value of each region of interest using a gradient variance weighted focus evaluation function. Based on the focus evaluation value, identify the out-of-focus areas and their corresponding degrees of out-of-focus in the image.

[0094] In this embodiment, the image center is used as the origin to divide the area into four quadrants because the inner wall of the pipe is distributed in a ring shape in the camera's field of view. The quadrant division can cover the inner wall area in different directions in the field of view, ensuring that defocusing in all directions can be detected.

[0095] The imaging quality of the central region of the camera is usually better than that of the edge region (less affected by lens distortion), and the core region is closer to the central axis of the pipe, so its depth information has a higher weight in pose calculation. Therefore, by increasing the weight of the core region, the accuracy of pose calculation can be improved.

[0096] In this application, the value of the preset radius needs to be determined based on the inner diameter of the pipe and the field of view of the camera, and is usually 1 / 4 to 1 / 3 of the image width to ensure that the core area can cover the key detection area of ​​the inner wall of the pipe.

[0097] Furthermore, the calculation formula for the gradient variance weighted focusing evaluation function is as follows:

[0098] ;

[0099] Where FM is the focus evaluation value of a single region of interest, and the focus evaluation value is positively correlated with image sharpness; M and N are the pixel width and pixel height of the region of interest, respectively. These are the pixel coordinates within the region of interest; The weighting coefficient of a pixel is positively correlated with the gradient magnitude of that pixel. For pixels The gradient magnitude at a given point can be calculated using the Sobel operator. This represents the average gradient magnitude of all pixels within the region of interest.

[0100] In images of the inner wall of a pipe, pixels with larger gradient magnitudes (such as defect edges and pipe wall texture boundaries) better reflect the image's sharpness, while pixels with smaller gradients (such as uniform pipe wall regions) contribute less to sharpness. Therefore, a weighting coefficient is introduced. The calculation formula is:

[0101] ;

[0102] in, Let be the maximum gradient magnitude within the region of interest. The local constant is 10. -8 .

[0103] Weighting coefficients can be applied to pixels with large gradients, enhancing the evaluation function's sensitivity to sharpness changes while suppressing noise interference in uniform regions. The average gradient magnitude of the region is used to calculate the deviation between the gradient of a single pixel and the average gradient of the region. The larger the sum of squared deviations, the more dispersed the pixel gradient distribution within the region, and the clearer the image edges. Therefore, the focus evaluation value FM is positively correlated with sharpness.

[0104] Therefore, by acquiring multiple frames of images of the inner wall of the pipe, the focus evaluation value of each region of interest is calculated, and the minimum value of these evaluation values ​​is taken as the sharpness threshold. When the FM of a certain region is lower than the threshold, it indicates that the image of that region is out of focus (out of focus), and the larger the FM, the more serious the out of focus.

[0105] S3. Based on the focus evaluation value of the defocused area and the preset focus depth mapping model, calculate the actual depth value of the inner wall of the pipe corresponding to the defocused area. Compare the actual depth value of each area with the depth value of the cylindrical geometric prior model to obtain the 6-DOF pose deviation of the robot body relative to the reference coordinate system of the pipe center axis. The 6-DOF pose deviation includes axial offset, radial offset, pitch angle deviation, yaw angle deviation, and roll angle deviation.

[0106] Specifically, the steps include:

[0107] S31. Convert the actual depth values ​​corresponding to each defocused area into a set of actual coordinate points in the pipeline reference coordinate system.

[0108] The actual depth value d is the straight-line distance from the camera's optical center to a point on the inner wall of the pipe in the camera coordinate system. It needs to be combined with the pixel coordinates of that point in the image. The coordinates are converted to camera coordinates using an intrinsic parameter matrix. Then through the extrinsic parameter matrix Convert to robot body coordinate system Finally, by mapping the robot's body coordinate system to the reference coordinate system, it is converted into a set of actual coordinate points in the reference coordinate system. k represents the number of regions of interest.

[0109] S32. Based on the least squares method, the actual coordinate point set is fitted and matched with the theoretical cylindrical surface of the prior cylindrical geometric model, and the optimal transformation matrix of the actual coordinate point set relative to the theoretical cylindrical surface is solved. The optimal transformation matrix includes the rotation matrix R and the translation vector T.

[0110] In this embodiment, the core of least squares fitting is to find the optimal rotation matrix R and translation vector T, such that the sum of the squares of the distances between the actual coordinate point set and the theoretical cylindrical surface of the prior cylindrical geometric model is minimized after the actual coordinate point set is rotated by R and translated by T.

[0111] S33. Decompose the optimal transformation matrix into the 6-DOF pose deviation of the robot body relative to the pipeline reference coordinate system;

[0112] Among them, the translation vector , , This is the radial offset. This is the axial offset.

[0113] Rotation matrix Decomposed into pitch angle deviation Yaw angle deviation Roll angle deviation The decomposition formula is:

[0114] ;

[0115] ;

[0116] ;

[0117] in, Let be the element in the i-th row and j-th column of the rotation matrix R.

[0118] S4. Based on the obtained 6-DOF pose deviation, confirm the control quantity of the robot pose execution unit and correct the robot's spatial pose; at the same time, based on the pose correction of the robot pose data, adjust the focusing parameters of the industrial camera through an adaptive variable step size focusing algorithm.

[0119] Specifically, step S4 includes:

[0120] S41. Construct the objective function for model predictive control, taking the minimization of the robot's 6-DOF pose deviation as the optimization objective:

[0121] ;

[0122] Where J is the objective function value, p is the prediction time domain, and c is the control time domain; Let k be the predicted pose deviation at time k; The pose reference value at time k is 0; Let be the control increment at time k; Q is the weight matrix of the pose deviation; and R is the weight matrix of the control increment.

[0123] In this objective function, the first term Represents the prediction time domain Accumulated pose deviation within. Second term. This represents the cumulative increment of the control quantity within the control time domain c. The values ​​of the prediction time domain p and the control time domain c can generally be determined based on the dynamic response characteristics of the robot. Typically, p can be 5~10 and c can be 2~5 to ensure the accuracy of prediction and the real-time performance of control.

[0124] S42. Under the condition of satisfying the displacement and velocity constraints of the actuator, solve the optimal solution of the objective function to obtain the optimal control quantity sequence in the control time domain, and output the first control quantity to the pose execution unit to drive the actuator to correct the robot's spatial pose.

[0125] In this embodiment, the constraints of the actuator include displacement constraints and velocity constraints. The displacement constraint can be that the maximum radial offset correction is 10% of the inner diameter of the pipe to avoid collision between the robot and the pipe wall. The velocity constraint can be that the maximum adjustment speed is 0.1 rad / s to avoid sudden changes in attitude. When solving the objective function, these constraints are substituted into the solution and the optimal control quantity sequence is solved by a quadratic programming algorithm.

[0126] S5. Repeat steps S2 to S4. Based on the pose correction data and focus compensation results of the continuous frame sequence, construct a pose cumulative error compensation model to iteratively correct the robot's global travel pose and update the parameters of the focus depth mapping model.

[0127] Specifically, the step of simultaneously adjusting the focus parameters of the industrial camera based on the pose-corrected robot pose data using an adaptive variable step-size focusing algorithm is as follows:

[0128] Based on the robot pose data after pose correction, the theoretical depth value from the camera to the inner wall of the pipe is calculated through a cylindrical geometry prior model. The theoretical depth value is then converted into the initial focus position of the camera. Starting from the initial focus position, the camera's focus travel is traversed using a variable step size hill climbing method. The global focus evaluation value of the image at different focus positions is calculated. The focus position corresponding to the maximum value of the global focus evaluation value is taken as the optimal focus position. The camera is then controlled to adjust to the optimal focus position to complete the focal length compensation.

[0129] When the difference between the global focus evaluation values ​​of two adjacent focus positions is greater than the preset step size threshold, a large step size traversal is used; when the difference is less than or equal to the preset step size threshold, a small step size traversal is used, wherein the large step size is 5 to 10 times the small step size.

[0130] In this embodiment, the length of the sliding window is set to 10-20 frames. If the length is too short, there will be insufficient observation data and the filtering accuracy will decrease; if the length is too long, the computational load will increase and the real-time performance will decrease.

[0131] Furthermore, embodiments of the present invention also propose a computer-readable storage medium storing a data calculation program, which, when executed by a processor, performs the following operations:

[0132] S1. Obtain the design parameters of the pipeline under test, construct the cylindrical geometric prior model of the pipeline, and determine the mapping relationship between the reference coordinate system of the pipeline's central axis and the robot's body coordinate system.

[0133] S2. Acquire a sequence of images of the inner wall of the pipeline continuously collected at a preset frame rate as the pipeline inspection robot moves along the pipeline. Divide the region of interest into regions of interest for each frame of the collected image. Calculate the focus evaluation value of each region of interest using the gradient variance weighted focus evaluation function. Based on the focus evaluation value, identify the out-of-focus areas and corresponding out-of-focus degrees in the image.

[0134] S3. Based on the focus evaluation value of the defocused area and the preset focus depth mapping model, calculate the actual depth value of the inner wall of the pipe corresponding to the defocused area, compare the actual depth value of each area with the depth value of the cylindrical geometry prior model, and obtain the 6-DOF pose deviation of the robot body relative to the reference coordinate system of the central axis of the pipe. The 6-DOF pose deviation includes axial offset, radial offset, pitch angle deviation, yaw angle deviation, and roll angle deviation.

[0135] S4. Based on the obtained 6-DOF pose deviation, confirm the control quantity of the robot pose execution unit and correct the robot's spatial pose; at the same time, based on the pose correction of the robot pose data, adjust the focusing parameters of the industrial camera through an adaptive variable step size focusing algorithm.

[0136] S5. Repeat steps S2 to S4. Based on the pose correction data and focus compensation results of the continuous frame sequence, construct a pose cumulative error compensation model to iteratively correct the robot's global travel pose and update the parameters of the focus depth mapping model.

[0137] Furthermore, when the data calculation program is executed by the processor, it also performs the following operations:

[0138] S11. Obtain the camera's focal length, principal point coordinates, radial distortion coefficient, and tangential distortion coefficient, and establish the mapping relationship between the image pixel coordinate system and the camera coordinate system.

[0139] S12. Obtain the rotation matrix and translation vector of the camera coordinate system relative to the robot body coordinate system, and establish the rigid transformation relationship between the camera coordinate system and the robot body coordinate system.

[0140] S13. Obtain the inner diameter, wall thickness, and design curvature parameters of the pipe to be tested, and construct a cylindrical geometric prior model of the pipe. Establish a reference coordinate system with the central axis of the pipe as the Z-axis, and determine the theoretical coordinates and theoretical depth value of any point on the inner wall of the pipe in the reference coordinate system. The theoretical depth value is the theoretical straight-line distance from the point to the optical center of the camera.

[0141] Furthermore, when the data calculation program is executed by the processor, it also performs the following operations:

[0142] The image is divided into four quadrants with the image center as the origin. Each quadrant is an independent region of interest. At the same time, the region within a preset radius of the image center is set as the core region of interest, and the weight of the core region of interest is higher than that of the quadrant regions.

[0143] Furthermore, when the data calculation program is executed by the processor, it also performs the following operations:

[0144] S31. Convert the actual depth values ​​corresponding to each defocused area into a set of actual coordinate points in the pipeline reference coordinate system;

[0145] S32. Based on the least squares method, the actual coordinate point set is fitted and matched with the theoretical cylindrical surface of the prior cylindrical geometric model, and the optimal transformation matrix of the actual coordinate point set relative to the theoretical cylindrical surface is solved. The optimal transformation matrix includes the rotation matrix R and the translation vector T.

[0146] S33. Decompose the optimal transformation matrix into the 6-DOF pose deviation of the robot body relative to the pipeline reference coordinate system.

[0147] Furthermore, when the data calculation program is executed by the processor, it also performs the following operations:

[0148] S41. With minimizing the robot's 6-DOF pose deviation as the optimization objective, construct the objective function for model predictive control;

[0149] S42. Under the conditions of satisfying the displacement and velocity constraints of the actuator, solve for the optimal solution of the objective function to obtain the optimal control quantity sequence in the control time domain. Output the first control quantity to the pose execution unit to drive the actuator to correct the robot's spatial pose.

[0150] Furthermore, when the data calculation program is executed by the processor, it also performs the following operations:

[0151] Based on the robot pose data after pose correction, the theoretical depth value from the camera to the inner wall of the pipe is calculated through a cylindrical geometry prior model. The theoretical depth value is then converted into the initial focus position of the camera. Starting from the initial focus position, the camera's focus travel is traversed using a variable step size hill climbing method. The global focus evaluation value of the image at different focus positions is calculated. The focus position corresponding to the maximum value of the global focus evaluation value is taken as the optimal focus position. The camera is then controlled to adjust to the optimal focus position to complete the focal length compensation.

[0152] When the difference between the global focus evaluation values ​​of two adjacent focus positions is greater than a preset step size threshold, a large step size traversal is used; when the difference is less than or equal to the preset step size threshold, a small step size traversal is used, where the large step size is 5 to 10 times the small step size.

[0153] The specific embodiments of the computer-readable storage medium of the present invention are basically the same as the embodiments of the above-described vision compensation method for pipeline inspection robots, and will not be described in detail here.

[0154] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0155] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0156] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0157] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for vision compensation of a pipeline inspection robot, characterized by, The vision compensation method for the pipeline inspection robot includes: S1. Obtain the design parameters of the pipeline under test, construct the cylindrical geometric prior model of the pipeline, and determine the mapping relationship between the reference coordinate system of the pipeline's central axis and the robot's body coordinate system. S2. Acquire a sequence of images of the inner wall of the pipeline continuously collected at a preset frame rate as the pipeline inspection robot moves along the pipeline. Divide the region of interest into regions of interest for each frame of the collected image. Calculate the focus evaluation value of each region of interest using the gradient variance weighted focus evaluation function. Based on the focus evaluation value, identify the out-of-focus areas and corresponding out-of-focus degrees in the image. S3. Based on the focus evaluation value of the defocused area and the preset focus depth mapping model, calculate the actual depth value of the inner wall of the pipe corresponding to the defocused area, compare the actual depth value of each area with the depth value of the cylindrical geometry prior model, and obtain the 6-degree-of-freedom pose deviation of the robot body relative to the reference coordinate system of the central axis of the pipe. The 6-degree-of-freedom pose deviation includes axial offset, radial offset, pitch angle deviation, yaw angle deviation, and roll angle deviation. S4. Based on the obtained 6-DOF pose deviation, confirm the control quantity of the robot pose execution unit and correct the robot's spatial pose; at the same time, based on the pose correction of the robot pose data, adjust the focusing parameters of the industrial camera through an adaptive variable step size focusing algorithm. S5. Repeat steps S2 to S4. Based on the pose correction data and focus compensation results of the continuous frame sequence, construct a pose cumulative error compensation model to iteratively correct the robot's global travel pose and update the parameters of the focus depth mapping model.

2. The vision compensation method for pipeline inspection robots according to claim 1, characterized in that, The steps of obtaining the design parameters of the pipe under test, constructing the cylindrical geometric prior model of the pipe, and determining the mapping relationship between the reference coordinate system of the pipe's central axis and the robot's body coordinate system are as follows: S11. Obtain the camera's focal length, principal point coordinates, radial distortion coefficient, and tangential distortion coefficient, and establish the mapping relationship between the image pixel coordinate system and the camera coordinate system. S12. Obtain the rotation matrix and translation vector of the camera coordinate system relative to the robot body coordinate system, and establish the rigid transformation relationship between the camera coordinate system and the robot body coordinate system. S13. Obtain the inner diameter, wall thickness, and design curvature parameters of the pipe to be tested, and construct a cylindrical geometric prior model of the pipe. Establish a reference coordinate system with the central axis of the pipe as the Z-axis, and determine the theoretical coordinates and theoretical depth value of any point on the inner wall of the pipe in the reference coordinate system. The theoretical depth value is the theoretical straight-line distance from the point to the optical center of the camera.

3. The vision compensation method for pipeline inspection robots according to claim 1, characterized in that, The specific steps for dividing each frame of the acquired image into regions of interest are as follows: The image is divided into four quadrants with the image center as the origin. Each quadrant is an independent region of interest. At the same time, the region within a preset radius of the image center is set as the core region of interest, and the weight of the core region of interest is higher than that of the quadrant regions.

4. The vision compensation method for pipeline inspection robots according to claim 1, characterized in that, The formula for calculating the gradient variance weighted focusing evaluation function is as follows: ； Where FM is the focus evaluation value of a single region of interest, and the focus evaluation value is positively correlated with image sharpness; M and N are the pixel width and pixel height of the region of interest, respectively. These are the pixel coordinates within the region of interest; The weighting coefficient of a pixel is positively correlated with the gradient magnitude of that pixel. For pixels The gradient magnitude at a given point can be calculated using the Sobel operator. This represents the average gradient magnitude of all pixels within the region of interest.

5. The vision compensation method for pipeline inspection robots according to claim 4, characterized in that, The weight coefficient of the pixel The calculation formula is: ； in, Let be the maximum gradient magnitude within the region of interest. The local constant is 10. -8 .

6. The vision compensation method for pipeline inspection robots according to claim 1, characterized in that, The specific steps for obtaining the 6-DOF pose deviation of the robot body relative to the reference coordinate system of the central axis of the pipeline are as follows: S31. Convert the actual depth values ​​corresponding to each defocused area into a set of actual coordinate points in the pipeline reference coordinate system; S32. Based on the least squares method, the actual coordinate point set is fitted and matched with the theoretical cylindrical surface of the prior cylindrical geometric model, and the optimal transformation matrix of the actual coordinate point set relative to the theoretical cylindrical surface is solved. The optimal transformation matrix includes the rotation matrix R and the translation vector T. S33. Decompose the optimal transformation matrix into the 6-DOF pose deviation of the robot body relative to the pipeline reference coordinate system; Among them, the translation vector , , This is the radial offset. This is the axial offset. Rotation matrix Decomposed into pitch angle deviation Yaw angle deviation Roll angle deviation The decomposition formula is: ； ； ； in, Let be the element in the i-th row and j-th column of the rotation matrix R.

7. The vision compensation method for pipeline inspection robots according to claim 1, characterized in that, The specific steps for determining the control quantity of the robot pose execution unit based on the obtained 6-DOF pose deviation and correcting the robot's spatial pose are as follows: S41. Construct the objective function for model predictive control, taking the minimization of the robot's 6-DOF pose deviation as the optimization objective: ； Where J is the objective function value, p is the prediction time domain, and c is the control time domain; Let k be the predicted pose deviation at time k; The pose reference value at time k is 0; Let be the control increment at time k; Q is the weight matrix of pose deviation; R is the weight matrix of control increment. S42. Under the condition of satisfying the displacement and velocity constraints of the actuator, solve the optimal solution of the objective function to obtain the optimal control quantity sequence in the control time domain, and output the first control quantity to the pose execution unit to drive the actuator to correct the robot's spatial pose.

8. The vision compensation method for pipeline inspection robots according to claim 1, characterized in that, The step of simultaneously adjusting the focus parameters of the industrial camera based on the robot pose data after pose correction using an adaptive variable step size focusing algorithm is as follows: Based on the robot pose data after pose correction, the theoretical depth value from the camera to the inner wall of the pipe is calculated through a cylindrical geometry prior model, and the theoretical depth value is converted into the initial focus position of the camera. Starting from the initial focus position, the camera's focus travel is traversed using the variable step size hill climbing method. The global focus evaluation value of the image at different focus positions is calculated. The focus position corresponding to the maximum global focus evaluation value is taken as the optimal focus position. The camera is then controlled to adjust to the optimal focus position to complete the focal length compensation. When the difference between the global focus evaluation values ​​of two adjacent focus positions is greater than the preset step size threshold, a large step size traversal is used. When the difference is less than or equal to the preset step size threshold, a small step size traversal is adopted, and the large step size is 5 to 10 times the small step size.

9. A pipeline inspection robot, characterized in that, The pipeline inspection robot includes a memory, a processor, and a vision compensation program stored in the memory and executable on the processor. When the vision compensation program is executed by the processor, it implements the steps of the vision compensation method for the pipeline inspection robot as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a vision compensation program, which, when executed by a processor, implements the steps of the vision compensation method for a pipeline inspection robot as described in any one of claims 1 to 8.

Images