Inspection robot stockade transfer and platform centralized path coordination control method

Abstract

The application relates to the technical field of track inspection, and discloses a track transfer and platform centralized control path cooperative control method of an inspection robot, which has the technical scheme points that the method comprises a data acquisition step, a relative position calculation step, a camera parameter solving step and a calibration precision judgment step. Through multi-height data acquisition, dynamic camera parameter calibration and a fault-tolerant mechanism, the problem of low visual positioning precision in a dynamic lifting environment is solved, and the method has the advantages of improving positioning precision and system reliability.

Description

Technical Field

[0001] This invention relates to the field of rail transit inspection technology, and more specifically to a method for coordinated control of track transport and platform centralized control path of inspection robots. Background Technology

[0002] To accommodate train operation, subway tracks are typically lower than station platforms or the external ground, forming a relatively enclosed recessed area. Inspection robots used in inspection systems need to inspect not only the sides of the train but also the underside within the tracks. Therefore, these robots must move back and forth between external charging areas, the train side inspection areas, and the track inspection areas. Due to the height difference between the tracks and the external environment, inspection robots cannot directly enter the tracks on a flat surface. Therefore, a lifting platform or ramp transfer structure is usually installed at the connection point between the tracks and the external environment. The lifting platform includes a vertically movable platform and a lifting drive mechanism. When the inspection robot moves onto the platform, the lifting drive mechanism lowers the platform until it is level with the track surface, allowing the robot to enter the track. The ramp transfer structure includes an inclined ramp, with one end connecting to the external ground and the other end extending to the track, allowing the inspection robot to enter the track. During inspection, images are captured by an image detection device mounted on the inspection robot. However, in actual use, under existing technology, there is a height difference between subway tracks and the external ground, requiring inspection robots to use lifting platforms or ramp structures to move between areas. Traditional positioning methods rely on mechanical encoders or inertial navigation, but mechanical vibrations, changes in lighting, and deformation of the support platform during lifting can easily cause visual sensor parameters to drift, resulting in inaccurate mapping between image coordinates and physical space coordinates, affecting the accuracy of inspection path planning. Summary of the Invention

[0003] To address the shortcomings of existing technologies, the present invention aims to provide a collaborative control method for the track-based transport and platform-based centralized control path of an inspection robot. Through strategies such as multimodal data acquisition, error decomposition and closed-loop calibration, and dynamic weight adaptive adjustment, the method achieves high-precision positioning and pose correction of the inspection robot on the lifting platform, improves the positioning reliability and path planning accuracy during transport, and optimizes the collaborative efficiency of the robotic arm and image acquisition module.

[0004] To achieve the above objectives, the present invention provides the following technical solution: The inspection robot is equipped with a robotic arm and an image acquisition module. The platform serves as the robot's transport carrier. Its key feature is that the platform is pre-set with positioning markers, which are high-precision calibration plates with known three-dimensional coordinates, such as a checkerboard calibration plate. The checkerboard unit size accuracy is ≤0.01mm, and the three-dimensional coordinates of its feature points in the platform's coordinate system are pre-calibrated. The control method includes a data acquisition step: when the inspection robot rises and falls with the platform, the platform pauses at intervals to N preset sampling heights, where N≥3, and these heights are evenly distributed along the platform's rising and falling stroke, covering the entire rising and falling range. At each preset sampling height, [the following is a diagram]. The acquisition module acquires a sequence of calibration images, representing continuously acquired calibration image information. The relative position calculation step involves feature extraction processing of each calibration image to identify feature points on the positioning markers, obtaining image coordinates and corresponding sampling heights to construct a camera imaging dataset. The image coordinates reflect the coordinates of the calibrated feature points in a preset platform coordinate system, i.e., the image pixel coordinates (u,v) corresponding to the three-dimensional coordinates (X,Y,0) of the feature point at Z=0 on the platform plane. The camera parameter solution step establishes a distorted pinhole camera imaging model based on the camera imaging dataset. The model expression is: S, Where K is the camera intrinsic parameter matrix, [R|t] is the camera extrinsic parameter relative to the platform, and the distortion compensation term includes radial distortion coefficients k1 and k2 and tangential distortion coefficients p1 and p2. To correspond to the sampling height, the camera imaging model is solved and the camera intrinsic parameters are calculated, including the focal length f. x The parameters are: fᵧ, principal point coordinates u0, v0, radial distortion coefficients k1, k2, and tangential distortion coefficients p1, p2; calibration accuracy judgment step: calculate the reprojection error of feature points at all sampling heights. The reprojection error is specifically the difference between the measured coordinates of the image and the predicted coordinates of the parameters. If the mean of the reprojection error is less than the preset reprojection error threshold, it is considered to meet the preset accuracy requirements. The currently calculated camera intrinsic parameters are used as the reference parameters. The reference parameters are used for image distortion correction and the transformation between image coordinates and the carrier platform coordinate system during the inspection phase. If the mean of the reprojection error is greater than or equal to the preset reprojection error threshold, the preset calibration quality control strategy is triggered, and the camera parameter solution step is re-executed. The calibration quality control strategy includes at least re-sampling calibration images at invalid sampling heights and removing low-precision feature points.

[0005] Furthermore, this application proposes a calibration quality control strategy including: a feature point screening sub-step, which removes low-precision points, specifically feature points in each calibrated image information whose distance from the image edge is less than a preset edge pixel distance threshold, or feature points whose distance from adjacent feature points is less than a preset feature point distance threshold; and a frame validity determination sub-step, which determines the image quality and the number of feature points for each calibrated image information, with image sharpness quantized using the Laplacian gradient mean, calculated using the following formula: , among which ▽ 2 I(x,y) is the second-order Laplacian operator of image I at pixel (x,y), M is the total number of valid pixels in the image, and valid feature points are the feature points remaining after removing low-precision points in the feature point filtering sub-step. If the preset valid conditions are met, specifically, the image sharpness is greater than or equal to a preset image sharpness threshold and the number of valid feature points is greater than or equal to a preset valid feature point number threshold, then the calibrated image information is marked as valid image information. If the preset valid conditions are not met, the calibrated image information is marked as invalid image information and a supplementary sampling signal is generated. The weight allocation sub-step assigns weights to each valid image information, and the weight calculation method is as follows: , Where α and β are weighting coefficients, α+β=1 and both α and β are 0.5; the higher the image clarity and the more effective feature points, the higher the weight of effective image information, and the higher the weight of high-weight image information is given higher constraint weight when solving camera parameters, that is, the feature point reprojection error weight of the image information is amplified by W times during the solution process.

[0006] Furthermore, this application proposes that the camera parameter solving step also includes a dynamic update sub-strategy, configured with a preset environmental parameter mapping model. The environmental parameter mapping model reflects the relationship between the initial parameter values ​​and the environmental parameters, where the environmental parameters include illumination intensity (unit: lux), platform vibration amplitude (unit: mm), and load status (no load / full load). The initial parameter values ​​are the initial values ​​of the camera intrinsic parameters, including the initial focal length, initial principal point coordinates, and initial distortion coefficients. After each calibration, the calculation deviation between the current benchmark parameters and the historical optimal parameters is calculated. The historical optimal parameters are those calculated from the camera intrinsic parameters generated in the past 10 valid calibration cycles stored in the system, which satisfy the condition of having the smallest average reprojection error and being used for subsequent inspection. The parameters whose measurement error does not exceed the preset inspection measurement error threshold are considered as follows: If the newly generated current benchmark parameter satisfies the condition that the average reprojection error is less than the average reprojection error of the historical best parameter, and no inspection measurement error exceeds the standard within one subsequent transport cycle, then the historical best parameter is updated to the new parameter; If the measurement deviation is less than the preset parameter deviation threshold, then the current benchmark parameter and the historical best parameter are weighted and fused to obtain a new benchmark parameter. The fusion formula is: new benchmark parameter = current benchmark parameter × 0.3 + historical best parameter × 0.7, where the current benchmark parameter accounts for 30% of the weight and the historical best parameter accounts for 70% of the weight; If the measurement deviation is greater than or equal to the preset parameter deviation threshold, then the current benchmark parameter replaces the historical best parameter.

[0007] Furthermore, this application also proposes that the data acquisition step further includes an observation-assisted constraint sub-strategy, including: configuring robotic arm action triggering conditions, with the field of view coverage rate being the ratio of the pixel area occupied by the effective feature region of the positioning marker in the calibration image information to the pixel area of ​​the total effective feature region of the positioning marker; when the field of view coverage rate of the positioning marker in the calibration image information is less than a preset field of view coverage rate threshold, triggering the robotic arm to perform action adjustment within a preset robotic arm adjustment range, the preset robotic arm adjustment range including a preset pitch angle adjustment threshold, a preset yaw angle adjustment threshold, and a preset end effector extension adjustment threshold.

[0008] Furthermore, this application also proposes that the calibration accuracy judgment step further includes a fault-tolerant application sub-strategy, including: if calibration failure occurs in a single calibration, after acquiring calibration images at 3 invalid sampling heights, the average reprojection error is still ≥ a preset reprojection error threshold; if the number of consecutive failures reaches a preset calibration failure error threshold, then the historical best parameters are used as a temporary reference and an error signal is output to prompt the inspection of the integrity of the positioning markers; if the image measurement error detected in the inspection section is greater than the preset inspection measurement error threshold, then the inspection is paused, and the simplified calibration strategy is executed on the carrier platform. The simplified calibration strategy is configured with 3 preset simplified sampling heights, namely the lowest height, the middle height, and the highest height of the carrier platform's lifting stroke; the control method re-executed is the data acquisition step in claim 1, which only acquires calibration images at 3 simplified sampling heights, including the relative position calculation step, the camera parameter solution step, and the calibration accuracy judgment step, omitting the weight allocation sub-step in the calibration quality control strategy.

[0009] Furthermore, the data acquisition step also includes data quality assurance strategies, including: Based on the lifting speed of the platform and the preset sampling height, the focus and exposure parameters of the next sampling of the predictive image acquisition module are adjusted in advance. If a blurred image with a resolution lower than a preset resolution threshold is acquired in the calibration image sequence, the image coordinates of the same feature point are extracted from the calibration image information adjacent to the blurred image before and after it, and their average value is taken as the image coordinates of the feature point in the blurred image.

[0010] Furthermore, the camera parameter solving step is configured with a convergence decision sub-strategy, including... When using an optimization algorithm to solve the camera imaging model, if the decrease in reprojection error after a preset number of iterations is less than a preset rate of change threshold, the algorithm is considered to have converged. If the number of iterations exceeds a preset maximum value and convergence is still not achieved, a fault-tolerant process is triggered, and historical parameters or an adjusted optimization strategy are used to recalculate. Furthermore, this application proposes that the camera parameter solving step also includes an offline model adaptation and online weighted update sub-strategy, including: recording the calibration image sequence, height data, camera parameters, and reprojection errors collected during the transportation process to form a calibration dataset; training an association model of environmental parameters including light intensity, platform vibration amplitude, bearing state, and parameter deviation based on the calibration dataset; adjusting parameter weights according to real-time environmental data during updates, the parameter weights including image feature point weight coefficients, error correction attenuation factors, and iteration convergence thresholds; wherein the image feature point weight coefficients are used to adjust the constraint weights of different feature points in parameter solving, the error correction attenuation factor is used to control the step size attenuation of parameter iteration correction, the initial value of the attenuation factor is 0.8, multiplied by 0.9 after each iteration until it drops to 0.3, the iteration convergence threshold is used to determine whether the parameter solving iteration has terminated, the iteration convergence threshold is that the parameter deviation between two adjacent iterations is <0.1%; when the updated parameters are substituted into the camera parameter solving calculation, the original parameters are directly replaced as the initial values ​​for solving.

[0011] Furthermore, the camera parameter solving step employs a nonlinear optimization algorithm to solve the camera imaging model; the camera imaging model is: , Where (u,v) are the image coordinates of the feature points, S represents the distortion compensation term, K is the camera intrinsic parameter matrix to be solved, [R|t] is the camera extrinsic parameter matrix determined by the robot arm pose and the platform height, and (X,Y,Z) are the world coordinates of the feature points on the positioning marker in the platform coordinate system.

[0012] Furthermore, the optimization objective of the nonlinear optimization algorithm is to minimize the reprojection error function: , Where E is the total reprojection error, and is the measured coordinate of the i-th feature point in the image. (·) represents the function that projects the world point M onto the image plane based on the camera model, K is the intrinsic parameter matrix, and and are the extrinsic parameters of the camera corresponding to the i-th observation position. As can be seen from the above, the collaborative control method and system for track transport and platform centralized control path of the inspection robot provided in this application solves the problem of low visual positioning accuracy in dynamic lifting environments through multi-height data acquisition, dynamic camera parameter calibration, and fault-tolerant mechanisms, and has the advantages of improving positioning accuracy and system reliability. Attached Figure Description

[0013] Figure 1This is a schematic diagram of the overall process of the collaborative control method of the present invention; Figure 2 This is a schematic diagram of the process for calibrating the quality control strategy in this invention; Figure 3 This is a flowchart illustrating the data acquisition and protection strategy in this invention; Figure 4 This is a schematic diagram of the camera parameter solving and convergence determination process in this invention; Figure 5 This is a schematic diagram illustrating the overall process of dynamic updating and fault tolerance applications in this invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] It should be noted that when a component is described as "fixed to" another component, it can be directly on the other component or may have a component in between. When a component is considered "connected to" another component, it can be directly connected to the other component or may have a component in between. When a component is considered "set on" another component, it can be directly set on the other component or may have a component in between. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0016] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0017] Please also see Figures 1 to 5This embodiment provides a method for coordinated control of the conveyor system and platform centralized control path of an inspection robot. This application proposes a method for coordinated control of the conveyor system and platform centralized control path of an inspection robot, including a data acquisition step, a relative position calculation step, a camera parameter solving step, and a calibration accuracy judgment step. In the data acquisition step, the platform stops at multiple uniformly distributed preset heights during its lifting stroke, acquiring a calibration image sequence; the relative position calculation step extracts the coordinates of feature point images and associates them with the platform coordinate system; the camera parameter solving step establishes a distorted pinhole camera model and solves for intrinsic parameters; the calibration accuracy judgment step evaluates the calibration quality through reprojection error and triggers a quality control strategy.

[0018] The positioning markers refer to high-precision calibration plates fixed on the support platform. Their surfaces have an array of feature points with known three-dimensional coordinates, such as a checkerboard calibration plate. Precision machining ensures controllable dimensional errors in the checkerboard units. This feature point array provides an absolute spatial reference for camera parameter calibration, eliminating coordinate deviations caused by mechanical deformation. The calibration image sequence refers to a dataset of images continuously acquired at different heights on the support platform. Multi-height sampling covers the entire lifting and lowering process, capturing the nonlinear characteristics of camera parameters changing with height, providing sufficient observational data for establishing an accurate camera model. The distorted pinhole camera imaging model refers to a projection equation that comprehensively considers radial and tangential distortion. Distortion compensation terms are introduced to correct lens nonlinear errors, such as using a distortion coefficient model containing k1, k2, p1, and p2, making the mapping relationship between image coordinates and three-dimensional spatial coordinates closer to actual physical laws. Reprojection error refers to a quantitative indicator used to verify calibration accuracy through inverse projection. The solved camera parameters are substituted back into the three-dimensional coordinates to calculate the theoretical pixel positions, and compared with the measured pixel positions to form an error assessment, providing an iterative basis for parameter optimization.

[0019] Specifically, the platform stops at its lowest, middle, and highest positions during the lifting and lowering process, and the image acquisition module continuously captures images of the calibration board at each height. A feature extraction algorithm identifies the checkerboard corner points on the calibration board and records their pixel coordinates in the image and the corresponding platform height. Based on multiple sets of height-image coordinate data, a nonlinear least squares method is used to solve for the camera intrinsic parameter matrix and distortion coefficients, constructing a mapping model from the image space to the platform coordinate system. If the average reprojection error exceeds a threshold, image re-acquisition or removal of abnormal feature points is triggered, and parameter calibration accuracy is improved through iterative optimization. A parameter variation model is established through multi-height sampling to achieve real-time calibration in dynamic environments, and calibration robustness is improved through feature point selection and weight allocation. Furthermore, distortion compensation terms correct imaging errors, improving visual positioning accuracy by an order of magnitude. This application effectively solves the positioning inaccuracy problem caused by camera parameter drift during lifting and lowering, ensuring the consistency of visual positioning of the inspection robot at different heights. Through dynamic calibration and error compensation mechanisms, the impact of environmental interference on coordinate mapping is reduced, improving the accuracy of track transport path planning. Based on automated quality control strategies, the need for manual intervention is reduced, and the system's adaptability under complex working conditions is enhanced.

[0020] This application further proposes a calibration quality control strategy, including a feature point screening sub-step to remove low-precision points. Low-precision points are specifically feature points in each calibration image information whose distance from the image edge is less than a preset edge pixel distance threshold, or feature points whose distance from adjacent feature points is less than a preset feature point distance threshold. A frame validity determination sub-step judges the image quality and the number of feature points for each calibration image information. Image sharpness is quantized using the mean of the Laplacian gradient, and the calculation formula is that sharpness equals the sum of the absolute values ​​of the second-order Laplacian operator of the image divided by the total number of effective pixels. Valid feature points are the feature points remaining after removing low-precision points in the feature point screening sub-step. If the preset validity conditions are met, the calibration image information is marked as valid image information; if the preset validity conditions are not met, the calibration image information is marked as invalid image information and a supplementary sampling signal is generated. A weight allocation sub-step assigns weights to each valid image information. The weight calculation method is the ratio of image sharpness to a high-sharpness threshold multiplied by a coefficient, plus the ratio of the number of effective feature points to a high feature point number threshold multiplied by another coefficient. High-weight image information is given higher constraint weights when solving camera parameters.

[0021] Low-precision points refer to feature points in image edge regions or densely populated areas. This can be achieved using edge distance detection algorithms combined with neighboring point spacing calculations, used to eliminate positioning errors caused by image distortion or occlusion. Image sharpness refers to the sharpness of image details, specifically calculated using the Laplacian operator to determine gradient magnitude, used to evaluate the stability and optical quality during image acquisition. Validity conditions refer to the criteria for determining image usability, specifically implemented using a dual-threshold comparison mechanism to ensure that the images used for calibration have sufficient feature points and sharpness. Weight calculation methods refer to the method for allocating the contribution of images of different quality to parameter optimization, specifically implemented using a linear weighted model combined with normalization processing, used to strengthen the influence of high-quality images in parameter solving.

[0022] In the feature point selection stage, feature points located in edge regions or with excessively dense distributions are automatically filtered out by calculating the Euclidean distance between feature points and image boundaries, as well as the distance between adjacent points. In the frame validity determination stage, a dual verification is performed based on the mean of the Laplacian gradient and the number of valid feature points, retaining only image data with high clarity and sufficient feature points. In the weight allocation stage, the weight coefficients of each frame in parameter optimization are dynamically adjusted according to image quality indicators, allowing high-resolution images with multiple feature points to have a greater impact on the camera model solution. Through multi-dimensional quality assessment and dynamic weight allocation, the negative impact of low-quality data on parameter optimization is effectively suppressed, improving the robustness of the calibration process. This application can significantly improve the accuracy of camera calibration parameters, reduce calculation errors caused by image blurring or missing feature points, and enhance the system's adaptability to complex working conditions through adaptive weight adjustment, ensuring the long-term stability of the inspection robot's positioning accuracy.

[0023] This application further proposes configuring a dynamic update sub-strategy in the camera parameter solving step. The environmental parameter mapping model reflects the relationship between the initial parameter values ​​and the environmental parameters, which include light intensity, platform vibration amplitude, and bearing state. The initial parameter values ​​are the initial values ​​of the camera intrinsic parameters and the initial values ​​of the distortion coefficients. After each calibration, the calculation deviation between the current benchmark parameters and the historical best parameters is calculated. The historical best parameters are the parameters that satisfy the minimum average reprojection error among the measured camera intrinsic parameters generated in the past valid calibration cycles stored in the system, and whose subsequent patrol measurement error does not exceed the preset patrol measurement error threshold. If the calculation deviation is less than the preset parameter deviation threshold, the current benchmark parameters and the historical best parameters are weighted and fused to obtain new benchmark parameters. If the calculation deviation is greater than or equal to the preset parameter deviation threshold, the current benchmark parameters replace the historical best parameters.

[0024] The environmental parameter mapping model refers to the correlation model between environmental variables and initial camera parameter values ​​established using historical calibration data. This model can be trained using regression analysis or machine learning algorithms and is used to dynamically adjust the initial values ​​of camera intrinsic parameters based on real-time environmental parameters, thereby improving the efficiency and accuracy of parameter solving. The measurement deviation refers to the degree of difference between the current baseline parameters and the historical best parameters. This can be calculated using Euclidean distance or weighted average error and is used to determine whether the current calibration results meet the stability requirements for long-term system operation. Weighted fusion refers to a calculation method that combines current parameters with historical parameters according to a preset ratio. This can be implemented using linear interpolation or moving average methods. By assigning higher weights to historical parameters, it ensures the adaptability of parameter updates while suppressing interference from short-term environmental fluctuations.

[0025] After each camera parameter calibration, the system automatically compares the current calibration result with the historical best parameters. If the deviation is small, it indicates that the current environmental change is within the range of historical experience. In this case, the current parameters and historical parameters are proportionally fused to generate new benchmark parameters, preserving both environmental adaptability and long-term stability. If the deviation is large, it is determined that the current environment has changed significantly, and the new parameters are directly used to replace the historical best parameters to quickly respond to sudden environmental changes. For example, in scenarios with sudden changes in light intensity or increased vibration of the platform under full load, the system adjusts the parameters in a timely manner through a dynamic update mechanism to avoid the accumulation of subsequent image coordinate transformation errors due to environmental factors. By fusing historical best parameters, abnormal parameter changes are suppressed during environmental fluctuations, and the system quickly switches to new parameters when the environment changes significantly, balancing system robustness and dynamic response capability. This application can effectively solve the problem of unstable camera parameter calibration results caused by environmental changes. Through a dynamic fusion mechanism, it balances short-term accuracy and long-term reliability, reduces measurement errors caused by sudden changes in factors such as light and vibration, and improves the accuracy of coordinate transformation of the inspection robot and the overall continuity of system operation.

[0026] This application further proposes an observation-assisted constraint sub-strategy, including configuring robotic arm action triggering conditions. When the effective feature area of ​​the positioning marker has a field of view coverage of less than a preset threshold in the calibration image information, the robotic arm is triggered to perform action adjustment within a preset adjustment range. The adjustment range includes pitch angle adjustment threshold, yaw angle adjustment threshold, and end effector extension adjustment threshold.

[0027] The field of view coverage refers to the ratio of the pixel area of ​​the effective feature region of the positioning marker in the calibration image information to the total effective feature region pixel area. This can be calculated using an image segmentation algorithm to determine the area ratio, and this parameter quantifies the completeness of feature point acquisition. The robotic arm action trigger condition refers to the logic for automatically initiating robotic arm adjustment when the field of view coverage is insufficient. This can be implemented using real-time image analysis and threshold comparison modules to ensure sufficient feature acquisition. The pitch angle adjustment threshold refers to the range of angles the robotic arm can rotate around its horizontal axis, such as ±5°. This can be achieved by controlling the pitch angle using a joint encoder, and is used to adjust the vertical coverage of the camera's view. The yaw angle adjustment threshold refers to the range of angles the robotic arm can rotate around its vertical axis, such as ±5°. This can be achieved by driving a horizontal rotation mechanism with a servo motor, and is used to adjust the horizontal coverage of the camera's view. The end effector extension adjustment threshold refers to the range of axial movement of the robotic arm's end effector, such as ±50mm. This can be achieved through coordinated control of a linear guide and a displacement sensor, and is used to adjust the relative distance between the camera and the positioning marker.

[0028] During data acquisition, the field-of-view coverage of the positioning marker is calculated in real time to determine whether the current image meets the feature point acquisition requirements. If the field-of-view coverage is lower than a preset threshold, such as 60%, the robotic arm is triggered to perform multi-degree-of-freedom adjustments within a preset range. For example, the vertical field-of-view coverage is expanded by adjusting the pitch angle, the horizontal field-of-view offset is compensated by yaw angle rotation, and the shooting distance is optimized by end-effector extension adjustment. The adjusted robotic arm pose is then used to reacquire images until the field-of-view coverage meets the requirements, thereby ensuring the integrity of subsequent calibration data. Through the robotic arm's automatic adjustment mechanism, the acquisition pose is actively optimized when image quality is insufficient, avoiding feature point loss due to limited viewing angle or improper distance, and improving the reliability of calibration data. This application can dynamically maintain the integrity of image acquisition under complex working conditions, reduce calibration errors caused by field-of-view occlusion or distance deviation, enhance the system's adaptability to interference factors such as mechanical vibration and pose offset, and provide a reliable data foundation for subsequent high-precision coordinate transformation.

[0029] This application further proposes a calibration accuracy judgment step including a fault-tolerant application sub-strategy. This includes a calibration failure condition where, after re-sampling three invalid sampling heights in a single calibration, the average reprojection error is still greater than or equal to a preset reprojection error threshold. If the number of consecutive failures reaches the calibration failure error threshold, the historical optimal parameters are used as a temporary benchmark, and an error signal is output to prompt for checking the integrity of the positioning markers. If an image measurement error is detected in the inspection section that exceeds the inspection measurement error threshold, the inspection is paused, and the system returns to the platform to execute a simplified calibration strategy. The simplified calibration strategy is configured with three preset simplified sampling heights: the lowest, middle, and highest heights of the platform's lifting stroke. The re-execution control method involves only acquiring calibration images at the three simplified sampling heights in the data acquisition step, the relative position calculation step, the camera parameter solution step, and the calibration accuracy judgment step, omitting the weight allocation sub-step in the calibration quality control strategy.

[0030] Among them, calibration failure refers to the state where the accuracy requirements cannot be met even after multiple supplementary acquisitions in a single calibration process. This can be achieved by setting an upper limit on the number of supplementary acquisitions and an error threshold for joint judgment. This definition can accurately identify abnormal situations in the calibration process. Historical optimal parameters refer to the camera parameters with the best performance in the past valid calibration periods stored in the system. This can be achieved by selecting parameter records with the smallest error and stable application from historical data. Its purpose is to provide reliable backup parameters for the system to maintain basic functionality. Simplified calibration strategy refers to a rapid calibration method that reduces the number of sampling points and optimizes the process. This can be achieved by combining key height point sampling with basic calculation steps. This strategy can quickly restore calibration functionality in emergency situations.

[0031] When the calibration process fails error verification multiple times consecutively, the system automatically switches to historically optimal parameters for maintenance and triggers a manual inspection command. During inspection, if measurement errors exceed the limit, the system immediately interrupts the current task and initiates a simplified calibration process, collecting image data only at the lowest, middle, and highest key heights for rapid parameter calculation. This process omits the weight allocation step to improve execution efficiency and quickly generates temporary parameters from the basic model to restore detection capabilities. By combining historical parameter retrieval with simplified calibration, the system significantly shortens fault recovery time while ensuring basic accuracy.

[0032] This application can maintain the basic operation of the system when calibration fails, avoiding a complete shutdown due to calibration problems; it can quickly respond to measurement error anomalies during inspection, and achieve rapid parameter calibration through streamlined processes, effectively reducing operation interruption time and ensuring the continuity of testing.

[0033] The data quality assurance strategy is a core auxiliary mechanism in the data acquisition process. It aims to solve the problems of image blurring and feature point loss caused by changes in motion state, such as speed fluctuations, and environmental interference, such as changes in lighting, during the lifting and lowering of the platform. It ensures that the acquired calibration image sequence meets the accuracy requirements of subsequent camera parameter solving, thus laying a data foundation for high-precision positioning.

[0034] During the lifting and lowering of the platform, the relative distance and relative speed between the image acquisition module and the positioning markers, such as the checkerboard calibration plate, change with height. If fixed parameters are used for image acquisition, problems such as inaccurate focus, overexposure, or underexposure can easily occur. This strategy employs a motion state prediction-based parameter pre-adjustment mechanism to proactively adapt to the environment and motion state before each sampling. The specific steps are as follows: The system acquires the lifting speed of the platform in real time, based on feedback from the encoder of the platform drive mechanism and the current sampling height; Based on the preset sampling height plan, if the lifting stroke of the bearing platform is evenly distributed with N≥3 sampling points, calculate the distance difference between the next sampling height and the current height and the estimated arrival time; Based on the distance difference, the relative distance between the image acquisition module and the positioning marker is predicted. Combined with the rise and fall speed, the risk of motion blur is predicted. The focusing parameters of the image acquisition module, such as lens focal length, focusing motor position, and exposure parameters, such as shutter speed and ISO sensitivity, are automatically adjusted. Once the platform reaches the next preset sampling height and stabilizes, it immediately acquires a calibration image using the pre-adjusted parameters to avoid motion blur or parameter adaptation delay.

[0035] If sudden interference, such as instantaneous vibration or sudden change in illumination, causes the clarity of the acquired calibration image to fall below a preset threshold (i.e., become a blurred image), directly discarding it would reduce the amount of sampled data and affect the accuracy of parameter solving. This strategy uses an adjacent frame data completion mechanism to recover the feature point coordinates from the effective images before and after the blurred image. The specific steps are as follows: After acquiring and calibrating the image sequence, the sharpness of each frame is calculated using the Laplacian gradient mean method: Sharpness = ∑|▽ 2 I(x,y)| / M, where ▽ 2 I(x,y) is the second-order Laplacian operator for pixel (x,y), and M is the total number of valid pixels; The calculated sharpness is compared with a preset sharpness threshold. If it is lower than the threshold, it is determined to be a blurry image. Extract 2-3 valid frames of images adjacent to the blurred image. It is necessary to ensure that the adjacent frames are acquired near the same preset sampling height and that the position of the feature points changes very little. Identify the same feature points in adjacent valid images as those in the blurred image, and extract the image coordinates of these feature points using feature point matching algorithms such as SIFT and ORB. The arithmetic mean of the coordinates of the same feature point in adjacent valid images is used as the complete coordinates of that feature point in the blurred image, ensuring the integrity of the calibration dataset.

[0036] This invention further proposes a convergence determination sub-strategy, which is the core termination condition of the camera parameter solution step. It is used to determine whether the nonlinear optimization algorithm has found the optimal camera intrinsic parameters, such as focal length and principal point coordinates, to avoid invalid iteration waves or non-convergence. At the same time, it ensures the continuity of the system through a fault-tolerant process.

[0037] Solving for camera parameters requires minimizing the reprojection error through an optimization algorithm. During the iteration process, it's necessary to dynamically determine whether convergence has occurred. The specific steps are as follows: Initialize the camera intrinsic parameters based on the environmental parameter mapping model, refer to historical data, and substitute them into a nonlinear optimization algorithm; After each iteration, the decrease in reprojection error is calculated. The decrease in reprojection error is equal to the current iteration error minus the previous iteration error. If the error decrease within a preset number of consecutive iterations is less than a preset rate of change threshold, the algorithm is considered to have converged, and the current camera intrinsic parameters are output. If the cumulative number of iterations reaches the preset maximum value, but the error reduction is still greater than or equal to the preset rate of change threshold, i.e., convergence has not occurred, then the fault tolerance process is triggered: Prioritize using the historical best parameters as temporary camera intrinsic parameters, which are the parameters with the smallest average reprojection error and whose survey measurement error does not exceed the standard among the 10 valid calibrations stored in the system. If the historical best parameters are unavailable, such as during the initial calibration, adjust the optimization strategy, such as increasing the iteration step size, updating the initial values ​​to the default parameters, and re-execute the optimization calculation.

[0038] This application further proposes that the camera parameter solving steps also include offline model adaptation and online weighted update sub-strategies, including: recording the calibration image sequence, height data, camera parameters and reprojection errors collected during the transportation process to form a calibration dataset; training the correlation model between environmental parameters and parameter deviations based on the calibration dataset, and adjusting the parameter weights according to real-time environmental data during updates; and directly replacing the original parameters as the initial values ​​for solving when the updated parameters are substituted into the camera parameter solving calculation.

[0039] The calibration dataset refers to a collection of calibration images acquired by the image acquisition module at different heights and under different environmental conditions, along with their corresponding height information, camera parameters, and error data. This can be implemented using a database storage structure and is used to establish the correlation between environmental parameters and camera parameter deviations. The correlation model between environmental parameters and parameter deviations is a mathematical model established using machine learning algorithms that reflects the mapping relationship between light intensity, vibration amplitude, load condition, and camera intrinsic parameter deviations. This can be implemented using linear regression or neural network algorithms and is used to predict parameter correction amounts under different environmental conditions. Parameter weights include image feature point weight coefficients, error correction attenuation factors, and iterative convergence thresholds. These can be implemented through a dynamic adjustment algorithm. The initial value of the error correction attenuation factor is set to 0.8 and decays successively at a rate of 0.9, used to control the convergence speed of parameter iteration.

[0040] Specifically, during the transport process, calibration image sequences and corresponding height data are continuously collected, and a calibration dataset is constructed by combining these with historical calibration results. Based on this dataset, an association model is trained to analyze the impact of environmental parameter changes on camera intrinsic parameters. When real-time environmental data is input, the model outputs parameter weight adjustment values; for example, when the light intensity is below a threshold, the weight coefficients of image feature points are increased to enhance the stability of feature recognition under low-light conditions. The updated parameters are used as initial values ​​in the camera parameter solution process, and parameter calibration is completed through a weighted iterative optimization algorithm. By combining offline training with online updates, a dynamic association mechanism between environmental parameters and weight adjustments is established, enabling the parameter solution process to adapt to different operating conditions and effectively suppress the impact of environmental interference on camera calibration.

[0041] This application achieves environmental adaptability in the camera parameter solving process, solving the problem of unstable calibration accuracy under complex working conditions in traditional methods. By dynamically adjusting the weight coefficients and iterative convergence conditions, the efficiency of parameter optimization is improved, the risk of calibration failure due to sudden environmental changes is reduced, and a reliable visual positioning foundation is provided for inspection robots.

[0042] This invention further proposes that the camera parameter solving step employs a nonlinear optimization algorithm to solve the camera imaging model; the camera imaging model is: , Where (u,v) are the image coordinates of the feature points, S represents the distortion compensation term, K is the camera intrinsic parameter matrix to be solved, [R|t] is the camera extrinsic parameter matrix determined by the robot arm pose and the platform height, and (X,Y,Z) are the world coordinates of the feature points on the positioning marker in the platform coordinate system.

[0043] The camera imaging model serves as a mathematical bridge connecting the world coordinates of feature points with the image coordinates. It describes how the feature points on the positioning marker, i.e., their 3D world coordinates, are projected through the camera's optical system into the 2D image coordinates of pixels in the image. This method employs a pinhole camera model with distortion compensation to correct lens optical distortions, such as radial and tangential distortions, ensuring accurate coordinate mapping. The optimization objective of the nonlinear optimization algorithm is to minimize the reprojection error function: , Where E is the total reprojection error. Let be the measured coordinates of the i-th feature point in the image. (·) is a function that projects world point M onto the image plane based on the camera model, and K is the intrinsic parameter matrix. and Let be the camera extrinsic parameters corresponding to the i-th observation position.

[0044] The core objective of nonlinear optimization algorithms is to minimize the reprojection error, that is, to minimize the deviation between the measured image coordinates of feature points and the predicted image coordinates of the model by adjusting the camera intrinsic parameters K and extrinsic parameters [R|t], thereby solving for the optimal camera parameters.

[0045] Objective function decomposition and optimization logic: The optimization logic is based on error backpropagation: by calculating the reprojection error of each feature point, the camera parameters are adjusted in reverse until the total error E reaches its minimum value. The specific decomposition is as follows: Error source: The reprojection error originates from the deviation between the measured coordinates and the predicted coordinates. The smaller the deviation, the closer the camera parameters are to the actual optical and motion state; Optimization process: A nonlinear optimization algorithm is used to iteratively adjust K. and After each iteration, the predicted coordinates of each feature point are calculated using the camera imaging model. Calculate the Euclidean distance between the measured coordinates and the predicted coordinates, and sum them to obtain the total reprojection error E; based on the gradient direction of the error E, adjust the parameters, such as increasing / decreasing the focal length and fine-tuning the rotation matrix. Termination condition: Stop iteration and output the optimal parameters when E reaches its minimum value.

[0046] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for coordinated control of track transport and platform centralized control path of an inspection robot, wherein the inspection robot is equipped with a robotic arm and an image acquisition module, characterized in that, The support platform is pre-set with positioning markers, and the control method includes: In the data acquisition step, when the inspection robot rises and falls with the carrier platform, the carrier platform stops at several preset sampling heights at intervals. At each preset sampling height, the image acquisition module acquires a calibration image sequence, which represents the continuously acquired calibration image information. The relative position calculation step involves performing feature extraction processing on each of the calibration image information to identify feature points on the positioning marker, obtaining image coordinates and corresponding preset sampling heights to construct a camera imaging dataset, wherein the image coordinates reflect the coordinates of the calibrated feature points in the preset carrier platform coordinate system; The camera parameter solving steps involve establishing a camera imaging model based on the camera imaging dataset, solving the camera imaging model, and outputting the calculated camera intrinsic parameters. The calibration accuracy judgment step involves calculating the reprojection error of feature points at all preset sampling heights. Specifically, the reprojection error is the difference between the measured coordinates of the image and the predicted coordinates of the parameters. If the mean of the reprojection error is less than a preset reprojection error threshold, it is considered to meet the preset accuracy requirements, and the currently calculated camera intrinsic parameters are used as reference parameters. These reference parameters are used for image correction during the inspection phase. If the mean of the reprojection error is greater than or equal to the preset reprojection error threshold, a preset calibration quality control strategy is triggered, and the camera parameter solving step is re-executed.

2. The method for coordinated control of inspection robot track transport and platform centralized control path according to claim 1, characterized in that, The calibration quality control strategy includes: The feature point filtering sub-step removes low-precision points. Specifically, each low-precision point is a feature point in the calibrated image information whose distance from the image edge is less than a preset edge pixel distance threshold, or a feature point whose distance from its adjacent feature points is less than a preset feature point distance threshold. The frame validity determination sub-step assesses the image quality and feature point count of each calibrated image information. If the image meets preset validity conditions, it is marked as valid. If it does not meet the preset validity conditions, it is marked as invalid and a supplementary sampling signal is generated. Specifically, the validity conditions are that the image sharpness is greater than or equal to a preset image sharpness threshold and the number of valid feature points is greater than or equal to a preset valid feature point count threshold. The weight allocation sub-step assigns weights to each valid image information. The higher the image clarity and the more valid feature points a valid image information has, the higher its weight.

3. The method for coordinated control of inspection robot track transport and platform centralized control path according to claim 2, characterized in that, The camera parameter solving step also includes dynamically updating a sub-strategy, configured with a preset environment parameter mapping model, which reflects the relationship between the initial parameter values ​​and the environment parameters. After each calibration, the measurement deviation is calculated. Specifically, the measurement deviation is the deviation between the current benchmark parameter and the historical best parameter. If the measurement deviation is less than the preset parameter deviation threshold, the current benchmark parameter and the historical best parameter are included in the historical dataset, and a new set of optimal parameters is recalculated based on the complete historical dataset as the new benchmark parameter. If the measurement deviation is greater than or equal to the preset parameter deviation threshold, the historical best parameter is replaced with the current benchmark parameter.

4. The method for coordinated control of inspection robot track transport and platform centralized control path according to claim 1, characterized in that, The data acquisition step also includes an observation-assisted constraint sub-strategy, including: The system is equipped with robotic arm action triggering conditions. The field of view coverage rate is defined as the ratio of the pixel area of ​​the effective feature area of ​​the positioning marker in the calibration image information to the pixel area of ​​the total effective feature area of ​​the positioning marker. When the field of view coverage rate of the positioning marker in the calibration image information is less than the preset field of view coverage rate threshold, the robotic arm is triggered to perform action adjustment within the preset robotic arm adjustment range. The preset robotic arm adjustment range includes preset pitch angle adjustment threshold, yaw angle adjustment threshold, and end effector extension adjustment threshold.

5. The method for coordinated control of inspection robot track transport and platform centralized control path according to claim 1, characterized in that, The calibration accuracy determination step also includes a fault-tolerant application sub-strategy, including: If the number of consecutive failures reaches the preset calibration failure error threshold, the historical best parameters are used as a temporary benchmark and an error signal is output to prompt the inspection of the integrity of the positioning marker. If an image measurement error is detected in the inspection section that is greater than the preset inspection measurement error threshold, the inspection is paused and the system returns to the carrier platform to execute a simplified calibration strategy. The simplified calibration strategy is configured with a preset simplified sampling height, and the control method is re-executed at the simplified preset sampling height.

6. The method for coordinated control of inspection robot track transport and platform centralized control path according to claim 1, characterized in that, The data acquisition steps also include data quality assurance strategies, including: Based on the lifting speed of the platform and the preset sampling height, the focus and exposure parameters of the next sampling of the predictive image acquisition module are adjusted in advance. If a blurred image with a resolution lower than a preset resolution threshold is acquired in the calibration image sequence, the image coordinates of the same feature point are extracted from the calibration image information of the adjacent images before and after the blurred image, and their average value is taken as the image coordinates of the feature point in the blurred image.

7. The method for coordinated control of inspection robot track transport and platform centralized control path according to claim 1, characterized in that, The camera parameter solving step is configured with a convergence decision sub-strategy, including: When using an optimization algorithm to solve the camera imaging model, if the decrease in reprojection error after a preset number of iterations is less than a preset rate of change threshold, the algorithm is considered to have converged. If the number of iterations exceeds a preset maximum value and convergence is still not achieved, a fault-tolerant process is triggered, and historical parameters or an adjusted optimization strategy are used to recalculate.

8. The method for coordinated control of inspection robot track transport and platform centralized control path according to claim 3, characterized in that, The camera parameter solving step also includes offline model adaptation and online weighted update sub-strategies, including: The calibration dataset is formed by recording the calibration image sequence, height data, camera parameters, and reprojection errors collected during the transportation process. The model is trained based on the calibration dataset to correlate environmental factors with parameter deviations. During the update, the parameter weights are adjusted according to real-time environmental data. The parameter weights include image feature point weight coefficients, error correction attenuation factors, and iterative convergence thresholds. The original parameters are then substituted into the camera parameter calculation.

9. The method for coordinated control of inspection robot track transport and platform centralized control path according to claim 1, characterized in that, The camera parameter solving step employs a nonlinear optimization algorithm to solve the camera imaging model; the camera imaging model is as follows: ， Where (u,v) are the image coordinates of the feature points, S represents the distortion compensation term, K is the camera intrinsic parameter matrix to be solved, [R|t] is the camera extrinsic parameter matrix determined by the robot arm pose and the platform height, and (X,Y,Z) are the world coordinates of the feature points on the positioning marker in the platform coordinate system.

10. The method for coordinated control of inspection robot track transport and platform centralized control path according to claim 9, characterized in that, The optimization objective of the nonlinear optimization algorithm is to minimize the reprojection error function: ， Where E is the total reprojection error. Let be the measured coordinates of the i-th feature point in the image. (·) is a function that projects world point M onto the image plane based on the camera model, and K is the intrinsic parameter matrix. and Let be the camera extrinsic parameters corresponding to the i-th observation position.

Images