Patrol robot station track transfer and platform centralized control path cooperative control method

Abstract

The invention relates to the technical field of rail traffic inspection, and discloses an inspection robot station track transfer and platform centralized control path cooperative control method, which is characterized by comprising a data acquisition step, a relative position calculation step, a camera parameter solving step and a calibration precision judgment step. Through multi-height data acquisition, dynamic camera parameter calibration and a fault-tolerant mechanism, the problem of low visual positioning precision in a dynamic lifting environment is solved, and the method has the advantages of improving positioning precision and system reliability.

Description

TECHNICAL FIELD

[0001] The present application relates to the rail transit inspection technical field, more particularly to a kind of inspection robot track transfer and platform centralized control path coordination control method. BACKGROUND

[0002] The subway track is adapted to the train driving demand, and the track plane is usually lower than the station platform or the external ground, forming a relatively closed recess area. The inspection robot is applied in the inspection system, and in addition to the need for detection on the side of the train, it also needs to enter the track to detect the bottom of the train. Therefore, the inspection robot needs to move back and forth between the external charging, train side inspection area and track inspection area. Due to the height difference between the track and the outside, the inspection robot cannot directly enter the track by walking on the plane, and a lifting platform or a slope transfer structure is usually set at the connection position of the track and the outside. The lifting platform includes a load-bearing platform that can be raised and lowered and a lifting drive mechanism. The inspection robot drives onto the load-bearing platform, and the lifting drive mechanism drives the load-bearing platform to descend to the same level as the track plane, and the inspection robot can drive into the track from the load-bearing platform. The slope transfer structure includes an inclined ramp, one end of which is connected with the external ground, and the other end of which extends to the track. The inspection robot can enter the track through the ramp. During inspection, the image detection device carried on the inspection robot is used to collect images. However, in actual use, there is a height difference between the subway track and the external ground, and the inspection robot needs to use the lifting platform or the slope structure to move between areas. The traditional positioning method relies on mechanical encoders or inertial navigation, but mechanical vibration, light changes and deformation of the load-bearing platform during lifting can easily cause parameter drift of the vision sensor, resulting in misalignment of the mapping of image coordinates and physical space coordinates, affecting the accuracy of the inspection path planning. SUMMARY

[0003] In view of the deficiencies in the prior art, the present application provides an inspection robot track transfer and platform centralized control path coordination control method, which realizes high-precision positioning and pose correction of the inspection robot on the lifting platform through multi-modal data acquisition, error decomposition and closed-loop calibration, dynamic weight self-adaptive adjustment and other strategies, improves the positioning reliability and path planning accuracy during transfer, and optimizes the coordination efficiency of the mechanical arm and the image acquisition module.

[0004] To achieve the above purpose, the present application provides the following technical solutions: The inspection robot is provided with a mechanical arm and an image acquisition module, and the carrying table is a track transfer carrier of the inspection robot, characterized in that a positioning marker is prearranged on the carrying table, the positioning marker is a high-precision calibration plate with known three-dimensional coordinates, such as a chessboard calibration plate, the chessboard unit size precision is less than or equal to 0.01 mm, and the three-dimensional coordinates of the characteristic points in the carrying table coordinate system are pre-calibrated, and the control method comprises: a data acquisition step, when the inspection robot is raised and lowered with the carrying table, the carrying table is stopped at intervals to N preset sampling heights, N is greater than or equal to 3, and is uniformly distributed along the lifting stroke of the carrying table to cover the whole lifting stroke, and the image acquisition module acquires a calibration image sequence at each preset sampling height, and the calibration image sequence represents the calibration image information acquired continuously; a relative position calculation step, feature extraction processing is performed on each calibration image information to identify the characteristic points on the positioning marker, and the image coordinates and the corresponding sampling height are acquired to construct a camera imaging dataset, and the image coordinates reflect the coordinates of the calibrated characteristic points in the preset carrying table coordinate system, that is, the three-dimensional coordinates (X, Y, 0) of the characteristic points at the carrying table plane Z=0 correspond to the image pixel coordinates (u, v); a camera parameter solving step, a pinhole camera imaging model with distortion is established based on the camera imaging dataset, and the model expression is: S, wherein K is a camera intrinsic matrix, [R|t] is an extrinsic parameter of the camera relative to the carrying table, the distortion compensation term contains radial distortion coefficients k1 and k2 and tangential distortion coefficients p1 and p2, and h is the corresponding sampling height, the camera imaging model is solved and a measured camera intrinsic parameter is output, the measured camera intrinsic parameter includes focal lengths f x , fᵧ, principal point coordinates u0 and v0, and radial distortion coefficients k1, k2, tangential distortion coefficients p1 and p2; a calibration accuracy judgment step, the re-projection error of the characteristic points at all sampling heights is calculated, the re-projection error is specifically the difference between the image measured coordinates and the parameter predicted coordinates, if the average value of the re-projection error is less than a preset re-projection error threshold, it is considered that the preset accuracy requirement is met, the current measured camera intrinsic parameter is taken as a reference parameter, and the reference parameter is used for image distortion correction in the inspection stage and conversion of the image coordinates and the carrying table coordinate system; if the average value of the re-projection error is greater than or equal to the preset re-projection error threshold, a preset calibration quality control strategy is triggered, the camera parameter solving step is re-executed, and the calibration quality control strategy at least includes resampling calibration images at invalid sampling heights and eliminating low-precision characteristic points.

[0005] Further, the application further provides a calibration quality control strategy, which comprises: a feature point screening sub-step of eliminating low-precision points, the low-precision points being specifically feature points in each calibration image information with a distance to an image edge less than a preset edge pixel distance threshold or feature points with a distance to adjacent feature points less than a preset feature point distance threshold; a frame validity determination sub-step of determining image quality and feature point quantity of each calibration image information, the image sharpness being quantified by using a Laplacian gradient mean value, and a calculation formula being: wherein ∇ 2 I(x, y) is a second-order Laplacian operator of the image I at a pixel (x, y), M is a total number of effective pixels of the image, and the effective feature points are the feature points remaining after the low-precision points are eliminated in the feature point screening sub-step; if a preset effective condition is met, the effective condition being specifically that the image sharpness is greater than or equal to a preset image sharpness threshold and the number of effective feature points is greater than or equal to a preset effective feature point number threshold, the calibration image information is marked as valid image information, and if the preset effective condition is not met, the calibration image information is marked as invalid image information and a supplementary sampling signal is generated; a weight distribution sub-step of distributing weights to each valid image information, the weight calculation mode being: wherein α and β are weight coefficients, α+β=1 and α and β are both 0.5; the higher the image sharpness and the more the number of effective feature points, the higher the weight of the valid image information, and the high-weight image information is given a higher constraint weight in solving the camera parameters, that is, the feature point re-projection error weight of the image information in the solving process is enlarged by W times.

[0006] Further, the application further proposes that the camera parameter solving step further comprises a dynamic updating sub-strategy, which is configured with a preset environment parameter mapping model reflecting the relationship between a parameter initial value and environment parameters, wherein the environment parameters include light intensity (unit: lux), bearing table vibration amplitude (unit: mm), and bearing state (empty / full), the parameter initial value is a camera intrinsic parameter initial value, and the camera intrinsic parameter initial value includes an initial focal length, an initial principal point coordinate, and a distortion coefficient initial value; after each calibration, a calculation deviation of a current reference parameter from a historical optimal parameter is calculated, the historical optimal parameter is a parameter in camera intrinsic parameters calculated in the past 10 effective calibration periods and stored by the system, which satisfies the minimum average re-projection error and does not exceed a preset patrol detection error threshold in subsequent application; if the newly generated current reference parameter satisfies the average re-projection error < the average re-projection error of the historical optimal parameter, and no patrol detection error exceeds the threshold in a subsequent application of one transfer cycle, the historical optimal parameter is updated to the new parameter; if the calculation deviation is less than a preset parameter deviation threshold, a new reference parameter is obtained by weighted fusion of the current reference parameter and the historical optimal parameter, the fusion formula is: new reference parameter = current reference parameter * 0.3 + historical optimal parameter * 0.7, wherein the weight proportion of the current reference parameter is 30%, and the weight proportion of the historical optimal parameter is 70%; and if the calculation deviation is greater than or equal to the preset parameter deviation threshold, the current reference parameter is used to replace the historical optimal parameter.

[0007] Further, the application further proposes that the data acquisition step further comprises an observation auxiliary constraint sub-strategy, which comprises: being configured with a mechanical arm action triggering condition, and a field of view coverage rate being a ratio of a pixel area of an effective feature area of a positioning marker in the calibration image information to a total effective feature area pixel area of the positioning marker; when the field of view coverage rate of the positioning marker in the calibration image information is less than a preset field of view coverage rate threshold, triggering the mechanical arm to perform action adjustment within a preset mechanical arm adjustment range, and the preset mechanical arm adjustment range includes a preset pitch angle adjustment threshold, a preset yaw angle adjustment threshold, and a preset end stretching amount adjustment threshold.

[0008] Further, the application also proposes that the calibration accuracy judgment step further includes a fault-tolerant application sub-strategy, including: in a single calibration failure, after 3 invalid sampling height calibration images are additionally sampled, the average re-projection error is still ≥ the preset re-projection error threshold; if the number of consecutive failures reaches the preset calibration failure error threshold, the historical optimal parameters are called as a temporary reference and an error signal is output to prompt the integrity of the positioning identification member; if the image measurement error detected in the inspection section is greater than the preset inspection measurement error threshold, the inspection is suspended and the bearing table is returned to execute a simplified calibration strategy, the simplified calibration strategy is configured with 3 preset simplified sampling heights, which are the lowest height, the middle height, and the highest height of the lifting stroke of the bearing table; the re-executed control method is the data collection step in claim 1, only collecting calibration images at 3 simplified sampling heights, including the relative position calculation step, the camera parameter solving step, the calibration accuracy judgment step, and omitting the weight distribution sub-step in the calibration quality control strategy.

[0009] Further, the data collection step further includes a data quality guarantee strategy, including: According to the lifting speed of the bearing table and the preset sampling height, the focus parameter and the exposure parameter of the next sampling of the predicted image collection module are adjusted in advance. If a blurred image with image clarity lower than the preset clarity threshold is collected in the calibration image sequence, the image coordinates of the same feature point are extracted from the adjacent calibration image information before and after the blurred image, and the average value is taken as the image coordinates of the feature point in the blurred image.

[0010] Further, the camera parameter solving step is configured with a convergence judgment sub-strategy, including When the camera imaging model is solved using an optimization algorithm, if the amount of decrease in the re-projection error is less than a preset change rate threshold after a preset number of consecutive iterations, it is determined that the algorithm converges; if the number of iterations exceeds a preset maximum value and still does not converge, a fault-tolerant process is triggered, and historical parameters or an adjustment optimization strategy is used to recalculate. Further, the application also provides that the camera parameter solving step further includes an offline model self-adaption and online weighted update sub-strategy, which includes: recording the calibration image sequence, height data, camera parameters and re-projection error formed during the transfer process to form a calibration data set; training an environmental parameter correlation model based on the calibration data set, including light intensity, carrier table vibration amplitude, bearing state and parameter deviation, updating the parameter weight according to real-time environmental data, the parameter weight including an image feature point weight coefficient, an error correction decay factor and an iteration convergence threshold; wherein the image feature point weight coefficient is used to adjust the constraint weight of different feature points in parameter solving, the error correction decay factor is used to control the step decay of parameter iteration correction, the initial value of the decay factor is 0.8, multiplied by 0.9 after each iteration, until it decreases to 0.3, and the iteration convergence threshold is used to determine whether the parameter solving iteration is terminated, and the iteration convergence threshold is that the parameter deviation of adjacent two iterations is less than 0.1%; when the updated parameters are substituted into the camera parameter solving calculation, the original parameters are directly replaced as the initial value of the solution.

[0011] Further, the camera parameter solving step uses a nonlinear optimization algorithm to solve the camera imaging model; the camera imaging model is: , wherein (u, v) is the image coordinate of the feature point, S represents the distortion compensation term, K is the camera intrinsic parameter matrix to be solved, [R|t] is the camera extrinsic parameter matrix determined by the mechanical arm pose and the height of the carrier table, (X, Y, Z) is the world coordinate of the feature point on the positioning marker in the carrier table coordinate system.

[0012] Further, the optimization objective of the nonlinear optimization algorithm is to minimize the re-projection error function: , wherein E is the total re-projection error, is the measured coordinate of the i-th feature point in the image, is a function of projecting the world point M to the image plane according to the camera model, K is the intrinsic parameter matrix, and is the camera extrinsic parameter corresponding to the i-th observation position. As can be seen from the above, the track transfer and platform centralized control path coordination control method and system provided by the application solve the problem of low visual positioning accuracy in a dynamic lifting environment through multi-height data acquisition, dynamic camera parameter calibration and fault-tolerant mechanism, and have the advantages of improving positioning accuracy and system reliability. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1is the overall flowchart of the cooperative control method of the present application; Figure 2 is the flowchart of the calibration quality control strategy in the present application; Figure 3 is the flowchart of the data acquisition and security strategy in the present application; Figure 4 is the flowchart of the camera parameter solving and convergence determination in the present application; Figure 5 is the flowchart of the dynamic updating and fault-tolerant application in the present application. DETAILED DESCRIPTION

[0014] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments of the present application, all the other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0015] It should be noted that when a component is referred to as being "fixed" to another component, it can be directly on the other component or there can be intervening components. When a component is referred to as being "connected" to another component, it can be directly connected to the other component or there can be intervening components. When a component is referred to as being "disposed" on another component, it can be directly on the other component or there can be intervening components. The terms "vertical", "horizontal", "left", "right", and the like as used herein are for purposes of illustration only.

[0016] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0017] Please refer to Figures 1 to 5The embodiment provides a path coordination control method for track transfer and platform centralized control of an inspection robot. The application provides a path coordination control method for track transfer and platform centralized control of an inspection robot, which comprises a data acquisition step, a relative position calculation step, a camera parameter solving step and a calibration accuracy judgment step. In the data acquisition step, the bearing table is stopped at a plurality of uniformly distributed preset heights in the lifting stroke, and a calibration image sequence is acquired; in the relative position calculation step, feature point image coordinates are extracted and associated with the bearing table coordinate system; in the camera parameter solving step, a pinhole camera model with distortion is established to solve the intrinsic parameters; and in the calibration accuracy judgment step, the calibration quality is evaluated by a re-projection error and a quality control strategy is triggered.

[0018] The positioning marker refers to a high-precision calibration plate fixed on the bearing table, and the surface of the calibration plate has a feature point array with known three-dimensional coordinates, for example, a chessboard calibration plate, which is processed precisely to ensure that the chessboard unit size error is controllable. The feature point array provides an absolute spatial reference datum for camera parameter calibration and eliminates the coordinate deviation caused by mechanical structure deformation. The calibration image sequence refers to a set of image data collected continuously at different heights of the bearing table, which covers the whole lifting process through multi-height sampling and captures the nonlinear characteristics of the camera parameters changing with the height, thereby providing sufficient observation data for establishing an accurate camera model. The pinhole camera imaging model with distortion refers to a projection equation that comprehensively considers radial distortion and tangential distortion, which corrects the nonlinear error of the lens by introducing a distortion compensation term, for example, a distortion coefficient model containing k1, k2, p1 and p2, so that the mapping relationship between the image coordinates and the three-dimensional space coordinates is closer to the actual physical law. The re-projection error refers to a quantitative index for verifying the calibration accuracy through inverse projection, which substitutes the solved camera parameters into the three-dimensional coordinate calculation to calculate the theoretical pixel position, compares it with the measured pixel position, and forms an error evaluation to provide an iteration basis for parameter optimization.

[0019] Specifically, the bearing table stops at the lowest, middle and highest positions during the lifting process, and the image acquisition module continuously captures the calibration plate image at each height. The feature extraction algorithm identifies the corner points of the checkerboard on the calibration plate, records their pixel coordinates in the image and the corresponding bearing table height. Based on multiple sets of height-image coordinate data, a nonlinear least squares method is used to solve the camera intrinsic matrix and distortion coefficient, and a mapping model from the image space to the bearing table coordinate system is constructed. If the average re-projection error exceeds the threshold, trigger the image acquisition or remove abnormal feature points, and improve the parameter calibration accuracy through iterative optimization. Through multi-height sampling to establish a parameter variation model, real-time calibration in a dynamic environment is realized, and the calibration robustness is improved through feature point screening and weight allocation. In addition, the imaging error is corrected by the distortion compensation term, which improves the visual positioning accuracy by an order of magnitude. The application effectively solves the positioning misalignment problem caused by camera parameter drift during the lifting process, ensuring the visual positioning consistency of the inspection robot at different heights. Through the dynamic calibration and error compensation mechanism, the influence of environmental interference on coordinate mapping is reduced, and the accuracy of the track transfer path planning is improved. Based on the automatic quality control strategy, the need for manual intervention is reduced, and the adaptive ability of the system in complex working conditions is enhanced.

[0020] The application further proposes a calibration quality control strategy, including a feature point screening sub-step, which removes low-precision points, specifically, feature points in each calibration image information that are less than a preset edge pixel distance threshold from the image edge, or feature points with a distance less than a preset feature point distance threshold from adjacent feature points; a frame validity determination sub-step, which determines the image quality and the number of feature points of each calibration image information, quantifies the image sharpness using the Laplacian gradient mean, and the calculation formula is sharpness equals the sum of the absolute values of the second-order Laplacian operator divided by the total number of effective pixels, and the effective feature points are the remaining feature points after the low-precision points are removed by the feature point screening sub-step; if the preset effective condition is met, the calibration image information is marked as valid image information, and if the preset effective condition is not met, the calibration image information is marked as invalid image information and a supplementary sampling signal is generated; a weight allocation sub-step, which allocates weights to each valid image information, the weight calculation method is the ratio of image sharpness to high sharpness threshold multiplied by a coefficient, plus the ratio of the number of effective feature points to the number of high feature points threshold multiplied by another coefficient, and high-weight image information is given higher constraint weight when solving the camera parameters.

[0021] The low-precision point refers to a feature point in an image edge region or a dense region, and can be specifically implemented by using an edge distance detection algorithm combined with a neighboring point spacing calculation, and is used to exclude positioning error points caused by image distortion or occlusion. The image definition refers to the sharpness of image details, and can be specifically implemented by using a Laplacian operator to calculate a gradient amplitude, and is used to evaluate the stability and optical quality during image acquisition. The effective condition refers to a judgment standard for image availability, and can be specifically implemented by using a double-threshold comparison mechanism, and is used to ensure that the images participating in the calibration have sufficient feature point quantity and definition. The weight calculation mode refers to a contribution degree distribution method of different quality images to parameter optimization, and can be specifically implemented by using a linear weighting model combined with normalization processing, and is used to strengthen the influence of high-quality images in parameter solving.

[0022] In the feature point screening stage, the feature points in the edge region or the dense distribution are automatically filtered by calculating the Euclidean distance of the feature points and the image boundary and the spacing between the adjacent points. In the frame validity determination stage, the double check is carried out based on the Laplacian gradient mean value and the effective feature point quantity, and only the image data with high definition and sufficient feature points are reserved. In the weight allocation stage, the weight coefficients of each frame in the parameter optimization are dynamically adjusted according to the image quality index, so that the images with high definition and many feature points have greater influence on the camera model solving. Through multi-dimensional quality evaluation and dynamic weight allocation, the negative influence of low-quality data on parameter optimization is effectively suppressed, and the robustness of the calibration process is improved. The present application can significantly improve the accuracy of the camera calibration parameters, reduce the calculation error caused by image blur or feature point loss, and at the same time, through adaptive weight adjustment, the adaptability of the system to complex working conditions is enhanced, and the long-term stability of the positioning accuracy of the inspection robot is ensured.

[0023] The present application further proposes to configure a dynamic update sub-strategy in the camera parameter solving step, and the environment parameter mapping model reflects the relationship between the parameter initial value and the environment parameter, the environment parameter includes the light intensity, the vibration amplitude of the bearing table, and the bearing state, and the parameter initial value is the camera intrinsic parameter initial value and the distortion coefficient initial value; after each calibration, the calculation deviation of the current reference parameter and the historical optimal parameter is calculated, the historical optimal parameter is the parameter in the calculated camera intrinsic parameter generated by the system in the past effective calibration period, which satisfies the minimum average re-projection error and the subsequent application error does not exceed the preset patrol detection error threshold; if the calculation deviation is less than the preset parameter deviation threshold, the current reference parameter and the historical optimal parameter are weighted and fused to obtain a new reference parameter; if the calculation deviation is greater than or equal to the preset parameter deviation threshold, the current reference parameter is replaced by the historical optimal parameter.

[0024] The environmental parameter mapping model is a model of association between environmental variables and initial values of camera parameters established by historical calibration data, and can be obtained by regression analysis or machine learning algorithm training, and is used to dynamically adjust the initial value of the camera intrinsic parameter according to the real-time environmental parameter, so as to improve the efficiency and accuracy of parameter solving. The measurement deviation refers to the difference between the current reference parameter and the historical optimal parameter, and can be calculated by using the Euclidean distance or the weighted average error, and is used to judge whether the current calibration result meets the stability requirements of long-term operation of the system. The weighted fusion refers to the calculation method of combining the current parameter and the historical parameter according to the preset proportion, and can be realized by using the linear interpolation or the moving average method, and by giving a higher weight to the historical parameter, the interference caused by short-term environmental fluctuations is suppressed while the adaptability of parameter updating is ensured.

[0025] After each camera parameter calibration is completed, the system automatically compares the current calibration result with the historical optimal parameter. If the deviation is small, it indicates that the current environmental change does not exceed the historical experience range, and the current parameter and the historical parameter are fused according to the proportion to generate a new reference parameter, which retains environmental adaptability and maintains long-term stability; if the deviation is large, it is determined that the current environment has changed significantly, and the new parameter is directly used to replace the historical optimal parameter to quickly respond to environmental mutations. For example, in the scene of sudden change of light intensity or intensification of vibration of the loaded platform, the system adjusts the parameters in time through the dynamic updating mechanism to avoid the accumulation of image coordinate conversion errors caused by environmental factors. By fusing the historical optimal parameter, the abnormal change of the parameter is suppressed when the environment fluctuates, and the new parameter is quickly switched when the environment changes significantly, which balances the system robustness and dynamic response capability. The application can effectively solve the problem of unstable camera parameter calibration results caused by environmental changes, balance short-term accuracy and long-term reliability through a dynamic fusion mechanism, reduce measurement errors caused by sudden changes of light, vibration and other factors, and improve the accuracy of coordinate conversion of the inspection robot and the overall operation continuity of the system.

[0026] The application further proposes an observation auxiliary constraint sub-strategy, including configuring a mechanical arm action trigger condition, when the effective feature area of the positioning mark is less than a preset threshold in the field of view coverage of the calibration image information, triggering the mechanical arm to adjust within a preset adjustment range, and the adjustment range includes a pitch angle adjustment threshold, a yaw angle adjustment threshold and an end stretching amount adjustment threshold.

[0027] The field of view coverage is a ratio of a pixel area of an effective feature area of a specified position identification element in the calibration image information to a total effective feature area pixel area, and can be realized by calculating the area ratio through an image segmentation algorithm. The mechanical arm action triggering condition is a determination logic for automatically starting the mechanical arm adjustment when the field of view coverage is insufficient, and can be realized by using a real-time image analysis and threshold comparison module to ensure the sufficiency of feature acquisition. The pitch angle adjustment threshold is an angle range of the mechanical arm rotating around the transverse axis, for example, ±5°, which can be realized by controlling the pitch angle through a joint encoder, and is used to adjust the vertical coverage range of the camera view angle. The yaw angle adjustment threshold is an angle range of the mechanical arm rotating around the vertical axis, for example, ±5°, which can be realized by driving a horizontal rotation mechanism through a servo motor, and is used to adjust the horizontal coverage range of the camera view angle. The end extension adjustment threshold is a displacement range of the mechanical arm end effector moving along the axis, for example, ±50 mm, which can be realized by controlling through a linear guide and a displacement sensor, and is used to adjust the relative distance between the camera and the positioning identification element.

[0028] In the data acquisition process, the field of view coverage of the positioning identification element is calculated in real time to determine whether the current image meets the feature point acquisition requirement. If the field of view coverage is lower than a preset threshold, for example, 60%, the mechanical arm is triggered to adjust in a preset range in multiple degrees of freedom. For example, the pitch angle is adjusted to expand the field of view coverage in the vertical direction, the yaw angle is rotated to compensate for the horizontal field of view offset, and the end extension is adjusted to optimize the shooting distance. The adjusted mechanical arm pose reacquires the image until the field of view coverage meets the requirement, thereby ensuring the completeness of the subsequent calibration data. Through the automatic adjustment mechanism of the mechanical arm, the acquisition pose is actively optimized when the image quality is insufficient, avoiding the problem of feature point loss caused by limited view angle or improper distance, and improving the reliability of the calibration data. The present application can dynamically maintain the completeness of image acquisition under complex working conditions, reduce the calibration error caused by field of view obstruction or distance deviation, and enhance the adaptability of the system to interference factors such as mechanical vibration and pose deviation, thereby providing a reliable data basis for subsequent high-precision coordinate conversion.

[0029] The calibration precision judgment step further comprises a fault-tolerant application sub-strategy, including that if the average re-projection error of the calibration image after the three invalid sampling heights are supplemented in the single calibration is greater than or equal to the preset re-projection error threshold, the number of consecutive failures reaches a calibration failure error threshold, the historical optimal parameters are called as temporary reference, and an error signal is output to prompt to check the integrity of the positioning identification member; if the image measurement error detected in the inspection section is greater than an inspection error threshold, the inspection is suspended, and a simplified calibration strategy is returned to the bearing table to execute, the simplified calibration strategy is configured with three preset simplified sampling heights, which are the lowest height, the intermediate height and the highest height of the lifting stroke of the bearing table, and the re-executed control method is that the data acquisition step only acquires the calibration images of the three simplified sampling heights, the relative position calculation step, the camera parameter solving step and the calibration precision judgment step, and the weight distribution sub-step in the calibration quality control strategy is omitted.

[0030] The calibration failure refers to a state that cannot meet the accuracy requirement after multiple supplementary acquisitions in a single calibration process, and the definition can accurately identify abnormal conditions in the calibration process by setting the upper limit of the supplementary acquisition number and the error threshold. The historical optimal parameters refer to the camera parameters with the best performance in the past effective calibration period stored by the system, and the parameters with the smallest error and stable application can be selected from the historical data to achieve this, which can provide reliable backup parameters for the system to maintain basic functions. The simplified calibration strategy refers to a fast calibration method that reduces the number of sampling points and optimizes the process, and the key height point sampling and basic calculation step combination can be used to achieve this, which can quickly recover the calibration function in an emergency.

[0031] When the calibration process fails continuously for multiple times, the system automatically switches to the historical optimal parameters to maintain operation, and triggers a manual inspection instruction. If the measurement error is detected during the inspection process, the system immediately interrupts the current task and starts the simplified calibration process, and only acquires image data of the lowest, intermediate and highest three key heights to quickly calculate the parameters. This process omits the weight distribution link to improve the execution efficiency, and quickly generates temporary parameters through the basic model to restore the detection ability. Through the combination of historical parameter calling and simplified calibration, the basic accuracy is guaranteed while the fault recovery time is significantly shortened.

[0032] The application can maintain the basic running state of the system when the calibration fails, avoiding overall shutdown caused by calibration problems; in the inspection process, the measurement error anomaly is quickly responded, the parameter is quickly calibrated through the simplified process, the operation interruption time is effectively reduced, and the detection continuity is guaranteed.

[0033] The data quality assurance strategy is the core auxiliary mechanism of the data acquisition step, aiming to solve the problems of image blurring and feature point loss caused by changes in motion state such as speed fluctuations and environmental disturbances such as changes in lighting during the lifting process of the bearing table, to ensure that the collected calibration image sequence meets the accuracy requirements of subsequent camera parameter solving, and to lay a data foundation for high-precision positioning.

[0034] During the lifting process of the bearing table, the relative distance and relative motion speed between the image acquisition module and the positioning marker, such as the checkerboard calibration board, will change with the height. If fixed parameters are used to acquire images, problems such as inaccurate focusing, overexposure, or underexposure are likely to occur. This strategy uses a parameter pre-adjustment mechanism based on motion state prediction to actively adapt to the environment and motion state before each sampling. The specific steps are as follows: The system obtains the lifting speed of the bearing table in real time from the encoder feedback of the bearing table driving mechanism and the current sampling height. According to the pre-set sampling height planning, such as evenly distributing N≥3 sampling points along the lifting stroke of the bearing table, calculate the distance difference between the next sampling height and the current height and the predicted arrival time. Based on the distance difference, predict the relative distance between the image acquisition module and the positioning marker, and predict the motion blur risk based on the lifting speed, and automatically adjust the focusing parameters of the image acquisition module, such as the lens focal length, focusing motor position, and exposure parameters, such as shutter speed and ISO sensitivity. After the bearing table reaches the next pre-set sampling height and stabilizes, the pre-adjusted parameters are used to acquire calibration images immediately to avoid motion blur or parameter adaptation delay.

[0035] If sudden disturbances such as instantaneous vibration or sudden changes in lighting cause the clarity of the acquired calibration image to be lower than the pre-set threshold, i.e. a blurred image, directly discarding the sampling data will reduce the amount of sampling data and affect the accuracy of parameter solving. This strategy uses an adjacent frame data completion mechanism to recover the feature point coordinates using the effective images before and after the blurred image. The specific steps are as follows: After acquiring the calibration image sequence, calculate the clarity of each frame of image using the Laplacian gradient mean method: Clarity = ∑|▽ 2 I(x,y)| / M, where ▽ 2 I(x,y) is the second-order Laplacian operator of pixel (x,y), and M is the total number of effective pixels. Compare the calculated clarity with the pre-set clarity threshold. If it is lower than the threshold, it is determined to be a blurred image. Extract 2-3 frames of effective images before and after the blurred image, ensuring that the adjacent frames are collected near the same pre-set sampling height with minimal change in feature point position. Identify the same feature points in the adjacent effective images as in the blurred image, and extract the image coordinates of these feature points using feature point matching algorithms such as SIFT and ORB. The coordinates of the same feature points in adjacent effective images are taken as arithmetic mean values, so as to ensure the integrity of the calibration data set.

[0036] The application further proposes a convergence determination sub-strategy, which is a core termination condition of the camera parameter solving step, is used for judging whether the nonlinear optimization algorithm finds optimal camera intrinsic parameters such as focal length and principal point coordinates, avoids invalid iteration waste or non-convergence, and guarantees system continuity through a fault-tolerant process.

[0037] The camera parameter solving needs to minimize the re-projection error through an optimization algorithm, and the convergence needs to be dynamically determined in the iteration process, and the specific steps are as follows: The initial value of the camera intrinsic parameter is initialized based on the environmental parameter mapping model, historical data is referred to, and the nonlinear optimization algorithm is substituted; The re-projection error reduction is calculated after each iteration, and the re-projection error reduction is equal to the current iteration error minus the last iteration error; The error reduction in a continuous preset number of iterations is counted, if all are less than the preset change rate threshold, the algorithm is determined to be converged, and the current solved camera intrinsic parameter is output; If the cumulative number of iterations reaches the preset maximum iteration value, but the error reduction is still greater than or equal to the preset change rate threshold, that is, it is not converged, the fault-tolerant process is triggered: The historical optimal parameter is preferentially called as a temporary camera intrinsic parameter, and the parameter with the smallest average re-projection error and the non-exceeding detection error in the past 10 effective calibrations stored by the system is used; If the historical optimal parameter is not available, such as the first calibration, the optimization strategy is adjusted, such as increasing the iteration step, updating the initial value to the default parameter, and re-executing the optimization calculation.

[0038] The application further proposes that the camera parameter solving step further includes an offline model self-adaption and online weighted update sub-strategy, which includes: recording the calibration image sequence, height data, camera parameters and re-projection error collected in the transfer process to form a calibration data set; training an environmental parameter and parameter deviation correlation model based on the calibration data set, adjusting the parameter weight according to the real-time environmental data during updating; and the updated parameter is directly replaced by the original parameter as the solving initial value when substituted into the camera parameter solving calculation.

[0039] The calibration data set refers to a set of calibration images collected by the image acquisition module under different heights and different environmental conditions, corresponding height information, camera parameters and error data, and can be specifically implemented by using a database storage structure, and is used to establish the association between the environmental parameters and the camera parameter deviation. The association model of the environmental parameters and the parameter deviation is a mathematical model reflecting the mapping relationship between the light intensity, vibration amplitude, bearing state and camera intrinsic parameter deviation, which can be specifically implemented by using a linear regression or neural network algorithm, and is used to predict the parameter correction amount under different environmental conditions. The parameter weight includes an image feature point weight coefficient, an error correction decay factor and an iterative convergence threshold, which can be specifically implemented by a dynamic adjustment algorithm, wherein the initial value of the error correction decay factor is set to 0.8 and is attenuated by 0.9 times each time, and is used to control the convergence speed of parameter iteration.

[0040] Specifically, the calibration image sequence and the corresponding height data are continuously collected during the transfer process, and the calibration data set is constructed in combination with the historical calibration results. The association model is trained based on the data set, and the influence law of the change of the environmental parameters on the camera intrinsic parameters is analyzed. When the real-time environmental data is input, the model outputs the parameter weight adjustment value, for example, the image feature point weight coefficient is increased when the light intensity is lower than the threshold value to enhance the stability of feature recognition under low illumination. The updated parameters are substituted into the camera parameter solving process as initial values, and the parameter calibration is completed by using a weighted iterative optimization algorithm. Through the combination of offline training and online updating, a dynamic association mechanism of the environmental parameters and the weight adjustment is established, so that the parameter solving process can adapt to different working conditions, and the influence of environmental interference on camera calibration is effectively suppressed.

[0041] The application realizes the environmental adaptability of the camera parameter solving process, solves the problem of unstable calibration accuracy of the traditional method under complex working conditions. By dynamically adjusting the weight coefficient and the iterative convergence condition, the parameter optimization efficiency is improved, and the risk of calibration failure caused by environmental mutation is reduced, thereby providing a reliable visual positioning basis for the inspection robot.

[0042] The application further proposes that the camera parameter solving step solves the camera imaging model by using a nonlinear optimization algorithm; and the camera imaging model is as follows: , wherein (u, v) is the image coordinates of the feature points, S represents a distortion compensation term, K is a camera intrinsic parameter matrix to be solved, [R|t] is a camera extrinsic parameter matrix determined by the mechanical arm pose and the height of the bearing table, and (X, Y, Z) is the world coordinates of the feature points on the positioning marker in the bearing table coordinate system.

[0043] wherein the camera imaging model is a mathematical bridge connecting the feature point world coordinates and the image coordinates, and is used to describe how the three-dimensional world coordinates of the feature points on the positioning marker are projected into the two-dimensional image coordinates of the pixel points in the image through the camera optical system. The method adopts a pinhole camera model with distortion compensation, which can correct the lens optical distortion such as radial distortion and tangential distortion, and ensure the accuracy of the coordinate mapping. The optimization objective of the nonlinear optimization algorithm is to minimize the re-projection error function: , wherein E is the total re-projection error, is the measured coordinate of the i th feature point in the image, is a function of projecting the world point M into the image plane according to the camera model, K is an intrinsic matrix, and is the camera extrinsic parameter corresponding to the i th observation position.

[0044] The core objective of the nonlinear optimization algorithm is to minimize the re-projection error, that is, to minimize the deviation between the measured image coordinates of the feature points and the predicted image coordinates by adjusting the camera intrinsic parameter K and the extrinsic parameter [R|t], so as to solve the optimal camera parameters.

[0045] The objective function is decomposed and the optimization logic is based on error back propagation: by calculating the re-projection error of each feature point, the camera parameters are adjusted in reverse until the total error E reaches the minimum value. The specific decomposition is as follows: error source: the re-projection error is derived from the deviation between the measured coordinates and the predicted coordinates, and the smaller the deviation, the closer the camera parameters to the actual optical and motion state; optimization process: using a nonlinear optimization algorithm, iteratively adjusting K, and : after each iteration, the predicted coordinates of each feature point are calculated by the camera imaging model; the Euclidean distance between the measured coordinates and the predicted coordinates is calculated, and the total re-projection error E is obtained; based on the gradient direction of the error E, the parameters are adjusted, such as increasing / decreasing the focal length, fine-tuning the rotation matrix; termination condition: when E reaches the minimum value, the iteration is stopped, and the optimal parameters are output.

[0046] The above is only a preferred embodiment of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any technical solutions falling within the scope of the present application should be considered as falling within the protection scope of the present application. It should be noted that, for ordinary skilled persons in the art, some improvements and refinements without departing from the principles of the present application should also be considered as falling within the protection scope of the present application.

Claims

1. A method for coordinated control of track transport and platform centralized control path of an inspection robot, wherein the inspection robot is equipped with a robotic arm and an image acquisition module, characterized in that, The support platform is pre-set with positioning markers, and the control method includes: In the data acquisition step, when the inspection robot rises and falls with the carrier platform, the carrier platform stops at several preset sampling heights at intervals. At each preset sampling height, the image acquisition module acquires a calibration image sequence, which represents the continuously acquired calibration image information. The relative position calculation step involves performing feature extraction processing on each of the calibration image information to identify feature points on the positioning marker, obtaining image coordinates and corresponding preset sampling heights to construct a camera imaging dataset, wherein the image coordinates reflect the coordinates of the calibrated feature points in the preset carrier platform coordinate system; The camera parameter solving steps involve establishing a camera imaging model based on the camera imaging dataset, solving the camera imaging model, and outputting the calculated camera intrinsic parameters. The calibration accuracy judgment step involves calculating the reprojection error of feature points at all preset sampling heights. Specifically, the reprojection error is the difference between the measured coordinates of the image and the predicted coordinates of the parameters. If the mean of the reprojection error is less than a preset reprojection error threshold, it is considered to meet the preset accuracy requirements, and the currently calculated camera intrinsic parameters are used as reference parameters. These reference parameters are used for image correction during the inspection phase. If the mean of the reprojection error is greater than or equal to the preset reprojection error threshold, a preset calibration quality control strategy is triggered, and the camera parameter solving step is re-executed.

2. The method for coordinated control of inspection robot track transport and platform centralized control path according to claim 1, characterized in that, The calibration quality control strategy includes: The feature point filtering sub-step removes low-precision points. Specifically, each low-precision point is a feature point in the calibrated image information whose distance from the image edge is less than a preset edge pixel distance threshold, or a feature point whose distance from its adjacent feature points is less than a preset feature point distance threshold. The frame validity determination sub-step assesses the image quality and feature point count of each calibrated image information. If the image meets preset validity conditions, it is marked as valid. If it does not meet the preset validity conditions, it is marked as invalid and a supplementary sampling signal is generated. Specifically, the validity conditions are that the image sharpness is greater than or equal to a preset image sharpness threshold and the number of valid feature points is greater than or equal to a preset valid feature point count threshold. The weight allocation sub-step assigns weights to each valid image information. The higher the image clarity and the more valid feature points a valid image information has, the higher its weight.

3. The method for coordinated control of inspection robot track transport and platform centralized control path according to claim 2, characterized in that, The camera parameter solving step also includes dynamically updating a sub-strategy, configured with a preset environment parameter mapping model, which reflects the relationship between the initial parameter values ​​and the environment parameters. After each calibration, the measurement deviation is calculated. Specifically, the measurement deviation is the deviation between the current benchmark parameter and the historical best parameter. If the measurement deviation is less than the preset parameter deviation threshold, the current benchmark parameter and the historical best parameter are included in the historical dataset, and a new set of optimal parameters is recalculated based on the complete historical dataset as the new benchmark parameter. If the measurement deviation is greater than or equal to the preset parameter deviation threshold, the historical best parameter is replaced with the current benchmark parameter.

4. The method for coordinated control of inspection robot track transport and platform centralized control path according to claim 1, characterized in that, The data acquisition step also includes an observation-assisted constraint sub-strategy, including: The system is equipped with robotic arm action triggering conditions. The field of view coverage rate is defined as the ratio of the pixel area of ​​the effective feature area of ​​the positioning marker in the calibration image information to the pixel area of ​​the total effective feature area of ​​the positioning marker. When the field of view coverage rate of the positioning marker in the calibration image information is less than the preset field of view coverage rate threshold, the robotic arm is triggered to perform action adjustment within the preset robotic arm adjustment range. The preset robotic arm adjustment range includes preset pitch angle adjustment threshold, yaw angle adjustment threshold, and end effector extension adjustment threshold.

5. The method for coordinated control of inspection robot track transport and platform centralized control path according to claim 1, characterized in that, The calibration accuracy determination step also includes a fault-tolerant application sub-strategy, including: If the number of consecutive failures reaches the preset calibration failure error threshold, the historical best parameters are used as a temporary benchmark and an error signal is output to prompt the inspection of the integrity of the positioning marker. If an image measurement error is detected in the inspection section that is greater than the preset inspection measurement error threshold, the inspection is paused and the system returns to the carrier platform to execute a simplified calibration strategy. The simplified calibration strategy is configured with a preset simplified sampling height, and the control method is re-executed at the simplified preset sampling height.

6. The method for coordinated control of inspection robot track transport and platform centralized control path according to claim 1, characterized in that, The data acquisition steps also include data quality assurance strategies, including: Based on the lifting speed of the platform and the preset sampling height, the focus and exposure parameters of the next sampling of the predictive image acquisition module are adjusted in advance. If a blurred image with a resolution lower than a preset resolution threshold is acquired in the calibration image sequence, the image coordinates of the same feature point are extracted from the calibration image information adjacent to the blurred image before and after it, and their average value is taken as the image coordinates of the feature point in the blurred image.

7. The method for coordinated control of inspection robot track transport and platform centralized control path according to claim 1, characterized in that, The camera parameter solving step is configured with a convergence decision sub-strategy, including: When using an optimization algorithm to solve the camera imaging model, if the decrease in reprojection error after a preset number of iterations is less than a preset rate of change threshold, the algorithm is considered to have converged. If the number of iterations exceeds a preset maximum value and convergence is still not achieved, a fault-tolerant process is triggered, and historical parameters or an adjusted optimization strategy are used to recalculate.

8. The method for coordinated control of inspection robot track transport and platform centralized control path according to claim 3, characterized in that, The camera parameter solving step also includes offline model adaptation and online weighted update sub-strategies, including: The calibration dataset is formed by recording the calibration image sequence, height data, camera parameters, and reprojection errors collected during the transportation process. The model is trained based on the calibration dataset to correlate environmental factors with parameter deviations. During the update, the parameter weights are adjusted according to real-time environmental data. The parameter weights include image feature point weight coefficients, error correction attenuation factors, and iterative convergence thresholds. The original parameters are then substituted into the camera parameter calculation.

9. The method for coordinated control of inspection robot track transport and platform centralized control path according to claim 1, characterized in that, The camera parameter solving step employs a nonlinear optimization algorithm to solve the camera imaging model; the camera imaging model is as follows: ， Where (u,v) are the image coordinates of the feature points, S represents the distortion compensation term, K is the camera intrinsic parameter matrix to be solved, [R|t] is the camera extrinsic parameter matrix determined by the robot arm pose and the platform height, and (X,Y,Z) are the world coordinates of the feature points on the positioning marker in the platform coordinate system.

10. The method for coordinated control of inspection robot track transport and platform centralized control path according to claim 9, characterized in that, The optimization objective of the nonlinear optimization algorithm is to minimize the reprojection error function: ， Where E is the total reprojection error. Let be the measured coordinates of the i-th feature point in the image. (·) is a function that projects world point M onto the image plane based on the camera model, and K is the intrinsic parameter matrix. and Let be the camera extrinsic parameters corresponding to the i-th observation position.

Images