Visual positioning system for balance oil cylinder of hydraulic support

Abstract

The invention discloses a visual positioning system for a balance oil cylinder of a hydraulic support, relates to the technical field of visual measurement, and is used for solving the problem that error decisions are generated under extreme conditions due to simple weighted fusion of data of sensors. A comprehensive scoring mechanism is generally adopted in the methods, different indexes are balanced through weight coefficients, but in the multi-interference-source environment of an underground coal mine, the weight coefficients are difficult to use, key defects are easy to cover due to an averaging effect, and systematic faults are caused. Calibration parameters are provided through the calibration initialization module, the visual perception module adopts multi-prior modeling, and a gating mechanism is implemented by the comprehensive analysis unit to output observation data; the pose calculation module calculates the pose of the oil cylinder in the outer circle channel, the step channel and the hole edge channel, the motion control module completes hole alignment positioning, and the hole alignment success rate and the operation reliability are improved.

Description

Technical Field

[0001] This invention relates to the field of visual measurement technology, and more specifically, to a visual positioning system for a hydraulic support balance cylinder. Background Technology

[0002] In fully mechanized coal mining faces, the hydraulic support's balance cylinders require high-precision operations such as hole positioning and pin insertion during maintenance and replacement. Traditional manual methods suffer from low efficiency, poor consistency, and high safety risks. Existing automation solutions mostly rely on preset trajectories or a simple fusion approach with single-channel vision. However, under complex interference conditions such as pervasive coal dust, strong metal reflections, obstruction by hoses and structural components, and deflection of the robotic arm and tooling due to heavy loads, leading to hand-eye extrinsic parameter drift, image and point cloud quality is difficult to stabilize, and single observation channels are prone to failure. At the same time, multi-sensor weighted average comprehensive scoring can dilute anomalies in extreme scenarios, masking key defects and leading to systematic misjudgments. Ultimately, this results in low hole positioning success rates and frequent need for manual intervention. To adapt to such multi-disturbance working conditions, the industry needs a systematic technical path that can perform gating quality checks before data enters the computing chain, robustly calculate pose even under obstructed or reflective conditions, and form an adaptive closed loop between perception and control.

[0003] To address the above problems, this invention proposes a solution. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a visual positioning system for a hydraulic support balancing cylinder to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] The hydraulic support balance cylinder vision positioning system includes:

[0007] Calibration initialization module: Loads historical calibration parameter set, reads and verifies camera intrinsic parameters, distortion coefficients and hand-eye relationship, performs electrical, hydraulic and sensor self-tests, and provides calibration parameters and self-test status to visual perception module and pose calculation module;

[0008] Visual perception module: acquires multi-view observation data, establishes priors by geometric prior unit, occlusion prior unit and reflection prior unit, and the comprehensive analysis unit judges the observation quality. When the quality meets the standard, the evaluated observation data is output to the pose calculation module. When the quality does not meet the standard, a re-observation is triggered.

[0009] Pose calculation module: Under the constraints of the calibration parameters and the evaluated observation data, the pose of the outer circular channel is calculated in parallel using ICP, the step channel using SVD, and the hole edge channel using PnP. Consistency verification and fine optimization are performed, and the pose of the hydraulic cylinder is output to the motion control module.

[0010] Motion control module: Based on the cylinder pose, it performs path planning, calls end-effector micro-attitude adjustment and pin control to generate robotic arm motion commands, switches between control modes by mode scheduling, sends robotic arm motion commands to the actuator, and feeds back the status and alignment error to the vision perception module and the pose calculation module.

[0011] In a preferred embodiment, the calibration initialization module further includes calibration board identification and working condition configuration parsing, generating a calibration version number and timestamp and writing them into the calibration parameters and operating environment identifier, distributing the calibration version number to the visual perception module and the pose calculation module, and switching to a backup parameter set and caching it to local non-volatile storage when a self-check anomaly occurs, for consistent reference across modules.

[0012] In a preferred embodiment, the visual perception module establishes an observation sequence number and a unified time base during multi-view observation and records inter-frame delay statistics. The comprehensive analysis unit records exposure and gain parameters according to the calibration version number. After receiving the alignment error feedback from the motion control module 104, the gating mechanism adjusts the next frame acquisition configuration and triggers a resampling marker.

[0013] In a preferred embodiment, the pose calculation module organizes the evaluated observation data into a candidate feature set, and performs parallel solutions using ICP for the outer circular channel, SVD for the step channel, and PnP for the hole edge channel, and generates a channel metadata table. The consistency verification unpacks the candidates to generate a solution set index, and the fine optimization performs joint solution with index constraints while maintaining the correspondence between the index and the metadata.

[0014] In a preferred embodiment, the consistency verification includes a threshold combiner and topological constraint rules. The threshold combiner includes amplitude threshold and structural threshold entries. The topological constraint rules are expressed as the connectivity of adjacent elements. Channel arbitration is performed based on the prior labels output by the geometric prior unit, the occlusion prior unit, and the reflection prior unit. The arbitration result, along with the solution set index, is then passed to the fine optimization.

[0015] In a preferred embodiment, the path planning generation of the motion control module includes an attitude field, a speed field, and a pin control field, and maintains an instruction sequence number and a timestamp. The mode scheduling maintains an alignment mode, an insertion mode, and a backtracking mode. While issuing the robotic arm motion instructions to the actuator, it also issues a resampling parameter set and a sampling period entry to the vision perception module.

[0016] In a preferred embodiment, after receiving the cylinder pose and alignment error output by the pose calculation module, the motion control module updates the viewpoint, exposure and frame rate entries in the resampling parameter set, records the effective time and source channel, and reports the usage record and record number of the calibration version number to the calibration initialization module through the status feedback channel.

[0017] In a preferred embodiment, a safety monitoring module is also included. The safety monitoring module receives data from sensors and limit switches, forms a monitoring status word, and establishes a linkage with the motion control module. The monitoring status word includes entries for door limit, emergency stop, stroke, and pressure. When the robot arm motion command and the alignment error are received, a safety arbitration indication is generated and sent back to the vision perception module.

[0018] In a preferred embodiment, the system further includes a data analysis module, which receives the observation data, the cylinder pose, the resampling parameter set, and the monitoring status word, establishes a data recording and query interface, establishes a cross-module data bus identifier and a time index, and provides the data recording index to the visual perception module and the pose calculation module.

[0019] In a preferred embodiment, after receiving the data record index and the resampling parameter set, the visual perception module backtracks multiple frames of observations of the same target according to the observation sequence number, and generates an observation quality summary and a backtracking batch number in the comprehensive analysis unit, which are then provided to the pose calculation module along with the observation data and the batch number.

[0020] The technical effects and advantages of the hydraulic support balance cylinder visual positioning system of the present invention are as follows:

[0021] This invention provides unified camera intrinsic parameters, distortion coefficients, and hand-eye relationships through a calibration and initialization module, forming a consistent parameter baseline across modules. The visual perception module uses geometric prior units, occlusion prior units, and reflection prior units to perform gating quality inspection and comprehensive analysis on multi-view observations, ensuring the consistency and usability of data entering the computation link. The pose calculation module uses ICP for the outer circle channel, SVD for the step channel, and PnP for the hole edge channel in parallel, and outputs the cylinder pose after consistency verification and fine optimization to meet the control side's requirements. The motion control module performs path planning and pin control based on the cylinder pose, and uses alignment error information to drive the re-sampling parameter set update, forming a closed loop of perception and control. The safety monitoring module uses monitoring status words to pre-arbitrate control commands and transmits constraints back to the vision side. The data analysis module establishes data records, data bus identifiers, and time indexes to support observation backtracking and cross-module tracking, thereby achieving link stability, adaptability to complex working conditions, and consistent data management. Attached Figure Description

[0022] Figure 1 This is a structural block diagram of a hydraulic support balance cylinder visual positioning system according to an embodiment of the present invention;

[0023] Figure 2 This is a structural block diagram of the visual perception module according to an embodiment of the present invention;

[0024] Figure 3 This is a structural block diagram of the pose calculation module according to an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0026] The embodiments of this application will be described below with reference to the accompanying drawings. This application provides a visual positioning system for a hydraulic support balance cylinder, see below. Figure 1 The system includes the following modules:

[0027] Calibration initialization module 101: After the system is powered on, it loads historical calibration parameters, performs electrical, hydraulic, and sensor self-tests, and starts the initial calibration process. This module reads the parameters from the last successfully calibrated system from the embedded Flash memory, including the camera intrinsic parameter matrix K, distortion coefficient vector D, hand-eye transformation matrix, and transformation of the tool coordinate system relative to the flange center, completing various equipment checks and calibration preparations.

[0028] Visual perception module 102: Used to acquire multi-view observation data, establish three types of models: geometric prior, occlusion prior and reflection prior, analyze the current scene features and interference factors, and provide prior information support for subsequent identification and positioning.

[0029] Pose calculation module 103: Calculates the current pose of the hydraulic cylinder based on multi-channel geometric feature matching and consistency judgment. This module extracts features and calculates the pose in three independent channels: the outer circle channel, the step channel, and the hole edge channel. Then, it performs consistency verification and fine optimization of the results, and outputs reliable hydraulic cylinder position and posture.

[0030] Motion control module 104 generates robotic arm motion commands based on pose calculation results. The path planning unit considers safety and stability constraints, the active sensing unit improves sensing performance through end-effector micro-posture adjustment, the pin control unit precisely controls the insertion and removal process, performs alignment error compensation and speed buffer control, and the mode scheduling unit switches between normal, cautious, and conservative control modes based on real-time status.

[0031] Safety monitoring module 105: Real-time assessment of system safety status. It continuously monitors data such as human-machine distance, robotic arm speed, chassis posture stability, and perception reliability. When a risk or anomaly is detected, it performs tiered degraded processing, such as pausing and reverting, adjusting observations, and tactile search. If necessary, it triggers an emergency stop and saves on-site data to ensure safety.

[0032] Data Analysis Module 106: Records and analyzes data throughout the entire process. The data recording unit stores key image frames, calculation results, trigger logs, mode switching events, and other information in a structured manner; the performance evaluation unit compares the effects of different strategy combinations offline; and the parameter optimization unit adjusts system parameters based on the evaluation conclusions to continuously improve system performance.

[0033] For calibration initialization module 101:

[0034] The calibration initialization module 101 performs a series of preparatory tasks upon system power-up. Initially, it loads the previously saved set of calibration parameters, making calibration data such as camera intrinsic parameters, distortion coefficients, and hand-eye relationship directly available. However, these historical parameters may have slight deviations in different environments. In actual operation, environmental changes may cause parameter drift; for example, the tool length at the robotic arm flange may stretch or contract by about 0.1% due to temperature differences. Therefore, positioning cannot be performed solely using old parameters. After loading historical calibration parameters, the system still needs to enter the initial calibration process to ensure that key parameters are accurate and reliable in the current environment.

[0035] Next, the calibration initialization module 101 performs a comprehensive electrical, hydraulic, and sensor self-test. The self-test process includes several items, specifically:

[0036] Electrical system insulation testing: Use a megohmmeter to measure the system's insulation resistance to ground. The resistance must be greater than the preset value. If it is lower than this value, it indicates that the electrical insulation may be damp or damaged, and the potential hazard needs to be eliminated before operation.

[0037] Hydraulic system pressure stability testing: This involves monitoring the pressure fluctuation range of the hydraulic circuit within a unit of time, requiring the fluctuation to not exceed the first predetermined positive and negative range of the rated value. This step can promptly detect problems such as hydraulic leaks or pump instability. In actual commissioning, it has been found that the pressure may gradually surge beyond the predetermined threshold after several seconds. Therefore, it is necessary to extend the testing time and strictly limit the threshold to avoid the risk of undetected pressure anomalies.

[0038] Camera focus adjustment mechanism check: Using the camera's built-in test pattern, move the focus adjustment component back and forth to ensure smooth movement without any jamming. Even slight jamming in the focus adjustment mechanism may cause subsequent images to be blurry or unstable.

[0039] Depth sensor transmit power detection: Read the power feedback from the infrared emitter of the depth camera. The measured value deviation should not exceed the preset range of the nominal value. Too weak a transmit power will result in insufficient depth data range, while too strong a power may pose a safety hazard. Therefore, it must be calibrated to the normal level before startup.

[0040] After completing the electrical and hydraulic component tests, the mechanical component verification begins. The mechanical verification unit controls the robotic arm to move sequentially to the zero position of each joint and uses a high-precision laser tracker to measure the actual position of the robotic arm's end effector. The positioning error of the end effector after each axis returns to zero must not exceed a second predetermined positive and negative range. If it does, it indicates a problem with the zero-position calibration or the sensor, requiring recalibration or repair. The gripper is controlled to open and close, and the corresponding limit switch signals are checked for timeliness and accuracy to avoid dangerous situations where the gripper actually closes but the sensor fails to provide feedback. Pressure sensors at the chassis support legs are read to verify the balanced grounding force on the four support legs. The difference in pressure readings between the support legs should not exceed a certain preset ratio to ensure chassis stability. If the support forces differ significantly, it indicates uneven ground or a faulty support leg, requiring adjustment before continuing operation.

[0041] After completing the above series of checks and confirming that everything is correct, the system enters the initial observation and calibration phase. The observation and acquisition unit will control the robotic arm to move the camera to three safe positions to perform a surround scan of the work area. Multiple frames of images are acquired in each position to build the environment model. As an optional example, the actual environment model could be:

[0042] The first orientation is a bird's-eye view, with the camera's optical axis at approximately 45° to the horizontal plane and about 1.2 meters away from the oil cylinder. This angle provides a general bird's-eye view of the oil cylinder and its surrounding environment, capturing features of the top and some sides of the cylinder.

[0043] The second posture is a side view, approximately 30° horizontally to the side of the cylinder, about 0.8 meters away from the cylinder. Side viewing helps to observe the details of the mounting surface and pin hole edge on the side of the cylinder flange.

[0044] The third posture is an oblique view, which allows for observation of both the flange face and the pin hole, at a distance of approximately 1.0 meter from the cylinder. The robotic arm views the cylinder obliquely from another side front, ensuring that both the flange face and the pin hole outline are visible.

[0045] For each pose, the camera continuously acquires 5 depth maps and 5 corresponding grayscale images, for a total of 15 depth maps and 15 grayscale images, forming a basic observation set for subsequent calculations. It is important to note that these image acquisitions must be performed under conditions of a stationary cylinder and stable lighting to avoid introducing motion blur or lighting changes that could interfere with subsequent recognition.

[0046] It should be noted that if only a single top-down view is used to acquire the image, the edge of the pin hole on one side of the hydraulic cylinder will be obscured by the gripper mechanism and completely invisible from the top view, resulting in excessively large initial positioning errors. Increasing the side and oblique view angles allows for multi-angle observation, clearly revealing the key features of the pin hole and improving the matching accuracy of key points. Therefore, multi-view data acquisition significantly improves stable recognition in complex scenes.

[0047] In a preferred embodiment, the calibration initialization module 101, in addition to loading historical calibration parameter sets, reading camera intrinsic parameters, verifying the relationship between distortion coefficients and hand-eye interaction, and performing electrical, hydraulic, and sensor self-tests, also includes calibration board identification and working condition configuration parsing, generating a calibration version number and timestamp and writing them into the calibration parameters and operating environment identifier, distributing the calibration version number to the visual perception module 102 and the pose calculation module 103 for consistent cross-module referencing, switching to a backup parameter set and caching it to local non-volatile storage when a self-test anomaly occurs; simultaneously, the calibration initialization module 101 receives the usage record and record number of the calibration version number reported by the motion control module 104 through the status feedback channel, which is used to form a traceable association between calibration parameters and the operating process.

[0048] For visual perception module 102:

[0049] The visual perception module 102 acquires and processes the aforementioned multi-view observation data, establishing geometric priors, occlusion priors, and reflection priors for the scene, such as... Figure 2 As shown, this is to improve the reliability of cylinder feature recognition. These three prior models are generated by sub-units, including a geometric prior unit, an occlusion prior unit, and a reflection prior unit. The comprehensive analysis unit integrates them for evaluation of the observation data.

[0050] First, the geometric prior unit extracts typical geometric features using the engineering CAD model of the hydraulic cylinder and predicts the projection positions of these features in the camera. Specifically, for example:

[0051] The following key structural elements were extracted from the CAD model: Outer Cylindrical Surface: The outer surface of a cylinder with a diameter of approximately 200mm and a height of approximately 850mm forms the main structure of the hydraulic cylinder. In the image, it is represented as an elliptical arc of a certain length. End Face Step: A step with a height of approximately 15mm exists at the end of the hydraulic cylinder, with a step diameter approximately 10mm different from the main body diameter. This step forms an annular flange in cross-section, and its edge is also a feature. Flange Mounting Surface: A circular flange at one end of the hydraulic cylinder, approximately 25mm thick, has six 22mm diameter mounting bolt holes evenly distributed around its circumference. The flange profile and hole positions are crucial for installation and positioning. Pin Hole Edge: Near the other end on the side of the hydraulic cylinder, there is a transverse pin hole with a diameter of approximately 50mm and a 2×45° chamfer. This makes the pin hole visually appear as a round hole with a highlighted edge, and its inner and outer ring contour information can be used for centering.

[0052] The aforementioned geometric features can be used to calculate the expected projection position on the camera image plane through perspective projection. The calculation considers the camera intrinsic matrix and distortion coefficients to correct the influence of fisheye distortion on the projection shape. During implementation, complex lighting and surface reflections may cause offsets. Therefore, the system not only relies on the ideal projection position based on geometric priors but also incorporates the occlusion and reflection priors below for correction.

[0053] Specifically, the occlusion prior unit analyzes the main occlusion sources in the current scene and assesses the probability of occlusion at each surface point of the hydraulic cylinder. Common occlusions at hydraulic support assembly / disassembly sites include:

[0054] Gripper body: The robotic gripper itself, when it extends to grasp the hydraulic cylinder, may block part of the camera's view.

[0055] Hydraulic hoses: The hoses connecting the oil cylinder and the hydraulic power source are often suspended on the side of the oil cylinder, which can obstruct observation from certain angles.

[0056] Camera body bracket: The large steel structure of the camera body bracket. If the camera angle is too low, the camera body frame will block the lower part of the oil cylinder and the side facing away from the camera.

[0057] Temporary obstacles: such as tools placed during the operation, the limbs of workers, etc., these moving objects may accidentally enter the field of vision and cause momentary obstruction.

[0058] The occlusion prior unit uses depth images and scene geometry models to calculate the visibility probability of each candidate feature point on the cylinder surface and generates a corresponding visibility probability map. For key feature points, such as the center of the pin hole or the edge of the flange mounting hole, the system requires a visibility probability of no less than 0.8 to consider the current observation pose sufficient to capture the feature. If a key area is predicted to be occluded with a high probability (visibility is far below the threshold), the observation angle needs to be changed and the system re-observed. It should be noted that visibility assessment involves a certain degree of uncertainty; for example, the hose may sway, causing changes in occlusion. Therefore, a certain margin is left in the threshold setting to avoid oversensitivity leading to frequent re-enhancing.

[0059] Then, the reflection prior unit analyzes the problem of reflection interference from the surface of the metal cylinder. High light reflection can cause overexposure in certain areas of the camera image or failure of depth measurement. Therefore, the system pre-identifies and marks highly reflective areas in the image as warning zones for subsequent positioning. Reflection detection is performed in three steps:

[0060] 1. Calculate the incident angle of light for each image pixel. When the incident angle is greater than 60°, the pixel is considered a high-risk area for potential reflection. A large incident angle means that the line of sight is very oblique to the object's surface, making specular reflection more likely.

[0061] 2. Pixel saturation detection: When the value of any channel in an RGB color image exceeds a preset saturation value, the pixel is marked as a saturated pixel. These pixels often correspond to areas with strong reflections or direct light sources.

[0062] 3. Perform connected component analysis on the marked high-risk areas and saturated pixels, clustering adjacent bright pixels into reflective candidate areas. If the area of ​​a connected region exceeds a preset number of pixels, it is identified as a significant reflective area.

[0063] Therefore, combining the above two points, a reflective prior is used to generate a labeled map, marking areas in the image that may be unreliable due to reflection. An example is the specular highlights on the chrome-plated surface of an oil cylinder in certain local areas, causing depth camera ranging errors. For instance, under midday sunlight, large areas of saturation patches appear on the cylinder surface, preventing ICP registration iterations from converging. Therefore, the aforementioned reflective detection is introduced, and these areas are temporarily excluded from key feature extraction. After adjustment, image areas marked with reflective properties do not participate in geometric matching calculations, effectively avoiding situations where local distortion misleads the system's judgment.

[0064] Finally, the comprehensive analysis unit fuses three types of prior information—geometric, occlusion, and reflection—to assess the quality of the base observation set and determine whether to proceed to the next step of pose calculation or trigger a re-observation. The fusion decision employs a gating mechanism, specifically including:

[0065] The requirement is that the matching degree between the projection of key features in the geometric prior and the actual observation exceeds the preset first matching degree threshold. That is, the observed image features must basically match the model projection; otherwise, it indicates that the perspective is poor or there is an error in recognition.

[0066] The visibility ratio of key points in the occlusion prior must exceed a preset second matching degree threshold, such as 90%, meaning that at least 90% of the key points are clearly visible in the image; otherwise, the observation perspective may need to be adjusted.

[0067] The requirement is that the proportion of key features covered by the reflective prior is less than a preset threshold to ensure that important features do not fall predominantly within the highlight area. For example, if more than 90% of the critical surface of the hydraulic cylinder is covered by highlight flicker, then the current lighting conditions are not suitable for precise positioning.

[0068] Only when all the above conditions are met is the observation data considered to cover the information required for localization and can be entered into the pose calculation module. If any condition is not met, the system will prioritize triggering a re-observation process. For example, it may slightly change the camera angle, adjust the exposure, or remove occlusions to re-acquire images to compensate for deficiencies. This decision-making mechanism ensures the quality of data entering subsequent calculations, reducing the occurrence of misidentification from the source.

[0069] In a preferred embodiment, when the visual perception module 102 acquires multi-view observation data, it establishes an observation sequence number and a unified time base, and records inter-frame delay statistics. The comprehensive analysis unit records exposure and gain parameters according to the calibration version number. After receiving the alignment error feedback from the motion control module 104, the gating mechanism adjusts the next frame acquisition configuration and triggers a resampling marker. Further, after receiving the data recording index provided by the data analysis module 106 and the resampling parameter set issued by the motion control module 104, the visual perception module 102 backtracks multiple frames of observation of the same target according to the observation sequence number, and generates an observation quality summary and a backtracking batch number in the comprehensive analysis unit. Together with the multi-view observation data and the backtracking batch number, it is provided again to the pose calculation module 103 to support pose recalculation and consistency verification in the backtracking scenario.

[0070] For pose calculation module 103:

[0071] like Figure 3 As shown, after acquiring high-quality observation data, the pose calculation module 103 begins to calculate the spatial position and attitude of the hydraulic cylinder. A multi-channel feature extraction and matching strategy is employed, that is, the pose is calculated in parallel from different types of features, and then the consistency of the results is verified to ensure the reliability of the final positioning. Specifically, the following steps are included:

[0072] Data preprocessing: First, the acquired multi-frame depth maps and grayscale images are time-synchronized by reading the acquisition timestamps of each frame and establishing a buffer queue. Depth frames and grayscale frames at the same time are paired using a recent-time matching method. When the time difference between two frames exceeds a synchronization threshold, the candidate pairing is discarded, and the system waits for the next frame. The synchronization threshold is a preset time threshold, preferably 10ms, to ensure that depth and grayscale information still correspond to the same pose even when the object is moving. Next, distortion correction is performed on all images. Using the camera intrinsic parameter matrix and distortion coefficient vector loaded and verified by the calibration initialization module 101, the image coordinates are corrected to remove the influence of fisheye and barrel distortion on the measurement. Finally, the processed point cloud and image coordinates are uniformly converted to the robot's base coordinate system for expression, so that the information from each sensor can be fused and calculated, providing a consistent geometric reference for subsequent feature extraction and pose calculation.

[0073] Feature Extraction: Pose calculation extracts different feature information in three independent channels. For the outer cylindrical channel: RANSAC random sampling consensus algorithm is used to fit a cylindrical model from the point cloud for the outer cylindrical surface of the cylinder. The direction vector of the cylinder's central axis and a point on the axis, such as the position of the cylinder's midpoint, are extracted. Cylinder fitting can robustly ignore outliers and converges quickly to obtain a rough axis pose of the cylinder. For the step channel: For the planar steps and flange surfaces at the cylinder end, a region growing method is used to segment the planar region from the point cloud, and its normal vector and boundary point cloud are calculated. This planar normal vector represents the orientation of the cylinder end face, while the boundary points can be compared with the CAD contour to obtain the cylinder's lateral position and rotational attitude. For the hole edge channel: For the pin hole on the side of the cylinder, Canny edge detection is performed using grayscale images, combined with an ellipse fitting algorithm, to extract the center position and axial direction of the pin hole from the contour edge pixels. Since the pin hole is elliptical in the image, its major and minor axis directions provide spatial orientation information. The aperture edge channel mainly extracts features from 2D images and then infers the corresponding 3D position based on depth data.

[0074] Each channel has its own unique feature description operator and matching strategy, which are independent of each other, ensuring that problems in one channel will not directly affect the solutions of other channels. For example, cylinder fitting is very sensitive to the cylinder axis direction, but not to rotation around the axis, while the flange plane provides a reference for the rotational direction. These channels complement each other in terms of information.

[0075] Pose estimation: After feature extraction in each channel, the system independently estimates the cylinder pose in each channel. For the outer circle channel, the ICP (Iterative Closest Point) algorithm is used to iteratively register the cylinder fitted from the measured point cloud with the theoretical cylinder in the CAD model, solving for the rigid body transformation that best coincides with the cylinder, thus obtaining the initial pose value of the cylinder relative to the robot base. For the step channel, the SVD decomposition method is used to perform least-squares fitting on the matched planar point cloud and the CAD plane, calculating the rotation and translation matrices required for planar alignment, and performing normal alignment and boundary contour alignment of the plane. For the hole edge channel, the ellipse center point and normal information in the image space are mapped to the 3D position of the pin hole in the CAD model, and the pose is solved using PnP (Perspective-n-Point). That is, by using the correspondence between several 3D and 2D points, the camera pose relative to the cylinder is solved, and then transformed into the robot base system to obtain the cylinder pose.

[0076] The three algorithms each have their strengths: ICP excels at global shape alignment, SVD offers fast and reliable plane fitting, and PnP provides accurate local feature point localization. However, the results obtained by different algorithms may vary slightly due to data noise, thus requiring further verification, as detailed below:

[0077] Consistency Verification: A strict consistency metric is set for the pose calculation module 103 to compare the pose results calculated by the three channels. If all channel results are basically consistent, the positioning is considered reliable; if the difference is large, the current frame result is rejected and a retry is performed. The specific criteria are: the position difference between any two channels must be less than a preset first difference value, and the difference in direction (attitude angle) must be less than a second difference value. For example, if the cylinder center position calculated by the cylindrical channel and the hole edge channel differs by 5mm, exceeding the threshold, the system will not readily believe either one, but will consider the current frame result unreliable, and may need to re-acquire data or wait for the next frame to recalculate. This rejection mechanism ensures that accidental single-channel errors will not propagate to the end. An optional example: if only the ICP result of the outer circle channel is relied upon for pin alignment, the point cloud distortion caused by the reflection on the cylinder surface may lead to a shift in the axis position given by the ICP. Without a second and third channel, namely the step channel and the hole edge channel, for cross-verification, the misaligned result may be used directly for operation, resulting in the pin head not being aligned with the hole. By introducing a multi-channel consistency verification check, such inconsistent frames will be automatically rejected and trigger a recalculation.

[0078] Once the calculation results from the three channels pass consistency verification, the system considers a relatively accurate initial pose of the hydraulic cylinder to have been obtained. At this point, the pose calculation module 103 will further enter the fine optimization stage: integrating the data from each channel to optimize the pose. Specifically, this includes: using the least squares method to fit the hydraulic cylinder's central axis based on the comprehensive point cloud, ensuring that the final axis direction error is less than the third difference value. Compared to single-channel fitting, global least squares can reduce the impact of noise and make the axis direction more stable. Similarly, the hydraulic cylinder end flange plane is fitted to ensure that the normal error is also within the third difference value. By integrating the point cloud data of the step and planar regions, it is closer to reality than simply relying on the ideal plane in CAD. Finally, the orthogonality relationship between the hydraulic cylinder axis and the flange plane is verified, i.e., it should be close to 90° (orthogonality error less than 1°). This geometric relationship is inherent to the hydraulic cylinder structure: the axis and the flange normal should be perpendicular. By adding this verification, on the one hand, a double guarantee mechanism confirms the rationality of the aforementioned fitting results, and on the other hand, it also monitors the assembly accuracy of the equipment—if the actual measured orthogonality error exceeds 1°, it may be due to the hydraulic cylinder itself being bent or the flange being improperly installed, which requires attention.

[0079] After the above process, the pose calculation module 103 outputs the final accurate pose of the hydraulic cylinder, providing a reliable basis for the subsequent motion control of the robotic arm.

[0080] In a preferred embodiment, the pose calculation module 103 organizes the evaluated observation data into a candidate feature set, and performs parallel solutions using ICP for the outer circular channel, SVD for the step channel, and PnP for the hole edge channel, generating a channel metadata table. The consistency verification unpacks the candidates to generate a solution set index, and the fine optimization uses the solution set index as an index constraint to perform joint solutions while maintaining the correspondence between the index and the metadata. The consistency verification includes a threshold combiner and topological constraint rules. The threshold combiner includes amplitude threshold and structural threshold entries. The topological constraint rules are expressed as the connectivity relationship between adjacent elements. Channel arbitration is performed based on the prior labels output by the geometric prior unit, the occlusion prior unit, and the reflection prior unit. The arbitration result, along with the solution set index, is passed to the fine optimization to achieve constraint screening and stable fusion of multi-channel solutions.

[0081] For motion control module 104:

[0082] The motion control module 104 plans and executes the robotic arm's movements based on the calculated cylinder pose, in order to adjust the cylinder's attitude and complete operations such as pin insertion. This module incorporates a series of safety, stability, and perception-priority strategies to ensure smooth and reliable operation.

[0083] First, the path planning unit incorporates multiple safety constraints on the macroscopic path, specifically including: Safety Distance: During path planning, the distance between the hydraulic cylinder and the personnel is always maintained greater than a preset first safety distance threshold. If the distance is expected to be less than the first safety distance threshold, the path is deemed unsafe and needs to be replanned. The first safety distance threshold is set based on the safety requirements of industrial robot collaboration and is more conservative than the 1-meter emergency stop distance for typical collaborative robots. Because the balancing hydraulic cylinder is a heavy-duty object, additional safety redundancy is required. Speed ​​Limit: The end effector speed of the robotic arm is limited to a preset safety speed threshold to prevent uncontrollable inertia and personnel risks associated with high-speed movement. In this embodiment, the safety speed threshold is set to 0.5 m / s. Curvature Constraint: The path curvature radius is required to be greater than a preset second safety distance threshold, meaning turns cannot be too abrupt. A gentle curvature reduces cylinder swaying, helping the robotic arm and load maintain stability. It should be noted that the second safety distance threshold is related to the end effector speed. Under speed constraints, the minimum allowable turning radius can be relatively small, but overall it should not be less than the cylinder length to avoid the risk of tail-wagging due to a small turning radius for long components.

[0084] Assuming safety constraints are met, stability constraints are also incorporated into the planning considerations to ensure that the overall center of gravity projection of the system always falls within the support polygon of the supporting chassis and maintains a safety margin of at least 150mm from the support boundary. If the center of gravity approaches the support edge in a certain posture, the system will fine-tune the robot arm posture or chassis position to avoid the risk of tipping over. This constraint is particularly critical when the hydraulic cylinders lift the robot to a high position.

[0085] Finally, perception requirements further influence path planning, demanding that the vision camera maintain a good field of view of key features of the hydraulic cylinder throughout the robotic arm's movement. For example, pin holes and flange faces must not move out of the camera's field of view, and the camera's top-down / angle of view of these features should be better than 45° (the more directly facing the camera, the better).

[0086] After the path planning provides an initial motion trajectory, the active sensing unit optimizes and adjusts the end effector's micro-pose near the target position. End effector micro-pose refers to subtle changes in the pose of the robotic arm's end effector, including small-scale modifications in three dimensions: orientation, lateral movement, and distance. This embodiment enumerates several candidate actions near the end effector's pose using discrete sampling. For example, micro-yaw: rotating left and right within ±5° of the current orientation, with each 1° representing one candidate, for a total of approximately 11 (including 0°). Micro-lateral movement: moving forward, backward, left, and right within ±50mm in the lateral plane of the current translation position, with each 10mm increment representing approximately 11×11=121 candidate translations. Micro-distance adjustment: moving forward and backward within ±100mm along the current line of sight, with each 20mm representing one candidate, for a total of 11 (including no movement).

[0087] The total number of combinations of these candidate actions may reach dozens, but considering that the combined effects of most dimensions are relatively independent, this embodiment typically evaluates the benefits of each of the three types of actions separately, and then selects the best ones for combination testing. Each candidate micro-action is evaluated based on the following three indicators: 1. Improvement in the visible area of ​​key surfaces: For example, whether the visible area of ​​the cylinder flange surface or the surface around the pin hole in the camera's field of view increases after fine-tuning, and by how much. This indicator measures the feature information gain brought about by the improved viewing angle. 2. Reduction in occlusion risk: Evaluating whether the fine-tuning avoids prior occlusions. For example, lateral adjustment may avoid the hydraulic pipe blocking the pin hole; how much does its occlusion probability decrease? 3. Avoidance effect in areas with excessive incident angles: Calculating whether the high-risk area of ​​reflection is reduced after fine-tuning. For example, a slight yaw of 2° may shift the angle of direct reflection from the original specular reflection, resulting in a reduction in the percentage of the highlight pixel area.

[0088] Since these three indicators may be inversely related, the motion control module 104 employs a Pareto optimization method. This involves first eliminating inferior solutions that are inferior to another candidate in all indicators, retaining only those Pareto optimal solutions that have an advantage in at least one aspect. Next, selection is made according to a pre-defined lexicographical priority, prioritizing visibility improvement, followed by occlusion risk reduction, and then incident angle avoidance. In practice, this embodiment prioritizes the candidate action with the greatest visibility improvement; if multiple candidate actions are tied in this indicator, their respective occlusion risk reduction degrees are compared; if still tied, the third indicator is compared. Finally, a set of end-point fine-tuning actions is determined and superimposed onto the original planned path for execution, thereby improving perception quality. An optional example: During a pin alignment process, if reflections at the pin hole edge in the camera image cause recognition instability, the system selects a combination of micro-yaw and lateral shift actions based on the reflection prior, shifting the camera's viewpoint away from the highlight area. This reduces the area of ​​saturated pixels due to reflection, making the pin hole features clearly visible, and subsequent alignment succeeds on the first attempt.

[0089] Once the robotic arm carrying the hydraulic cylinder has moved to the vicinity of the mounting position, the precise alignment and insertion phase, handled by the pin control unit, begins. To ensure safe and smooth insertion of the cylinder pin into the mounting hole, the system employs closed-loop control for alignment errors and actuator status. First, the relative error between the cylinder and the mounting hole is decomposed into axial and radial errors. Axial error refers to the distance deviation along the pin's axis, which must be less than a preset first distance deviation threshold, such as 0.5mm, to essentially align the pin tip with the depth of the hole. Radial error refers to the lateral deviation perpendicular to the pin's direction, requiring a more stringent requirement: less than a preset second distance deviation threshold, such as 0.2mm. Only when both errors are within the thresholds is the system allowed to perform the insertion action; otherwise, the posture is continuously adjusted. Simultaneously, the micro-vibration level of the end effector (gripper and cylinder) and the steady-state quality factor of the hydraulic system are monitored. Micro-vibration is detected by an acceleration / force sensor installed at the end. The amplitude of the micro-vibration at the moment of insertion must be less than 5μm; otherwise, the vibration may cause hard impact and scraping of the pin hole. A hydraulic steady-state quality factor greater than 2.0 indicates that the hydraulic control has entered a stable state. This indicator combines parameters such as the damping ratio and overshoot of the hydraulic oil pressure; a higher value indicates smoother hydraulic execution. These two conditions ensure that the mechanical state is sufficiently stable before the insertion action, preventing shaking or impact at critical moments. Based on the above parameters, an optional example is that the system can limit the maximum insertion speed to no more than 0.1m / s and set the approach buffer section length to 50mm. In actual operation, this means that when the pin tip is still 50mm from the hole inlet, it begins to slowly advance into the buffer section at a low speed. If micro-vibration or abnormal pressure is detected within the buffer section, the machine can be stopped immediately for adjustment.

[0090] Furthermore, the motion control module 104 is designed with a mode scheduling unit to dynamically switch between different operating strategies to adapt to real-time safety and accuracy requirements. The system presets three control modes: Normal Mode: The default mode used under safe and good conditions, with an insertion speed limit of 0.1 m / s. A pre-pressure (applying slight force to ensure alignment) is applied before pin insertion for 2 seconds, followed by insertion. Caution Mode: Activated when a potential risk is detected in the environment or condition, such as a person approaching a warning area or a decrease in visual confidence. In this mode, the motion speed is halved to 0.05 m / s, and the buffer section length is increased to 100 mm for more secure pin insertion. Conservative Mode: Used in high-risk situations or when multiple attempts fail. The speed is further reduced to 0.02 m / s, and there is a 1-second pause before contact before slowly advancing to allow hydraulic pressure to stabilize. Conservative mode enhances operational safety.

[0091] Mode switching follows a fixed priority order: security constraints first, followed by stability domain requirements, then perceived trustworthiness, and finally efficiency considerations. In other words, as soon as a security or stability warning is triggered, the system immediately switches from normal mode to cautious or conservative mode, without taking risks to save time. Furthermore, only after conditions have recovered, such as personnel leaving the area and system recalibration being completed, does it gradually transition back to a more efficient mode. This scheduling mechanism is akin to programming human experience, enabling effective real-time intervention and strategy adjustments.

[0092] In a preferred embodiment, the motion control module 104 performs path planning based on the cylinder pose and generates robotic arm motion commands. The robotic arm motion commands include posture fields, speed fields, and pin control fields, and maintain command sequence numbers and timestamps. The mode scheduling maintains alignment mode, insertion mode, and backoff mode. While issuing the robotic arm motion commands to the actuator, it also issues resampling parameter sets and sampling period entries to the vision perception module 102. When the motion control module 104 receives the cylinder pose and alignment error output by the pose calculation module 103, it updates the viewpoint, exposure, and frame rate entries in the resampling parameter set and records the effective time and source channel. It also reports the usage record and record number of the calibration version number to the calibration initialization module 101 through the status feedback channel, thereby forming a closed-loop scheduling link of perception, calculation, and control.

[0093] For security monitoring module 105:

[0094] The safety monitoring module 105 operates throughout the entire process, providing real-time assessments of key safety indicators for the system and environment. This module comprises three units: a real-time monitoring unit, a degradation processing unit, and an emergency stop unit, ensuring that the system remains safe and controllable even in complex and dynamic environments.

[0095] The real-time monitoring unit collects multi-source data at high frequency, using a TOF (Time-of-Flight) sensor to measure the distance between the robot and the human, with a sampling frequency of 10Hz. The system can detect someone approaching within 0.1 seconds. Based on safety experience, this embodiment sets a safe warning distance of 1.5 meters and an emergency braking distance of 0.5 meters. This threshold is stricter than the 1-meter stopping distance standard for some collaborative robots because this embodiment considers the complex environment of coal mine sites, making earlier braking more reliable. Joint speeds are acquired through encoders at each joint of the robotic arm, and the controller calculates the Cartesian velocity of the end effector, updating at a frequency of 100Hz. This high acquisition frequency ensures timely detection of any excessive speed. Once the speed exceeds the defined maximum speed threshold, such as 125% (i.e., 0.625 m / s), theoretically, it means increased risk, and the system will immediately take measures. The IMU installed on the chassis measures the chassis's attitude angles in real time, and combines this with information from the support leg force sensors to calculate the support polygon range and the center of gravity projection position. The offset of the center of gravity relative to the support surface is updated every tens of milliseconds. If the detected center-of-gravity margin is less than the predetermined 50mm, it indicates that the platform is very close to the edge of instability. In terms of visual perception, the reliability of perception is assessed using feature matching between camera image frames, with an update frequency of approximately 30Hz. For example, a sudden drop in the number and consistency of key feature matches in consecutive frames indicates that the vision system has lost reliable tracking of the target, resulting in reduced reliability.

[0096] Real-time monitoring data is fed into a safety state machine model to determine whether to trigger degradation processing. The degradation processing unit has a pre-set three-step progressive fault / risk response strategy, which includes: First, a small retraction. When a minor anomaly is detected, such as a pin failing to insert smoothly or signs of contact friction, the robotic arm first retracts to the previous safe posture, i.e., the most recent collision-free position with normal indicators continuously recorded by the system during operation. The retraction distance is usually about 100–200mm, which avoids the current potential jamming or danger without wasting too much progress. Next, a viewpoint reselection is performed. If the problem is determined to be vision-related, such as occlusion or recognition loss, the system will refer to the occlusion prior and reselect an observation posture to acquire the image again. Empirically, the new viewpoint is at least 30° different from the previous one to see enough new information. Therefore, the robotic arm will move around the cylinder or target hole to another side to avoid the original occlusion path. This step is equivalent to trying again from a different angle, which can often eliminate misjudgments caused by the limitations of a single viewpoint. Finally, a force-guided tactile search is performed. If the problem persists after re-observation, and the system determines that the cylinder is very close to the target position, the third and final safeguard step is implemented: a force sensor-guided tactile search. The controller instructs the gripper (along with the cylinder) to make small circular movements near the hole, or to gently swing at a preset amplitude, probing the edge of the hole. Feedback from the six-dimensional force sensor detects when and where minute contact occurs. Once a certain direction first touches the edge of the hole, the direction and distance of the cylinder pin's offset relative to the hole can be deduced, thus guiding adjustments to align it.

[0097] The emergency stop unit is the highest priority safety protection measure, which will immediately trigger an emergency stop of the entire system in the following extreme situations: when personnel enter a dangerous area at a distance far below the normal safe distance (e.g., less than 0.5 meters, indicating an emergency), an emergency stop will be initiated immediately.

[0098] In one optional example, the 1-meter stop distance mentioned earlier in the collaborative robot safety standards is a common practice. This embodiment shortens it to 0.5 meters because the system's built-in ultrasonic / radar sensors also have this threshold, and any movement within this range could cause injury. When the robotic arm's movement speed exceeds the limit by 125%, it indicates that control may be out of control or that external interference is causing abnormal acceleration. In this case, the system must be stopped immediately for inspection. When the chassis support polygon margin is less than 50mm, it is determined that instability is imminent, and all actions must be stopped immediately to prevent tipping. If any major sensor (camera, force sensor, hydraulic pressure sensor, etc.) malfunctions or loses data, the system enters an emergency stop to avoid continuing to operate blindly, awaiting manual intervention.

[0099] After an emergency stop, the system will immediately lock all joints of the robotic arm and cut off hydraulic power to ensure that the cylinders remain in their current positions. At the same time, it will trigger automatic data saving, caching and writing the image sequence of the most recent 10 seconds, control command history, and sensor readings into the log for easy post-accident analysis of the cause of the accident.

[0100] In a preferred embodiment, the system further includes a safety monitoring module 105. The safety monitoring module 105 connects to sensor and limit switch data to form a monitoring status word and establishes linkage with the motion control module 104. The monitoring status word includes gate limit, emergency stop, stroke and pressure entries. When the robot arm motion command and the alignment error are received, the safety monitoring module 105 generates a safety arbitration instruction and sends it back to the vision perception module 102 to coordinate the observation and execution when there is a safety risk or structural constraint trigger.

[0101] For data analysis module 106:

[0102] The data analysis module 106 is used to record data throughout the process and analyze it after the task is completed to continuously improve the system algorithm and parameter configuration. It includes three sub-units: a data recording unit, a performance evaluation unit, and a parameter optimization unit.

[0103] The data recording unit stores various key data in a pre-designed structured format to facilitate future retrieval and analysis. These key data include:

[0104] Keyframe images: These are camera shots capturing crucial moments during the operation, in JPEG format. Each frame includes a timestamp and metadata such as the cylinder's pose at the time of capture. Keyframes include representative frames from the initial observation set, frames at the moment alignment is complete, and frames where re-observation or downgrading occurs.

[0105] Pose calculation results: Each successfully estimated cylinder pose, along with intermediate results from each channel, is saved as a JSON file. This file contains data such as the estimated pose, matching error, and consistency metrics for each of the three channels. These detailed records facilitate post-calculation analysis to identify any abnormal deviations. For example, it allows us to trace why a frame with a sudden increase in ICP error still passed verification.

[0106] Gating Trigger Log: Records each gating condition-triggered event, re-observation, or degraded processing event in CSV format, including the trigger time, cause determination (i.e., which condition was not met, such as insufficient visibility or excessive reflection), and subsequent processing measures, such as adjusting the angle and reshooting. By accumulating these logs, it is possible to identify which problems occur most frequently, thus enabling targeted improvements.

[0107] Mode switching events: Details of mode switching are recorded using XML format, including the time of the switch, the mode names before and after the switch, changes in key parameters, and environmental status indicators at the time. This is used to check the rationality of mode scheduling. For example, excessively frequent switching may indicate an inappropriate threshold setting, which can be identified and optimized by analyzing the logs.

[0108] The performance evaluation unit performs offline comparative analysis on accumulated data when the task is completed or the system is idle. This involves selecting task records completed under similar conditions and comparing the performance of different parameter or strategy combinations. For example, this embodiment analyzes the impact of the following strategies on system performance: Whether active perception is enabled: Comparing how much the visibility of key system features improves and whether the pin alignment success rate increases when active end-effector micro-pose optimization is enabled. Statistics show that enabling active perception under complex lighting conditions can reduce the feature loss rate from approximately 30% to below 10%, proving its effectiveness. Whether online extrinsic parameter calibration is used: Some tasks allow for minor adjustments to camera extrinsic parameters mid-process, such as when the robotic arm experiences hand-eye matrix drift due to temperature, in which case a new calibration plate is used for adjustment. Comparing the changes in positioning accuracy in these two cases, it was found that the consistency residual of pose calculation decreased in batches that underwent online calibration, indicating that extrinsic parameter fine-tuning helps compensate for system deviations during long-term operation. However, such calibration requires downtime, so the benefits and efficiency losses must be weighed. Whether tactile search is used: Comparative analysis is performed on tasks where end-effector insertion difficulties occur. Without the support of tactile search by force sensors, the success rate of latching operations drops significantly when encountering visual instability, requiring more manual intervention. However, with the introduction of tactile search, the overall success rate without human intervention improves.

[0109] The parameter optimization unit periodically adjusts various system parameters and strategy priorities based on performance evaluation results to continuously improve overall performance. For example, analysis revealed that a certain fixed observation azimuth (low-angle upward view) would cause re-shooting in 80% of cases due to occlusion, wasting time and posing safety hazards. Therefore, this angle can be removed from the preset observation sequence. For strategies with excellent performance, their call frequency or priority is increased. For instance, comparison showed that adding just one frame of oblique angle observation can significantly improve positioning accuracy at a minimal cost, so oblique angle observation is upgraded from optional to mandatory. Furthermore, adding monitoring of center of gravity margin in path planning has averted several dangerous situations, so this function will be extended to all motion planning stages. Regarding parameters, if a certain threshold is consistently too conservative, leading to efficiency losses, it can be appropriately relaxed, while other thresholds that are frequently triggered at critical points may need to be tightened. For example, the human-machine distance warning threshold of 1.5 meters has never been actually triggered in the actual field. Therefore, we considered slightly reducing the warning radius to improve the sensitivity to real dangers. Conversely, the speed limit of 0.5 m / s was almost triggered twice by 125% emergency braking. Therefore, the control margin can be tightened to 120% for greater stability.

[0110] In a preferred embodiment, the system further includes a data analysis module 106, which receives observation data, cylinder pose, resampling parameter set and monitoring status word, establishes data recording and query interface, establishes cross-module data bus identifier and time index, generates data record index and provides the data record index to the visual perception module 102 and the pose calculation module 103 to support process recording and result verification under gating trigger, resampling strategy and backtracking mechanism.

[0111] Through this series of revised and supplemented descriptions of embodiments, it can be seen that the technical solution of this application has been fully disclosed and verified in detail. It not only describes the normal operation process of each module, but also provides how to handle abnormal situations, as well as the negative consequences and comparative data that may result from not using certain key features. This ensures that those skilled in the art can understand and practice this solution after reading it, and understand the importance and scope of each element. In summary, although the foregoing embodiments have been described in detail, those skilled in the art may still modify or replace some modules or steps without departing from the scope of this application. Therefore, the protection scope of this application is not limited to the specific embodiments described above, but covers various equivalent variations of its technical solution.

Claims

1. A hydraulic support balance cylinder visual positioning system, characterized in that, include: Calibration initialization module: Loads historical calibration parameter set, reads and verifies camera intrinsic parameters, distortion coefficients and hand-eye relationship, performs electrical, hydraulic and sensor self-tests, and provides calibration parameters and self-test status to visual perception module and pose calculation module; Visual perception module: acquires multi-view observation data, establishes priors by geometric prior unit, occlusion prior unit and reflection prior unit, and the comprehensive analysis unit judges the observation quality. When the quality meets the standard, the evaluated observation data is output to the pose calculation module. When the quality does not meet the standard, a re-observation is triggered. Pose calculation module: Under the constraints of the calibration parameters and the evaluated observation data, the pose of the outer circular channel is calculated in parallel using ICP, the step channel using SVD, and the hole edge channel using PnP. Consistency verification and fine optimization are performed, and the pose of the hydraulic cylinder is output to the motion control module. Motion control module: Based on the cylinder pose, it performs path planning, calls end-effector micro-attitude adjustment and pin control to generate robotic arm motion commands, switches between control modes by mode scheduling, sends robotic arm motion commands to the actuator, and feeds back the status and alignment error to the vision perception module and the pose calculation module.

2. The hydraulic support balance cylinder visual positioning system according to claim 1, characterized in that: The calibration initialization module also includes calibration board identification and working condition configuration parsing, generating a calibration version number and timestamp and writing them into the calibration parameters and operating environment identifier, distributing the calibration version number to the visual perception module and the pose calculation module, and switching to a backup parameter set and caching it to local non-volatile storage when a self-check anomaly occurs, for consistent reference across modules.

3. The hydraulic support balance cylinder visual positioning system according to claim 2, characterized in that: The visual perception module establishes an observation sequence number and a unified time base during multi-view observation, and records inter-frame delay statistics. The comprehensive analysis unit records exposure and gain parameters according to the calibration version number, and adopts a gating mechanism to adjust the next frame acquisition configuration and trigger resampling markers after receiving the alignment error feedback from the motion control module.

4. The hydraulic support balance cylinder visual positioning system according to claim 3, characterized in that; The pose calculation module organizes the evaluated observation data into a candidate feature set, and performs parallel solutions using ICP for the outer circular channel, SVD for the step channel, and PnP for the hole edge channel, and generates a channel metadata table. The consistency verification unpacks the candidates to generate a solution set index, and the fine optimization performs joint solution with index constraints while maintaining the correspondence between the index and the metadata.

5. The hydraulic support balance cylinder visual positioning system according to claim 4, characterized in that: The consistency verification includes a threshold combiner and topological constraint rules. The threshold combiner includes amplitude threshold and structural threshold entries. The topological constraint rules are expressed as the connectivity relationship between adjacent elements. Channel arbitration is performed based on the prior labels output by the geometric prior unit, the occlusion prior unit and the reflection prior unit, and the arbitration result, together with the solution set index, is passed to the fine optimization.

6. The hydraulic support balance cylinder visual positioning system according to claim 5, characterized in that: The path planning generation of the motion control module includes attitude field, speed field and pin control field, and maintains instruction sequence number and timestamp. The mode scheduling maintains alignment mode, insertion mode and backtracking mode. While issuing the robotic arm motion command to the actuator, it also issues the resampling parameter set and sampling period entry to the vision perception module.

7. The hydraulic support balance cylinder visual positioning system according to claim 6, characterized in that: After receiving the cylinder pose and alignment error output by the pose calculation module, the motion control module updates the viewpoint, exposure and frame rate entries in the resampling parameter set, records the effective time and source channel, and reports the usage record and record number of the calibration version number to the calibration initialization module through the status feedback channel.

8. The hydraulic support balance cylinder visual positioning system according to claim 7, characterized in that: It also includes a safety monitoring module, which connects to sensor and limit switch data to form a monitoring status word and establishes linkage with the motion control module. The monitoring status word includes door limit, emergency stop, stroke and pressure entries. When the robot arm motion command and the alignment error are received, a safety arbitration instruction is generated and sent back to the vision perception module.

9. The hydraulic support balance cylinder visual positioning system according to claim 8, characterized in that: It also includes a data analysis module, which receives observation data, cylinder pose, resampling parameter set and monitoring status word, establishes data recording and query interface, establishes cross-module data bus identifier and time index, and provides data record index to the visual perception module and the pose calculation module.

10. The hydraulic support balance cylinder visual positioning system according to claim 9, characterized in that: After receiving the data record index and the resampling parameter set, the visual perception module backtracks multiple frames of observations of the same target according to the observation sequence number, and generates an observation quality summary and a backtracking batch number in the comprehensive analysis unit, which are then provided to the pose calculation module along with the observation data and the backtracking batch number.

Images