Method and device for realizing track control of hydraulic mechanical arm

Abstract

The invention discloses a method and device for achieving track control of a hydraulic mechanical arm, a target operation area is perceived through a depth camera, division of the operation area and marking of obstacles are completed in combination with an artificial intelligence algorithm, and tight coupling of environment modeling and task planning is achieved. According to the method, the plurality of candidate operation areas can be quickly generated and transmitted back to the ground station, an operator only needs to visually select or correct, the manual operation amount is greatly reduced, and the operation efficiency is remarkably improved. According to the embodiment of the invention, the track control precision and the working efficiency of the hydraulic mechanical arm are improved, the dependency degree of operation on artificial experience is reduced, the friendliness of man-machine interaction is enhanced, the safety and the reliability of the system can be ensured in a complex mining area scene, and the method has good application value and popularization prospect.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of automation control technology, and in particular to a method and apparatus for realizing trajectory control of a hydraulic robotic arm. Background Technology

[0002] With the increasing demand for automation in mining, tunnel construction, and high-risk environment operations, hydraulic robotic arms, as typical heavy-duty work equipment, play a vital role in remote operation and replacing manual labor. Compared to electric robotic arms, hydraulic robotic arms have advantages such as higher output force, stronger impact resistance, and adaptability to harsh environments, and are therefore widely deployed in mining, hazardous material handling, and disaster relief scenarios. However, the complex dynamic characteristics of hydraulic robotic arms and the variability of mining environments make precise, stable, and efficient trajectory control a persistent challenge in the industry. Summary of the Invention

[0003] This application provides a method and apparatus for realizing trajectory control of a hydraulic robotic arm, which can achieve high-precision, low-latency, and safe and reliable remote intelligent operation of the hydraulic robotic arm.

[0004] This invention provides a method for implementing trajectory control of a hydraulic robotic arm, comprising: Acquire depth and image data of the target work area, and divide the target work area into walkable areas and label obstacle areas to obtain a target work area model; Multiple candidate work areas are divided on the target work area model, and the divided candidate work areas are overlaid on the target work area screen in the form of images for remote operation by the operator. Receive operation information from the operator and generate a work point layout within the selected target candidate work area for the operator to perform remote operation again; Receive further operation information from the operator and confirm the target operation point within the selected target candidate operation area; Transform the confirmed target work point from the camera coordinate system to the robot arm base coordinate system; Perform path planning on the transformed target work points to generate an obstacle avoidance path point sequence; The path planning results are converted into joint motion control information for the hydraulic robotic arm.

[0005] In one exemplary instance, before converting the path planning result into joint motion control information for the hydraulic robotic arm, the method further includes: Based on the flow-pressure difference model of the hydraulic valve orifice and the dynamic response of the hydraulic cylinder, frequency scheduling and nonlinear compensation are performed on the joint motion control information to obtain the compensated target control quantity. The process of converting the path planning results into joint motion control information for the hydraulic robotic arm also includes: Frequency scheduling and signal compensation of the target control quantity are performed based on the nonlinear characteristics of the hydraulic system in which the hydraulic robotic arm is located.

[0006] In one exemplary instance, the step of dividing the target work area into walkable areas and labeling obstacle areas to obtain a target work area model includes: The target work area is perceived using a depth camera to obtain depth data and image data of the target work area; The acquired data is preprocessed; The target work area is identified using artificial intelligence (AI) algorithms or based on the Transformer recognition framework using multimodal visual fusion. Obstacles in the environment are detected using data from a depth camera. The target work area is then divided into walkable areas and obstacle areas are labeled to obtain a model of the target work area.

[0007] In one exemplary instance, the step of dividing the target job area model into multiple candidate job areas includes: Based on the data collected by the depth camera and the artificial intelligence recognition algorithm, the target work area is analyzed and the candidate work areas are divided.

[0008] In one exemplary instance, the remote operation includes selection or correction.

[0009] In one exemplary instance, the step of performing path planning for the selected work points and generating an obstacle avoidance path point sequence includes: At the global level, the selected job points are treated as a coverage path optimization problem, and the optimal access order of job points is generated based on the improved Traveling Salesman Problem (TSP) algorithm or a heuristic coverage path planning algorithm. At the local level, the Fast Extended Random Tree (RRT) algorithm or the Covariant Hamiltonian Optimization Motion Planning (CHOMP) algorithm is used to generate obstacle avoidance trajectories in the 3D environment grid, and spline interpolation is performed on the generated obstacle avoidance trajectories to achieve trajectory smoothing and time parameterization. During the path planning process, the inverse kinematics solver of the robotic arm is called to ensure that each transition posture is reachable and does not cause self-collision. When a dynamic obstacle is detected, the trajectory is replanned online based on a local cost map to generate the obstacle avoidance path point sequence that meets the obstacle avoidance requirements.

[0010] In one exemplary instance, the step of performing path planning for the selected work points and generating an obstacle avoidance path point sequence includes: Based on the dynamic parameters of the robotic arm, the selected work points are screened for dynamic feasibility, and work points that require exceeding the preset joint rotation angle or may cause joint overload are eliminated. A B-spline interpolation method with dynamic constraints is used to generate trajectory curves that satisfy the constraints of joint angular velocity, joint torque, and trajectory curvature change rate. When the trajectory approaches a high-risk area of ​​obstacles, the trajectory is locally adjusted for obstacle avoidance based on the artificial potential field method to ensure that the trajectory and obstacles maintain a preset safe distance. The planned trajectory is corrected online based on real-time feedback to generate the obstacle avoidance path point sequence that satisfies dynamic and safety constraints.

[0011] In one exemplary instance, the conversion of path planning results into joint motion control information for a hydraulic robotic arm includes: The trajectory displayed by the path planning results is subjected to inverse kinematics solution to obtain the sequence of angles, angular velocities, and angular accelerations of each joint; based on the valve control characteristic model of the hydraulic system, the joint motion quantities are converted into valve control current or PWM control signals, and the joint motion control information is obtained through a combination of feedforward compensation and PID regulation to drive the hydraulic multi-way valve; or... A nonlinear dynamic model of the hydraulic manipulator is established to predict the future motion state of the hydraulic manipulator. Based on the nonlinear dynamic model, the predictive control optimizes the trajectory deviation and control quantity changes within a preset prediction window, and obtains the joint motion control information under the conditions of satisfying the current range and joint speed deviation constraints.

[0012] This application also provides a computer-readable storage medium storing computer-executable instructions for executing the method for implementing trajectory control of a hydraulic robotic arm as described above.

[0013] This application embodiment further provides a device for realizing trajectory control of a hydraulic robotic arm, installed at a ground station, including: a UI interaction module, an operation input parsing module, a remote communication management module, and a video receiving and display module; wherein, The UI interaction module is used for depth camera images and provides an operation interface for area selection and work point confirmation or adjustment. The operation input parsing module is used to parse input commands from the handle and convert them into standardized movement, operation, or confirmation control commands; The remote communication management module is used to encapsulate and decapsulate control commands in order to issue commands and receive status information. The video receiving and display module is used to receive video streams pushed by the robot and achieve real-time display by combining caching and low-latency processing strategies.

[0014] In one exemplary instance, the UI interaction module is also used to display a 360° panoramic view of the target work area, a front-view camera view, and to display robot status information such as battery level, signal quality, joint angle, and work progress.

[0015] This application embodiment further provides a device for realizing trajectory control of a hydraulic robotic arm, installed on the robot end, including: a perception processing module, a work point planning module, a robotic arm trajectory planning module, a hydraulic control module, a remote communication management module, and a video transmission module; wherein, The perception processing module is used to collect depth camera data and identify the work area and obstacles through artificial intelligence algorithms, while performing filtering and coordinate transformation preprocessing on the original point cloud or image. The task planning module is used to generate candidate task points based on the identified task area and form a set of spatial coordinate points. At the same time, the results are visualized and sent back to the ground station. The robotic arm trajectory planning module is used to convert spatial coordinate points into a sequence of trajectory points; The hydraulic control module is used to convert the trajectory point sequence into joint motion commands, and generate current or voltage signals through PID and feedforward compensation, thereby controlling the multi-way valve to realize the action of the hydraulic cylinder. The remote communication management module is used to receive and parse instructions from the ground station, package robot status information, and send it back. The video sending module is used to push camera video to the ground station as a video stream.

[0016] In one exemplary instance, the robotic arm trajectory planning module is further configured to: before the control signal generated by trajectory planning is sent to the hydraulic control module, perform frequency scheduling and nonlinear compensation on the joint motion control signal based on the flow-pressure difference model of the hydraulic valve port and the dynamic response of the hydraulic cylinder, so as to obtain the compensated target control quantity; The hydraulic control module is also used for: frequency scheduling and signal compensation of the target control quantity based on the nonlinear characteristics of the hydraulic system.

[0017] In one exemplary instance, the perception processing module includes an explosion-proof depth camera.

[0018] The method for trajectory control of a hydraulic robotic arm provided in this application uses a depth camera to perceive the target work area and combines it with artificial intelligence algorithms to divide the work area and label obstacles, achieving tight coupling between environmental modeling and task planning. It enables the rapid generation of multiple candidate work areas and their transmission back to the ground station, allowing operators to simply select or correct them intuitively, significantly reducing manual operations and thus greatly improving work efficiency. This application not only improves the trajectory control accuracy and work efficiency of the hydraulic robotic arm but also reduces the reliance on human experience, enhances the user-friendliness of human-machine interaction, and ensures the safety and reliability of the system in complex mining environments, demonstrating good application value and promising prospects for widespread adoption.

[0019] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the description, claims, and drawings. Attached Figure Description

[0020] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0021] Figure 1 This is a flowchart illustrating the method for implementing trajectory control of a hydraulic robotic arm in an embodiment of this application. Figure 2 This is a schematic diagram of a system embodiment for implementing trajectory control of a hydraulic robotic arm in this application. Figure 3 This is a schematic diagram of the architecture of the method for implementing trajectory control of a hydraulic robotic arm according to an embodiment of this application; Figure 4 This is a first schematic diagram of the operation process of human-machine collaborative scenario guidance, taking a mining operation robot as an example in the embodiments of this application; Figure 5 This is a second schematic diagram illustrating the operation process of human-machine collaborative scenario guidance using a mining operation robot as an example in this application embodiment; Figure 6 This is a schematic diagram of the overall operation process of human-machine collaborative scenario guidance, using a mining operation robot as an example in this application embodiment. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be arbitrarily combined with each other.

[0023] To facilitate understanding of this application, a more complete description will be provided below with reference to the accompanying drawings, which illustrate embodiments of the present application. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of this application will be thorough and complete.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0025] It is understood that the terms "first" and "second" used in this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0026] It is understood that the term "connection" in the following embodiments should be understood as "electrical connection," "communication connection," etc., if the connected circuits, modules, units, etc., have electrical signal or data transmission with each other.

[0027] When used herein, the singular forms of “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising / including” or “having,” etc., specify the presence of the stated features, wholes, steps, operations, components, parts, or combinations thereof, but do not preclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts, or combinations thereof. Meanwhile, the term “and / or” as used in this specification includes any and all combinations of the associated listed items.

[0028] The steps illustrated in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases the steps shown or described may be performed in a different order than that presented here.

[0029] Hydraulic robotic arm control largely relies on manual operators controlling each joint individually via levers. Because path planning depends entirely on experience, operational efficiency is low, and the high skill requirement for operators increases training costs and limits the widespread application of the equipment. In complex working environments, the accuracy of manual judgment of the work area is limited, easily leading to improper area selection or unreasonable trajectory planning, thus posing safety risks.

[0030] On the other hand, hydraulic systems inherently possess characteristics such as strong nonlinearity, time-varying parameters, and multi-joint coupling, resulting in lag and instability in joint motion response. Manual operation struggles to guarantee smooth and high-precision trajectories. Furthermore, most existing remote control solutions only offer video feedback and basic command transmission functions, lacking intelligent environmental perception and decision-making support. They cannot provide operators with efficient work area recommendations and trajectory planning support, leading to operational efficiency and safety issues that fail to meet actual requirements in complex mining environments.

[0031] Therefore, in order to introduce depth perception, artificial intelligence recognition, and trajectory planning methods into the control of hydraulic robotic arms, and combine them with human-machine collaboration mechanisms and closed-loop hydraulic control to achieve high-precision, low-latency, and safe and reliable remote intelligent operation of hydraulic robotic arms, embodiments of this application provide a method for implementing trajectory control of hydraulic robotic arms, such as... Figure 1 As shown, it may include: Step 100: Obtain the depth and image data of the target work area, and divide the target work area into walkable areas and label the obstacle areas to obtain the target work area model.

[0032] In one exemplary instance, step 100 may include: The target work area can be perceived by a depth camera (sensor) to obtain depth data and image data of the target work area; in one embodiment, the depth camera is an explosion-proof depth camera.

[0033] The collected data is preprocessed, including filtering and coordinate transformation of the raw point cloud or image data, in order to improve data quality and enable integration with subsequent processing modules. While processing perception, artificial intelligence (AI) algorithms identify the target work area, combine depth camera data to detect obstacles in the environment, and divide the target work area into walkable areas and label obstacle areas to obtain a target work area model.

[0034] Unlike related technologies that only perform static geometric reconstruction during the modeling stage, this step generates accessibility and stability prior information in parallel while generating a map of the target work area. This provides a fast and effective criterion for scoring and selecting subsequent candidate work areas. This processing in the embodiments of this application significantly improves the coupling between environmental modeling and task planning, laying the foundation for efficient and safe trajectory planning.

[0035] In one exemplary instance, step 100 may further include: the image data may include images from a 360° surround view camera and a forward view camera to achieve multi-source information acquisition of the target work area.

[0036] In one embodiment, step 100 of this application can be based on a target work area identification method using artificial intelligence algorithms. By fusing depth camera point clouds and visible light images, and utilizing joint modeling of geometric semantics and visual features, automatic identification and classification of work areas in complex environments can be achieved. More specifically, the artificial intelligence (AI) algorithm in this step for identifying the target work area may include: the system acquiring 3D point cloud data output from a depth camera, and combining it with image information from a forward-looking camera or a 360° surround-view camera. After voxel downsampling and denoising, the data is input into a 3D semantic segmentation network based on PointNet++ and a lightweight convolutional neural network for feature extraction, respectively. Subsequently, a multimodal fusion module based on Bayesian inference is used to fuse the geometric probability distribution of the point cloud with the image semantic mask to generate a confidence distribution map of the target work area. This method can maintain high recognition accuracy even in environments with low illumination and dust obstruction in mines. By using a Bayesian probabilistic modeling method based on visual geometric fusion, the spatial structure information of the point cloud and image texture features are utilized in synergy, thereby improving the robustness of recognition in complex environments.

[0037] In one embodiment, step 100 of this application can be based on a Transformer recognition framework using multimodal visual fusion to achieve automatic identification and classification of work areas in complex environments. More specifically, a dual-branch Transformer network structure is adopted, with each branch including 6 encoder layers and 8 attention heads: the depth branch is used to extract 3D geometric features from point cloud data; the visible light image branch is used to extract image features such as texture, edges, and structure from visible light images. The two branches achieve cross-modal fusion of depth information and image features through a cross-attention module, generating highly consistent spatial semantic features. The fused features are input to an improved YOLOv8 object detection network (whose Neck layer introduces a CBAM attention module) to identify key components (such as equipment bolts and interfaces) and various obstacles (such as tools and material stacks) in the work area. This embodiment can maintain robust recognition performance under various lighting conditions and complex texture environments, improving the accuracy of target work area modeling and reachable area segmentation.

[0038] Through the implementation process of step 100, geometric semantic information, reachable area information and obstacle distribution information are generated simultaneously in the environmental modeling stage, realizing the deep integration of mapping and planning pre-planning logic. Compared with the traditional serial approach of modeling first and then planning, it significantly improves the real-time performance of target operation area modeling, the safety of planning and the efficiency of overall task execution.

[0039] Step 101: Divide the target work area model into multiple candidate work areas, and overlay the divided candidate work areas onto the target work area screen in the form of an image for the operator to perform remote operation.

[0040] In one exemplary instance, step 101 may include: Based on data collected by depth cameras and artificial intelligence recognition algorithms, the target work area can be analyzed to automatically delineate ideal candidate work areas. Specifically, depth camera data combined with AI recognition algorithms can be used to detect areas in the work scene and mark multiple candidate work areas (such as area A, area B, and area C) within the target work area. The obtained candidate work areas are overlaid on the target work area image and transmitted wirelessly to the ground station computer for remote operation by the operator, such as selection or correction.

[0041] In this embodiment, step 101 combines the candidate work area automatic division method based on depth perception and artificial intelligence recognition results. By establishing a three-dimensional semantic map and integrating accessibility, stability, and security indicators, the environment is quantitatively analyzed and candidate areas are selected. In one embodiment, based on the semantic tag point cloud obtained in step 100, a semantic occupancy grid is generated using voxel modeling, and parameters such as surface normal vector, curvature, accessibility ratio, and obstacle distance are calculated for each grid cell. Based on these features, a region scoring function is constructed: where represents the accessibility index, represents surface flatness, represents obstacle spacing, and represents the workable area; the system generates several candidate work areas based on their scores. This method can filter out unworkable areas in real time based on the solvability of the robot arm's inverse kinematics and present the results visually through a ground station interface. It is the first method to directly introduce robot arm accessibility constraints into the candidate region scoring system, achieving coupled optimization of work area selection and mechanical structure matching.

[0042] Step 102: Receive operation information from the operator and generate a work point layout in the selected target candidate work area for the operator to perform remote operation again.

[0043] In one exemplary instance, step 102 may include: By displaying candidate work areas identified and divided by the onboard computer on the ground station user interface, operators can perform operations such as selection or modification of the candidate areas through the interface, and send the operation information back to the onboard computer via the data transmission module. The onboard computer automatically generates a work point layout within the selected target candidate work area and sends the results back to the ground station for operator confirmation or further adjustments. Only after the operator finally confirms the work area and points will an execution signal be issued, thereby triggering the subsequent trajectory planning and control process.

[0044] In one embodiment, the generated job points are represented as a set of spatial coordinate points and transmitted back in graphical form.

[0045] Step 103: Receive the reoperation information from the operator and confirm the target work point within the selected target candidate work area.

[0046] Once the operator confirms or modifies the target work area at the ground station, the onboard computer will automatically generate a target work point layout within the selected candidate work area. In one embodiment, the generated target work points are represented as a set of spatial coordinate points, and the result is visualized and transmitted back to the ground station for the operator to make final confirmation or further adjustments. The target work points are then obtained after the operator's confirmation.

[0047] The automatic work point layout method for irregular work surfaces in steps 102 and 103 of this application adopts a layered sampling and local feasibility correction strategy to achieve uniform distribution of work points and adaptive adjustment of safety constraints.

[0048] In one embodiment, generating a target work point layout within a selected candidate work area may include: first, generating an initial work point grid using Poisson-disk sampling based on the geometric boundaries of the candidate work area to ensure that the distance between adjacent points meets a set minimum safety distance; then, performing normal vector and curvature checks on each initial work point; if the angle with the drilling direction exceeds a threshold or the point is located in an obstacle shadow area, local point resampling or removal is performed; finally, a main work point set and a backup work point set are formed, and each work point is assigned a priority score based on reachability and efficiency. This method introduces a two-layer layout strategy of macro-uniformity + micro-correction and generates a work point database containing priority information, realizing the automation of work tasks and human-machine co-planning.

[0049] Step 104: Transform the confirmed target work point from the camera coordinate system to the robot arm base coordinate system.

[0050] In one exemplary instance, the onboard computer converts the confirmed target work point from the camera coordinate system to the robot arm base coordinate system in real time to ensure that the work point is consistent with the robot arm's control coordinate system.

[0051] Step 105: Perform path planning on the transformed target work points to generate an obstacle avoidance path point sequence.

[0052] In one exemplary instance, after completing the coordinate transformation, the onboard computer can invoke the trajectory planning module to plan a path to the transformed target work point, generating a sequence of obstacle avoidance path points. In one embodiment, when generating the trajectory point sequence, constraints such as position, attitude, and speed must be met to ensure the smoothness and executability of the trajectory.

[0053] Steps 104 and 105 of this application combine a multi-layer path planning method that integrates global coverage and local obstacle avoidance, and generate a continuous, smooth, and executable sequence of job path points through heuristic task order optimization and sampling obstacle avoidance algorithm.

[0054] In one embodiment, step 105 may include: At the global level, the task points are treated as a coverage path optimization problem, and optimal access sequences can be generated using algorithms such as the improved Traveling Salesman Problem (TSP) or heuristic coverage path planning algorithms. At the local level, obstacle avoidance trajectories can be generated in a 3D environment grid using algorithms such as Rapid Expanding Random Tree (RRT) or Covariant Hamiltonian Optimized Motion Planning (CHOMP), and trajectory smoothing and time parameterization are performed through spline interpolation. The inverse kinematics solver of the robotic arm is invoked synchronously during the planning process to ensure that each transition posture is solvable and collision-free. When dynamic obstacles are detected in the work environment, online replanning can be performed based on a local cost map to generate the obstacle avoidance path point sequence that meets the obstacle avoidance requirements. This method integrates a path planning system that combines robotic arm reachability and work efficiency constraints, enabling real-time generation of safe and efficient obstacle avoidance paths in complex environments.

[0055] In another embodiment, step 105 can also be implemented based on an adaptive trajectory planning method with dynamic constraints. In this embodiment, the selected work points are first screened according to the dynamic parameters of the robotic arm, including determining the dynamic feasibility of each candidate work point based on the maximum joint torque, maximum angular velocity, and the current load of the robotic arm. Work points that require a large range of rotation under heavy load conditions or may cause joint overload are eliminated, thus retaining a relatively concise set of feasible work points that all meet the dynamic constraints. For example, work points with rotation angles > 80° are eliminated under heavy load, retaining 10-15 feasible work points. Then, using the selected work points as key nodes, a work trajectory satisfying dynamic constraints is generated. The trajectory curve is constructed using a B-spline interpolation method with dynamic constraints. The overall smoothness of the trajectory and minimization of joint load are used as optimization objectives. By limiting joint angular velocity to no more than 30° / s, joint torque to no more than 90% of the rated value, and maintaining the interpolation node spacing at 3-5mm (which can be further compressed to 3mm under heavy load conditions), the trajectory, while maintaining geometric continuity, further meets the dynamic safety requirements of the robotic arm during actual execution. In other words, the B-spline interpolation method with dynamic constraints generates a trajectory curve that satisfies the constraints of joint angular velocity, joint torque, and trajectory curvature change rate. During this process, the trajectory curvature change rate is also constrained to not exceed 0.3 rad / m. 2 To reduce the instantaneous load impact on the robotic arm joints caused by high curvature segments in the trajectory, a local pre-obstacle avoidance adjustment is performed based on an artificial potential field method when the trajectory approaches a high-risk obstacle area. This ensures a preset safe distance between the trajectory and obstacles. Specifically, when the planned trajectory segment traverses an area with dense obstacles or potential interference risks, a local pre-obstacle avoidance adjustment based on an artificial potential field method is performed. When the trajectory is detected to be approaching a high-risk area, a rapid replanning mechanism is triggered. By constructing a repulsive potential energy field, the trajectory is shifted to a safe passage within a local range, thereby ensuring a safe distance of at least 0.4m between the trajectory and obstacles. Small-scale trajectory correction is performed without changing the global access order. That is, the planned trajectory is corrected online based on real-time feedback to generate an obstacle avoidance path sequence that meets dynamic and safety constraints. This local replanning process has a real-time response capability of no more than 0.2s, enabling the system to maintain high trajectory stability and execution safety under dynamic or complex environmental conditions. In this embodiment, by introducing the aforementioned dynamic screening, dynamic constraint interpolation, and real-time obstacle avoidance adjustment, an executable trajectory that simultaneously satisfies geometric constraints, dynamic constraints, and safety constraints can be generated in irregular working surfaces, heavy-load conditions, and dynamic disturbance environments, further improving the reliability, robustness, and engineering feasibility of path planning.

[0056] Step 106: Convert the path planning results into joint motion control information for the hydraulic robotic arm.

[0057] In one exemplary instance, the path planning result is converted into joint motion control information for the hydraulic robotic arm and sent to a wire-controlled unit at a specific frequency to drive a multi-way valve to achieve precise movements of the robotic arm. Thus, the wire-controlled unit can control the opening of the hydraulic multi-way valve based on the received trajectory commands, thereby driving the hydraulic cylinder to achieve automatic and precise movement of the robotic arm.

[0058] In one embodiment, step 106 can further convert the trajectory point sequence into current or voltage signals through PID control and feedforward compensation to ensure a smooth response of the hydraulic actuator. Regarding trajectory control, this embodiment automatically maps the work points from the camera coordinate system to the robot arm base coordinate system, avoiding manual conversion errors; and introduces constraints such as position, attitude, and speed during trajectory planning to ensure the smoothness and executability of the generated trajectory. Simultaneously, by combining PID control and feedforward compensation, the trajectory point sequence is converted into current or voltage signals to drive the hydraulic actuator, ensuring the stability and accuracy of the robot arm's movement.

[0059] Step 106 of this application provides a method for generating motion control information for a hydraulic robotic arm, which achieves trajectory-level precise control through inverse kinematics solution and hydraulic dynamics feedforward compensation.

[0060] In one embodiment, step 106 may specifically include: performing inverse kinematics calculation on the spatial trajectory in the path planning result to obtain the sequence of joint angles, angular velocities, and angular accelerations; calculating the corresponding target valve control current or PWM control signal based on the hydraulic system valve orifice flow-pressure difference model and hydraulic cylinder dynamic equations, and executing it using a combination of frequency scheduling feedforward control and adaptive feedback control. Furthermore, pressure disturbance estimation based on an extended state observer can be introduced into the control algorithm to achieve real-time correction of hydraulic hysteresis and leakage errors. This method, with its feedforward-adaptive composite control structure designed for the frequency domain characteristics of the hydraulic system, significantly improves high-frequency response stability and low-frequency tracking accuracy.

[0061] In one embodiment, step 106 can employ a closed-loop control strategy based on hydraulic nonlinear model predictive control (MPC) to further improve the control accuracy and response speed of the hydraulic manipulator under complex load conditions and dynamic environments. In this embodiment, a nonlinear dynamic model of the hydraulic manipulator is first established, as shown in equation (1): (1) In formula (1), Represents the inertia matrix. Represents the Coriolis force and centrifugal force terms. Indicates the gravity compensation term. This represents the hydraulic gain matrix as a function of the joint angle. This indicates the drive current of the hydraulic multi-way valve. The external disturbance torque is used. Based on the model shown in formula (1), it is assumed that the MPC controller uses a time range of about 0.3 seconds as the prediction window, minimizes the trajectory deviation and smooths the control quantity as the optimization objectives, and uses the current range (e.g., 0-20mA) and joint speed deviation (e.g., not exceeding 5° / s) as constraints. The optimal valve control current command for future control moments is obtained through quadratic programming. The feedback data of the joint sensor used to measure the state of each joint of the robot is collected at a frequency of 100Hz, and the MPC parameters are dynamically adjusted according to the real-time trajectory deviation. When the trajectory deviation is detected to exceed 0.5mm, the hydraulic gain can be automatically increased or the prediction model can be adjusted under heavy load conditions to keep the control law consistent with the actual hydraulic response, and finally achieve high-precision closed-loop control with a trajectory tracking deviation not exceeding 0.3mm. This implementation method introduces nonlinear dynamic modeling, forward optimization and high-speed feedback correction, so that the hydraulic robot arm can still maintain stable, fast and high-precision trajectory tracking capability under high power load, long stroke and dynamic disturbance environment.

[0062] In one exemplary instance, step 106 may also include: Based on the flow-pressure difference model of the hydraulic valve port and the dynamic response of the hydraulic cylinder, frequency scheduling and nonlinear compensation are performed on the joint motion control signal to obtain the compensated target control quantity, thereby improving the trajectory following accuracy; correspondingly, step 106 may also include: frequency scheduling and signal compensation of the target control quantity based on the nonlinear characteristics of the hydraulic system in which the hydraulic robot arm is located, to generate joint motion control information suitable for hydraulic multi-way valve drive, thereby improving the stability of trajectory following.

[0063] In one exemplary instance, to ensure the safety and reliability of the operation during trajectory planning and execution, the following may also be included: Step 107: Optimize and monitor the end-to-end control link latency for remote operation of hydraulic robotic arms.

[0064] In one embodiment, the end-to-end latency of the video transmission link can be controlled to within, for example, 180 milliseconds (ms), and the latency of the control feedback link can be controlled to within, for example, 20 ms, thereby ensuring real-time performance between remote operation and execution. This enables the system to support obstacle avoidance and safe operation in complex mining environments, improving the overall stability and adaptability of the operation. Regarding safety, this embodiment strictly controls the end-to-end communication link latency during trajectory planning and execution, with the video transmission link latency controlled to within 180 milliseconds and the control feedback link latency controlled to within 20 milliseconds, thus ensuring real-time performance between remote operation and execution. Furthermore, the system supports obstacle detection and avoidance in complex mining environments, preventing risks caused by environmental complexity or operator misjudgment, and significantly improving the overall safety of the operation.

[0065] Step 107 of this application describes an end-to-end control link delay optimization and monitoring method for remote operation of hydraulic robotic arms. This method combines delay measurement, predictive display, and model predictive control to achieve stable operation in high-latency environments. In one embodiment, a timestamp mechanism can be embedded in each stage of sensing, communication, and execution to sample and statistically analyze link delay, jitter, and packet loss rate in real time. When the detected delay exceeds a preset threshold, a delay compensation mechanism is automatically triggered. In some embodiments, this mechanism may include, but is not limited to, enabling predictive display at the ground station, predicting the robotic arm posture within a preset timeframe (e.g., 100–200 milliseconds) based on a motion model, and rendering the image; and / or employing a model predictive control (MPC) algorithm with a delay prediction term at the robot end to compensate for the lead time of control commands; and / or automatically switching to a local safety mode to ensure equipment safety when the link is interrupted or the delay is too large. This method is an end-to-end delay adaptive system combining predictive display, delay compensation control, and local bypass protection, achieving stable and reliable control for remote hydraulic operations.

[0066] The method for trajectory control of a hydraulic robotic arm provided in this application uses a depth camera to perceive the target work area and combines it with artificial intelligence algorithms to divide the work area and label obstacles, achieving tight coupling between environmental modeling and task planning. It enables the rapid generation of multiple candidate work areas and their transmission back to the ground station, allowing operators to simply select or correct them intuitively, significantly reducing manual operations and thus greatly improving work efficiency. This application not only improves the trajectory control accuracy and work efficiency of the hydraulic robotic arm but also reduces the reliance on human experience, enhances the user-friendliness of human-machine interaction, and ensures the safety and reliability of the system in complex mining environments, demonstrating good application value and promising prospects for widespread adoption.

[0067] This application also provides a computer-readable storage medium storing computer-executable instructions for performing the method for implementing trajectory control of a hydraulic robotic arm as described in any of the preceding claims.

[0068] This application further provides a computer device, including a memory and a processor, wherein the memory stores the following instructions executable by the processor: steps for performing the method for implementing the trajectory control of a hydraulic robotic arm as described in any of the preceding claims.

[0069] This application also provides a device for implementing trajectory control of a hydraulic robotic arm, such as... Figure 2 As shown, the device installed at the ground station end can include: The UI interaction module displays the depth camera view of the target work area and provides an interface for selecting the area and confirming or adjusting work points. Furthermore, it displays a 360° panoramic view of the target work area, a front-view camera view, and shows robot status information such as battery level, signal quality, joint angles, and work progress.

[0070] The operation input parsing module is used to parse input commands from the handle and convert them into standardized movement, operation, or confirmation control commands; The remote communication management module is used to encapsulate and decapsulate control commands, such as the Mavlink protocol, to issue commands and receive status information, while also having the ability to handle exceptions and packet loss. The video receiving and display module is used to receive video streams pushed by the robot, such as Real-Time Streaming Protocol (RTSP) video streams, and to achieve real-time display by combining caching and low-latency processing strategies.

[0071] This application also provides a device for implementing trajectory control of a hydraulic robotic arm, such as... Figure 2 As shown, the device set on the robot end can include: The perception processing module is used to collect depth camera data and identify the work area and obstacles through artificial intelligence algorithms, while performing filtering and coordinate transformation preprocessing on the original point cloud or image. The task planning module is used to generate candidate task points based on the identified task area and form a set of spatial coordinate points. At the same time, the results are visualized and sent back to the ground station. The robotic arm trajectory planning module is used to convert spatial coordinate points into a sequence of trajectory points, perform trajectory interpolation, and combine obstacle avoidance calculations and redundant joint calculations. The hydraulic control module is used to convert the trajectory point sequence into joint motion commands, and generate current or voltage signals through PID and feedforward compensation methods, thereby controlling the multi-way valve to achieve precise action of the hydraulic cylinder. The remote communication management module is used to receive and parse instructions from the ground station, such as Mavlink instructions, package robot status information and send it back, and also works in conjunction with the video transmission module. The video transmission module is used to push camera video to the ground station in the form of a video stream (such as an RTSP video stream), and combines caching and low-latency processing to ensure real-time communication.

[0072] The device for trajectory control of a hydraulic robotic arm provided in this application uses a depth camera to perceive the target work area and combines it with artificial intelligence algorithms to divide the work area and mark obstacles, achieving tight coupling between environmental modeling and task planning. It enables the rapid generation of multiple candidate work areas and their transmission back to the ground station, allowing operators to simply select or correct them intuitively, significantly reducing manual operations and thus greatly improving work efficiency. This application not only improves the trajectory control accuracy and work efficiency of the hydraulic robotic arm but also reduces the reliance on human experience, enhances the user-friendliness of human-machine interaction, and ensures the safety and reliability of the system in complex mining environments, demonstrating good application value and promising prospects for widespread adoption.

[0073] In one exemplary instance, the robotic arm trajectory planning module is further configured to: before the control signal generated by trajectory planning is sent to the hydraulic control module, perform frequency scheduling and nonlinear compensation on the control signal based on the flow-pressure difference model of the hydraulic valve port and the dynamic response of the hydraulic cylinder, so as to improve the trajectory following accuracy; correspondingly, the hydraulic control module is further configured to: perform frequency scheduling and signal compensation based on the nonlinear characteristics of the hydraulic system, so as to improve the stability of trajectory following.

[0074] In one embodiment, when generating trajectory commands, the onboard computer combines the flow-pressure difference model of the hydraulic valve port with the dynamic response characteristics of the hydraulic cylinder to perform frequency scheduling and nonlinear compensation on the control signal after trajectory interpolation. By combining dynamic adjustment of the signal frequency with feedforward compensation, the nonlinear hysteresis effect in the hydraulic system can be effectively suppressed, improving the accuracy and stability of trajectory following.

[0075] In one exemplary embodiment, the sensing and processing module includes an explosion-proof depth camera to meet the safety requirements of complex mining environments such as coal mines and tunnels. The sensing and processing module is further configured to: employ an explosion-proof depth camera to meet the safety requirements of special operating environments such as coal mines and tunnels, thereby improving the safety and adaptability of the system in mining environments. The explosion-proof depth camera can operate stably in high-dust and low-light environments and avoids sparks or electrical hazards, thus ensuring the adaptability and safety of the system in complex mining environments.

[0076] Figure 2The provided hydraulic robotic arm trajectory control system consists of two main parts: a ground station and the equipment (i.e., the robot end), which can interact via a communication link. The system is designed in layers, including a human-machine interaction layer, a transmission and communication layer, a perception layer, a decision-making layer, and a control layer.

[0077] At the ground station, in the human-machine interface (HMI) layer, the UI module displays a 360° surround view of the target work area, along with images from the front-view and depth cameras, and shows robot status information (such as battery level, signal strength, joint angles, and work progress). It also provides an interactive interface for area selection and work point confirmation / adjustment. The operation input / output module parses the operator's commands issued via input devices such as handles and converts them into standardized movement, operation, or confirmation commands. In the transmission and communication layer, the video receiving and display module receives the RTSP video stream pushed from the robot and, using caching and low-latency processing, displays the video in real time. The remote communication management module uses the MAVLink protocol to encapsulate / decapsulate control commands, enabling command issuance and robot status information reception, and supports anomaly detection and packet loss handling.

[0078] On the device side (robot side), at the transmission and communication layer, the video sending module encapsulates the camera images captured by the device into an RTSP video stream and pushes it to the ground station, while using caching and low-latency processing to ensure real-time performance. The remote communication management module receives and parses remote control commands from the ground station and packages the robot's status information for transmission back. At the perception layer, the perception processing module collects depth camera data and uses artificial intelligence algorithms to detect the work area and identify obstacles, while performing filtering and coordinate transformation preprocessing on the raw point cloud or image data. At the decision layer, the work point planning module generates candidate work points within the identified work area, forming a set of spatial coordinate points, visualizes the results, and transmits them back to the ground station for confirmation or correction. The robotic arm trajectory planning module converts the work points into a sequence of trajectory points, performs trajectory interpolation, and performs obstacle avoidance calculations and redundant joint calculations. At the control layer, the hydraulic control module converts the sequence of trajectory points into joint motion commands and generates current or voltage signals through PID regulation and feedforward compensation to control the hydraulic multi-way valve to drive the hydraulic cylinder, thereby achieving precise movements of the robotic arm.

[0079] from Figure 2As shown in the system operation process, in this embodiment, the ground station is responsible for information display and human decision-making, while the device (robot) is responsible for environmental perception, trajectory planning, and hydraulic execution. The device's perception and processing module collects environmental data, generates work points, and feeds them back to the ground station via the video and data feedback module. The ground station's UI interaction module displays candidate work areas and work points, and the operator selects or corrects them through the operation input module. The corrected instructions are transmitted to the device via the remote communication management module, where the work point planning and trajectory planning module generates the path. Finally, the hydraulic control module converts the trajectory into control signals, driving the robotic arm to complete the action.

[0080] Figure 3 The system operation process is demonstrated, such as Figure 3 As shown, the ground station is primarily used for human-machine interaction and remote command issuance. The operator generates control signals using a control handle (such as a combination of joystick and buttons). These signals are input to the ground station computer, which converts them into standardized control commands and encapsulates them into MAVLink data packets. These data packets are then sent to the device via a data transmission module, enabling the issuance of control commands. Simultaneously, the ground station computer receives video streams from the device and decodes and processes them via a wireless video transmission module. The video data is then played back on the display screen, providing the operator with a real-time view of the work environment. The operator can intuitively observe the environment and work status through the display screen and complete the operation loop by combining the input from the control handle.

[0081] The equipment primarily handles environmental perception, trajectory planning, and hydraulic execution. This includes: A hydraulic actuator: supplied with hydraulic oil by a hydraulic pump station, controlled via a multi-way valve to drive the hydraulic robotic arm. The opening of the multi-way valve is adjusted by electrical signals received by the drive-by-wire unit, thus achieving precise motion control of the hydraulic cylinder. Sensing and perception section: The equipment is equipped with a depth camera to collect depth and image data of the environment and transmit the perception results back to the onboard computer. Onboard computer: Processes data from the depth camera, performing functions such as work area detection, trajectory planning, and motion generation. Simultaneously, the onboard computer receives MAVLink control commands from the ground station and converts them into corresponding hydraulic control signals, outputting them to the drive-by-wire unit. Communication and video transmission: The onboard computer packages its generated control information into MAVLink data packets and transmits them back to the ground station via a data transmission module; simultaneously, video data collected by the depth camera is encoded into a video stream via a wireless video transmission module and transmitted back, enabling real-time visualization of the work site.

[0082] like Figure 3As shown, during system operation, the operator issues operating commands via a handle, which are encapsulated by the ground station computer and transmitted to the device via a wireless link. After the on-board computer parses the control commands, it drives the hydraulic control unit to adjust the multi-way valve, thereby enabling the hydraulic robotic arm to perform its actions. At the same time, environmental video and image data collected by the depth camera on the device are transmitted back to the ground station in real time via a wireless video transmission module and displayed on the screen, thus forming a closed-loop control system of remote control, hydraulic execution, and video feedback.

[0083] The method for implementing trajectory control of a hydraulic robotic arm provided in this application is described in detail below with reference to an embodiment.

[0084] like Figure 4 As shown, after the equipment enters the designated work area, the onboard computer uses the environmental depth and image data collected by the depth camera, combined with artificial intelligence recognition algorithms, to analyze the target work area, thereby automatically dividing it into multiple candidate work areas, such as... Figure 5 Regions A, B, and C are shown in the image. The resulting division is overlaid on the environmental image using image annotations and transmitted back to the ground station computer in real time via a wireless video transmission module.

[0085] like Figure 5 As shown, the operator selects or modifies the target work area through the ground station interface. For example, the user can select area A using a handle and send the result back to the onboard computer via the data transmission module. The onboard computer automatically generates a work point layout within the selected target work area, forming a set of spatial coordinate points, and displays the generated result as a graphical overlay, such as the work point layout shown by the green dots in the figure. The result is then transmitted back to the ground station for manual confirmation by the operator.

[0086] like Figure 6 As shown, the operator confirms or modifies the generated work point layout on the ground station interface and sends an execution signal after final confirmation. Upon receiving the execution signal, the onboard computer first converts the work points from the camera coordinate system to the robotic arm base coordinate system and calls the trajectory planning module to generate an obstacle avoidance path point sequence. The trajectory planning result is converted into joint motion control information for the hydraulic robotic arm and sent to the drive-by-wire unit at a specific frequency. The drive-by-wire unit generates electrical signals based on the received motion control information, driving the multi-way valve opening for adjustment. The hydraulic system responds to the above control signals, controlling the hydraulic cylinders to move, thereby driving the hydraulic robotic arm to achieve automated and precise movement.

[0087] This embodiment realizes a complete closed-loop process from environmental perception, work area division, and manual confirmation to trajectory planning and hydraulic execution, which effectively improves the operating efficiency and control accuracy of the hydraulic robotic arm in complex working environments, while reducing the operator's workload.

[0088] Although the embodiments disclosed in this application are as described above, the content described is merely for the purpose of understanding this application and is not intended to limit this application. Any person skilled in the art to which this application pertains may make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in this application; however, the scope of patent protection of this application shall still be determined by the scope defined in the appended claims.

Claims

1. A method for trajectory control of a hydraulic robotic arm, characterized in that, include: Acquire depth and image data of the target work area, and divide the target work area into walkable areas and label obstacle areas to obtain a target work area model; Multiple candidate work areas are divided on the target work area model, and the divided candidate work areas are overlaid on the target work area screen in the form of images for remote operation by the operator. Receive operation information from the operator and generate a work point layout within the selected target candidate work area for the operator to perform remote operation again; Receive further operation information from the operator and confirm the target operation point within the selected target candidate operation area; Transform the confirmed target work point from the camera coordinate system to the robot arm base coordinate system; Perform path planning on the transformed target work points to generate an obstacle avoidance path point sequence; The path planning results are converted into joint motion control information for the hydraulic robotic arm.

2. The method according to claim 1, further comprising, before converting the path planning result into joint motion control information of the hydraulic robotic arm: Based on the flow-pressure difference model of the hydraulic valve orifice and the dynamic response of the hydraulic cylinder, frequency scheduling and nonlinear compensation are performed on the joint motion control information to obtain the compensated target control quantity. The process of converting the path planning results into joint motion control information for the hydraulic robotic arm also includes: Frequency scheduling and signal compensation of the target control quantity are performed based on the nonlinear characteristics of the hydraulic system in which the hydraulic robotic arm is located.

3. The method according to claim 1 or 2, further comprising: The end-to-end control link latency for remote operation of the hydraulic robotic arm is optimized and monitored.

4. The method according to claim 1 or 2, wherein, The process of dividing the target work area into walkable areas and labeling obstacle areas to obtain a target work area model includes: The target work area is perceived using a depth camera to obtain depth data and image data of the target work area; The acquired data is preprocessed; The target work area is identified using artificial intelligence (AI) algorithms or based on the Transformer recognition framework using multimodal visual fusion. Obstacles in the environment are detected using data from a depth camera. The target work area is then divided into walkable areas and obstacle areas are labeled to obtain a model of the target work area.

5. The method according to claim 4, wherein, The process of dividing the target work area model into multiple candidate work areas includes: Based on the data collected by the depth camera and the artificial intelligence recognition algorithm, the target work area is analyzed and the candidate work areas are divided.

6. The method according to claim 5, wherein, The remote operation includes selection or correction.

7. The method according to claim 1, wherein, Path planning is performed on the transformed target work points to generate an obstacle avoidance path point sequence, including: At the global level, the selected job points are treated as a coverage path optimization problem, and the optimal access order of job points is generated based on the improved Traveling Salesman Problem (TSP) algorithm or a heuristic coverage path planning algorithm. At the local level, the Fast Extended Random Tree (RRT) algorithm or the Covariant Hamiltonian Optimization Motion Planning (CHOMP) algorithm is used to generate obstacle avoidance trajectories in the 3D environment grid, and spline interpolation is performed on the generated obstacle avoidance trajectories to achieve trajectory smoothing and time parameterization. During the path planning process, the inverse kinematics solver of the robotic arm is called to ensure that each transition posture is reachable and does not cause self-collision. When a dynamic obstacle is detected, the trajectory is replanned online based on a local cost map to generate the obstacle avoidance path point sequence that meets the obstacle avoidance requirements.

8. The method according to claim 1, wherein, Path planning is performed on the transformed target work points to generate an obstacle avoidance path point sequence, including: Based on the dynamic parameters of the robotic arm, the selected work points are screened for dynamic feasibility, and work points that require exceeding the preset joint rotation angle or may cause joint overload are eliminated. A B-spline interpolation method with dynamic constraints is used to generate trajectory curves that satisfy the constraints of joint angular velocity, joint torque, and trajectory curvature change rate. When the trajectory approaches a high-risk area of ​​obstacles, the trajectory is locally adjusted for obstacle avoidance based on the artificial potential field method to ensure that the trajectory and obstacles maintain a preset safe distance. The planned trajectory is corrected online based on real-time feedback to generate the obstacle avoidance path point sequence that satisfies dynamic and safety constraints.

9. The method according to claim 1, wherein, The process of converting path planning results into joint motion control information for a hydraulic robotic arm includes: The trajectory displayed by the path planning results is subjected to inverse kinematics solution to obtain the sequence of angles, angular velocities, and angular accelerations of each joint; based on the valve control characteristic model of the hydraulic system, the joint motion quantities are converted into valve control current or PWM control signals, and the joint motion control information is obtained through a combination of feedforward compensation and PID regulation to drive the hydraulic multi-way valve; or... A nonlinear dynamic model of the hydraulic manipulator is established to predict the future motion state of the hydraulic manipulator. Based on the nonlinear dynamic model, the predictive control optimizes the trajectory deviation and control quantity changes within a preset prediction window, and obtains the joint motion control information under the conditions of satisfying the current range and joint speed deviation constraints.

10. A computer-readable storage medium storing computer-executable instructions for performing the method for implementing trajectory control of a hydraulic robotic arm as described in any one of claims 1-9.

11. A device for realizing trajectory control of a hydraulic robotic arm, characterized in that, Located at the ground station, it includes: a UI interaction module, an operation input parsing module, a remote communication management module, and a video receiving and display module; among which, The UI interaction module is used for depth camera images and provides an operation interface for area selection and work point confirmation or adjustment. The operation input parsing module is used to parse input commands from the handle and convert them into standardized movement, operation, or confirmation control commands; The remote communication management module is used to encapsulate and decapsulate control commands in order to issue commands and receive status information. The video receiving and display module is used to receive video streams pushed by the robot and achieve real-time display by combining caching and low-latency processing strategies.

12. The device according to claim 11, wherein the UI interaction module is further configured to display a 360° panoramic view of the target work area, a front-view camera view, and display robot status information such as battery level, signal quality, joint angle, and work progress.

13. A device for realizing trajectory control of a hydraulic robotic arm, characterized in that, The components installed on the robot include: a perception and processing module, a work point planning module, a robotic arm trajectory planning module, a hydraulic control module, a remote communication management module, and a video transmission module; among which, The perception processing module is used to collect depth camera data and identify the work area and obstacles through artificial intelligence algorithms, while performing filtering and coordinate transformation preprocessing on the original point cloud or image. The task planning module is used to generate candidate task points based on the identified task area and form a set of spatial coordinate points. At the same time, the results are visualized and sent back to the ground station. The robotic arm trajectory planning module is used to convert spatial coordinate points into a sequence of trajectory points; The hydraulic control module is used to convert the trajectory point sequence into joint motion commands, and generate current or voltage signals through PID and feedforward compensation, thereby controlling the multi-way valve to realize the action of the hydraulic cylinder. The remote communication management module is used to receive and parse instructions from the ground station, package robot status information, and send it back. The video sending module is used to push camera video to the ground station as a video stream.

14. The device according to claim 13, wherein the robotic arm trajectory planning module is further configured to: before the control signal generated by trajectory planning is sent to the hydraulic control module, perform frequency scheduling and nonlinear compensation on the joint motion control information based on the flow-pressure difference model of the hydraulic valve port and the dynamic response of the hydraulic cylinder, so as to obtain the compensated target control quantity; The hydraulic control module is also used for: frequency scheduling and signal compensation of the target control quantity based on the nonlinear characteristics of the hydraulic system.

15. The apparatus according to claim 13 or 14, wherein the sensing processing module includes an explosion-proof depth camera.

Images