Intelligent control method and system for coal mine underground hydraulic drive movable type mechanical arm

Abstract

The invention provides an intelligent control method and system for an underground coal mine hydraulic drive movable type mechanical arm. The method comprises the steps that multi-mode sensing data are obtained; according to the multi-mode sensing data, an initial control instruction of a mechanical arm maneuvering unit is determined; compensating the initial control instruction to obtain a first control instruction of the mechanical arm maneuvering unit; the simulation mechanical arm maneuvering unit is simulated based on the first control instruction, and a safety detection result of the simulation mechanical arm maneuvering unit in the simulation process is obtained; correcting the first control instruction to obtain a target control instruction of the mechanical arm maneuvering unit in response to the safety detection result indicating that the risk exists; and the mechanical arm maneuvering unit is controlled based on the target control instruction. Therefore, the autonomous operation capacity of the mechanical arm in the unstructured roadway is improved, the control precision of the mechanical arm maneuvering unit is improved, and the safety, reliability and efficiency of the mechanical arm maneuvering unit in the operation process are enhanced.

Description

Technical Field

[0001] This invention relates to the field of coal mining equipment technology, and in particular to an intelligent control method and system for a hydraulically driven mobile robotic arm in underground coal mines. Background Technology

[0002] Coal mining in my country is primarily underground, characterized by harsh underground environments, cramped spaces, insufficient lighting, pervasive dust, and dynamic obstacles. Currently, heavy-duty operations such as tunnel support, equipment installation and maintenance, and material handling still heavily rely on manual labor, resulting in high safety risks and low efficiency. The application of robots to replace manual labor is a trend in the intelligent development of coal mines.

[0003] In coal mine special robots, hydraulically driven mobile robotic arms are the preferred choice for heavy-duty tasks due to their high power density and strong impact resistance. However, when the robotic arm is mounted on a mobile chassis, the chassis motion and the boom motion are strongly dynamically coupled, leading to decreased accuracy and trajectory oscillations in traditional independent control methods. Furthermore, the inherent valve nonlinearity, pressure pulsation, and external load disturbances of the hydraulic system are amplified under complex underground working conditions, severely affecting the stability and accuracy of control. Related technologies often rely on preset trajectories and lack the ability to perceive and autonomously adapt to real-time changes in unstructured environments such as unknown obstacles and tunnel deformation, resulting in insufficient intelligence. Summary of the Invention

[0004] This application aims to at least partially address one of the technical problems in the related art.

[0005] According to a first aspect of this application, a method for intelligent control of a hydraulically driven mobile robotic arm in a coal mine is provided, comprising: acquiring multimodal perception data, wherein the multimodal perception data includes three-dimensional point cloud data of the roadway, obstacle information within the roadway, image data of the work target, and body state data of the robotic arm's mobile unit; determining an initial control command for the robotic arm's mobile unit based on the multimodal perception data; compensating the initial control command to obtain a first control command for the robotic arm's mobile unit; simulating the robotic arm's mobile unit based on the first control command, and obtaining a safety detection result of the simulated robotic arm's mobile unit during the simulation process; in response to the safety detection result indicating a risk, correcting the first control command to obtain a target control command for the robotic arm's mobile unit; and controlling the robotic arm's mobile unit based on the target control command.

[0006] According to a second aspect of this application, an intelligent control system for a hydraulically driven mobile robotic arm in a coal mine is provided, comprising: a first acquisition module for acquiring multimodal perception data, wherein the multimodal perception data includes three-dimensional point cloud data of the roadway, obstacle information within the roadway, image data of the work target, and body state data of the robotic arm's mobile unit; a determination module for determining an initial control command for the robotic arm's mobile unit based on the multimodal perception data; a second acquisition module for compensating the initial control command to acquire a first control command for the robotic arm's mobile unit; a simulation module for simulating the robotic arm's mobile unit based on the first control command and acquiring a safety detection result of the simulated robotic arm's mobile unit during the simulation process; a correction module for correcting the first control command in response to the safety detection result indicating a risk, thereby obtaining a target control command for the robotic arm's mobile unit; and a control module for controlling the robotic arm's mobile unit based on the target control command.

[0007] According to a third aspect of the present application, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the instructions to implement the intelligent control method for a hydraulically driven mobile robotic arm in a coal mine as described in the first aspect of the present application.

[0008] According to a fourth aspect of the present application, a computer-readable storage medium is provided, which, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the intelligent control method for a hydraulically driven mobile robotic arm in a coal mine as described in the first aspect of the present application.

[0009] The intelligent control method and system for a hydraulically driven mobile robotic arm in coal mines provided in this application acquires multimodal sensing data, determines the initial control command for the robotic arm's mobile unit based on the multimodal sensing data, compensates for the initial control command to obtain the first control command for the robotic arm's mobile unit, simulates the robotic arm's mobile unit based on the first control command, obtains the safety detection results of the simulated robotic arm's mobile unit during the simulation process, and corrects the first control command in response to the safety detection results indicating a risk, thereby obtaining the target control command for the robotic arm's mobile unit. Based on the target control command, the robotic arm's mobile unit is controlled. Therefore, by determining the target control command and controlling the robotic arm's mobile unit based on the target control command, this application improves the autonomous operation capability of the robotic arm in unstructured roadways, enhances the control accuracy of the robotic arm's mobile unit, and strengthens the safety, reliability, and efficiency of the robotic arm's mobile unit during operation.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided for a better understanding of this solution and do not constitute a limitation of this application. Wherein: Figure 1 A flowchart illustrating an intelligent control method for a hydraulically driven mobile robotic arm in a coal mine, provided in an embodiment of this application; Figure 2 A flowchart illustrating an intelligent control method for a hydraulically driven mobile robotic arm in a coal mine, provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an intelligent control device for a hydraulically driven mobile robotic arm in a coal mine, provided as an embodiment of this application.

[0012] Figure 4 This is a schematic diagram of the structure of an intelligent control system for a hydraulically driven mobile robotic arm in a coal mine, provided as an embodiment of this application.

[0013] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0014] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0015] The following embodiments provide a detailed description of the intelligent control method for a hydraulically driven mobile robotic arm in coal mines according to this application.

[0016] Figure 1 This is a flowchart illustrating the intelligent control method for a hydraulically driven mobile robotic arm in a coal mine, as provided in an embodiment of this application.

[0017] like Figure 1 As shown in this embodiment, the intelligent control method for a hydraulically driven mobile robotic arm in coal mines specifically includes the following steps: S101. Acquire multimodal perception data, including three-dimensional point cloud data of the tunnel, obstacle information in the tunnel, image data of the working target, and body state data of the robotic arm's motor unit.

[0018] In this embodiment of the application, the robotic arm motor unit includes: a robotic arm, a mobile transport platform associated with the robotic arm, an electro-hydraulic servo valve group, a hydraulic cylinder, a hydraulic pressure and flow sensor, and a hydraulic power source.

[0019] It should be noted that this application does not limit the specific method for acquiring multimodal sensing data, and the appropriate method can be selected according to the actual situation.

[0020] Optionally, multimodal sensing data can be continuously collected based on the multimodal fusion sensing unit deployed on the mobile platform and robotic arm to obtain multimodal sensing data.

[0021] For example, a multimodal fusion sensing unit is constructed based on explosion-proof solid-state lidar, explosion-proof millimeter-wave radar, joint absolute encoder, joint torque sensor, platform inertial measurement unit, explosion-proof tilt sensor, and intrinsically safe binocular infrared camera for mining. The explosion-proof solid-state lidar collects high-precision three-dimensional point cloud data of the roadway, the millimeter-wave radar obtains the point traces and speed information of dynamic obstacles (personnel, mine cars, etc.) in the roadway, the intrinsically safe binocular infrared camera for mining obtains image data of the work target, and the joint absolute encoder, joint torque sensor, and platform inertial measurement unit obtain body state data such as the joint angle of the robotic arm, hydraulic cylinder pressure, and the pose and speed of the mobile transport platform.

[0022] In this embodiment of the application, after acquiring multimodal sensing data, the multimodal sensing data is preprocessed to obtain target sensing data, and a global map of the tunnel is constructed based on the target sensing data.

[0023] Optionally, preprocessing operations such as data cleaning and calibration can be performed on the multimodal sensing data to obtain target sensing data. An improved Simultaneous Localization and Mapping (SLAM) algorithm is then used to fuse the target sensing data, build and update the global map of the tunnel (globally consistent environment map) in real time, and fuse the data obtained by the intrinsically safe binocular infrared camera and the millimeter-wave radar through an adaptive Kalman filter to achieve trajectory prediction of dynamic obstacles.

[0024] S102. Based on the multimodal sensing data, determine the initial control commands for the robotic arm's motion unit.

[0025] In this application, multimodal perception data is analyzed based on a deep reinforcement learning framework to determine the initial control commands for the robotic arm's motion unit online.

[0026] S103. Compensate the initial control command and obtain the first control command of the robotic arm's motion unit.

[0027] In this embodiment, a first desired motion trajectory of the robotic arm end effector can be generated based on the work objective and the global map of the tunnel. Based on the first control command, the first desired motion trajectory is locally optimized and time-calibrated to obtain a second desired motion trajectory. Based on the extended state observer and fractional sliding mode controller of the robotic arm motor unit, environmental disturbance information is determined. Based on the environmental disturbance information, the second desired motion trajectory is compensated to generate a third desired motion trajectory. Based on the third desired motion trajectory, a first control command of the robotic arm motor unit is generated.

[0028] S104. Simulate the simulated robotic arm motor unit based on the first control command, and obtain the safety detection results of the simulated robotic arm motor unit during the simulation process.

[0029] It should be noted that, in order to improve the safety and reliability of the control process of the simulated robotic arm's motor unit, before controlling the robotic arm's motor unit, a millisecond-level simulation pre-run and safety test are performed on the simulated robotic arm's motor unit based on the first control command to obtain the safety test results of the simulated robotic arm's motor unit during the simulation process.

[0030] S105. In response to the safety detection result indicating a risk, the first control command is modified to obtain the target control command for the robotic arm motor unit.

[0031] In this embodiment of the application, after obtaining the safety detection results of the simulated robotic arm motor unit during the simulation process, in response to the safety detection results indicating that there is a risk, the first control command is corrected in a feedforward manner to obtain the target control command of the robotic arm motor unit; in response to the safety detection results indicating that there is no risk, there is no need to correct the first control command, and the first control command is directly used as the target control command of the robotic arm motor unit.

[0032] In this embodiment, in response to the safety detection result indicating a risk, the first control command is modified to form a safety intervention closed loop, and the target control command of the robotic arm motor unit is obtained.

[0033] S106. Control the robotic arm's motion unit based on the target control command.

[0034] In this embodiment of the application, after obtaining the target control command, the robotic arm motor unit is controlled based on the target control command.

[0035] In this embodiment of the application, in response to the safety detection result indicating that there is no risk, the robotic arm motor unit is controlled based on the first control command.

[0036] The intelligent control method for a hydraulically driven mobile robotic arm in coal mines provided in this application acquires multimodal sensing data, determines the initial control command for the robotic arm's mobile unit based on the multimodal sensing data, compensates for the initial control command to obtain the first control command for the robotic arm's mobile unit, simulates the robotic arm's mobile unit based on the first control command, obtains the safety detection results of the simulated robotic arm's mobile unit during the simulation process, and corrects the first control command in response to the safety detection results indicating a risk, thereby obtaining the target control command for the robotic arm's mobile unit. Based on the target control command, the robotic arm's mobile unit is controlled. Therefore, by determining the target control command and controlling the robotic arm's mobile unit based on the target control command, this application improves the autonomous operation capability of the robotic arm in unstructured roadways, enhances the control accuracy of the robotic arm's mobile unit, and strengthens the safety, reliability, and efficiency of the robotic arm's mobile unit during operation.

[0037] Figure 2 This is a flowchart illustrating the intelligent control method for a hydraulically driven mobile robotic arm in a coal mine, as provided in an embodiment of this application.

[0038] like Figure 2 As shown in this embodiment, the intelligent control method for a hydraulically driven mobile robotic arm in coal mines specifically includes the following steps: S201. Acquire multimodal sensing data.

[0039] Optionally, step S102 in the above embodiments, "determining the initial control command of the robotic arm's motor unit based on multimodal perception data", may specifically include the following steps S202-S206.

[0040] S202. Determine the operating condition data of the robotic arm motor unit and the relative position data of the robotic arm motor unit and the roadway.

[0041] S203. Extract features from the working condition data to obtain the working condition feature vector, and extract features from the relative position data to obtain the relative position feature vector.

[0042] S204. Extract features from the target perception data to obtain a standardized state feature vector.

[0043] S205. Summarize the working condition feature vector, relative position feature vector, and standardized state feature vector to obtain the target feature vector.

[0044] S206. The pre-trained agent outputs the initial control commands for the robotic arm's motion unit based on the target feature vector.

[0045] In this embodiment, the target feature vector is input into the intelligent agent, which analyzes and processes the target feature vector and outputs the initial control command of the robotic arm's motor unit to achieve an adaptive balance between trajectory tracking accuracy, energy consumption and motion stability, and to naturally handle the coupling relationship between movement and operation.

[0046] Here, the agent can be understood as a Deep Reinforcement Learning (DRL) agent.

[0047] In the embodiments of this application, an intelligent agent can be pre-constructed, the control problem can be modeled as a partially observable Markov decision process, a proximal policy optimization algorithm framework can be adopted, and a long short-term memory network can be embedded to process the temporal state. Massive offline training can be carried out in a simulation environment. The trained deep reinforcement learning policy network can output control commands online according to the target feature vector, and can be fine-tuned through online data.

[0048] It should be noted that the intelligent agent can dynamically allocate control commands to the mobile transport platform and the robotic arm based on real-time operational data.

[0049] For example, when the robot system moves to the target location but has not yet started dust removal, it outputs a control command of "mobile transport platform as the main component, arm movement as the auxiliary component." The mobile transport platform is used as the sole path planning object, and horizontal movement is completed by the mobile transport platform. The robotic arm actively avoids obstacles only through vertical adjustment and end effector pose adjustment. When the robot system moves to the target location and has started dust removal, it uses cooperative coupling control to treat the mobile transport platform and the robotic arm as a whole as a redundant degree-of-freedom system for unified planning. This achieves optimal efficiency while meeting accuracy and stability requirements, with the effective working distance between the end effector and the target being maintained as a constraint. When the robot system moves along a fixed route to monitor dust accumulation, it adopts a control strategy of "arm movement as the main component, mobile transport platform as the auxiliary component." Without changing the global path of the mobile transport platform, the robotic arm dynamically adjusts its pose to maintain the stability of the relative positions of each sensor and the alley wall, ensuring the accuracy of the monitoring data.

[0050] It should be noted that the intelligent agent can output different control commands based on the real-time relative position data of the robotic arm's motor unit and the tunnel.

[0051] For example, when the mobile transport platform and the robotic arm are in a reasonable working position, the rotation of the robotic arm base and the stroke of the upper arm are locked, and fine adjustments are made through the rotation of the forearm joint and the end effector to maintain the optimal working angle. When the mobile transport platform is in a reasonable position but the working position deviates, the robotic arm is the primary means of adjustment, while the travel speed of the mobile transport platform is reduced. After automatic adjustment, the speed is restored to normal. When the position of the mobile transport platform exceeds the adjustment range of the robotic arm, the robotic arm automatically retracts to a safe position, the mobile transport platform makes lateral position adjustments, and the robotic arm resumes its working posture after re-entering the reasonable area. When the distance between the mobile transport platform and the tunnel wall is less than the safety threshold, the mobile transport platform decelerates while the robotic arm quickly retracts its extended volume according to a preset collision avoidance path. If the calculated end effector shows a collision tendency, emergency braking is performed.

[0052] Optionally, step S103 in the above embodiment, "compensating the initial control command and obtaining the first control command of the robotic arm motor unit", may specifically include the following steps S207-S2010.

[0053] S207. Based on the task objective and the global map of the tunnel, generate the first desired motion trajectory of the robotic arm end effector.

[0054] Optionally, based on the task objective (such as moving bundles of anchor bolts to a designated location) and the overall roadway map, the first desired motion trajectory of the robotic arm end effector is planned and generated.

[0055] S208. Based on the first control command, perform local trajectory optimization and time calibration on the first desired motion trajectory to obtain the second desired motion trajectory.

[0056] In this embodiment of the application, after obtaining the first desired motion trajectory, the first control command is combined with the first desired motion trajectory to perform local trajectory optimization and time calibration to obtain the second desired motion trajectory.

[0057] S209. An extended state observer and a fractional-order sliding mode controller based on the robotic arm's motion unit determine disturbance information, compensate for the second desired motion trajectory based on the disturbance information, and generate a third desired motion trajectory.

[0058] For example, a fractional-order sliding mode controller can be set as the error dynamic characteristic benchmark for each hydraulic joint of the robotic arm's motor unit, and an extended state observer (ESO) can be equipped to estimate and compensate for the total disturbance caused by the nonlinearity of the hydraulic system, parameter uncertainty, and external load changes in real time. The equivalent control term based on the fractional-order sliding mode controller and the disturbance compensation term based on the ESO estimate together constitute the disturbance information. The second desired motion trajectory is compensated based on the disturbance information to generate the third desired motion trajectory, so as to ensure high-precision and robust tracking of the joint-level motion of the robotic arm's motor unit and achieve high-precision and strong disturbance rejection tracking control.

[0059] S2010. Generate the first control command for the robotic arm's motion unit based on the third desired motion trajectory.

[0060] In this embodiment of the application, after obtaining the third desired motion trajectory, the third desired motion trajectory is transformed to generate the first control command of the robotic arm motor unit.

[0061] Optionally, step S104 in the above embodiment, "simulating the simulated robotic arm motor unit based on the first control command and obtaining the safety detection results of the simulated robotic arm motor unit during the simulation process", may specifically include the following steps S2011-S2012.

[0062] S2011. Based on the first control command, drive the simulated robotic arm motor unit to move in the simulation environment model, and obtain the motion state data of the simulated robotic arm motor unit within the preset simulation time.

[0063] S2012. Perform security checks on motion state data and obtain security check results.

[0064] Optionally, by performing safety checks on the motion state data, the safety checks can be performed to determine whether the simulated robotic arm's motor unit collides with any object in the simulation environment model, whether the joints exceed their limits, and whether the robotic arm's motor unit becomes unstable, in order to obtain safety check results.

[0065] Optionally, step S105 in the above embodiment, "in response to the safety detection result indicating a risk, the first control command is modified to obtain the target control command for the robotic arm motor unit", may specifically include the following steps S2013-S2014.

[0066] S2013. Generate corresponding correction parameters based on the risk type.

[0067] For example, if the simulated robotic arm's motor unit collides with any object in the simulation environment model, and a risk is identified, then based on the principle of minimum intervention and different risk types, correction parameters such as adjusting waypoints, reducing speed, and adjusting target endpoint coordinates are generated.

[0068] Optionally, in response to a safety test result indicating a risk, an early warning message can be generated.

[0069] S2014. Based on the correction parameters, the first control command is corrected to obtain the target control command.

[0070] In this embodiment of the application, after obtaining the correction parameters, the control parameters corresponding to the first control command can be corrected based on the correction parameters to obtain the target control command, thereby blocking potential risks.

[0071] S2015. Control the robotic arm's motion unit based on target control commands.

[0072] For example, after receiving the target control command, the target control command is sent to the robotic arm motor unit to drive the electro-hydraulic proportional servo valve group and hydraulic cylinders, etc., to complete the actual movement of the robotic arm.

[0073] Optionally, real-time operational status data of the robotic arm's motor unit can be collected synchronously to update the parameters of the simulated robotic arm's motor unit online and fine-tune the agent's strategy network, enabling the entire system to have the ability to continuously learn and self-optimize, forming an evolutionary closed loop of "execution-feedback-learning".

[0074] The intelligent control method for hydraulically driven mobile robotic arms in coal mines provided in this application constructs an adaptive coordinated control architecture for mobile robotic arms based on deep reinforcement learning. This architecture can learn and handle the strong dynamic coupling between the mobile transport platform and the robotic arm online, adapt to the complex nonlinear characteristics of hydraulic systems, and does not rely on precise analytical models. It has a high degree of intelligence and compensates for disturbance information while correcting based on safety detection results, thus upgrading safety control from "passive response" to "active prediction and prevention." This improves the safety and reliability of the robotic arm control process. Through data closed-loop, it can continuously update the parameters of the intelligent agent and optimize control commands, enabling the system to have self-learning and self-optimization capabilities, with performance improving over time. This method does not rely on specific operating scenarios or precise roadway models. Through the generalization ability of the intelligent agent and the underlying robust controller, it can adapt to various heavy-duty underground operations and constantly changing environmental conditions, significantly improving the autonomous operation capability, safety, and efficiency of the robotic arm in unstructured underground environments.

[0075] Based on the intelligent control method for hydraulically driven mobile robotic arms in coal mines provided in this application, an intelligent control device for hydraulically driven mobile robotic arms in coal mines can be constructed.

[0076] Figure 3 This is a schematic diagram of the structure of an intelligent control device for a hydraulically driven mobile robotic arm in a coal mine, provided as an embodiment of this application.

[0077] like Figure 3 As shown, the intelligent control device for the hydraulically driven mobile robotic arm in coal mines includes: a multimodal fusion sensing unit, a data transmission unit, an adaptive intelligent decision-making unit, a hierarchical collaborative control unit, a digital twin verification unit, and a robotic arm mobility unit.

[0078] The multimodal fusion sensing unit includes an explosion-proof solid-state lidar, an explosion-proof millimeter-wave radar, a joint absolute encoder, a joint torque sensor, a platform inertial measurement unit, an explosion-proof tilt sensor, and a mining intrinsically safe binocular infrared camera, used to collect multimodal sensing data from all directions.

[0079] Among them, the multimodal fusion sensing unit is used to collect multimodal sensing data and transmit it to the adaptive intelligent decision-making unit through the data transmission unit.

[0080] The data transmission unit includes an intrinsically safe mining base station based on 5G / Wi-Fi 6 technology, an explosion-proof and intrinsically safe mining switch, an industrial Ethernet gateway, and underground communication optical cables, forming a high-bandwidth, low-latency underground data communication network to ensure reliable data transmission.

[0081] The data transmission unit is used to transmit multimodal perception data to the adaptive intelligent decision-making unit.

[0082] The core of the adaptive intelligent decision-making unit is a high-performance edge computing server that has passed the mine explosion-proof certification. It is equipped with a graphics processing unit (GPU) or a dedicated artificial intelligence (AI) computing card to run deep reinforcement learning agent algorithms (for determining initial control commands) and environmental modeling algorithms (for building a global map of the tunnel). The adaptive intelligent decision-making unit is deployed in underground chambers or central control centers.

[0083] The adaptive intelligent decision-making unit is used to receive multimodal perception data, analyze and process the multimodal perception data, and determine the initial control commands for the robotic arm's motion unit.

[0084] The hierarchical collaborative control unit consists of an intrinsically safe programmable automation controller for mining and a matching drive module. It is used to run the underlying fractional sliding mode control algorithm and ESO observer algorithm, and is directly connected to the amplifier of the electro-hydraulic servo valve to achieve precise current / voltage output.

[0085] The hierarchical collaborative control unit is used to compensate for the initial control commands and obtain the first control commands of the robotic arm's motion unit.

[0086] The digital twin verification unit is deployed on a high-performance graphics workstation and server cluster in the ground control center, running a physical simulation engine and maintaining real-time data synchronization with the underground system through the mine ring network.

[0087] The digital twin verification unit is used to simulate the simulated robotic arm motor unit based on the first control command, obtain the safety detection results of the simulated robotic arm motor unit during the simulation process, and correct the first control command in response to the safety detection results indicating that there is a risk, so as to obtain the target control command of the robotic arm motor unit.

[0088] The robotic arm motor unit includes a heavy-duty hydraulic robotic arm body, a mobile transport platform, an electro-hydraulic proportional servo valve group, hydraulic cylinders, oil pressure and flow sensors, and a hydraulic power source. It is the final actuator that controls the robotic arm motor unit based on target control commands.

[0089] The intelligent control device for hydraulically driven mobile robotic arms in coal mines provided in this application addresses three core challenges in the complex environment of underground coal mine roadways: dynamic coupling control, hydraulic system nonlinearity and disturbance suppression, and adaptation to unstructured environments. It constructs an intelligent control device integrating perception, decision-making, and control. This device achieves high-precision, strong anti-disturbance, and autonomous intelligent control during the robotic arm's mobile unit movements, significantly improving its intelligence level in auxiliary operations. For the first time in underground coal mines, this device deeply integrates deep reinforcement learning algorithms with hydraulic system characteristic compensation and achieves "predictive-prevention" safety control through digital twins. This effectively solves the problem of strong coupling control between the mobile platform and the robotic arm, greatly improving the robotic arm's autonomous operation capability, safety, and efficiency in unstructured underground environments. To achieve the above embodiments, this embodiment provides an intelligent control system for a hydraulically driven mobile robotic arm in coal mines. Figure 4 This is a schematic diagram of the structure of an intelligent control system for a hydraulically driven mobile robotic arm in a coal mine, provided as an embodiment of this application.

[0090] like Figure 4As shown, the intelligent control system 1000 for the hydraulically driven mobile robotic arm in coal mine includes: a first acquisition module 110, a determination module 120, a second acquisition module 130, a simulation module 140, a correction module 150, and a control module 160.

[0091] The first acquisition module 110 is used to acquire multimodal perception data, wherein the multimodal perception data includes three-dimensional point cloud data of the tunnel, obstacle information in the tunnel, image data of the work target, and body state data of the robotic arm's motor unit; The determining module 120 is used to determine the initial control command of the robotic arm motion unit based on the multimodal sensing data; The second acquisition module 130 is used to compensate the initial control command and acquire the first control command of the robotic arm motor unit. Simulation module 140 is used to simulate the simulated robotic arm motor unit based on the first control command and obtain the safety detection results of the simulated robotic arm motor unit during the simulation process; The correction module 150, in response to the safety detection result indicating a risk, corrects the first control command to obtain the target control command for the robotic arm motor unit. The control module 160 is used to control the robotic arm's motor unit based on the target control command.

[0092] According to one embodiment of this application, the robotic arm motor unit includes: a robotic arm; a mobile transport platform associated with the robotic arm; an electro-hydraulic servo valve group; a hydraulic cylinder; a hydraulic pressure and flow sensor; and a hydraulic power source.

[0093] According to one embodiment of this application, after acquiring the multimodal sensing data, the device 1000 is further configured to: preprocess the multimodal sensing data to obtain target sensing data; and construct a global map of the alleyway based on the target sensing data.

[0094] According to one embodiment of this application, the determining module 120 is further configured to: determine the working condition data of the robotic arm mobile unit and the relative position data of the robotic arm mobile unit and the roadway; extract features from the working condition data to obtain a working condition feature vector, and extract features from the relative position data to obtain a relative position feature vector; extract features from the target perception data to obtain a standardized state feature vector; summarize the working condition feature vector, the relative position feature vector, and the standardized state feature vector to obtain a target feature vector; and output the initial control command of the robotic arm mobile unit based on the target feature vector by a pre-trained agent.

[0095] According to one embodiment of this application, the second acquisition module 130 is further configured to: generate a first desired motion trajectory for the end effector of the robotic arm based on the work target and the global map of the tunnel; perform local trajectory optimization and time calibration on the first desired motion trajectory based on a first control command to obtain a second desired motion trajectory; determine disturbance information based on the extended state observer and fractional sliding mode controller of the robotic arm maneuvering unit; compensate the second desired motion trajectory based on the disturbance information to generate a third desired motion trajectory; and generate a first control command for the robotic arm maneuvering unit based on the third desired motion trajectory.

[0096] According to one embodiment of this application, the simulation module 140 is further configured to: drive the simulated robotic arm motor unit to move in a simulation environment model based on the first control command, and obtain motion state data of the simulated robotic arm motor unit within a preset simulation time; perform safety detection on the motion state data, and obtain the safety detection result.

[0097] According to one embodiment of this application, the correction module 150 is further configured to: in response to the safety detection result indicating the existence of a risk, generate corresponding correction parameters according to the risk type; and correct the first control command based on the correction parameters to obtain the target control command.

[0098] According to one embodiment of this application, the device 1000 is further configured to: control the robotic arm motor unit based on the first control command in response to the safety detection result indicating that no risk exists.

[0099] It should be noted that the foregoing explanation of the intelligent control method embodiment for the hydraulically driven mobile robotic arm in coal mines also applies to the intelligent control system of the hydraulically driven mobile robotic arm in coal mines in this embodiment, and will not be repeated here.

[0100] The intelligent control system for a hydraulically driven mobile robotic arm in coal mines provided in this application acquires multimodal perception data, determines the initial control command for the robotic arm's mobile unit based on the multimodal perception data, compensates for the initial control command to obtain the first control command for the robotic arm's mobile unit, simulates the robotic arm's mobile unit based on the first control command, obtains the safety detection results of the simulated robotic arm's mobile unit during the simulation process, and corrects the first control command in response to the safety detection results indicating a risk, thereby obtaining the target control command for the robotic arm's mobile unit. Based on the target control command, the robotic arm's mobile unit is controlled. Therefore, by determining the target control command and controlling the robotic arm's mobile unit based on the target control command, this application improves the autonomous operation capability of the robotic arm in unstructured roadways, enhances the control accuracy of the robotic arm's mobile unit, and strengthens the safety, reliability, and efficiency of the robotic arm's mobile unit during operation.

[0101] To implement the above embodiments, as shown in FIG5, this disclosure also proposes an electronic device 2000, which includes a memory 210, a processor 220, and a computer program stored on the memory and executable on the processor 220. When the processor 220 executes program instructions, it implements the intelligent control method for the hydraulically driven mobile robotic arm in coal mines described above.

[0102] In the foregoing descriptions of the embodiments, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0103] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0104] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0105] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0106] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0107] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0108] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0109] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for intelligent control of a hydraulically driven mobile robotic arm in coal mines, characterized in that, The method includes: Acquire multimodal perception data, wherein the multimodal perception data includes three-dimensional point cloud data of the tunnel, obstacle information in the tunnel, image data of the working target, and body state data of the robotic arm's motor unit; Based on the multimodal sensing data, the initial control command for the robotic arm's motion unit is determined; The initial control command is compensated to obtain the first control command of the robotic arm motion unit; The simulation robot arm motor unit is simulated based on the first control command, and the safety detection results of the simulation robot arm motor unit during the simulation process are obtained. In response to the safety detection result indicating a risk, the first control command is modified to obtain the target control command for the robotic arm motor unit; The robotic arm's motion unit is controlled based on the target control command.

2. The method according to claim 1, characterized in that, The robotic arm motion unit includes: robotic arm; A mobile transport platform associated with the robotic arm; Electro-hydraulic servo valve assembly; Hydraulic cylinder; Oil pressure and flow sensors; Hydraulic power source.

3. The method according to claim 1, characterized in that, After acquiring the multimodal sensing data, the process also includes: The multimodal sensing data is preprocessed to obtain target sensing data; Based on the target perception data, a global map of the alleyway is constructed.

4. The method according to any one of claims 1-3, characterized in that, The step of determining the initial control command for the robotic arm's motion unit based on the multimodal data includes: Determine the operating condition data of the robotic arm mobile unit and the relative position data of the robotic arm mobile unit and the roadway; Feature extraction is performed on the work condition data to obtain a work condition feature vector, and feature extraction is performed on the relative position data to obtain a relative position feature vector; Feature extraction is performed on the target perception data to obtain a standardized state feature vector; The target feature vector is obtained by summing the working condition feature vector, the relative position feature vector, and the standardized state feature vector. The pre-trained agent outputs the initial control commands for the robotic arm's motion unit based on the target feature vector.

5. The method according to claim 4, characterized in that, The step of compensating the initial control command to obtain the first control command of the robotic arm's motion unit includes: Based on the task objective and the global map of the tunnel, a first desired motion trajectory of the robotic arm end effector is generated. Based on the first control command, the first desired motion trajectory is optimized locally and time-calibrated to obtain the second desired motion trajectory. Based on the extended state observer and fractional sliding mode controller of the robotic arm's motion unit, disturbance information is determined, and the second desired motion trajectory is compensated based on the disturbance information to generate a third desired motion trajectory. Based on the third desired motion trajectory, the first control command of the robotic arm motion unit is generated.

6. The method according to claim 1, characterized in that, The simulation of the robotic arm's motion unit based on the first control command, and the acquisition of the safety detection results of the robotic arm's motion unit during the simulation process, include: Based on the first control command, the simulated robotic arm motor unit is driven to move in the simulation environment model to obtain the motion state data of the simulated robotic arm motor unit within a preset simulation time. Perform security checks on the motion state data and obtain the security check results.

7. The method according to claim 1, characterized in that, In response to the safety detection result indicating a risk, the first control command is modified to obtain the target control command for the robotic arm's motion unit, including: Generate corresponding correction parameters based on the risk type; The first control command is modified based on the correction parameters to obtain the target control command.

8. The method according to claim 1, characterized in that, The method further includes: In response to the safety detection result indicating no risk, the robotic arm motor unit is controlled based on the first control command.

9. An intelligent control system for a hydraulically driven mobile robotic arm in coal mines, characterized in that, The system includes: The first acquisition module is used to acquire multimodal perception data, wherein the multimodal perception data includes three-dimensional point cloud data of the tunnel, obstacle information in the tunnel, image data of the work target, and body state data of the robotic arm's motor unit; The determination module is used to determine the initial control command of the robotic arm motion unit based on the multimodal sensing data; The second acquisition module is used to compensate the initial control command and acquire the first control command of the robotic arm motion unit; The simulation module is used to simulate the simulated robotic arm motor unit based on the first control command and obtain the safety detection results of the simulated robotic arm motor unit during the simulation process. The correction module, in response to the safety detection result indicating a risk, corrects the first control command to obtain the target control command for the robotic arm's motor unit; The control module is used to control the robotic arm's motion unit based on the target control command.

10. An electronic device, characterized in that, It includes a processor and a memory; wherein the processor runs a program corresponding to the executable program code stored in the memory to implement the method as described in any one of claims 1-8.

Images