A collaborative robot system for autonomous operation of a machine tool numerical control system panel

Abstract

A kind of collaborative robot system for machine tool numerical control system panel autonomous operation, including collaborative robot hardware system module, motion planning algorithm module for key operation, collaborative robot software module and the collaborative operation implementation module of numerical control machine tool panel, convert machine tool numerical control system panel operation process into schedulable execution task flow;In the process of system running, first, the space calibration of machine tool numerical control system panel is carried out and the collision-free pose database is established, the automatic execution of machine tool numerical control system panel key operation is realized through task construction and scheduling mechanism;At the same time, combined with machine tool running state monitoring and multi-source information feedback mechanism, the operation process is judged and handled in real time, so as to ensure the continuity and safety of operation process;The present application realizes the automatic control of machine tool numerical control system panel operation process, effectively improves the efficiency of machine tool operation, reduces the intensity of artificial repetitive labor, and improves the generality and engineering application value of system.

Description

Technical Field

[0001] This invention relates to the field of industrial automation and intelligent manufacturing technology, specifically to a collaborative robot system for autonomous operation of machine tool CNC system panels. Background Technology

[0002] With the continuous development of CNC machining technology, CNC machine tools have been widely used in aerospace manufacturing, precision machining, and other fields, playing a vital role in the machining of complex structural parts and high-precision components. In actual production, the operation of CNC machine tools not only depends on the execution of CNC programs, but also requires operators to complete a series of operations through the machine tool's CNC system panel, including powering on the equipment, switching modes, setting parameters, starting the program, and monitoring the operation process. These operations are typically completed through panel buttons or a human-machine interface, characterized by numerous operation steps and frequent interactions.

[0003] In traditional production models, the operation of machine tool CNC system panels is primarily manual. Operators need to be present at the machine for extended periods, continuously monitoring its operational status and performing corresponding operational controls based on the machining progress. This method is not only labor-intensive but also prone to operator fatigue due to the highly repetitive nature of the operations, leading to misoperations or response delays, which in turn affect processing efficiency and product quality. In some complex machining scenarios, the machine tool's operating time is long and the operation process is cumbersome, further exacerbating the burden on manual operators and hindering the achievement of efficient and stable production processes.

[0004] With the development of intelligent manufacturing and industrial automation technologies, the demand for automation upgrades of CNC machine tools is constantly increasing. Existing research mainly focuses on automated machine tool loading and unloading, workpiece handling, and program invocation via communication interfaces. However, there is relatively little research on the automation of the control panel button operation of CNC machine tools. Existing methods typically rely on open machine tool interfaces or specific communication protocols for control. For example, Liu et al. constructed a Cyber-Physical Machine Tool platform based on OPC UA and MTConnect, achieving standardized communication and collaborative control between machine tools and external applications (Liu C, Vengayil H, Lu Y, Xu X. A Cyber-Physical Machine Tools Platformusing OPC UA and MTConnect[J]. Journal of Manufacturing Systems, 2019, 51:61-74.). However, with inconsistent machine tool interface standards from different manufacturers, problems such as poor compatibility and complex system integration exist. Furthermore, for machine tools lacking open interfaces or with limited interfaces, automated control cannot be achieved through these methods. In addition, existing technologies generally lack a unified operational process organization mechanism and stable execution control methods in the process of automating panel operation, making it difficult to meet the flexibility and reliability requirements of machine tool operation under complex processes. Therefore, it is necessary to propose a technical solution that can adapt to different machine tool scenarios, has good versatility, and enables autonomous operation of the machine tool CNC system panel. Summary of the Invention

[0005] To overcome the shortcomings of the existing technology, the present invention aims to provide a collaborative robot system for autonomous operation of machine tool CNC system panels. This system transforms the operation process of the machine tool CNC system panel into a schedulable task flow. During system operation, the machine tool CNC system panel is first spatially calibrated and a collision-free pose database is established. Based on this, a task construction and scheduling mechanism is used to automatically execute the button operations on the machine tool CNC system panel. Simultaneously, combined with machine tool operation status monitoring and multi-source information feedback mechanisms, the operation process is judged and anomalies are handled in real time, thereby ensuring the continuity and safety of the operation process. By transforming the operation of the machine tool CNC system panel from traditional manual operation to autonomous execution by a collaborative robot, the present invention achieves automated control of the machine tool CNC system panel operation process without relying on the machine tool's open interface. This effectively improves machine tool operation efficiency, reduces the intensity of repetitive manual labor, and enhances the system's versatility and engineering application value.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A collaborative robot system for autonomous operation of a machine tool CNC system panel includes the following modules: Module 1) Collaborative Robot Hardware Module: Constructs a collaborative robot hardware system for performing CNC system panel operation tasks on machine tools. The collaborative robot hardware system is designed around the operating mechanism of "command issuance - visual perception - system decision-making - collaborative execution - machine tool movement". The collaborative robot hardware system includes the collaborative robot body, vision camera, membrane keypad, multi-port switch, mounting base and related assembly components, as well as end effector for completing keypad operations. Module 2) Motion planning algorithm module for panel button operation: The coordinate system of the machine tool CNC system panel is calibrated, the spatial transformation relationship between the panel coordinate system and the collaborative robot coordinate system is established, and the panel button positions are mapped based on the transformation relationship. The corresponding robot arm button operation poses are obtained by solving the forward and inverse kinematics. The operation space is modeled and the button operation process is simulated in Unity. Collision detection is performed on the button operation process, and the set of operation poses that meet the collision-free constraints is selected to build a collision-free pose database. Module 3) Collaborative Robot Software Module: Construct a collaborative robot software control system for executing machine tool CNC system panel operation tasks. The software control system calls the button operation poses based on a collision-free pose database, and organizes and schedules the machine tool CNC system panel operation tasks in a process-oriented manner. It drives the collaborative robot to complete button operations through preset task sequences, and realizes status monitoring and hierarchical alarm processing during the execution process. Module 4) Collaborative Operation Implementation Module of CNC System Panel: Based on the collaborative robot hardware system, motion planning results and collaborative robot software control system, construct the autonomous operation process of the machine tool CNC system panel. By calling the operation poses in the collision-free pose database and combining the task scheduling mechanism, the automatic execution of various button operations on the machine tool CNC system panel is realized, thereby completing the autonomous operation of the machine tool CNC system.

[0007] The specific process of module 1) is as follows: 1.1) First, construct a system operation mechanism for the CNC system panel operation tasks of machine tools. The operation mechanism includes the execution process of "instruction issuance - visual perception - system decision-making - collaborative execution - machine tool movement". During the operation, "visual perception" continuously acquires interface status information recognition, alarm information monitoring and environmental status monitoring, and participates in system decision-making by providing visual recognition information. A system-level hardware structure is constructed, consisting of a sensing unit, an execution unit, a control unit, and a communication unit. The sensing unit includes a vision camera and membrane keys, used to acquire machine tool interface status information, alarm information, and operating environment status information, and feed the necessary information back to the control unit for system decision-making. The execution unit includes the collaborative robot body and its end effector, used to complete the machine tool CNC system panel key operation tasks based on the system decision results, realizing the collaborative execution process. The control unit is the collaborative robot software control system, used to realize task parsing, scheduling control, and execution logic judgment, and to uniformly manage and process the tasks after the instructions are issued. The communication unit is a multi-port switching device, used to establish data interaction channels between various hardware modules, realizing the transmission and sharing of instruction information and status information. 1.2) Based on the structural dimensions of the machine tool edge operating space and the distribution characteristics of the CNC panel buttons, the overall layout design of the installation method of the collaborative robot in the machine tool working space is carried out; by designing the structure of the mounting base and the robotic arm mounting frame, and optimizing the installation height and relative position relationship, the effective working space of the collaborative robot can cover the operating area of ​​the machine tool CNC system panel, while avoiding spatial interference with the machine tool body and surrounding equipment. 1.3) The end effector of a collaborative robot includes the following modules: Button mechanism module: miniature electric cylinder, conductive rubber flexible end; Feed mechanism module: precision slide table, stepper motor driver; Displacement detection module: Infrared laser displacement sensor; Connector module: Robot end effector flange and slide plate connector, button mechanism base; The sensor is coaxially mounted with the miniature electric cylinder to measure the distance from the end to the working surface in real time; the precision slide is responsible for long-distance coarse positioning, and the miniature electric cylinder is responsible for short-stroke fine pressing; the closed-loop electric cylinder of the miniature electric cylinder is equipped with a silicon-based conductive rubber end, which is compatible with capacitive touch screens and physical buttons. 1.4) The overall layout of the connector module must ensure coaxiality and rigidity, and be suitable for operation in narrow spaces at the edge of the machine tool. The connector module is made of integrated aluminum alloy and includes robot flange interface, slide mounting surface, electric cylinder fixing lug, and sensor mounting slot. 1.5) The displacement sensor outputs a distance signal in real time, which is then used by the control system. The system execution process is as follows: 1.5.1) After the collaborative robot moves the end effector above the work area, the feed mechanism module drives the precision slide to feed towards the CNC panel work surface at a set speed; the displacement sensor collects the distance between the end effector and the panel in real time until the distance between the end effector and the panel reaches the preset reference distance; 1.5.2) After the precision slide is locked, the miniature electric cylinder starts smoothly with a set constant acceleration. The push rod extends with uniform acceleration from rest. Before approaching the button surface, the miniature electric cylinder maintains a constant acceleration. 1.5.3) After the miniature electric cylinder accelerates to the target speed, it drives the flexible end to contact and press the button in a uniform manner; at the same time, the control system realizes constant force closed-loop control based on the displacement sensor to keep the pressing force stable within the trigger range; 1.5.4) After pressing the button, the miniature electric cylinder first retracts to the initial position at a constant speed; then the precision slide is unlocked and returns to the standby position at a constant speed; the end effector returns to the safe posture and waits for the next operation command.

[0008] The specific process of module 2) is as follows: 2.1) First, uniformly distributed calibration reference points are set up on the machine tool operation panel. The corresponding joint angle data are collected by touching each point using the robot teaching mode. A base coordinate system is established with the robot base mounting center as the reference. Combined with the calibration reference points, a three-point plane fitting and vector orthogonalization algorithm is used to build user coordinate systems for the keyboard and screen operation areas in the Unity environment. The pose transformation between the base coordinate system and each user coordinate system is realized through homogeneous coordinate transformation. All target points of the buttons on the machine tool CNC system panel are converted and mapped from the user coordinate system to the robot base coordinate system. At the same time, the XYZ spatial offset of the end effector relative to the robot flange is measured, the end effector coordinate system is established and incorporated into the overall coordinate transformation system, and the button center and robot control target pose are accurately matched. 2.2) Determine the improved DH parameters, including link length, link twist angle, joint offset, and joint rotation angle; establish the coordinate system of adjacent links, derive the homogeneous transformation matrix between links, and form a complete kinematic model of the robot, providing a mathematical basis for forward and inverse kinematics solutions. The general formula for the transformation matrix between adjacent links is: ; In the formula: i 1 T i — Link transformation matrix; θ i — Joint angle; a i 1— Link length; α i 1 — Linkage torsion angle; d i — Joint offset; 2.3) Multiply the homogeneous transformation matrices of each link sequentially from the base to the end effector to obtain the total pose transformation matrix from the base coordinate system to the end effector; input the joint angles to directly solve for the end effector position and attitude, realizing the mapping from joint space to working space, where the homogeneous matrix from the base to the end effector is: In the formula: ; ; ; ; ; ; ; ; ; ; n i It is the unit vector component of the X-axis of the end coordinate system in the X / Y / Z directions of the base coordinate system; o i It is the unit vector component of the Y-axis of the end coordinate system in the X / Y / Z directions of the base coordinate system; a i It is the unit vector component of the Z-axis of the end coordinate system in the X / Y / Z directions of the base coordinate system; P i The X / Y / Z coordinates of the origin of the final coordinate system in the base coordinate system; 2.4) Given the end-effector pose, perform inverse operation on the total pose transformation matrix to obtain multiple analytical solutions for the joint angles; for each target pose, obtain multiple feasible inverse solutions, covering different mechanical configurations. The results of the inverse kinematics analytical solutions are shown below: In the formula: ; ; ; ; ; 2.5) Based on the physical limits of robot joints, motion singularities, and configuration constraints, invalid solutions that exceed limits, interfere, or have abnormal postures are eliminated, while valid inverse solutions that are free of singularities, wrist flips, and have smooth joint movements are retained. 2.6) Substitute the selected valid inverse kinematics back into the forward kinematics equations, recalculate the end pose and compare the error with the target pose to ensure that the pose error is less than the system's allowable threshold; the valid inverse kinematics that have passed the verification are applied to the valid joint parameter combinations in the simulation and execution. 2.7) In the Unity 3D simulation platform, import the 3D models of the machine tool body, operation panel, camera bracket, and collaborative robot linkage, and complete the scene assembly according to the actual size and installation position on site; add mesh collision body components to all models, divide the collision levels, set the collision detection rules between the robot linkage and environmental obstacles, and construct a confined space virtual working environment consistent with the real working conditions; 2.8) By combining offline simulation with on-site teaching experiments, a set of safe transition poses is determined and used as the unified starting pose for all button target point pose traversal and motion simulation, ensuring that each motion trajectory has consistent and safe initial conditions. 2.9) Starting from the safe transition pose and ending with the effective inverse kinematics of each target point after screening, drive the robot to perform point-to-point motion simulation; enable real-time collision detection throughout the simulation process, synchronously determine whether the robot has its own linkage interference or collision with the external environment such as the machine tool or camera, and record the collision status and motion process data in real time. 2.10) Pose that detects a collision is directly eliminated; for valid poses without collision, the motion time, joint angular velocity change, and trajectory smoothness are statistically analyzed; according to the principles of prioritizing no collisions, shortest motion time, and best trajectory smoothness, a unique optimal execution pose is selected and determined for each button target point. 2.11) First, standardize and normalize the optimal joint angle, motion time, and collision state of the button target point; then define a collision-free pose database containing fields such as button number and joint angle; subsequently, perform validation and deduplication on the data in the database and remove invalid entries; finally, serialize the collision-free pose database into a TXT / JSON file to generate a collision-free pose library that can be directly called by the robot controller.

[0009] The specific process of module 3) is as follows: 3.1) Construct a collaborative robot software control system. The software control system is used to realize the unified scheduling and execution control of the machine tool CNC system panel operation tasks, and to visualize the task execution status, system operation status and alarm information through the software interactive interface, so that the operator can obtain system operation information in real time and monitor and intervene in the collaborative robot execution process.

[0010] The collaborative robot software control system includes a user interaction module, a task scheduling module, a collaborative function module, a communication module, and an anomaly and alarm module. The user interaction module is used not only for task construction and execution control but also for real-time updating and displaying of operation log information during task execution, thus intuitively reflecting the task's progress and results. The collaborative function module encapsulates collaborative function functions related to the button operations on the machine tool CNC system panel and provides the basic interfaces required for robotic arm motion control and end effector control. The communication module receives, parses, and stores status information transmitted from the machine tool system and external sensing modules. The anomaly and alarm handling module identifies and classifies abnormal information generated during system operation.

[0011] In addition, the collaborative robot software control system interacts with the collision-free pose database established in module 2) during motion control. By calling the safe operating poses pre-calculated and verified in the database, it can directly call each button operation.

[0012] During the overall task scheduling, execution control, status monitoring and alarm handling process, the functional modules interact with each other through a unified data interface, forming a closed-loop control structure for the task execution process.

[0013] 3.2) In the collaborative function module, the various button operations of the machine tool CNC system panel are encapsulated according to the machine tool operation process to form multiple collaborative function units. Each collaborative function consists of multiple execution steps, and each execution step corresponds to one or more button operations and their operation result judgment conditions, thereby realizing the functional abstraction and structured organization of the machine tool CNC system panel operation process. The collaborative function is designed based on the actual button flow of the machine tool, combined with interface recognition and panel information feedback, to realize the automatic execution of machine tool operations. The collaborative functions implemented cover the operation process of machine tool startup preparation, machining operation and safety control, including machine tool power-on, initial self-test, tool magazine return to zero, power off, screen wake-up, tool change, probe change, zero offset setting, program execution, program pause, coolant switching, emergency stop control and machine tool door opening and closing. After the collaborative function is encapsulated, it is invoked through the user interaction module. Users can click the corresponding function button to load the selected collaborative function into a task data structure and store it in the task queue for unified management. The task data structure includes task type, task name, execution parameters, and corresponding execution step information. Through the above task queue construction mechanism, multiple collaborative functions are combined according to actual processing needs to form different machine tool operation task sequences, which are executed in sequence according to the set order, thereby realizing the flexible organization and scheduling of machine tool CNC system panel operation tasks.

[0014] 3.3) During task execution, the corresponding collaborative function encapsulation functions are called sequentially according to the task sequence, and the robotic arm operation pose parameters required for the button operation are called according to the execution steps. The collaborative robot's movement is controlled to complete the button operation on the machine tool CNC system panel. Specifically, the execution control process includes: 3.3.1) Control the collaborative robot to move from its current pose to a preset safe transition pose; 3.3.2) Control the collaborative robot to move from the safe transition pose to the operation pose corresponding to the target button; 3.3.3) After the robotic arm reaches the target pose, the control end effector completes the button operation, including using an infrared sensor to obtain the distance information between the end effector and the machine tool CNC system panel, and using a precision slide to make fine adjustments to the position, and then driving a micro electric cylinder to complete the button pressing operation on the machine tool CNC system panel or screen; 3.3.4) After the button operation is completed, control the collaborative robot to return to a safe transition pose to wait for the next operation step to be executed.

[0015] The safe transition pose is a common transition state that satisfies collision-free constraints within the confined space of the machine tool and is achievable for all target operation poses.

[0016] 3.4) During task execution, the communication module monitors the operating status of the collaborative robot system in real time. The monitoring content includes the movement status of the robotic arm, the task execution status, and the panel visual monitoring information. The module also collects, records, and displays the status information. 3.5) Based on the aforementioned status monitoring results, a tiered alarm handling mechanism is constructed to classify and handle abnormal situations according to the system's operating status, including: 3.5.1) When a suggestive anomaly is detected, an alarm message is output without interrupting the execution of the current task; 3.5.2) When a general anomaly is detected, pause the current task execution and stop the collaborative robot's current movement; 3.5.3) When a serious anomaly is detected, immediately stop the movement of the collaborative robot and control it to immediately perform an emergency stop operation on the machine tool to protect the operational safety of the machine tool and the collaborative robot system.

[0017] The specific process of module 4) is as follows: 4.1) Before the machine tool CNC system panel operation task is executed, the machine tool CNC system panel is first spatially calibrated to establish the spatial transformation relationship between the machine tool CNC system panel coordinate system and the collaborative robot coordinate system; the positions of each button on the machine tool CNC system panel in the panel coordinate system are mapped to the collaborative robot coordinate system, and the corresponding robot arm operation pose is obtained through pose calculation.

[0018] Furthermore, collision detection and reachability analysis are performed on the above poses in the operation space model to select a set of robot arm operation poses that meet the collision-free constraints, thereby constructing a pose database for the operation of the control panel buttons of the machine tool CNC system.

[0019] 4.2) The machine tool CNC system panel operation tasks are constructed through the software interactive interface, and the actual machine tool operation requirements are transformed into a task sequence consisting of multiple collaborative functions and their corresponding button operation processes; each task is encapsulated in a unified data structure form and loaded into the task queue for unified management according to the execution order set by the user.

[0020] The task data structure includes task type, task name, execution parameters, and corresponding operation steps, which are used to describe the execution content and execution order of each collaborative function.

[0021] 4.3) During task execution, the task data structure in the task queue is sequentially traversed and scheduled for execution by the execution thread; during execution, the robotic arm operation pose parameters required for the corresponding button operation are called sequentially according to each task step.

[0022] The operation pose parameters are derived from the pose database constructed in step 4.1). By associating and calling the task steps with the pose data, the connection between the button operation task and the motion control of the robotic arm is realized.

[0023] 4.4) During the execution of button operation, the collaborative robot is controlled to complete the button operation according to the preset execution process; first, the collaborative robot moves from the current pose to the preset safe transition pose, and then moves from the safe transition pose to the operation pose corresponding to the target button.

[0024] Once the robotic arm moves to the target pose, the end effector completes the specific button operation process, including: 4.4.1) Obtain distance information between the end effector and the machine tool CNC system panel using an infrared sensor in the end effector; 4.4.2) Based on the distance information, the position of the end effector is finely adjusted by a precision slide in the end effector to improve the alignment accuracy between the end effector and the target button; 4.4.3) The miniature electric cylinder in the drive end effector completes the button pressing action, thereby realizing stable triggering of the buttons on the CNC system panel of the machine tool; 4.4.4) After the button operation is completed, control the collaborative robot to return to the safe transition pose to wait for the next operation step to be executed.

[0025] 4.5) After the key operation is completed, the current operating status of the machine tool is determined; the determination process includes membrane key feedback detection based on key trigger and visual status recognition based on changes in the CNC system panel interface of the machine tool.

[0026] 4.6) Anomaly Detection and Handling: During task execution, the machine tool's operating status and working environment are monitored in real time; anomaly information comes from the vision module's identification of alarm information on the machine tool's CNC system interface and the identification results of the working environment status.

[0027] The abnormal information includes machine tool operation alarm information and human-machine environment abnormal information; human-machine environment abnormal information includes personnel not dressed in accordance with requirements, incorrect tool type identification, and milling fluid not turned on; when an abnormal situation is detected, the corresponding handling strategy is executed according to the abnormality type.

[0028] Compared with the prior art, the present invention has the following advantages: (1) By constructing a collaborative robot system for machine tool CNC system panel operation, this invention transforms the traditional manual operation of machine tool CNC system panel button operation into an automated execution process, which can effectively reduce the labor intensity of operators, reduce human operation errors, and improve the stability and production efficiency of machine tool operation. (2) Through the flexible conductive end, displacement sensing and slide table-electric cylinder coordinated control end execution structure, smooth contact and high-precision positioning of the button process are realized, which not only effectively avoids damage to the CNC system panel and screen of the machine tool, but also significantly reduces the false touch rate and improves the stability and reliability of button operation. (3) By combining kinematic modeling with collision detection in a simulation environment, the operation posture of the robotic arm is optimized and screened, and the motion stability and execution efficiency are improved under the premise of ensuring obstacle avoidance safety, so as to adapt to the complex operation requirements in the narrow space at the edge of the machine tool. (4) The present invention organizes the operation process of the CNC system panel of the machine tool into a configurable task sequence through the task queue construction and scheduling mechanism, supports the flexible combination of various machine tool operation processes, can adapt to the needs of different processing scenarios, and improves the versatility and scalability of the system; (5) By designing an end-effector and combining infrared ranging, slide table fine adjustment and electric cylinder pressing in a coordinated control mode, this invention achieves precise positioning and stable pressing of the buttons on the CNC system panel of the machine tool, thereby improving the reliability and consistency of button operation. (6) This invention achieves accurate judgment of machine tool operation status by integrating the dual judgment mechanism of membrane button feedback and visual status recognition, thereby ensuring the correct progress of the operation process and improving the reliability of system operation. (7) By constructing an anomaly detection and hierarchical processing mechanism based on visual information, this invention can respond in real time to machine tool operation alarms and abnormal operating environment, and stop the task or perform emergency stop operation in time under abnormal conditions, thereby effectively ensuring the operation safety of machine tools and collaborative robot systems. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the system hardware structure according to an embodiment of the present invention.

[0030] Figure 2 This is a schematic diagram of the overall operation mechanism of the system in an embodiment of the present invention.

[0031] Figure 3 This is a schematic diagram of the collaborative robot installation platform according to an embodiment of the present invention.

[0032] Figure 4 This is a schematic diagram of the end effector of a collaborative robot according to an embodiment of the present invention.

[0033] Figure 5 This is a physical diagram of the end effector of the collaborative robot according to an embodiment of the present invention.

[0034] Figure 6 This is a schematic diagram illustrating the complete filtering process for keyboard and screen buttons according to an embodiment of the present invention.

[0035] Figure 7 This is a Unity virtual scene simulation interface according to an embodiment of the present invention.

[0036] Figure 8 This is a schematic diagram of the collaborative robot software control system structure according to an embodiment of the present invention.

[0037] Figure 9 This is a schematic diagram of the collaborative function machine tool operation process according to an embodiment of the present invention.

[0038] Figure 10 This is a schematic diagram of the autonomous operation method of the CNC system panel of the machine tool according to an embodiment of the present invention. Detailed Implementation

[0039] The present invention will now be described in detail with reference to embodiments and accompanying drawings. This embodiment provides a collaborative robot system for autonomous operation of a machine tool CNC system panel. It employs a strategy combining system integration and phased execution. Through hardware system construction, panel calibration and pose database establishment, software control, and task scheduling, it gradually automates the button operations on the machine tool CNC system panel. Each step has a clear logical relationship, and through pose data-driven operation and task flow organization, stable operation of the machine tool is achieved.

[0040] A collaborative robot system for autonomous operation of a machine tool CNC system panel includes the following modules: Module 1) Collaborative Robot Hardware Module: Based on the application requirements of autonomous operation of machine tool CNC system panels, a collaborative robot hardware system is constructed to perform machine tool CNC system panel operation tasks, such as... Figure 1 As shown, the collaborative robot hardware system is designed around the operating mechanism of "command issuance—visual perception—system decision-making—collaborative execution—machine tool movement." The collaborative robot hardware system includes the collaborative robot body, a vision camera, membrane buttons, a multi-port switch, a mounting base and related assembly components, and an end effector for performing button operations. The mounting base is used to ensure stable installation of the collaborative robot within the machine tool's workspace, and the end effector is used to perform pressing operations on the buttons on the machine tool's CNC system panel. The specific process is as follows: 1.1) System Hardware Composition and Overall Architecture Design: Based on the application requirements of autonomous operation of the machine tool CNC system panel, the system operation mechanism for machine tool CNC system panel operation tasks is first constructed, such as... Figure 2 As shown, the operating mechanism includes the execution process of "instruction issuance - visual perception - system decision-making - collaborative execution - machine tool movement"; during the operation, "visual perception" continuously acquires interface status information recognition, alarm information monitoring and environmental status monitoring, and participates in system decision-making by providing visual recognition information; Based on the above operating mechanism, the overall hardware architecture of the collaborative robot system is designed, and a system hardware structure consisting of a perception unit, an execution unit, a control unit and a communication unit is constructed. The sensing unit includes a vision camera and membrane keys, used to acquire machine tool interface status information, alarm information, and operating environment status information, and feed this information back to the control unit for system decision-making. The execution unit includes a collaborative robot body and its end effector, used to complete the machine tool CNC system panel key operation tasks according to the system decision results, realizing the collaborative execution process. The control unit is a collaborative robot software control system, used to realize task parsing, scheduling control, and execution logic judgment, and to uniformly manage and process the tasks after the instructions are issued. The communication unit is a multi-port switching device, used to establish data interaction channels between various hardware modules, realizing the transmission and sharing of instruction information and status information. Through the above hardware structure design, the system can support the "instruction issuance—visual perception—system decision-making—collaborative execution—machine tool movement" operation mechanism, thereby realizing the automated execution and stable operation of the machine tool CNC system panel operation tasks.

[0041] 1.2) System Spatial Layout and Installation Structure Design: Based on the above system hardware architecture, and according to the spatial structure of the machine tool operating area and the layout characteristics of the CNC panel, the installation position of the collaborative robot within the machine tool workspace is planned as a whole. For example... Figure 3As shown, it includes a robotic arm 1, a robotic arm mounting base 2, a robotic arm mounting frame 3, and a collaborative robot end effector button tool 4; the robotic arm 1 is connected to the robotic arm mounting frame 3 via the robotic arm mounting base 2, and the robotic arm 1 and the collaborative robot end effector button tool 4 are connected.

[0042] The CNC machine tool operation panel consists of a display screen area and a button operation area, which are connected by a rotating shaft. To ensure that the collaborative robot can cover both the display screen and the button area at the same time, the first joint axis (J1 axis) of the robotic arm and the rotating shaft are basically on the same horizontal plane in the height direction during installation, thereby improving the accessibility of the robotic arm to the panel operation area.

[0043] Furthermore, a robotic arm mounting frame is designed for stable installation. This frame employs a layered structure, comprising a bottom support platform and an upper mounting platform. The bottom platform increases the support area and lowers the system's center of gravity, while the upper platform mounts the collaborative robot and provides a suitable working height. The overall structure uses a frame design with reinforcing beams to improve rigidity and stability. By matching the installation height and structural dimensions, the J1 axis of the robotic arm is aligned with the rotation axis of the machine tool's CNC system panel. An adjustment mechanism allows for fine-tuning of the installation angle, and a connection interface is provided for integration with the vision equipment mounting structure. This spatial layout and mounting structure design ensures the collaborative robot has good operational accessibility and stability within the confined space of the machine tool, providing a spatial foundation for subsequent motion planning.

[0044] 1.3) Overall Design of the End Effector: The end effector comprises four main functional modules: a button mechanism module—a miniature electric cylinder and a conductive rubber flexible end; a feed mechanism module—a precision slide and a stepper motor driver; a displacement detection module—an infrared laser displacement sensor; and a connector module—the robot end effector flange and slide connector, and the button mechanism base. The displacement sensor and electric cylinder are coaxially mounted, allowing for real-time acquisition of the distance between the end effector and the working surface. The precision slide handles long-distance coarse positioning tasks, while the miniature electric cylinder handles short-stroke precise pressing tasks. The overall design utilizes a lightweight aluminum alloy structure, resulting in low weight and high rigidity, matching the rated load requirements of collaborative robots. Its overall structure is as follows: Figure 4 As shown, the actual object is as follows Figure 5 As shown, it includes a slide-to-flange connector 7, a miniature electric cylinder 8, a conductive rubber head 9, a displacement sensor 10, an electric cylinder-to-slide connector 5, and a precision slide 6; the precision slide 6 is connected to the electric cylinder-to-slide connector 5, the miniature electric cylinder 8 is connected to the electric cylinder-to-slide connector 5, the miniature electric cylinder 8 and the conductive rubber head 9 are connected; the electric cylinder-to-slide connector 5 and the displacement sensor 10 are connected, and the precision slide 6 and the slide-to-flange connector 7 are connected.

[0045] The miniature closed-loop electric cylinder is equipped with a flexible silicon-based conductive rubber end, which can be adapted to both capacitive touch screens and physical buttons. It has flexible buffering characteristics, will not scratch the screen surface, and the button triggering is stable.

[0046] The parameters of the miniature electric cylinder are as follows: effective stroke: 30mm; rated thrust: 100N; minimum thrust: 30N; maximum speed: 100mm / s; repeatability: ±0.02mm; The precision slide is controlled in a closed loop by a stepper motor driver to achieve high-speed feeding and precise locking. Its parameters are as follows: transmission type: linear ball screw; positioning accuracy: ≤15μm; repeatability: ±1.5μm; load: ≥10kg; stroke: ≥50mm; 1.4) Design of connectors: The overall layout of the connector module must ensure coaxiality and rigidity. It should be small in size and light in weight, suitable for operation in narrow spaces at the edge of the machine tool. The connector module is made of integrated aluminum alloy and includes robot flange interface, slide mounting surface, electric cylinder fixing lug, and sensor mounting slot. 1.5) Execution of the end-button operation: The displacement sensor outputs a distance detection signal in real time and uploads it to the control system to achieve distance closed-loop control and avoid collisions and false triggers. The complete system execution process is as follows: 1.5.1) After the collaborative robot moves the end effector above the work area, the feed mechanism module drives the precision slide to feed towards the CNC panel work surface at a set speed; the displacement sensor collects the distance between the end effector and the panel in real time until the distance between the end effector and the panel reaches the preset reference distance (e.g., 70mm), and the positioning is completed. 1.5.2) After the precision slide is locked, the miniature electric cylinder starts smoothly with a set constant acceleration. The push rod extends from rest with uniform acceleration, and the speed rises steadily without sudden changes or impacts. Before approaching the button surface, the miniature electric cylinder maintains a constant acceleration to ensure smooth movement and avoid instantaneous impact on the button and screen. 1.5.3) After the miniature electric cylinder accelerates to the target speed, it enters a constant speed motion stage, driving the flexible end to contact and press the button in a uniform manner; at the same time, the control system realizes constant force closed-loop control based on the displacement sensor, keeping the pressing force stable within the trigger range; 1.5.4) After pressing the button, the miniature electric cylinder retracts to its initial position at a constant speed; then the precision slide is unlocked and returns to the standby position at a constant speed; the entire reset process is uniform, smooth, and shock-free, and the end effector returns to a safe posture, waiting for the next operation command.

[0047] Module 2) Motion planning algorithm module for panel button operation: The coordinate system of the machine tool CNC system panel is calibrated, the spatial transformation relationship between the panel coordinate system and the collaborative robot coordinate system is established, and the panel button positions are mapped based on the transformation relationship. The corresponding robot arm button operation poses are obtained by solving the forward and inverse kinematics. The operation space is modeled and the button operation process is simulated in Unity. Collision detection is performed on the button operation process, and a set of operation poses that meet the collision-free constraints is selected to build a collision-free pose database. This embodiment uses a six-axis collaborative robot as the hardware platform, targeting confined workspaces such as machine tool CNC system panels and control screens. Through end-effector integration, multi-axis system calibration, improved DH kinematics calculation, Unity real-time collision detection, and optimal pose library construction, it achieves high-precision, interference-free, and directly executable button operation motion planning; such as... Figure 6 As shown, the specific process is as follows: 2.1) Multi-coordinate system calibration and point conversion of machine tool CNC system panel: Calibration control points are evenly distributed in the machine tool operation panel area. Each point is reached sequentially using robot teaching, and the joint angle parameters corresponding to each point are collected synchronously. A base coordinate system is constructed with the robot base mounting center as the origin. Using the calibration control points combined with three-point plane fitting and vector orthogonalization algorithms, custom user coordinate systems for the keyboard and screen operation areas are established in the Unity simulation environment. A pose transformation model between the coordinate systems is established through a homogeneous coordinate transformation algorithm. All target points of the machine tool CNC system panel buttons are transformed from the local user coordinate system to the robot base coordinate system, avoiding positioning deviations and coordinate transformation errors caused by multi-coordinate system switching. The XYZ spatial offset of the button execution end relative to the robot flange end face is measured, and an end effector coordinate system is created and integrated into the overall coordinate transformation system. This accurately matches the physical center of the button with the robot control target pose, fundamentally compensating for positioning errors caused by assembly and installation, and stably ensuring the positioning accuracy of the robot button operation. 2.2) Improved DH (MD-H) Kinematic Model Establishment: For the structure of a six-axis collaborative robot, the improved DH modeling parameters are determined, including link length, link twist angle, joint offset, and joint rotation angle. The following adjacent link coordinate system is established, and the homogeneous transformation matrix between links is derived to form a complete kinematic model of the robot, providing mathematical support for forward and inverse kinematics solutions.

[0048] ; In the formula: i 1 T i — Link transformation matrix; θ i — Joint angle; a i 1— Link length; α i 1 — Linkage torsion angle; d i — Joint offset; 2.3) Forward Kinematics Solution: Multiply the homogeneous transformation matrices of each link sequentially from the base to the end effector to obtain the total pose transformation matrix from the base coordinate system to the end effector. Inputting joint angles allows for direct solution of the end effector position and attitude, achieving a fast and stable mapping from joint space to the working space. The homogeneous matrix from the base to the end effector is: In the formula: ; ; ; ; ; ; ; ; ; ; n i It is the unit vector component of the X-axis of the end coordinate system in the X / Y / Z directions of the base coordinate system; o i It is the unit vector component of the Y-axis of the end coordinate system in the X / Y / Z directions of the base coordinate system; a i It is the unit vector component of the Z-axis of the end coordinate system in the X / Y / Z directions of the base coordinate system; P i The X / Y / Z coordinates of the origin of the final coordinate system in the base coordinate system; 2.4) Inverse Kinematics Solution: Based on the known end-effector pose, the total pose transformation matrix is ​​inversely calculated to obtain the analytical formulas for the six-axis joint angles. Multiple feasible inverse solutions can be obtained for each target pose, covering various mechanical configurations and providing ample solution space for subsequent selection.

[0049] In the formula: ; ; ; ; ; 2.5) Valid solution screening: Based on the robot's joint motion range, singular point constraints and configuration rationality requirements, invalid solutions that exceed the limits, cause interference, or have abnormal postures are eliminated, and reasonable and valid inverse solutions that have no singular points, no wrist flipping, and continuous and smooth joint motion are retained to ensure the safe and smooth movement of the robot. 2.6) Verification of solution results: Substitute the selected effective inverse kinematics solutions into the forward kinematics equations to recalculate the end pose and compare the error with the target pose to ensure that the pose deviation is less than the system's allowable range. The verified inverse kinematics solutions can be used as effective joint parameters for simulation and actual execution. 2.7) Virtual working environment setup: Import 3D models of machine tools, control panels, camera brackets, and robot linkages into the Unity 3D simulation platform. Assemble the scene according to the actual dimensions and installation positions on site, such as... Figure 7 As shown, mesh colliders are added to all models and collision levels and detection rules are set to construct a confined space simulation scene consistent with the real working environment; 2.8) Determination of safe transition pose: By combining offline simulation and on-site teaching, a set of safe transition poses that do not interfere with any environmental components, have uniform joint angle distribution, and have no motion singularities are determined. These poses are used as the unified starting pose for all button point traversal and simulated motion, ensuring that all motion trajectories have a consistent and safe initial state. 2.9) Point-to-point motion simulation and collision detection: Starting from the safe transition pose and ending with the effective inverse solution of each target point, drive the robot to perform point-to-point motion simulation. Real-time collision detection is enabled throughout the simulation process to simultaneously determine whether the robot will interfere with itself or collide with external equipment, and to record the collision status and motion data in real time. 2.10) Optimal Collision-Free Pose Selection: Directly exclude poses that cause collisions, and statistically analyze indicators such as motion time, joint angular velocity change, and trajectory smoothness for effective collision-free poses. Based on the principle of prioritizing collision-free poses and minimizing motion time, determine a unique optimal execution pose for each button target.

[0050] 2.11) Collision-Free Optimal Pose Database Construction and Standardized Output: First, the optimal joint pose, motion duration, collision detection results, and multi-coordinate system pose parameters corresponding to each button target are collected and organized, and standardized according to a unified specification. Next, data fields such as button number, joint angle, motion time, coordinate attribution, valid identifier, and verification status are set to build a well-organized collision-free pose database. Then, the integrity and rationality of the input data are verified, and duplicate data, joint over-range, format errors, and abnormal records are cleaned up to ensure that all pose data is safe, reliable, and can be called normally. Finally, the verified pose data is packaged and exported as a TXT or JSON format file to form a collision-free pose library that can be directly loaded and parsed by the robot controller, providing data support for offline path planning and one-click operation on site, thereby completing the fully automatic, high-precision, and collision-free operation control of the robot on the CNC system panel buttons of the machine tool.

[0051] Module 3) Collaborative Robot Software Module: Construct a collaborative robot software control system for executing machine tool CNC system panel operation tasks. The software control system calls the button operation poses based on the collision-free pose database, and organizes and schedules the machine tool CNC system panel operation tasks in a process-oriented manner. It drives the collaborative robot to complete the button operation through a preset task sequence, and realizes status monitoring and hierarchical alarm processing during the execution process, thereby ensuring the safety and stability of the operation process. This embodiment constructs a collaborative robot software control system for autonomous operation of a machine tool CNC system panel. Its overall structure includes a user interaction module, a task scheduling module, a collaborative function module, a communication module, and an anomaly and alarm module. These modules interact via data interfaces and jointly complete task construction, execution control, status monitoring, and anomaly handling, thus forming a closed-loop control system. The specific process is as follows: 3.1) Overall Software System Construction: such as Figure 8 As shown, the user interaction module is located at the upper layer of the system and is used to realize task construction, execution control and visualization of system operation status. Users can select collaboration functions and build task queues through this module, and can also view task execution status, system operation status and alarm information in real time.

[0052] During task execution, the system generates corresponding operation log data based on the running status information of each execution step, and updates and displays it in real time through the user interaction module to reflect the task execution progress and results.

[0053] The task scheduling module is used for unified management and scheduling of the task queue, and drives the collaborative function module to execute corresponding operations according to the task order in the task queue. The collaborative function module is used to encapsulate collaborative function functions related to the button operation of the machine tool CNC system panel, and completes the motion control of the robotic arm and the control of the end effector according to the scheduling instructions during task execution, thereby realizing specific button operations. The communication module is used to receive status information transmitted by the collaborative robot system and external sensing modules, including the collaborative robot system operating status information and visual recognition results, and parses and stores them to provide real-time data support for the system. The anomaly and alarm module is used to identify and classify the abnormal information generated during system operation, and execute corresponding control strategies according to the anomaly level.

[0054] In addition, the collaborative robot software control system interacts with the collision-free pose database established in module 2) during execution. By calling the calculated safe operation pose, it realizes the direct invocation of button operation, thereby avoiding the uncertainty brought about by real-time path planning and improving the system execution efficiency and stability.

[0055] The various functional modules interact with each other through a unified data interface, forming a closed-loop control structure for task construction, execution control, status monitoring, and anomaly handling, thereby ensuring that the machine tool CNC system panel operation tasks can be executed stably according to the predetermined process.

[0056] 3.2) Cooperative Function Encapsulation and Task Queue Construction: Based on the operational requirements of the machine tool CNC system panel, the cooperative function module encapsulates various button operations of the machine tool CNC system panel according to the machine tool operation process, forming multiple cooperative function units. Each cooperative function consists of multiple execution steps, and each execution step corresponds to one or more button operations and their operation result judgment conditions, thereby realizing the functional abstraction and structured organization of the machine tool CNC system panel operation process. The cooperative function is designed according to the actual button flow of the machine tool, combined with interface recognition and panel information feedback, to realize the automatic execution of machine tool operations. Furthermore, the collaborative functions cover typical operation processes such as machine tool startup preparation and machining operation, including but not limited to 15 collaborative operation functions such as machine tool power-on, initial self-test, tool magazine homing, power-off, screen wake-up, tool change, probe change, zero offset setting, program execution, program pause, coolant switching, emergency stop control, and machine tool door opening and closing. The machine tool operation flow of these 15 collaborative functions is as follows: Figure 9 As shown; After the collaborative function is encapsulated, it is invoked through the user interaction module. Users can click the corresponding function button to load the selected collaborative function into a task data structure and store it in the task queue for unified management. The task data structure includes task type, task name, execution parameters, and corresponding execution step information. Through the above task queue construction mechanism, multiple collaborative functions can be combined according to actual processing needs to form different machine tool operation task sequences, which are then executed in a set order, thereby realizing the flexible organization and scheduling of machine tool CNC system panel operation tasks.

[0057] 3.3) Task scheduling and execution control process: During task execution, the task scheduling module sequentially schedules the task data in the task queue and calls the function units in the corresponding collaborative function modules in turn. Simultaneously, it uses the pose database in module 2) to call the corresponding key operation pose parameters. Specifically, the execution control process includes: 3.3.1) Control the collaborative robot to move from its current pose to a preset safe transition pose; 3.3.2) Control the collaborative robot to move from the safe transition pose to the operation pose corresponding to the target button; 3.3.3) After the robotic arm reaches the target pose, the control end effector completes the button operation. That is, the distance information between the end effector and the CNC system panel of the machine tool is obtained through the infrared sensor, and the position is finely adjusted by the precision slide, and then the micro electric cylinder is driven to complete the button pressing. 3.3.4) After the button operation is completed, control the collaborative robot to return to the safe transition pose to wait for the next operation step to be executed; wherein, the safe transition pose is a common intermediate pose that satisfies the collision-free constraint and is reachable from all target poses.

[0058] By employing the above execution method, the continuous path planning problem is transformed into an execution process based on discrete pose calls, thereby reducing the complexity of path planning and improving the stability of system operation.

[0059] 3.4) Status monitoring mechanism: During task execution, the communication module continuously receives system status information, and the software system monitors the operating status of the collaborative robot in real time. The monitoring content includes the movement status of the robotic arm, the task execution status, and the visual monitoring information of the panel.

[0060] Specifically, by identifying changes in the machine tool interface status through visual monitoring information, the system can determine whether button operations were successful and whether the triggering conditions for the next execution step were met, thereby achieving dynamic judgment and control of the task execution process. Simultaneously, the system collects, records, and feeds this status information back to the user interaction module for display, providing users with real-time monitoring data.

[0061] 3.5) Tiered Alarm Handling Mechanism: Based on the status monitoring results, the anomaly and alarm module identifies abnormal information generated during system operation and handles it in a tiered manner according to preset rules, including: 3.5.1) When a suggestive anomaly is detected, only an alarm message is output without interrupting the execution of the current task; 3.5.2) When a general anomaly is detected, pause the current task execution and stop the collaborative robot's current movement; 3.5.3) When a serious anomaly is detected, immediately stop the movement of the collaborative robot and control it to perform an emergency stop operation on the machine tool to ensure the safe operation of the machine tool and the collaborative robot system.

[0062] The above-mentioned hierarchical alarm handling mechanism enables differentiated handling of abnormal situations at different levels, ensuring system security while maintaining the continuity of task execution and improving the overall reliability of system operation.

[0063] Module 4) Collaborative Operation Implementation Case of CNC System Panel: The overall operation flow of the machine tool CNC system panel is as follows: Figure 10 As shown, based on the collaborative robot hardware system, motion planning results, and collaborative robot software control system, an autonomous operation process for the machine tool CNC system panel is constructed. By calling the operation poses in the collision-free pose database and combining them with a task scheduling mechanism, the automatic execution of various button operations on the machine tool CNC system panel is achieved, thereby completing the autonomous operation of the machine tool CNC system. The specific process is as follows: 4.1) Panel calibration and pose database preparation: Before the machine tool CNC system panel operation task is executed, the machine tool CNC system panel is first spatially calibrated to establish the spatial transformation relationship between the machine tool CNC system panel coordinate system and the collaborative robot coordinate system. Based on this, the positions of each button on the machine tool CNC system panel in the panel coordinate system are mapped to the collaborative robot coordinate system, and the corresponding robot arm operation pose is obtained through pose calculation.

[0064] Furthermore, collision detection and reachability analysis are performed on the above poses in the operation space model to filter out the set of operation poses that meet the collision-free constraints, thereby constructing a collision-free pose database; the pose database is used to provide stable and reliable pose input for various subsequent button operations.

[0065] 4.2) Task Queue Construction and Operation Flow Organization: After completing the pose database construction, the machine tool CNC system panel operation tasks are constructed through the user interface, transforming the actual machine tool operation requirements into a task sequence composed of multiple collaborative functions. Each task is encapsulated in a unified data structure and loaded into the task queue for unified management according to a set order; the task data structure includes task type, task name, execution parameters, and corresponding step information.

[0066] Based on the above task construction method, various machine tool operation processes can be flexibly combined according to processing requirements. For example, in the machine tool power-on initialization and program execution scenario, a task sequence including machine tool power-on, self-test control, tool magazine homing, and program startup can be constructed and executed sequentially. Furthermore, this task construction method is not limited to the example process described above; users can construct different operation processes according to their actual needs.

[0067] 4.3) Task Queue Scheduling and Pose Invocation: During task execution, the execution thread sequentially traverses and schedules the tasks in the task queue for execution. During execution, the robot arm's pose parameters required for the corresponding button operation are invoked sequentially according to the task steps; these pose parameters originate from the pose database constructed in step 4.1). By associating and invoking task steps with pose data, an effective connection between machine tool operation tasks and robot arm motion control is achieved, thereby providing stable pose input for button operations.

[0068] 4.4) Button operation execution process: During the button operation execution process, the collaborative robot completes the button operation according to the preset execution process.

[0069] Specifically, the collaborative robot is first controlled to move from its current pose to a preset safe transition pose, and then from the safe transition pose to the operation pose corresponding to the target button, so as to ensure the safety of the movement process.

[0070] Once the robotic arm reaches the target pose, the button operation is completed through the end effector. The specific process includes: 4.4.1) Obtain distance information between the end effector and the machine tool CNC system panel using an infrared sensor in the end effector; 4.4.2) Based on the distance information, the position is finely adjusted using the slide assembly to improve alignment accuracy; 4.4.3) Drive the electric cylinder to complete the button action, so as to achieve stable triggering of the buttons or screen of the machine tool CNC system panel; 4.4.4) After pressing the button, control the collaborative robot to return to the safe transition pose and wait for the next operation step.

[0071] The above execution method enables stable and repeatable operation of the buttons on the CNC system panel of the machine tool.

[0072] 4.5) Status Determination and Process Progression: After the button operation is completed, the current machine tool operation status is determined. The determination includes membrane key feedback detection based on button triggering and visual status recognition based on changes in the machine tool CNC system panel interface; wherein, button feedback is used to determine whether the button was effectively triggered, and visual recognition is used to determine whether the current operation is completed and whether the conditions for entering the next operation step are met.

[0073] When the judgment result meets the preset conditions, the task queue is pushed forward to execute subsequent tasks, thereby realizing the continuous execution of operation processes such as machine tool power-on initialization and program operation.

[0074] 4.6) Anomaly Detection and Handling: During task execution, the machine tool's operating status and the working environment are monitored in real time. Anomaly information originates from the vision module's identification of alarm information on the machine tool's CNC system interface and the identification of the working environment status; specifically, anomaly information includes machine tool operation alarm information and human-machine environment anomaly information, such as abnormal personnel attire, incorrect tool type, and milling fluid not being turned on. When an anomaly is detected, corresponding handling strategies are executed according to the anomaly type, including stopping the current task execution and providing software prompts; in the case of severe anomalies, the collaborative robot is controlled to perform an emergency stop operation on the machine tool to ensure the operational safety of the machine tool and collaborative robot system.

[0075] The above-mentioned exception handling mechanism improves the security and reliability of system operation.

Claims

1. A collaborative robot system for autonomous operation of a machine tool CNC system panel, characterized in that, Includes the following modules: Module 1) Collaborative Robot Hardware Module: Constructs a collaborative robot hardware system for performing CNC system panel operation tasks on machine tools. The collaborative robot hardware system is designed around the operating mechanism of "command issuance - visual perception - system decision-making - collaborative execution - machine tool movement". The collaborative robot hardware system includes the collaborative robot body, vision camera, membrane keypad, multi-port switch, mounting base and related assembly components, as well as end effector for completing keypad operations. Module 2) Motion planning algorithm module for button operation: The coordinate system of the machine tool CNC system panel is calibrated, the spatial transformation relationship between the panel coordinate system and the collaborative robot coordinate system is established, and the panel button positions are mapped based on the transformation relationship. The corresponding robot arm button operation poses are obtained by solving the forward and inverse kinematics. The operation space is modeled and the button operation process is simulated in Unity. Collision detection is performed on the button operation process, and the set of operation poses that meet the collision-free constraints is selected to build a collision-free pose database. Module 3) Collaborative Robot Software Module: Construct a collaborative robot software control system for executing machine tool CNC system panel operation tasks. The software control system calls the button operation poses based on a collision-free pose database, and organizes and schedules the machine tool CNC system panel operation tasks in a process-oriented manner. It drives the collaborative robot to complete button operations through preset task sequences, and realizes status monitoring and hierarchical alarm processing during the execution process. Module 4) Collaborative Operation Implementation Module for CNC Machine Tool Panel: Based on the collaborative robot hardware system, motion planning results and collaborative robot software control system, an autonomous operation process for the machine tool CNC system panel is constructed. By calling the operation poses in the collision-free pose database and combining them with the task scheduling mechanism, the automatic execution of various button operations on the machine tool CNC system panel is realized, thereby completing the autonomous operation of the machine tool CNC system.

2. The collaborative robot system according to claim 1, characterized in that, The specific process of module 1) is as follows: 1.1) First, construct a system operation mechanism for the CNC system panel operation tasks of machine tools. The operation mechanism includes the execution process of "instruction issuance - visual perception - system decision-making - collaborative execution - machine tool movement". During the operation, "visual perception" continuously acquires interface status information recognition, alarm information monitoring and environmental status monitoring, and participates in system decision-making by providing visual recognition information. A system-level hardware structure is constructed, consisting of a sensing unit, an execution unit, a control unit, and a communication unit. The sensing unit includes a vision camera and membrane keys, used to acquire machine tool interface status information, alarm information, and operating environment status information, and feed the necessary information back to the control unit for system decision-making. The execution unit includes the collaborative robot body and its end effector, used to complete the machine tool CNC system panel key operation tasks based on the system decision results, realizing the collaborative execution process. The control unit is the collaborative robot software control system, used to realize task parsing, scheduling control, and execution logic judgment, and to uniformly manage and process the tasks after the instructions are issued. The communication unit is a multi-port switching device, used to establish data interaction channels between various hardware modules, realizing the transmission and sharing of instruction information and status information. 1.2) Based on the structural dimensions of the machine tool edge operating space and the distribution characteristics of the CNC panel buttons, the overall layout design of the installation method of the collaborative robot in the machine tool working space is carried out; by designing the structure of the mounting base and the robotic arm mounting frame, and optimizing the installation height and relative position relationship, the effective working space of the collaborative robot can cover the operating area of ​​the machine tool CNC system panel, while avoiding spatial interference with the machine tool body and surrounding equipment. 1.3) The end effector of a collaborative robot includes the following modules: Button mechanism module: miniature electric cylinder, conductive rubber flexible end; Feed mechanism module: precision slide table, stepper motor driver; Displacement detection module: Infrared laser displacement sensor; Connector module: Robot end effector flange and slide plate connector, button mechanism base; The sensor is coaxially mounted with the miniature electric cylinder to measure the distance from the end to the working surface in real time; the precision slide is responsible for long-distance coarse positioning, and the miniature electric cylinder is responsible for short-stroke fine pressing; the closed-loop electric cylinder of the miniature electric cylinder is equipped with a silicon-based conductive rubber end, which is compatible with capacitive touch screens and physical buttons. 1.4) The overall layout of the connector module must ensure coaxiality and rigidity, and be suitable for operation in narrow spaces at the edge of the machine tool. The connector module is made of integrated aluminum alloy and includes robot flange interface, slide mounting surface, electric cylinder fixing lug, and sensor mounting slot. 1.5) The displacement sensor outputs a distance signal in real time, which is then used by the control system. The system execution process is as follows: 1.5.1) After the collaborative robot moves the end effector above the work area, the feed mechanism module drives the precision slide to feed towards the CNC panel work surface at a set speed; the displacement sensor collects the distance between the end effector and the panel in real time until the distance between the end effector and the panel reaches the preset reference distance; 1.5.2) After the precision slide is locked, the miniature electric cylinder starts smoothly with a set constant acceleration. The push rod extends with uniform acceleration from rest. Before approaching the button surface, the miniature electric cylinder maintains a constant acceleration. 1.5.3) After the miniature electric cylinder accelerates to the target speed, it drives the flexible end to contact and press the button in a uniform manner; at the same time, the control system realizes constant force closed-loop control based on the displacement sensor to keep the pressing force stable within the trigger range; 1.5.4) After pressing the button, the miniature electric cylinder first retracts to the initial position at a constant speed; then the precision slide is unlocked and returns to the standby position at a constant speed; the end effector returns to the safe posture and waits for the next operation command.

3. The collaborative robot system according to claim 1, characterized in that, The specific process of module 2) is as follows: 2.1) First, uniformly distributed calibration reference points are set up on the machine tool operation panel. The corresponding joint angle data are collected by touching each point using the robot teaching mode. A base coordinate system is established with the robot base mounting center as the reference. Combined with the calibration reference points, a three-point plane fitting and vector orthogonalization algorithm is used to build user coordinate systems for the keyboard and screen operation areas in the Unity environment. The pose transformation between the base coordinate system and each user coordinate system is realized through homogeneous coordinate transformation. All target points of the buttons on the machine tool CNC system panel are converted and mapped from the user coordinate system to the robot base coordinate system. At the same time, the XYZ spatial offset of the end effector relative to the robot flange is measured, the end effector coordinate system is established and incorporated into the overall coordinate transformation system, and the button center and robot control target pose are accurately matched. 2.2) Determine the improved DH parameters, including link length, link twist angle, joint offset, and joint rotation angle; establish the coordinate system of adjacent links, derive the homogeneous transformation matrix between links, and form a complete kinematic model of the robot, providing a mathematical basis for forward and inverse kinematics solutions. The general formula for the transformation matrix between adjacent links is: ； In the formula: i 1 T i — Link transformation matrix; θ i — Joint angle; a i 1— Link length; α i 1 — Linkage torsion angle; d i — Joint offset; 2.3) Multiply the homogeneous transformation matrices of each link sequentially from the base to the end effector to obtain the total pose transformation matrix from the base coordinate system to the end effector; input the joint angles to directly solve for the end effector position and attitude, realizing the mapping from joint space to working space, where the homogeneous matrix from the base to the end effector is: ； In the formula: ; ; ； ； ； ； ； ； ； ； n i It is the unit vector component of the X-axis of the end coordinate system in the X / Y / Z directions of the base coordinate system; o i It is the unit vector component of the Y-axis of the end coordinate system in the X / Y / Z directions of the base coordinate system; a i It is the unit vector component of the Z-axis of the end coordinate system in the X / Y / Z directions of the base coordinate system; P i The X / Y / Z coordinates of the origin of the final coordinate system in the base coordinate system; 2.4) Given the end-effector pose, perform inverse operation on the total pose transformation matrix to obtain multiple analytical solutions for the joint angles; for each target pose, obtain multiple feasible inverse solutions, covering different mechanical configurations. The results of the inverse kinematics analytical solutions are shown below: In the formula: ; ; ; ； ； 2.5) Based on the physical limits of robot joints, motion singularities, and configuration constraints, invalid solutions that exceed limits, interfere, or have abnormal postures are eliminated, while valid inverse solutions that are free of singularities, wrist flips, and have smooth joint movements are retained. 2.6) Substitute the selected valid inverse kinematics back into the forward kinematics equations, recalculate the end pose and compare the error with the target pose to ensure that the pose error is less than the system's allowable threshold; the valid inverse kinematics that have passed the verification are applied to the valid joint parameter combinations in the simulation and execution. 2.7) In the Unity 3D simulation platform, import the 3D models of the machine tool body, operation panel, camera bracket, and collaborative robot linkage, and complete the scene assembly according to the actual size and installation position on site; add mesh collision body components to all models, divide the collision levels, set the collision detection rules between the robot linkage and environmental obstacles, and construct a confined space virtual working environment consistent with the real working conditions; 2.8) By combining offline simulation with on-site teaching experiments, a set of safe transition poses is determined and used as the unified starting pose for all button target point pose traversal and motion simulation, ensuring that each motion trajectory has consistent and safe initial conditions. 2.9) Starting from the safe transition pose and ending with the effective inverse kinematics of each target point after screening, drive the robot to perform point-to-point motion simulation; enable real-time collision detection throughout the simulation process, synchronously determine whether the robot has its own linkage interference or collision with the external environment such as the machine tool or camera, and record the collision status and motion process data in real time. 2.10) Pose that detects a collision is directly eliminated; for valid poses without collision, the motion time, joint angular velocity change, and trajectory smoothness are statistically analyzed; according to the principles of prioritizing no collisions, shortest motion time, and best trajectory smoothness, a unique optimal execution pose is selected and determined for each button target point. 2.11) First, standardize and normalize the optimal joint angle, motion time, and collision state of the button target point; then define a collision-free pose database containing fields such as button number and joint angle; subsequently, perform validation and deduplication on the data in the database and remove invalid entries; finally, serialize the collision-free pose database into a TXT / JSON file to generate a collision-free pose library that can be directly called by the robot controller.

4. The collaborative robot system according to claim 1, characterized in that, The specific process of module 3) is as follows: 3.1) Construct a collaborative robot software control system. The software control system is used to realize the unified scheduling and execution control of the machine tool CNC system panel operation tasks. The software interactive interface displays the task execution status, system operation status and alarm information in a visual manner, so that the operator can obtain system operation information in real time and monitor and intervene in the collaborative robot execution process. 3.2) In the collaborative function module, the various button operations of the machine tool CNC system panel are encapsulated according to the machine tool operation process to form multiple collaborative function units. Each collaborative function consists of multiple execution steps, and each execution step corresponds to one or more button operations and their operation result judgment conditions, thereby realizing the functional abstraction and structured organization of the machine tool CNC system panel operation process. The collaborative function is designed based on the actual button flow of the machine tool, combined with interface recognition and panel information feedback, to realize the automatic execution of machine tool operations. 3.3) During task execution, the corresponding collaborative function encapsulation function is called sequentially according to the task sequence, and the robot arm operation posture parameters required for the button operation are called according to the execution steps. The button operation of the machine tool CNC system panel is completed by controlling the movement of the collaborative robot. 3.4) During task execution, the communication module monitors the operating status of the collaborative robot system in real time. The monitoring content includes the movement status of the robotic arm, the task execution status, and the panel visual monitoring information. The module also collects, records, and displays the status information. 3.5) Based on the status monitoring results, a hierarchical alarm processing mechanism is constructed to classify and process abnormal situations according to the system operating status.

5. The collaborative robot system according to claim 4, characterized in that, Step 3.1) describes the collaborative robot software control system, which includes a user interaction module, a task scheduling module, a collaborative function module, a communication module, and an anomaly and alarm module. The user interaction module is used not only for task construction and execution control but also for real-time updating and displaying of the task execution log information, thus intuitively reflecting the task execution progress and results. The collaborative function module encapsulates collaborative function functions related to the button operations on the machine tool CNC system panel and provides the basic interfaces required for robotic arm motion control and end effector control. The communication module receives status information transmitted from the machine tool system and external sensing modules, and parses and stores it. The anomaly and alarm handling module identifies and classifies abnormal information generated during system operation. During motion control, the collaborative robot software control system interacts with the collision-free pose database established in module 2), directly invoking each button operation by calling the pre-calculated and verified safe operating poses in the database.

6. The collaborative robot system according to claim 4, characterized in that, Step 3.2) implements collaborative functions covering the machine tool startup preparation, machining operation, and safety control processes, including machine tool power-on, initial self-test, tool magazine homing, power-off, screen wake-up, tool change, probe change, zero offset setting, program execution, program pause, coolant switching, emergency stop control, and machine tool door opening and closing. After the collaborative functions are encapsulated, they are invoked through the user interaction module. Users can load the selected collaborative functions into task data structures by clicking the corresponding function buttons and store them in the task queue for unified management. The task data structure includes task type, task name, execution parameters, and corresponding execution step information. Through the above task queue construction mechanism, multiple collaborative functions are combined according to actual machining needs to form different machine tool operation task sequences, which are executed sequentially according to the set order, thereby realizing flexible organization and scheduling of machine tool CNC system panel operation tasks.

7. The collaborative robot system according to claim 4, characterized in that, Step 3.3) The execution control process includes: 3.3.1) Control the collaborative robot to move from its current pose to a preset safe transition pose; 3.3.2) Control the collaborative robot to move from the safe transition pose to the operation pose corresponding to the target button; 3.3.3) After the robotic arm reaches the target pose, the control end effector completes the button operation, including using an infrared sensor to obtain the distance information between the end effector and the machine tool CNC system panel, and using a precision slide to make fine adjustments to the position, and then driving a micro electric cylinder to complete the button pressing operation on the machine tool CNC system panel or screen; 3.3.4) After the button operation is completed, control the collaborative robot to return to a safe transition pose to wait for the next operation step to be executed; The safe transition pose is a common transition state that satisfies collision-free constraints within the confined space of the machine tool and is achievable for all target operation poses.

8. The collaborative robot system according to claim 1, characterized in that, The specific process of module 4) is as follows: 4.1) Before executing the operation task on the machine tool CNC system panel, the machine tool CNC system panel is first spatially calibrated to establish the spatial transformation relationship between the machine tool CNC system panel coordinate system and the collaborative robot coordinate system; the positions of each button on the machine tool CNC system panel in the panel coordinate system are mapped to the collaborative robot coordinate system, and the corresponding robot arm operation pose is obtained through pose calculation. 4.2) The machine tool CNC system panel operation tasks are constructed through the software interactive interface, and the actual machine tool operation requirements are transformed into a task sequence consisting of multiple collaborative functions and their corresponding button operation processes; each task is encapsulated in a unified data structure form and loaded into the task queue for unified management according to the execution order set by the user. 4.3) During task execution, the task data structure in the task queue is sequentially traversed and scheduled for execution by the execution thread; during execution, the robotic arm operation pose parameters required for the corresponding button operation are called sequentially according to each task step; 4.4) During the execution of button operation, the collaborative robot is controlled to complete the button operation according to the preset execution process; first, the collaborative robot moves from the current pose to the preset safe transition pose, and then moves from the safe transition pose to the operation pose corresponding to the target button; Once the robotic arm moves to the target pose, the end effector completes the specific button operation process, including: 4.4.1) Obtain distance information between the end effector and the machine tool CNC system panel using an infrared sensor in the end effector; 4.4.2) Based on the distance information, the position of the end effector is finely adjusted by a precision slide in the end effector to improve the alignment accuracy between the end effector and the target button; 4.4.3) The miniature electric cylinder in the drive end effector completes the button pressing action, thereby realizing stable triggering of the buttons on the CNC system panel of the machine tool; 4.4.4) After the button operation is completed, control the collaborative robot to return to the safe transition pose to wait for the next operation step to be executed; 4.5) After the key operation is completed, the current operating status of the machine tool is determined; the determination process includes membrane key feedback detection based on key trigger and visual status recognition based on changes in the CNC system panel interface of the machine tool. 4.6) Anomaly Detection and Handling: During task execution, the machine tool's operating status and working environment are monitored in real time. Anomaly information comes from the vision module's identification of alarm information on the machine tool's CNC system interface and the identification results of the working environment status. Among them, anomaly information includes machine tool operation alarm information and human-machine environment anomaly information. Human-machine environment anomaly information includes personnel not wearing the required clothing, incorrect tool type identification, and milling fluid not being turned on. When an anomaly is detected, the corresponding handling strategy is executed according to the anomaly type.

9. The collaborative robot system according to claim 8, characterized in that, Step 4.1) Perform collision detection and reachability analysis on the above poses in the operation space model, and select the set of robot arm operation poses that meet the collision-free constraints, thereby constructing a pose database for the operation of the CNC system panel buttons on the machine tool.

10. The collaborative robot system according to claim 8, characterized in that, The task data structure described in step 4.2) includes task type, task name, execution parameters and corresponding operation steps, which are used to describe the execution content and execution order of each collaborative function.

Images