Mechanical arm end system integrated with teaching execution and teaching and execution method

Abstract

The application discloses a kind of demonstration execution integrated mechanical arm end system and demonstration and execution method, the system includes detachable demonstration tool, end effector main body and learning and control module.Demonstration tool is integrated with multiple sensors, and end effector main body is provided with composite interface.The method includes: when demonstration tool is detached and operated by artificial, system generates job model automatically based on the multimodal data collected by demonstration tool itself;When demonstration tool is installed back to end effector main body, system automatically calls job model and controls mechanical arm to drive the same demonstration tool to execute job.The application solves the problems of current demonstration technology, such as calibration error caused by separation of demonstration and execution tools, single information collection, low deployment efficiency and poor adaptability, etc., realizes natural and intuitive demonstration, high-precision job reproduction and rapid production changeover, and improves the application efficiency of robots in flexible manufacturing.

Description

Technical Field

[0001] This invention relates to the field of teaching robotic arms, and more particularly to a robotic arm end effector system that integrates teaching and execution, as well as a teaching and execution method. Background Technology

[0002] With the deepening development of industrial automation and intelligent manufacturing, industrial robots have become core equipment for modern manufacturing to improve production efficiency and product quality, playing a crucial role, especially in scenarios with extremely high requirements for flexibility and precision, such as high-precision assembly and electronic product assembly. Faced with the increasing demand for multi-variety, small-batch, and customized production, the traditional working mode of robots, which relies on pre-programming or complex offline programming, is no longer able to adapt to the production needs of rapid production changeover and flexible response.

[0003] Currently, common robot teaching methods mainly include point-to-point teaching with a teach pendant, drag-and-drop teaching, and vision-based assisted teaching. Teach pendant teaching requires operators to manually control the robot to record key points using a handheld device, which is cumbersome and relies on human experience. While drag-and-drop teaching is relatively intuitive, allowing operators to directly drag the robot's end effector to record the trajectory, it still requires a separate tool for teaching, separating it from the actual execution tool. Vision-based teaching uses an external camera to model the environment to assist in path generation, but the system is complex and susceptible to calibration errors.

[0004] These methods generally focus on recording robot pose information, making it difficult to simultaneously acquire multi-dimensional information such as force feedback, near-field posture, and tool status during operation. Furthermore, the tools used in the teaching phase are independent of the actual tools used in the execution phase, with differences in size, weight, and installation method. This results in the inability to accurately reproduce the robot's learned motion trajectory and mechanical model, introducing additional calibration errors and accuracy losses. Consequently, when facing real-world industrial scenarios such as frequent production changes and delicate operations, the system suffers from low deployment efficiency, poor adaptability, and difficulty in meeting the requirements of modern flexible manufacturing for rapid response and high-precision reproduction. Summary of the Invention

[0005] This invention provides a robotic arm end effector system that integrates teaching and execution, as well as a teaching and execution method. This solves the problems in current teaching technologies, such as calibration errors, limited information acquisition, low deployment efficiency, and poor adaptability caused by the separation of teaching and execution tools. It achieves natural and intuitive teaching, high-precision operation reproduction, and rapid production changeover, thereby improving the application efficiency of robots in flexible manufacturing.

[0006] This invention provides a teaching-execution integrated robotic arm end effector system, comprising a teaching tool, an end effector body, and a learning and control module. The teaching tool is a detachable tool. The front end of the teaching tool is provided with an operating part for performing operations. The teaching tool integrates a first sensor group for collecting operation data. The end effector body is mounted on the end of the robotic arm. The end effector body is provided with a composite interface for docking with the teaching tool. The composite interface is used to realize the mechanical installation, electrical connection and vacuum circuit communication of the teaching tool on the end effector body. The learning and control module is communicatively connected to the end effector body and the teaching tool. When the teaching tool is detached from the end effector body, it receives and integrates operation data from the first sensor group to generate a work model. When the teaching tool is installed on the end effector body through the composite interface, it controls the robotic arm to drive the teaching tool to perform the work based on the work model.

[0007] Optionally, the first sensor group includes at least: a multi-dimensional force sensor for acquiring operational force information, and a macro camera for acquiring local visual information of the assembly area.

[0008] Optionally, the teaching tool is a pen-shaped structure suitable for manual one-handed holding, with a vacuum nozzle at its front end; the teaching tool also integrates a vacuum barometer for detecting the adsorption state of the vacuum nozzle.

[0009] Optionally, the end effector body integrates a wide-angle camera for acquiring environmental visual information; When generating the task model, the learning and control module performs spatiotemporal alignment and fusion of local visual information from the macro camera and environmental visual information from the wide-angle camera.

[0010] Optionally, the end effector is also integrated with a microphone array for receiving voice command information during teaching or execution. The learning and control module semantically correlates the voice command information with visual data from macro and wide-angle cameras, as well as force data from a multi-dimensional force sensor, and incorporates the semantic correlation results into the operation model.

[0011] Optionally, the composite interface on the end effector body includes a mechanical locking unit, an electrical connection unit, and a fluid passage unit; The mechanical locking unit is used to achieve detachable mechanical fixation of the teaching tool on the end effector body; The electrical connection unit includes an electrical connector for establishing a bidirectional data communication and power supply connection with the teaching tool after the teaching tool is installed. The fluid passage unit includes a pneumatic connector for connecting the air passage of the end effector body to the air passage of the operating part of the teaching tool after the teaching tool is installed.

[0012] Optionally, the mechanical locking unit, electrical connection unit, and fluid passage unit are configured to work together so that when the teaching tool performs an installation or removal action relative to the end effector body, the mechanical locking / release, electrical connection / disconnection, and air passage connection / closure are switched synchronously.

[0013] Furthermore, to achieve the aforementioned objectives, this invention also provides a teaching and execution method for a robotic arm that integrates teaching and execution, utilizing the aforementioned robotic arm end effector system. The method includes: When the teaching pendant is detached from the end effector body and is performing the target task, a task model for the target task is automatically generated based on the operation data collected by the teaching pendant itself. In response to the teaching tool being installed on the end effector body, the work model is automatically invoked, and the robotic arm is controlled to drive the same teaching tool to execute the target work defined by the work model.

[0014] Optionally, the step of automatically generating the task model of the target task based on the operation data collected by the teaching tool itself includes: The system receives and fuses force data from the multi-dimensional force sensor within the teaching tool with local visual data from the macro camera to generate a task model that includes force control strategies and visual trajectories.

[0015] Optionally, the step of automatically generating the job model for the target job further includes: Receives environmental visual information from a wide-angle camera on the end effector body; The environmental visual information is aligned and fused with the local visual data to generate an operational model with environmental perception and adaptive capabilities.

[0016] One or more technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages: 1. By integrating a detachable teaching pendant with the end effector body, the calibration error and accuracy loss caused by the independence of the teaching pendant and the execution tool in related technologies are solved. The teaching pendant used for data acquisition is directly used as the execution tool, achieving physical unification of the teaching and execution tools, eliminating installation and calibration errors caused by tool switching, and improving the accuracy and reliability of job reproduction.

[0017] 2. By integrating a multi-dimensional force sensor and a macro camera into the teaching tool, the problem of traditional teaching tools being able to only record pose information and unable to acquire multi-dimensional data of the fine operation process is solved. This allows the system to simultaneously collect key information such as force perception and local vision during the teaching phase, thereby learning and generating a work model that includes force control strategies and fine visual trajectories, enabling the robotic arm to complete complex tasks such as high-precision flexible assembly.

[0018] 3. By designing a composite interface that integrates mechanical, electrical, and vacuum circuit connections, and integrating it with the learning and control module, the problems of cumbersome system deployment and low changeover efficiency were solved. This composite interface supports rapid plugging and unplugging of teaching tools and automatic status recognition. Combined with the automatic modeling and calling functions of the learning and control module, it enables a rapid deployment process that can be automatically executed after a single manual teaching, greatly shortening production line changeover time and effectively adapting to the flexible production needs of multi-variety, small-batch production.

[0019] 4. By integrating a wide-angle camera and microphone array as secondary sensing modules into the end effector and fusing them with the sensing data collected by the teaching tool, the problems of the robot's operational model lacking environmental understanding and semantic interaction capabilities, as well as poor adaptability, are solved. Spatiotemporal alignment and semantic association of environmental-level visual information, voice information, and operational-level multimodal data are achieved, enabling the generated operational model to possess environmental perception and adaptive adjustment capabilities, and to respond to advanced semantic commands, thus improving the system's intelligence in complex or dynamic scenarios.

[0020] 5. The above system and technical processes are refined into a state-driven teaching and execution method, which automatically switches learning and execution modes in response to the disassembly / installation state of the teaching tool, providing a natural, efficient, and precise robotic operation paradigm. This directly and seamlessly transforms human operational skills into reproducible machine instructions, reducing the technical requirements for operators while simultaneously improving the overall intelligence level and application efficiency of the robotic system. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the end effector system of the robotic arm integrating teaching and execution according to the present invention; Figure 2 This is a flowchart illustrating the integrated teaching and execution method for a robotic arm according to the present invention. Figure 3 This is a schematic diagram of the terminal structure of the hardware operating environment involved in an embodiment of the present invention. Detailed Implementation

[0022] To address the issues of calibration errors, limited information acquisition, and low deployment efficiency caused by the separation of teaching and execution tools in robot teaching technology, this invention proposes an integrated teaching and execution robotic arm end effector system and method. It employs a detachable teaching tool that combines data acquisition and task execution functions, connecting to the robotic arm end effector via a composite interface. During the teaching phase, the tool is manually disassembled and operated, while the system automatically collects multimodal data from its built-in sensors, such as force sensors and macro cameras, to generate a task model. During the execution phase, the teaching tool is reassembled, and the system uses the model to drive the same tool to accurately reproduce the task. This achieves natural and intuitive teaching, unified tools, and comprehensive data integration, eliminating calibration errors that may arise from the calibration process. It significantly improves task accuracy, deployment speed, and system flexibility, meeting the requirements of intelligent manufacturing for rapid changeover and high-precision replication.

[0023] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the invention to those skilled in the art.

[0024] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0025] Example 1 In this embodiment, a robotic arm end effector system integrating teaching and execution is provided.

[0026] Reference Figure 1 The teaching and execution integrated robotic arm end effector system is connected to the robotic arm body 1001. The system includes a teaching tool 1003, an end effector body 1002, and a learning and control module (not shown in the figure).

[0027] The teaching pendant 1003 is a detachable tool. The front end of the teaching pendant is provided with an operating part for performing operations, and the inside of the teaching pendant integrates a first sensor group for collecting operation data. Optionally, the teaching pendant 1003 has a slender, pen-like structure, suitable for one-handed holding. Its front end has a vacuum nozzle 1006, serving as the operating part for directly performing tasks. The teaching pendant integrates miniaturized multimodal sensors, forming the first sensor group.

[0028] Specifically, the first sensor group includes: a multi-dimensional force sensor 1007, used for real-time, high-precision acquisition of multi-dimensional force or torque information experienced by the pen body during manual operation; a proximity sensor 1004, used for non-contact detection of the precise distance between the tip of the teaching pen nozzle and the workpiece surface; a macro camera 1008, with its optical axis pointing towards the nozzle working area, used to acquire local, high-magnification visual information of the assembly contact point, providing precise visual feedback at the operator level; and a vacuum barometer (not shown in the illustration, integrated inside the teaching tool), used to monitor the air pressure in the vacuum adsorption pipeline in real time, accurately determining the adsorption and release states.

[0029] Optionally, during manual teaching, the teaching tool can simultaneously capture multi-dimensional real-world operational data such as force perception information, macroscopic visual images, proximity distance, and the tool's own functional status (e.g., adsorption start / stop, air pressure value), fully recording the detailed process and dynamic strategies of manual operation, thus solving the shortcomings of traditional teaching methods that mainly rely on recording the spatial pose of the robot's end effector.

[0030] Optionally, the teaching pendant has a dual function. In teaching mode, it acts as a handheld data acquisition tool for workers, recording the detailed processes and dynamic strategies of manual operations. In execution mode, it is installed behind the end effector body, serving as the end effector of the robotic arm, replicating the detailed processes and dynamic strategies of manual operations. That is, the physical coordinate system, mechanical sensors, and visual sensors referenced by the teaching pendant for learning and execution are completely consistent, fundamentally avoiding calibration errors and model distortions caused by tool switching, ensuring high-fidelity skill reproduction.

[0031] The end effector body 1002 is installed at the end of the robotic arm. The end effector body is provided with a composite interface for docking with the teaching tool. The composite interface is used to realize the mechanical installation, electrical connection and vacuum circuit communication of the teaching tool on the end effector body. Optionally, the end effector body 1002 is fixedly mounted on the end flange of the robotic arm via its base interface, forming a multi-functional integrated platform for the entire system. This body specifically integrates a global vision perception module, a voice interaction module, a teaching tool docking and locking mechanism, and an energy and data hub module.

[0032] The global vision perception module, integrated into the main housing, includes at least one wide-angle camera 1005 facing the work area. This high-resolution camera continuously acquires images of the overall work scene centered on the robotic arm during the teaching phase, providing the system with global spatial information and context of the work environment. The global vision perception module enables the system to gain macroscopic scene recognition capabilities. Based on the visual information acquired by the global vision perception module, the robotic arm's work path and strategy can be adjusted online according to actual environmental characteristics, rather than mechanically replicating fixed points, thus improving the adaptability of task execution to changes in workpiece position and environmental layout.

[0033] The voice interaction module integrates a high-sensitivity microphone or an acoustically designed microphone array 1009, with its acoustic aperture facing the operator's usual position. This module is used to capture and parse natural language voice commands issued by the operator during the teaching process or before and after task execution. The voice interaction module enables the system to transform abstract task descriptions, such as "gentle placement" or "limited assembly of red parts," into specific constraints on force control parameters and execution sequence in the work model, enhancing the system's usability and the flexibility of task description.

[0034] The teaching pendant docking and locking mechanism, also known as the composite interface, is a key mechanical-electrical-fluid composite interface on the end effector body that directly couples with the detachable teaching pendant 1003. It enables quick, one-handed insertion and removal of the teaching pendant on the end effector body and automatic locking, while simultaneously establishing electrical connections and vacuum circuit communication.

[0035] Specifically, the composite interface consists of three functional units working together: a mechanical locking unit, providing reliable theoretical fixation and rapid release to ensure a precise and stable rigid connection between the teaching pendant and the robotic arm end effector when switching between teaching and execution states; an electrical connection unit, including a multi-pin electrical connector, responsible for transmitting sensor data and control commands and providing power after the teaching pendant is installed; and a fluid pathway unit, integrating a quick-release self-sealing pneumatic connector to instantly connect the internal air path of the end effector body with the vacuum nozzle air path at the front end of the teaching pendant after installation. The composite interface is the core of the integrated teaching and execution system, ensuring seamless connection and instantaneous recovery of the teaching pendant's spatial reference, data link, and power source when it functions as a data acquisition end and as an execution end, eliminating the need for repeated calibration and accumulated errors caused by tool replacement.

[0036] The energy and data hub module, built into the end effector's main body, is an electronic subsystem integrating power management, pneumatic control, and data exchange functions. Through the aforementioned composite interface, it continuously supplies the teaching pendant with the necessary power, provides and controls vacuum pressure on demand, and manages the high-speed, low-latency bidirectional transmission of all sensor data and control commands.

[0037] The learning and control module is communicatively connected to the end effector body 1002 and the teaching tool 1003. When the teaching tool is detached from the end effector body, it receives and integrates operation data from the first sensor group to generate a work model. When the teaching tool is installed on the end effector body through the composite interface, it controls the robotic arm to drive the teaching tool to perform the work based on the work model.

[0038] In this embodiment, the learning and control module is the core information processing and control hub of the system. In terms of physical deployment, it can adopt a distributed or centralized architecture: it can be integrated as an embedded system within the end effector body 1002, deployed in the robot arm's main controller, or run as a remote service on a cloud computing platform. Its core function is to implement a multi-level perception and coordination mechanism and complete the entire process from raw data to executable control commands, specifically including the synchronous collection, fusion analysis, and generation and invocation of the operation model of the perceived data.

[0039] Optionally, the multi-level perception and collaboration mechanism is as follows: when the operator performs the target task by holding a teaching tool, the system initiates the synchronous acquisition and fusion of multi-source asynchronous data, which includes environmental-level perception data stream, operation-level perception data stream, and instruction-level perception data stream.

[0040] Optionally, the environmental perception data stream is generated by a wide-angle camera 1005 on the end effector body, providing macroscopic spatial structure and static / dynamic environmental information of the work scene. The operation-level perception data stream is synchronously generated by multimodal sensors integrated into the teaching tool, specifically including: a multi-dimensional force sensor 1007 for capturing contact force and torque, a macro camera 1008 for acquiring microscopic features and relative pose of the workpiece surface, a proximity sensor 1004 for non-contact ranging, and a vacuum pressure sensor for monitoring the working status of the adsorption actuator. The command-level perception data stream is captured by a microphone array 1009 integrated into the end effector body, receiving and parsing natural language voice commands or descriptions generated by the operator during the teaching process.

[0041] Optionally, the multi-level perception collaboration mechanism also includes data fusion and model building processes. Specifically, the learning and control module assigns precise timestamps to all the aforementioned asynchronous data streams through a unified hardware clock or software synchronization protocol, achieving time alignment of cross-modal data. Then, it invokes embedded machine learning algorithm frameworks, such as imitation learning, dynamic motion primitives, or deep learning models, to perform feature extraction, correlation analysis, and fusion processing on the aligned environmental visual data, local macro visual data, voice command text (converted through speech recognition), multi-dimensional force perception data, tool spatial pose data, and tool state data. The output of this process is a structured digital operation model that not only encodes continuous motion trajectories but also integrates force interaction strategies, vision-based servo adjustment parameters, and response logic for specific semantic commands.

[0042] In this embodiment, the system workflow includes a teaching process and an execution process. During the teaching process, the operator removes the teaching tool from the end effector body and performs a complete real-world operation while holding the teaching tool. The system automatically and synchronously completes the collection and modeling of the aforementioned multi-level data in the background. During the execution process, the operator reattaches the view tool to the end effector body and issues execution commands via voice or interface. The learning and control module calls the pre-built operation model, obtains real-time sensor feedback from the teaching tool through the composite interface, and drives the robotic arm body 1001 to reproduce the previous manual operation.

[0043] In this embodiment, the learning and control module integrates information from three levels: environment, operation, and semantics. This enables the system to construct a flexible work strategy with environmental context awareness, haptic feedback adjustment, and natural language interaction capabilities. This enhances the work model's ability to effectively generalize from a single teaching demonstration, allowing it to adapt to minor changes in workpiece pose and adjustments to the environmental layout, and to understand the high-level intent of the task execution. This makes it better suited for flexible, small-batch production scenarios.

[0044] Based on the same inventive concept, this invention also provides a teaching and execution method corresponding to the system in Embodiment 1, as shown in Embodiment 2.

[0045] Example 2 Based on Embodiment 1, another embodiment of the present invention is proposed, with reference to... Figure 2 A teaching and execution method for a robotic arm that integrates teaching and execution, comprising: Step S1: In response to the teaching pendant being detached from the end effector body and performing the target task, an operation model of the target task is automatically generated based on the operation data collected by the teaching pendant itself. In this embodiment, in response to the system detecting that the teaching pendant is detached from the end effector body and is manually operated to perform the target task, the learning and control module automatically starts, receives and integrates multimodal operation data collected from the first sensor group on the teaching pendant.

[0046] Specifically, the six-dimensional force / torque sequence from the multi-dimensional force sensor is fused with the high-resolution image sequence from the macro camera at the feature level to generate a preliminary operational model that simultaneously encodes the continuous spatial trajectory, contact force control strategy, and vision-based fine pose adjustment parameters.

[0047] Optionally, robot tasks that rely on offline programming or tedious point-to-point teaching can be transformed into an intuitive and natural operation. Through the underlying fusion of force perception and macro vision, the robot can directly learn and reproduce the high-precision assembly skills that are extremely sensitive to force and microscopic alignment in human operations, providing a high-fidelity foundation for the work model.

[0048] Optionally, to enable the operation model to adapt to macroscopic working scenarios, while executing step S1, the learning and control module can simultaneously receive a global scene video stream from a wide-angle camera on the end effector body. The learning and control module uses a spatiotemporal registration algorithm to align and contextually fuse the aforementioned environmental-level visual information with the operational-level local visual information from the macro camera. This enhances the operation model into an enhanced operation model with environmental awareness and online adaptive capabilities. For example, the operation model can not only perform fixed relative movements but also dynamically adjust its absolute coordinate system based on the actual position of the workpiece as determined by global visual recognition, thereby possessing the ability to compensate for workpiece placement tolerances.

[0049] Optionally, if the operator provides voice input during the teaching process, the learning and control module also simultaneously receives an audio stream from the microphone array. The module's built-in speech recognition and natural language understanding components parse the audio, extract structured semantic instructions, and then, through attention mechanisms or semantic annotation technology, associate and bind the structured semantic constraints with the sensor data stream and force sensor data stream being constructed during the above-mentioned operation model construction process. The association relationship is then integrated into the final operation model as metadata or control parameters.

[0050] Step S2: In response to the teaching tool being installed on the end effector body, the work model is automatically invoked, and the robotic arm is controlled to drive the same teaching tool to execute the target work defined by the work model.

[0051] In this embodiment, in response to the system detecting that the teaching pendant has been installed through the mechanical locking mechanism of the composite interface and a stable electrical and pneumatic connection has been established, the learning and control module automatically switches from modeling mode to online execution mode. In online execution mode, the constructed task model is invoked in real time and used as a reference for the desired trajectory and strategy. Simultaneously, the joint states fed back by the robotic arm and the real-time sensor feedback from force sensors, proximity sensors, etc., on the teaching pendant are continuously read, forming a closed-loop control system based on the task model. This system uses preset algorithms, such as impedance control and visual servoing, to dynamically calculate control commands, driving the robotic arm to precisely operate the same teaching pendant, thus smoothly and accurately reproducing the target task defined by the task model.

[0052] This embodiment completes a closed-loop process from skill learning to skill execution. Using the physical connection status of the teaching tool as a trigger, it achieves seamless switching between two working modes. It simplifies the deployment process, enabling production deployment after a single teaching demonstration, meeting the high-efficiency production changeover requirements of flexible manufacturing systems.

[0053] Example 3 In this embodiment of the invention, a robotic arm teaching and execution device integrating teaching and execution is proposed.

[0054] Reference Figure 3 , Figure 3 This is a schematic diagram of the terminal structure of the hardware operating environment involved in an embodiment of the present invention.

[0055] like Figure 3 As shown, the control terminal may include: a processor 1001, such as a CPU, a network interface 1003, a memory 1004, and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The network interface 1003 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1004 may be high-speed RAM or stable non-volatile memory, such as disk storage. Alternatively, the memory 1004 may be a storage device independent of the aforementioned processor 1001.

[0056] Those skilled in the art will understand that Figure 3 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0057] like Figure 3 As shown, the memory 1004, which serves as a computer storage medium, may include an operating system, a network communication module, and a teaching and execution program for a robotic arm that integrates teaching and execution.

[0058] exist Figure 3 In the hardware structure of the integrated teaching and execution robotic arm teaching and execution device shown, the processor 1001 can call the integrated teaching and execution robotic arm teaching and execution program stored in the memory 1004 and perform the following operations: When the teaching pendant is detached from the end effector body and is performing the target task, a task model for the target task is automatically generated based on the operation data collected by the teaching pendant itself. In response to the teaching tool being installed on the end effector body, the work model is automatically invoked, and the robotic arm is controlled to drive the same teaching tool to execute the target work defined by the work model.

[0059] Optionally, the processor 1001 may call the teaching and execution program of the integrated teaching and execution robotic arm stored in the memory 1004, and further perform the following operations: The system receives and fuses force data from the multi-dimensional force sensor within the teaching tool with local visual data from the macro camera to generate a task model that includes force control strategies and visual trajectories.

[0060] Optionally, the processor 1001 may call the teaching and execution program of the integrated teaching and execution robotic arm stored in the memory 1004, and further perform the following operations: Receives environmental visual information from a wide-angle camera on the end effector body; The environmental visual information is aligned and fused with the local visual data to generate an operational model with environmental perception and adaptive capabilities.

[0061] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0062] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0063] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0064] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0065] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, third, etc., does not indicate any order. These words can be interpreted as names.

[0066] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0067] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A robotic arm end effector system integrating teaching and execution, characterized in that, The system includes a teaching tool, an end effector body, and a learning and control module. The teaching tool is a detachable tool. The front end of the teaching tool is provided with an operating part for performing operations. The teaching tool integrates a first sensor group for collecting operation data. The end effector body is mounted on the end of the robotic arm. The end effector body is provided with a composite interface for docking with the teaching tool. The composite interface is used to realize the mechanical installation, electrical connection and vacuum circuit communication of the teaching tool on the end effector body. The learning and control module is communicatively connected to the end effector body and the teaching tool. When the teaching tool is detached from the end effector body, it receives and integrates the operation data from the first sensor group to generate a work model. When the teaching tool is installed on the end effector body through the composite interface, the control robot arm drives the teaching tool to perform the task based on the work model.

2. The system as described in claim 1, characterized in that, The first sensor group includes at least: a multi-dimensional force sensor for collecting operational force information, and a macro camera for collecting local visual information of the assembly area.

3. The system as described in claim 2, characterized in that, The teaching tool is a pen-shaped structure suitable for one-handed holding, with a vacuum nozzle at its front end; the teaching tool also integrates a vacuum barometer for detecting the adsorption state of the vacuum nozzle.

4. The system as described in claim 1 or 3, characterized in that, The end effector body integrates a wide-angle camera for collecting environmental visual information; When generating the task model, the learning and control module performs spatiotemporal alignment and fusion of local visual information from the macro camera and environmental visual information from the wide-angle camera.

5. The system as described in claim 4, characterized in that, The end effector also integrates a microphone array for receiving voice command information during teaching or execution. The learning and control module semantically correlates the voice command information with visual data from macro and wide-angle cameras, as well as force data from a multi-dimensional force sensor, and incorporates the semantic correlation results into the operation model.

6. The system as described in claim 5, characterized in that, The composite interface on the end effector body includes a mechanical locking unit, an electrical connection unit, and a fluid passage unit; The mechanical locking unit is used to achieve detachable mechanical fixation of the teaching tool on the end effector body; The electrical connection unit includes an electrical connector for establishing a bidirectional data communication and power supply connection with the teaching tool after the teaching tool is installed. The fluid passage unit includes a pneumatic connector for connecting the air passage of the end effector body to the air passage of the operating part of the teaching tool after the teaching tool is installed.

7. The system as described in claim 6, characterized in that, The mechanical locking unit, electrical connection unit, and fluid passage unit are configured to work together so that when the teaching tool performs an installation or removal action relative to the end effector body, the mechanical locking / release, electrical connection / disconnection, and air passage connection / closure are switched synchronously.

8. A teaching and execution method for a robotic arm that integrates teaching and execution, characterized in that, The method of using the robotic arm end effector system as described in any one of claims 1-7 includes: When the teaching pendant is detached from the end effector body and is performing the target task, a task model for the target task is automatically generated based on the operation data collected by the teaching pendant itself. In response to the teaching tool being installed on the end effector body, the work model is automatically invoked, and the robotic arm is controlled to drive the same teaching tool to execute the target work defined by the work model.

9. The method as described in claim 8, characterized in that, The step of automatically generating the task model of the target task based on the operation data collected by the teaching tool itself includes: The system receives and fuses force data from a multi-dimensional force sensor within the teaching tool with local visual data from a macro camera to generate a task model that includes force control strategies and visual trajectories.

10. The method as described in claim 9, characterized in that, The step of automatically generating the job model for the target job further includes: Receives environmental visual information from a wide-angle camera on the end effector body; The environmental visual information is aligned and fused with the local visual data to generate an operational model with environmental perception and adaptive capabilities.

Images