Multi-joint decoupled robotic arm control method, system, device and storage medium

Abstract

The present application relates to the technical field of mechanical arm control, and particularly relates to a multi-end decoupling mechanical arm control method, system, device and storage medium, the method comprising: setting a vision end, an algorithm end and a mechanical arm end as independently deployed running units, which interact through a network protocol and do not share internal states; the algorithm end reserves a main flow vacancy at a key node of a grabbing process, the main flow vacancy does not perform any operation in a default state; remote calls of the algorithm end to the vision end and the mechanical arm end are encapsulated by a fault-tolerant retry layer, and when communication is instantaneously interrupted, automatic retry is performed according to a preset number of times and a waiting interval, and after continuous retries all fail, an error code is returned and a degradation strategy is executed according to an error type. When hardware is replaced, only an interface implementation needs to be newly added and switching is performed through configuration, and failure / upgrade of any end does not affect running of other ends, calibration parameters are centrally managed by the algorithm end and a same matrix is uniformly issued to the vision end and the mechanical arm end, online update of the calibration parameters is supported, and the system does not need to be restarted.

Description

Technical Field

[0001] This invention relates to the field of robotic arm control technology, and in particular to a multi-terminal decoupled robotic arm control method, system, device and storage medium. Background Technology

[0002] A robotic arm vision grasping system typically consists of a camera acquisition module, a vision algorithm inference module, a task scheduling module, and a robotic arm control module. The communication and organization between these modules directly affect the system's reliability, maintainability, and hardware adaptability. As industrial scenarios increasingly demand greater flexibility and stability from grasping systems, the degree of coupling between modules has become a key factor restricting engineering implementation.

[0003] Existing robotic arm grasping systems mainly fall into three engineering organization forms: (i) Single-process tightly coupled, where all modules communicate directly within the same process via function calls and share the same memory space; this form has a low development threshold and is suitable for prototype verification stages with simple functions and fixed hardware, but as functions increase, the dependencies between modules become more complex, and any modification may cause global side effects that are difficult to detect, and the SDK interfaces of different brands of cameras and robotic arms are directly scattered in the business code, resulting in a wide range of modifications when replacing hardware. (ii) Node-based architecture based on robot middleware such as Robot Operating System (ROS) / ROS2, where each module interacts as an independent node through topic publishing / subscription or service calls, is widely used in academic research and laboratory prototypes, but it has a strong binding to the middleware version, and upgrading the middleware version in the production environment can easily cause compatibility issues, and the configuration and debugging of the DDS communication stack is highly complex. (iii) A dedicated control cabinet (Programmable Logic Controller (PLC)) plus a host computer solution is commonly found in traditional industrial automation. The PLC is responsible for the underlying motion control and safety interlocks, with a response time in the millisecond range. The host computer runs vision algorithms and task logic. The two communicate through industrial buses such as Modbus, Profinet or EtherCAT. It has strong real-time performance but high system integration costs. The flexible iteration of vision algorithms is limited by the industrial PC development environment.

[0004] The information disclosed in this background section is intended only to enhance the understanding of the general background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0005] This invention provides a multi-terminal decoupled robotic arm control method, system, device, and storage medium, thereby effectively solving the problems in the background art.

[0006] To achieve the above objectives, the technical solution adopted by this invention is: a multi-end decoupled robotic arm control method, comprising the following steps:

[0007] The vision end, algorithm end, and robotic arm end are set as independently deployed operating units that interact through network protocols and do not share internal states;

[0008] The vision terminal encapsulates camera hardware access and image acquisition functions. It defines standard calling methods for camera initialization, color image reading, depth image reading, and intrinsic parameter acquisition through a hardware abstraction interface. It also allows switching camera brands by specifying the currently loaded camera implementation class through a configuration file.

[0009] The algorithm encapsulates multimodal segmentation, point cloud processing, grasping pose solving and calibration calculation functions, centrally manages the homogeneous transformation matrix calibration parameters from the camera to the robotic arm base and loads them uniformly at startup, and sends the same matrix to the vision end and the robotic arm end through a centralized access interface.

[0010] The robotic arm end encapsulates the robotic arm hardware control functions, defines standard calling methods for connection, disconnection, Cartesian motion, joint motion, pose query, gripper opening and closing, and emergency stop through a hardware abstraction interface, and specifies the currently loaded robotic arm implementation class through a configuration file to switch robotic arm brands;

[0011] The algorithm reserves a main process slot at key nodes in the capture process. The main process slot does not perform any operation by default. The remote calls between the vision end and the robotic arm end by the algorithm are encapsulated by a fault-tolerant retry layer. When communication is interrupted, it automatically retryes according to a preset number of times and waiting interval. After consecutive retries fail, it returns an error code and executes a degradation strategy according to the error type.

[0012] Furthermore, the algorithm defines a unified grasping strategy interface, which specifies the processing flow of generating approach pose, grasp pose, lift pose and placement pose from grasping center point and grasping angle. Different grasping methods are implementations of the grasping strategy interface, which are instantiated by the strategy selection module according to the configuration file.

[0013] Furthermore, the key nodes reserved in the main process empty space include the image acquisition completion node, the mask generation completion node, the capture pose solution completion node, the capture action completion node, and the single loop start and end nodes.

[0014] Furthermore, the fault-tolerant retry layer adopts a fixed-interval backoff strategy, with a preset retry count of 3 times and a single call timeout of 5 seconds; when the vision end fails to retry continuously, it switches to manual waiting mode; when the robotic arm end fails to retry continuously, it triggers a safety stop and alarm.

[0015] Furthermore, the vision end exposes services externally using the HTTP / REST protocol, providing image acquisition interfaces and status query interfaces. The image acquisition interface returns Base64 encoded color images and depth maps, as well as JSON format parameters. The vision end and the algorithm end agree on the interface path, request method, and return field format through interface documentation.

[0016] Furthermore, after loading the configuration file and calibration parameters upon startup, the algorithm initiates a connection detection to the vision end. After a successful connection, it sends a handshake request to the robotic arm and obtains the current pose. Once all three ends are ready, it enters a state waiting for the grasping instruction. The vision end starts first and exposes the network service interface, the algorithm end starts next, and the robotic arm starts last.

[0017] Furthermore, the calibration parameters are homogeneous transformation matrices from the camera to the robotic arm base; after recalibration, the calibration parameters are updated online by clearing the cache so that the latest file is read on the next access, without needing to restart the algorithm.

[0018] The present invention also includes a multi-terminal decoupled robotic arm control system, using the method described above, wherein the system comprises:

[0019] On the visual end, as an independently deployed operating unit, it is used to encapsulate camera hardware access and image acquisition functions, define the standard calling method of camera operation through the hardware abstraction interface and expose services to the outside world through network protocols.

[0020] On the algorithm side, as an independently deployed running unit, it is used to encapsulate multimodal segmentation, point cloud processing, capture pose solving and calibration calculation functions, centrally manage calibration parameters and load them uniformly at startup, reserve space in the main process at key nodes of the capture process, and encapsulate a fault-tolerant retry layer for remote calls.

[0021] The robotic arm end, as an independently deployed operating unit, is used to encapsulate the hardware control functions of the robotic arm. It defines the standard calling method for robotic arm operations through a hardware abstraction interface and receives control commands through a network protocol.

[0022] The vision terminal, the algorithm terminal, and the robotic arm terminal interact via a network protocol and do not share their internal states. Independent upgrades or hardware replacements of any terminal do not affect the operation of the other terminals.

[0023] The present invention also includes a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as described above.

[0024] The present invention also includes a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described above.

[0025] The beneficial effects of this invention are as follows: the camera and robotic arm are connected through hardware abstraction interfaces, and replacing hardware only requires adding an interface and switching via configuration. Failure / upgrade of one end does not affect the operation of the other end; changing the brand of camera or robotic arm only requires adding an adapter; calibration parameters are centrally managed by the algorithm end and uniformly distributed to the vision end and robotic arm end with the same matrix, supporting online updates of calibration parameters without restarting the system; the grasping strategy is switched according to configuration through a unified interface and strategy selection module, and key nodes of the grasping process on the algorithm end reserve space in the main process. New functions are accessed by writing processing logic into the space in the main process without modifying the core control process; remote calls between the vision end and the robotic arm end are encapsulated by a fault-tolerant retry layer, and automatic retry is performed when communication is interrupted. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart of the method of the present invention;

[0028] Figure 2 This is a schematic diagram of the system structure of the present invention;

[0029] Figure 3 This is a schematic diagram of the structure of the computer device of the present invention. Detailed Implementation

[0030] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0031] Example 1:

[0032] like Figure 1 As shown: A multi-terminal decoupled robotic arm control method, comprising the following steps:

[0033] The vision end, algorithm end, and robotic arm end are set as independently deployed operating units that interact through network protocols and do not share internal states;

[0034] The vision side encapsulates camera hardware access and image acquisition functions. It defines standard calling methods for camera initialization, color image reading, depth image reading, and intrinsic parameter acquisition through a hardware abstraction interface. It also allows switching camera brands by specifying the currently loaded camera implementation class through a configuration file.

[0035] The algorithm encapsulates multimodal segmentation, point cloud processing, grasping pose solving and calibration calculation functions, centrally manages the homogeneous transformation matrix calibration parameters from the camera to the robotic arm base and loads them uniformly at startup, and sends the same matrix to the vision end and the robotic arm end through a centralized access interface;

[0036] The robotic arm end encapsulates the hardware control functions of the robotic arm. It defines the standard calling methods for connection, disconnection, Cartesian motion, joint motion, pose query, gripper opening and closing, and emergency stop through the hardware abstraction interface. It can also switch robotic arm brands by specifying the currently loaded robotic arm implementation class through the configuration file.

[0037] The algorithm reserves a main process slot at key nodes in the capture process. The main process slot does not perform any operation by default. The remote calls between the algorithm and the vision and robotic arm are encapsulated by the fault tolerance and retry layer. When communication is interrupted, it will automatically retry according to the preset number of times and waiting interval. After consecutive retries fail, an error code is returned and a degradation strategy is executed according to the error type.

[0038] The camera and robotic arm are connected through hardware abstraction interfaces. Replacing hardware only requires adding an interface and switching via configuration. Failure / upgrade of one end does not affect the operation of the other end. Changing the brand of camera or robotic arm only requires adding an adapter. Calibration parameters are centrally managed by the algorithm end and uniformly distributed to the vision end and robotic arm end with the same matrix. Online updates of calibration parameters are supported without restarting the system. Grasping strategies are switched by configuration through a unified interface and strategy selection module. Key nodes in the algorithm end's grasping process reserve space in the main process. New functions are added by writing processing logic into the space in the main process without modifying the core control flow. Remote calls between the vision end and the robotic arm end are encapsulated by a fault-tolerant retry layer, and automatic retry is performed when communication is interrupted.

[0039] In this embodiment, a unified grasping strategy interface is defined on the algorithm side. The grasping strategy interface specifies the processing flow of generating approach pose, grasp pose, lift pose and placement pose from grasping center point and grasping angle. Different grasping methods are implemented as grasping strategy interface, and are instantiated by strategy selection module according to configuration file.

[0040] Among them, the key nodes reserved in the main process include the image acquisition completion node, the mask generation completion node, the capture pose solution completion node, the capture action completion node, and the start and end nodes of a single loop.

[0041] As a preferred embodiment of the above, the fault-tolerant retry layer adopts a fixed-interval backoff strategy, with a preset retry count of 3 times and a single call timeout of 5 seconds; when the vision end fails to retry continuously, it switches to manual waiting mode; when the robotic arm fails to retry continuously, it triggers a safety stop and alarm.

[0042] The vision side exposes its services to the outside world using the HTTP / REST protocol, providing image acquisition interfaces and status query interfaces. The image acquisition interface returns Base64 encoded color images and depth maps, as well as JSON format parameters. The vision side and the algorithm side agree on the interface path, request method, and return field format through interface documentation.

[0043] In this embodiment, the algorithm loads the configuration file and calibration parameters upon startup and initiates a connection detection to the vision end. After a successful connection, it sends a handshake request to the robotic arm and obtains the current pose. Once all three ends are ready, they enter a state of waiting for the grasping instruction. The vision end starts first and exposes the network service interface, followed by the algorithm end, and the robotic arm end starts last.

[0044] As a preferred embodiment of the above, the calibration parameters are the homogeneous transformation matrix from the camera to the robotic arm base; after recalibration, the calibration parameters are updated online by clearing the cache so that the latest file is read on the next access, without the need to restart the algorithm.

[0045] Example 2:

[0046] Figure 2 The diagram shows the overall architecture of the three-terminal decoupled control system of this invention. In the diagram, the topmost large box represents the algorithm side, containing core algorithm modules such as multimodal segmentation, point cloud processing, grasping pose solving, and calibration calculation. The task scheduling module coordinates the execution order of each algorithm step and sends call requests to both the vision end and the robotic arm end. The centralized calibration parameter management module provides the hand-eye calibration matrix to the entire system via a single-point access method, supporting online updates. The grasping strategy selection module switches between different strategies such as vertical grasping and parallel grasping according to configuration. Spare spaces in the main process are reserved at key nodes for extended logic such as data acquisition to be integrated without modifying the main process code. Below the algorithm side, the left box represents the vision end, which exposes image acquisition services through an HTTP network protocol interface. The camera hardware abstraction interface encapsulates the differences in camera software development kits (SDKs) for various brands, supporting the access and switching of multiple camera brands. The right box represents the robotic arm end. The robotic arm hardware abstraction interface encapsulates the differences in robotic arm SDKs for various brands. Motion control and gripper control are called through a unified interface, supporting brands such as Lebai, UR, and Franka. The fault-tolerant retry layer at the bottom of the diagram encapsulates remote calls between the vision end and the robotic arm end. When communication is interrupted, it automatically retryes according to a preset number of times and intervals. After consecutive failures, it returns an error code to the algorithm end.

[0047] (1) Three-end division

[0048] Visual side: camera hardware access, image acquisition and intrinsic parameter acquisition, and external interfaces are exposed through network protocols;

[0049] Algorithm side: Multimodal segmentation, point cloud processing, grasping pose solving, calibration calculation, and sequentially calling the vision side and robotic arm side according to the grasping process, centrally managing calibration parameters and grasping strategy selection;

[0050] Robotic arm end: hardware control of the robotic arm, encapsulating specific vendor SDKs.

[0051] Each terminal communicates through clearly defined interface boundaries and does not share internal state.

[0052] System startup process: The vision end starts first and exposes the network service interface; after the algorithm end starts, it loads the configuration file and calibration matrix and initiates a connection detection to the vision end; then the algorithm end connects to the robotic arm end and sends a handshake request to obtain the current pose of the robotic arm; after all three ends are ready, the system enters the waiting state for grasping instructions.

[0053] (2) Hardware Abstraction Layer

[0054] Camera Hardware Abstraction Interface: Defines the standard calling methods for camera initialization, color image reading, depth image reading, and intrinsic parameter acquisition; the differences in the Software Development Kit (SDK) of various camera brands are encapsulated in the corresponding interface implementations.

[0055] The robotic arm hardware abstraction interface defines the standard calling methods for connection, disconnection, Cartesian motion, joint motion, pose query, and gripper opening and closing; the SDK differences of various brands of robotic arms are encapsulated in the corresponding interface implementations.

[0056] Replacing hardware only requires adding the corresponding interface implementation and switching via configuration; the camera hardware abstract interface includes at least initialization, color image reading, depth map reading, intrinsic parameter acquisition, and resource release; the robotic arm hardware abstract interface includes at least connection, disconnection, Cartesian motion, joint motion, pose query, gripper opening and closing, and emergency stop; the input parameter types and return value types of each interface are uniformly defined in the abstract layer, and the actual operation is completed by calling the corresponding vendor's software development kit in the specific implementation; the system specifies the currently loaded camera and robotic arm implementation class names through the configuration file, and when replacing hardware, only the configuration needs to be modified and the corresponding end restarted.

[0057] (3) Centralized management of calibration parameters

[0058] The homogeneous transformation matrix from the camera to the robotic arm base is configured as a system-level parameter, loaded uniformly by the algorithm at startup, and distributed to both the vision and robotic arm ends via a centralized access interface. After recalibration, the cache is cleared so that the latest file is read on the next access, achieving online updates of calibration parameters without requiring a system restart.

[0059] (4) Fetching strategy switching mechanism

[0060] Define a unified grasping strategy interface, specifying the processing flow of generating approach / grab / lift / place poses based on the grasping center point and grasping angle; specific processing methods such as vertical grasping and parallel grasping are implemented as interfaces, and are instantiated by the strategy selection module according to the configuration file.

[0061] (5) Main process empty space

[0062] The algorithm reserves slots in the main workflow at key nodes during the crawling process. These key nodes include image acquisition completion, mask generation completion, crawling pose calculation completion, crawling action completion, and single loop start / end. By default, these main workflow slots do not perform any operations. When additional functions such as data acquisition are needed, the corresponding processing logic can be written into the main workflow slot of the corresponding node for integration; the main workflow code does not require modification.

[0063] (6) Fault-tolerant retry mechanism

[0064] Remote calls from both the vision end and the robotic arm end are uniformly encapsulated by a fault-tolerant retry layer. In the event of a momentary communication interruption, the calls are automatically retried according to a preset number of times and a waiting interval. The fault-tolerant retry layer adopts a fixed-interval backoff strategy, with a preset number of retries of 3 and a timeout of 5 seconds for each call. If all retries fail, an error code is returned to the algorithm end, which determines the degradation strategy based on the error type: if the vision end fails, it switches to manual waiting mode; if the robotic arm end fails, it triggers a safety stop and alarm.

[0065] (7) Single-time capture workflow:

[0066] 1) The algorithm sends an image acquisition request to the vision unit;

[0067] 2) The visual terminal returns the current color image, depth image, and camera intrinsics;

[0068] 3) The algorithm calls the model to calculate motion commands.

[0069] 4) The algorithm sends approach, grasp, and lift motion commands sequentially through the robotic arm abstract interface;

[0070] 5) The robotic arm executes each step sequentially and returns a confirmation of arrival, triggering the "grabbing action completed" main process empty slot;

[0071] 6) The algorithm determines whether to continue the next loop or end the task.

[0072] Vision-side interface specifications: The vision-side exposes services via HTTP / REST protocol, providing an image acquisition interface (returning Base64 encoded PNG color images, depth maps, and JSON format parameters) and a status query interface (returning camera connection status and frame rate information). Interface paths, request methods, and return field formats are agreed upon in the interface documentation between the vision-side and algorithm-side. Differences in acquisition between different camera brands are encapsulated within the vision-side; the algorithm-side does not need to be aware of the camera brand. The robotic arm interface specifications are similar, encapsulating SDKs from various brands and providing the algorithm-side with unified interfaces for Cartesian motion, joint motion, pose query, gripper opening / closing, and emergency stop. Interface parameters (coordinate system, units) are uniformly defined in the abstraction layer.

[0073] Standard deployment for a single workstation: The vision end is deployed as an independent HyperText Transfer Protocol (HTTP) service on a host directly connected to the camera; the algorithm end and the robotic arm end are deployed in different processes on the same industrial control computer and interact through inter-process communication. The algorithm end completes inference calculation and task scheduling internally.

[0074] Deployment with data acquisition: Data processing logic is written into nodes such as image acquisition completion and grasping action completion in the algorithm process. Each grasping loop automatically saves color images, depth maps, robotic arm poses and results to the local dataset directory. The main process code does not need to be modified.

[0075] Example 3:

[0076] Please see Figure 3 The diagram shows a structural schematic of a computer device provided in an embodiment of this application. An embodiment of this application provides a computer device 400, including a processor 410 and a memory 420. The memory 420 stores a computer program executable by the processor 410. When the computer program is executed by the processor 410, it performs the method described above.

[0077] This application embodiment also provides a storage medium 430, on which a computer program is stored, and the computer program is executed by a processor 410 to perform the above method.

[0078] The storage medium 430 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0079] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. "A plurality of" means two or more, unless otherwise explicitly specified.

[0080] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0081] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0082] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0083] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0084] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0085] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0086] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A multi-terminal decoupled robotic arm control method, characterized in that, include: The vision end, algorithm end, and robotic arm end are set as independently deployed operating units that interact through network protocols and do not share internal states; The vision terminal encapsulates camera hardware access and image acquisition functions. It defines standard calling methods for camera initialization, color image reading, depth image reading, and intrinsic parameter acquisition through a hardware abstraction interface. It also allows switching camera brands by specifying the currently loaded camera implementation class through a configuration file. The algorithm encapsulates multimodal segmentation, point cloud processing, grasping pose solving and calibration calculation functions, centrally manages the homogeneous transformation matrix calibration parameters from the camera to the robotic arm base and loads them uniformly at startup, and sends the same matrix to the vision end and the robotic arm end through a centralized access interface. The robotic arm end encapsulates the robotic arm hardware control functions, defines standard calling methods for connection, disconnection, Cartesian motion, joint motion, pose query, gripper opening and closing, and emergency stop through a hardware abstraction interface, and specifies the currently loaded robotic arm implementation class through a configuration file to switch robotic arm brands; The algorithm reserves a main process slot at key nodes in the capture process. The main process slot does not perform any operation by default. The remote calls between the vision end and the robotic arm end by the algorithm are encapsulated by a fault-tolerant retry layer. When communication is interrupted, it automatically retryes according to a preset number of times and waiting interval. After consecutive retries fail, it returns an error code and executes a degradation strategy according to the error type.

2. The method according to claim 1, characterized in that, The algorithm defines a unified grasping strategy interface, which specifies the processing flow of generating approach pose, grasp pose, lift pose and placement pose from grasping center point and grasping angle. Different grasping methods are implemented as the grasping strategy interface and are instantiated by the strategy selection module according to the configuration file.

3. The method according to claim 1, characterized in that, The key nodes reserved in the main process include the image acquisition completion node, the mask generation completion node, the capture pose solution completion node, the capture action completion node, and the start and end nodes of a single loop.

4. The method according to claim 1, characterized in that, The fault-tolerant retry layer adopts a fixed-interval backoff strategy, with a preset retry count of 3 times and a single call timeout of 5 seconds; when the vision end fails to retry continuously, it switches to manual waiting mode; when the robotic arm end fails to retry continuously, it triggers a safety stop and alarm.

5. The method according to claim 1, characterized in that, The vision terminal exposes services externally via HTTP / REST protocol, providing image acquisition interface and status query interface. The image acquisition interface returns Base64 encoded color image and depth image, as well as JSON format parameters. The vision terminal and the algorithm terminal agree on the interface path, request method, and return field format through interface documentation.

6. The method according to claim 1, characterized in that, Upon startup, the algorithm loads the configuration file and calibration parameters and initiates a connection detection to the vision end. After a successful connection, it sends a handshake request to the robotic arm and obtains the current pose. Once all three ends are ready, it enters a state of waiting for a grasping command. The vision end starts first and exposes the network service interface, the algorithm end starts next, and the robotic arm starts last.

7. The method according to claim 1, characterized in that, The calibration parameters are the homogeneous transformation matrix from the camera to the robotic arm base; after recalibration, the calibration parameters are updated online by clearing the cache so that the latest file is read on the next access, without the need to restart the algorithm.

8. A multi-terminal decoupled robotic arm control system, characterized in that, Using the method of any one of claims 1 to 7, the system comprises: On the visual end, as an independently deployed operating unit, it is used to encapsulate camera hardware access and image acquisition functions, define the standard calling method of camera operation through the hardware abstraction interface and expose services to the outside world through network protocols. On the algorithm side, as an independently deployed running unit, it is used to encapsulate multimodal segmentation, point cloud processing, capture pose solving and calibration calculation functions, centrally manage calibration parameters and load them uniformly at startup, reserve space in the main process at key nodes of the capture process, and encapsulate a fault-tolerant retry layer for remote calls. The robotic arm end, as an independently deployed operating unit, is used to encapsulate the hardware control functions of the robotic arm. It defines the standard calling method for robotic arm operations through a hardware abstraction interface and receives control commands through a network protocol. The vision terminal, the algorithm terminal, and the robotic arm terminal interact via a network protocol and do not share their internal states. Independent upgrades or hardware replacements of any terminal do not affect the operation of the other terminals.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1-7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images