Mechanical arm control method, device, equipment and storage medium

Abstract

The application discloses a mechanical arm control method and device, equipment and a storage medium, and relates to the technical field of industrial mechanical arms. The mechanical arm comprises at least two skeletal arms connected in cascade, and adjacent skeletal arms in the at least two skeletal arms are connected through joints. The method comprises the following steps: collecting the coordinates of a workpiece to be processed in a two-dimensional coordinate system through a vision sensor; determining the expected coordinate posture of the end of the mechanical arm in a three-dimensional coordinate system based on the coordinates of the workpiece to be processed in the two-dimensional coordinate system; determining the planned motion trajectory of the end of the mechanical arm from an initial coordinate posture to the expected coordinate posture based on the expected coordinate posture; and controlling the end of the mechanical arm to move from the initial coordinate posture to the expected coordinate posture based on the planned motion trajectory, so as to improve the control precision of the mechanical arm.

Description

Technical Field

[0001] This application relates to the field of industrial robotic arm technology, and in particular to a robotic arm control method, device, equipment and storage medium. Background Technology

[0002] Robotics technology is being used more and more widely in the manufacturing industry, especially in the fields of automobiles, electronic information, rail trains and aerospace manufacturing.

[0003] During the trajectory control of robotic arms, it is difficult to achieve high-precision trajectory reproduction due to human operation and robot errors, especially in complex tasks, which severely limits work efficiency and quality. Summary of the Invention

[0004] This application provides a robotic arm control method, device, equipment, and storage medium. The technical solution provided by this application includes the following aspects.

[0005] According to one aspect of the embodiments of this application, a robotic arm control method is provided, the robotic arm including at least two cascaded skeletal arms, adjacent skeletal arms being connected by joints, the method comprising:

[0006] The coordinates of the workpiece to be processed in a two-dimensional coordinate system are acquired using a vision sensor;

[0007] Based on the coordinates of the workpiece to be processed in the two-dimensional coordinate system, the desired coordinate posture of the end effector of the robotic arm in the three-dimensional coordinate system is determined.

[0008] Based on the desired coordinate posture, the planned motion trajectory of the end effector of the robotic arm from the initial coordinate posture to the desired coordinate posture is determined;

[0009] Based on the planned motion trajectory, the end effector of the robotic arm is controlled to move from the initial coordinate posture to the desired coordinate posture.

[0010] According to one aspect of the embodiments of this application, a robotic arm control device is provided. The robotic arm includes at least two skeletal arms linked in a series, wherein adjacent skeletal arms are connected by joints. The device includes:

[0011] The acquisition module is used to acquire the coordinates of the workpiece to be processed in a two-dimensional coordinate system through a vision sensor;

[0012] The first determining module is used to determine the desired coordinate posture of the end effector of the robotic arm in three-dimensional coordinates based on the coordinates of the workpiece to be processed in a two-dimensional coordinate system.

[0013] The second determining module is used to determine the planned motion trajectory of the end effector of the robotic arm from the initial coordinate posture to the desired coordinate posture based on the desired coordinate posture.

[0014] The control module is used to control the end effector of the robotic arm to move from the initial coordinate posture to the desired coordinate posture based on the planned motion trajectory.

[0015] In one possible implementation, the control module is used to determine the desired coordinate posture of the end effector of the robotic arm at the t-th moment during the motion process, based on the planned motion trajectory, where t is a positive integer;

[0016] Based on the desired coordinate posture at the t-th time, the joint sub-posture of each joint of the robotic arm at the t-th time is determined, and the joint sub-posture at the t-th time is characterized by the relative change of the joint sub-posture at the (t-1)-th time.

[0017] Based on the joint sub-pose of each joint at the t-th time, each joint is controlled so that the end effector of the robotic arm moves from the initial coordinate pose to the desired coordinate pose.

[0018] In one possible implementation, the control module is configured to determine the joint sub-coordinates of each joint at the t-th time based on the joint sub-pose of each joint at the t-th time.

[0019] Based on the joint sub-pose and the joint sub-coordinate, each joint is controlled so that the end effector of the robotic arm moves from the initial coordinate pose to the desired coordinate pose.

[0020] In one possible implementation, the control module is used to determine the correspondence between the joint sub-postures of each joint of the robotic arm and the trajectory motion time based on the desired coordinate posture corresponding to the t-th time, the initial postures corresponding to each joint, and the trajectory motion time.

[0021] Based on the correspondence, the joint sub-pose of each joint of the robotic arm at the t-th time is determined.

[0022] In one possible implementation, the acquisition module is further configured to acquire the error at the (t-1)th moment through the vision sensor, the error being used to characterize the error between the actual coordinate posture of the end effector of the robotic arm and the expected coordinate posture at the (t-1)th moment; the control module is configured to correct the planned motion trajectory based on the error to obtain a corrected motion trajectory;

[0023] Based on the corrected motion trajectory, the desired coordinate posture of the end effector of the robotic arm at the t-th moment during the motion process is determined.

[0024] In one possible implementation, the first determining module is used to determine the desired coordinate posture of the end effector of the robotic arm in three-dimensional coordinates based on the coordinates of the workpiece to be processed in a two-dimensional coordinate system and the transformation relationship.

[0025] The transformation relationship is the transformation from the vision system coordinate system to the end effector coordinate system of the robotic arm.

[0026] According to one aspect of the embodiments of this application, a terminal device is provided, the terminal device including a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-described robotic arm control method.

[0027] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, the computer program being loaded and executed by a processor to implement the above-described robotic arm control method.

[0028] According to one aspect of the embodiments of this application, a computer program product is provided, the computer program product including a computer program stored in a computer-readable storage medium, and a processor reading from the computer-readable storage medium and executing the computer program to implement the above-described robotic arm control method.

[0029] The technical solution provided in this application can bring the following beneficial effects:

[0030] This application determines the planned motion trajectory based on the desired coordinate posture of the end effector of the robotic arm, and controls the end effector of the robotic arm to move from the initial coordinate posture to the desired coordinate posture along the planned motion trajectory. It realizes the automatic determination of the planned motion trajectory based on the posture data of the end effector of the robotic arm, which solves the problems of low accuracy due to manual pre-operation in the teaching method and the heavy reliance on the accuracy and precision of the 3D model of the robot and the workpiece in the computer-aided design method. It improves the precision of robotic arm control, thereby improving the efficiency of workpiece processing by the robotic arm. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of a computer system provided in one embodiment of this application;

[0032] Figure 2 This is a flowchart of a robotic arm control method provided in one embodiment of this application;

[0033] Figure 3This is a flowchart of the operation of a fuzzy PID controller provided in one embodiment of this application;

[0034] Figure 4 This is a flowchart of a robotic arm control system provided in one embodiment of this application;

[0035] Figure 5 This is a schematic diagram of the structure of a robotic arm control system provided in one embodiment of this application;

[0036] Figure 6 This is a schematic diagram of a six-degree-of-freedom robotic arm provided in one embodiment of this application;

[0037] Figure 7 This is a block diagram of a robotic arm control device provided in one embodiment of this application;

[0038] Figure 8 This is a structural block diagram of a terminal device provided in one embodiment of this application. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0040] It should be noted that the terms "first," "second," etc., used in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0041] First, the terms used in this application will be explained.

[0042] Robotic arm: An automated device that mimics the functions of a human arm, capable of performing various tasks such as grasping, moving, and assembling. A robotic arm typically consists of multiple links (skeletal arms), joints, end effectors (such as grippers, spray guns, etc.), and a control system.

[0043] The skeletal arms of a robotic arm, often simply referred to as "links" or "arms," ​​are the main components of the robotic arm's structure. They are the "skeleton" of the robotic arm, connecting the various joints and forming the main structure of the robotic arm. The length, shape, and layout of the skeletal arms directly affect the robotic arm's working range and flexibility.

[0044] Joints: Moving parts that connect the various skeletal arms, allowing relative movement between the skeletal arms.

[0045] Forward kinematics: Given the angles of each joint of a robotic arm, calculate the coordinate orientation of the end effector. This involves combining the rotational and translational transformations of each joint to form a total transformation matrix from the base of the robotic arm to its end effector.

[0046] Inverse kinematics: Given the desired coordinate orientation of the end effector of a robotic arm, calculate the joint angles required to achieve that desired orientation.

[0047] Hand-eye calibration: A vision sensor, such as a camera, is installed at the end of the robotic arm, and a calibration target is set in the working area, for example, using the ArucoMarker method in the opencv_contrtb module. Through the calibration process, the relative position and orientation between the vision sensor (eye) and the end of the robotic arm (hand) are determined, thereby achieving hand-eye calibration.

[0048] To further illustrate the technical solutions provided in the embodiments of this application, a detailed description is provided below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of this application provide method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on conventional or non-inventive effort. For steps that do not logically have a necessary causal relationship, the execution order of these steps is not limited to the execution order provided in the embodiments of this application. In actual processing or when the control device executes the method, it may be executed sequentially or in parallel according to the method shown in the embodiments or drawings.

[0049] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0050] Figure 1 A schematic diagram of a computer system provided in an embodiment of this application is shown. The computer system includes a computer device 101 and a server 102.

[0051] In one possible implementation, computer device 101 is any electronic product capable of human-computer interaction with an interactive object through one or more means such as a keyboard, touchpad, touchscreen, remote control, voice interaction, or handwriting device. Examples include PCs (Personal Computers), mobile phones, smartphones, PDAs (Personal Digital Assistants), wearable devices, handheld portable gaming devices, PPCs (Pocket PCs), tablets, laptops, desktop computers, smart car systems, smart TVs, smart speakers, smartwatches, and in-vehicle terminals, but it is not limited to these.

[0052] Server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. This application embodiment does not limit this. Server 102 communicates directly or indirectly with computer device 101 via wired or wireless communication methods, which is not limited here. Server 102 has data receiving, data processing, and data sending functions. Of course, server 102 may also have other functions, which are not limited in this application embodiment.

[0053] Server 102 provides background services to the clients installed on computer device 101. In one possible implementation, server 102 undertakes the primary computing task, and computer device 101 undertakes the secondary computing task. Alternatively, server 102 undertakes the secondary computing task, and computer device 101 undertakes the primary computing task. Or, computer device 101 and server 102 collaborate on computing using a distributed computing architecture.

[0054] Computer device 101 can refer to one of a plurality of computer devices. This embodiment uses computer device 101 as an example only. Those skilled in the art will know that the number of computer devices 101 can be more or less. For example, there may be only one computer device 101, or there may be dozens or hundreds of computer devices 101, or more. This application embodiment does not limit the number or type of computer devices 101.

[0055] The robotic arm control method provided in this application embodiment can be executed by computer device 101, server 102, or interactively by computer device 101 and server 102. This application embodiment does not limit the specific execution method. In some embodiments, computer device 101 can send uplink synchronization data to server 102. The uplink synchronization data includes the coordinates of the workpiece to be processed in a two-dimensional coordinate system acquired by a vision sensor. Server 102 uses the robotic arm control method provided in this application embodiment to process the coordinates of the workpiece to be processed in a two-dimensional coordinate system acquired by the vision sensor sent by computer device 101, obtains the final planned motion trajectory, and sends downlink synchronization data to computer device 101. The downlink synchronization data includes the planned motion trajectory.

[0056] For example, see Figure 1Step 103: Acquire the coordinates of the workpiece to be processed in a two-dimensional coordinate system using a vision sensor; Step 104: Determine the desired coordinate posture of the end effector of the robotic arm in a three-dimensional coordinate system based on the coordinates of the workpiece to be processed in a two-dimensional coordinate system and the transformation relationship; Step 105: Determine the planned motion trajectory of the end effector of the robotic arm from the initial coordinate posture to the desired coordinate posture based on the desired coordinate posture; Step 106: Determine the desired coordinate posture of the end effector of the robotic arm at time t during the motion based on the planned motion trajectory; Step 107: Determine the joint sub-posture of each joint of the robotic arm at time t based on the desired coordinate posture at time t; Step 108: Determine the joint sub-coordinates of each joint at time t based on the joint sub-postures of each joint at time t; Step 109: Control each joint based on the joint sub-postures and joint sub-coordinates to move the end effector of the robotic arm from the initial coordinate posture to the desired coordinate posture.

[0057] Those skilled in the art should understand that the computer device 101 and server 102 described above are merely illustrative examples. Other existing or future computer devices or servers that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0058] Based on the above Figure 1 The computer system shown in this application provides a method for sharding load balancing between nodes. This method can be executed by computer device 101, server 102, or by interaction between computer device 101 and server 102. This application does not limit the specific implementation of this method.

[0059] In related technologies, robotics is increasingly being applied across various industries, such as automotive, electronics, rail transit, and aerospace manufacturing. Compared to traditional manual design and production processes, robotics offers advantages such as higher efficiency, greater precision, safety, and increased intelligence. With the development of robotics, industrial robots are increasingly being used in manufacturing processes such as grinding and welding. Robotic arms, as multi-degree-of-freedom robotic devices based on electronics, mechanics, and control technologies, possess advantages such as high flexibility and low labor costs, and are also conducive to achieving green and intelligent manufacturing. In recent years, machine vision technology has received widespread attention in trajectory planning. Based on visual guidance, it acquires contour images through visual sensors and obtains robot target trajectory data through image processing, reducing the dependence on the 3D models and relative positions of the robot and workpiece.

[0060] For typical robot integration applications, the main methods for controlling the trajectory of robotic arms are teach programming and offline programming. Teach programming typically involves setting the robot's trajectory points on-site within the corresponding robot teach pendant. Due to human error and inherent robot errors, high-precision trajectory reproduction is difficult to achieve, especially in complex tasks, severely limiting efficiency and quality. Offline programming, on the other hand, does not require online programming of the robot's trajectory, making it significantly advantageous in complex environments. Currently, offline programming methods can be divided into computer-aided design (CAD) based and vision-based methods. CAD-based offline programming heavily relies on the accuracy and precision of the 3D models of the robot and workpiece. In practice, deviations or interference from the original workpiece position will affect operational accuracy.

[0061] This application embodiment illustrates the method by example, where the method is executed by a computer device, which can be either a computer or a server. Figure 2 As shown, the robotic arm control method provided in this application embodiment may include at least one of the following steps 201 to 204, wherein the robotic arm includes at least two cascaded skeletal arms, and adjacent skeletal arms are connected by joints.

[0062] In step 201, the coordinates of the workpiece to be processed in a two-dimensional coordinate system are acquired by a vision sensor.

[0063] A vision sensor is a device used to acquire image information of a workpiece to be processed and convert it into electrical or digital signals. It can quickly and accurately capture image data of the workpiece to be processed or the working scene of a robotic arm, providing a basis for subsequent analysis, measurement and control.

[0064] Vision sensors can be installed at different parts of the robotic arm. They can be directly mounted on the end effector, allowing the sensor to move along with it. This is suitable for applications requiring close-up, dynamic observation of workpiece details, such as assembly operations where the robotic arm's end effector, carrying the vision sensor, approaches the workpiece for precise assembly. Alternatively, the vision sensor can be fixed at a specific location within the workspace, providing a relatively fixed perspective to observe the entire work area. This is suitable for locating and inspecting workpieces over a large area. This application does not restrict the installation location of the vision sensor on the robotic arm.

[0065] The workpiece to be processed is the workpiece that requires further processing, assembly, inspection or other operations during the manufacturing or processing process.

[0066] For example, in the automobile manufacturing process, the body panel is a workpiece to be processed, and the formed panel is welded to the body frame by a robotic arm; or, in a food production line, a robotic arm sorts different types of food into the corresponding packaging area.

[0067] Optionally, the coordinates of the workpiece to be processed in a two-dimensional coordinate system are acquired by a vision sensor, including: acquiring image information of the workpiece to be processed by a vision sensor; determining the workpiece features based on the image information and an edge detection algorithm; and determining the coordinates of the workpiece to be processed in a two-dimensional coordinate system based on the workpiece features.

[0068] Optionally, the workpiece to be processed includes at least two workpiece regions, and the coordinates of the workpiece to be processed in a two-dimensional coordinate system are acquired by a vision sensor, including: acquiring the coordinates of at least two workpiece regions in a two-dimensional coordinate system by a vision sensor respectively.

[0069] By refining the extraction process, the robotic arm can accurately understand the working trajectory of each workpiece area, thereby improving the precision and reliability of the robotic arm operation. Multi-area batch processing allows the robotic arm to flexibly adapt to different tasks and workpiece shapes, improving the system's adaptability and flexibility. Batch processing can also optimize the workflow, making the robotic arm more efficient in performing tasks and reducing unnecessary movement and waiting time.

[0070] Optionally, the determination of at least two workpiece regions may include at least one of the following methods: determining at least two workpiece regions based on the geometry of the workpiece to be processed; determining at least two workpiece regions based on the requirements of the task; or determining at least two workpiece regions based on the field of view of a vision sensor.

[0071] Optionally, the method further includes storing at least two workpiece regions in a matrix arrangement.

[0072] For example, the workpiece to be processed is divided into at least two workpiece regions named: Vtew1, Vtew2, Vtew3, ..., VtewX.

[0073] The coordinates of the workpiece to be processed in a two-dimensional coordinate system are acquired, and hand-eye calibration is performed on it before the robotic arm performs the task. The coordinate data points on the surface of the workpiece are located in the vision system coordinate system of the vision sensor. The data is converted to the robotic arm coordinate system through hand-eye calibration. A calibration code is attached to the end of the robotic arm. After calibration, the coordinates of the workpiece to be processed in a two-dimensional coordinate system are acquired using the vision system and its vision sensor.

[0074] In step 202, the desired coordinate orientation of the end effector of the robotic arm in three-dimensional coordinates is determined based on the coordinates of the workpiece to be processed in the two-dimensional coordinate system.

[0075] The desired coordinate orientation is the position (coordinates) and orientation (orientation) of the end effector in three-dimensional space after processing the workpiece. It includes three position coordinates (X, Y, Z) of the end effector relative to the reference coordinate system of the robot arm and three orientation angles, where the reference coordinate system of the robot arm can be the base coordinate system of the robot arm.

[0076] The three attitude angles can be understood as roll angle, pitch angle, and yaw angle, or Euler angles.

[0077] In step 203, based on the desired coordinate posture, the planned motion trajectory of the end effector of the robotic arm is determined to move from the initial coordinate posture to the desired coordinate posture.

[0078] The initial coordinate orientation is the position (coordinates) and orientation (orientation) of the end effector of the robotic arm in three-dimensional space before processing of the workpiece begins.

[0079] The planned motion trajectory is the set of continuous points traversed by the end effector of a robotic arm as it moves along a planned path in three-dimensional space. The planned motion trajectory reflects the spatial motion path of the end effector when handling a workpiece, such as grasping, moving, placing, or processing.

[0080] In step 204, based on the planned motion trajectory, the end effector of the robotic arm is controlled to move from the initial coordinate posture to the desired coordinate posture.

[0081] The end effector of the robotic arm moves from the initial coordinate posture to the desired coordinate posture. That is, according to the planned motion trajectory, the movement of each joint of the robotic arm is controlled to ensure that the end effector of the robotic arm moves along the planned motion trajectory.

[0082] This application determines the planned motion trajectory based on the desired coordinate posture of the end effector of the robotic arm, and controls the end effector of the robotic arm to move from the initial coordinate posture to the desired coordinate posture along the planned motion trajectory. It realizes the automatic determination of the planned motion trajectory based on the posture data of the end effector of the robotic arm, which solves the problems of low accuracy due to manual pre-operation in the teaching method and the heavy reliance on the accuracy and precision of the 3D model of the robot and the workpiece in the computer-aided design method. It improves the precision of robotic arm control, thereby improving the efficiency of workpiece processing by the robotic arm.

[0083] Transformation of the two-dimensional coordinates of the workpiece to be processed into three-dimensional coordinates.

[0084] In some embodiments, determining the desired coordinate posture of the end effector of the robotic arm in three-dimensional coordinates based on the coordinates of the workpiece to be processed in a two-dimensional coordinate system includes: determining the desired coordinate posture of the end effector of the robotic arm in three-dimensional coordinates based on the coordinates of the workpiece to be processed in a two-dimensional coordinate system and a transformation relationship; wherein, the transformation relationship is a relationship of transformation from the vision system coordinate system to the end effector coordinate system of the robotic arm.

[0085] Since the vision sensor acquires a two-dimensional image, the pixel position (x, y) of the workpiece in the image is determined through the two-dimensional image. The pixel position is the coordinate in the two-dimensional coordinate system. However, the robotic arm operates in the three-dimensional real world, so it is necessary to convert the coordinates (x, y) in the two-dimensional coordinate system into the desired coordinate posture (P) in the three-dimensional coordinate system. x P y P z ).

[0086] Establishing the relationship between the vision system coordinate system and the end effector coordinate system of the robotic arm requires the integration and calibration of the two systems to enable the vision system and the end effector coordinate system of the robotic arm to work together.

[0087] Optionally, the vision sensor includes at least one of a binocular sensor and a monocular sensor.

[0088] For example, a binocular sensor, simulating the principle of human binocular vision, uses two cameras to observe the same object from different angles and calculates the parallax to obtain the desired coordinate pose of the robotic arm's end effector in three-dimensional coordinates. A monocular vision system, on the other hand, acquires images through a single camera and infers the desired coordinate pose of the workpiece in three-dimensional coordinates by combining the known dimensions of the workpiece or other sensor data.

[0089] Optionally, the transformation relationship includes a transformation matrix. Based on the coordinates of the workpiece to be processed in the two-dimensional coordinate system and the transformation relationship, the desired coordinate posture of the end effector of the robotic arm in the three-dimensional coordinate system is determined, including: based on the coordinates of the workpiece to be processed in the two-dimensional coordinate system and the transformation matrix, the desired coordinate posture of the end effector of the robotic arm in the three-dimensional coordinate system is determined.

[0090] The transformation matrix expression is used to represent the transformation relationship between two-dimensional image coordinates (i.e., the coordinates of the workpiece to be processed in the two-dimensional coordinate system acquired by the robot's vision sensor) and point coordinates in three-dimensional space (i.e., the desired coordinate orientation of the end effector of the robotic arm in the three-dimensional coordinate system). The transformation matrix combines the intrinsic and extrinsic parameters of the camera to achieve the transformation from three-dimensional world coordinates to two-dimensional image coordinates.

[0091]

[0092] in, Let x be the coordinate vector of the workpiece to be processed in the two-dimensional coordinate system, and let y represent the horizontal and vertical positions of the coordinate points on the workpiece to be processed on the image plane, respectively. It includes the camera's focal length f, which is in pixels, and converts the three-dimensional coordinates into normalized image coordinates. This matrix consists of a rotation matrix R and a translation vector T, and is used to describe the orientation of the two-dimensional coordinate system relative to the robot arm's coordinate system.

[0093] The pinhole imaging model is a fundamental principle of vision systems. It describes how light rays from an object are projected onto the imaging plane through the optical axis center of the camera. In the pinhole imaging model, every point on the object is projected onto the image plane through the camera's optical center, forming an image. Using the pinhole imaging model, along with the camera's intrinsic parameters (such as focal length and principal point coordinates) and extrinsic parameters (such as the camera's coordinate orientation relative to the robotic arm's base), a transformation relationship between the vision system and the robotic arm's end effector coordinate system is established.

[0094] Matrix R is a 3x3 matrix used to describe the rotation of the two-dimensional coordinate system relative to the robot arm coordinate system. Translation vector T is a 3x1 vector used to describe the position of the origin of the two-dimensional coordinate system in the robot arm coordinate system.

[0095] Let P be a coordinate vector in three-dimensional coordinates. x P y P z These represent the x, y, and z coordinates of the point in the robotic arm's coordinate system, respectively. c It is depth information in a two-dimensional coordinate system, that is, the distance from the point to the camera; Used to normalize three-dimensional coordinates so that they can be represented in two-dimensional image coordinates.

[0096] Optionally, based on the coordinates of the workpiece to be processed in the two-dimensional coordinate system and the transformation matrix, the desired coordinate posture of the end effector of the robotic arm in the three-dimensional coordinate system is determined, including: based on the coordinates of the workpiece to be processed in the two-dimensional coordinate system, the intrinsic and extrinsic parameters of the camera, the desired coordinate posture of the end effector of the robotic arm in the three-dimensional coordinate system is determined.

[0097] In summary, the process of converting the two-dimensional coordinates of the workpiece to three-dimensional coordinates yields the desired coordinate orientation that the end effector of the robotic arm needs to reach.

[0098] The joint sub-pose of each joint is determined based on the coordinate pose of the end effector of the robotic arm.

[0099] Since the workpiece to be processed represents at least two coordinate points in a two-dimensional coordinate system, and at least two transformed three-dimensional coordinate points are obtained through the above process, the next step is to generate a continuous and smooth motion trajectory between these discrete three-dimensional coordinate points. The motion trajectory is used to determine the joint sub-pose of each joint of the robotic arm at time t throughout the entire time period from the starting point to the ending point.

[0100] In some embodiments, determining the joint sub-pose of each joint of the robotic arm at time t based on the desired coordinate pose at time t includes: determining the correspondence between the joint sub-pose and the trajectory motion time of each joint of the robotic arm based on the desired coordinate pose at time t, the initial pose of each joint, and the trajectory motion time; and determining the joint sub-pose of each joint of the robotic arm at time t based on the correspondence. The trajectory motion time is the time required for the end effector of the robotic arm to move from the starting position to the ending position of the planned motion trajectory.

[0101] Optionally, the trajectory motion time can be determined by at least one of the following methods: determining the trajectory motion time based on the area of ​​the workpiece to be processed; determining the trajectory motion time based on the length of the trajectory; or determining the trajectory motion time based on the historical operation data of the robotic arm.

[0102] For example, using the client as a carrier, the Matrix Laboratory (MATLAB) or MATLAB function Simulink is used as software to build a motion trajectory planning control system. First, a controller is established in the motion trajectory planning control system, and the coordinate information of the controller's working trajectory is used as the coordinates of the planned motion trajectory. Then, the planned motion trajectory is programmed by the trajectory planning algorithm program.

[0103] The correspondence between the joint sub-poses of each joint of the robotic arm and the trajectory motion time can be understood as follows: based on the trajectory planning algorithm, the relationship between the sub-poses of each joint of the robotic arm and the trajectory motion time is determined. Through the correspondence, the sub-pose of each joint at any moment in the trajectory motion time can be determined.

[0104] In some embodiments, controlling the end effector of the robotic arm to move from an initial coordinate posture to a desired coordinate posture based on a planned motion trajectory includes: determining the desired coordinate posture of the end effector of the robotic arm at time t during the motion process based on the planned motion trajectory, where t is a positive integer; determining the joint sub-posture of each joint of the robotic arm at time t based on the desired coordinate posture at time t, wherein the joint sub-posture at time t is characterized by a relative change relative to the joint sub-posture at time t-1; and controlling each joint based on the joint sub-posture of each joint at time t, so that the end effector of the robotic arm moves from the initial coordinate posture to the desired coordinate posture.

[0105] The planned motion trajectory of the end effector of the robotic arm (i.e., the coordinate posture of the end effector at each moment) is planned in the previous step. Through the inverse kinematics model, the angles (i.e., joint sub-postures) that each joint of the robotic arm needs to rotate in order for the end effector to reach the corresponding coordinate posture at each moment are determined.

[0106] Optionally, a cubic polynomial can be used to describe the correspondence between the sub-pose θ(t) of each joint of the robotic arm and time t.

[0107] θ(0)=a0=θ0 Formula (1)

[0108]

[0109] θ'(0)=a1=0 Formula (3)

[0110]

[0111] Formulas (1) to (4) are the trajectory planning algorithms. To achieve smooth movement of the robotic arm, the trajectory function θ(t) of each joint of the robotic arm needs to satisfy at least four constraints, where θ(0) = θ0 represents the initial angle θ0 of each joint of the robotic arm at time t = 0; θ(t f )=θ f This indicates that at time t = t f At that time, the end effector angle of the robotic arm should reach angle θ. f θ'(0)=0, indicating that at the start of the motion, the velocities of all joints of the robotic arm are zero, i.e., starting from rest; θ'(t f ) = 0, which means that at the end of the movement, the speed of each joint of the robotic arm is zero, that is, a smooth stop.

[0112] The parameters a0, a1, a2, and a3 are the coefficients of a cubic polynomial, determined by satisfying the above constraints. This allows us to obtain the correspondence between the sub-pose θ(t) of each joint of the robotic arm and time t.

[0113] Based on the joint sub-pose of each joint, the sub-coordinates of each joint are determined.

[0114] In some embodiments, based on the joint sub-pose of each joint at time t, controlling each joint to move the end effector of the robotic arm from the initial coordinate pose to the desired coordinate pose includes: determining the joint sub-coordinates of each joint at time t based on the joint sub-pose of each joint at time t; and controlling each joint based on the joint sub-pose and the joint sub-coordinates to move the end effector of the robotic arm from the initial coordinate pose to the desired coordinate pose.

[0115] See Equation (5), which is a parametric representation (Denavit-Hartenberg, DH) of the coordinate system transformation relationship between two adjacent joints (e.g., the i-th joint and the (i-1)-th joint) in a robotic arm.

[0116] Formula (5) can be used to represent the transformation matrix between the end coordinate system and the base coordinate system of the robotic arm in the robotic arm system by connecting the various joints of the robotic arm.

[0117]

[0118] Where, θ i Indicates the linkage angle; d i Represents the link offset, and α represents the translation distance along the Z-axis of the i-th joint coordinate system; i Represents the link twist angle; R represents rotational transformation, T represents translational transformation; c is an abbreviation for cosine, s is an abbreviation for sinine, R x (α i-1 ) represents rotation α around the X-axis i-1 Rotation matrix of the angle; T x (a i-1 ) indicates a translation along the X-axis. i-1 Translation matrix of distance; R z (θ i ) represents a rotation θ around the Z-axis i Rotation matrix of the angle; T z (d i ) indicates a translation d along the Z-axis i The translation matrix of the distance.

[0119] Formula (5) above describes the transformation from one joint coordinate system to the next, including rotation and translation. By multiplying the transformation matrices of all joints, the total transformation matrix from the base coordinate system to the end-effector coordinate system of the robotic arm can be obtained, see formula (6).

[0120]

[0121] Equation (6) describes the total transformation matrix of the robotic arm from the base coordinate system to the end effector of the robotic arm. It is achieved by transforming the local transformation matrices between the joints. The product is obtained by multiplying. This represents a chain multiplication operation, which is the product of all local transformation matrices from the first joint to the nth joint; n represents the number of joints in the robotic arm; n x n y n z This represents the unit vector component of the Z-axis of the end effector coordinate system of the robotic arm in the base coordinate system; o x o y o z a represents the unit vector component of the Y-axis of the end effector coordinate system of the robotic arm in the base coordinate system; x a y a z p represents the unit vector component of the X-axis of the end effector coordinate system of the robotic arm in the base coordinate system; x p y p z This indicates the position coordinates of the origin of the end effector coordinate system of the robotic arm in the base coordinate system.

[0122] Correct the trajectory of motion.

[0123] In some embodiments, the method further includes: acquiring the error at time t-1 using a visual sensor, the error being used to characterize the error between the actual coordinate posture of the end effector of the robotic arm and the expected coordinate posture at time t-1; determining the expected coordinate posture of the end effector of the robotic arm at time t during the motion process based on the planned motion trajectory, including: correcting the planned motion trajectory based on the error to obtain a corrected motion trajectory; and determining the expected coordinate posture of the end effector of the robotic arm at time t during the motion process based on the corrected motion trajectory.

[0124] Error is an indicator that measures the difference between the actual result and the expected result, characterizing the deviation between the actual coordinate posture and the expected coordinate posture of the end effector of a robotic arm. Deviation can originate from multiple sources, including manufacturing errors of the robotic arm, measurement errors of sensors, calculation errors of the control system, and the influence of environmental factors.

[0125] In vision sensors, error is acquired by comparing the actual coordinate posture of the robotic arm's end effector at time t-1 with the pre-set desired coordinate posture. The vision sensor captures the current coordinate posture of the robotic arm's end effector and feeds this information back to the control system. The control system then calculates the error, i.e., the difference between the actual value and the target value, based on this feedback.

[0126]

[0127] Formula (7) is used to calculate the position error e(t) of the robotic arm's end effector at time t. This error is the difference between the actual position (x, y, z) and the desired position (P). x P y P z The Euclidean distance between them.

[0128] In robotic arm control systems, error e(t) is fed back for control, such as using a proportional-integral-derivative (PID) controller. The control system corrects the planned motion trajectory based on the error, resulting in a revised trajectory to reduce the error and bring the robotic arm's end effector as close as possible to the desired trajectory. Real-time monitoring and adjustment can improve the accuracy and efficiency of robotic arm operations.

[0129] Correcting the motion trajectory refers to adjusting the originally planned desired motion trajectory during the movement of a robotic arm, based on the error between the actual and expected trajectory, to obtain a new trajectory. This correction is to ensure that the end effector of the robotic arm can operate more precisely according to the predetermined goal, thereby improving the accuracy and efficiency of the operation.

[0130]

[0131] Formula (8) is the PID control formula, used to calculate the control input u(t) based on the error e(t). The control input is adjusted through three parts: proportional (P), integral (I), and derivative (D) to reduce the error. p e(t) is the proportional part, which is directly proportional to the error. The larger the error, the larger the control input. The integral part is proportional to the accumulation of error and can eliminate steady-state error; The differential part is proportional to the rate of change of the error, predicts the future trend of the error, and improves the system's response speed and stability.

[0132] Optionally, the planned motion trajectory is corrected based on the error to obtain the corrected motion trajectory, including: calculating the control input u(t) using the PID formula based on the error, and updating the motion trajectory of the robotic arm based on the control input u(t) to obtain the corrected motion trajectory.

[0133] For example, suppose a robotic arm needs to move from point A to point B along a straight line. During the movement, due to external disturbances, the actual path of the robotic arm deviates from the planned trajectory. A vision sensor detects the actual coordinate posture of the robotic arm's end effector and calculates the error between it and the desired coordinate posture. Using a PID control formula, the control input that needs adjustment at each time step is calculated. If the error is large, the proportional component increases the control input, the integral component adjusts based on the accumulated error, and the derivative component predicts and adjusts based on the rate of change of the error. By updating the robotic arm's trajectory in real time and adjusting according to the PID control input, it is ensured that the end effector of the robotic arm can accurately move along the corrected trajectory and ultimately reach the position of the workpiece to be processed.

[0134] Optionally, the number of visual sensors is at least two, including at least one horizontally placed lateral visual sensor and at least one vertically placed longitudinal visual sensor; the error at time t-1 is collected through the visual sensors, including: collecting the lateral error at time t-1 through at least one lateral visual sensor and collecting the longitudinal error at time t-1 through at least one longitudinal visual sensor; the planned motion trajectory is corrected based on the error to obtain a corrected motion trajectory, including: correcting the planned motion trajectory laterally based on the lateral error and correcting the planned motion trajectory longitudinally based on the longitudinal error, and obtaining the corrected motion trajectory based on the lateral and longitudinal corrections.

[0135] Lateral and longitudinal vision sensors can capture deviation information of the workpiece being processed by the robotic arm's end effector from two different directions. Through two independent corrections, the robotic arm can more precisely adjust the position of its end effector, ensuring that the workpiece is placed in the predetermined precise position. This dual correction mechanism reduces accumulated errors and improves positioning accuracy.

[0136] For example, if a robotic arm needs to screw a screw into a hole in a steel plate on an assembly line, a lateral vision sensor detects the horizontal deviation of the screw relative to the hole. Based on this, the end effector of the robotic arm performs a lateral correction in the X-axis direction, adjusting its position to align with the center of the hole. A longitudinal vision sensor detects the vertical deviation of the screw relative to the hole. Based on this, the end effector of the robotic arm performs a longitudinal correction in the Y-axis direction, further adjusting its position. Ultimately, the lateral correction in the X-axis direction and the longitudinal correction in the Y-axis direction form a correction motion trajectory.

[0137] For example, see Figure 3The flowchart shown illustrates the operation of a fuzzy PID controller, an advanced control strategy that combines fuzzy logic with PID control. It leverages the flexibility of fuzzy logic and the stability of PID control to improve the performance of the control system. The fuzzy PID controller includes fuzzification (301), fuzzy inference (302), and defuzzification (303).

[0138] Fuzzification 301 is the process of converting precise input values ​​into fuzzy sets. In control systems, input values ​​are typically the error e(t) and the rate of change of the error e'(t). Precise values ​​are mapped to fuzzy sets, such as "negative large," "negative small," "zero," "positive small," and "positive large." The fuzzification process involves defining input membership functions 304, which describe the degree to which the input value belongs to each fuzzy set. For example, the error e(t) and the rate of change of the error e'(t) can be represented by triangular or trapezoidal membership functions, respectively. The value of the input membership function 304 is between 0 and 1, indicating the degree to which the input value belongs to a certain fuzzy set.

[0139] Fuzzy inference 302 is the process of reasoning about the fuzzified input based on the rule base 305. The rule base 305 contains a series of if-then rules that define the relationship between the input fuzzy set and the output fuzzy set. For example, a fuzzy rule is "If the error is negatively large and the rate of change of the error is negatively small, then the control output has a large proportional gain".

[0140] Defuzzification 303 is the process of converting the fuzzy output set obtained from fuzzy inference 302 into precise control output values. Since the fuzzy output set is a fuzzy value, it needs to be converted into a specific numerical value based on the output membership function 306 for use in actual control.

[0141] Through the above steps, a fuzzy PID controller is designed in Matlab / Simulink to achieve the goal of motion trajectory planning and control. The fuzzy PID controller can dynamically adjust the PID parameters based on fuzzy logic reasoning of the error and its rate of change, thereby improving the robustness and adaptability of the control system.

[0142] For example, see Figure 4The illustrated robotic arm control flowchart shows "Machine Vision System Connection Successful" 401 and "Robotic Arm System Connection Successful" 402 as the starting points of the process, ensuring that the two key systems are ready to work together. "Real-time Robotic Arm System Operation Trajectory Map" 403 indicates that the robotic arm system is operating according to a preset or real-time calculated planned motion trajectory. "Planned Motion Trajectory Information" 404 provides the expected trajectory that the robotic arm's end effector needs to follow. "Real-time Operation Trajectory Information" 405 is the actual trajectory followed by the robotic arm's end effector during execution, which has errors compared to "Planned Motion Trajectory Information" 404 and needs to be adjusted by the control system. "Control System Status" 406 indicates that the control system is monitoring and comparing "Planned Motion Trajectory Information" 404 with the real-time trajectory and correcting errors accordingly.

[0143] For example, see Figure 5 The schematic diagram shown is of the structure of the robotic arm control system, which includes a trajectory control system 501, a robotic arm system 502, and a vision system 503.

[0144] The trajectory control system 501 includes a planned motion trajectory 504 and a controller 505. The planned motion trajectory 504 is the desired trajectory that the end effector of the robotic arm is expected to follow by the robotic arm control system. The controller 505 is the core of the robotic arm control system. It receives the planned motion trajectory and calculates the control signal based on the error 508 between the actual coordinate posture 512 transmitted by the vision system 503 and the desired coordinate posture.

[0145] The robotic arm system 502 includes a job program input 506, a robotic arm operation 507, and an actual coordinate posture 512. The job program input 506 is used to input control signals generated by the controller 505 into the robotic arm system 502 to guide the movement of the robotic arm. The robotic arm operation 507 is used to execute corresponding actions, such as rotation and translation, according to the received control signals to realize the planned motion trajectory 504. The actual coordinate posture 512 is the coordinate posture of the actual trajectory that the end effector of the robotic arm actually travels in space.

[0146] The vision system 503 includes a real position 509, an image processing module 510, and a camera 511. The real position 509 outputs the actual three-dimensional coordinates of the end effector of the robotic arm to the image processing module, which are used to compare with the planned motion trajectory 504 and calculate the error 508. The image processing module 510 is responsible for extracting the position information of the end effector of the robotic arm from the image captured by the camera. The camera 511 is used to capture the actual position of the end effector of the robotic arm, and the image processing module 510 converts the image data into the three-dimensional coordinates of the end effector of the robotic arm.

[0147] For example, see Figure 6The diagram shows a six-DOF robotic arm. Each joint and skeletal arm is defined by parameters: joint angle (θ), skeletal arm length (a), and skeletal arm offset (d). The diagram shows six joints (rotational joints), each represented by a circle 601, and the skeletal arms between the joints represented by rectangles 602. Each joint can rotate about its axis, thereby moving the skeletal arm. Each joint has its own local coordinate system, represented by three coordinate axes X, Y, and Z. The base coordinate system X... w Y w Z w The initial reference coordinate system of the robotic arm is 603; the coordinate system of the skeletal arm is represented by (X1-X6, Y1-Y6, Z1-Z6); a1 is the distance along the X-axis of the previous coordinate system, describing the horizontal or axial offset of the adjacent skeletal arm (e.g., a1 is the distance from {X1} to {X2} along the X1 axis), and a2 and a3 are similar to a1; d1 is the distance along the Z-axis of the previous coordinate system, describing the vertical or axial offset of the adjacent skeletal arm (e.g., d1 is the distance from {X1} to {X2} along the X1 axis). w} to {X1} along Z w (Distance between axes), d5, d6 and d1 are similar.

[0148] This application boasts low operating costs, high compatibility, and flexible application capabilities. It can plan and control work trajectories for various sizes and types of workpieces, meeting the needs of users with diverse work types. This application utilizes visual image acquisition to plan motion trajectories and the actual working trajectory of the robotic arm, enabling real-time trajectory control, reducing operational errors, and adapting to workpieces of different structures, types, and materials for trajectory planning. It solves the problems of low precision and manual pre-operation in robotic arm processing trajectories using teaching methods, and the heavy reliance on the accuracy and precision of the 3D models of the robot and workpiece in computer-aided design methods.

[0149] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0150] Please refer to Figure 7 The diagram illustrates a block diagram of a robotic arm control system according to an embodiment of this application. The robotic arm includes at least two cascaded skeletal arms, with adjacent skeletal arms connected by joints. The device includes:

[0151] The acquisition module 701 is used to acquire the coordinates of the workpiece to be processed in a two-dimensional coordinate system through a vision sensor;

[0152] The first determining module 702 is used to determine the desired coordinate posture of the end effector of the robotic arm in three-dimensional coordinates based on the coordinates of the workpiece to be processed in a two-dimensional coordinate system.

[0153] The second determining module 703 is used to determine the planned motion trajectory of the end effector of the robotic arm from the initial coordinate posture to the desired coordinate posture based on the desired coordinate posture.

[0154] The control module 704 is used to control the end effector of the robotic arm to move from the initial coordinate posture to the desired coordinate posture based on the planned motion trajectory.

[0155] In one possible implementation, the control module 704 is used to determine the desired coordinate posture of the end effector of the robotic arm at the t-th moment during the motion process, based on the planned motion trajectory, where t is a positive integer;

[0156] Based on the desired coordinate posture at time t, the joint sub-posture of each joint of the robotic arm at time t is determined. The joint sub-posture at time t is characterized by the relative change of the joint sub-posture at time t-1.

[0157] Based on the joint sub-pose of each joint at time t, control each joint so that the end effector of the robotic arm moves from the initial coordinate pose to the desired coordinate pose.

[0158] In one possible implementation, the control module 704 is used to determine the joint sub-coordinates of each joint at time t based on the joint sub-pose of each joint at time t.

[0159] Based on the joint sub-pose and joint sub-coordinate, each joint is controlled so that the end effector of the robotic arm moves from the initial coordinate pose to the desired coordinate pose.

[0160] In one possible implementation, the control module 704 is used to determine the correspondence between the joint sub-poses and the trajectory motion time of each joint of the robotic arm based on the expected coordinate pose at time t, the initial poses of each joint, and the trajectory motion time.

[0161] Based on the correspondence, determine the joint sub-pose of each joint of the robotic arm at time t.

[0162] In one possible implementation, the acquisition module 701 is further configured to acquire the error at time t-1 using a vision sensor. The error is used to characterize the difference between the actual coordinate posture of the end effector of the robotic arm and the expected coordinate posture at time t-1. The control module 704 is configured to correct the planned motion trajectory based on the error to obtain the corrected motion trajectory.

[0163] The desired coordinate posture of the robotic arm's end effector at time t during the motion process is determined based on the corrected motion trajectory.

[0164] In one possible implementation, the first determining module 702 is used to determine the desired coordinate posture of the end effector of the robotic arm in three-dimensional coordinates based on the coordinates of the workpiece to be processed in the two-dimensional coordinate system and the transformation relationship.

[0165] The transformation relationship refers to the transformation from the vision system coordinate system to the end effector coordinate system of the robotic arm.

[0166] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the content structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0167] This application also provides a computer device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set. The processor loads and executes the at least one instruction, at least one program, code set, or instruction set to implement the robotic arm control method provided in the above-described method embodiments.

[0168] For example, Figure 8 This is a structural block diagram of a computer device 1000 provided in an exemplary embodiment of this application. The computer device 1000 may be... Figure 1 The computer device shown is used to implement the robotic arm control method provided in the above embodiments.

[0169] The computer device 1000 includes a central processing unit (CPU) 1001, a system memory 1004 including random access memory (RAM) 1002 and read-only memory (ROM) 1003, and a system bus 1005 connecting the system memory 1004 and the CPU 1001. The computer device 1000 also includes a basic input / output system (I / O system) 1006 to facilitate information transfer between various components within the computer device, and a mass storage device 1007 for storing the operating system 1013, application programs 1014, and other program modules 1015.

[0170] The basic input / output system 1006 includes a display 1008 for displaying information and an input device 1009 for user input, such as a mouse or keyboard. Both the display 1008 and the input device 1009 are connected to the central processing unit 1001 via an input / output controller 1010 connected to the system bus 1005. The basic input / output system 1006 may also include the input / output controller 1010 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1010 also provides output to a display screen, printer, or other types of output devices.

[0171] The mass storage device 1007 is connected to the central processing unit 1001 via a mass storage controller (not shown) connected to the system bus 1005. The mass storage device 1007 and its associated computer-readable storage media provide non-volatile storage for the computer device 1000. That is, the mass storage device 1007 may include computer-readable storage media (not shown), such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0172] Without loss of generality, the computer-readable storage medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable storage instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage devices, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage medium is not limited to the above-mentioned types. The system memory 1004 and mass storage device 1007 described above can be collectively referred to as memory.

[0173] The memory stores one or more programs, which are configured to be executed by one or more central processing units 1001. The one or more programs contain instructions for implementing the above method embodiments. The central processing unit 1001 executes the one or more programs to implement the robotic arm control methods provided by the above method embodiments.

[0174] According to various embodiments of this application, the computer device 1000 can also be connected to a remote computer device on a network, such as the Internet. That is, the computer device 1000 can be connected to the network 1012 via the network interface unit 1011 connected to the system bus 1005, or the network interface unit 1011 can be used to connect to other types of networks or remote computer device systems (not shown).

[0175] The memory also includes one or more programs stored in the memory, and the one or more programs include steps executed by a computer device in the robotic arm control method provided in the embodiments of this application.

[0176] This application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set. When the at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor of a computer device, the robotic arm control method provided in the above-described method embodiments is implemented.

[0177] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the robotic arm control method provided in the above-described method embodiments.

[0178] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0179] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent switching, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A robotic arm control method, characterized in that, The robotic arm includes at least two cascaded skeletal arms, with adjacent skeletal arms connected by joints, and the method includes: The coordinates of the workpiece to be processed in a two-dimensional coordinate system are acquired using a vision sensor; Based on the coordinates of the workpiece to be processed in the two-dimensional coordinate system and the transformation relationship, the desired coordinate posture of the end effector of the robotic arm in the three-dimensional coordinate system is determined. The transformation relationship is the relationship of transformation from the vision system coordinate system to the end effector coordinate system of the robotic arm. Based on the desired coordinate posture, the planned motion trajectory of the end effector of the robotic arm from the initial coordinate posture to the desired coordinate posture is determined; The error at time t-1 is acquired by the vision sensor, and the error is used to characterize the error between the actual coordinate posture of the end of the robotic arm and the expected coordinate posture at time t-1. The planned motion trajectory is corrected based on the error to obtain the corrected motion trajectory; Based on the corrected motion trajectory, the desired coordinate posture of the end effector of the robotic arm at the t-th moment during the motion process is determined, where t is a positive integer; Based on the desired coordinate posture at the t-th time, the joint sub-posture of each joint of the robotic arm at the t-th time is determined, and the joint sub-posture at the t-th time is characterized by the relative change with respect to the joint sub-posture at the (t-1)-th time. Based on the joint sub-pose of each joint at the t-th time, each joint is controlled so that the end effector of the robotic arm moves from the initial coordinate pose to the desired coordinate pose.

2. The method according to claim 1, characterized in that, The step of controlling each joint based on its corresponding joint sub-pose at time t, so that the end effector of the robotic arm moves from the initial coordinate pose to the desired coordinate pose, includes: Based on the joint sub-pose of each joint at the t-th time, determine the joint sub-coordinates of each joint at the t-th time. Based on the joint sub-pose and the joint sub-coordinate, each joint is controlled so that the end effector of the robotic arm moves from the initial coordinate pose to the desired coordinate pose.

3. The method according to claim 1 or 2, characterized in that, The step of determining the joint sub-pose of each joint of the robotic arm at the t-th time based on the expected coordinate pose at the t-th time includes: Based on the desired coordinate posture at the t-th time, the initial posture of each joint, and the trajectory motion time, the correspondence between the joint sub-posture of each joint of the robotic arm and the trajectory motion time is determined. Based on the correspondence, the joint sub-pose of each joint of the robotic arm at the t-th time is determined.

4. A robotic arm control device, characterized in that, The robotic arm includes at least two skeletal arms linked in a series, with adjacent skeletal arms connected by joints. The device includes: The acquisition module is used to acquire the coordinates of the workpiece to be processed in a two-dimensional coordinate system through a vision sensor; The first determining module is used to determine the desired coordinate posture of the end effector of the robotic arm in three-dimensional coordinates based on the coordinates of the workpiece to be processed in a two-dimensional coordinate system and the transformation relationship, wherein the transformation relationship is the relationship of transformation from the vision system coordinate system to the end effector coordinate system of the robotic arm. The second determining module is used to determine the planned motion trajectory of the end effector of the robotic arm from the initial coordinate posture to the desired coordinate posture based on the desired coordinate posture. The control module is configured to: acquire the error at time t-1 using the vision sensor, wherein the error characterizes the difference between the actual coordinate posture of the end effector of the robotic arm and the desired coordinate posture at time t-1; correct the planned motion trajectory based on the error to obtain a corrected motion trajectory; determine the desired coordinate posture of the end effector of the robotic arm at time t during the motion process based on the corrected motion trajectory, where t is a positive integer; determine the joint sub-postures of each joint of the robotic arm at time t based on the desired coordinate posture at time t, wherein the joint sub-postures at time t are characterized by relative changes to the joint sub-postures at time t-1; and control each joint based on the joint sub-postures at time t to move the end effector of the robotic arm from the initial coordinate posture to the desired coordinate posture.

5. A terminal device, characterized in that, The terminal device includes a processor and a memory, the memory storing a computer program, which is loaded and executed by the processor to implement the robotic arm control method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the robotic arm control method as described in any one of claims 1 to 3.

7. A computer program product, characterized in that, The computer program product includes a computer program stored in a computer-readable storage medium, and a processor reads from and executes the computer program to implement the robotic arm control method as described in any one of claims 1 to 3.

Images