Control method and control device of mechanical arm

Abstract

The present invention proposes a control method and a control device for a robotic arm, which relates to the technical field of industrial automation equipment. The control method includes the following steps: the robotic arm acquires an environmental image; calculates the conversion relationship between the pixel coordinates in the environmental image and the physical coordinates of the robotic arm, and obtains the conversion relationship between the control coordinates and the physical coordinates; and controls the robotic arm to move to the physical coordinate position selected by the user. By combining image processing and coordinate conversion technology, the present invention can automatically plan a path and perform operations according to the target specified by the user through the image interface, thereby improving the robotic arm's perception of the environment and positioning accuracy, simplifying the user interaction process, improving the convenience of operation, and improving the automation level and flexibility of the robotic arm.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial automation equipment, and in particular to a control method and a control device for a robotic arm. Background Art

[0002] A robotic arm is an automated mechanical device that has functions similar to human arms and can perform various tasks. It is programmable and can accept instructions to accurately move objects according to their spatial positions and postures. The robotic arm can replace manual labor to perform repetitive, high-intensity and high-risk tasks, such as carrying heavy objects and assembling parts, thereby greatly improving production efficiency and output. Through precise operation and continuous operation, the robotic arm can complete a large amount of work in a short period of time, reducing the time and labor costs of manual operation.

[0003] When operating in complex or dynamic environments, traditional robotic arms often rely on preset coordinate systems or external positioning devices, which may result in insufficient positioning accuracy or slow response speed; their control systems may require users to input complex coordinate values ​​or use professional control software, which increases the user's operating difficulty and learning cost. Therefore, there is a need for a control method and device that can control the operation of a robotic arm by combining image processing and coordinate conversion technology, improve the robotic arm's environmental and positioning accuracy, and simplify the user interaction process. Summary of the invention

[0004] The main purpose of the present invention is to provide a control method and a control device for a robotic arm, aiming to solve the problems of insufficient positioning accuracy of the robotic arm and great difficulty in user operation.

[0005] To achieve the above object, the present invention provides a method for controlling a robotic arm, comprising the following steps:

[0006] The robotic arm acquires an image of the environment;

[0007] Calculating the conversion relationship between the pixel coordinates in the environment image and the physical coordinates of the robotic arm;

[0008] Acquire the conversion relationship between the control coordinates and the pixel coordinates, and obtain the conversion relationship between the control coordinates and the physical coordinates;

[0009] A user setting instruction is obtained, and according to the user setting instruction and the conversion relationship between the control coordinates and the physical coordinates, the robot arm is controlled to move to the physical coordinate position selected by the user.

[0010] Optionally, the specific steps of the robotic arm acquiring the environment image include:

[0011] Collecting real-time environmental images of the robotic arm through a camera;

[0012] Calibrate the camera to obtain internal parameter data and external parameter data of the camera;

[0013] The environment image captured by the camera is preprocessed.

[0014] Optionally, the specific step of calculating the conversion relationship between the pixel coordinates in the environment image and the physical coordinates of the robotic arm includes:

[0015] Based on the starting position and orientation of the robot arm, a physical coordinate system of the robot arm is established { W};

[0016] Based on the camera's internal and external parameter data, establish the camera coordinate system { C} and obtain the pixel coordinate system of the environment image through the conversion relationship { I};

[0017] Based on the physical coordinate system { W} and the camera coordinate system { C}, the camera coordinate system { C} and the pixel coordinate system { I}, and calculate the conversion relationship between pixel coordinates and physical coordinates.

[0018] Optionally, the camera coordinate system is established based on the internal parameter data and the external parameter data of the camera. C} and obtain the pixel coordinate system of the environment image through the conversion relationship { I Also includes:

[0019] The camera coordinate system { C}Based on its own and imaging plane coordinate system{ F} and the imaging plane coordinate system { F} and the pixel coordinate system { I} to obtain the pixel coordinate system { I}.

[0020] Optionally, the specific steps of acquiring the conversion relationship between the control coordinates and the pixel coordinates and deriving the conversion relationship between the control coordinates and the physical coordinates include:

[0021] Displaying the acquired environment image on a user interface;

[0022] Based on the position and size of the controls displayed in the user interface, a control coordinate system is constructed { L};

[0023] Through the control coordinate system { L} and pixel coordinate system { I} and the conversion relationship between pixel coordinates and physical coordinates to obtain the conversion relationship between control coordinates and physical coordinates.

[0024] Optionally, the specific steps of obtaining the user setting instruction and controlling the robot arm to move to the physical coordinate position selected by the user according to the user setting instruction and the conversion relationship between the control coordinates and the physical coordinates include:

[0025] Obtaining a user setting instruction, and obtaining the physical coordinate position of the robot arm according to the user setting instruction and the conversion relationship between the control coordinates and the physical coordinates;

[0026] After obtaining the physical coordinate position of the robot arm, path planning is performed;

[0027] Based on the results of path planning, the robot arm is controlled to move to the physical coordinate position selected by the user.

[0028] The present invention also provides a control device, including a memory, a processor, and a robot arm control program stored in the memory and executable on the processor, wherein the robot arm control program implements the steps of the above-mentioned robot arm control method when executed by the processor.

[0029] The present invention provides a robotic arm and a control method thereof. By combining image processing and coordinate conversion technology, the robotic arm can automatically plan a path and perform operations according to a target specified by a user through an image interface, thereby improving the robotic arm's ability to perceive the environment and positioning accuracy, simplifying the user interaction process, improving the convenience of operation, and improving the robotic arm's automation level and flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0031] Figure 1 A flowchart of a control method for a mechanical arm according to the present invention;

[0032] Figure 2 This is a flow chart of step S100 of an embodiment of a method for controlling a robot arm of the present invention;

[0033] Figure 3 This is a flow chart of step S200 of an embodiment of a method for controlling a robot arm of the present invention;

[0034] Figure 4 This is a flow chart of step S220 of an embodiment of a method for controlling a robot arm of the present invention;

[0035] Figure 5 This is a flow chart of step S300 of an embodiment of a method for controlling a robot arm of the present invention;

[0036] Figure 6 This is a flow chart of step S400 of an embodiment of a method for controlling a robotic arm of the present invention.

[0037] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0040] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0041] In addition, in the present invention, descriptions such as "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0042] The present invention provides a control method for a robotic arm, such as Figure 1 As shown, the following steps are included:

[0043] Step S100: the robot arm acquires an environment image;

[0044] The robot arm obtains environmental image information through image acquisition devices such as cameras, etc. It is worth noting that the image acquisition device needs to be installed on the robot arm or fixed at an appropriate position in the working area to ensure that the target area can be covered and a clear image can be obtained.

[0045] The image acquisition device captures the light signals of the surrounding environment and converts them into digital signals, namely environmental image data. The environmental image data is transmitted to the control system or host computer of the robot arm via a data line or wirelessly for processing.

[0046] Step S200: Calculate the conversion relationship between the pixel coordinates in the environment image and the physical coordinates of the robot arm;

[0047] First, it is necessary to align the image acquisition device and calculate the conversion relationship between the coordinate system of the image acquisition device itself and the coordinate system of the robotic arm. Then, the conversion relationship between the image acquisition device coordinate system and the pixel coordinates in the environmental image is obtained by combining the image acquisition device coordinate system with the imaging principle and the projection principle. The conversion includes translation conversion, rotation conversion and composite conversion.

[0048] Step S300: obtaining the conversion relationship between the control coordinates and the pixel coordinates, and obtaining the conversion relationship between the pixel coordinates and the physical coordinates of the robot arm through S200, thereby obtaining the conversion relationship between the control coordinates and the physical coordinates;

[0049] Usually refers to the relative coordinate system inside a control (such as a button, text box, etc.), whose origin (0,0) is usually located at the upper left corner of the control or according to the specific layout properties of the control. In most cases, if the control is drawn directly on the screen without transformations such as scaling or rotation, the relationship between the control coordinates and the pixel coordinates on the screen may be a simple offset. That is, the control coordinates plus the position of the control on the screen (usually in pixels) can get the corresponding pixel coordinates.

[0050] Step S400: Obtain the user setting instruction, and according to the user setting instruction, convert the control coordinates into pixel coordinates, convert the pixel coordinates into physical coordinates, and control the robot arm to move to the physical coordinate position selected by the user.

[0051] Get user setting instructions. The user inputs the control coordinates, which first need to be converted into pixel coordinates on the screen. This usually involves the position and size of the control, as well as the resolution of the screen. Next, the pixel coordinates need to be converted into physical coordinates that the robot arm can understand. This may require considering factors such as the calibration data of the camera or sensor, the basic position of the robot arm, and the size and layout of the workspace.

[0052] Once the physical coordinates are obtained, motion instructions can be sent to the robotic arm to move to that location; during the movement of the robotic arm, it may be necessary to receive real-time feedback from the robotic arm in order to monitor its status or make necessary adjustments.

[0053] The present invention provides a robotic arm and a control method and device thereof. By combining image processing and coordinate conversion technology, the robotic arm can automatically plan a path and perform operations according to a target specified by a user through an image interface, thereby improving the robotic arm's ability to perceive the environment and positioning accuracy, simplifying the user interaction process, improving the convenience of operation, and improving the robotic arm's automation level and flexibility.

[0054] In one implementation, if Figure 2 As shown, the specific steps for the robotic arm to obtain the environment image include:

[0055] Step S110: collecting a real-time environmental image of the robotic arm through a camera;

[0056] In this step, the

[0057] The camera needs to ensure that it can cover the target area and obtain a comprehensive image of the environment. The camera's field of view, light conditions, and possible obstructions need to be considered during installation.

[0058] Select the appropriate camera type according to the requirements of the application scenario, and select a camera with appropriate resolution and frame rate to ensure that a clear image can be captured and meet the real-time requirements. In this embodiment, the camera is preferably an industrial camera, which focuses light on a photosensitive element through a lens to form an image. Thereafter, the environmental image data is acquired and transmitted to the processing system in real time through the camera driver or API interface.

[0059] Step S120: calibrating the camera to obtain internal parameter data and external parameter data of the camera;

[0060] Camera calibration is the process of obtaining the camera's intrinsic and extrinsic parameters, which are crucial for converting image coordinates into physical coordinates in the robot's workspace.

[0061] Intrinsic parameter calibration: mainly includes calibration of parameters such as focal length, optical center (image center), distortion coefficient, etc. This usually requires shooting with a specific calibration plate (such as a chessboard calibration plate) and calculating the intrinsic parameter matrix through image processing algorithms.

[0062] External parameter calibration: that is, determining the rotation matrix and translation vector between the camera coordinate system and the physical coordinate system of the robot. This can be done by placing markers or objects with known positions in the scene and using image processing algorithms to estimate the camera's pose.

[0063] Step S130: pre-processing the environment image captured by the camera.

[0064] Image preprocessing is an important step in image processing, which aims to improve image quality and make it more suitable for subsequent target recognition, positioning and other tasks.

[0065] Among them, they include: Grayscale: Convert color images to grayscale images to reduce the amount of calculation and storage space while retaining sufficient image information. Filtering: Use filters (such as Gaussian filters, median filters, etc.) to remove noise and interference in the image.

[0066] Edge detection: Extract edge information from the image through edge detection algorithms (such as Canny edge detector, Sobel operator, etc.), providing a basis for subsequent feature extraction and target recognition. Histogram equalization: Adjust the contrast of the image to make the details of the image clearer.

[0067] In one implementation, if Figure 3 As shown, the specific steps of calculating the conversion relationship between the pixel coordinates in the environment image and the physical coordinates of the robotic arm include:

[0068] Step S210: Based on the starting position and orientation of the robot arm, establish the physical coordinate system of the robot arm { W};

[0069] The position and posture description of the robot arm is usually expressed using a homogeneous transformation matrix. This matrix describes the robot arm from the physical coordinate system { W} to another position.

[0070] Physical coordinate system { W} is a fixed coordinate system, usually used to describe the position and posture of the robot arm in the working area. The origin and axes of this coordinate system are usually related to the starting position and orientation of the robot arm. The physical coordinates of this coordinate system ( x w , y w , z w ).

[0071] Step S220: Establish a camera coordinate system based on the camera's internal parameter data and external parameter data { C} and obtain the pixel coordinate system of the environment image through the conversion relationship { I};

[0072] In this embodiment, the image acquisition device selects a camera.

[0073] Camera coordinate system { C} is the coordinate system used when the camera is fixed on the robot arm. In this coordinate system, the optical center of the camera is located at the origin of the coordinate system, and the optical axis is z The axes coincide with each other. x Axis and y The axes are aligned with the horizontal and vertical directions of the camera respectively. The camera coordinates in this coordinate system ( x c , y c , z c ).

[0074] Pixel coordinate system { I}: The pixel coordinate system is a two-dimensional coordinate system. Each pixel has a unique coordinate, and its coordinate value is expressed as a discrete pixel point coordinate. In the pixel coordinate system, the origin of the coordinate is usually in the upper left corner of the image. x The axis extends to the right. y The axis extends downward. Therefore, the coordinates in the pixel coordinate system are usually P =( u,v ) form, where u and v Represent the column and row of the pixel in the image respectively. The coordinates of the optical center in the pixel coordinate system ( u 0 ，v 0).

[0075] Step S230: Based on the physical coordinate system { W} and the camera coordinate system { C} conversion relationship, camera coordinate system { C} and pixel coordinate system { I}, and calculate the conversion relationship between pixel coordinates and physical coordinates.

[0076] The above physical coordinate system { W}、Camera coordinate system{ C} and pixel coordinate system { I The conversion between} can be achieved through the transformation matrix. The three-dimensional space point representation is usually used to describe the three-dimensional position of an object. It consists of three values, which represent the x, y and z coordinates in the three-dimensional coordinate system. In the rectangular coordinate system, any point can be PIt is expressed as: .

[0077] Translation transformation refers to moving a point a certain distance in a specified direction in a coordinate system. The translation makes the origins of the two coordinate systems coincide. Use the position vector Represents the coordinate system { A}The origin is in the coordinate system{ B}, the translation equation is: ;

[0078] Rotation transformation refers to a transformation method that changes the direction and posture of the coordinate system through rotation operations. Rotation transformation is usually performed around a certain axis. Rotation transformation can be described by a rotation matrix, which is an orthogonal matrix that represents the new vector obtained by rotating a vector around a certain axis by a certain angle. B} can be obtained by rotating the matrix Get the coordinate system { A}, then click P In the coordinate system { A} can be expressed as: ;

[0079] Composite transformation refers to the process of performing multiple transformation operations in a certain order to obtain a new transformation. Common composite transformations include translation transformation, rotation transformation, scaling transformation, etc. The order of composite transformation is very important, and different orders may produce completely different results. For example, when performing rotation transformation and translation transformation, their order cannot be interchanged at will, otherwise it will lead to different results. In a three-dimensional coordinate system, if we need to transform an object from one coordinate system to another, we can use a composite transformation of multiple transformation operations to achieve it. The usual practice is to perform a translation transformation first, moving the object from its original coordinate system to its original coordinate system.

[0080] The origin of the reference system is translated to the origin of the target coordinate system, and then rotation and scaling operations are performed to finally obtain the transformation matrix that represents the position and direction of the object in the target coordinate system. P In the coordinate system { A} Need to be from the coordinate system { B} firstly uses rotation transformation and then translation transformation, which can be expressed as: .

[0081] Camera coordinate system { C} and physical coordinate system { W}The conversion relationship between:

[0082]

[0083] Camera coordinate system {C} and pixel coordinate system { I}The conversion relationship between:

[0084] ;

[0085] .

[0086] in, f is the focal length of the camera, dx and dy are the physical sizes of each pixel on the image sensor in the x-axis and y-axis directions respectively.

[0087] In one implementation, if Figure 4 As shown, based on the camera's internal parameter data and external parameter data, the camera coordinate system is established { C} and obtain the pixel coordinate system of the environment image through the conversion relationship { I Also includes:

[0088] Camera coordinate system { C}Based on its own and imaging plane coordinate system{ F} and the imaging plane coordinate system { F} and pixel coordinate system { I} to obtain the pixel coordinate system { I}.

[0089] The specific steps of step S220 include:

[0090] Step S221: Establish a camera coordinate system based on the camera's internal parameter data and external parameter data { C};

[0091] Step S222: Camera coordinate system { C}Based on its own and imaging plane coordinate system{ F} and the imaging plane coordinate system { F} and pixel coordinate system { I} to obtain the pixel coordinate system { I};

[0092] Step S223: Obtain camera coordinate system { C} and pixel coordinate system { I} conversion relationship.

[0093] Imaging plane coordinate system { F} is a two-dimensional coordinate system used to represent camera imaging. It is usually a plane that can be regarded as the photoreceptor surface of the camera or the plane of the imaging sensor (such as CCD or CMOS). On this plane, the image formed by the camera is projected into the two-dimensional coordinate system, and each pixel has a unique coordinate in this coordinate system. The imaging plane coordinate system is usually defined as perpendicular to the camera optical axis and aligned with the horizontal and vertical directions of the camera. In the imaging plane coordinate system, it is usually used x and y The coordinates represent the column and row of the pixel in the image, respectively.

[0094] The camera imaging model refers to the process of imaging a three-dimensional object in a camera as an abstract mathematical model for use in related applications in computer vision and robot vision. Pinhole imaging is a commonly used imaging model, which refers to the introduction of light into the camera's imaging system through a small circular hole, and the formation of an image through the refraction and focusing of light. It is suitable for complex optical imaging systems and has relatively good image quality. The camera's imaging system can be regarded as a concave lens that projects points in three-dimensional space onto the camera's imaging plane.

[0095] In one implementation, if Figure 5 As shown, the specific steps of obtaining the conversion relationship between the control coordinates and the pixel coordinates and obtaining the conversion relationship between the control coordinates and the physical coordinates include:

[0096] Step S310: The environment image captured by the camera is displayed on the user interface; the user obtains information about the environment image through this interface, and performs human-computer interaction through the interface, thereby controlling the position of the robot arm movement.

[0097] Step S320: Construct a control coordinate system based on the position and size of the control displayed on the user interface { L}; First, you need to determine the exact position of each control on the user interface (usually in pixels). This is usually set through the user interface layout file and / or programming logic. For each control, you can build a local coordinate system { L}, whose origin (0,0) is usually set at the upper left corner of the control or determined according to the layout characteristics of the control. L}of X Axis and Y The axes are parallel to the horizontal and vertical directions of the user interface, respectively.

[0098] Step S330: Through the control coordinate system { L} and pixel coordinate system { I} and the conversion relationship between pixel coordinates and physical coordinates to obtain the conversion relationship between control coordinates and physical coordinates.

[0099] Control coordinate system {L} to pixel coordinate system { I} is a simple translation transformation, because the control coordinate system { L}'s origin and direction have been aligned with the pixel coordinate system { I}aligned (just offset in position). L} = ( u ' ， v ′), pixel coordinate system { I} = ( u,v );

[0100] u ' =u+a ;

[0101] v ' =v+b .

[0102] in, a, b is the offset 。

[0103] In one implementation, if Figure 6 As shown, the specific steps of obtaining the user setting instruction and controlling the robot arm to move to the physical coordinate position selected by the user according to the user setting instruction and the conversion relationship between the control coordinate and the physical coordinate include:

[0104] Step S410: Obtain the user setting instruction, and obtain the physical coordinate position of the robot arm according to the user setting instruction and the conversion relationship between the control coordinates and the physical coordinates;

[0105] According to the control coordinate system established previously { L} and pixel coordinate system { I}, and the pixel coordinate system { I} and physical coordinate system { W}, converting the position specified by the user through the control (control coordinates) into physical coordinates that the robot can understand. This step may include the application of multiple mathematical transformations and calibration parameters. Through the above conversion process, the system finally obtains the physical coordinate position that the robot needs to reach.

[0106] Step S420: After obtaining the physical coordinate position of the robot arm, perform path planning;

[0107] Path planning is an important part of robot motion control, which involves planning an optimal or feasible path for the robot arm from its current position to its target position. This path needs to avoid obstacles while taking into account the kinematic limitations and dynamic characteristics of the robot arm.

[0108] Step S430: Based on the results of path planning, the robot arm is controlled to move to the physical coordinate position selected by the user. Based on the results of path planning, instructions for controlling the movement of the robot arm are generated. These instructions usually include parameters such as the target angle, speed, and acceleration of each joint. The control instructions are sent to the control system of the robot arm. The control system drives the motor and other actuators of the robot arm according to the instructions, so that the robot arm moves along the planned path.

[0109] During the movement, it may be necessary to receive feedback from the robot in real time and adjust the control instructions based on this information to ensure that the robot can reach the target position accurately and stably. When the robot reaches the physical coordinate position selected by the user, the control system will stop sending control instructions and the robot will stop moving.

[0110] The present invention also provides a control device, including a memory, a processor, and a robot arm control program stored in the memory and executable on the processor. When the robot arm control program is executed by the processor, the steps of the above-mentioned robot arm control method are implemented.

[0111] The present invention also provides a mechanical arm, comprising the above-mentioned control device.

[0112] In one implementation, the robotic arm further includes a camera, which is rotatably mounted at the end of the robotic arm.

[0113] The camera is rotatably mounted at the end of the robot arm, which brings many advantages. On the one hand, it expands the camera's field of view, allowing the robot arm to operate in a wider working area. On the other hand, by adjusting the angle and position of the camera, the robot arm can more flexibly handle objects of different shapes and sizes and achieve more complex operation tasks.

[0114] In one embodiment, the robot arm is a selective compliance assembly robot arm. Selective Compliance Assembly Robot Arm (SCARA) is actually a specific type of industrial robot that is well known for its unique design features and application areas. SCARA robots are widely used in a variety of industrial automation tasks such as assembly, pick and place, packaging and inspection due to their high flexibility, accuracy and ability to move quickly in two-dimensional space.

[0115] SCARA robots usually have four degrees of freedom, of which the first three joints allow the arm to be accurately positioned and oriented in the horizontal plane, while the fourth joint is used to provide additional flexibility in the vertical direction or to install and rotate tools. At the same time, SCARA robots have high rigidity in certain directions to ensure that the necessary force can be applied during assembly or operation, and high flexibility in other directions for precise position adjustment.

[0116] The above descriptions are only optional embodiments of the present invention, and are not intended to limit the patent scope of the present invention. All equivalent structural changes made using the contents of the present invention's specification and drawings, or directly / indirectly applied in other related technical fields, are included in the patent protection scope of the present invention.

Claims

1. A method for controlling a robotic arm, characterized in that: The following steps are involved: The robot arm acquires an environmental image; the robot arm acquires a real-time environmental image through a camera; the camera is calibrated to acquire internal and external parameter data of the camera; and the environmental image acquired by the camera is preprocessed; Calculate the conversion relationship between the pixel coordinates in the environment image and the physical coordinates of the robotic arm; establish the physical coordinate system of the robotic arm based on the starting position and orientation of the robotic arm { W }; Based on the camera's internal and external parameter data, establish the camera coordinate system { C } and obtain the pixel coordinate system of the environment image through the conversion relationship { I }; Based on the physical coordinate system { W } and the camera coordinate system { C }, the camera coordinate system { C } and the pixel coordinate system { I }, calculate the conversion relationship between pixel coordinates and physical coordinates; Acquire the conversion relationship between the control coordinates and the pixel coordinates, and obtain the conversion relationship between the control coordinates and the physical coordinates; A user setting instruction is obtained, and according to the user setting instruction and the conversion relationship between the control coordinates and the physical coordinates, the robot arm is controlled to move to the physical coordinate position selected by the user.

2. The control method of the robot arm according to claim 1, characterized in that: The camera coordinate system is established based on the internal parameter data and external parameter data of the camera. C } and obtain the pixel coordinate system of the environment image through the conversion relationship { I Also includes: The camera coordinate system { C }Based on its own and imaging plane coordinate system{ F } and the imaging plane coordinate system { F } and the pixel coordinate system { I } to obtain the pixel coordinate system { I }.

3. The control method of the robot arm according to claim 1, characterized in that: The specific steps of acquiring the conversion relationship between the control coordinates and the pixel coordinates and obtaining the conversion relationship between the control coordinates and the physical coordinates include: Displaying the acquired environment image on a user interface; Based on the position and size of the controls displayed in the user interface, a control coordinate system is constructed { L }; Through the control coordinate system { L } and pixel coordinate system { I } and the conversion relationship between pixel coordinates and physical coordinates to obtain the conversion relationship between control coordinates and physical coordinates.

4. The control method of the robot arm according to claim 1, characterized in that: The specific steps of obtaining the user setting instruction and controlling the robot arm to move to the physical coordinate position selected by the user according to the user setting instruction and the conversion relationship between the control coordinate and the physical coordinate include: Obtaining a user setting instruction, and obtaining the physical coordinate position of the robot arm according to the user setting instruction and the conversion relationship between the control coordinates and the physical coordinates; After obtaining the physical coordinate position of the robot arm, path planning is performed; Based on the results of path planning, the robot arm is controlled to move to the physical coordinate position selected by the user.

5. A control device, characterized in that: It comprises a memory, a processor and a robot arm control program stored in the memory and executable on the processor, wherein the robot arm control program implements the steps of the robot arm control method as claimed in any one of claims 1 to 4 when executed by the processor.

Images