A multi-joint robot arm high-precision trajectory optimization method, system, device and medium using visual perception constraints

Abstract

The application is suitable for the field of mechanical arm movement and trajectory planning, and discloses a multi-joint mechanical arm high-precision trajectory optimization method, system, device and medium using visual perception constraints, which comprises the following steps: obtaining target features of real-time visual images and performing three-dimensional reconstruction to obtain a desired pose of the target; obtaining an actual pose based on a forward kinematics model according to joint angle information; constructing a visual constraint error vector by aligning the desired pose and the actual pose; constructing a trajectory optimization model by combining the visual constraint error vector with mechanical arm constraint conditions; and obtaining an optimal joint trajectory sequence by solving the trajectory optimization model and driving the multi-joint mechanical arm to perform an action. The application uses visual perception constraints in combination with a trajectory optimization model, and cooperates with visual closed-loop feedback to solve problems such as easy deviation and instability of the trajectory of the mechanical arm in a dynamic disturbance environment, and significantly improves the operation precision and robustness in a complex operation scenario.

Description

Technical Field

[0001] This invention relates to the field of robotic arm motion and trajectory planning, and in particular to a method, system, device and medium for high-precision trajectory optimization of multi-joint robotic arms using visual perception constraints. Background Technology

[0002] In existing research on motion control and trajectory planning for multi-joint robotic arms, visual perception has gradually become a key technology for improving control precision and enhancing environmental adaptability. However, current mainstream methods typically use visual information as an auxiliary positioning or target detection signal, and the participation of visual feedback in trajectory optimization is limited, making it difficult to form dynamic constraints on the robotic arm's operation. Especially in the presence of occlusion, minute target movements, or visual noise, it is difficult for the robotic arm trajectory to maintain stable and precise operation.

[0003] While some existing research attempts to construct end-to-end control frameworks based on depth vision, the lack of explicit and interpretable constraint models generally leads to problems such as trajectory instability and unclear optimization objectives. For example, in a scenario where a UAV carrying a flexible robotic arm is used to attach grounding wires on a power transmission line, the visual target may vibrate with the wind and the UAV's attitude may be disturbed. Existing vision-guided trajectory planning methods cannot provide stable and real-time optimization capabilities, leading to the risk of end-point trajectory deviation or even operation failure. Furthermore, due to the motion uncertainties brought about by the high-dimensionality and flexible structure of multi-joint robotic arms, existing trajectory optimization methods generally lack a unified approach to handling the consistency of robotic arm dynamics and task geometric constraints.

[0004] Therefore, in order to operate robotic arms in complex dynamic environments, it is urgent to develop an intelligent, efficient, and high-precision trajectory optimization method that accurately and deeply incorporates visual perception as a real-time constraint input into the trajectory optimization process and uniformly handles multiple constraints of visual perception and robotic arm dynamics. Summary of the Invention

[0005] In view of the aforementioned existing problems, the present invention is proposed.

[0006] Therefore, this invention provides a method, system, device, and medium for high-precision trajectory optimization of multi-joint robotic arms using visual perception constraints to solve the problems of not being able to construct dynamic constraints using visual perception, not being able to achieve stable high-precision trajectory generation under disturbed environments, and not being able to uniformly handle multiple constraints of visual perception and robotic arm dynamics.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints, comprising: The target features are acquired from real-time visual images and 3D reconstruction is performed to obtain the target's desired pose. Based on the joint angle information, the actual pose is obtained using a forward kinematics model. A visual constraint error vector is constructed by aligning the desired pose with the actual pose. A trajectory optimization model is constructed by combining the visual constraint error vector with the constraints of the robotic arm. By solving the trajectory optimization model, the optimal joint trajectory sequence is obtained and used to drive the multi-joint robotic arm to perform actions.

[0008] As a preferred embodiment of the high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints described in this invention, the method includes: acquiring target features from a real-time visual image and performing 3D reconstruction to obtain the target's desired pose, comprising: By performing target detection and feature extraction on real-time visual images, the two-dimensional pixel coordinates of the target in the image coordinate system are obtained; Obtain the depth information corresponding to the two-dimensional pixel coordinates; The two-dimensional pixel coordinates and the depth information are back-projected into three-dimensional space based on the camera intrinsic parameter matrix to obtain the three-dimensional point set of the target; By performing geometric analysis on the three-dimensional point set, the three-dimensional position vector and attitude rotation matrix of the target are determined, and the desired pose of the target is obtained.

[0009] As a preferred embodiment of the high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints described in this invention, the method comprises: constructing a visual constraint error vector by aligning the desired pose with the actual pose, including: Based on the attitude rotation matrix of the actual pose and the desired pose, the relative rotation relationship is converted into a three-dimensional vector form using the logarithmic mapping operator from Lie group to Lie algebra to obtain the attitude error. The position error is obtained based on the deviation between the actual pose and the desired pose in three dimensions. The position error and attitude error are combined to obtain the visual constraint error vector.

[0010] As a preferred embodiment of the high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints described in this invention, the trajectory optimization model is constructed by combining the visual constraint error vector with constraint terms, including: Construct a trajectory optimization objective function in the form of a weighted summation of multiple cost terms; The multiple cost terms include: visual constraint error vector, joint acceleration vector, and reference joint trajectory preset in the task planning stage. At the same time, the dynamic constraints of the robotic arm are added as hard constraints, which together with the trajectory optimization objective function constitute the trajectory optimization model.

[0011] As a preferred embodiment of the high-precision trajectory optimization method for multi-joint robotic arms utilizing visual perception constraints described in this invention, the robotic arm dynamic constraints include the dynamic balance relationship between the inertia matrix, Coriolis force and centrifugal force terms, gravity term, and joint driving torque vector.

[0012] As a preferred embodiment of the high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints as described in this invention, the optimal joint trajectory sequence is obtained by solving the trajectory optimization model, including: The model predictive control method is adopted, taking the current state of the robotic arm joints as the initial condition. Within a finite prediction time domain, the joint motion states at multiple discrete future moments are jointly optimized and solved to obtain the optimal joint position, joint velocity, and joint acceleration at each discrete moment within the prediction time window, which are then used as the optimal joint trajectory sequence.

[0013] As a preferred embodiment of the high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints described in this invention, the method of driving the multi-joint robotic arm to perform actions further includes: During the execution of the multi-joint robotic arm along the optimal joint trajectory sequence, the latest joint status is continuously read back and the actual pose of the end effector is updated. Simultaneously acquire real-time visual images, update the target's desired pose, and recalculate the visual constraint error vector; In response to the visual constraint error vector not meeting the preset accuracy threshold, the updated visual constraint error vector and the latest joint state are input into the trajectory optimization objective function to trigger the next round of online solution and control iteration.

[0014] Secondly, the present invention provides a high-precision trajectory optimization system for a multi-joint robotic arm utilizing visual perception constraints, comprising: The visual perception module is used to acquire target features from real-time visual images and perform 3D reconstruction to obtain the target's desired pose. The state acquisition module is used to obtain the actual pose based on the joint angle information and the positive kinematics model; A constraint construction module is used to construct a visual constraint error vector by aligning the desired pose with the actual pose. The model building module is used to construct a trajectory optimization model by combining the visual constraint error vector with the constraints of the robotic arm; The optimization execution module is used to obtain the optimal joint trajectory sequence by solving the trajectory optimization model and drive the multi-joint robotic arm to perform actions.

[0015] Thirdly, the present invention provides an electronic device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of a high-precision trajectory optimization method for a multi-joint robotic arm that utilizes visual perception constraints.

[0016] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints.

[0017] Compared with existing technologies, the beneficial effects of this invention are as follows: By introducing a logarithmic mapping operator from Lie groups to Lie algebras, complex nonlinear attitude differences are transformed into continuously differentiable mathematical vectors, making the visual perception results into constraints with clear physical meaning; by using visual errors as a cost term and combining them with hard constraints of robotic arm dynamics to construct a trajectory optimization model, the generated trajectory is ensured to achieve precise alignment visually; through a model predictive control and visual feedback closed-loop mechanism, the planned trajectory can be automatically updated as the target state changes. When faced with external disturbances such as UAV attitude disturbances or target drift due to wind, the system can instantly replan the trajectory, giving the robotic arm extremely strong anti-disturbance robustness and dynamic convergence capability. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the overall process of a high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints, as described in an embodiment of the present invention. Detailed Implementation

[0020] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0021] Example 1, referring to Figure 1 As an embodiment of the present invention, a high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints is provided, comprising: S100: Acquire the target features of the real-time visual image and perform 3D reconstruction to obtain the target's desired pose; S200: Based on the joint angle information, the actual pose is obtained using a positive kinematics model; S300: Construct a visual constraint error vector by aligning the desired pose with the actual pose; S400: Construct a trajectory optimization model by combining the visual constraint error vector with the robotic arm constraints; S500: By solving the trajectory optimization model, the optimal joint trajectory sequence is obtained and the multi-joint robotic arm is driven to perform actions.

[0022] Specifically, through steps S100-S500, this invention maps the target geometric state acquired by the front-end vision system into quantifiable mathematical constraints and embeds them into the robotic arm trajectory optimization model. This breaks the limitations of traditional trajectory planning, which relies heavily on offline calculations or static models, enabling the robotic arm's motion trajectory to not only follow its own dynamic boundaries but also be dynamically constrained by external visual information, thereby achieving high-precision, low-latency trajectory generation in complex and disturbed environments.

[0023] Example 2, refer to Figure 1 As an embodiment of the present invention, based on the above embodiment, a high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints is provided.

[0024] In this embodiment of the application, step S100, which involves acquiring target features from a real-time visual image and performing 3D reconstruction to obtain the target's desired pose, includes: Specifically, firstly, an industrial camera installed on the fuselage of a drone or the end of a multi-jointed robotic arm captures a sequence of environmental images near the power transmission line and grounding point at a set frame rate, serving as real-time visual images. The image then undergoes preprocessing, including noise filtering, lens distortion correction, and brightness and contrast normalization, to reduce the impact of imaging noise, lens distortion, and complex lighting conditions on the visual perception results. The preprocessed visual image is then input into a convolutional neural network for target detection and feature extraction, yielding the two-dimensional pixel coordinates of the target in the image coordinate system.

[0025] The convolutional neural network model specifically employs a target detection network based on the YOLO architecture. This network includes a backbone network for feature extraction, a neck network for multi-scale feature fusion, and a detection head for target category prediction and bounding box regression. The YOLO network was trained offline before system deployment. Training samples were derived from image data collected in real power transmission line operation scenarios or simulation environments. Training labels included target category annotations and the corresponding bounding box positions of the targets in the images. Supervised learning was employed during training, optimizing network parameters by minimizing target category prediction errors and bounding box regression errors, enabling the network to stably output the two-dimensional pixel coordinate information of targets in complex backgrounds and dynamic environments.

[0026] Simultaneously, a D435i depth camera is used to acquire depth information corresponding to two-dimensional pixel coordinates.

[0027] In one optional implementation, after obtaining the desired pose of the target in step S100, a Kalman filter algorithm can be introduced to smooth and predict the pose state. By introducing the Kalman filter algorithm, the kinematic model of the target is used to estimate and smooth the three-dimensional pose of consecutive frames, filtering out high-frequency observation noise. Thus, even when the visual signal is briefly lost or there are abnormal jumps, it can still provide a stable and continuous desired pose input to the system.

[0028] In another optional implementation, the desired pose of the target obtained in step S100 can also be reconstructed using multi-camera collaborative multi-view stereo vision technology through feature fusion. By distributing multiple industrial camera nodes at the robotic arm base and end effector, two-dimensional pixel coordinates from multiple perspectives are extracted synchronously. Epipolar geometric constraints are used for multi-view feature matching and triangulation ranging, and the three-dimensional coordinates of the target are fused and calculated. Thus, even when one side of the line of sight is blocked or a single depth sensor fails, the three-dimensional point set and desired pose of the target can still be calculated with high accuracy.

[0029] Furthermore, by using the PnP framework, target feature points in the image coordinate system are... Position vector mapped to a three-dimensional coordinate system Its three-dimensional spatial reconstruction process satisfies: in, These are the pixel coordinates in the image; D( are homogeneous coordinates;) The depth value of a pixel; This is the camera intrinsic parameter matrix; These are the coordinates of the corresponding three-dimensional point in space; Based on 3D spatial reconstruction, the output is a set of 3D points of the target. Subsequently, geometric analysis is performed on these points to determine the target's 3D position vector in a unified reference coordinate system. and attitude rotation matrix The combination of the two is the desired pose of the target. : In the formula, SE(3) is a three-dimensional special Euclidean group.

[0030] It should be noted that the S100 addresses the limitation of visual information serving only as an auxiliary positioning signal. By fusing deep learning with depth information, the two-dimensional image is transformed into a three-dimensional spatial desired pose with clear geometric meaning, providing a precise and real-time spatial reference benchmark for subsequent trajectory optimization.

[0031] In this embodiment of the application, step S200, which obtains the actual pose based on the joint angle information and the positive kinematics model, includes: Specifically, during system operation, the robotic arm controller collects joint angle and angular velocity information of each joint in real time through encoders at each joint, and calculates the actual pose of the end effector in a unified coordinate system based on the forward kinematics model of the robotic arm. : In the formula, This is the current Cartesian position vector of the robotic arm's end effector; The pose rotation matrix is ​​the actual pose, which is calculated by the robot arm controller within the real-time control cycle and serves as the spatial representation of the robot arm's current state.

[0032] In this embodiment of the application, step S300, which involves constructing a visual constraint error vector by aligning the desired pose with the actual pose, includes: Specifically, after obtaining the actual pose of the robotic arm's end effector and the desired pose of the target, the system aligns the two in the trajectory optimization module and constructs a visual perception constraint error vector. : In the formula, Position error, representing the deviation between the actual 3D position and the desired 3D position, is defined as: ; The attitude error is used to measure the relative rotational relationship between the actual pose and the desired pose, and it is expressed by a Lie group. To Lie algebra The construction of the logarithmic map is specifically represented as: In the formula, The logarithmic mapping operator represents the mapping from the rotation matrix to the Lie algebra vector space. It is used to convert nonlinear attitude differences into three-dimensional vector form, thereby obtaining a continuous, differentiable attitude error description with a clear geometric meaning.

[0033] It should be noted that by obtaining the true state in step S200 and combining it with the introduction of the logarithmic mapping operator from Lie groups to Lie algebras in step S300, the nonlinear singularity and discontinuity problems caused by directly calculating attitude errors in traditional Euclidean space are effectively avoided. This step successfully transforms the visual perception results into continuous and differentiable constraints that can be directly used by the optimization solver, fundamentally improving the trajectory oscillations and non-convergence phenomena commonly found in high-dimensional nonlinear systems.

[0034] In this embodiment of the application, step S400, which constructs a trajectory optimization model by combining the visual constraint error vector with constraint terms, includes: Specifically, a trajectory optimization model is constructed using time-domain discretization.

[0035] First, a trajectory optimization objective function in the form of a weighted sum of multiple cost terms is constructed. These multiple cost terms include: a visual constraint error vector, a joint acceleration vector, and a reference joint trajectory preset during the task planning phase.

[0036] Furthermore, the trajectory optimization objective function The specific expression is: In the formula, This represents the visual constraint error vector at the k-th discrete time step. This is the joint acceleration vector, used to suppress drastic acceleration changes to improve trajectory smoothness and executability; Let be the joint position vector of the robotic arm at discrete time points, denoted as , where n is the number of joint degrees of freedom of the robotic arm; This represents the reference joint trajectory given during the task planning phase, used to constrain the overall shape of the trajectory from deviating from the predetermined operation path; , , These are the weight matrices for the corresponding constraint terms, used to balance visual accuracy, motion smoothness, and task consistency.

[0037] Furthermore, the aforementioned objective function is not used to solve for a single control variable, but rather serves as a performance measure of the entire joint trajectory within the prediction time domain. Based on this, the trajectory optimization process must also satisfy the dynamic constraints of the robotic arm, which, together with the trajectory optimization objective function, constitute the trajectory optimization model.

[0038] Its constraint form is specifically expressed as follows: In the formula, Indicated by the forward kinematics model The calculated end effector pose; Here is the inertia matrix of the robotic arm; These are the Coriolis force and centrifugal force terms; This is the term related to gravity. This represents the joint driving torque vector.

[0039] The above constraints ensure that the generated trajectory is physically feasible and will not cause joint overruns or dynamic inconsistencies.

[0040] It should be noted that a unified multi-objective optimization framework was constructed by using visual error as a soft cost term and the robotic arm dynamics equations as hard constraints. This solves the problem that traditional trajectory planning methods cannot simultaneously consider control input constraints, attitude stability, and environmental safety constraints, ensuring that the generated trajectory has both high-precision visual alignment capability and meets the physical execution boundaries of the actual mechanical system.

[0041] In an optional implementation, the trajectory optimization model constructed in step S400 can also incorporate an obstacle avoidance safety distance penalty term based on the artificial potential field method. By introducing the obstacle avoidance penalty term, the shortest Euclidean distance between the three-dimensional bounding box of each link of the robotic arm and the point cloud of environmental obstacles is calculated in real time. When this distance is less than a preset safety threshold, the penalty value is increased exponentially in the objective function, thereby automatically planning a safe joint obstacle avoidance trajectory while guiding the end effector to approach the target.

[0042] In another optional implementation, the trajectory optimization model constructed in step S400 can also employ an adaptive weight adjustment mechanism to dynamically allocate the weight matrix of each cost term in the objective function. Based on the distance between the current actual position of the end effector and the desired target position, the weight coefficients of the visual constraint term and the acceleration smoothing term are dynamically adjusted, thereby significantly improving the local alignment accuracy and convergence speed during end effector approximation while ensuring global motion stability.

[0043] In this embodiment of the application, step S500 involves solving the trajectory optimization model to obtain the optimal joint trajectory sequence and driving the multi-joint robotic arm to perform actions, including: Specifically, the model predictive control method is adopted, using the current state of the robotic arm joints as the initial condition. Within a finite prediction time domain, the joint motion states at multiple discrete future moments are jointly optimized and solved to obtain the optimal joint position, joint velocity, and joint acceleration at each discrete moment within the prediction time window, which serve as the optimal joint trajectory sequence.

[0044] Furthermore, the optimal joint trajectory sequence obtained at this point constitutes the reference motion trajectory of the robotic arm in the next control time window, and serves as the basic input for the generation of subsequent control reference quantities.

[0045] In an optional implementation, the trajectory optimization model solved in step S500 can also incorporate a variable step size prediction mechanism to dynamically adjust the prediction time domain of the model predictive control. The prediction time window length of the MPC is dynamically scaled according to the norm of the current visual constraint error, thereby effectively reducing the real-time computing load on the airborne computer and improving the system's response frequency while ensuring control stability.

[0046] In another optional implementation, the trajectory optimization model solved in step S500 can also employ a low-level servo drive strategy combining feedforward torque compensation and impedance control. The optimal joint position and velocity solved by MPC are used as reference commands, while the joint torque calculated by the dynamic model is used as a feedforward quantity. Combined with an impedance control algorithm based on a six-dimensional force sensor at the end effector, the performance stiffness and damping of the robotic arm's end effector are dynamically adjusted. This achieves high-precision spatial trajectory following while providing compliant physical contact capability, ensuring the safety of contact operations.

[0047] Furthermore, to ensure that the trajectory continuously conforms to visual constraints during execution, a closed-loop visual feedback mechanism is constructed, including: During the execution of the multi-joint robotic arm along the optimal joint trajectory sequence, the latest joint status is continuously read back and the actual pose of the end effector is updated. Simultaneously acquire real-time visual images, update the target's desired pose, and recalculate the visual constraint error vector; In response to the visual constraint error vector not meeting the preset accuracy threshold, the updated visual constraint error vector and the latest joint state are input into the trajectory optimization objective function, triggering the next round of online solution and control iteration. This enables the system to converge stably even under conditions such as UAV attitude disturbances and slight target movement, achieving high-precision end-point trajectory following.

[0048] It should be noted that the model predictive control solution and visual closed-loop feedback mechanism in step S500 endow the system with extremely strong dynamic adaptive capabilities. When faced with external disturbances such as UAV attitude disturbances and target drift due to wind, this invention can instantly acquire the latest visual error and re-trigger optimization iterations, guiding the robotic arm end effector to dynamically follow the target, thereby ensuring absolute stability and high precision in complex and high-risk operation scenarios.

[0049] In summary, this invention, through the fusion of deep learning and depth information, accurately reconstructs discrete two-dimensional pixels into physically meaningful three-dimensional desired poses; by introducing a logarithmic mapping operator from Lie groups to Lie algebras, it transforms nonlinear spatial pose differences into continuously differentiable vectors that can be directly processed by the optimization solver, breaking down the underlying data barriers between visual perception and mathematical optimization; it constructs a trajectory optimization model, using visual errors as a soft cost term to guide the end effector to approach the target, and uses complex robotic arm dynamics equations as hard constraints to safeguard the physical safety boundary, solving the trajectory oscillation problem caused by multiple constraint conflicts; at the same time, it sets up a visual feedback closed-loop mechanism, giving the system extremely strong anti-disturbance capabilities.

[0050] Example 3 illustrates a schematic scheme for a high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints. It should be noted that the technical solution of this system for high-precision trajectory optimization of a multi-joint robotic arm utilizing visual perception constraints is based on the same concept as the aforementioned method for high-precision trajectory optimization of a multi-joint robotic arm utilizing visual perception constraints. Details not described in detail in this embodiment can be found in the description of the aforementioned method for high-precision trajectory optimization of a multi-joint robotic arm utilizing visual perception constraints.

[0051] This embodiment also provides a high-precision trajectory optimization system for a multi-joint robotic arm that utilizes visual perception constraints, including: The visual perception module is used to acquire target features from real-time visual images and perform 3D reconstruction to obtain the target's desired pose. The state acquisition module is used to obtain the actual pose based on the joint angle information and the positive kinematics model; A constraint construction module is used to construct a visual constraint error vector by aligning the desired pose with the actual pose. The model building module is used to construct a trajectory optimization model by combining the visual constraint error vector with the constraints of the robotic arm; The optimization execution module is used to obtain the optimal joint trajectory sequence by solving the trajectory optimization model and drive the multi-joint robotic arm to perform actions.

[0052] This embodiment also provides an electronic device suitable for high-precision trajectory optimization of multi-joint robotic arms using visual perception constraints, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the high-precision trajectory optimization method for multi-joint robotic arms using visual perception constraints as proposed in the above embodiment.

[0053] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints as proposed in the above embodiments.

[0054] The storage medium proposed in this embodiment and the method for optimizing the high-precision trajectory of a multi-joint robotic arm using visual perception constraints proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0055] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0056] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints, characterized in that, include: The target features are acquired from real-time visual images and 3D reconstruction is performed to obtain the target's desired pose. Based on the joint angle information, the actual pose is obtained using a forward kinematics model. A visual constraint error vector is constructed by aligning the desired pose with the actual pose. A trajectory optimization model is constructed by combining the visual constraint error vector with the constraints of the robotic arm. By solving the trajectory optimization model, the optimal joint trajectory sequence is obtained and used to drive the multi-joint robotic arm to perform actions.

2. The high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints as described in claim 1, characterized in that, Acquire target features from real-time visual images and perform 3D reconstruction to obtain the target's desired pose, including: By performing target detection and feature extraction on real-time visual images, the two-dimensional pixel coordinates of the target in the image coordinate system are obtained; Obtain the depth information corresponding to the two-dimensional pixel coordinates; The two-dimensional pixel coordinates and the depth information are back-projected into three-dimensional space based on the camera intrinsic parameter matrix to obtain the three-dimensional point set of the target; By performing geometric analysis on the three-dimensional point set, the three-dimensional position vector and attitude rotation matrix of the target are determined, and the desired pose of the target is obtained.

3. The high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints as described in claim 2, characterized in that, A visual constraint error vector is constructed by aligning the desired pose with the actual pose, including: Based on the attitude rotation matrix of the actual pose and the desired pose, the relative rotation relationship is converted into a three-dimensional vector form using the logarithmic mapping operator from Lie group to Lie algebra to obtain the attitude error. The position error is obtained based on the deviation between the actual pose and the desired pose in three dimensions. By combining the position error and the attitude error, a visual constraint error vector is obtained.

4. The high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints as described in claim 3, characterized in that, A trajectory optimization model is constructed by combining the visual constraint error vector with constraint terms, including: Construct a trajectory optimization objective function in the form of a weighted summation of multiple cost terms; The multiple cost terms include: visual constraint error vector, joint acceleration vector, and reference joint trajectory preset in the task planning stage. At the same time, the dynamic constraints of the robotic arm are added as hard constraints, which together with the trajectory optimization objective function constitute the trajectory optimization model.

5. The high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints as described in claim 4, characterized in that, The dynamic constraints of the robotic arm include the dynamic balance relationship between the inertia matrix, the Coriolis force and centrifugal force terms, the gravity term, and the joint driving torque vector.

6. The high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints as described in claim 5, characterized in that, By solving the trajectory optimization model, the optimal joint trajectory sequence is obtained, including: The model predictive control method is adopted, taking the current state of the robotic arm joints as the initial condition. Within a finite prediction time domain, the joint motion states at multiple discrete future moments are jointly optimized and solved to obtain the optimal joint position, joint velocity, and joint acceleration at each discrete moment within the prediction time window, which are then used as the optimal joint trajectory sequence.

7. The high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints as described in claim 6, characterized in that, The method for driving the multi-joint robotic arm to perform actions also includes: During the execution of the multi-joint robotic arm along the optimal joint trajectory sequence, the latest joint status is continuously read back and the actual pose of the end effector is updated. Simultaneously acquire real-time visual images, update the target's desired pose, and recalculate the visual constraint error vector; In response to the visual constraint error vector not meeting the preset accuracy threshold, the updated visual constraint error vector and the latest joint state are input into the trajectory optimization objective function to trigger the next round of online solution and control iteration.

8. A high-precision trajectory optimization system for a multi-joint robotic arm utilizing visual perception constraints, employing the method described in any one of claims 1-7, characterized in that, include: The visual perception module is used to acquire target features from real-time visual images and perform 3D reconstruction to obtain the target's desired pose. The state acquisition module is used to obtain the actual pose based on the joint angle information and the positive kinematics model; A constraint construction module is used to construct a visual constraint error vector by aligning the desired pose with the actual pose. The model building module is used to construct a trajectory optimization model by combining the visual constraint error vector with the constraints of the robotic arm; The optimization execution module is used to obtain the optimal joint trajectory sequence by solving the trajectory optimization model and drive the multi-joint robotic arm to perform actions.

9. An electronic device, characterized in that, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the high-precision trajectory optimization method for a multi-joint robotic arm using visual perception constraints as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It includes storing computer-executable instructions that, when executed by a processor, implement the steps of the high-precision trajectory optimization method for a multi-joint robotic arm utilizing visual perception constraints as described in any one of claims 1 to 7.

Images