Robot moving trajectory optimization control method and device, electronic equipment and medium

Abstract

The embodiment of the invention relates to the technical field of robot moving trajectory planning, and provides a robot moving trajectory optimization control method and device, electronic equipment and a medium, and the method comprises the steps: obtaining visual observation data and a natural language task instruction when a robot executes a current moving trajectory; joint reasoning is carried out on the visual observation data and the natural language task instruction, a discrete action sequence is generated, smooth optimization is carried out, and a first continuous running track is generated; the robot is controlled to be converted from the current moving track to execute the first continuous moving track, and the visual observation data and the natural language task instruction of the robot executing the first continuous moving track continue to be obtained; and smooth transition processing is carried out on the newly generated second continuous moving track and the first continuous moving track, and the robot is controlled to be transited to the second continuous moving track from the first continuous moving track. Therefore, smooth trajectory generation meeting dynamic constraints, continuous trajectory transition under dynamic instructions and safety control in an abnormal state are realized.

Description

Technical Field

[0001] This invention relates to the field of robot trajectory planning technology, and in particular to a robot trajectory optimization control method, device, electronic device and medium. Background Technology

[0002] In recent years, with the development of multimodal large-scale models, Vision-Language-Action (VLA) models have received widespread attention in robot task planning. These models take visual images and natural language commands as input and directly output discrete robot action sequences (such as grasping points and placement point poses) through an end-to-end network. Their advantage lies in their ability to understand complex environmental semantics and quickly provide high-level decisions; typical applications include human-robot collaboration and autonomous assembly. However, the action sequences output by VLA models only contain spatial position or posture information, lacking consideration for physical constraints such as velocity, acceleration, and joint torques during robot movement. Directly executing such discrete actions will lead to trajectory jitter, joint impact, and long-term operation can easily cause motor overheating, mechanical wear, and even action failure during cold starts due to instantaneous torque exceeding limits.

[0003] MoveIt is a widely adopted open-source motion planning framework in the robotics field. It includes planners such as OMPL (Open Motion Planning Library) and can fit discrete target points into smooth, continuous trajectories that satisfy physical constraints such as robot joint velocities, accelerations, and torques. MoveIt supports Cartesian path planning, joint space interpolation, and dynamic optimization, enabling smooth, collision-free trajectory generation given fixed target points. However, MoveIt relies on static input. When the environment changes or task instructions are dynamically updated, current technologies cannot achieve seamless transitions between old and new trajectories. Common issues include motion interruptions or sudden velocity changes at non-zero velocities, leading to decreased task execution efficiency, loss of positioning accuracy, and potential safety hazards.

[0004] The existing "serial" architecture of VLA and MoveIt lacks a real-time monitoring and dynamic feedback mechanism for robot physical parameters (motor temperature, joint torque, current, etc.). Once the temperature or torque exceeds the limit, the system cannot adjust the trajectory online and can only passively stop after the anomaly occurs, posing safety risks such as motor burnout and collisions. Summary of the Invention

[0005] This invention provides a method, device, electronic device, and medium for optimizing and controlling robot trajectory, which addresses the shortcomings of existing technologies that lack real-time monitoring and dynamic feedback mechanisms for robot physical parameters, cannot dynamically adjust the trajectory when abnormalities occur, and pose safety hazards. It achieves smooth trajectory generation that meets dynamic constraints, continuous trajectory transition under dynamic commands, and safety control under abnormal conditions.

[0006] This invention provides a method for optimizing and controlling the trajectory of a robot, comprising: Acquire visual observation data and natural language task instructions as the robot executes its current trajectory; Joint reasoning is performed on the visual observation data and the natural language task instructions to generate a discrete action sequence; The discrete motion sequence is smoothed and optimized to generate a first continuous running trajectory that satisfies the physical constraints of the robot's motion process, wherein the physical constraints include joint velocity, joint acceleration and joint torque constraints. The robot is controlled to switch from the current running trajectory to execute the first continuous running trajectory; During the process of the robot executing the first continuous running trajectory, the step of acquiring the visual observation data and natural language task instructions when the robot is executing the current running trajectory continues to be executed; The newly generated second continuous running trajectory is smoothly transitioned to the first continuous running trajectory obtained in the previous round, and the robot is controlled to transition from the first continuous running trajectory to the second continuous running trajectory.

[0007] In one possible implementation, the method further includes: Extract the visual features and language features corresponding to the visual observation data and the natural language task instructions, respectively. The visual and linguistic features are input into a pre-trained visual-linguistic action model to generate discrete action sequences.

[0008] In one possible implementation, the method further includes: The discrete motion sequence is input into the trajectory planning module, which generates a geometrically continuous path based on the robot joint motion mechanics model. The geometrically continuous path is smoothed by fifth-order polynomial spline interpolation, and the physical constraints are explicitly applied during the optimization process, and the smoothed path is parameterized over time. The first continuous running trajectory is generated based on the smoothed and optimized geometrically continuous path.

[0009] In one possible implementation, the method further includes: Obtain the end velocity of the robot executing the current running trajectory and the initial target velocity of the smoothed and optimized geometrically continuous path; Based on the terminal velocity and the initial target velocity, a linear acceleration interpolation algorithm is used to generate the transition segment trajectory, wherein the velocity change rate of the transition segment trajectory is less than a preset threshold. The current running trajectory, the transition segment running trajectory, and the smoothed and optimized geometrically continuous path are subjected to global time parameterization to generate the first continuous running trajectory.

[0010] In one possible implementation, the method further includes: While the robot is executing its current trajectory, the robot's physical operating parameters are collected in real time. Anomaly detection is performed on the physical operating parameters; The robot is controlled based on the corresponding exception handling scheme triggered according to the exception level.

[0011] In one possible implementation, the method further includes: When the anomaly level is the first level, the local trajectory corresponding to the abnormal physical operating parameters in the current running trajectory is replanned, and the robot is controlled to execute the replanned trajectory. When the anomaly level is the second level, emergency braking is immediately triggered, wherein the severity of the anomaly at the first level is less than that at the second level.

[0012] The present invention also provides a robot trajectory optimization control device, comprising the following modules: The acquisition module is used to acquire visual observation data and natural language task instructions when the robot is executing the current running trajectory; The reasoning module is used to perform joint reasoning on the visual observation data and the natural language task instructions to generate discrete action sequences; An optimization module is used to smooth the discrete motion sequence and generate a first continuous running trajectory that satisfies the physical constraints in the robot's motion process, wherein the physical constraints include joint velocity, joint acceleration and joint torque constraints. The control module is used to control the robot to switch from the current running trajectory to execute the first continuous running trajectory; The acquisition module is further configured to continue executing the step of acquiring the visual observation data and natural language task instructions of the robot when it executes the current running trajectory during the process of the robot executing the first continuous running trajectory; The control module is also used to perform a smooth transition between the newly generated second continuous running trajectory and the first continuous running trajectory obtained in the previous round, and control the robot to transition from the first continuous running trajectory to the second continuous running trajectory.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the robot trajectory optimization control method as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the robot trajectory optimization control method as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the robot trajectory optimization control method as described above.

[0016] The robot trajectory optimization control method, device, electronic device, and medium provided by this invention acquire visual observation data and natural language task instructions when the robot executes the current trajectory; perform joint reasoning on the visual observation data and the natural language task instructions to generate a discrete action sequence; perform smooth optimization on the discrete action sequence to generate a first continuous trajectory that satisfies the physical constraints of the robot's motion process, wherein the physical constraints include joint velocity, joint acceleration, and joint torque constraints; control the robot to switch from the current trajectory to execute the first continuous trajectory; during the execution of the first continuous trajectory, continue to execute the step of acquiring visual observation data and natural language task instructions when the robot executes the current trajectory; and perform a smooth transition process between the newly generated second continuous trajectory and the first continuous trajectory obtained in the previous round to control the robot to transition from the first continuous trajectory to the second continuous trajectory. Compared to existing technologies that lack real-time monitoring and dynamic feedback mechanisms for robot physical parameters and cannot dynamically adjust trajectories when anomalies occur, posing safety hazards, this solution integrates the high-level decision-making capabilities of VLA with the low-level trajectory optimization capabilities of MoveIt to construct a closed-loop control system of "perception-planning-execution-feedback". This system enables smooth trajectory generation that meets dynamic constraints, continuous trajectory transition under dynamic commands, and safety control under abnormal conditions. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a system architecture diagram of the robot trajectory optimization control method provided by the present invention.

[0019] Figure 2 This is one of the flowcharts illustrating the robot trajectory optimization and control method provided by the present invention.

[0020] Figure 3 This is the second flowchart of the robot trajectory optimization control method provided by the present invention.

[0021] Figure 4 This is a schematic diagram of the asynchronous reasoning and trajectory grafting principle provided by the present invention.

[0022] Figure 5 This is a schematic diagram of the robot trajectory optimization and control device provided by the present invention.

[0023] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] To facilitate understanding of the embodiments of the present invention, further explanations and descriptions will be provided below with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0026] Figure 1 This is a system architecture diagram of the robot trajectory optimization control method provided by the present invention, as shown below. Figure 1 As shown, this system includes the following five core modules: VLA Model Module: Visual observation data and natural language task instructions for the robot to execute the current running trajectory, outputting discrete action sequences (including absolute position or attitude points). MoveIt trajectory planning module: Fits and optimizes discrete action sequences to generate smooth running trajectories that meet physical constraints; Asynchronous inference module: Drives the VLA model to update the action sequence in real time while the robot executes its trajectory; Trajectory grafting module: Seamlessly connects new action sequences with the current running trajectory to ensure motion continuity; Execution and Feedback Module: Drives the robot to execute its running trajectory and monitors physical parameters (such as motor temperature and torque) in real time.

[0027] Figure 2 This is one of the flowcharts illustrating the robot trajectory optimization and control method provided by the present invention, such as... Figure 2 As shown, the method includes the following: S21. Obtain visual observation data and natural language task instructions when the robot is executing the current running trajectory.

[0028] In this embodiment of the invention, when the robot executes its current trajectory, the system acquires environmental images (visual observation data) in real time through a visual sensor (such as an RGB-D camera), and simultaneously receives natural language task instructions (such as "insert part A into station B") from the human-machine interface or task management system. This data serves as dynamic input to the VLA model for subsequent inference. It should be noted that this process is performed in parallel with the execution of the robot's current trajectory to ensure immediate response to environmental changes or instruction updates.

[0029] S22. Perform joint reasoning on the visual observation data and the natural language task instructions to generate a discrete action sequence.

[0030] The VLA model extracts image features (such as part position and obstacle information) through a visual encoder, combines them with semantic parsing of text instructions (such as grasping action type and target workstation), and outputs discrete action sequences (such as critical path point coordinates or end-effector pose). For example, if the instruction is updated to "grab the part after avoiding obstacles", VLA will generate a new grasping point sequence (Pn+1, Pn+2,...), but physical constraints have not yet been considered.

[0031] S23. The discrete action sequence is smoothed and optimized to generate a first continuous running trajectory that satisfies the physical constraints of the robot's motion process.

[0032] First, the discrete points output by VLA (e.g., P1→P2→...→Pn) are used to generate an initial continuous trajectory through Cartesian path planning. Then, fifth-order polynomial spline interpolation is used to optimize the smoothness of the trajectory, while simultaneously satisfying constraints on the robot joints' maximum speed (e.g., ≤2m / s), acceleration (e.g., ≤1m / s²), and torque (e.g., ≤50N·m). The optimization objective is to minimize acceleration (jerk) to avoid mechanical shock. For example, if the acceleration of a joint exceeds the limit, the algorithm will reallocate time parameters or adjust the path curvature.

[0033] S24. Control the robot to switch from the current running trajectory to execute the first continuous running trajectory.

[0034] In this embodiment of the invention, it is necessary to ensure the velocity continuity between the current trajectory and the new trajectory: by extracting the end velocity of the current trajectory (v_current) and the starting velocity of the new trajectory (v_target), a transition segment is generated using linear acceleration interpolation (LAI) to make the velocity change rate ≤ 0.5m / s³ (configurable threshold).

[0035] For example, if the robot is currently moving at a speed of 0.3 m / s, and the starting speed of the new trajectory is 0.8 m / s, LAI will calculate a uniformly accelerated transition trajectory to avoid jitter caused by sudden speed changes.

[0036] S25. During the process of the robot executing the first continuous running trajectory, continue to execute the step of obtaining the visual observation data and natural language task instructions when the robot executes the current running trajectory.

[0037] While the robot executes the first continuous trajectory, the VLA model continues the S21-S22 process at a fixed frequency (e.g., 10Hz), monitoring environmental changes (e.g., part position shifts, new obstacles appearing) or instruction updates (e.g., "change to station C"). This process runs in parallel with the trajectory execution, ensuring dynamic adaptability, but it can generate a new discrete action sequence (i.e., the prototype of the second continuous trajectory) without waiting for the current trajectory to complete.

[0038] S26. Perform a smooth transition process between the newly generated second continuous running trajectory and the first continuous running trajectory obtained in the previous round, and control the robot to transition from the first continuous running trajectory to the second continuous running trajectory.

[0039] When the second continuous running trajectory (based on the latest observations and command generation) is ready, the system repeats the optimization and switching process of S23-S24, but resolves conflicts through more refined transition processing: Speed ​​matching: If the speed difference between the starting point of the second continuous running trajectory and the end point of the first continuous running trajectory is too large (e.g., v_target=1.2m / s vs v_current=0.1m / s), the OMPL planner is enabled to re-optimize the time parameterization of the second trajectory to ensure overall smoothness.

[0040] Dynamic replanning: If abnormal physical parameters are detected (such as motor temperature > 80℃), the system will trigger the Elastic Band algorithm to locally adjust the second continuous running trajectory, reduce speed or avoid high-risk areas.

[0041] Emergency braking: If the joint torque exceeds the limit (e.g., >120% of the maximum threshold), the transition is immediately interrupted and a safety brake is executed to avoid hardware damage.

[0042] For example, if the target part is found to have shifted during the execution of the first continuous running trajectory, the gripping point needs to be corrected in the second continuous running trajectory. At this time, the system will complete the transition within 0.1s to ensure that the robot switches to the new trajectory without interruption and the speed fluctuation is ≤0.5m / s².

[0043] The robot trajectory optimization and control method provided by this invention acquires visual observation data and natural language task instructions when the robot executes the current trajectory; performs joint reasoning on the visual observation data and the natural language task instructions to generate a discrete action sequence; performs smooth optimization on the discrete action sequence to generate a first continuous trajectory that satisfies the physical constraints of the robot's motion process, wherein the physical constraints include joint velocity, joint acceleration, and joint torque constraints; controls the robot to switch from the current trajectory to execute the first continuous trajectory; during the execution of the first continuous trajectory, the step of acquiring the visual observation data and natural language task instructions when the robot executes the current trajectory continues to be executed; and performs a smooth transition processing between the newly generated second continuous trajectory and the first continuous trajectory obtained in the previous round, controlling the robot to transition from the first continuous trajectory to the second continuous trajectory. Compared to existing technologies that lack real-time monitoring and dynamic feedback mechanisms for robot physical parameters and cannot dynamically adjust trajectories when anomalies occur, posing safety hazards, this method integrates the high-level decision-making capabilities of VLA with the low-level trajectory optimization capabilities of MoveIt to construct a closed-loop control system of "perception-planning-execution-feedback". This system enables smooth trajectory generation that meets dynamic constraints, continuous trajectory transition under dynamic commands, and safety control under abnormal conditions.

[0044] Figure 3 This is one of the flowcharts illustrating the robot trajectory optimization and control method provided by the present invention, such as... Figure 3As shown, the method includes the following: S31. Extract the visual features and language features corresponding to the visual observation data and the natural language task instructions, respectively.

[0045] The embodiments of the present invention will be explained in conjunction with the asynchronous reasoning and trajectory grafting principle shown in Figure 2, as follows: Figure 2 As shown, in step 1: when the robot executes the current running trajectory, the asynchronous inference module drives the VLA model to generate a new action sequence (Pn+1, Pn+2, ...) based on new observations (such as environmental changes) and instructions. Step 2: The trajectory grafting module extracts the end velocity v_current of the current trajectory and the target velocity v_target of the starting point of the new sequence; Step 3: Calculate the transition trajectory using the Linear Acceleration Interpolation (LAI) algorithm to ensure that the rate of change of velocity at the junction is ≤0.5m / s³ (threshold is configurable), thus achieving a continuous transition between the old and new trajectories; Step 4: Optimize the updated complete trajectory with time parameters using MoveIt's OMPL planner to ensure overall smoothness.

[0046] Specifically, the images captured by the RGB-D camera are first processed by an encoder (the visual sub-network in the VLA model) to extract high-dimensional visual feature vectors.

[0047] Natural language instructions are processed by a BERT-like text encoder to obtain language feature vectors, which are then mapped to a dimension aligned with the high-dimensional visual feature vectors.

[0048] S32. Input the visual features and language features into the pre-trained visual-language action model to generate discrete action sequences.

[0049] High-dimensional visual feature vectors and language feature vectors can be fused using an attention mechanism to obtain a joint representation. This joint representation is then fed into the action decoder of a pre-trained visual-language action model, directly outputting a discrete action sequence.

[0050] S33. Input the discrete motion sequence into the trajectory planning module, and generate a geometrically continuous path based on the robot joint motion mechanics model through the trajectory planning module.

[0051] The trajectory planning module, or MoveIt framework, first performs inverse kinematics calculations to transform Cartesian keypoints into joint space waypoints. Then, it calls OMPL's PRM / RRT* algorithms to generate a "geometrically continuous path" in joint space or Cartesian space—a series of collision-free, densely packed pose points within joint constraints. This path only guarantees spatial continuity; the temporal dimension has not yet been processed.

[0052] S34. The geometrically continuous path is smoothed by fifth-order polynomial spline interpolation, and the physical constraints are explicitly applied during the optimization process, and the smoothed path is time-parameterized.

[0053] For each adjacent pair of segments on the geometric path, construct the joint position-velocity-acceleration curve using a fifth-order polynomial.

[0054] Constraints: Joint velocity ≤ 2 m / s (or rad / s) Joint acceleration ≤ 1 m / s² (or rad / s²) Joint torque ≤ 50 N·m The "time-parameterized trajectory" that satisfies all constraints is obtained by iterative time scaling.

[0055] S35. Obtain the end velocity of the robot executing the current running trajectory and the starting target velocity of the smoothed and optimized geometrically continuous path.

[0056] Obtain the end joint velocity of the current trajectory, and extract the target velocity at the start time from the new trajectory (i.e., the smoothed and optimized geometrically continuous path).

[0057] S36. Based on the end velocity and the initial target velocity, a linear acceleration interpolation algorithm is used to generate the transition segment trajectory.

[0058] Among them, the rate of change of speed of the transition section trajectory is less than the preset threshold.

[0059] Linear acceleration interpolation algorithm: Construct a uniform acceleration / deceleration segment (transition segment) between the joint velocity at the end of the current trajectory and the target velocity at the start of the new trajectory, ensuring that the rate of change of acceleration (jerk) ≤ 0.5 m / s³ (configurable). This transition segment ensures no sudden velocity changes at the switching point.

[0060] S37. Perform global time parameterization processing on the current running trajectory, the transition segment running trajectory, and the smoothed and optimized geometric continuous path to generate the first continuous running trajectory.

[0061] The three segments are spliced ​​together: the remaining part of the current trajectory (keeping the original time parameters); the transition segment trajectory (time length Δt); and the newly generated smooth trajectory (already parameterized by time).

[0062] Use MoveIt's Time-Optimal Trajectory Generation (TOTG) or TOPP-RA to perform a global time reparameterization on the entire trajectory, ensuring that the entire trajectory still satisfies all velocity, acceleration, and torque constraints in the shortest possible time. The output is the final "first continuous running trajectory," which can be directly sent to the robot controller for execution.

[0063] It should be noted that while the robot is executing the current running trajectory, the robot's physical operating parameters can also be collected in real time; anomaly detection can be performed on the physical operating parameters to determine whether there is an anomaly and the anomaly level; based on the anomaly level, the corresponding anomaly handling scheme can be triggered, and the robot can be controlled based on the anomaly handling scheme.

[0064] When the anomaly level is Level 1, the local trajectory corresponding to the abnormal physical operating parameters in the current running trajectory is replanned, and the robot is controlled to execute the replanned trajectory; when the anomaly level is Level 2, emergency braking is triggered immediately, wherein the severity of the anomaly in Level 1 is less than that in Level 2.

[0065] For example, the execution and feedback module collects parameters such as motor temperature (T) and joint torque (M) in real time through sensors (such as temperature sensors and torque sensors); when it detects that T > 80℃ (threshold is configurable) or M > maximum torque threshold, it triggers abnormal handling.

[0066] Non-urgent anomalies (e.g., T=85℃): MoveIt's Elastic Band algorithm is called to perform local trajectory replanning, reducing running speed; Emergency abnormality (e.g., M > 120% of maximum torque): Immediately trigger emergency braking to stop the robot's movement.

[0067] This invention addresses the problem of VLA models neglecting dynamic characteristics by adapting discrete actions generated by MoveIt to physical constraints; it achieves continuous trajectory updates under dynamic commands through velocity matching and interpolation transition algorithms, avoiding non-zero velocity interruptions; it links sensor monitoring with trajectory replanning to achieve a full-process response of "anomaly detection - dynamic adjustment - safe braking"; and it ensures trajectory smoothness and reduces mechanical wear by minimizing acceleration and time parameterization adjustments.

[0068] Compared with the prior art, the embodiments of the present invention have the following advantages: 1. Improved motion stability: Through dynamic constraint optimization, trajectory jitter amplitude is reduced by more than 60%, motor overheating rate is reduced by 80%, and cold start success rate is increased from 85% to 99%; 2. Enhanced dynamic adaptability: The asynchronous trajectory grafting mechanism shortens the trajectory transition time during instruction updates to less than 0.1s, with no obvious impact (speed change ≤0.5m / s²). 3. Enhanced safety: The real-time monitoring and anomaly handling mechanism can respond to over-limit conditions within 10ms, preventing motor damage or collision accidents; 4. Wide applicability: In industrial assembly (such as precision parts docking) and service robot (such as human-robot collaborative handling) scenarios, the task completion accuracy is improved to ±0.1mm, and the efficiency is improved by 30%.

[0069] The robot trajectory optimization control device provided by the present invention is described below. The robot trajectory optimization control device described below and the robot trajectory optimization control method described above can be referred to in correspondence.

[0070] Figure 5 This is a schematic diagram of the robot trajectory optimization and control device provided by the present invention, specifically including: The acquisition module 501 is used to acquire visual observation data and natural language task instructions of the robot while it is executing the current running trajectory. For detailed explanations, please refer to the relevant descriptions in the above method embodiments; they will not be repeated here.

[0071] The inference module 502 is used to perform joint inference on the visual observation data and the natural language task instructions to generate a discrete action sequence. For detailed explanations, please refer to the relevant descriptions in the above method embodiments; they will not be repeated here.

[0072] The optimization module 503 is used to smooth and optimize the discrete motion sequence to generate a first continuous running trajectory that satisfies the physical constraints during the robot's motion process. These physical constraints include joint velocity, joint acceleration, and joint torque constraints. For detailed explanations, please refer to the relevant descriptions in the above method embodiments; they will not be repeated here.

[0073] The control module 504 is used to control the robot to switch from the current running trajectory to execute the first continuous running trajectory. For detailed explanation, please refer to the relevant descriptions in the above method embodiments; they will not be repeated here.

[0074] The acquisition module 501 is further configured to continue executing the step of acquiring the visual observation data and natural language task instructions of the robot while it is executing the first continuous running trajectory, during the process of the robot executing the first continuous running trajectory. For detailed explanation, please refer to the relevant descriptions in the above method embodiments, which will not be repeated here.

[0075] The control module 504 is further configured to perform a smooth transition processing between the newly generated second continuous running trajectory and the first continuous running trajectory obtained in the previous round, controlling the robot to transition from the first continuous running trajectory to the second continuous running trajectory. For detailed explanations, please refer to the relevant descriptions in the above method embodiments, which will not be repeated here.

[0076] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communications bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other through the communications bus 640. The processor 610 can call logic instructions in the memory 630 to execute a robot trajectory optimization control method. This method includes: acquiring visual observation data and natural language task instructions when the robot is executing the current trajectory; performing joint reasoning on the visual observation data and the natural language task instructions to generate a discrete action sequence; performing smooth optimization on the discrete action sequence to generate a first continuous trajectory that satisfies the physical constraints of the robot's motion process, wherein the physical constraints include joint velocity, joint acceleration, and joint torque constraints; controlling the robot to switch from the current trajectory to execute the first continuous trajectory; continuing to execute the step of acquiring the visual observation data and natural language task instructions when the robot is executing the first continuous trajectory during the execution of the first continuous trajectory; and performing a smooth transition between the newly generated second continuous trajectory and the first continuous trajectory obtained in the previous round, controlling the robot to transition from the first continuous trajectory to the second continuous trajectory.

[0077] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0078] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the robot trajectory optimization control method provided by the above methods. The method includes: acquiring visual observation data and natural language task instructions when the robot is executing the current trajectory; performing joint reasoning on the visual observation data and the natural language task instructions to generate a discrete action sequence; performing smooth optimization on the discrete action sequence to generate a first continuous trajectory that satisfies the physical constraints of the robot's motion process, wherein the physical constraints include joint velocity, joint acceleration, and joint torque constraints; controlling the robot to switch from the current trajectory to execute the first continuous trajectory; continuing to execute the step of acquiring visual observation data and natural language task instructions when the robot is executing the first continuous trajectory during the execution of the first continuous trajectory; and performing a smooth transition processing between the newly generated second continuous trajectory and the first continuous trajectory obtained in the previous round to control the robot to transition from the first continuous trajectory to the second continuous trajectory.

[0079] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a robot trajectory optimization control method provided by the above methods. The method includes: acquiring visual observation data and natural language task instructions when the robot is executing the current trajectory; performing joint reasoning on the visual observation data and the natural language task instructions to generate a discrete action sequence; performing smooth optimization on the discrete action sequence to generate a first continuous trajectory that satisfies the physical constraints of the robot's motion process, wherein the physical constraints include joint velocity, joint acceleration, and joint torque constraints; controlling the robot to switch from the current trajectory to execute the first continuous trajectory; continuing to execute the step of acquiring visual observation data and natural language task instructions when the robot is executing the first continuous trajectory during the execution of the first continuous trajectory; and performing a smooth transition processing between the newly generated second continuous trajectory and the first continuous trajectory obtained in the previous round to control the robot to transition from the first continuous trajectory to the second continuous trajectory.

[0080] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0081] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for optimizing and controlling the trajectory of a robot, characterized in that, include: Acquire visual observation data and natural language task instructions as the robot executes its current trajectory; Joint reasoning is performed on the visual observation data and the natural language task instructions to generate a discrete action sequence; The discrete motion sequence is smoothed and optimized to generate a first continuous running trajectory that satisfies the physical constraints of the robot's motion process, wherein the physical constraints include joint velocity, joint acceleration and joint torque constraints. The robot is controlled to switch from the current running trajectory to execute the first continuous running trajectory; During the process of the robot executing the first continuous running trajectory, the step of acquiring the visual observation data and natural language task instructions when the robot is executing the current running trajectory continues to be executed; The newly generated second continuous running trajectory is smoothly transitioned to the first continuous running trajectory obtained in the previous round, and the robot is controlled to transition from the first continuous running trajectory to the second continuous running trajectory.

2. The method according to claim 1, characterized in that, The step of jointly reasoning about the visual observation data and the natural language task instructions to generate a discrete action sequence includes: Extract the visual features and language features corresponding to the visual observation data and the natural language task instructions, respectively. The visual and linguistic features are input into a pre-trained visual-linguistic action model to generate discrete action sequences.

3. The method according to claim 1 or 2, characterized in that, The step of smoothing and optimizing the discrete action sequence to generate a first continuous running trajectory that satisfies the physical constraints of the robot's motion process includes: The discrete motion sequence is input into the trajectory planning module, which generates a geometrically continuous path based on the robot joint motion mechanics model. The geometrically continuous path is smoothed by fifth-order polynomial spline interpolation, and the physical constraints are explicitly applied during the optimization process, and the smoothed path is parameterized over time. The first continuous running trajectory is generated based on the smoothed and optimized geometrically continuous path.

4. The method according to claim 3, characterized in that, The generation of the first continuous running trajectory based on the smoothed and optimized geometrically continuous path includes: Obtain the end velocity of the robot executing the current running trajectory and the initial target velocity of the smoothed and optimized geometrically continuous path; Based on the terminal velocity and the initial target velocity, a linear acceleration interpolation algorithm is used to generate the transition segment trajectory, wherein the velocity change rate of the transition segment trajectory is less than a preset threshold. The current running trajectory, the transition segment running trajectory, and the smoothed and optimized geometrically continuous path are subjected to global time parameterization to generate the first continuous running trajectory.

5. The method according to claim 1, characterized in that, The method further includes: While the robot is executing its current trajectory, the robot's physical operating parameters are collected in real time. Anomaly detection is performed on the physical operating parameters; The robot is controlled based on the corresponding exception handling scheme triggered according to the exception level.

6. The method according to claim 5, characterized in that, The method of triggering a corresponding exception handling scheme based on the exception level, and controlling the robot based on the exception handling scheme, includes: When the anomaly level is the first level, the local trajectory corresponding to the abnormal physical operating parameters in the current running trajectory is replanned, and the robot is controlled to execute the replanned trajectory. When the anomaly level is the second level, emergency braking is immediately triggered, wherein the severity of the anomaly at the first level is less than that at the second level.

7. A method for optimizing and controlling the trajectory of a robot, characterized in that, include: The acquisition module is used to acquire visual observation data and natural language task instructions when the robot is executing the current running trajectory; The reasoning module is used to perform joint reasoning on the visual observation data and the natural language task instructions to generate discrete action sequences; An optimization module is used to smooth the discrete motion sequence and generate a first continuous running trajectory that satisfies the physical constraints in the robot's motion process, wherein the physical constraints include joint velocity, joint acceleration and joint torque constraints. The control module is used to control the robot to switch from the current running trajectory to execute the first continuous running trajectory; The acquisition module is further configured to continue executing the step of acquiring the visual observation data and natural language task instructions of the robot when it is executing the first continuous running trajectory during the process of the robot executing the first continuous running trajectory; The control module is also used to perform a smooth transition between the newly generated second continuous running trajectory and the first continuous running trajectory obtained in the previous round, and control the robot to transition from the first continuous running trajectory to the second continuous running trajectory.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the robot trajectory optimization control method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the robot trajectory optimization control method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the robot trajectory optimization control method as described in any one of claims 1 to 6.

Images