Smooth processing method and device of action block, electronic equipment and storage medium

Abstract

The invention provides a smoothing processing method and device for an action block, electronic equipment and a storage medium. The method comprises the steps that S1, a current action block is segmented and divided into necessary action fragments; s2, constructing an expression of an interpolation function of an interpolation track and an interpolation curve expression of a multi-order kinematics parameter according to the inevitable action segments; s3, performing smooth interpolation, and determining an interpolation trajectory curve and a plurality of kinematics parameter curves; s4, constructing a target optimization function according to the optimization target and the motion constraint condition; s5, solving the least square problem with constraints by adopting a partial derivative numerical approximate calculation mode, carrying out convergence judgment, if convergence is carried out, executing S6, and if not, updating the optimization variables, and returning to execute S3; and S6, outputting the optimal moving track. Therefore, by means of the scheme, the problems of motion sudden change and jitter between action blocks output by the action inference model can be effectively solved, and the requirements of the action track on the aspects of continuity, smoothness and physical performability are met.

Description

Technical Field

[0001] This application relates to the field of robotics, and in particular to a method, apparatus, electronic device, and storage medium for smoothing motion blocks. Background Technology

[0002] In the current field of humanoid robot and intelligent agent control, large-scale models based on Vision-Language-Action (VLA) are gradually becoming an important bridge from high-level semantics to low-level action control. However, existing VLA models generally suffer from discontinuities and lack of smoothness between action chunks when inferring and generating continuous action sequences. This is mainly due to the fact that VLA models are designed to predict discrete semantic instructions or temporal segments, and their output actions are often generated in blocks. While each action block is relatively consistent internally, there are abrupt changes in velocity, acceleration, or posture between blocks, leading to jitter, instability, or even task failure during actual robot execution.

[0003] Existing methods primarily focus on directly smoothing action sequences within a single action block or between adjacent blocks, lacking overall constraints on global motion characteristics. Since action blocks inferred from VLA models are semantically independent, significant abrupt changes in posture or velocity often occur between their final actions and the starting actions of the next action block. Even when traditional smoothing algorithms interpolate or filter these points locally, noticeable kinematic discontinuities still arise. This is because conventional interpolation methods typically require the trajectory to strictly pass through each discrete point, thus forcing the introduction of high-frequency changes or unnatural dynamic transitions when abrupt changes occur. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method, apparatus, electronic device and storage medium for smoothing action blocks. By flexibly dividing the action blocks and performing time optimization and kinematic constraint smooth interpolation on the global necessary point sequence containing the end point of the previous action block, a continuous and physically feasible interpolation trajectory on multiple kinematic parameters is generated, thereby fundamentally eliminating the motion abrupt change and jitter problem between action blocks output by the action inference model.

[0005] This application provides a method for smoothing action blocks, the smoothing method including: S1. Divide the current action block (excluding the first action block) output by the action reasoning model into optional action segments and mandatory action segments; wherein, the optional action segments contain the first M actions in the current action block, and the mandatory action segments contain the subsequent N actions in the current action block; S2. Based on the number of action points included in the necessary action segment and the smoothness requirements, construct the expression of the interpolation function of the interpolation trajectory and the interpolation curve expression of the multi-order kinematic parameters. S3. Perform smooth interpolation based on multiple target path points, the current movement time interval between adjacent target path points, the expression of the interpolation function of the interpolation trajectory, and the interpolation curve expression of the multi-order kinematic parameters to determine the interpolation trajectory curve and multiple kinematic parameter curves; wherein, the target path points include the end point of the previous action block of the current action block and the action points included in the necessary action segment. S4. Based on the optimization objective, construct an initial optimization function, and modify the initial optimization function through motion constraints to obtain the target optimization function expressed in least squares form; wherein, the optimization objective includes minimizing the execution time and minimizing the deviation between the interpolated trajectory and the original trajectory; the motion constraints are determined based on multiple kinematic parameter curves; S5. Based on the target optimization function, solve the constrained least squares problem using partial derivative numerical approximation and determine convergence. If converged, proceed to step S6; if not converged, update the optimization variables and return to step S3. The optimization variables are the current movement time interval between adjacent target path points. S6. Output the optimal movement time interval between the target path points, and determine the currently determined interpolation trajectory as the optimal movement trajectory for the controlled object to move from the end point of the previous action block to the end point of the current action block.

[0006] Optionally, the step of constructing the expression for the interpolation function of the interpolation trajectory and the interpolation curve expression for the multi-order kinematic parameters based on the number of action points included in the necessary action segment and the smoothness requirement includes: The total number of path points is determined based on the number of action points included in the required action segment, combined with the number of virtual path points and the end point of the previous action block; Based on the smoothness requirements, and in combination with the selected target interpolation method, the constraints of the trajectory endpoints, and the computational efficiency requirements, determine the target order of the interpolation function; Based on the total number of path points and the target order, the expression of the interpolation function for the interpolation trajectory is determined according to the interpolation function construction method of the target interpolation method. Depending on the type of kinematic parameters, the expression of the interpolation function of the interpolation trajectory is differentiated multiple times to obtain the interpolation curve expression of multi-order kinematic parameters.

[0007] Optionally, the step of performing smooth interpolation based on multiple target waypoints, the current movement time interval between adjacent target waypoints, the expression of the interpolation function of the interpolation trajectory, and the interpolation curve expression of the multi-order kinematic parameters to determine the interpolation trajectory curve and multiple kinematic parameter curves includes: Substitute the location information of all target waypoints and the current movement time interval between adjacent target waypoints into the expression of the interpolation function of the interpolation trajectory to obtain the interpolation trajectory curve; Substitute the location information of all target waypoints and the current movement time interval between adjacent target waypoints into the interpolation curve expression of each kinematic parameter to obtain multiple kinematic parameter curves; among them, the multiple kinematic parameter curves include velocity curves, acceleration curves and jerk curves.

[0008] Optionally, constructing the initial optimization function based on the optimization objective includes: Based on the current movement time interval between all adjacent target waypoints, determine the total movement time of the controlled object from the end point of the previous action block to the end point of the current action block according to the currently determined interpolation trajectory curve. Extract the interpolation trajectory curve segment corresponding to the selectable action segment from the currently determined interpolation trajectory curve; The trajectory offset is determined based on the interpolated trajectory curve segment and the optional action segment; The initial optimization function is determined by using a weighted summation of total travel time and trajectory offset.

[0009] Optionally, the step of modifying the initial optimization function through motion constraints to obtain the objective optimization function expressed in least squares form includes: Based on multiple kinematic parameter curves and their corresponding kinematic constraint parameters, determine whether there is a constraint violation. Based on the determination of constraint violations, determine the correction coefficients of the initial optimization function; The initial optimization function is modified according to the correction coefficient and expressed in least squares form to obtain the target optimization function.

[0010] Optionally, the step of solving the constrained least squares problem using partial derivative numerical approximation based on the objective optimization function and performing convergence judgment includes: Based on the objective optimization function, the constrained least squares problem is solved using the numerical Jacobi central difference method, and gradient convergence and variable convergence are judged.

[0011] Optionally, the optimization variables can be updated using the following steps: Based on the numerical Jacobian matrix and the objective function value, an iterative calculation is performed using a nonlinear least squares solver to determine the variable update amount; Based on the variable update amount, the optimization variable is updated according to the update rules to complete this update of the optimization variable.

[0012] This application embodiment also provides a smoothing processing device for action blocks, the smoothing processing device comprising: The segmentation module is used to segment the current action block (excluding the first action block output by the action reasoning model) into optional action segments and mandatory action segments; wherein, the optional action segments include the first M actions in the current action block, and the mandatory action segments include the subsequent N actions in the current action block; The first construction module is used to construct the expression of the interpolation function of the interpolation trajectory and the interpolation curve expression of the multi-order kinematic parameters according to the number of action points included in the necessary action segment and the smoothness requirements. The interpolation module is used to perform smooth interpolation based on multiple target path points, the current movement time interval between adjacent target path points, the expression of the interpolation function of the interpolation trajectory, and the interpolation curve expression of the multi-order kinematic parameters, to determine the interpolation trajectory curve and multiple kinematic parameter curves; wherein, the target path points include the end point of the previous action block of the current action block and the action points included in the necessary action segment. The second construction module is used to construct an initial optimization function based on the optimization objective, and to modify the initial optimization function through motion constraints to obtain the target optimization function expressed in least squares form; wherein, the optimization objective includes minimizing the execution time and minimizing the deviation between the interpolated trajectory and the original trajectory; the motion constraints are determined based on multiple kinematic parameter curves; The judgment module is used to solve the constrained least squares problem by using partial derivative numerical approximation based on the target optimization function, and to perform convergence judgment. If convergence is achieved, the output module is controlled to work; if convergence is not achieved, the optimization variables are updated and the interpolation module is controlled to work. The optimization variables are the current movement time interval between adjacent target path points. The output module is used to output the optimal movement time interval between the target's path points and to determine the currently determined interpolation trajectory as the optimal movement trajectory for the controlled object to move from the end point of the previous action block to the end point of the current action block.

[0013] This application also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the smoothing method described above are performed.

[0014] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the smoothing method described above.

[0015] This application provides a method, apparatus, electronic device, and storage medium for smoothing action blocks. The method includes: S1, dividing the current action block (excluding the first action block output by the action inference model) into optional action segments and mandatory action segments; wherein the optional action segments include the first M actions in the current action block, and the mandatory action segments include the subsequent N actions in the current action block; S2, constructing an expression for the interpolation function of the interpolation trajectory and an interpolation curve expression for multi-order kinematic parameters based on the number of action points included in the mandatory action segments and the smoothness requirement; S3, performing smooth interpolation based on multiple target path points, the current movement time interval between adjacent target path points, the expression for the interpolation function of the interpolation trajectory, and the interpolation curve expression for multi-order kinematic parameters to determine the interpolation trajectory curve and multiple kinematic parameter curves; wherein the target path point includes the end point of the previous action block of the current action block. And the action points included in the necessary action segment; S4. Based on the optimization objective, construct an initial optimization function, and modify the initial optimization function through motion constraints to obtain the target optimization function expressed in least squares form; wherein, the optimization objective includes minimizing the execution time and minimizing the deviation between the interpolated trajectory and the original trajectory; the motion constraints are determined based on multiple kinematic parameter curves; S5. Based on the target optimization function, solve the constrained least squares problem using partial derivative numerical approximation, and perform convergence judgment. If converged, execute step S6; if not converged, update the optimization variables and return to execute step S3; wherein, the optimization variable is the current movement time interval between adjacent target path points; S6. Output the optimal movement time interval between target path points, and determine the currently determined interpolated trajectory as the optimal movement trajectory of the controlled object from the end point of the previous action block to the end point of the current action block.

[0016] Thus, this application firstly divides the current action block into optional action segments and mandatory action segments, using only the action points of the mandatory action segments as the core basis for interpolation, breaking through the technical limitation of conventional interpolation requiring strict passage through all discrete points and avoiding high-frequency distortion at abrupt change points; secondly, by constructing a cross-block target path point system combining the end action point of the previous action block with the mandatory action point of the current action block, a global motion characteristic constraint system is built to replace the limitations of traditional local smoothing schemes; thirdly, by constructing multi-order kinematic parameter interpolation curves and determining motion constraints based on these curves, the initial optimization function is modified, suppressing abrupt changes in kinematic parameters from both mathematical modeling and optimization constraints dimensions; furthermore, using the movement time interval between adjacent target path points as optimization variables, the constrained least squares problem is solved using the partial derivative numerical approximation method to achieve a dynamic balance between execution efficiency and trajectory fit; finally, through iterative optimization and convergence judgment closed-loop design, the optimality and engineering feasibility of the final output trajectory are ensured.

[0017] In summary, the technical solution of this application can effectively eliminate abrupt changes in motion between moving blocks, avoid unnatural dynamic transitions and overall motion discontinuity, and improve the dynamic rationality of the trajectory. Furthermore, this solution ensures that the core motion intent of the motion reasoning model does not deviate, while also taking into account the efficiency of motion execution and adapting to the physical motion constraints of the robot body. This significantly improves the stability of robot motion execution and reduces the risk of task failure caused by discontinuous motion. Therefore, this solution has excellent adaptability and engineering feasibility.

[0018] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating the smoothing method for action blocks provided in an embodiment of this application; Figure 2 An example of an interpolation trajectory for an action block provided in this application; Figure 3 A schematic diagram of the structure of a smoothing device for action blocks provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0022] In the current field of humanoid robot and intelligent agent control, large-scale models based on Vision-Language-Action (VLA) are gradually becoming an important bridge from high-level semantics to low-level action control. However, existing VLA models generally suffer from discontinuities and lack of smoothness between action chunks when inferring and generating continuous action sequences. This is mainly due to the fact that VLA models are designed to predict discrete semantic instructions or temporal segments, and their output actions are often generated in blocks. While each action block is relatively consistent internally, there are abrupt changes in velocity, acceleration, or posture between blocks, leading to jitter, instability, or even task failure during actual robot execution.

[0023] Existing methods primarily focus on directly smoothing action sequences within a single action block or between adjacent blocks, lacking overall constraints on global motion characteristics. Since action blocks inferred from VLA models are semantically independent, significant abrupt changes in posture or velocity often occur between their final actions and the starting actions of the next action block. Even when traditional smoothing algorithms interpolate or filter these points locally, noticeable kinematic discontinuities still arise. This is because conventional interpolation methods typically require the trajectory to strictly pass through each discrete point, thus forcing the introduction of high-frequency changes or unnatural dynamic transitions when abrupt changes occur.

[0024] Based on this, embodiments of this application provide a method, apparatus, electronic device, and storage medium for smoothing action blocks, so as to effectively eliminate motion abrupt changes and jitter problems between action blocks output by the action inference model.

[0025] Please see Figure 1 , Figure 1 This is a flowchart illustrating the smoothing method for action blocks provided in an embodiment of this application. Figure 1As shown in the embodiments of this application, the smoothing processing method includes: S1. Divide the current action block that is not the first action block output by the action reasoning model into optional action segments and mandatory action segments.

[0026] S2. Based on the number of action points included in the necessary action segment and the smoothness requirements, construct the expression of the interpolation function of the interpolation trajectory and the expression of the interpolation curve of the multi-order kinematic parameters.

[0027] S3. Perform smooth interpolation based on multiple target waypoints, the current movement time interval between adjacent target waypoints, the expression of the interpolation function of the interpolation trajectory, and the interpolation curve expression of the multi-order kinematic parameters to determine the interpolation trajectory curve and multiple kinematic parameter curves.

[0028] S4. Based on the optimization objective, construct an initial optimization function, and modify the initial optimization function through motion constraints to obtain the objective optimization function expressed in least squares form.

[0029] S5. Based on the objective optimization function, solve the constrained least squares problem using the partial derivative numerical approximation method, and perform convergence judgment.

[0030] S6. Output the optimal movement time interval between the target path points, and determine the currently determined interpolation trajectory as the optimal movement trajectory for the controlled object to move from the end point of the previous action block to the end point of the current action block.

[0031] The exemplary steps of the embodiments of this application are described below: Specifically, step S1 may include: identifying the type of each action block in the action sequence output by the action inference model in chronological order, and determining whether the action block is the first action block; if it is not the first action block, then segmenting the current action block to obtain optional action segments and necessary action segments.

[0032] Here, the action reasoning model can be a large vision-language-action model, the optional action fragments include the first M actions in the current action block, and the mandatory action fragments include the subsequent N actions in the current action block. The M actions and N actions can be specifically understood as M action points and N action points.

[0033] The segmentation position when dividing the current action block can be adaptively selected according to the work requirements, where M+N = the sum of actions of the current action block.

[0034] The action points in the optional action segments are not forced to be interpolated through, but the action points in the mandatory action segments are precisely interpolated and optimized, thereby ensuring the overall dynamic smoothness and continuity while maintaining trajectory accuracy.

[0035] Regarding step S2, in one embodiment provided in this application, the step of constructing the expression for the interpolation function of the interpolation trajectory and the interpolation curve expression for the multi-order kinematic parameters based on the number of action points included in the necessary action segment and the smoothness requirement includes: S21. Determine the total number of path points based on the number of action points included in the required action segment, combined with the number of virtual path points and the last action point of the previous action block.

[0036] S22. Based on the smoothness requirements, and in conjunction with the selected target interpolation method, the constraints of the trajectory endpoints, and the computational efficiency requirements, determine the target order of the interpolation function.

[0037] S23. Based on the total number of path points and the target order, determine the expression of the interpolation function of the interpolation trajectory according to the interpolation function construction method of the target interpolation method.

[0038] S24. Based on the type of kinematic parameters, differentiate the expression of the interpolation function of the interpolation trajectory multiple times to obtain the interpolation curve expression of the multi-order kinematic parameters.

[0039] For step S21, this step specifically includes: determining the total number of path points by summing the number of action points included in the necessary action segment, the number of virtual path points, and the value 1.

[0040] For example, two virtual path points can be set, with the two virtual path points introduced at the second and second-to-last positions of the joint path point sequence.

[0041] It should be noted that the reason for introducing two virtual path points is that when the selected target interpolation method is B-spline interpolation, if the jerk at the starting and ending points of the trajectory needs to be constrained to zero, a 7th-order B-spline curve needs to be set. However, in order to reduce the increase in computational efficiency requirements, two virtual path points are introduced at the 2nd and 2nd-to-last positions of the joint path point sequence, and the B-spline order is reduced back to 5th order.

[0042] In this way, the computational efficiency requirements are met while the constraints on the trajectory endpoints are also ensured.

[0043] Regarding step S22, the smoothness requirement can be set to ensure that the acceleration of the interpolated trajectory is smooth and continuous; the target interpolation method can specifically be B-spline interpolation, and the constraint requirement for the trajectory endpoints can be set to constrain the acceleration of the trajectory start point and end point to zero. Therefore, according to the above example, the determined target order can be 5.

[0044] For step S23, the expression for the interpolation function of the interpolation trajectory is as follows:

[0045] in, This refers to the joint position. To control the vertices, It is a vector consisting of non-decreasing normalized time nodes, where n is the number of control vertices. The basis functions represent the B-spline curves.

[0046] It should be noted that if the controlled object includes multiple controlled joints, the motion inference model can output a corresponding motion sequence for each controlled joint, which requires interpolation processing of the movement trajectory of each joint.

[0047] For step S24, the kinematic parameters may include velocity, acceleration, and jerk.

[0048] When the kinematic parameter is velocity, the first derivative of the interpolation function expression of the interpolated trajectory yields the interpolation curve expression of the velocity of the trajectory; when the kinematic parameter is acceleration, the second derivative of the interpolation function expression of the interpolated trajectory yields the interpolation curve expression of the acceleration of the trajectory; when the kinematic parameter is jerk, the third derivative of the interpolation function expression of the interpolated trajectory yields the interpolation curve expression of the jerk of the trajectory.

[0049] For example, this application provides the velocity, acceleration, and jerk interpolation curve expressions for the trajectory, as shown below:

[0050] In the formula The vertices are the velocity, acceleration, and jerk control vertices of the B-spline curve.

[0051] For step S3, the target path point includes the end point of the previous action block of the current action block and all action points included in the necessary action segment.

[0052] The current movement time interval between adjacent target waypoints can be the same or different. Before priority is given, the current movement time interval between adjacent target waypoints is the initial time interval, which can be output by the action reasoning model or adaptively set.

[0053] Continuing with step S3, in one embodiment provided in this application, the step of performing smooth interpolation based on multiple target waypoints, the current movement time interval between adjacent target waypoints, the expression of the interpolation function of the interpolation trajectory, and the interpolation curve expression of multiple kinematic parameters to determine the interpolation trajectory curve and multiple kinematic parameter curves includes: substituting the position information of all target waypoints and the current movement time interval between adjacent target waypoints into the expression of the interpolation function of the interpolation trajectory to obtain the interpolation trajectory curve; and substituting the position information of all target waypoints and the current movement time interval between adjacent target waypoints into the interpolation curve expression of each kinematic parameter to obtain multiple kinematic parameter curves.

[0054] Here, the multiple kinematic parameter curves include velocity curves, acceleration curves, and jerk curves.

[0055] For an example, please refer to Figure 2 , Figure 2 An example of an interpolated trajectory for an action block provided in this application. For example... Figure 2 As shown, For the current action block, The previous action block of the current action block. Optional action segments The dashed line represents the interpolation trajectory curve determined by smoothing the current action block, and the starting point of this curve is... The point of action at the end of the curve is the endpoint of the curve. The point of action at the end of the process.

[0056] For step S4, the optimization objectives include minimizing the execution time and minimizing the deviation between the interpolated trajectory and the original trajectory; the motion constraints are determined based on multiple kinematic parameter curves.

[0057] Regarding step S4, in one embodiment provided in this application, constructing the initial optimization function according to the optimization objective includes: S411. Based on the current movement time interval between all adjacent target transit points, determine the total movement time of the controlled object from the end point of the previous action block to the end point of the current action block according to the currently determined interpolation trajectory curve. S412. Extract the interpolation trajectory curve segment corresponding to the selectable action segment from the currently determined interpolation trajectory curve.

[0058] S413. Determine the trajectory offset based on the interpolated trajectory curve segment and the optional action segment.

[0059] S414. The initial optimization function is determined by using a weighted summation method that combines total travel time and trajectory offset.

[0060] For step S411, this step specifically includes adding the current movement time intervals between all adjacent target path points to obtain the total movement time of the controlled object from the end point of the previous action block to the end point of the current action block.

[0061] The controlled object mentioned here can be an intelligent agent, such as a robot.

[0062] For steps S412-S413, when calculating the trajectory offset, it can also be done by determining the corresponding interpolation point in the interpolation trajectory for each action point in the optional action segment; calculating the offset between each action point in the optional action segment and the corresponding interpolation point and adding them together to obtain the trajectory offset.

[0063] For step S414, different weight values ​​can be set for the total travel time and trajectory offset.

[0064] Continuing with step S4, in one embodiment provided in this application, the step of modifying the initial optimization function through motion constraints to obtain the objective optimization function expressed in least squares form includes: S421. Based on multiple kinematic parameter curves and their corresponding kinematic constraint parameters, determine whether there is a constraint violation.

[0065] S422. Based on the determination of the constraint violation, determine the correction coefficient of the initial optimization function.

[0066] S423. The initial optimization function is modified according to the correction coefficient and expressed in least squares form to obtain the target optimization function.

[0067] For step S421, when the kinematic parameters are velocity, acceleration and jerk, the corresponding motion constraint parameters include maximum velocity and minimum velocity, maximum acceleration and minimum acceleration, and maximum jerk and minimum jerk.

[0068] Based on multiple kinematic parameter curves and their corresponding motion constraint parameters, it is determined whether there are any constraint violations. Specifically, it is determined whether there are any values ​​in the velocity curve of the controlled object that are less than the minimum velocity and / or greater than the maximum velocity; whether there are any values ​​in the acceleration curve of the controlled object that are less than the minimum acceleration and / or greater than the maximum acceleration; and whether there are any values ​​in the jerk curve of the controlled object that are less than the minimum jerk and / or greater than the maximum jerk.

[0069] Regarding step S422, if a constraint violation exists, the correction coefficient of the initial optimization function is determined to be a larger value; if no constraint violation exists, the correction coefficient of the initial optimization function is determined to be a smaller value. The specific value can be determined adaptively.

[0070] The correction factor can also be called the penalty factor.

[0071] For step S423, the initial optimization function is modified according to the correction coefficient and expressed in least squares form, resulting in the target optimization function as shown below.

[0072] ,

[0073] In the formula As the weight of the total travel time, The weight of the trajectory offset can be set. , This is a correction factor.

[0074] Regarding step S5, in one embodiment provided in this application, the step of solving the constrained least squares problem using partial derivative numerical approximation based on the objective optimization function and performing convergence judgment includes: solving the constrained least squares problem using numerical Jacobi central difference based on the objective optimization function and performing gradient convergence and variable convergence judgment.

[0075] If both gradient convergence and variable convergence are successful, proceed to step S6. If either gradient convergence or variable convergence is unsuccessful, update the optimization variable and return to step S3. The optimization variable is the current movement time interval between adjacent target waypoints.

[0076] Furthermore, in one embodiment provided in this application, the optimization variables are updated through the following steps: based on the numerical Jacobian matrix and the objective function value, a nonlinear least squares solver is used to perform iterative calculations to determine the variable update amount; according to the variable update amount, the optimization variables are updated according to the update rules to complete this update of the optimization variables.

[0077] Thus, using the updated movement time interval, return to step S3 to redetermine the interpolation trajectory curve and multiple kinematic parameter curves, and execute S4-S6 sequentially to find the optimal interpolation trajectory.

[0078] For step S6, the current movement time interval between the target path points is determined as the optimal movement time interval between the target path points and output. The currently determined interpolation trajectory is also determined as the optimal movement trajectory, thereby controlling the controlled object to move according to the optimal movement trajectory, thus meeting the requirements of continuity, smoothness, and physical executability of the motion trajectory.

[0079] Based on the same inventive concept, this application also provides a smoothing device corresponding to the smoothing method. Since the principle of the device in this application is similar to the smoothing method described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0080] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a smoothing device for action blocks provided in an embodiment of this application. Figure 3 As shown, the smoothing processing device 300 includes: The segmentation module 310 is used to segment the current action block, which is not the first action block output by the action reasoning model, into optional action segments and mandatory action segments; wherein, the optional action segments include the first M actions in the current action block, and the mandatory action segments include the subsequent N actions in the current action block. The first construction module 320 is used to construct an expression for the interpolation function of the interpolation trajectory and an interpolation curve expression for the multi-order kinematic parameters based on the number of action points included in the necessary action segment and the smoothness requirements. The interpolation module 330 is used to perform smooth interpolation based on multiple target path points, the current movement time interval between adjacent target path points, the expression of the interpolation function of the interpolation trajectory, and the interpolation curve expression of the multi-order kinematic parameters, to determine the interpolation trajectory curve and multiple kinematic parameter curves; wherein, the target path points include the end point of the previous action block of the current action block and the action points included in the necessary action segment. The second construction module 340 is used to construct an initial optimization function based on the optimization objective, and to modify the initial optimization function through motion constraints to obtain the target optimization function expressed in least squares form; wherein, the optimization objective includes minimizing the execution time and minimizing the deviation between the interpolated trajectory and the original trajectory; the motion constraints are determined based on multiple kinematic parameter curves; The judgment module 350 is used to solve the constrained least squares problem by using the partial derivative numerical approximation method according to the target optimization function, and to perform convergence judgment. If convergence is achieved, the output module is controlled to work; if convergence is not achieved, the optimization variables are updated and the interpolation module is controlled to work. The optimization variables are the current movement time interval between adjacent target path points. The output module 360 ​​is used to output the optimal movement time interval between the target path points and to determine the currently determined interpolation trajectory as the optimal movement trajectory for the controlled object to move from the end point of the previous action block to the end point of the current action block.

[0081] Optionally, when the first construction module 320 is used to construct the expression of the interpolation function of the interpolation trajectory and the interpolation curve expression of the multi-order kinematic parameters based on the number of action points included in the necessary action segment and the smoothness requirements, the first construction module 320 is used to: The total number of path points is determined based on the number of action points included in the required action segment, combined with the number of virtual path points and the end point of the previous action block; Based on the smoothness requirements, and in combination with the selected target interpolation method, the constraints of the trajectory endpoints, and the computational efficiency requirements, determine the target order of the interpolation function; Based on the total number of path points and the target order, the expression of the interpolation function for the interpolation trajectory is determined according to the interpolation function construction method of the target interpolation method. Depending on the type of kinematic parameters, the expression of the interpolation function of the interpolation trajectory is differentiated multiple times to obtain the interpolation curve expression of multi-order kinematic parameters.

[0082] Optionally, when the interpolation module 330 is used to perform smooth interpolation based on multiple target waypoints, the current movement time interval between adjacent target waypoints, the expression of the interpolation function of the interpolation trajectory, and the interpolation curve expression of the multi-order kinematic parameters to determine the interpolation trajectory curve and multiple kinematic parameter curves, the interpolation module 330 is used to: Substitute the location information of all target waypoints and the current movement time interval between adjacent target waypoints into the expression of the interpolation function of the interpolation trajectory to obtain the interpolation trajectory curve; Substitute the location information of all target waypoints and the current movement time interval between adjacent target waypoints into the interpolation curve expression of each kinematic parameter to obtain multiple kinematic parameter curves; among them, the multiple kinematic parameter curves include velocity curves, acceleration curves and jerk curves.

[0083] Optionally, when the second construction module 340 is used to construct the initial optimization function according to the optimization objective, the second construction module 340 is used to: Based on the current movement time interval between all adjacent target waypoints, determine the total movement time of the controlled object from the end point of the previous action block to the end point of the current action block according to the currently determined interpolation trajectory curve. Extract the interpolation trajectory curve segment corresponding to the selectable action segment from the currently determined interpolation trajectory curve; The trajectory offset is determined based on the interpolated trajectory curve segment and the optional action segment; The initial optimization function is determined by using a weighted summation of total travel time and trajectory offset.

[0084] Optionally, when the second construction module 340 is used to modify the initial optimization function through motion constraints to obtain the objective optimization function expressed in least squares form, the second construction module 340 is used to: Based on multiple kinematic parameter curves and their corresponding kinematic constraint parameters, determine whether there is a constraint violation. Based on the determination of constraint violations, determine the correction coefficients of the initial optimization function; The initial optimization function is modified according to the correction coefficient and expressed in least squares form to obtain the target optimization function.

[0085] Optionally, when the judgment module 350 is used to solve the constrained least squares problem using the partial derivative numerical approximation method based on the objective optimization function and to perform convergence judgment, the judgment module 350 is used to: Based on the objective optimization function, the constrained least squares problem is solved using the numerical Jacobi central difference method, and gradient convergence and variable convergence are judged.

[0086] Optionally, the judgment module 350 is further configured to update the optimization variable through the following steps: Based on the numerical Jacobian matrix and the objective function value, an iterative calculation is performed using a nonlinear least squares solver to determine the variable update amount; Based on the variable update amount, the optimization variable is updated according to the update rules to complete this update of the optimization variable.

[0087] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 400 includes a processor 410, a memory 420, and a bus 430.

[0088] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, they can perform the operations described above. Figure 1 as well as Figure 2 The steps in the method embodiment shown are specifically implemented in the method embodiment and will not be repeated here.

[0089] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 as well as Figure 2 The steps in the method embodiment shown are specifically implemented in the method embodiment and will not be repeated here.

[0090] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0091] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0092] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0093] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0094] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0095] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for smoothing action blocks, characterized in that, The smoothing method includes: S1. Divide the current action block (excluding the first action block) output by the action reasoning model into optional action segments and mandatory action segments; wherein, the optional action segments contain the first M actions in the current action block, and the mandatory action segments contain the subsequent N actions in the current action block; S2. Based on the number of action points included in the necessary action segment and the smoothness requirements, construct the expression of the interpolation function of the interpolation trajectory and the interpolation curve expression of the multi-order kinematic parameters. S3. Perform smooth interpolation based on multiple target path points, the current movement time interval between adjacent target path points, the expression of the interpolation function of the interpolation trajectory, and the interpolation curve expression of the multi-order kinematic parameters to determine the interpolation trajectory curve and multiple kinematic parameter curves; wherein, the target path points include the end point of the previous action block of the current action block and the action points included in the necessary action segment. S4. Based on the optimization objective, construct an initial optimization function, and modify the initial optimization function through motion constraints to obtain the target optimization function expressed in least squares form; wherein, the optimization objective includes minimizing the execution time and minimizing the deviation between the interpolated trajectory and the original trajectory; the motion constraints are determined based on multiple kinematic parameter curves; S5. Based on the target optimization function, solve the constrained least squares problem using partial derivative numerical approximation and determine convergence. If converged, proceed to step S6; if not converged, update the optimization variables and return to step S3. The optimization variables are the current movement time interval between adjacent target path points. S6. Output the optimal movement time interval between the target path points, and determine the currently determined interpolation trajectory as the optimal movement trajectory for the controlled object to move from the end point of the previous action block to the end point of the current action block.

2. The smoothing method according to claim 1, characterized in that, The step of constructing the expression for the interpolation function of the interpolation trajectory and the expression for the interpolation curve of the multi-order kinematic parameters based on the number of action points included in the necessary action segment and the smoothness requirements includes: The total number of path points is determined based on the number of action points included in the required action segment, combined with the number of virtual path points and the end point of the previous action block; Based on the smoothness requirements, and in combination with the selected target interpolation method, the constraints of the trajectory endpoints, and the computational efficiency requirements, determine the target order of the interpolation function; Based on the total number of path points and the target order, the expression of the interpolation function for the interpolation trajectory is determined according to the interpolation function construction method of the target interpolation method. Depending on the type of kinematic parameters, the expression of the interpolation function of the interpolation trajectory is differentiated multiple times to obtain the interpolation curve expression of multi-order kinematic parameters.

3. The smoothing method according to claim 1, characterized in that, The process of performing smooth interpolation based on multiple target waypoints, the current movement time interval between adjacent target waypoints, the expression of the interpolation function of the interpolation trajectory, and the interpolation curve expression of multi-order kinematic parameters to determine the interpolation trajectory curve and multiple kinematic parameter curves includes: Substitute the location information of all target waypoints and the current movement time interval between adjacent target waypoints into the expression of the interpolation function of the interpolation trajectory to obtain the interpolation trajectory curve; Substitute the location information of all target waypoints and the current movement time interval between adjacent target waypoints into the interpolation curve expression of each kinematic parameter to obtain multiple kinematic parameter curves; among them, the multiple kinematic parameter curves include velocity curves, acceleration curves and jerk curves.

4. The smoothing method according to claim 1, characterized in that, The step of constructing an initial optimization function based on the optimization objective includes: Based on the current movement time interval between all adjacent target waypoints, determine the total movement time of the controlled object from the end point of the previous action block to the end point of the current action block according to the currently determined interpolation trajectory curve. Extract the interpolation trajectory curve segment corresponding to the selectable action segment from the currently determined interpolation trajectory curve; The trajectory offset is determined based on the interpolated trajectory curve segment and the optional action segment; The initial optimization function is determined by using a weighted summation of total travel time and trajectory offset.

5. The smoothing method according to claim 1, characterized in that, The step of modifying the initial optimization function through motion constraints to obtain the objective optimization function expressed in least squares form includes: Based on multiple kinematic parameter curves and their corresponding kinematic constraint parameters, determine whether there is a constraint violation. Based on the determination of constraint violations, determine the correction coefficients of the initial optimization function; The initial optimization function is modified according to the correction coefficient and expressed in least squares form to obtain the target optimization function.

6. The smoothing method according to claim 1, characterized in that, The step of solving the constrained least squares problem using partial derivative numerical approximation based on the objective optimization function, and then performing convergence judgment, includes: Based on the objective optimization function, the constrained least squares problem is solved using the numerical Jacobi central difference method, and gradient convergence and variable convergence are judged.

7. The smoothing method according to claim 1, characterized in that, The optimization variables are updated using the following steps: Based on the numerical Jacobian matrix and the objective function value, an iterative calculation is performed using a nonlinear least squares solver to determine the variable update amount; Based on the variable update amount, the optimization variable is updated according to the update rules to complete this update of the optimization variable.

8. A smoothing device for motion blocks, characterized in that, The smoothing processing device includes: The segmentation module is used to segment the current action block (excluding the first action block output by the action reasoning model) into optional action segments and mandatory action segments; wherein, the optional action segments include the first M actions in the current action block, and the mandatory action segments include the subsequent N actions in the current action block; The first construction module is used to construct the expression of the interpolation function of the interpolation trajectory and the interpolation curve expression of the multi-order kinematic parameters according to the number of action points included in the necessary action segment and the smoothness requirements. The interpolation module is used to perform smooth interpolation based on multiple target path points, the current movement time interval between adjacent target path points, the expression of the interpolation function of the interpolation trajectory, and the interpolation curve expression of the multi-order kinematic parameters, to determine the interpolation trajectory curve and multiple kinematic parameter curves; wherein, the target path points include the end point of the previous action block of the current action block and the action points included in the necessary action segment. The second construction module is used to construct an initial optimization function based on the optimization objective, and to modify the initial optimization function through motion constraints to obtain the target optimization function expressed in least squares form; wherein, the optimization objective includes minimizing the execution time and minimizing the deviation between the interpolated trajectory and the original trajectory; the motion constraints are determined based on multiple kinematic parameter curves; The judgment module is used to solve the constrained least squares problem by using partial derivative numerical approximation based on the target optimization function, and to perform convergence judgment. If convergence is achieved, the output module is controlled to work; if convergence is not achieved, the optimization variables are updated and the interpolation module is controlled to work. The optimization variables are the current movement time interval between adjacent target path points. The output module is used to output the optimal movement time interval between the target's path points and to determine the currently determined interpolation trajectory as the optimal movement trajectory for the controlled object to move from the end point of the previous action block to the end point of the current action block.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and the machine-readable instructions are executed by the processor to perform the steps of the smoothing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the smoothing method as described in any one of claims 1 to 7.

Images