Track generation method and device and electronic equipment

Abstract

The embodiment of the invention provides a track generation method and device and electronic equipment, and relates to the technical field of data processing, and the track generation method comprises the steps: carrying out the interpolation processing of at least one pair of adjacent track points in a plurality of track points after a VLA model outputs a plurality of track points for a target task of a robot at any time, obtaining interpolation points, and carrying out the interpolation processing of the interpolation points; under the condition that the first track exists, based on the first track, position information of at least one target point in the multiple track points and the interpolation points is subjected to preset optimization processing, and the preset optimization processing is processing used for optimizing track smoothness; and according to the current track point and the current interpolation point existing after the preset optimization processing, generating a to-be-utilized second track which corresponds to the plurality of track points output at the time and is used for executing the target task by the robot. According to the scheme, the smoothness of the generated track can be improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to trajectory generation methods, apparatus and electronic devices. Background Technology

[0002] The Vision-Language-Action (VLA) model aims to integrate visual perception, language understanding, and physical actions, enabling robots to intelligently interact with the physical world. In scenarios where robots are controlled to perform tasks, image data collected by the robot and user commands can be input into the VLA model. The model outputs multiple trajectory points to form a trajectory, which the robot then follows to perform the task. These trajectory points refer to the points the robot needs to reach during the task. Furthermore, for the same task, the VLA model can infer and output multiple trajectory points at a set frequency to dynamically update the robot's trajectory during task execution.

[0003] However, the VLA model outputs multiple trajectory points with large intervals each time, resulting in poor trajectory smoothness. This leads to abrupt changes in the robot's position, velocity, and acceleration, damaging the robot's hardware structure, thus shortening the robot's lifespan and increasing the probability of failure. Summary of the Invention

[0004] The purpose of this invention is to provide a trajectory generation method, apparatus, and electronic device to improve the smoothness of the generated trajectory. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of the present invention provide a trajectory generation method, the method comprising:

[0006] After the Vision-Language-Motion (VLA) model outputs multiple trajectory points for the robot's target task, interpolation is performed on at least one pair of adjacent trajectory points to obtain interpolated points.

[0007] In the presence of a first trajectory, based on the first trajectory, the position information of at least one target point among multiple trajectory points and interpolation points is subjected to predetermined optimization processing. The first trajectory is: the trajectory obtained before the output of multiple trajectory points for the target task. The predetermined optimization processing is a process used to optimize the smoothness of the trajectory.

[0008] Based on the current trajectory points and interpolation points that exist after the predetermined optimization process, a second trajectory corresponding to the multiple trajectory points output in this iteration is generated, which is to be used by the robot to perform the target task.

[0009] Secondly, embodiments of the present invention provide a trajectory generation apparatus, the apparatus comprising:

[0010] The interpolation module is used to interpolate at least one pair of adjacent trajectory points among the multiple trajectory points after the vision-language-action VLA model outputs multiple trajectory points for the robot's target task, so as to obtain interpolated points.

[0011] The pre-optimization module is used to perform pre-optimization processing on the position information of at least one target point among multiple trajectory points and interpolation points based on the first trajectory when a first trajectory exists. The first trajectory is the trajectory obtained before the output of multiple trajectory points for the target task. The pre-optimization processing is a process used to optimize the smoothness of the trajectory.

[0012] The first generation module is used to generate a second trajectory corresponding to the multiple trajectory points output in this output, based on the current trajectory points and the current interpolation points after predetermined optimization processing, for the robot to perform the target task.

[0013] Thirdly, embodiments of the present invention provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0014] Memory, used to store computer programs;

[0015] The processor, when executing a program stored in memory, implements any of the above trajectory generation methods.

[0016] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-described trajectory generation methods.

[0017] Beneficial effects of the embodiments of the present invention:

[0018] The technical solution provided by this invention can, after the VLA model outputs multiple trajectory points for the robot's target task at any time, perform interpolation processing on at least one pair of adjacent trajectory points among the multiple trajectory points to obtain interpolation points. If a first trajectory exists, based on the first trajectory, perform predetermined optimization processing on the position information of at least one target point among the multiple trajectory points and interpolation points respectively. Subsequently, based on the current trajectory points and the current interpolation points that exist after the predetermined optimization processing, generate a second trajectory corresponding to the multiple trajectory points output this time, which is to be used by the robot to perform the target task. As can be seen, the embodiments of the present invention can not only perform interpolation processing on the VLA model and multiple trajectory points output for any given time for the robot's target task, but also perform predetermined optimization processing on the position information of at least one target point among the multiple trajectory points and interpolation points based on the first trajectory (the trajectory obtained before the output of multiple trajectory points for the target task). This improves the smoothness of the second trajectory to be used by combining the two aspects. Specifically, the interpolation processing can insert interpolation points between at least one pair of adjacent trajectory points, increasing the number of points, filling the position gaps between discrete trajectory points, effectively mitigating abrupt position changes of adjacent trajectory points, and achieving smooth and continuous position of adjacent trajectory points, thereby improving the smoothness of the trajectory. Furthermore, the historical trajectory (i.e., the first trajectory) is used as a reference for predetermined optimization processing (i.e., trajectory smoothing optimization of the target point). This ensures that the trajectory to be generated this time can continue the pattern of the historical trajectory (given certain position constraints) relative to the historical trajectory, mitigating abrupt position changes of the generated trajectory compared to the historical trajectory, and improving the smoothness of the generated trajectory. Finally, combining the two aspects can improve the smoothness of the generated trajectory.

[0019] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0021] Figure 1(a) is a schematic diagram of the effect of a trajectory generation method in related technologies;

[0022] Figure 1(b) is a schematic diagram of the principle of a trajectory generation method in related technologies;

[0023] Figure 2 This is a flowchart illustrating a trajectory generation method provided in an embodiment of the present invention;

[0024] Figure 3(a) is a flowchart illustrating another trajectory generation method provided in an embodiment of the present invention;

[0025] Figure 3(b) is a schematic diagram of the principle of a trajectory generation method provided in an embodiment of the present invention;

[0026] Figure 4 A flowchart illustrating another trajectory generation method provided in an embodiment of the present invention;

[0027] Figure 5 This is a schematic diagram of the structure of a trajectory generation device provided in an embodiment of the present invention;

[0028] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on the present invention are within the scope of protection of the present invention.

[0030] First, some technical terms in the embodiments of this invention will be introduced:

[0031] Temporal Ensemble is a semi-supervised learning method that improves the performance and stability of a model by integrating the inference results of the model over time.

[0032] To better understand this solution, the following description, in conjunction with the accompanying drawings, introduces the related technologies most similar to the trajectory generation method provided in the embodiments of this invention, as shown in Figures 1(a) and 1(b):

[0033] In related technology 1, for the same task, the VLA model can infer multiple times at a set frequency and output multiple trajectory points to form a trajectory, thereby dynamically updating the trajectory of the robot during task execution. Since the multiple trajectory points output by the VLA model all take the robot's position at the start of inference as the starting point, and each inference by the VLA model consumes a period of time, the robot is still moving during the model's inference period. As a result, when the inference is completed, the robot has moved to another position (different from the starting position at the start of inference). The robot's current position is not on the trajectory formed by the multiple trajectory points output by the VLA model, and the robot's current position is discontinuous with the multiple trajectory points, which causes the robot to malfunction in task execution.

[0034] In Figure 1(a), the robot's target task is to move to position ①. The VLA model takes the robot's position at the start of inference as the starting point and outputs multiple trajectory points to form trajectory 100. However, during the inference process of the VLA model, the robot has moved to another position, namely the current position. The robot's current position is not on trajectory 100. If the robot is controlled to move according to trajectory 100, it will eventually move to position ②, causing the robot to malfunction in performing the target task.

[0035] In related technology 2, Temporal Ensemble technology can be used to combine the trajectory points output by the VLA model in the past and perform weighted calculation on the current trajectory points to determine the position information of the next trajectory point to be reached by the robot.

[0036] As shown in Figure 1(b), the VLA model outputs four trajectory points (1-4) for each inference. These four trajectory points represent the points the robot needs to reach to perform the target task. D=1 represents the first inference by the VLA model, D=2 represents the second inference, D=3 represents the third inference, and D=4 represents the fourth inference. The fourth inference is the most recent inference. There is a time step between two adjacent inferences. Therefore, multiple trajectory points can exist at the same time step. Ensemble technology considers the position information of the trajectory point corresponding to a time step as a weighted sum of the position information of all trajectory points at that time step. Taking time step 110 as an example, the position information of trajectory point 1 obtained from the fourth inference is multiplied by a weight of 0.5, the position information of trajectory point 2 obtained from the third inference is multiplied by a weight of 0.3, the position information of trajectory point 3 obtained from the second inference is multiplied by a weight of 0.2, and the position information of trajectory point 4 obtained from the first inference is multiplied by a weight of 0.1. The sum of the above weighted position information is then calculated to obtain the position information of the trajectory point corresponding to time step 110.

[0037] However, this related technology only weights the positional information of trajectory points at the same time step, failing to address the problem of excessively large spacing between adjacent trajectory points in a single inference output, resulting in poor trajectory smoothness. Furthermore, the aforementioned issue of discontinuity between the robot's current position and multiple trajectory points remains unresolved.

[0038] Based on the problems described above, embodiments of the present invention provide a trajectory generation method, apparatus, and electronic device.

[0039] The trajectory generation method provided in the embodiments of the present invention will be described below.

[0040] The trajectory generation method provided in this embodiment of the invention can be applied to a trajectory generation device, which is located at the back end of a VLA model and can communicate with the VLA model to further process the output results of the VLA model. The trajectory generation device can be in software form, such as trajectory generation software deployed on a personal computer (PC); or it can be in hardware form, such as a tablet computer, mobile phone, or other mobile device used for trajectory generation. This embodiment of the invention does not specifically limit the form of the trajectory generation device. Furthermore, the robot targeted in this embodiment of the invention can be a robot equipped with an image acquisition device, such as a humanoid robot (the image acquisition device can be located on the two arms and head of the humanoid robot), or a robot dog (the image acquisition device can be located on the head of the robot dog). In addition, the VLA model in this embodiment of the invention performs model inference based on the input prompt words and the image data acquired by the image acquisition device to output multiple trajectory points for the robot's target task. The trajectory generation device then generates the trajectory; this process can also be called "end-to-end".

[0041] Furthermore, the trajectory generation method provided in this embodiment of the invention can be applied to various scenarios, such as: trajectory generation during the process of using a robot to perform a picking task in a warehouse; trajectory generation during the process of using a robot to perform a medicine picking task in a hospital; and trajectory generation during the process of performing a task of pulling out a drawer to retrieve items in a home setting.

[0042] The trajectory generation method provided in this embodiment of the invention includes:

[0043] After the Vision-Language-Motion (VLA) model outputs multiple trajectory points for the robot's target task, interpolation is performed on at least one pair of adjacent trajectory points to obtain interpolated points.

[0044] In the presence of a first trajectory, based on the first trajectory, the position information of at least one target point among multiple trajectory points and interpolation points is subjected to predetermined optimization processing. The first trajectory is: the trajectory obtained before the output of multiple trajectory points for the target task. The predetermined optimization processing is a process used to optimize the smoothness of the trajectory.

[0045] Based on the current trajectory points and interpolation points that exist after the predetermined optimization process, a second trajectory corresponding to the multiple trajectory points output in this iteration is generated, which is to be used by the robot to perform the target task.

[0046] The technical solution provided by this invention can, after the VLA model outputs multiple trajectory points for the robot's target task at any time, perform interpolation processing on at least one pair of adjacent trajectory points among the multiple trajectory points to obtain interpolation points. If a first trajectory exists, based on the first trajectory, perform predetermined optimization processing on the position information of at least one target point among the multiple trajectory points and interpolation points respectively. Subsequently, based on the current trajectory points and the current interpolation points that exist after the predetermined optimization processing, generate a second trajectory corresponding to the multiple trajectory points output this time, which is to be used by the robot to perform the target task. As can be seen, the embodiments of the present invention can not only perform interpolation processing on the VLA model and multiple trajectory points output for any given time for the robot's target task, but also perform predetermined optimization processing on the position information of at least one target point among the multiple trajectory points and interpolation points based on the first trajectory (the trajectory obtained before the output of multiple trajectory points for the target task). This improves the smoothness of the second trajectory to be used by combining the two aspects. Specifically, the interpolation processing can insert interpolation points between at least one pair of adjacent trajectory points, increasing the number of points, filling the position gaps between discrete trajectory points, effectively mitigating abrupt position changes of adjacent trajectory points, and achieving smooth and continuous position of adjacent trajectory points, thereby improving the smoothness of the trajectory. Furthermore, the historical trajectory (i.e., the first trajectory) is used as a reference for predetermined optimization processing (i.e., trajectory smoothing optimization of the target point). This ensures that the trajectory to be generated this time can continue the pattern of the historical trajectory (given certain position constraints) relative to the historical trajectory, mitigating abrupt position changes of the generated trajectory compared to the historical trajectory, and improving the smoothness of the generated trajectory. Finally, combining the two aspects can improve the smoothness of the generated trajectory.

[0047] The trajectory generation method provided by the embodiments of the present invention will be described below with reference to the accompanying drawings.

[0048] like Figure 2 As shown in the figure, an embodiment of the present invention provides a trajectory generation method, the method comprising:

[0049] S201: After the Vision-Language-Motion (VLA) model outputs multiple trajectory points for the robot's target task, interpolation is performed on at least one pair of adjacent trajectory points among the multiple trajectory points to obtain interpolated points.

[0050] It is understood that the VLA model in this embodiment of the invention can also perform multiple inferences at a set frequency and output multiple trajectory points for forming a trajectory. For example, the VLA model can perform an inference every 10 seconds. After the VLA model outputs multiple trajectory points for the robot's target task, the multiple trajectory points output by the VLA model can be obtained. However, the interval between any two adjacent trajectory points output by the VLA model is relatively large, and the smoothness of the trajectory formed by the multiple trajectory points output by the VLA model is poor. Therefore, at least one pair of adjacent trajectory points can be interpolated to obtain interpolated points. After interpolation, there can be multiple interpolated points between the at least one pair of adjacent trajectory points, thereby increasing the number of points, filling the position gaps of discrete trajectory points, effectively mitigating the positional abrupt changes of adjacent trajectory points, achieving smooth and continuous position of adjacent trajectory points, and thus improving the smoothness of the trajectory. Furthermore, the target task can be a task that the robot needs to perform. For example, when the robot is a humanoid robot, the target task can be a task of pulling out a drawer or retrieving an object. When the robot is a robot dog, the target task can be a patrol task. This embodiment of the invention does not specifically limit this.

[0051] It should be emphasized that any interpolation method in the related technologies is applicable to the embodiments of the present invention, such as nonlinear interpolation, cosine interpolation, polynomial interpolation, etc. For clarity, the specific process of interpolating at least one pair of adjacent trajectory points among multiple trajectory points will be described in other embodiments, and will not be elaborated here.

[0052] In addition, a trajectory point can also be referred to as the position point of the robot's end effector. For example, when the gripper at the end of a humanoid robot's arm performs a target task, the position point of the gripper can be considered as a trajectory point. Or, when the robotic arm gripper of a robot dog performs a target task, the position point of the robotic arm gripper can be considered as a trajectory point.

[0053] S202, In the case of the existence of the first trajectory, based on the first trajectory, perform predetermined optimization processing on the position information of at least one target point among multiple trajectory points and interpolation points;

[0054] The first trajectory is: the trajectory obtained before the output of multiple trajectory points for the target task, and the predetermined optimization process is a process to optimize the smoothness of the trajectory.

[0055] Step S202 addresses the case where a first trajectory exists. This can be considered as the existence of a trajectory obtained prior to the current output of multiple trajectory points for the target task, or as the existence of a historical trajectory. Accordingly, in step S202, the multiple trajectory points output by the VLA model for the robot's target task are not the first output trajectory points. Furthermore, in the case of a first trajectory, the obtained historical trajectory (first trajectory) can be used as a reference for optimizing trajectory smoothness (pre-defined optimization processing).

[0056] It is important to emphasize that each target point is either a trajectory point or an interpolation point. Correspondingly, at least one target point among multiple trajectory points and interpolation points may include all or some of the trajectory points and interpolation points. In one implementation, the position information of any target point may include the three-dimensional coordinates of the robot when it reaches the target point, or it may include the joint angles of the robot when it reaches the target point. For clarity, the specific process of performing predetermined optimization processing on the position information of at least one target point among multiple trajectory points and interpolation points based on the first trajectory will be described in other embodiments and will not be elaborated upon here.

[0057] In one implementation, after interpolating at least one pair of adjacent trajectory points from a plurality of trajectory points to obtain interpolated points, the method further includes step A1:

[0058] Step A1: In the absence of a first trajectory, a trajectory containing multiple trajectory points and interpolation points is generated based on the position information of multiple trajectory points and the position information of interpolation points. This trajectory serves as the second trajectory to be used by the robot to perform the target task, corresponding to the multiple trajectory points output in this step.

[0059] Step A1 addresses the case where a first trajectory does not exist. This can be interpreted as the absence of a trajectory prior to the current output of multiple trajectory points for the target task, or the absence of historical trajectories. Accordingly, the multiple trajectory points output by the VLA model for the robot's target task in step A1 are the first output trajectory points. Furthermore, in the absence of a first trajectory, the multiple trajectory points are the first output trajectory points. Based on the position information of these multiple trajectory points and the position information of the interpolation points, a trajectory containing multiple trajectory points and interpolation points can be generated and used as the second trajectory to be utilized by the robot for executing the target task, corresponding to the multiple trajectory points output in this step. Additionally, after step A1, if the VLA model outputs multiple trajectory points again for the robot's target task, the second trajectory to be utilized determined in step A1 can be used as the first trajectory.

[0060] It is evident that, in the absence of a first trajectory, a trajectory containing multiple trajectory points and interpolation points can be generated based on the position information of multiple trajectory points and the position information of interpolation points. This trajectory serves as the second trajectory to be used by the robot to perform the target task, corresponding to the multiple trajectory points output in this instance. In the absence of a first trajectory, the second trajectory to be used can also be determined, thus providing a basis for trajectory generation.

[0061] S203, based on the current trajectory points and current interpolation points existing after predetermined optimization processing, generate the second trajectory to be used by the robot to perform the target task, corresponding to the multiple trajectory points output in this time.

[0062] Understandably, after performing pre-optimization processing on the position information of at least one target point among multiple trajectory points and interpolation points, the current trajectory point and the current interpolation point existing after pre-optimization processing can be obtained. Furthermore, the current trajectory point and the current interpolation point existing after pre-optimization processing can be connected to generate a second trajectory corresponding to the multiple trajectory points output in this iteration, which is to be utilized by the robot to perform the target task.

[0063] The technical solution provided by this invention can, after the VLA model outputs multiple trajectory points for the robot's target task at any time, perform interpolation processing on at least one pair of adjacent trajectory points among the multiple trajectory points to obtain interpolation points. If a first trajectory exists, based on the first trajectory, perform predetermined optimization processing on the position information of at least one target point among the multiple trajectory points and interpolation points respectively. Subsequently, based on the current trajectory points and the current interpolation points that exist after the predetermined optimization processing, generate a second trajectory corresponding to the multiple trajectory points output this time, which is to be used by the robot to perform the target task. As can be seen, the embodiments of the present invention can not only perform interpolation processing on the VLA model and multiple trajectory points output for any given time for the robot's target task, but also perform predetermined optimization processing on the position information of at least one target point among the multiple trajectory points and interpolation points based on the first trajectory (the trajectory obtained before the output of multiple trajectory points for the target task). This improves the smoothness of the second trajectory to be used by combining the two aspects. Specifically, the interpolation processing can insert interpolation points between at least one pair of adjacent trajectory points, increasing the number of points, filling the position gaps between discrete trajectory points, effectively mitigating abrupt position changes of adjacent trajectory points, and achieving smooth and continuous position of adjacent trajectory points, thereby improving the smoothness of the trajectory. Furthermore, the historical trajectory (i.e., the first trajectory) is used as a reference for predetermined optimization processing (i.e., trajectory smoothing optimization of the target point). This ensures that the trajectory to be generated this time can continue the pattern of the historical trajectory (given certain position constraints) relative to the historical trajectory, mitigating abrupt position changes of the generated trajectory compared to the historical trajectory, and improving the smoothness of the generated trajectory. Finally, combining the two aspects can improve the smoothness of the generated trajectory.

[0064] Optionally, in one implementation, as shown in Figure 3(a), based on the first trajectory, the position information of at least one target point among multiple trajectory points and interpolation points is subjected to predetermined optimization processing, including:

[0065] S301, for each target point among multiple trajectory points and interpolation points, determine the specified trajectory point corresponding to the target point from the trajectory points included in the first trajectory;

[0066] Among them, the designated trajectory point corresponding to each target point is: a trajectory point with matching time information to the target point; for the target point and the trajectory points included in the first trajectory, the time information of each point represents the time of arrival at the point during the task execution process;

[0067] It is understood that each target point corresponds to a designated trajectory point. The designated trajectory point for each target point is a trajectory point with matching time information. For example, the time information of the target point can be the same as the time information of the designated trajectory point corresponding to the target point, or the time information of the target point can be close to the time information of the designated trajectory point corresponding to the target point. Furthermore, the designated trajectory point corresponding to any target point is a trajectory point included in the first trajectory, and the number of designated points corresponding to any target point can be one or more.

[0068] In one implementation, determining the designated trajectory point corresponding to the target point from the trajectory points included in the first trajectory includes at least one of steps B1 and B2:

[0069] Step B1: In response to the existence of a trajectory point with the same time information as the target point among the trajectory points included in the first trajectory, the trajectory point with the same time information as the target point is determined as the designated trajectory point corresponding to the target point.

[0070] Understandably, we can first determine from the trajectory points included in the first trajectory whether there exists a trajectory point with the same time information as the target point. The trajectory points included in the first trajectory are those obtained before the current output of multiple trajectory points for the target task. Determining whether there exists a trajectory point with the same time information as the target point will result in two possibilities: either it exists, or it does not. For example, if the time information of target point 1 is 'a', then we can determine from the trajectory points included in the first trajectory whether there exists a trajectory point with time information 'a'. Alternatively, we can iterate through the time information of the trajectory points included in the first trajectory to determine whether there exists a trajectory point with the same time information as the target point.

[0071] Step B1 is in response to the case that there is a trajectory point with the same time information as the target point among the trajectory points included in the first trajectory. The trajectory point with the same time information as the target point can be directly determined as the designated trajectory point corresponding to the target point. For example, the time information of the target point 1 is 'a'. Among the trajectory points included in the first trajectory, trajectory points 2 and 3 with time information 'a' are found. Trajectory points 2 and 3 can be determined as the designated trajectory points corresponding to the target point 1.

[0072] Step B2: In response to the absence of a trajectory point with the same time information as the target point among the trajectory points included in the first trajectory, a trajectory point that meets a predetermined matching condition is selected from the trajectory points included in the first trajectory as the designated trajectory point corresponding to the target point; the matching condition includes: the time difference between the time information of the trajectory point and the time information of the target point is less than a predetermined threshold.

[0073] Step B2 is in response to the situation where no trajectory point in the first trajectory has the same time information as the target point. Therefore, a trajectory point whose time information differs from the target point's time information by less than a predetermined threshold can be selected from the trajectory points in the first trajectory as the designated trajectory point corresponding to the target point. The trajectory point whose time information differs from the target point's time information by less than the predetermined threshold can be the trajectory point whose time information is closest to the target point. For example, if the predetermined threshold is 0.1s and the target point 5's time information is b, and it is determined that no trajectory point with time information b exists in the first trajectory, then a trajectory point whose time information differs from time information b by less than 0.1s can be selected from the trajectory points in the first trajectory as the designated trajectory point corresponding to the target point 5.

[0074] In another implementation, different strategies can be adopted for different target points to determine the designated trajectory points corresponding to the target points. For target points that are trajectory points, trajectory points with the same time information as the target point can be determined from the trajectory points included in the first trajectory and used as the designated trajectory points corresponding to the target point. For target points that are interpolation points, trajectory points whose time information is less than a predetermined threshold can be directly selected from the trajectory points included in the first trajectory and used as the designated trajectory points corresponding to the target point. Of course, the above is only an illustrative example and does not specifically limit the embodiments of the present invention.

[0075] As can be seen, in response to the existence of a trajectory point with the same time information as the target point among the trajectory points included in the first trajectory, the trajectory point with the same time information as the target point is determined as the designated trajectory point corresponding to the target point. In response to the absence of a trajectory point with the same time information as the target point among the trajectory points included in the first trajectory, a trajectory point that meets the predetermined matching conditions is selected from the trajectory points included in the first trajectory as the designated trajectory point corresponding to the target point. The designated trajectory point corresponding to the target point can be determined in all cases, thus providing a basis for trajectory generation.

[0076] S302, determine the weight of the target point and the weight of the specified trajectory point corresponding to the target point, and based on the determined weights, perform weighted fusion of the position information of the target point and the position information of the specified trajectory point corresponding to the target point to obtain the position information of the target point after predetermined optimization.

[0077] It is understandable that each point has a corresponding weight, and the weight of any point can be related to its order in the trajectory. Based on the determined weights, the position information of the target point and the position information of the specified trajectory point corresponding to the target point can be weighted and calculated to obtain the position information of the target point after predetermined optimization. For example, the weighted fusion process may include: calculating the product of the weight of the specified trajectory point corresponding to the target point and the position information of the specified trajectory point corresponding to the target point to obtain the weighted position information of the specified trajectory point corresponding to the target point; calculating the product of the weight of the target point and the position information of the target point to obtain the weighted position information of the target point; subsequently, calculating the sum of the weighted position information of the target point and the weighted position information of the specified trajectory point corresponding to the target point to obtain a first result; calculating the sum of the weight of the target point and the weight of the specified trajectory point corresponding to the target point to obtain a second result; and calculating the ratio of the first result to the second result to obtain the optimized position information of the target point. In addition, if there are multiple specified trajectory points corresponding to the target point, the weighted position information of the specified trajectory points corresponding to the target point can be considered as the sum of the weighted position information of the multiple specified trajectory points; if there is only one specified trajectory point corresponding to the target point, the weighted position information of the specified trajectory point corresponding to the target point can be the product of the position information of the specified trajectory point and the weight of the specified trajectory point.

[0078] To better understand the above weighted fusion method, the following explanation uses the formula as an example:

[0079] ;

[0080] in, The optimized location information of the target point, The VLA model outputs multiple trajectory points in the i-th iteration. Representing time, This can be considered the first result mentioned above. This can be considered the second result mentioned above. Let be a smooth function used to characterize the weight at any point. Let be a function used to characterize the location information of any point.

[0081] Of course, the weighted fusion method described above is only an example, and other methods can also be used to achieve weighted fusion.

[0082] To better understand the above content, the following explanation is provided with reference to the accompanying diagram, as shown in Figure 3(b):

[0083] In the first inference of the VLA model, k trajectory points can be inferred. The position information of the k trajectory points obtained in this inference is as follows: , ..., The time information of the k trajectory points obtained in this inference is as follows: , ..., Interpolation is performed on k trajectory points, and the resulting interpolated points and the k trajectory points can be represented as f 1（t） The weights of the interpolation points and the k trajectory points are λ. 1（t） In the i-th inference of the VLA model, k trajectory points can be inferred. The position information of the k trajectory points obtained in this inference is as follows: , ..., The time information of the k trajectory points obtained in this inference is as follows: , ..., Interpolation is performed on k trajectory points, and the resulting interpolated points and the k trajectory points can be represented as f i（t） The weights of the interpolation points and the k trajectory points are λ. i（t） Furthermore, based on the weights and position information of trajectory points obtained from each inference, weighted fusion can be performed to obtain the position information p of multiple target points after predetermined optimization.

[0084] In one implementation, for the target point and the designated trajectory point corresponding to the target point, the weight of each point is set based on the order in which the point is located in its respective trajectory, and the trajectory point located at the first or last position has the smallest weight.

[0085] If the point is a designated trajectory point corresponding to the target point, then the trajectory to which the point belongs is the first trajectory; if the point is the target point, then the trajectory to which the point belongs is a trajectory formed by multiple trajectory points and interpolation points.

[0086] For each point, its weight can be considered to be based on its position within the trajectory. Points in the middle of the trajectory have higher weights, while points at the beginning or end have the lowest weights, which can be 0. For example, target point 1 is at the beginning of the trajectory, and target point 2 is the next point after it. Therefore, target point 1's weight can be 0, and target point 2's weight can be 0.2.

[0087] In one implementation, a function can be used. Determine the weight of each point. It can be considered as a convex, smooth function. The x-axis represents the time information of multiple trajectory points output by the VLA model for any given time in the robot's target task, with a value range of []. , ],in, The temporal information of the first trajectory point among the multiple trajectory points output by the VLA model in this iteration is represented. The time information of the last trajectory point among the multiple trajectory points output by the VLA model in this iteration is represented. The y-axis represents the weight, with a value range of [0, 1). Since the trajectory points located at the first or last position have the smallest weight, , Of course, this implementation method is only presented as an example.

[0088] In another implementation, the target point is given the largest weight, while the corresponding trajectory point is given a smaller weight.

[0089] As can be seen, the weight of each point is set based on its position in the trajectory, with the first or last trajectory point having the smallest weight. This reduces the impact of the VLA model's inference time on trajectory generation. The first and last trajectory points output by the VLA model have the smallest weight, which improves the continuity between the robot's current position and the multiple trajectory points output by the VLA model, and also reduces the probability of the robot performing abnormal tasks.

[0090] As can be seen, the embodiments of the present invention can determine the designated trajectory point corresponding to the target point from the first trajectory, and based on the determined weight, perform weighted fusion of the position information of the target point and the position information of the designated trajectory point corresponding to the target point. The position information of the target point after predetermined optimization can be considered as the position information generated by combining the position information of the target point and the position information of the trajectory point in the historical trajectory that has matching time information with the target point. Furthermore, the historical trajectory (i.e., the first trajectory) is used as a reference for predetermined optimization (i.e., trajectory smoothing optimization of the target point). This allows the trajectory to be generated this time to continue the pattern of the historical trajectory (giving certain position constraints) relative to the historical trajectory, alleviate the positional abruptness of the generated trajectory compared to the historical trajectory, and improve the smoothness of the generated trajectory.

[0091] Optionally, in one implementation, before interpolating at least one pair of adjacent trajectory points among multiple trajectory points to obtain the interpolated point, the method further includes step C1:

[0092] Step C1: Determine the state information of at least one pair of adjacent trajectory points among multiple trajectory points;

[0093] Accordingly, interpolation is performed on at least one pair of adjacent trajectory points among multiple trajectory points to obtain interpolated points, including step D1:

[0094] Step D1: Based on the state information of at least one pair of adjacent trajectory points among multiple trajectory points, perform interpolation processing on at least one pair of adjacent trajectory points among multiple trajectory points to obtain interpolated points.

[0095] Regarding step C1, the state information of at least one pair of adjacent trajectory points among the multiple trajectory points may include: position information, time information, velocity, and acceleration. Of course, it may also include other information, such as the robot's posture information. Furthermore, in this embodiment of the invention, the state information of multiple trajectory points can be determined first, and then the state information of at least one pair of adjacent trajectory points among the multiple trajectory points can be determined.

[0096] The following describes, with reference to the accompanying drawings, an exemplary method for determining the state information of multiple trajectory points, such as... Figure 4 As shown:

[0097] S401, in response to the VLA model outputting multiple trajectory points, obtain the position information of multiple trajectory points;

[0098] It is understandable that the positional information of multiple trajectory points can be output by the VLA model.

[0099] S402, based on the position information of multiple trajectory points, the time information of multiple trajectory points can be determined;

[0100] The time information of multiple trajectory points can be set based on the order of the trajectory points within their respective trajectories. The time information of trajectory points with earlier order is earlier than that of trajectory points with later order. The details of determining the time information of multiple trajectory points will be introduced in subsequent embodiments and will not be elaborated here. For example, the VLA model outputs four trajectory points in this instance, in the order of 1-4. The time information of trajectory point 1 can be determined as 10min1s, the time information of trajectory point 2 as 10min4s, the time information of trajectory point 3 as 10min7s, and the time information of trajectory point 4 as 10min10s.

[0101] S403, calculates the velocity of multiple trajectory points based on the position information and time information of multiple trajectory points;

[0102] It is understandable that after determining the time information of multiple trajectory points, the velocities of these multiple trajectory points can be calculated. To better understand the content of step S403, the following is an illustrative explanation using formulas:

[0103] ;

[0104] in, Let j be the velocity of the trajectory point j. For the position information of trajectory point j+1, For the position information of trajectory point j, For the time information of trajectory point j+1, This refers to the time information of trajectory point j.

[0105] Of course, the above formula for calculating the velocity of a trajectory point is only an example, and other methods can also be used to calculate the velocity of a trajectory point.

[0106] S404, calculates the acceleration of multiple trajectory points based on the velocity and time information of multiple trajectory points;

[0107] It is understandable that after calculating the velocities of multiple trajectory points, the accelerations of these points can be calculated. To better understand step S404, the following example, using formulas, provides an illustrative explanation:

[0108] ;

[0109] in, Let j be the acceleration of the trajectory point j. Let j be the velocity of the trajectory point j. Let j-1 be the velocity of the trajectory point. For the time information of trajectory point j-1, This refers to the time information of trajectory point j.

[0110] Of course, the formula for calculating the acceleration of a trajectory point is only an example and other methods can also be used to calculate the acceleration of a trajectory point.

[0111] In addition, based on the position information, time information, velocity and acceleration of the multiple trajectory points determined above, the state information of the multiple trajectory points can be determined.

[0112] Regarding step D1, at least one pair of adjacent trajectory points can be interpolated using the position information, time information, velocity, and acceleration of at least one pair of adjacent trajectory points to obtain interpolated points. The specific interpolation process will be described in subsequent embodiments.

[0113] As can be seen, after interpolating adjacent trajectory points, the state information of at least one pair of adjacent trajectory points among multiple trajectory points can be determined. Based on the determined state information, interpolation is performed on at least one pair of adjacent trajectory points among multiple trajectory points to obtain interpolated points, thus providing a basis for the subsequent generation of smooth trajectories.

[0114] Optionally, in one implementation, the state information of at least one pair of adjacent trajectory points among the multiple trajectory points includes: position information, time information, velocity, and acceleration;

[0115] Accordingly, based on the state information of at least one pair of adjacent trajectory points among multiple trajectory points, interpolation processing is performed on at least one pair of adjacent trajectory points among multiple trajectory points to obtain interpolated points, including steps D11-D13:

[0116] Step D11: For each pair of adjacent trajectory points included in at least one pair of adjacent trajectory points among multiple trajectory points, substitute the state information of the pair of adjacent trajectory points into the polynomial coefficient matrix to obtain the polynomial coefficients.

[0117] Step D12: Based on the polynomial coefficients, construct a formula to determine the location information of the interpolation point located in the target time period, and obtain the target formula; the target time period is the time interval represented by the time information of the pair of adjacent trajectory points.

[0118] Step D13: Substitute multiple times within the target time period into the target formula to determine the position information of the interpolation points corresponding to the pair of adjacent trajectory points, so as to obtain the interpolation points between the pair of adjacent trajectory points.

[0119] Regarding step D11, the polynomial coefficient matrix can be used to calculate the polynomial coefficients. In this implementation, the polynomial coefficient matrix can be used to calculate the coefficients of a fifth-order polynomial. Also, for each pair of adjacent trajectory points included in at least one pair of adjacent trajectory points among a plurality of trajectory points, the state information of the pair of adjacent trajectory points is substituted into the polynomial coefficient matrix to calculate the polynomial coefficients. Any pair of adjacent trajectory points includes two adjacent trajectory points.

[0120] For example, the components in the m-th row of the polynomial coefficient matrix are:

[0121] ;

[0122] in, The coefficient matrix of the polynomial The components in the m-th row are represented by m, j, and i, where m represents the dimension, j represents the trajectory point j, and i represents the number of trajectory points output by the VLA model in the i-th iteration, i being the number of inferences of the VLA model. This represents the positional information of trajectory point j among multiple trajectory points in the i-th output of the VLA model in dimension m. Characterizes the velocity of trajectory point j among multiple trajectory points in the i-th output of the VLA model in dimension m. The acceleration of trajectory point j among multiple trajectory points in the i-th output of the VLA model in dimension m is represented. This represents the position information of trajectory point j+1 among multiple trajectory points in the i-th output of the VLA model in dimension m. The velocity of trajectory point j+1 among multiple trajectory points output by the VLA model in dimension m is represented. The acceleration of trajectory point j+1 among multiple trajectory points in the i-th output of the VLA model in dimension m is represented. The temporal information of trajectory point j among multiple trajectory points output by the VLA model in the i-th iteration is represented. T represents the time information of trajectory point j+1 among multiple trajectory points output by the VLA model in the i-th time, and T represents the transpose.

[0123] Furthermore, the number of rows in the polynomial coefficient matrix is ​​related to the dimension of the position information. When the position information includes three-dimensional coordinate information, the number of rows in the polynomial coefficient matrix can be 3. When the position information includes three-dimensional coordinate information and rotation angle, the number of rows in the polynomial coefficient matrix can be 6. When the position information includes joint angle information when the robot reaches the target point, the number of rows in the polynomial coefficient matrix can also be the same as the number of joint axes that the robot needs to rotate when performing the target task.

[0124] Regarding step D12, after calculating the polynomial coefficients, a formula can be constructed to determine the location information of the interpolation point located in the target time period, thereby obtaining the target formula. The target time period is the time interval represented by the time information of the pair of adjacent trajectory points. In other words, the target formula can be a formula for two adjacent trajectory points. By substituting the time within the target time period into the target formula, the location information of the interpolation point within the target time period can be determined.

[0125] For example, taking a fifth-degree polynomial as an example, the target formula is:

[0126] ;

[0127] in, For the location information of the interpolation points, Let be the calculated polynomial coefficients, and t be the time within the target time period. The range of values ​​for t is ( ). , ), T is the transpose; and, by transforming the above objective formula, we can obtain the formula: ,in, - These are the polynomial coefficients.

[0128] For step D13, multiple moments within the target time period can be substituted into the target formula. Based on the target formula, the position information of the interpolation points corresponding to the pair of adjacent trajectory points can be determined. The determined position information of the interpolation points is all located between the pair of adjacent trajectory points, thus obtaining the interpolation points between the pair of adjacent trajectory points.

[0129] As can be seen, in this embodiment of the invention, the state information of the pair of adjacent trajectory points can be substituted into the polynomial coefficient matrix to obtain polynomial coefficients. Based on the polynomial coefficients, a formula for determining the position information of the interpolation point located in the target time period is constructed to obtain the target formula. Multiple moments within the target time period are substituted into the target formula to determine the position information of the interpolation point corresponding to the pair of adjacent trajectory points, thereby obtaining the interpolation point between the pair of adjacent trajectory points. This achieves interpolation, reduces the interval between adjacent trajectory points, fills the position gaps of discrete trajectory points, effectively alleviates the positional abrupt changes of adjacent trajectory points, and achieves smooth and continuous position of adjacent trajectory points, thereby improving the smoothness of the generated trajectory.

[0130] Optionally, in one implementation, the determination of the time information of each of the multiple trajectory points includes method E1:

[0131] Method E1 determines the order of multiple trajectory points in their respective trajectories, and sets the time information of the multiple trajectory points based on the determined order to obtain the time information of each trajectory point; the time information of the trajectory point with the earlier order is earlier than the time information of the trajectory point with the later order.

[0132] Based on the VLA model, for the robot's target task, multiple trajectory points output in this iteration can be determined, their order within the trajectory, and their timing information can be set. This can be done by setting the timing information for each trajectory point individually, or by setting a time interval. It's important to emphasize that the timing information of earlier-ordered trajectory points is earlier than that of later-ordered points. For example, if the VLA model outputs five trajectory points (1-5), with trajectory point 1 being the first and trajectory point 5 the last, the timing information for trajectory points 1-5 can be set as follows: trajectory point 1: 2 min 10 s; trajectory point 2: 2 min 16 s; trajectory point 3: 2 min 20 s; trajectory point 4: 2 min 27 s; and trajectory point 5: 2 min 34 s.

[0133] It is evident that the method for determining the time information of multiple trajectory points can be to determine the order of the multiple trajectory points in their respective trajectories, and based on the determined order, set the time information of the multiple trajectory points to obtain the time information of each of the multiple trajectory points, thus providing a basis for the subsequent generation of smooth trajectories.

[0134] In one implementation, based on a determined order, the time information of multiple trajectory points is set to obtain the time information of each trajectory point, including step E11:

[0135] Step E11: Based on the determined order, set the time information of multiple trajectory points according to a predetermined time interval to obtain the time information of multiple trajectory points; the time difference represented by the time information of any two adjacent trajectory points is the predetermined time interval.

[0136] Because the interval between any two trajectory points output by the VLA model is relatively large, the predetermined time interval cannot be set to a small value. If the predetermined time interval is set too small, the speed and acceleration of multiple trajectory points will be excessively high. If the robot performs the target task at the calculated high speed and acceleration, it may damage the robot's hardware structure. For example, if the predetermined time interval is set to 10 seconds, and the VLA model outputs 5 trajectory points (a-e), with trajectory point a being the first and trajectory point e the last, the time information for trajectory points a-e is set as follows: trajectory point a = 1 minute 10 seconds, trajectory point b = 1 minute 20 seconds, trajectory point c = 1 minute 30 seconds, trajectory point d = 1 minute 40 seconds, and trajectory point e = 1 minute 50 seconds.

[0137] As can be seen, based on the determined order, the time information of multiple trajectory points can be set according to a predetermined time interval to obtain the time information of each trajectory point. Setting the time information of multiple trajectory points according to the time interval ensures that the time difference represented by the time information of any two adjacent trajectory points is the predetermined time interval, providing a basis for the subsequent generation of smooth trajectories.

[0138] Optionally, in one implementation, the method further includes step F1:

[0139] Step F1: The second trajectory to be used for the robot to perform the target task is sent to the robot's control terminal, so that the control terminal selects the trajectory corresponding to the unexecuted task in the target task from the second trajectory to be used based on the progress of the target task currently being performed by the robot, obtains the third trajectory, and controls the robot to move according to the third trajectory.

[0140] After determining the second trajectory to be utilized, it can be sent to the robot's control terminal. This allows the control terminal to identify unexecuted tasks within the target task based on the robot's current progress, select the trajectory corresponding to the unexecuted task from the second trajectory, and use this selected trajectory as the third trajectory. The control terminal can then control the robot to move according to this third trajectory to continue executing the unexecuted tasks within the target task. Alternatively, the trajectory generation device can select the trajectory corresponding to the unexecuted task from the second trajectory based on the robot's current progress, obtain the third trajectory, and directly send it to the control terminal.

[0141] As can be seen, the second trajectory to be used can be sent to the robot's control terminal. The control terminal can provide the robot with the progress of the target task it is currently executing. From the second trajectory to be used, the trajectory corresponding to the unexecuted task in the target task can be selected to obtain the third trajectory. The robot can then be controlled to move according to the third trajectory. This reduces the impact of the VLA model's inference time on trajectory generation, improves the continuity between the robot's current position and the multiple trajectory points output by the VLA model, and also reduces the probability of abnormalities occurring during robot task execution.

[0142] Based on the method embodiments described above, such as Figure 5 As shown, an embodiment of the present invention provides a trajectory generation device, which includes:

[0143] The interpolation processing module 510 is used to interpolate at least one pair of adjacent trajectory points among the multiple trajectory points after the vision-language-action VLA model outputs multiple trajectory points for the robot's target task at any time, so as to obtain interpolated points.

[0144] The pre-optimization module 520 is used to perform pre-optimization processing on the position information of at least one target point among multiple trajectory points and interpolation points based on the first trajectory when a first trajectory exists. The first trajectory is the trajectory obtained before the output of multiple trajectory points for the target task. The pre-optimization processing is a process for optimizing the smoothness of the trajectory.

[0145] The first generation module 530 is used to generate a second trajectory corresponding to the multiple trajectory points output in this operation, based on the current trajectory points and the current interpolation points after predetermined optimization processing, for the robot to perform the target task.

[0146] Optionally, the pre-optimization module includes: a determination submodule, used to determine a designated trajectory point corresponding to each target point from the trajectory points included in the first trajectory, for each target point among multiple trajectory points and interpolation points; the designated trajectory point corresponding to each target point is a trajectory point with matching time information to the target point; for multiple target points and trajectory points included in the first trajectory, the time information of each point represents the time of arrival at the point during task execution; and a weighted fusion submodule, used to determine the weight of the target point and the weight of the designated trajectory point corresponding to the target point, and based on the determined weight, to perform weighted fusion of the position information of the target point and the position information of the designated trajectory point corresponding to the target point to obtain the position information of the target point after pre-optimization processing.

[0147] Optionally, for the target point and the designated trajectory point corresponding to the target point, the weight of each point is set based on the order in which the point is located in its respective trajectory, and the trajectory point located at the first or last position has the smallest weight.

[0148] Optionally, the determining submodule is specifically configured to: in response to the existence of a trajectory point with the same time information as the target point among the trajectory points included in the first trajectory, determine the trajectory point with the same time information as the target point as the designated trajectory point corresponding to the target point; in response to the absence of a trajectory point with the same time information as the target point among the trajectory points included in the first trajectory, select a trajectory point that meets a predetermined matching condition from the trajectory points included in the first trajectory as the designated trajectory point corresponding to the target point; the matching condition includes: the time difference between the time information of the selected trajectory point and the time information of the target point is less than a predetermined threshold.

[0149] Optionally, the device further includes: a first determining module, configured to perform interpolation processing on at least one pair of adjacent trajectory points among a plurality of trajectory points, and determine the state information of at least one pair of adjacent trajectory points among a plurality of trajectory points before obtaining the interpolation point; and an interpolation processing module, including: an interpolation processing submodule, configured to perform interpolation processing on at least one pair of adjacent trajectory points among a plurality of trajectory points based on the state information of at least one pair of adjacent trajectory points among a plurality of trajectory points, and obtain the interpolation point.

[0150] Optionally, the state information of at least one pair of adjacent trajectory points among the multiple trajectory points includes: position information, time information, velocity, and acceleration; and / or, the interpolation processing submodule is specifically used for: for each pair of adjacent trajectory points included in at least one pair of adjacent trajectory points among the multiple trajectory points, substituting the state information of the pair of adjacent trajectory points into a polynomial coefficient matrix to obtain polynomial coefficients; based on the polynomial coefficients, constructing a formula for determining the position information of the interpolation point located in the target time period to obtain the target formula; the target time period is the time interval between the times represented by the time information of the pair of adjacent trajectory points; substituting multiple moments within the target time period into the target formula to determine the position information of the interpolation point corresponding to the pair of adjacent trajectory points to obtain the interpolation point between the pair of adjacent trajectory points.

[0151] Optionally, the method for determining the time information of each of the multiple trajectory points includes: determining the order of the multiple trajectory points in their respective trajectories, and setting the time information of the multiple trajectory points based on the determined order to obtain the time information of each of the multiple trajectory points; the time information of the trajectory point with the earlier order is earlier than the time information of the trajectory point with the later order.

[0152] Optionally, based on the determined order, the time information of multiple trajectory points is set to obtain the time information of each trajectory point, including: based on the determined order, the time information of multiple trajectory points is set according to a predetermined time interval to obtain the time information of multiple trajectory points; the time difference represented by the time information of any two adjacent trajectory points is the predetermined time interval.

[0153] Optionally, the device further includes: a sending module, used to send the second trajectory to be used for the robot to perform the target task to the robot's control terminal, so that the control terminal selects the trajectory corresponding to the unexecuted task in the target task from the second trajectory to be used based on the progress of the target task currently being performed by the robot, to obtain the third trajectory, and controls the robot to move according to the third trajectory.

[0154] Optionally, the device further includes: a second generation module, used to perform interpolation processing on at least one pair of adjacent trajectory points among the multiple trajectory points, and after obtaining interpolation points, generate a trajectory containing multiple trajectory points and interpolation points based on the position information of the multiple trajectory points and the position information of the interpolation points in the absence of a first trajectory, as a second trajectory to be used by the robot to perform the target task corresponding to the multiple trajectory points output in this output.

[0155] This invention also provides an electronic device, such as... Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0156] Memory 603 is used to store computer programs;

[0157] The processor 601, when executing the program stored in the memory 603, implements any of the above trajectory generation methods.

[0158] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0159] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0160] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0161] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0162] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and the computer program, when executed by a processor, implements any of the above-described trajectory generation methods.

[0163] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the trajectory generation methods described above.

[0164] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0165] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0166] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0167] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A trajectory generation method, characterized in that, The method includes: After the Visual-Language-Motion (VLA) model outputs multiple trajectory points for the robot's target task, at least one pair of adjacent trajectory points among the multiple trajectory points are interpolated to obtain interpolated points. In the presence of a first trajectory, based on the first trajectory, the position information of at least one target point among the plurality of trajectory points and the interpolation points is subjected to a predetermined optimization process. The first trajectory is: the trajectory obtained before the output of the plurality of trajectory points for the target task. The predetermined optimization process is a process for optimizing the smoothness of the trajectory. Based on the current trajectory points and interpolation points existing after the predetermined optimization process, a second trajectory to be utilized by the robot to perform the target task is generated, corresponding to the multiple trajectory points output in this instance.

2. The method according to claim 1, characterized in that, The step of performing predetermined optimization processing on the position information of at least one target point among the plurality of trajectory points and the interpolation points based on the first trajectory includes: For each target point among the plurality of trajectory points and the interpolation points, a designated trajectory point corresponding to the target point is determined from the trajectory points included in the first trajectory; the designated trajectory point corresponding to each target point is: a trajectory point with matching time information to the target point; for the target point and the trajectory points included in the first trajectory, the time information of each point represents the time of arrival at the point during the task execution process; Determine the weight of the target point and the weight of the specified trajectory point corresponding to the target point. Based on the determined weights, perform weighted fusion of the position information of the target point and the position information of the specified trajectory point corresponding to the target point to obtain the position information of the target point after predetermined optimization. Among them, for the target point and the designated trajectory point corresponding to the target point, the weight of each point is set according to the order in which the point is located in the trajectory, and the trajectory point located at the first or last position has the smallest weight.

3. The method according to claim 2, characterized in that, Determining the specified trajectory point corresponding to the target point from the trajectory points included in the first trajectory includes at least one of the following: If, in response to the existence of a trajectory point among the trajectory points included in the first trajectory that has the same time information as the target point, the trajectory point that has the same time information as the target point is determined as the designated trajectory point corresponding to the target point; If there is no trajectory point with the same time information as the target point among the trajectory points included in the first trajectory, then a trajectory point that meets a predetermined matching condition is selected from the trajectory points included in the first trajectory as the designated trajectory point corresponding to the target point; the matching condition includes: the time difference between the time information of the selected trajectory point and the time information of the target point is less than a predetermined threshold.

4. The method according to any one of claims 1 to 3, characterized in that, Before performing interpolation on at least one pair of adjacent trajectory points among the plurality of trajectory points to obtain interpolated points, the method further includes: Determine the state information of at least one pair of adjacent trajectory points among the plurality of trajectory points; The step of interpolating at least one pair of adjacent trajectory points from the plurality of trajectory points to obtain interpolated points includes: Based on the state information of at least one pair of adjacent trajectory points among the plurality of trajectory points, interpolation processing is performed on at least one pair of adjacent trajectory points among the plurality of trajectory points to obtain interpolated points.

5. The method according to claim 4, characterized in that, The state information of at least one pair of adjacent trajectory points among the plurality of trajectory points includes: position information, time information, velocity, and acceleration; and / or, The step of interpolating at least one pair of adjacent trajectory points based on the state information of at least one pair of adjacent trajectory points among the plurality of trajectory points to obtain interpolated points includes: For each pair of adjacent trajectory points included in at least one pair of adjacent trajectory points among the plurality of trajectory points, the state information of the pair of adjacent trajectory points is substituted into the polynomial coefficient matrix to obtain the polynomial coefficients. Based on the polynomial coefficients, a formula is constructed to determine the position information of the interpolation point located in the target time period, thus obtaining the target formula; the target time period is the time interval represented by the time information of the pair of adjacent trajectory points. Substitute multiple moments within the target time period into the target formula to determine the position information of the interpolation points corresponding to the pair of adjacent trajectory points, so as to obtain the interpolation points between the pair of adjacent trajectory points.

6. The method according to any one of claims 2 to 5, characterized in that, The methods for determining the time information of each of the multiple trajectory points include: The order of the multiple trajectory points in their respective trajectories is determined, and the time information of the multiple trajectory points is set based on the determined order to obtain the time information of each of the multiple trajectory points; the time information of the trajectory points with earlier order is earlier than the time information of the trajectory points with later order.

7. The method according to claim 6, characterized in that, The step of setting the time information of the multiple trajectory points based on the determined order to obtain the time information of each of the multiple trajectory points includes: Based on the determined order, the time information of the multiple trajectory points is set according to a predetermined time interval to obtain the time information of the multiple trajectory points; the time difference represented by the time information of any two adjacent trajectory points is the predetermined time interval.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The second trajectory to be used by the robot to perform the target task is sent to the robot's control terminal, so that the control terminal selects the trajectory corresponding to the unexecuted task in the target task from the second trajectory to be used based on the progress of the target task currently being performed by the robot, obtains the third trajectory, and controls the robot to move according to the third trajectory; And / or, After interpolating at least one pair of adjacent trajectory points from the plurality of trajectory points to obtain interpolated points, the method further includes: In the absence of the first trajectory, a trajectory containing the multiple trajectory points and the interpolation points is generated based on the position information of the multiple trajectory points and the position information of the interpolation points. This trajectory serves as the second trajectory to be used by the robot to perform the target task, corresponding to the multiple trajectory points output in this instance.

9. A trajectory generation device, characterized in that, The device includes: The interpolation processing module is used to interpolate at least one pair of adjacent trajectory points among the multiple trajectory points after the vision-language-action VLA model outputs multiple trajectory points for the robot's target task at any time, so as to obtain interpolated points. A pre-optimization module is used to perform pre-optimization processing on the position information of at least one target point among the plurality of trajectory points and the interpolation points, based on the first trajectory, when a first trajectory exists. The first trajectory is the trajectory obtained before the output of the plurality of trajectory points for the target task. The pre-optimization processing is a process for optimizing the smoothness of the trajectory. The first generation module is used to generate a second trajectory to be used by the robot to perform the target task, corresponding to the multiple trajectory points output in this output, based on the current trajectory points and the current interpolation points that exist after the predetermined optimization processing.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method of any one of claims 1 to 8.

Images