Robot motion control method and apparatus, and computer device and storage medium

Abstract

A robot motion control method and apparatus, and a computer device and a storage medium. The robot motion control method comprises: acquiring a path parameterization model (401), wherein the path parameterization model is obtained by means of performing training on the basis of sample trajectory data, and the sample trajectory data comprises respective sample coordinates of at least two sample path points in a sample path, and respective sample timestamps; acquiring a first path, and acquiring respective first coordinates of at least two first path points in the first path (402); and determining first trajectory data by means of the path parameterization model and on the basis of the first coordinates, wherein the first trajectory data comprises first timestamps respectively corresponding to the at least two first path points (403), and a robot arrives at the at least two first path points at first path point moments indicated by the first timestamps. The robot motion control method ensures that trajectory data conforms to the motion rule of a training object, and improves the rationality of the trajectory data; thus, the motion of a robot better matches motion habits of the training object.

Description

Robot motion control method, device, computer equipment and storage medium

[0001] This application claims priority to Chinese patent application No. 202410174125.X, filed on February 6, 2024, entitled “Path parameterization method, device, computer equipment and storage medium”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The embodiments of the present application relate to the field of computer technology, and in particular to a method and apparatus for controlling robot motion, a computer device, and a storage medium. Background Art

[0003] With the development of computer technology and the widespread application of robots, safe and reasonable robot motion planning has become a research hotspot, and path parameterization is a crucial aspect of robot motion planning. Path parameterization involves converting the path data of a given robot path into trajectory data while satisfying system constraints. Trajectory data contains not only the path data but also the timestamps of each waypoint along the path when the robot passes through it.

[0004] A commonly used control method for robot motion in related technologies is TOPP (Time-Optimal Path Parameterization). TOPP generally aims to minimize the time it takes to pass through a path so that the robot can pass through the path quickly.

[0005] However, TOPP only focuses on the time of passing the path, resulting in the trajectory data obtained after the path data is converted not conforming to the movement law of humans passing the path, and the rationality of the trajectory data is poor. Summary of the Invention

[0006] The present invention provides a method, apparatus, computer device, and storage medium for controlling robot motion. The trajectory data generated by the path parameterization model includes the characteristics of the training object, is more consistent with the motion laws of the virtual object, and improves the rationality of the trajectory data. The technical solution is as follows:

[0007] In one aspect, a method for controlling robot motion is provided, the method comprising:

[0008] Obtaining a path parameterized model, the path parameterized model trained based on sample trajectory data, the sample trajectory data including sample coordinates and sample timestamps of at least two sample waypoints in the sample path, the sample timestamps indicating the sample waypoint moments when the training subject respectively arrives at the at least two sample waypoints while traversing the sample path;

[0009] Acquire a first path, and acquire first coordinates of at least two first waypoints in the first path;

[0010] First trajectory data is determined based on the first coordinates using the path parameterized model, where the first trajectory data includes first timestamps corresponding to each of the at least two first waypoints. The first trajectory data is used to control the robot to perform movement on the first path so that the robot reaches the at least two first waypoints at the first waypoint times indicated by the first timestamps.

[0011] In another aspect, a method for controlling robot motion is provided, the method comprising:

[0012] Obtaining a path parameterized model, the path parameterized model trained based on sample trajectory data, the sample trajectory data including sample coordinates and sample timestamps of at least two sample waypoints in the sample path, the timestamps indicating the sample waypoint moments when the training subject respectively arrived at the at least two sample waypoints while traversing the sample path;

[0013] Obtain a second path and obtain at least two second timestamps;

[0014] Second trajectory data is determined based on the second timestamp using the path parameterization model, where the second trajectory data includes second coordinates corresponding to at least two second waypoints, respectively. The second trajectory data is used to control the robot to move on the second path so that the robot reaches the at least two second waypoints at the second waypoint times indicated by the second timestamp.

[0015] In another aspect, a device for controlling robot motion is provided, the device comprising:

[0016] a model acquisition module, configured to acquire a path parameterized model, the path parameterized model being trained based on sample trajectory data, the sample trajectory data including sample coordinates and sample timestamps of at least two sample waypoints in a sample path, the sample timestamps indicating the sample waypoint moments when the training subject respectively reaches the at least two sample waypoints while traversing the sample path;

[0017] a path acquisition module, configured to acquire a first path and obtain first coordinates of at least two first waypoints in the first path;

[0018] a path parameterization module, configured to determine first trajectory data based on the first coordinates, the first trajectory data including first timestamps corresponding to each of the at least two first waypoints, the first trajectory data being used to control the robot to perform movement on the first path so that the robot reaches the at least two first waypoints at first waypoint times indicated by the first timestamps.

[0019] In another aspect, a device for controlling robot motion is provided, the device comprising:

[0020] a model acquisition module, configured to acquire a path parameterized model, the path parameterized model being trained based on sample trajectory data, the sample trajectory data including sample coordinates and sample timestamps of at least two sample waypoints in a sample path, the timestamps indicating the sample waypoint moments at which the training subject respectively reaches the at least two sample waypoints while traversing the sample path;

[0021] A path acquisition module, configured to acquire a second path and at least two second timestamps;

[0022] a path parameterization module, configured to determine, using the path parameterization model and based on the second timestamp, second trajectory data, wherein the second trajectory data includes second coordinates corresponding to at least two second waypoints, respectively; and the second trajectory data is used to control the robot to move along the second path so that the robot reaches the at least two second waypoints at the second waypoint times indicated by the second timestamps.

[0023] On the other hand, a computer device is provided, which includes a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed by the robot motion control method as described in the above aspects.

[0024] On the other hand, a computer-readable storage medium is provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the operations performed by the robot motion control method as described in the above aspects.

[0025] On the other hand, a computer program product is provided, comprising a computer program, wherein the computer program is loaded and executed by a processor to implement the operations performed by the robot motion control method as described in the above aspects.

[0026] In an embodiment of the present application, a path parameterization model is trained based on sample trajectory data. Since the sample trajectory data includes sample coordinates and sample timestamps of at least two sample waypoints in the sample path, and the sample timestamps refer to the sample waypoint moments when the training object arrives at at least two sample waypoints respectively in the process of passing through the sample path, the sample trajectory data is trajectory data that conforms to the motion laws of the training object. The trained path parameterization model has the ability to generate trajectory data that conforms to the motion laws of the training object. Therefore, the trajectory data corresponding to the path data of the robot is determined through the path parameterization model, which ensures that the trajectory data contains the characteristics of the training object, is more consistent with the motion laws of the training object, and improves the rationality of the trajectory data. When the trajectory data is used to control the robot to move on the path, the robot's movement is more in line with the training object's movement habits. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] FIG1 is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0028] FIG2 is a flow chart of a path parameterization model training method provided in an embodiment of the present application;

[0029] FIG3 is a flow chart of another path parameterization model training method provided in an embodiment of the present application;

[0030] FIG4 is a flow chart of a method for controlling robot motion provided by an embodiment of the present application;

[0031] FIG5 is a schematic diagram of the structure of a path parameterization model provided in an embodiment of the present application;

[0032] FIG6 is a flow chart of another method for controlling robot motion provided by an embodiment of the present application;

[0033] FIG7 is a schematic diagram of the structure of another path parameterization model provided in an embodiment of the present application;

[0034] FIG8 is a comparison diagram of trajectories determined by a TOPP method and a robot motion control method provided in an embodiment of the present application;

[0035] FIG9 is a schematic diagram of a learning curve of a path parameterization model provided in an embodiment of the present application;

[0036] FIG10 is a schematic diagram of an embedding feature provided in an embodiment of the present application;

[0037] FIG11 is a schematic diagram of a speed curve provided in an embodiment of the present application;

[0038] FIG12 is a schematic diagram of a trajectory provided by an embodiment of the present application;

[0039] FIG13 is a schematic structural diagram of a robot motion control device provided in an embodiment of the present application;

[0040] FIG14 is a schematic structural diagram of another robot motion control device provided in an embodiment of the present application;

[0041] FIG15 is a schematic structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0043] It is understood that the terms "first," "second," and the like used herein may be used to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are used solely to distinguish one concept from another. For example, a first sequence may be referred to as a second sequence, and similarly, a second sequence may be referred to as a first sequence without departing from the scope of this application.

[0044] Here, "at least two" refers to two or more than two. For example, the at least two sequences can be two sequences, three sequences, or any other integer greater than or equal to two. "Each" refers to each of the at least two sequences. For example, "each sequence" refers to each sequence in the at least two sequences. If the at least two sequences are three sequences, "each sequence" refers to each sequence in the three sequences.

[0045] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, input data, etc.) and signals (including but not limited to signals transmitted between user terminals and other devices, etc.) involved in this application are all fully authorized by users or relevant parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0046] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0047] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, pre-trained models, operating / interaction systems, and mechatronics. Pre-trained models, also known as large models or basic models, can be fine-tuned and widely applied to downstream tasks across various AI disciplines. AI software technologies primarily encompass computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and smart transportation.

[0048] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, autonomous driving, drones, digital twins, virtual humans, robots, artificial intelligence-generated content, conversational interaction, smart medical care, smart customer service, game AI, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0049] A pre-training model (PTM), also known as a cornerstone model or a large model, refers to a deep neural network (DNN) with large parameters. It is trained on massive amounts of unlabeled data, and the function approximation capability of the large-parameter DNN is used to enable the PTM to extract common features from the data. Through fine tuning, parameter-efficient fine-tuning (PEFT), prompt-tuning and other techniques, it is suitable for downstream tasks. Therefore, the pre-training model can achieve ideal results in few-shot or zero-shot scenarios. PTM can be divided into language models, visual models, speech models, multimodal models, etc. according to the data modality processed. Among them, a multimodal model refers to a model that establishes feature representations of two or more data modalities. The pre-training model is an important tool for outputting AI-generated content, and can also be used as a general interface to connect multiple specific task models.

[0050] The solutions provided in the embodiments of this application involve artificial intelligence technology, which is specifically illustrated by the following embodiments:

[0051] The robot motion control method provided in the embodiment of the present application is applied to computer equipment, which includes terminals, servers, or robot controllers.

[0052] In one possible implementation, the computer program involved in the embodiments of the present application can be deployed and executed on a computer device, or on multiple computer devices located at one location, or on multiple computer devices distributed at multiple locations and interconnected through a communication network. Multiple computer devices distributed at multiple locations and interconnected through a communication network can constitute a blockchain system.

[0053] In one possible implementation, the computer device in the embodiment of the present application is a node in a blockchain system, which can store the robot's data in the blockchain. Thereafter, the node or the node corresponding to other devices in the blockchain can query the robot's data by accessing the blockchain.

[0054] Optionally, the terminal is a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a smart game console, a smart TV, etc., but is not limited thereto.

[0055] Optionally, the server is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0056] Optionally, the robot controller is a device that controls the robot to complete certain actions or tasks based on instructions and sensor information, that is, the robot controller is used to control the robot, including controlling the trajectory of the robot's movement, the actions performed by the robot, and the posture changes of various parts of the robot.

[0057] In one possible implementation, the robot controller is located on the robot and is a component of the robot. In another possible implementation, the robot controller is an electronic device independent of the robot, and the robot controller is connected to the robot by wire or wirelessly.

[0058] A robot is an intelligent machine capable of semi-autonomous or fully autonomous operation. It can be programmed and automatically controlled to perform tasks such as moving, throwing, and hitting. The robot can be semi-mobile, meaning the entire robot is fixed in place with only parts able to move, or mobile, meaning the entire robot can move. Such robots can include drones, delivery robots, cooking robots, juggling robots, and other types. The embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, and smart transportation.

[0059] Figure 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application. Referring to Figure 1, the implementation environment includes: a server 101 and a robot 102. The server 101 and the robot 102 are connected through a communication network. The robot 102 includes an end effector 1021 and a robot controller 1022. The end effector 1021 refers to any tool connected to the edge of the robot and having certain functions, including a robot gripper, a robot collision sensor, a robot rotary connector, etc.

[0060] In an embodiment of the present application, the server 101 trains a path parameterization model based on sample trajectory data. The path parameterization model is used to determine trajectory data based on the path data of any robot, thereby achieving parameterization of the robot path, so that the robot's trajectory data not only contains the coordinates of at least two waypoints, but also includes the timestamps of at least two waypoints. The sample trajectory data is obtained by the training object through the sample path. Taking the training object as a human as an example, since the sample trajectory data is obtained by the human through the sample path, the path parameterization model has anthropomorphic capabilities, and the obtained trajectory data conforms to the laws of human movement. The robot path is composed of multiple waypoints. Waypoints are a key element for defining the robot path. They are a series of preset position points that the robot passes through during the movement process. These preset position points constitute the robot's movement path. The path data of the robot path includes the coordinates of the waypoints, and the trajectory data of the robot path includes the coordinates of the waypoints and the timestamps of the waypoints.

[0061] Then, when the starting point and end point of the end effector 1021 of the robot 102 have been determined, it is necessary to control the end effector 1021 to move from the starting point to the end point. The robot controller 1022 will determine the path from the starting point to the end point, and then determine the trajectory data corresponding to the path data through the path parameterization model, which can ensure that the trajectory data contains the characteristics of the training object, and the speed change of the end effector 1021 in the process of passing through the path is more in line with the motion law of the training object.

[0062] The implementation environment shown in FIG1 is only an exemplary implementation environment, and the embodiments of the present application can also be applied in other implementation environments.

[0063] The embodiments of the present application relate to a path parameterization model training method and a method for controlling robot motion through a path parameterization model. The following embodiments first describe the path parameterization model training method and then describe the robot motion control method.

[0064] FIG2 is a flow chart of a path parameterization model training method provided in an embodiment of the present application. The embodiment of the present application is executed by a computer device, which is a terminal or a server. Referring to FIG2 , the method includes:

[0065] 201. A computer device obtains sample trajectory data, where the sample trajectory data includes sample coordinates and sample timestamps of at least two sample waypoints in a sample path, where the sample timestamps refer to sample waypoint moments when a training subject reaches the at least two sample waypoints respectively while passing through the sample path.

[0066] In the embodiments of the present application, the training object is, for example, a human, a robot, or an animal, but is not limited thereto. Taking a human as an example, considering that human motion is related in space and time, and that the trajectory of human motion is not solely based on efficiency as the sole motion goal, but is also affected by factors such as human intention, emotion, and ability, the trajectory of human motion contains useful clues on how to pass through the path. Therefore, in order to ensure that the trajectory data generated by the trained path parameterization model conforms to the laws of human motion, a human can demonstrate the process of passing through a sample path, and record the sample coordinates of each sample waypoint in the process of the human passing through the sample path, as well as the sample timestamp of the human when arriving at each sample waypoint. The sample coordinates of the sample waypoint are coordinates in a three-dimensional coordinate system, which can represent the position and posture of the sample waypoint in three-dimensional space. For the sample timestamp of any sample waypoint, the sample timestamp of the sample waypoint refers to the sample waypoint moment when the training object arrives at the sample waypoint in the process of passing through the sample path.

[0067] Schematically, the calculation begins with the first moment as the starting moment, and the moment when the training subject arrives at the sample waypoint is determined as the sample waypoint moment of the sample waypoint. In some embodiments, the first moment is the moment when the training subject enters the sample path. For example, the moment when the training subject enters the sample path is 00:00, the moment when the training subject arrives at the first sample waypoint is 00:02 (i.e., the sample timestamp of the first sample waypoint), and the moment when the training subject arrives at the second sample waypoint is 00:05 (i.e., the sample timestamp of the second sample waypoint). In other embodiments, the first moment may also be the moment when the training subject arrives at the first sample waypoint on the sample path; or, the first moment is the moment when the video recording in the video corresponding to the sample trajectory data starts, and so on. This application does not limit the method for determining the first moment.

[0068] The sample path includes at least two sample waypoints from the starting point to the end point, and the at least two sample waypoints are sampled from the sample path. For example, the distance between each two adjacent sample waypoints in the at least two sample waypoints is equal, or the at least two sample waypoints are randomly sampled from the sample path.

[0069] From the perspective of the entire sample path, after determining the sample coordinates and sample timestamps of at least two sample waypoints, not only the time when the training object arrives at the sample waypoint is determined, but also the speed of the training object passing through each sample waypoint in the sample path is determined, thereby determining the characteristics of the speed change of the training object when passing through the sample path.

[0070] In one possible implementation, a demonstration video is obtained, the demonstration video including footage of a training subject passing through a sample path. For example, in the case of a human being, the demonstration video includes footage of a human hand passing through the sample path. Sample trajectory data is obtained based on the coordinates of the human hand in the footage and the corresponding timestamps of the footage in the demonstration video.

[0071] The demonstration video is a video captured of a human hand as it passes through a sample path. In each frame of the video, the human hand is in a certain position and posture. The coordinates of the human hand can be obtained, and each frame has a corresponding timestamp. For example, taking one frame in the video as one frame, at least two frames in the demonstration video are sampled to obtain at least two sets of coordinates and timestamps, which are then combined to form sample trajectory data. For example, OptiTrack (an optical motion capture system) is used to collect data during the human demonstration to obtain the sample trajectory data.

[0072] 202. The computer device trains a path parameterization model based on the sample trajectory data, where the path parameterization model is used to determine trajectory data corresponding to the path based on the path.

[0073] The computer device trains a path parameterization model based on the sample trajectory data, so that the path parameterization model has the ability to accurately determine the trajectory data based on the path, that is, the ability to parameterize the path. The path parameterization model can be a deep learning network model, a Transformer model (a model based on an attention mechanism), or other types of models. Optionally, the path parameterization model is an implicit neural function (HMTF) (Human Mimic Timing Function).

[0074] In one possible implementation, the path parameterization model takes robot path data as input and outputs trajectory data. Compared to the path data, the trajectory data also includes timestamps for at least two waypoints. A computer then compiles sample path data from the sample coordinates of at least two waypoints in the sample trajectory data. The computer then uses the sample path data as input and output for training the path parameterization model.

[0075] In another possible implementation, the path parameterization model takes as input a coordinate sequence consisting of the coordinates of at least two waypoints, and outputs a timestamp sequence consisting of the timestamps of at least two waypoints. The computer device then extracts sample coordinates of at least two sample waypoints from the sample trajectory data to form a sample coordinate sequence, and extracts sample timestamps of at least two sample waypoints from the sample trajectory data to form a sample timestamp sequence. The path parameterization model is trained using the sample coordinate sequence as input and the sample timestamp sequence as output.

[0076] In another possible implementation, the path parameterization model takes as input at least two timestamps and outputs the coordinates of at least two waypoints. The computer device then extracts sample coordinates of the at least two sample waypoints from the sample trajectory data to form a sample coordinate sequence, and extracts sample timestamps of the at least two sample waypoints from the sample trajectory data to form a sample timestamp sequence. The path parameterization model is trained using the sample timestamp sequence as input and the sample coordinate sequence as output.

[0077] After the path parameterization model is trained, the path parameterization model can be used to determine the trajectory data corresponding to the path based on the path, and the trajectory data is used to control the robot to move on the corresponding path.

[0078] In some embodiments, taking a first path as an example, the path data for the first path includes the coordinates of at least two first waypoints in the first path, and the first trajectory data includes the first timestamps of at least two first waypoints. Therefore, taking a human as an example, training a path parameterization model involves modeling the spatiotemporal correlations in human motion. Once modeled, the path parameterization model can be used to generate anthropomorphic temporal features for the robot's motion that match the robot's first path.

[0079] The robot can be any type of robot, and the first path is any path of the robot, and the first path includes at least two first waypoints, where the at least two first waypoints are sampled from the first path. For example, the distance between each two adjacent first waypoints in the at least two first waypoints is equal, or the at least two first waypoints are randomly sampled from the first path. In addition, the first timestamp of the first waypoint is the timestamp when the robot reaches the first waypoint.

[0080] In other embodiments, taking the second path as an example, the path data for the second path includes at least two second timestamps, and the second trajectory data includes second coordinates corresponding to at least two second waypoints. Therefore, taking a human as an example, training a path parameterization model involves modeling the spatial correlations in human motion. Once modeled, the path parameterization model can be used to generate anthropomorphic spatial features for the robot's motion that match the robot's second path.

[0081] In another possible implementation, in order to ensure the accuracy of the path parameterization model, it is necessary to train the path parameterization model based on multiple sample trajectory data. This ensures that the path parameterization model can learn the motion patterns of the training object when passing through different paths and remove the influence of abnormal trajectories, making it more universal and more accurate.

[0082] Taking humans as an example, as they traverse a sample path, their speed may vary, sometimes even significantly, causing local fluctuations in acceleration and discontinuous jumps. Directly applying this to a robot could cause severe vibration and damage. Training a path parameterization model based on sample trajectory data from multiple human demonstrations can learn the global characteristics of human demonstrations and filter out local fluctuations. This allows the model to generate anthropomorphic trajectories with smoother speed variations, improving trajectory smoothness.

[0083] For example, different human operation modes will generate different forms of sample paths and trajectories. For example, when a human picks up an object, the human hand will move directly to the location of the object. When a human needs to pour water from a handheld container into another container, the human hand will hold the container and move to the location of the other container before pouring the water. In these two cases, the paths that the human hand takes are different, and the speeds are also different. Therefore, by collecting sample trajectory data under multiple operation modes and training a path parameterization model based on sample trajectory data under multiple operation modes, the diversity of sample trajectory data can be enhanced, thereby allowing the path parameterization model to generalize different operation modes, thereby training a more generalized path parameterization model. Subsequently, path parameterization can be performed on the path data under any operation mode through the path parameterization model.

[0084] For example, path parameterization models can be trained separately for different operating modes. Specifically, sample trajectory data for each operating mode is obtained, and a path parameterization model corresponding to each operating mode is trained based on the sample trajectory data for each operating mode to improve the specificity of the path parameterization model. Subsequently, when the robot is about to perform a target operation in any operating mode, the robot's path data is determined, and the trajectory data corresponding to that path data is determined using the path parameterization model corresponding to that operating mode.

[0085] Compared with training the path parameterization model corresponding to each operation mode separately, the path parameterization model trained based on sample trajectory data under multiple operation modes performs multimodal learning, which can potentially generate embedded features and share these embedded features in relatively similar paths, thereby improving learning efficiency and accuracy.

[0086] In an embodiment of the present application, sample trajectory data is obtained and a path parameterization model is trained based on the sample trajectory data. Since the sample trajectory data includes sample coordinates and sample timestamps of at least two sample waypoints in the sample path, and the sample timestamps refer to the sample waypoint moments when the training object respectively arrives at at least two sample waypoints in the process of passing through the sample path, the sample trajectory data is trajectory data that conforms to the motion laws of the training object. The trained path parameterization model has the ability to generate trajectory data that conforms to the motion laws of the training object. Therefore, the trajectory data corresponding to the path data of the robot is determined through the path parameterization model, which ensures that the trajectory data contains the characteristics of the training object, is more consistent with the motion laws of the training object, and improves the rationality of the trajectory data. When the trajectory data is applied to control the robot to move on the first path, the movement of the robot is more in line with the motion habits of the training object.

[0087] Based on the embodiment shown in FIG. 2 above, an embodiment of the present application further provides another path parameterization model training method. FIG. 3 is a flow chart of another path parameterization model training method provided by an embodiment of the present application. The embodiment of the present application is executed by a computer device, which is a terminal or a server. Referring to FIG. 3 , the method includes:

[0088] 301. A computer device obtains first sample trajectory data, where the first sample trajectory data includes initial sample coordinates and sample timestamps of at least two sample waypoints.

[0089] The first sample trajectory data is the initial sample trajectory data acquired by the computer device. For example, the first sample trajectory data is the trajectory data acquired based on the human demonstration video in the above step 201 .

[0090] 302. The computer device normalizes initial sample coordinates and sample timestamps of at least two sample waypoints respectively to obtain normalized sample coordinates and normalized sample timestamps of the at least two sample waypoints.

[0091] Taking humans as training subjects as an example, the sample coordinates of the human hand's position in different demonstration videos are affected by the initial posture of the human hand in the real world. Moreover, the sample timestamps corresponding to the scenes in the demonstration videos are affected by real time. Therefore, the first sample trajectory data itself is subject to many interference factors and cannot accurately reflect the actual movement patterns of the human as it passes through the sample path. To this end, the first sample trajectory data is normalized to remove interference factors. After such processing, at least two normalized sample coordinates and normalized sample timestamps of sample waypoints are obtained. The normalized sample coordinates only reflect the changes in the position of the human hand in the sample path, without considering the influence of the initial posture of the human hand in the real world and the duration of the trajectory. The normalized sample timestamps reflect the changes in the relative time when the human hand reaches each sample waypoint in the sample path, thereby reflecting the changes in the speed of the human hand in the sample path without being affected by real time.

[0092] In one possible implementation, step 302 includes: determining normalized sample coordinates for each sample waypoint based on the initial sample coordinates of each sample waypoint and the initial sample coordinates of the first sample waypoint among the at least two sample waypoints; determining the duration corresponding to each sample waypoint as the difference between the sample timestamp of each sample waypoint and the sample timestamp of the first sample waypoint; and determining the normalized sample timestamp of each sample waypoint as the ratio between the duration corresponding to each sample waypoint and the duration corresponding to the last sample waypoint among the at least two sample waypoints. For any sample waypoint, the duration corresponding to the sample waypoint refers to the duration from the first sample waypoint to the sample waypoint.

[0093] For example, the following formula (1) is used to normalize the initial sample coordinates of at least two sample waypoints:

[0094] In formula (1), i represents the serial number of the sample waypoint, m represents the number of sample waypoints, i and m are positive integers, represents the initial sample coordinates of the i-th sample waypoint, Indicates the normalized sample coordinates of the i-th sample waypoint. h indicates before normalization, |h| indicates after normalization, and h can be replaced by any other letter. By the initial sample coordinates of the first sample waypoint Perform inverse transformation to obtain the inverse transformation result The inverse transformation result is combined with the initial sample coordinates of the i-th sample waypoint Multiply to get the normalized sample coordinates of the i-th sample waypoint

[0095] For example, the following formula (2) is used to normalize the initial sample timestamps of at least two sample waypoints:

[0096] In formula (2), represents the initial sample timestamp of the i-th sample waypoint, Represents the normalized sample timestamp of the i-th sample waypoint. That is, the normalized sample timestamp of each sample waypoint belongs to the interval [0, 1], where the sample timestamp of the starting point (the first sample waypoint) is 0 and the sample timestamp of the end point (the last sample waypoint) is 1. Indicates the duration from the first sample waypoint to the i-th sample waypoint, Indicates the duration from the first sample waypoint to the last sample waypoint.

[0097] By using the above formula (1), the product of the initial sample coordinates of each sample waypoint and the inverse transformation result of the initial sample coordinates of the first sample waypoint is calculated. This allows the absolute posture of each sample waypoint to be converted into a posture difference relative to the posture of the first sample waypoint. In other words, the posture of each sample waypoint is transformed relative to the posture of the first sample waypoint, and a relative posture description based on the first sample waypoint is constructed, thereby achieving consistency between different sample waypoints and eliminating the influence of the initial posture of the human hand. In addition, by using the above formula (2), the duration corresponding to the last sample waypoint is the duration of the entire trajectory. The normalized sample timestamp of each sample waypoint represents the ratio between the sample timestamp when arriving at the sample waypoint and the duration of the trajectory, rather than the actual time when arriving at the sample waypoint. This better reflects the relative position of the sample waypoint in the entire trajectory, thereby eliminating the influence of actual time.

[0098] 303. The computer device combines the normalized sample coordinates and the normalized sample timestamps of at least two sample waypoints into second sample trajectory data.

[0099] For example, the first sample trajectory data is represented as ξ h Represents the first sample trajectory data, i represents the sequence number of the sample waypoint, m represents the number of sample waypoints, i and m are positive integers, represents the initial sample coordinates of the i-th sample waypoint, represents the initial sample timestamp of the i-th sample waypoint, represents the normalized sample coordinates of the i-th sample waypoint, represents the normalized sample timestamp of the i-th sample waypoint, and the second sample trajectory data is expressed as ξ |h| Represents the second sample trajectory data.

[0100] 304. The computer device trains a path parameterization model based on the second sample trajectory data.

[0101] Step 304 is similar to step 202 and will not be repeated here.

[0102] It should also be noted that in one possible implementation, the path parameterization model's input is a coordinate sequence consisting of the coordinates of at least two waypoints, and its output is a timestamp sequence consisting of the timestamps of at least two waypoints. That is, the path parameterization model is a sequence-to-sequence neural network model. Based on sample trajectory data, the process of training the path parameterization model includes: forming a sample coordinate sequence from the sample coordinates of the at least two sample waypoints, and forming a first sample timestamp sequence from the sample timestamps of the at least two sample waypoints. Using the path parameterization model, a second sample timestamp sequence corresponding to the sample coordinate sequence is determined. The first sample timestamp sequence is a sample timestamp sequence that conforms to human motion patterns, and the second sample timestamp sequence is a sample timestamp sequence predicted by the path parameterization model. Based on the error between the first and second sample timestamp sequences, the model parameters of the path parameterization model are adjusted, thereby improving the performance of the path parameterization model so that the sample timestamp sequence generated by the path parameterization model gradually approaches a sample timestamp sequence that conforms to human motion patterns.

[0103] In another possible implementation, if the sample trajectory data is normalized, the sample timestamps in the normalized sample timestamp sequence will fall within the interval [0, 1]. This sample timestamp sequence can reflect the speed changes in the sample path, but it cannot reflect the actual duration of the sample path. Therefore, when training the path parameterization model, it is necessary to consider not only the sample timestamp sequence but also the duration of the sample path.

[0104] Therefore, based on the sample trajectory data, the path parameterization model is trained, which also includes:

[0105] The difference between the sample timestamp of the last sample waypoint and the sample timestamp of the first sample waypoint is determined as the first duration; the second duration corresponding to the second sample timestamp sequence is determined through the path parameterization model, the first duration being the time required for a human hand to pass through the sample path, and the second duration representing the time required to pass through the sample path as predicted by the path parameterization model. Based on the error between the first sample timestamp sequence and the second sample timestamp sequence, the model parameters of the path parameterization model are adjusted, including: based on the error between the first sample timestamp sequence and the second sample timestamp sequence, and the error between the first duration and the second duration, the model parameters of the path parameterization model are adjusted, thereby improving the performance of the path parameterization model, so that the sample timestamp sequence generated by the path parameterization model gradually approaches the sample timestamp sequence that conforms to the laws of human motion, and the duration required to pass through the path is also more consistent with the laws of human motion.

[0106] In addition, the model parameters of the path parameterized model are adjusted based on the error, including: creating a loss function, the loss value in the loss function is positively correlated with the error, based on the error, with the loss value close to the minimum value as the goal, adjusting the model parameters of the path parameterized model until the number of training times reaches the target number of times or until the loss function converges, and the path parameterized model training is completed.

[0107] For example, the sample timestamp sequence in the first sample trajectory data is The first sample timestamp sequence obtained after normalization is The second sample timestamp sequence is The first sample timestamp sequence Γ |h| and the second sample timestamp sequence Γ |k| The errors between include: a first error and a second error.

[0108] The distance between the sample timestamp of each sample waypoint in the first sample timestamp sequence and the sample timestamp of the same sample waypoint in the second sample timestamp sequence is positively correlated with the first error, that is, the distance between the sample timestamp of each sample waypoint in the first sample timestamp sequence and the sample timestamp of the same sample waypoint in the second sample timestamp sequence is positively correlated with the first error. The distance can be the Euclidean distance, L2 distance, or other distance between two sample timestamps of the same sample waypoint. The first error can be the sum of the distances between the two sample timestamps of each sample waypoint, or the product of the distances between the two sample timestamps of each sample waypoint, or other numerical values ​​positively correlated with the distances between the two sample timestamps of each sample waypoint, which are not limited in this embodiment of the present application. The distance between the two sample timestamps is used to measure the difference between the sample timestamp output by the path parameterization model and the sample timestamp in the sample trajectory data, or to quantify the degree of deviation in the time dimension between the robot reaching a specified sample waypoint and the human hand reaching the same sample waypoint.

[0109] For example, the first error is calculated by the following formula (3):

[0110] In formula (3), i represents the serial number of the sample waypoint, m represents the number of sample waypoints, i and m are positive integers, represents the sample timestamp of the i-th sample waypoint in the first sample timestamp sequence, represents the sample timestamp of the i-th sample waypoint in the second sample timestamp sequence, and ||·||2 represents the L2 distance. In addition, h and k are only used to distinguish the letters of the first sample timestamp sequence from the second sample timestamp sequence, and can be replaced by any other letters.

[0111] In addition, the distance between the second duration and the first duration is positively correlated with the second error. The distance can be the Euclidean distance, L2 distance or other distance between the second duration and the first duration. The second error can be equal to the distance between the second duration and the first duration, or the product of the distance between the second duration and the first duration and a preset value, or other values ​​that are positively correlated with the distance between the second duration and the first duration, and the embodiments of the present application do not limit this. Among them, the distance between the two durations is used to measure the difference between the time required to pass the sample path predicted by the path parameterization model and the time required for a human hand to pass through the sample path.

[0112] For example, the second error is calculated by the following formula (4): ||TT h ||2 (4)

[0113] In formula (4), T represents the second duration, T h Indicates the first duration, That is, the duration from the first sample waypoint to the last sample waypoint.

[0114] The first sample timestamp sequence Γ |h| and the second sample timestamp sequence Γ |k| The error between includes a first error and a second error, including: the first sample timestamp sequence Γ |h| and the second sample timestamp sequence Γ |k| The error between them is the sum of the first error and the second error, or the product of the sum of the first error and the second error and a preset value, or the weighted sum of the first error and the second error, etc., which is not limited in the embodiment of the present application.

[0115] In one possible implementation, the path parameterized loss function is: L=ω1L1+ω2L2, where L represents the loss value, L1 represents the first error, L2 represents the second error, ω1 represents the first weight, and ω2 represents the second weight, where the first weight and the second weight can be set by a computer device, and the loss value is the weighted sum of the first error and the second error.

[0116] In addition, in addition to the above-mentioned first error and second error, the error of the path parameterization model may also include other errors, which are not limited in the embodiment of the present application.

[0117] After the path parameterization model is trained, the trajectory data corresponding to the robot's path data can be determined through the path parameterization model.

[0118] In the embodiment of the present application, in addition to requiring the robot to produce safe and reasonable movements, it is more important to make the robot operate in a way that humans can accept, understand, and even predict. In the embodiment of the present application, by obtaining sample trajectory data, a path parameterization model is trained based on the sample trajectory data. Since the sample trajectory data contains the sample coordinates of at least two sample waypoints in the sample path and the sample timestamps when the training object reaches each sample waypoint in the process of passing through the sample path, the sample trajectory data is trajectory data that conforms to the motion law of the training object. The trained path parameterization model has the ability to generate trajectory data that conforms to the motion law of the training object. Therefore, when the training object is a human, the trajectory data generated by the path parameterization model contains anthropomorphic and smooth time characteristics of speed changes, which is more in line with the laws of human motion and improves the rationality of the trajectory data. Compared with the TOPP method, the path parameterization model provided in the embodiment of the present application is more versatile and has achieved significant improvements in anthropomorphism.

[0119] In addition, by normalizing the initial sample coordinates and sample timestamps of at least two sample waypoints in the first sample trajectory data, respectively, and training a path parameterization model based on the second sample trajectory data obtained after normalization, the normalized sample coordinates reflect the changes in the position and posture of the human hand in the sample path, without considering the influence of the initial posture of the human hand in the real world and the duration of the trajectory. The normalized sample timestamps reflect the changes in the relative time when the human hand reaches each sample waypoint in the sample path, thereby reflecting the changes in the speed of the human hand in the sample path without being interfered with by real time, ensuring that the path parameterization model can accurately learn the laws of human movement, thereby improving the accuracy of the path parameterization model.

[0120] In addition, by obtaining a demonstration video during the human demonstration, based on the coordinates of the human hand in the picture in the demonstration video and the timestamp corresponding to the picture, we can obtain sample trajectory data that accurately reflects the laws of human movement, thereby improving the accuracy of the path parameterization model.

[0121] Furthermore, the path parameterization model takes as input a coordinate sequence consisting of the coordinates of at least two waypoints, and outputs a timestamp sequence consisting of the timestamps of at least two waypoints. The first sample timestamp sequence determined based on the sample trajectory data is a sample timestamp sequence that conforms to the laws of human motion, and the second sample timestamp sequence output by the path parameterization model is a timestamp sequence predicted by the path parameterization model. Based on the error between the first and second sample timestamp sequences, the model parameters of the path parameterization model are adjusted, thereby improving the performance of the path parameterization model. This allows the timestamp sequence generated by the path parameterization model to gradually approach a timestamp sequence that conforms to the laws of human motion, thereby enhancing the accuracy of the path parameterization model.

[0122] Furthermore, if the sample trajectory data is normalized, the normalized sample timestamp sequence can reflect the speed changes along the sample path, but it cannot reflect the actual duration of the sample path. Therefore, a path parameterization model is trained based on the sample timestamp sequence and the duration of the sample path. This allows the timestamp sequence generated by the path parameterization model to gradually approach a timestamp sequence that conforms to the laws of human motion. Furthermore, the duration required to pass the path also better conforms to human motion laws, improving the accuracy of the path parameterization model.

[0123] Based on the embodiments shown in Figures 2 and 3 above, the present invention also provides a method for controlling robot motion, illustrating the process of determining trajectory data using a path parameterization model. Figure 4 is a flow chart of a method for controlling robot motion provided by the present invention. The present invention uses the method executed by a robot controller as an example. Referring to Figure 4, the method includes:

[0124] 401. The robot controller obtains a path parameterized model.

[0125] The path parameterization model is trained based on sample trajectory data, which includes the coordinates and timestamps of at least two waypoints along the sample path. The timestamp represents the time a person arrives at a waypoint while traversing the sample path. The training process for the path parameterization model is similar to that of the embodiments shown in Figures 2 and 3 above and will not be further described here.

[0126] In one possible implementation, after training the path parameterization model, the computer device transmits the path parameterization model to the robot controller, which then stores the path parameterization model. Each time the robot's path data is determined through path planning, the robot controller uses the path parameterization model to determine the trajectory data corresponding to the path data.

[0127] 402. The robot controller obtains a first path and obtains first coordinates of at least two first waypoints in the first path.

[0128] The first path represents the path that the robot needs to pass through, and the first path includes at least two first waypoints. The first first waypoint of the at least two first waypoints is the starting point of the robot, and the last first waypoint is the end point of the robot.

[0129] In one possible implementation, the robot controller determines a start point and an end point, determines a path from the start point to the end point based on the robot's start point and end point, and smoothes the path to obtain a first path that satisfies a smoothing condition. For example, the robot controller employs an RRT (Rapidly-Exploring Random Tree) algorithm or an RRT-Connect (Rapidly-Exploring Random Tree-Connect) algorithm to perform path planning based on the robot's start point and end point, and determines a path from the start point to the end point.

[0130] In one possible implementation, the robot controller adopts Cartesian space, and the coordinates involved are coordinates in the Cartesian space coordinate system of the robot. Alternatively, the robot controller may also adopt other coordinate systems, which is not limited in the embodiments of the present application. For example, the robot controller creates Cartesian space and joint space, performs path parameterization in Cartesian space, and after determining the first trajectory data corresponding to the first path, the first trajectory data can be converted into trajectory data in the joint space. Alternatively, the robot controller creates Cartesian space and joint space, converts the first path in the joint space into the first path in Cartesian space, and thus performs path parameterization in Cartesian space.

[0131] 403. The robot controller determines first trajectory data based on the first coordinates using a path parameterized model, where the first trajectory data includes first timestamps corresponding to at least two first waypoints.

[0132] The first trajectory data represents the first trajectory of the robot. The first trajectory data is used to control the robot to perform movement on the first path so that the robot reaches at least two first waypoints at the first waypoint time indicated by the first timestamp. The first path is regarded as a geometric object without a time feature, while the first trajectory contains a time feature. For example, the first path of the robot is represented by P = {x1, x2, ...x m}, m represents the number of waypoints, m is a positive integer, and the first trajectory is represented by ξ = {(x1, t1), (x2, t2), ..., (x m , t m )}, path parameterization refers to adding a timestamp and a first duration of the trajectory on the basis of the first path to obtain a first trajectory, so that the robot performs movement according to the first trajectory.

[0133] For example, in addition to the first timestamps of at least two first waypoints, the first trajectory data may also include the first coordinates of at least two first waypoints. Based on the first trajectory data, the changes in position and posture as well as the changes in speed of the robot during the process of passing through the first path can be represented.

[0134] In one possible implementation, the input of the path parameterization model is a coordinate sequence consisting of the coordinates of at least two waypoints, and the output is a timestamp sequence consisting of the timestamps of at least two waypoints. Based on the first coordinates of the at least two first waypoints, a first coordinate sequence is determined, and through the path parameterization model, a first timestamp sequence and a first duration corresponding to the first coordinate sequence are determined, where the first duration represents the time required for the robot to pass through the first path.

[0135] In one possible implementation, when sample coordinates of at least two sample waypoints in the sample trajectory data are not normalized during training of the path parameterization model, determining the first coordinate sequence based on the first coordinates of the at least two first waypoints includes: forming the first coordinates of the at least two first waypoints into the first coordinate sequence.

[0136] In another possible implementation, when the sample coordinates of at least two sample waypoints in the sample trajectory data are normalized during training of the path parameterization model, determining the first coordinate sequence based on the coordinates of the at least two first waypoints includes: forming an initial coordinate sequence from the first coordinates of the at least two first waypoints, and then normalizing the at least two first coordinates in the initial coordinate sequence to obtain the first coordinate sequence. The normalization process is similar to the normalization process in step 302 above and is not further described here.

[0137] Furthermore, if the sample trajectory data is normalized during training of the path parameterization model, the trained path parameterization model is capable of generating a normalized timestamp sequence. Specifically, in the generated timestamp sequence, the timestamp of the first waypoint is 0, and the timestamp of the last waypoint is 1. While the first timestamp sequence can reflect the robot's speed changes, it cannot accurately reflect the time at which the robot reaches each first waypoint. Therefore, after determining the first timestamp sequence and the first duration corresponding to the first coordinate sequence using the path parameterization model, the product of each timestamp in the first timestamp sequence and the first duration is calculated to generate an updated first timestamp sequence. The timestamps of at least two first waypoints included in the updated first timestamp sequence represent the interval durations between the first waypoints, i.e., the interval durations between the arrival time of each first waypoint and the starting time point. Subsequently, the robot can be controlled, starting from the starting point, to reach each first waypoint after the interval durations between each first waypoint have elapsed, according to the updated first timestamp sequence.

[0138] In one possible implementation, after obtaining the first trajectory data through the path parameterization model, the robot controller may use the first trajectory data as the final trajectory data of the robot. Alternatively, the robot is further provided with constraints, and the robot's trajectory data must satisfy the constraints, including constraints on the robot's speed, or constraints on the speed or torque of the robot's joints. After determining the first timestamp sequence and the first duration, the robot controller determines whether the first timestamp sequence and the first duration satisfy the constraints. If so, the robot is controlled based on the first timestamp sequence and the first duration. If not, the first timestamp sequence and the first duration are adjusted until the adjusted first timestamp sequence and the first duration satisfy the constraints, and the robot is then controlled based on the adjusted first timestamp sequence and the first duration.

[0139] For example, T'←T+Δt means increasing the original first duration T by the adjustment amplitude Δt to obtain the updated first duration T', and Γ'←T'×Γ means multiplying the original first timestamp sequence Γ by the updated first duration T' to obtain the updated first timestamp sequence Γ'.

[0140] It should be noted that the embodiment of the present application takes the path parameterization of the robot controller as an example, but in another embodiment, the path parameterization for the robot can also be performed by a computer device, that is, after the robot controller determines the path of the robot, it sends it to the computer device, and the computer device obtains the trajectory data of the path through the path parameterization model and sends it to the robot controller. The robot controller then controls the robot to move according to the trajectory data so that the movement law of the robot conforms to the movement law of the training object.

[0141] Another point that needs to be explained is that the control method of robot motion provided in the embodiment of the present application can be applied to the motion planning of a bionic robot or a non-bionic robot, wherein a bionic robot refers to a robot that looks similar to a human, and a non-bionic robot refers to a robot that does not look similar to a human. In addition, the control method of robot motion provided in the embodiment of the present application can be applied to a path planning algorithm based on optimization. The path planning algorithm based on optimization will set an objective function (a first function or other naming method may also be used, which is not limited). The objective function represents the optimization target. When performing path planning, it is necessary to determine the trajectory with the minimum value of the objective function, that is, the trajectory that can achieve the optimization target. After determining the trajectory data using the method provided in the embodiment of the present application, the value of the objective function can be determined based on the trajectory data, thereby selecting the trajectory data with the minimum value of the objective function.

[0142] For example, the first trajectory data meets the following conditions:

[0143] Where u(ξ) represents the objective function, which is the optimization cost of the robot, such as the first time spent on the first trajectory or the energy consumed by the first trajectory, Ξ represents the space to which the first trajectory data belongs, and ξ * Represents the first trajectory data that satisfies the minimum value of the objective function. i represents the sequence number of the waypoint, m represents the number of waypoints, i and m are positive integers, x i represents the coordinates of the i-th waypoint, t i represents the timestamp of the i-th waypoint, Γ={t1,t2,…,t m}, Γ represents the first timestamp sequence, P = {x1, x2, ..., x m}, P represents the first coordinate sequence, C OTHERCONSTRAINTS Represents the constraints that the robot needs to meet.

[0144] In an embodiment of the present application, a path parameterization model is trained based on sample trajectory data. Since the sample trajectory data includes sample coordinates and sample timestamps of at least two sample waypoints in the sample path, and the sample timestamps refer to the sample waypoint moments when the training object arrives at at least two sample waypoints respectively in the process of passing through the sample path, the sample trajectory data is trajectory data that conforms to the motion laws of the training object. The trained path parameterization model has the ability to generate trajectory data that conforms to the motion laws of the training object. Therefore, the trajectory data corresponding to the path data of the robot is determined through the path parameterization model, which ensures that the trajectory data contains the characteristics of the training object, is more consistent with the motion laws of the training object, and improves the rationality of the trajectory data. When the trajectory data is used to control the robot to move on the path, the robot's movement is more in line with the training object's movement habits.

[0145] Based on the embodiment shown in FIG4 above, an embodiment of the present application further provides another method for controlling robot motion. FIG5 is a schematic structural diagram of a path parameterization model provided in an embodiment of the present application. The path parameterization model includes at least one encoder, at least one decoder, a normalization layer, and a summation layer. FIG6 is a flow chart of another method for controlling robot motion provided in an embodiment of the present application. The embodiment of the present application is performed by a robot controller as an example. Referring to FIG6, the method includes:

[0146] 601. The robot controller obtains a path parameterized model.

[0147] 602. The robot controller obtains a first path and obtains first coordinates of at least two first waypoints in the first path.

[0148] Steps 601-602 are similar to the above steps 401-402 and will not be repeated here.

[0149] 603. The robot controller determines a first coordinate sequence based on the coordinates of at least two first waypoints.

[0150] The possible implementation of step 603 is similar to the method of determining the first coordinate sequence in step 403 above, and will not be described again here.

[0151] 604. The robot controller encodes the first coordinate sequence through at least one encoder to obtain an encoding feature.

[0152] 605. The robot controller decodes the encoded feature through at least one decoder to obtain a decoded feature.

[0153] 606. The robot controller normalizes the decoded features through a normalization layer to obtain a first timestamp sequence.

[0154] After the decoded features are normalized by the normalization layer, the obtained first timestamp sequence is the normalized timestamp sequence, that is, the first timestamp sequence includes at least two normalized first timestamps of the first waypoints. For example, the first timestamp of the first first waypoint is 0, indicating that at t=0, the robot needs to start from the first first waypoint (i.e., the starting point). The first timestamp of the last first waypoint is 1, indicating that the ratio of the time taken by the robot to reach the last first waypoint to the total first time is 1.

[0155] 607. The robot controller sums the decoded features through a summation layer to obtain a first duration.

[0156] The decoding features include multiple features, such as decoding features corresponding to at least two first waypoints, and the multiple features are summed through a summation layer to obtain the first duration.

[0157] For example, referring to FIG7 , at least one encoder includes an MLP (MultiLayer Perceptron) layer, and at least one decoder includes multiple Transformer Blocks (a network based on an attention mechanism) and an MLP layer. The first coordinate sequence is Represents the sequence number of the waypoint, m represents the number of waypoints, i and m are positive integers, Represents the coordinates of the i-th waypoint, encodes the first coordinate sequence through the first MLP layer to obtain the encoded feature, the encoded feature includes the encoded sub-feature corresponding to each waypoint, then based on the position of the encoded sub-feature corresponding to each waypoint in the encoded feature, determine the position sub-feature corresponding to each waypoint, and the position sub-features corresponding to at least two waypoints constitute the position feature, then input the encoded feature and the position feature into the first Transformer Block, thereby decoding the encoded feature based on the position feature through multiple Transformer Blocks and the second MLP layer to obtain the decoded feature, and then process the decoded features respectively through the normalization layer and the summation layer to obtain the first timestamp sequence And the first duration T.

[0158] Exemplarily, the normalization layer outputs the time difference between every two adjacent first waypoints. Since the timestamp of the first first waypoint is 0, starting from the second first waypoint, the sum of the time difference corresponding to each first waypoint and the timestamp of the previous first waypoint of each first waypoint is calculated to obtain the timestamp of each first waypoint, thereby ensuring that the timestamps are monotonically increasing.

[0159] In an embodiment of the present application, a path parameterization model is trained based on sample trajectory data. Since the sample trajectory data includes sample coordinates and sample timestamps of at least two sample waypoints in the sample path, and the sample timestamps refer to the sample waypoint moments when the training object arrives at at least two sample waypoints respectively in the process of passing through the sample path, the sample trajectory data is trajectory data that conforms to the motion laws of the training object. The trained path parameterization model has the ability to generate trajectory data that conforms to the motion laws of the training object. Therefore, the trajectory data corresponding to the path data of the robot is determined through the path parameterization model, which ensures that the trajectory data contains the characteristics of the training object, is more consistent with the motion laws of the training object, and improves the rationality of the trajectory data. When the trajectory data is used to control the robot to move on the path, the robot's movement is more in line with the training object's movement habits.

[0160] In addition, an embodiment of the present application provides a model architecture of a path parameterization model, which includes at least one encoder, at least one decoder, a normalization layer and a summation layer. The use of this model architecture can improve the learning efficiency of the path parameterization model, thereby improving the accuracy of the path parameterization model.

[0161] Furthermore, based on the foregoing description, in some embodiments, taking the second path as an example, the path data for the second path includes at least two second timestamps, and the second trajectory data includes second coordinates corresponding to at least two second waypoints. Therefore, taking a human as an example, training a path parameterization model involves modeling the spatial correlations in human motion. Once modeled, the path parameterization model can be used to generate anthropomorphic spatial features for the robot's motion that match the second path.

[0162] Based on this, an embodiment of the present application also provides another method for controlling robot motion, which is executed by a computer device. The method includes: obtaining a path parameterization model, the path parameterization model is trained based on sample trajectory data, the sample trajectory data includes sample coordinates and sample timestamps of at least two sample waypoints in the sample path, and the timestamp refers to the sample waypoint moments when the training object reaches the at least two sample waypoints respectively in the process of passing through the sample path; obtaining a second path and obtaining at least two second timestamps; determining second trajectory data based on the second timestamp through the path parameterization model, the second trajectory data includes second coordinates corresponding to at least two second waypoints respectively, and the second trajectory data is used to control the robot to perform motion on the second path, so that the robot reaches at least two second waypoints respectively at the second waypoint moments referred to by the second timestamps.

[0163] In some embodiments, second trajectory data is determined based on a second timestamp through a path parameterization model, including: determining a second timestamp sequence and a second duration based on at least two second timestamps, the second duration representing the duration required for the robot to pass through the second path; and determining a second coordinate sequence corresponding to the second timestamp sequence and the second duration through the path parameterization model, the second coordinate sequence including second coordinates corresponding to at least two second waypoints.

[0164] Through the above method, since the sample trajectory data includes the sample coordinates and sample timestamps of at least two sample waypoints in the sample path, and the sample timestamps refer to the sample waypoint moments when the training object respectively reaches at least two sample waypoints in the process of passing through the sample path, the sample trajectory data is trajectory data that conforms to the motion laws of the training object. The trained path parameterization model has the ability to generate trajectory data that conforms to the motion laws of the training object. Therefore, the trajectory data corresponding to the path data of the robot is determined through the path parameterization model, which ensures that the trajectory data contains the characteristics of the training object, is more consistent with the motion laws of the training object, and improves the rationality of the trajectory data. When the trajectory data is used to control the robot to move on the path, the robot's movement is more in line with the training object's movement habits.

[0165] The TOPP method is compared with the robot motion control method provided in the embodiments of the present application below.

[0166] Taking the scenario where the robot's end effector needs to move from the starting point to the end point and bypass the obstacles between the starting point and the end point as an example, the trajectory determined by the TOPP method is shown in the first figure in Figure 8, and the trajectory determined by the robot motion control method provided in the embodiment of the present application is shown in the second figure in Figure 8, where the more transparent the trajectory curve is, the smaller the speed of the robot's end effector is, and the more opaque the trajectory curve is, the faster the speed of the robot's end effector is.

[0167] As shown in Figure 8, the TOPP method drives the robot to move at maximum capacity in order to minimize duration. Therefore, the robot's end-effector speed increases and then decreases, with the end-effector's speed being highest near the obstacle, which violates the laws of human motion. However, the robot motion control method provided in the present embodiment exhibits temporal characteristics related to space. Therefore, the robot's end-effector speed increases, decreases before approaching the obstacle, increases after bypassing the obstacle, and then decreases again just before reaching the end point. This change in end-effector speed conforms to the laws of human motion. When bypassing an obstacle, humans also slow down when approaching the obstacle and accelerate again after bypassing it, in order to avoid or protect themselves from it. Therefore, the robot trajectory determined in the present embodiment not only completes the movement task from the starting point to the end point, but also incorporates anthropomorphic temporal characteristics, thereby conveying richer semantics and expressing the human's intention to avoid the obstacle, making it more expressive.

[0168] The present application also conducted experiments in which humans demonstrated the following four operating modes:

[0169] Operation mode 1: Human grasping objects;

[0170] Operation mode 2: Human placement of objects;

[0171] Operation mode 3: A human holds a container filled with water and moves it;

[0172] Operation mode 4: A human moves a container filled with water and pours the water into another container.

[0173] The four aforementioned operating modes consist of human hand movements while avoiding collisions. The geometry of the paths traversed by the hand and the corresponding temporal characteristics vary significantly across these modes. For example, operating mode 3 does not involve a change in the container's orientation, while operating mode 4 requires reorientation of the container.

[0174] OptiTrack (a motion capture system) was used to capture data during human demonstrations of the four aforementioned operating modes to construct a dataset. Considering that the paths actually demonstrated by humans are composed of sparse and uneven waypoints, random sampling was performed within each human-demonstrated trajectory in this embodiment to ensure that m to u [40, 120], l to u [30, 50], where u represents a uniform distribution, the number m of waypoints in the sample trajectory data follows a uniform distribution within the interval [40, 120], and the number l of sample trajectory data follows a uniform distribution within the interval [30, 50], to ensure a uniform distribution of waypoints in the dataset.

[0175] In one possible implementation, the dataset is divided into three subsets: a training set, a validation set, and a test set. For example, the ratio of the number of sample trajectory data in the training set, validation set, and test set is fixed. For example, the training set contains 70% of the sample trajectory data, the validation set contains 20% of the sample trajectory data, and the test set contains 10% of the sample trajectory data. The path parameterization model is trained using the training set, and the trained path parameterization is verified using the validation set. Based on the verification results, the path parameterization model is trained again. After training, the path parameterization model is evaluated using the test set to understand its performance.

[0176] FIG9 is a schematic diagram of a learning curve for a path parameterization model provided in an embodiment of the present application. Referring to FIG9 , the horizontal axis of the learning curve represents the number of iterative training cycles, the vertical axis represents the error, the solid line represents the training curve, and the dashed line represents the verification curve. As the number of iterative training cycles increases, the first error L1, the second error L2, and the total error L = ω1L1 + ω2L2 of the path parameterization model decrease to very low levels. It can be seen that the path parameterization model has achieved high prediction accuracy, proving that human motion is time- and space-dependent. The method employed in the embodiment of the present application can extract the temporal and spatial correlation from human-demonstrated operations and extract accurate temporal features from the path, thereby constructing an accurate path parameterization model.

[0177] The embodiment of the present application also adopts PCA (Principal Components Analysis) technology to project the embedded features of each path into a two-dimensional space, and obtains the embedded feature diagram shown in Figure 10. Each point in Figure 10 represents the embedded feature of a path. The embedded feature refers to the feature obtained in the process of the path parameterization model processing the path data. For example, the model architecture of the path parameterization model is shown in Figure 7. The Transformer Block includes an encoding layer and a decoding layer. When the feature is processed by the Transformer Block, it will first be encoded to obtain the embedded feature, and then the embedded feature will be decoded. Therefore, the embedded feature generated during the Transformer Block processing is projected into a two-dimensional space. As can be seen from Figure 10, the embedded features of the paths corresponding to the above four operation modes are respectively clustered in four clusters (represented by dotted lines). The distance between the embedded features from the same operation mode is short and can form a cluster. This proves that the embedded features learned by the path parameterization model are operation mode-oriented, so that the path parameterization model has strong generalization ability and flexibility, and can parameterize random paths from different modes.

[0178] In addition, there are three TOPP methods in the related art: IPTP (Iterative Parabolic Time Parameterization), ISTP (Iterative Spline Time Parameterization), and TOTG (Time-Optimal Trajectory Generation). The method provided in the embodiment of the present application is compared with the above three TOPP methods as follows:

[0179] Given a human-demonstrated trajectory The path parameterization is performed using the method (HMTF method) provided in the embodiment of the present application and the three TOPP methods mentioned above, and the trajectory is obtained again. The following two evaluation indicators are used:

[0180] First indicator: Where i represents the sequence number of the waypoint in the trajectory, its initial value is 1, the maximum value is m, m represents the number of waypoints contained in the trajectory, i and m are positive integers, represents the timestamp of the i-th waypoint in the trajectory produced by the HMTF method or each of the three TOPP methods mentioned above, represents the timestamp of the i-th waypoint in the trajectory of the human demonstration. timestamp represents the first metric, which quantifies the temporal similarity of the trajectory obtained by re-parameterizing the path to the trajectory of human demonstration.

[0181] Second indicator: f duration =T k / T h , where T k represents the duration of the trajectory determined by the HMTF method or each of the three TOPP methods mentioned above, T h represents the duration of the trajectory demonstrated by the human. duration represents the second metric, which quantifies the similarity between the duration of the trajectory obtained by re-parameterizing the path and the trajectory demonstrated by humans.

[0182] Based on the datasets of the four operating modes mentioned above, the HMTF method and the three TOPP methods mentioned above are used to parameterize the paths, and the two evaluation indicators mentioned above for each method are obtained. The evaluation results are shown in Table 1 below, where the values ​​in front of the brackets represent f timestamp or f duration The values ​​in brackets represent the average value of f timestamp or f duration The standard deviation of .

[0183] Table 1

[0184] As shown in Table 1, while all methods reparameterize the path while satisfying system constraints, the HMTF method achieves greater improvements in the temporal anthropomorphism of the resulting trajectories. Figure 11 shows the velocity curves of the trajectories generated by the four methods for a given path under four operational modes. (The HMTF method handles both translational and rotational motion; for simplicity, Figure 11 only shows the translational velocity curve.) Figure 11 shows that during human motion, acceleration can fluctuate locally, resulting in discontinuous jumps. Direct application to robots can cause severe vibration and damage. Experiments demonstrate that the HMTF method learns the global characteristics of humans while filtering out local fluctuations. Consequently, the velocity curves are smoother, indicating smoother velocity variations in the generated trajectories. This demonstrates that the HMTF method improves trajectory smoothness. The TOPP method, on the other hand, mostly generates faster but less anthropomorphic trajectories. Specifically, because the TOPP method always drives the robot to its maximum capacity through the path, the velocity distributions across all trajectories under different operational modes are similar. For example, the velocity curves exhibit bell-shaped or ∧-shaped curves at the robot's maximum velocity. In contrast, the velocity distribution of the trajectory obtained by the HMTF method exhibits different temporal characteristics for different operation modes. For example, in operation mode 1, humans may first accelerate and then decelerate at a relatively low acceleration when they are about to grasp the object in order to stabilize the desired grasping posture. In contrast, in operation mode 4, when they are close to the container to be filled with water, humans may decelerate at a relatively high acceleration to avoid leaking when pouring water. The HMTF method exploits the spatiotemporal correlations in the human manipulation trajectory and successfully parameterizes the robot's path into a trajectory with anthropomorphic temporal characteristics without the need for explicit prompts from the task. In addition, the results show that humans tend to move in a relatively slow and stable manner. This shows that the TOPP method, which only emphasizes the time-optimal method, may mislead the robot's motion planning.

[0185] Experiments have shown that the robot trajectories planned by the TOPP method are unlike human motion trajectories and are more suitable for scenarios that emphasize operational efficiency. However, human motion trajectories are temporally and spatially correlated and highly expressive. By modeling these temporal and spatial correlations, the trained path parameterization model learns the underlying anthropomorphic temporal features for mobility tasks, achieving significant improvements in anthropomorphism and smoothness, achieving reasonable, high-quality, and anthropomorphic path parameterization. The method can also summarize motion trajectories under different operating modes, making it flexible enough to be applied to motion planning under multiple operating modes.

[0186] In addition, the method provided in the embodiment of the present application was used to generate multiple random robot trajectories. In Figure 12, eight trajectories and a speed bar are shown in a three-dimensional coordinate system consisting of the x-axis, y-axis, and z-axis. The speed bar indicates that the lower the transparency of the trajectory, the greater the speed. The position of the arrow indicates the starting point of the trajectory, and the arrow points to the direction of movement. As can be seen from Figure 12, in most cases, the robot starts to accelerate, then decelerates, and finally stops at the end point. The robot often decelerates before entering a highly curved turning point where a collision may be avoided, and then accelerates or decelerates on a flatter path, which is similar to the movement patterns of humans. Compared with straight or planar trajectories, curved trajectories have been widely proven to be more expressive, and experimental results show that the method provided in the embodiment of the present application can utilize the learned spatiotemporal correlations to express anthropomorphic time characteristics in the trajectory, thereby reflecting the robot's intentions or emotions.

[0187] FIG13 is a schematic diagram of the structure of a path parameterization device provided in an embodiment of the present application. Referring to FIG13 , the device includes:

[0188] A model acquisition module 1301 is configured to acquire a path parameterization model. The path parameterization model is trained based on sample trajectory data. The sample trajectory data includes sample coordinates and sample timestamps of at least two sample waypoints in the sample path. The sample timestamps indicate the times when the training subject reaches each of the at least two sample waypoints while traversing the sample path.

[0189] A path acquisition module 1302 is configured to acquire a first path and obtain first coordinates of at least two first waypoints in the first path;

[0190] The path parameterization module 1303 is configured to determine first trajectory data based on the first coordinates, where the first trajectory data includes first timestamps corresponding to at least two first waypoints. The first trajectory data is configured to control the robot to move along the first path so that the robot reaches the at least two first waypoints at the first waypoint times indicated by the first timestamps.

[0191] In an embodiment of the present application, a path parameterization model is trained based on sample trajectory data. Since the sample trajectory data includes the sample coordinates and sample timestamps of at least two sample waypoints in the sample path, and the sample timestamps refer to the sample waypoint moments when the training object arrives at at least two sample waypoints respectively in the process of passing through the sample path, the sample trajectory data is trajectory data that conforms to the motion laws of the training object. The trained path parameterization model has the ability to generate trajectory data that conforms to the motion laws of the training object. Therefore, the trajectory data corresponding to the path data of the robot is determined through the path parameterization model, which ensures that the trajectory data contains the characteristics of the training object, is more consistent with the motion laws of the training object, and improves the rationality of the trajectory data. When the trajectory data is used to control the robot to move on the path, the robot's movement is more in line with the training object's movement habits.

[0192] In one possible implementation, referring to FIG. 14 , the path parameterization module 1303 includes:

[0193] A sequence forming unit 1313 is configured to determine a first coordinate sequence based on the first coordinates of at least two first waypoints;

[0194] The parameterization unit 1323 is configured to determine a first timestamp sequence and a first duration corresponding to the first coordinate sequence through a path parameterization model, where the first duration represents the duration required for the robot to pass through the first path.

[0195] In one possible implementation, the path parameterization model includes at least one encoder, at least one decoder, a normalization layer, and a summation layer, and the parameterization unit 1323 is configured to:

[0196] Encoding the first coordinate sequence by at least one encoder to obtain an encoded feature;

[0197] decoding the encoded features by at least one decoder to obtain decoded features;

[0198] Normalizing the decoded features through a normalization layer to obtain a first timestamp sequence;

[0199] The decoded features are summed through the summation layer to obtain the first duration.

[0200] In one possible implementation, referring to FIG14 , the apparatus further includes:

[0201] The sample data acquisition module 1304 is configured to acquire first sample trajectory data, where the first sample trajectory data includes initial coordinates and timestamps of at least two waypoints.

[0202] A normalization module 1305 is configured to normalize the initial sample coordinates and sample timestamps of at least two sample waypoints to obtain normalized sample coordinates and normalized sample timestamps of the at least two sample waypoints;

[0203] A composition module 1306 is configured to combine the normalized sample coordinates and the normalized sample timestamps of at least two sample waypoints into second sample trajectory data;

[0204] The model training module 1307 is configured to train a path parameterization model based on the second sample trajectory data.

[0205] In one possible implementation, the normalization module 1305 is configured to:

[0206] Determine the normalized sample coordinates of each sample waypoint based on the initial sample coordinates of each sample waypoint and the initial sample coordinates of the first sample waypoint of the at least two sample waypoints;

[0207] The difference between the sample timestamp of each sample waypoint and the sample timestamp of the first sample waypoint is determined as the duration corresponding to each sample waypoint, where the duration corresponding to a sample waypoint refers to the duration from the first sample waypoint to the sample waypoint;

[0208] The ratio of the duration corresponding to each sample waypoint to the duration corresponding to the last sample waypoint of the at least two sample waypoints is respectively determined as the normalized sample timestamp of each sample waypoint.

[0209] In one possible implementation, referring to FIG14 , the apparatus further includes:

[0210] The sample data acquisition module 1304 is configured to acquire a demonstration video, the demonstration video including a frame of a training object passing through a sample path; and acquire sample trajectory data based on the coordinates of the training object in the frame of the demonstration video and the timestamp corresponding to the frame.

[0211] The model training module 1307 is used to train a path parameterization model based on sample trajectory data.

[0212] In one possible implementation, referring to FIG14 , the apparatus further includes a model training module 1307, which includes:

[0213] A first sequence unit 1317 is configured to combine the coordinates of at least two sample waypoints into a coordinate sequence;

[0214] A second sequence unit 1327 is configured to combine sample timestamps of at least two sample waypoints into a first sample timestamp sequence;

[0215] A first parameterization unit 1337 is configured to determine a second sample timestamp sequence corresponding to the coordinate sequence through a path parameterization model;

[0216] The parameter adjustment unit 1347 is configured to adjust the model parameters of the path parameterization model based on the error between the first sample timestamp sequence and the second sample timestamp sequence.

[0217] In one possible implementation, referring to FIG. 14 , the model training module 1307 further includes:

[0218] A first duration unit 1357 is configured to determine a difference between a sample timestamp of the last sample waypoint and a sample timestamp of the first sample waypoint as a first duration;

[0219] A second parameterization unit 1367 is configured to determine a second duration corresponding to the second sample timestamp sequence using a path parameterization model, where the second duration represents a duration required to pass through the sample path as predicted by the path parameterization model.

[0220] The parameter adjustment unit 1347 is configured to adjust the model parameters of the path parameterization model based on the error between the first sample timestamp sequence and the second sample timestamp sequence, and the error between the first duration and the second duration.

[0221] In one possible implementation, the error between the first sample timestamp sequence and the second sample timestamp sequence includes: a first error and a second error;

[0222] wherein the distance between the sample timestamp of each sample waypoint in the first sample timestamp sequence and the sample timestamp of the same sample waypoint in the second sample timestamp sequence is positively correlated with the first error;

[0223] The distance between the second duration and the first duration is positively correlated with the second error.

[0224] It should be noted that the robot motion control device provided in the above embodiment is merely exemplified by the division of the aforementioned functional modules. In actual applications, the aforementioned functions can be assigned to different functional modules as needed, i.e., the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the robot motion control device provided in the above embodiment and the robot motion control method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0225] The present application also provides a robot motion control device, the device comprising:

[0226] a model acquisition module, configured to acquire a path parameterized model, the path parameterized model being trained based on sample trajectory data, the sample trajectory data including sample coordinates and sample timestamps of at least two sample waypoints in a sample path, the timestamps indicating the sample waypoint moments at which the training subject respectively reaches the at least two sample waypoints while traversing the sample path;

[0227] A path acquisition module, configured to acquire a second path and at least two second timestamps;

[0228] a path parameterization module, configured to determine, using the path parameterization model and based on the second timestamp, second trajectory data, wherein the second trajectory data includes second coordinates corresponding to at least two second waypoints, respectively; and the trajectory data is used to control the robot to move along the second path so that the robot reaches the at least two second waypoints at the second waypoint times indicated by the second timestamps.

[0229] In some embodiments, a path parameterization module is configured to determine a second timestamp sequence and a second duration based on the at least two second timestamps, wherein the second duration represents a duration required for the robot to pass through the second path;

[0230] A second coordinate sequence corresponding to the second timestamp sequence and the second duration is determined by using the path parameterization model, where the second coordinate sequence includes second coordinates corresponding to each of the at least two second waypoints.

[0231] An embodiment of the present application also provides a computer device, which includes a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed in the robot motion control method of the above embodiment.

[0232] FIG15 shows a schematic structural diagram of a computer device 1500 provided in an exemplary embodiment of the present application.

[0233] The computer device 1500 includes a processor 1501 and a memory 1502 .

[0234] The processor 1501 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1501 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), or PLA (Programmable Logic Array). The processor 1501 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In one possible implementation, the processor 1501 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1501 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0235] The memory 1502 may include one or more computer-readable storage media, which may be non-transitory. The memory 1502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In one possible implementation, the non-transitory computer-readable storage medium in the memory 1502 is used to store at least one computer program, which is used by the processor 1501 to implement the robot motion control method provided in the method embodiment of the present application.

[0236] In one possible implementation, computer device 1500 may further optionally include a peripheral device interface 1503 and at least one peripheral device. Processor 1501, memory 1502, and peripheral device interface 1503 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 1503 via a bus, signal lines, or circuit boards. Optionally, the peripheral device includes at least one of a radio frequency circuit 1504, a display screen 1505, a camera assembly 1506, and a power supply 1507.

[0237] The peripheral device interface 1503 can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor 1501 and the memory 1502. In one possible implementation, the processor 1501, the memory 1502, and the peripheral device interface 1503 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1501, the memory 1502, and the peripheral device interface 1503 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0238] RF circuit 1504 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. RF circuit 1504 communicates with communication networks and other communication devices via electromagnetic signals. RF circuit 1504 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. RF circuit 1504 optionally includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. RF circuit 1504 can communicate with other devices via at least one wireless communication protocol. Such wireless communication protocols include, but are not limited to, metropolitan area networks, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In one possible implementation, RF circuit 1504 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0239] Display screen 1505 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1505 is a touch screen display, display screen 1505 also has the ability to collect touch signals on or above the surface of display screen 1505. The touch signals can be input as control signals to processor 1501 for processing. In this case, display screen 1505 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In one possible implementation, display screen 1505 can be one, disposed on the front panel of computer device 1500; in other embodiments, display screen 1505 can be at least two, disposed on different surfaces of computer device 1500 or in a foldable design; in other embodiments, display screen 1505 can be a flexible display, disposed on a curved surface or a foldable surface of computer device 1500. Display screen 1505 can even be configured as a non-rectangular irregular shape, i.e., a special-shaped screen. The display screen 1505 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0240] The camera assembly 1506 is used to capture images or videos. Optionally, the camera assembly 1506 includes a front camera and a rear camera. The front camera is set on the front panel of the computer device 1500, and the rear camera is set on the back of the computer device 1500. In one possible implementation, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In one possible implementation, the camera assembly 1506 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. The dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation under different color temperatures.

[0241] Power supply 1507 is used to power the various components of computer device 1500. Power supply 1507 can be AC ​​power, DC power, disposable batteries, or rechargeable batteries. When power supply 1507 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0242] Those skilled in the art will appreciate that the structure shown in FIG. 15 does not limit the computer device 1500 , and may include more or fewer components than shown, or combine certain components, or adopt a different component arrangement.

[0243] An embodiment of the present application also provides a computer-readable storage medium, which stores at least one computer program. The at least one computer program is loaded and executed by a processor to implement the operations performed by the robot motion control method of the above embodiment.

[0244] An embodiment of the present application further provides a computer program product, including a computer program, which is loaded and executed by a processor to implement the operations performed by the robot motion control method of the above embodiment.

[0245] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0246] The above description is merely an optional embodiment of the embodiments of the present application and is not intended to limit the embodiments of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the embodiments of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for controlling robot motion, executed by a computer device, the method comprising: Obtaining a path parameterized model, the path parameterized model trained based on sample trajectory data, the sample trajectory data including sample coordinates and sample timestamps of at least two sample waypoints in the sample path, the sample timestamps indicating the sample waypoint moments when the training subject respectively arrives at the at least two sample waypoints while traversing the sample path; Acquire a first path, and acquire first coordinates of at least two first waypoints in the first path; First trajectory data is determined based on the first coordinates using the path parameterized model, where the first trajectory data includes first timestamps corresponding to each of the at least two first waypoints. The first trajectory data is used to control the robot to perform movement on the first path so that the robot reaches the at least two first waypoints at the first waypoint times indicated by the first timestamps.

2. The method according to claim 1, wherein The determining first trajectory data based on the first coordinate using the path parameterized model includes: determining a first coordinate sequence based on the first coordinates of at least two of the first waypoints; A first timestamp sequence and a first duration corresponding to the first coordinate sequence are determined using the path parameterization model, where the first duration represents the duration required for the robot to pass through the first path.

3. The method according to claim 2, wherein: The path parameterization model includes at least one encoder, at least one decoder, a normalization layer, and a summation layer. Determining the first timestamp sequence and the first duration corresponding to the first coordinate sequence through the path parameterization model includes: Encoding the first coordinate sequence by at least one encoder to obtain an encoding feature; Decoding the encoded features by at least one decoder to obtain decoded features; Normalizing the decoded features through the normalization layer to obtain the first timestamp sequence; The decoded features are summed through the summation layer to obtain the first duration.

4. The method according to any one of claims 1 to 3, wherein The method further comprises: Acquire first sample trajectory data, where the first sample trajectory data includes initial sample coordinates and sample timestamps of at least two sample waypoints; Normalizing the initial sample coordinates and sample timestamps of the at least two sample waypoints respectively to obtain normalized sample coordinates and normalized sample timestamps of the at least two sample waypoints; Combining the normalized sample coordinates and the normalized sample timestamps of the at least two sample waypoints into second sample trajectory data; The path parameterization model is trained based on the second sample trajectory data.

5. The method according to claim 4, wherein Normalizing the initial sample coordinates and sample timestamps of the at least two sample waypoints to obtain normalized sample coordinates and normalized sample timestamps of the at least two sample waypoints includes: Determining normalized sample coordinates of each sample waypoint based on the initial sample coordinates of each sample waypoint and the initial sample coordinates of the first sample waypoint of the at least two sample waypoints; Determine the difference between the sample timestamp of each sample waypoint and the sample timestamp of the first sample waypoint as the duration corresponding to each sample waypoint, where the duration corresponding to the sample waypoint refers to the duration from the first sample waypoint to the sample waypoint; The ratio of the duration corresponding to each sample waypoint to the duration corresponding to the last sample waypoint of the at least two sample waypoints is respectively determined as the normalized sample timestamp of each sample waypoint.

6. The method according to any one of claims 1 to 5, wherein The method further comprises: Obtaining a demonstration video, wherein the demonstration video includes a picture of the training object passing through the sample path; Acquire the sample trajectory data based on the coordinates of the training object in the picture of the demonstration video and the timestamp corresponding to the picture; The path parameterization model is trained based on the sample trajectory data.

7. The method according to any one of claims 1 to 6, wherein Training the path parameterization model based on the sample trajectory data includes: Combining the sample coordinates of the at least two sample waypoints into a sample coordinate sequence; Combining the sample timestamps of the at least two sample waypoints into a first sample timestamp sequence; Determining a second sample timestamp sequence corresponding to the sample coordinate sequence using the path parameterization model; Based on the error between the first sample timestamp sequence and the second sample timestamp sequence, a model parameter of the path parameterization model is adjusted.

8. The method according to claim 7, wherein: Training the path parameterization model based on the sample trajectory data further includes: Determine a difference between a sample timestamp of a last sample waypoint among the at least two sample waypoints and a sample timestamp of a first sample waypoint among the at least two sample waypoints as a first duration; Determining a second duration corresponding to the second sample timestamp sequence using the path parameterization model, where the second duration represents a duration required to pass through the sample path as predicted by the path parameterization model; The adjusting the model parameters of the path parameterization model based on the error between the first sample timestamp sequence and the second sample timestamp sequence includes: Model parameters of the path parameterization model are adjusted based on an error between the first sample timestamp sequence and the second sample timestamp sequence, and an error between the first duration and the second duration.

9. The method according to claim 8, wherein The error between the first sample timestamp sequence and the second sample timestamp sequence includes: a first error and a second error; wherein the distance between the sample timestamp of each sample waypoint in the first sample timestamp sequence and the sample timestamp of the same sample waypoint in the second sample timestamp sequence is positively correlated with the first error; The distance between the second duration and the first duration is positively correlated with the second error.

10. A method for controlling robot motion, executed by a computer device, the method comprising: Obtaining a path parameterized model, the path parameterized model trained based on sample trajectory data, the sample trajectory data including sample coordinates and sample timestamps of at least two sample waypoints in the sample path, the timestamps indicating the sample waypoint moments when the training subject respectively arrived at the at least two sample waypoints while traversing the sample path; Obtain a second path and obtain at least two second timestamps; Second trajectory data is determined based on the second timestamp using the path parameterization model, where the second trajectory data includes second coordinates corresponding to at least two second waypoints, respectively. The second trajectory data is used to control the robot to move on the second path so that the robot reaches the at least two second waypoints at the second waypoint times indicated by the second timestamp.

11. The method according to claim 10, wherein: The determining, using the path parameterized model and based on the second timestamp, second trajectory data includes: determining a second timestamp sequence and a second duration based on the at least two second timestamps, wherein the second duration represents a duration required for the robot to pass through the second path; A second coordinate sequence corresponding to the second timestamp sequence and the second duration is determined by using the path parameterization model, where the second coordinate sequence includes second coordinates corresponding to each of the at least two second waypoints.

12. A robot motion control device, comprising: a model acquisition module, configured to acquire a path parameterized model, the path parameterized model being trained based on sample trajectory data, the sample trajectory data including sample coordinates and sample timestamps of at least two sample waypoints in a sample path, the sample timestamps indicating the sample waypoint moments when the training subject respectively reaches the at least two sample waypoints while traversing the sample path; a path acquisition module, configured to acquire a first path and obtain first coordinates of at least two first waypoints in the first path; a path parameterization module, configured to determine first trajectory data based on the first coordinates, the first trajectory data including first timestamps corresponding to each of the at least two first waypoints, the first trajectory data being used to control the robot to perform movement on the first path so that the robot reaches the at least two first waypoints at first waypoint times indicated by the first timestamps.

13. A robot motion control device, comprising: a model acquisition module, configured to acquire a path parameterized model, the path parameterized model being trained based on sample trajectory data, the sample trajectory data including sample coordinates and sample timestamps of at least two sample waypoints in a sample path, the timestamps indicating the sample waypoint moments at which the training subject respectively reaches the at least two sample waypoints while traversing the sample path; A path acquisition module, configured to acquire a second path and at least two second timestamps; a path parameterization module, configured to determine, using the path parameterization model and based on the second timestamp, second trajectory data, wherein the second trajectory data includes second coordinates corresponding to at least two second waypoints, respectively; and the trajectory data is used to control the robot to move along the second path so that the robot reaches the at least two second waypoints at the second waypoint times indicated by the second timestamps.

14. A computer device, comprising a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed by the robot motion control method according to any one of claims 1 to 11.

15. A computer-readable storage medium, wherein at least one computer program is stored in the computer-readable storage medium, wherein the at least one computer program is loaded and executed by a processor to implement the operations performed by the robot motion control method according to any one of claims 1 to 11.

16. A computer program product, comprising a computer program, wherein the computer program is loaded and executed by a processor to implement the operations performed by the robot motion control method according to any one of claims 1 to 11.

Images