Method, device and processing equipment for generating a motion trajectory

Abstract

This application relates to the field of motion path generation technology, and provides a method, apparatus, and processing device for generating motion trajectories, including: receiving a three-dimensional point cloud image of a target object sent by an image acquisition device; receiving a two-dimensional trajectory image, the two-dimensional trajectory image being obtained by determining a first motion trajectory in the two-dimensional image of the target object; determining the projection of the first motion trajectory in the two-dimensional trajectory image onto the three-dimensional point cloud image according to the mapping relationship between the two-dimensional image and the three-dimensional point cloud image, thereby obtaining a first target point cloud; determining the trajectory points of a robotic arm and the corresponding postures of each trajectory point according to the first target point cloud; and generating a second motion trajectory of the robotic arm based on the trajectory points of the robotic arm and the corresponding postures of each trajectory point, wherein the second motion trajectory is a three-dimensional motion trajectory. This method can improve the speed and accuracy of the obtained motion trajectory.

Description

Technical Field

[0001] This application belongs to the field of motion path generation technology, and particularly relates to methods, apparatus, processing equipment and computer-readable storage media for generating motion trajectories. Background Technology

[0002] When a robotic arm performs grinding or massage on a specific area, it needs to follow a specified trajectory path. This specified trajectory path is usually determined by manually dragging the robotic arm across the surface of the object for teaching purposes. In other words, the robotic arm is pre-taught to move along a specified trajectory, and then the motion path of the specified trajectory is generated.

[0003] However, since drag-and-teach involves a human dragging the robotic arm, requiring human-machine collaboration, the accuracy of the motion path generated by drag-and-teach varies from person to person. Furthermore, if the required motion trajectory is complex, the teaching process will be time-consuming. Additionally, when drag-and-teach on uneven curved surfaces, the handling of turns may be poor, resulting in an uneven teaching trajectory. In such cases, many attempts may be required to achieve success, significantly reducing efficiency.

[0004] In summary, it is difficult to quickly and accurately determine the movement trajectory of a robotic arm by manually dragging and teaching it. Summary of the Invention

[0005] This application provides a method, apparatus, and processing device for generating motion trajectories, which can solve the problems of slow speed and low accuracy in generating motion trajectories for robotic arms.

[0006] In a first aspect, embodiments of this application provide a method for generating motion trajectories, applied to a controller of a robotic arm, comprising:

[0007] Receive a 3D point cloud image of the target object sent by the image acquisition device;

[0008] Receive a two-dimensional trajectory map, which is obtained by determining a first motion trajectory in a two-dimensional image of the target body, wherein the first motion trajectory is a two-dimensional motion trajectory;

[0009] Based on the mapping relationship between the two-dimensional image and the three-dimensional point cloud map, the projection of the first motion trajectory in the two-dimensional trajectory map onto the three-dimensional point cloud map is determined to obtain the first target point cloud.

[0010] The trajectory points of the robotic arm and the posture corresponding to each trajectory point are determined based on the first target point cloud.

[0011] Based on the trajectory points of the robotic arm and the postures corresponding to each trajectory point, a second motion trajectory of the robotic arm is generated. The second motion trajectory is a three-dimensional motion trajectory.

[0012] Secondly, embodiments of this application provide a motion trajectory generation device, applied to a controller of a robotic arm, comprising:

[0013] The 3D point cloud acquisition module is used to receive the 3D point cloud image of the target object sent by the image acquisition device;

[0014] A two-dimensional trajectory acquisition module is used to receive a two-dimensional trajectory, which is obtained by determining a first motion trajectory in a two-dimensional image of the target body, wherein the first motion trajectory is a two-dimensional motion trajectory;

[0015] The first target point cloud determination module is used to determine the projection of the first motion trajectory in the two-dimensional trajectory map onto the three-dimensional point cloud map based on the mapping relationship between the two-dimensional image and the three-dimensional point cloud map, so as to obtain the first target point cloud.

[0016] The attitude determination module is used to determine the trajectory points of the robotic arm and the attitude corresponding to each trajectory point based on the first target point cloud.

[0017] The second motion trajectory determination module is used to generate a second motion trajectory of the robotic arm based on the trajectory points of the robotic arm and the postures corresponding to each trajectory point. The second motion trajectory is a three-dimensional motion trajectory.

[0018] Thirdly, embodiments of this application provide a processing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect.

[0019] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0020] Fifthly, embodiments of this application provide a computer program product that, when run on a processing device, causes the processing device to perform the method described in the first aspect above.

[0021] The beneficial effects of the embodiments in this application compared with the prior art are:

[0022] In this embodiment, since the first motion trajectory is a two-dimensional motion trajectory determined in the two-dimensional image of the target body, the first motion trajectory can be quickly and accurately determined based on the two-dimensional trajectory map. At the same time, since there is a specific mapping relationship between the two-dimensional image of the target body and the three-dimensional point cloud map, the second motion trajectory (i.e., the three-dimensional motion trajectory) corresponding to the first motion trajectory can be determined based on the mapping relationship. Furthermore, since the generation process of the second motion trajectory does not require manual dragging of the robotic arm, the generation accuracy and precision of the second motion trajectory are further improved. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0024] Figure 1 This is a flowchart of a method for generating a motion trajectory according to an embodiment of this application;

[0025] Figure 2 This is an interactive flowchart of a 3D camera, a host computer, and a controller provided in one embodiment of this application;

[0026] Figure 3 This is a schematic diagram illustrating an application scenario of a 3D camera, host computer, and controller provided in an embodiment of this application;

[0027] Figure 4 This is a schematic diagram of the structure of a motion trajectory generation device provided in an embodiment of this application;

[0028] Figure 5 This is a schematic diagram of the structure of a processing device provided in one embodiment of this application. Detailed Implementation

[0029] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0030] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0031] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0032] Furthermore, in the description of this application and the appended claims, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0033] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0034] Example 1:

[0035] When a robot arm is manually dragged along a specified trajectory to polish a certain area, the accuracy and speed of the resulting motion trajectory are not high because the coordination between the human and the robot arm varies from person to person and the speed at which the human drags the robot arm is relatively low.

[0036] To improve the accuracy and speed of the motion trajectory obtained when a robotic arm grinds a certain area, this application provides a method for generating a motion trajectory. In this method, a two-dimensional image and a three-dimensional point cloud map of the target object from which the motion trajectory is to be generated are first acquired. After determining the two-dimensional motion trajectory on the two-dimensional image, the two-dimensional motion trajectory is projected onto the point cloud of the three-dimensional point cloud map to obtain a first target point cloud. Then, the trajectory points and posture of the robotic arm are determined based on the first target point cloud to generate a three-dimensional motion trajectory.

[0037] The method for generating motion trajectories according to embodiments of this application will now be described with reference to the accompanying drawings.

[0038] Figure 1 A flowchart illustrating a motion trajectory generation method provided in an embodiment of this application is shown, applied to a robotic arm controller, and is described in detail below:

[0039] Step S11: Receive the three-dimensional point cloud map of the target object sent by the image acquisition device.

[0040] The target body is the object on which the robotic arm needs to acquire the motion trajectory. The target body includes one or more of the following: human body, designated area, and designated type of object.

[0041] In this embodiment, the target object can be photographed using an image acquisition device (such as a 3D camera) to obtain a two-dimensional image and a three-dimensional point cloud map corresponding to the target object (i.e., the three-dimensional point cloud map is a three-dimensional image of the target object, and the two-dimensional image is a two-dimensional image of the target object). The three-dimensional point cloud map of the target object is then sent to the controller, or both the two-dimensional image and the three-dimensional point cloud map of the target object are sent to the controller. The three-dimensional point cloud map of the target object includes the point cloud of the target object.

[0042] Step S12: Receive a two-dimensional trajectory map. The two-dimensional trajectory map is obtained by determining the first motion trajectory in the two-dimensional image of the target body, wherein the first motion trajectory is a two-dimensional motion trajectory.

[0043] In this embodiment, the image acquisition device sends a two-dimensional image to a host computer (such as a computer or mobile phone). After displaying the two-dimensional image using image editing software (such as a drawing board) installed on the host computer, the user can directly draw the first motion trajectory of the two-dimensional image in the image editing software. After the drawing is completed, the host computer sends the two-dimensional image with the first motion trajectory drawn (i.e., the two-dimensional trajectory diagram) to the controller. The first motion trajectory is the two-dimensional motion trajectory that the user wants the robotic arm to move on the target body. For example, if the robotic arm is a massage type robotic arm and the target body is a human body, and the user wants the robotic arm to move from the left shoulder to the right shoulder, then the user draws the motion trajectory from the left shoulder to the right shoulder in the two-dimensional image corresponding to the human body as the first motion trajectory in this embodiment.

[0044] In some embodiments, the image acquisition device sends a two-dimensional image to a host computer. The host computer stores preset algorithms. After receiving the two-dimensional image, the host computer draws a first motion trajectory on the two-dimensional image according to the algorithm, generating a two-dimensional trajectory map, which is then sent to the controller. In some embodiments, the host computer stores multiple preset algorithms. The host computer draws a first motion trajectory on the two-dimensional image according to the algorithm selected by the user, generating a two-dimensional trajectory map. In some embodiments, the host computer selects an algorithm that matches the rules set by the user from multiple preset algorithms, and then draws a first motion trajectory on the two-dimensional image according to the algorithm, generating a two-dimensional trajectory map.

[0045] In some embodiments, the image acquisition device sends a two-dimensional image to a host computer, which stores preset algorithms. The host computer then sends the two-dimensional image and the preset algorithm to a controller. After receiving the two-dimensional image and the algorithm, the controller draws a first motion trajectory on the two-dimensional image according to the algorithm, generating a two-dimensional trajectory map. In some embodiments, the host computer stores multiple preset algorithms. The host computer sends the algorithm selected by the user and the two-dimensional image to the controller. In some embodiments, the host computer selects an algorithm that matches the rules set by the user from multiple preset algorithms, and then sends the algorithm and the two-dimensional image to the controller, so that the controller draws a first motion trajectory on the two-dimensional image according to the algorithm, generating a two-dimensional trajectory map. That is, when the controller receives the two-dimensional trajectory map, it is essentially receiving the two-dimensional image and the corresponding algorithm.

[0046] In some embodiments, the controller stores a preset algorithm. The image acquisition device sends a two-dimensional image to the controller, and the controller draws a first motion trajectory on the two-dimensional image according to the preset algorithm to generate a two-dimensional trajectory map.

[0047] Step S13: Based on the mapping relationship between the two-dimensional image and the three-dimensional point cloud map, determine the projection of the first motion trajectory in the two-dimensional trajectory map onto the three-dimensional point cloud map to obtain the first target point cloud.

[0048] In this embodiment of the application, since there is a certain mapping relationship (or projection relationship) between the two-dimensional image and the three-dimensional point cloud map of the same target, and there is also the same mapping relationship between the two-dimensional trajectory map and the three-dimensional point cloud map, the point cloud corresponding to the first motion trajectory can be found in the three-dimensional point cloud map according to the mapping relationship between the two-dimensional image and the three-dimensional point cloud map.

[0049] In some embodiments, the image acquisition device is a 3D camera (or stereo camera). The 2D image acquired by the 3D camera has a pixel dimension of w*h, where w is the width of the 2D image and h is the height of the 2D image. After acquiring the image, the 3D camera uses an alignment algorithm (an algorithm built into the 3D camera) to align the acquired 2D image with the depth image. The pixel dimension of the aligned depth image is also w*h, thus establishing a one-to-one correspondence between the pixels of the 2D image and the pixels of the depth image. The 3D camera then calculates a 3D point cloud map based on its intrinsic and extrinsic parameters and the depth image. The 3D point cloud map is a vector of dimension w*h, where each value is a 3D coordinate representing the coordinates of points corresponding to the target object. Therefore, each pixel in the 2D image corresponds one-to-one with each vector value in the 3D point cloud map, forming a mapping relationship between the 2D image and the 3D point cloud map. The 3D camera sends the 2D image, the 3D point cloud map, and the mapping relationship to the controller.

[0050] Step S14: Determine the trajectory points of the robotic arm and the postures corresponding to each trajectory point based on the first target point cloud.

[0051] Since the first motion trajectory is the two-dimensional motion trajectory that the user wants the robotic arm to move on the target body, and the first target point cloud is the three-dimensional coordinate points determined based on the determined first motion trajectory, the three-dimensional trajectory points of the robotic arm and the posture corresponding to each trajectory point can be determined based on each three-dimensional coordinate point.

[0052] Step S15: Based on the trajectory points of the robotic arm and the postures corresponding to each trajectory point, generate the second motion trajectory of the robotic arm. The second motion trajectory is a three-dimensional motion trajectory.

[0053] In this embodiment, after acquiring the three-dimensional point cloud map and the two-dimensional trajectory map of the target object, the point cloud corresponding to the first motion trajectory in the two-dimensional trajectory map is determined according to the mapping relationship between the two-dimensional image and the three-dimensional point cloud map of the target object, thus obtaining the first target point cloud. Then, the trajectory points of the robotic arm and the postures corresponding to each trajectory point are determined according to the first target point cloud to determine the second motion trajectory of the robotic arm. Since the first motion trajectory is a two-dimensional motion trajectory determined in the two-dimensional image of the target object, the first motion trajectory can be determined quickly and accurately according to the two-dimensional trajectory map. At the same time, since there is a specific mapping relationship between the two-dimensional image and the three-dimensional point cloud map of the target object, the second motion trajectory corresponding to the first motion trajectory can be determined according to the mapping relationship. Furthermore, since the generation process of the second motion trajectory does not require manual dragging of the robotic arm, the generation accuracy and precision of the second motion trajectory are further improved.

[0054] In this application, if the image acquisition device is a 3D camera and the image editing software is image editing software on a host computer (such as a drawing board), then the interaction flowchart between the 3D camera, the host computer, and the controller can be as follows: Figure 2 As shown.

[0055] To more clearly describe the motion trajectory generation method provided in the embodiments of this application, an application scenario is described below.

[0056] like Figure 3As shown, the image acquisition device takes pictures of a human body (or a human body model) to obtain a 3D point cloud map and a 2D image of the human body (which is the target body of this application). The image acquisition device then sends the 3D point cloud map and the 2D image to a controller. The controller sends the 2D image to a host computer (in some embodiments, the image acquisition device sends the 2D image to the host computer). The host computer is equipped with image editing software. The user draws a first motion trajectory on the 2D image displayed by the image editing software through the host computer, obtaining a 2D image with the first motion trajectory drawn (i.e., a 2D trajectory map). The host computer then sends the 2D trajectory map to the controller. The controller determines the projection of the first motion trajectory in the 2D trajectory map onto the 3D point cloud map based on the mapping relationship between the 2D image and the 3D point cloud map, obtaining a first target point cloud. Based on the first target point cloud, the controller determines the trajectory points of the robotic arm and the corresponding postures of each trajectory point. Then, based on the trajectory points of the robotic arm and the corresponding postures of each trajectory point, it generates a second motion trajectory for the robotic arm. When the robotic arm needs to massage the human body, the controller controls the robotic arm to massage the human body according to the second motion trajectory.

[0057] In some embodiments, to improve the speed of obtaining the first target point cloud, step S13 includes:

[0058] A1. Perform skeleton extraction processing on the first motion trajectory in the above two-dimensional trajectory diagram to obtain the target skeleton.

[0059] Skeleton extraction, also known as skeleton extraction, can be performed using existing methods, such as methods based on fire simulation or methods based on the maximum disk. The extracted skeleton can highlight the main structure and shape information of the object and remove redundant information.

[0060] In this embodiment, a first motion trajectory is extracted from the two-dimensional trajectory map as the skeleton to be extracted (i.e., the target skeleton) through skeleton extraction processing. That is, in this embodiment, since only the first motion trajectory is extracted as the target skeleton, subsequent processing only needs to be performed on the pixel coordinates of the target skeleton, without processing the pixel coordinates of the entire two-dimensional trajectory map, thereby effectively reducing the number of pixel coordinates that need to be processed.

[0061] A2. Sort the pixel coordinates of the target skeleton to obtain the sorted pixel coordinates.

[0062] Since the target skeleton is the skeleton corresponding to the first motion trajectory, and the first motion trajectory has an order, for example, if the starting point of the first motion trajectory is A and the ending point is B, then the order of the pixel coordinates corresponding to the first motion trajectory is that the closer to point A, the earlier the sorting, and the closer to point B, the later the sorting.

[0063] A3. Based on the mapping relationship between the two-dimensional image and the three-dimensional point cloud, the sorted pixel coordinates are projected onto the point cloud of the three-dimensional point cloud to obtain the first target point cloud.

[0064] Among them, each target point in the first target point cloud also has an order relationship with the sorted pixel coordinates.

[0065] In this embodiment, since the first motion trajectory is extracted from the two-dimensional trajectory map and then projected onto the three-dimensional point cloud map, the number of pixel coordinates that need to be projected is reduced, thereby improving the speed of obtaining the first target point cloud.

[0066] In some embodiments, to improve the accuracy of the obtained target trajectory, step A1 above includes:

[0067] A11. Convert the above two-dimensional trajectory diagram into a grayscale image and perform binarization.

[0068] A12. Perform skeleton extraction processing on the first motion trajectory in the binarized two-dimensional trajectory graph to obtain the target skeleton.

[0069] In this embodiment of the application, when the two-dimensional trajectory image is a red-green-blue (RGB) color image, the two-dimensional trajectory image is converted into a grayscale image and then binarized to increase the difference between the first motion trajectory and other objects in the two-dimensional trajectory image, thereby facilitating the accurate extraction of the target skeleton from the binarized two-dimensional trajectory image.

[0070] In some embodiments, to improve the speed of obtaining trajectory points, step S14 includes:

[0071] B1. Downsample the first target point cloud to obtain the second target point cloud.

[0072] Specifically, after downsampling the first target point cloud, the number of target points in the resulting second target point cloud is less than the number of target points in the first target point cloud. In some embodiments, to facilitate extraction, corresponding target points are extracted from the first target point cloud at equal intervals, so that the distance between adjacent target points in the second target point cloud is equal.

[0073] B2. Determine the trajectory points of the robotic arm and the corresponding postures of each trajectory point based on the second target point cloud.

[0074] In this embodiment of the application, since the second target point cloud is obtained by downsampling the first target point cloud, the number of target points in the obtained second target point cloud is less than the number of target points in the first target point cloud. Therefore, when determining the trajectory points and posture of the robotic arm based on the second target point cloud, the speed of obtaining the trajectory points and posture can be greatly improved.

[0075] In some embodiments, step B2 above includes:

[0076] B21. Based on the transformation relationship between the camera coordinate system and the robotic arm coordinate system and the aforementioned second target point cloud, determine the matrix of the grasping points. This matrix of grasping points includes the coordinates of the target points in the second target point cloud and information used to determine the posture of the robotic arm at the trajectory points.

[0077] The coordinates of the robotic arm's trajectory points can be determined based on the coordinates of the target points in the second target point cloud. For example, the coordinates of each target point can be used as the coordinates of each trajectory point of the robotic arm.

[0078] Since the second target point cloud is captured by the camera, the coordinate system of the second target point cloud is the camera coordinate system, while the coordinates of the trajectory points of the robotic arm belong to the robotic arm coordinate system. Therefore, before determining the trajectory points of the robotic arm, it is necessary to transform the second target point cloud from the camera coordinate system to the robotic arm coordinate system.

[0079] In this embodiment, the transformation relationship between the camera coordinate system and the robotic arm coordinate system can be represented by a matrix.

[0080] B22. Determine the coordinates of the trajectory points of the robotic arm and the posture of the robotic arm at the trajectory points based on the matrix of the above gripping points.

[0081] Specifically, since the second target point cloud is determined based on the first motion trajectory, the trajectory points of the robotic arm include the target points in the second target point cloud, and correspondingly, the coordinates of the trajectory points of the robotic arm include the coordinates of the target points in the second target point cloud.

[0082] In this embodiment, since the coordinate transformation of the second target point cloud is performed according to the transformation relationship between the camera coordinate system and the robotic arm coordinate system, it can be ensured that the coordinates of the obtained trajectory points are more compatible with the robotic arm, thereby improving the accuracy of the subsequently obtained second motion trajectory.

[0083] In some embodiments, step B21 above includes:

[0084] For any two adjacent target points in the second target point cloud, the following steps are performed, wherein the sorting relationship of each target point in the second target point cloud is determined based on the sorting relationship of the pixel coordinates as described above:

[0085] B211. Determine the first straight line (let's call it Lb) on the point cloud map where the target point that is ranked first is located.

[0086] For example, suppose the second target point cloud includes three target points W1, W2, and W3. W1 and W2 are two adjacent target points, and W2 and W3 are also two adjacent target points. If we consider target points W1 and W2, W1 is the target point ranked first, and W2 is the target point ranked second. However, if we consider target points W2 and W3, W2 is the target point ranked first, and W3 is the target point ranked second.

[0087] In this embodiment, Lb is perpendicular to the point cloud map where the first target point is located, and it is the straight line where the normal vector of the first target point is located on the point cloud map.

[0088] B212. Determine the second straight line (let's call it La) that passes through the target point after the sorting and is perpendicular to the above Lb.

[0089] Among them, La passes through the target point that is ranked later among two adjacent target points, and La is perpendicular to Lb.

[0090] B213. Based on Lb and La mentioned above, determine the midpoint.

[0091] In this context, the intermediate point is usually a point between two adjacent target points. Assuming W1 is the target point ranked first and W2 is the target point ranked second, and the intermediate point is denoted by P, then P is usually a point between W1 and W2.

[0092] In some embodiments, the midpoint may be the intersection of Lb and La.

[0093] B214. Determine the direction vector from the above intermediate point to the target point in the above order, wherein the direction vector is perpendicular to the above Lb.

[0094] In the embodiments of this application, when determining the direction vector, the movement is from the midpoint to the target point that is ranked later, while when the robotic arm is running, the movement is from the target point that is ranked earlier to the target point that is ranked later.

[0095] B215. Determine the trajectory normal vector based on the above normal vector and the above direction vector. The trajectory normal vector is perpendicular to the plane formed by the above Lb and the above La.

[0096] Among them, the normal vector, the aforementioned direction vector, and the trajectory normal vector follow the right-hand rule of the cross product, and the direction of the trajectory normal vector is the direction pointed to by the right thumb.

[0097] B216. Determine the transformation matrix based on the above direction vector, the above normal vector, the above trajectory normal vector, and the coordinates of the target points listed first.

[0098] In some embodiments, it is assumed that the direction vector is represented by dir_v, the normal vector by nor_v, and the trajectory normal vector by res_v. The components of each vector on the X-axis, Y-axis, and Z-axis are marked with corresponding numbers. For example, when the number is "0", it represents the component of the corresponding vector on the X-axis (e.g., dir_v[0] represents the component of the direction vector on the X-axis), when the number is "1", it represents the component of the corresponding vector on the Y-axis (e.g., dir_v[1] represents the component of the direction vector on the Y-axis), and when the number is "2", it represents the component of the corresponding vector on the Z-axis (e.g., dir_v[2] represents the component of the direction vector on the Y-axis). x0, y0, and z0 represent the coordinates of the target point W1 that is ranked first. Then the transformation matrix can be represented in the following form:

[0099]

[0100] B216. Based on the transformation relationship between the camera coordinate system and the robotic arm coordinate system and the above transformation matrix, determine the matrix of the grasping points corresponding to the two target points.

[0101] The matrix of the grasping points can be obtained by cross-product of the transformation matrix and the transformation relationship between the camera coordinate system and the robotic arm coordinate system. For example, assuming the transformation relationship between the camera coordinate system and the robotic arm coordinate system is represented by a matrix (which is usually set as a 4*4 homogeneous matrix, mainly used to describe translation and perspective projection transformations), the transformation matrix is ​​represented by a matrixpick (which is usually set as a 4*4 homogeneous matrix), and the matrix of the grasping points is represented by grabCordinate, then grabCordinate = matrix * matrixpick.

[0102] In this embodiment, the transformation matrix is ​​determined based on the direction vector, normal vector, trajectory normal vector, and the coordinates of the first-order target point. Since the direction vector, normal vector, and trajectory normal vector are mutually perpendicular, and the pose of the trajectory point needs to be determined using three mutually perpendicular vectors, the transformation matrix includes information for determining the pose of the first-order target point as well as information for determining the pose of the robotic arm at the coordinates of the first-order target point. Therefore, after determining the grab point matrix `grabCordinate` based on the transformation relationship between the camera coordinate system and the robotic arm coordinate system and the transformation matrix, it can be guaranteed that `grabCordinate` includes information for determining the pose of the robotic arm at the first-order target point as well as information for determining the pose of the robotic arm at the coordinates of the first-order target point. The information of the coordinates of the trajectory points (i.e., the coordinates of the target points in the second target point cloud) is obtained. In this way, Euler angles can be calculated based on the direction vector, normal vector and trajectory normal vector in grabCordinate, so as to obtain the posture of the trajectory points corresponding to the first target points in the order. Also, the coordinates of the corresponding trajectory points can be determined based on the coordinates of the first target points in grabCordinate, such as determining the coordinates of the corresponding trajectory points based on the last item of each row in grabCordinate (grabCordinate(0,3), grabCordinate(1,3), and grabCordinate(2,3) in grabCordinate are x0, y0 and z0 of the trajectory points, respectively).

[0103] In some embodiments, the transformation relationship between the camera coordinate system and the robotic arm coordinate system can be determined in the following manner. In this case, the motion trajectory generation method provided in this application embodiment further includes:

[0104] C1. Obtain the two-dimensional image and three-dimensional point cloud map corresponding to the space where the motion trajectory needs to be determined. The space where the motion trajectory needs to be determined has at least 4 specified graphics placed in advance.

[0105] The specified graphics include QR code graphics, such as the Apriltag QR code image.

[0106] In this embodiment of the application, at least four Apriltag QR code images can be pre-placed in the space where the motion trajectory needs to be determined, and then the space where the motion trajectory needs to be determined can be photographed by a 3D camera to obtain the corresponding two-dimensional image and three-dimensional point cloud map.

[0107] In some embodiments, nine Apriltag QR code images may be pre-placed in the space where the motion trajectory needs to be determined in order to improve the accuracy of subsequent calibration.

[0108] C2. Identify at least four specified graphics in the two-dimensional image corresponding to the space where the motion trajectory needs to be determined, and obtain the two-dimensional coordinates of each specified graphic.

[0109] Specifically, the coordinates of a point in a specified graphic (such as the center point or the top left corner of the specified graphic) can be used as the two-dimensional coordinates of the specified graphic.

[0110] In this embodiment of the application, assuming that there are 9 pre-placed specified graphics, the positions of the 9 specified graphics in the two-dimensional image are identified respectively, and the two-dimensional coordinates of the 9 specified graphics are obtained.

[0111] C3. Align the two-dimensional image and three-dimensional point cloud map corresponding to the space where the motion trajectory needs to be determined, and determine the three-dimensional point cloud coordinates corresponding to the two-dimensional coordinates of each of the above-mentioned specified graphics.

[0112] C4. Obtain the coordinates of the robotic arm when its end point points to at least four specified shapes.

[0113] Specifically, the position pointed to by the end of the robotic arm is the same as the position of the selected two-dimensional coordinate point that can be used as the specified graphic. For example, if the coordinates of the center of the specified graphic are selected as the two-dimensional coordinates of the specified graphic, then the coordinates of the robotic arm when the end of the robotic arm points to the center of the specified graphic are obtained.

[0114] In this embodiment, the end effector of the robotic arm can be moved to point to designated shapes in a preset order, thereby obtaining the coordinates of the robotic arm as many as the number of designated shapes. The preset order refers to the same order in which the two-dimensional coordinates of each designated shape are determined. For example, assuming four designated shapes are placed at the top left, bottom left, top right, and bottom right, and the two-dimensional coordinates of the designated shapes are recorded in the order of top left, bottom left, top right, and bottom right, then the end effector of the robotic arm will also move in the order of top left, bottom left, top right, and bottom right.

[0115] It should be noted that this step can be performed before step C1. Correspondingly, the order in which the two-dimensional coordinates of the specified graphic are recorded must be the same as the order of the end effector of the moving robotic arm, which will not be elaborated here.

[0116] C5. Based on the coordinates of each of the above-mentioned three-dimensional point clouds and the coordinates of each of the above-mentioned robotic arms, determine the transformation relationship between the camera coordinate system and the robotic arm coordinate system.

[0117] Specifically, the coordinates of each 3D point cloud are aligned with the coordinates of the corresponding robotic arm to perform coordinate system transformation calculations, thereby obtaining the transformation relationship between the camera coordinate system and the robotic arm coordinate system and completing the calibration.

[0118] In this embodiment of the application, since a specified graphic is placed in the space where the motion trajectory needs to be determined, and the specified graphic is easily identifiable, the robotic arm can be calibrated quickly and accurately based on the identification result.

[0119] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0120] Example 2:

[0121] Corresponding to the motion trajectory generation method in Embodiment 1 above, Figure 4 The diagram shows a structural block diagram of a motion trajectory generation device provided in an embodiment of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0122] Reference Figure 4 The motion trajectory generation device 4, applied to the controller of the robotic arm, includes: a three-dimensional point cloud acquisition module 41, a two-dimensional trajectory acquisition module 42, a first target point cloud determination module 43, an attitude determination module 44, and a second motion trajectory determination module 45. Wherein:

[0123] The 3D point cloud acquisition module 41 is used to receive the 3D point cloud image of the target body sent by the image acquisition device.

[0124] The target body is the object on which the robotic arm needs to acquire the motion trajectory. The target body includes one or more of the following: human body, designated area, and designated type of object.

[0125] In this embodiment, a 3D camera can be used to photograph the target object to obtain a two-dimensional image and a three-dimensional point cloud map corresponding to the target object. The three-dimensional point cloud map of the target object is then sent to the controller, or both the two-dimensional image and the three-dimensional point cloud map of the target object are sent to the controller. The three-dimensional point cloud map of the target object includes the point cloud of the target object.

[0126] The two-dimensional trajectory acquisition module 42 is used to receive a two-dimensional trajectory, which is obtained by determining a first motion trajectory in the two-dimensional image of the target body, wherein the first motion trajectory is a two-dimensional motion trajectory.

[0127] In this embodiment, the 3D camera can send a two-dimensional image to a host computer (such as a computer or mobile phone). After displaying the two-dimensional image through image editing software (such as a drawing board) installed on the host computer, the user can directly draw the first motion trajectory of the two-dimensional image in the image editing software. After the drawing is completed, the host computer sends the two-dimensional image with the first motion trajectory drawn (i.e., the two-dimensional trajectory diagram) to the controller. The first motion trajectory is the two-dimensional motion trajectory that the user wants the robotic arm to move on the target object.

[0128] The first target point cloud determination module 43 is used to determine the projection of the first motion trajectory in the two-dimensional trajectory map onto the three-dimensional point cloud map based on the mapping relationship between the two-dimensional image and the three-dimensional point cloud map, so as to obtain the first target point cloud.

[0129] The attitude determination module 44 is used to determine the trajectory points of the robotic arm and the attitude corresponding to each trajectory point based on the first target point cloud.

[0130] The second motion trajectory determination module 45 is used to generate the second motion trajectory of the robotic arm based on the trajectory points of the robotic arm and the postures corresponding to each trajectory point. The second motion trajectory is a three-dimensional motion trajectory.

[0131] In this embodiment, after acquiring the three-dimensional point cloud map and the two-dimensional trajectory map of the target object, the point cloud corresponding to the first motion trajectory in the two-dimensional trajectory map is determined according to the mapping relationship between the two-dimensional image and the three-dimensional point cloud map of the target object, thus obtaining the first target point cloud. Then, the trajectory points and posture of the robotic arm are determined according to the first target point cloud to determine the second motion trajectory of the robotic arm. Since the first motion trajectory is a two-dimensional motion trajectory determined in the two-dimensional image of the target object, the first motion trajectory can be determined quickly and accurately according to the two-dimensional trajectory map. At the same time, since there is a specific mapping relationship between the two-dimensional image and the three-dimensional point cloud map of the target object, the second motion trajectory corresponding to the first motion trajectory can be determined according to the mapping relationship. Furthermore, since the generation process of the second motion trajectory does not require manual dragging of the robotic arm, the generation accuracy and precision of the second motion trajectory are further improved.

[0132] In some embodiments, the first target point cloud determination module 43 includes:

[0133] The target skeleton determination unit is used to perform skeleton extraction processing on the first motion trajectory in the two-dimensional trajectory diagram to obtain the target skeleton.

[0134] In this embodiment, since only the first motion trajectory is extracted as the target skeleton, only the pixel coordinates of the target skeleton need to be processed subsequently, without processing the pixel coordinates of the entire two-dimensional trajectory map, thereby effectively reducing the number of pixel coordinates that need to be processed.

[0135] The sorted pixel coordinate determination unit is used to sort the pixel coordinates of the target skeleton to obtain the sorted pixel coordinates.

[0136] The first target point cloud determination unit is used to project the sorted pixel coordinates onto the point cloud of the three-dimensional point cloud based on the mapping relationship between the two-dimensional image and the three-dimensional point cloud, thereby obtaining the first target point cloud. The target points in the first target point cloud also have an order relationship with the sorted pixel coordinates.

[0137] In this embodiment, since the first motion trajectory is extracted from the two-dimensional trajectory map and then projected onto the three-dimensional point cloud map, the number of pixel coordinates that need to be projected is reduced, thereby improving the speed of obtaining the first target point cloud.

[0138] In some embodiments, the target skeleton determination unit includes:

[0139] The binarization processing unit is used to convert the above two-dimensional trajectory map into a grayscale image and perform binarization processing.

[0140] The skeleton extraction processing unit is used to perform skeleton extraction processing on the first motion trajectory in the binarized two-dimensional trajectory map to obtain the target skeleton.

[0141] In this embodiment of the application, when the two-dimensional trajectory map is an RGB color image, the two-dimensional trajectory map is converted into a grayscale image and then binarized to increase the difference between the first motion trajectory and other objects in the two-dimensional trajectory map, thereby facilitating the accurate extraction of the target skeleton from the binarized two-dimensional trajectory map.

[0142] In some embodiments, the attitude determination module 44 includes:

[0143] The downsampling processing unit is used to downsample the first target point cloud to obtain the second target point cloud.

[0144] Specifically, after downsampling the first target point cloud, the number of target points in the resulting second target point cloud is less than the number of target points in the first target point cloud. In some embodiments, to facilitate extraction, corresponding target points are extracted from the first target point cloud at equal intervals, so that the distance between adjacent target points in the second target point cloud is equal.

[0145] The trajectory point determination unit is used to determine the trajectory points and attitude of the robotic arm based on the second target point cloud.

[0146] In this embodiment of the application, since the second target point cloud is obtained by downsampling the first target point cloud, the number of target points in the obtained second target point cloud is less than the number of target points in the first target point cloud. Therefore, when determining the trajectory points and posture of the robotic arm based on the second target point cloud, the speed of obtaining the trajectory points and posture can be greatly improved.

[0147] In some embodiments, the trajectory point determination unit includes:

[0148] The grasping point matrix determination unit is used to determine the grasping point matrix based on the transformation relationship between the camera coordinate system and the robotic arm coordinate system and the aforementioned second target point cloud. The grasping point matrix includes the coordinates of the target points in the second target point cloud and information for determining the posture of the robotic arm at the trajectory points.

[0149] The trajectory point coordinate determination unit is used to determine the coordinates of the trajectory points of the robotic arm and the posture of the robotic arm at the trajectory points based on the matrix of the grasping points.

[0150] Specifically, since the second target point cloud is determined based on the first motion trajectory, the trajectory points of the robotic arm include the target points in the second target point cloud, and correspondingly, the coordinates of the trajectory points of the robotic arm include the coordinates of the target points in the second target point cloud.

[0151] In this embodiment, since the coordinate transformation of the second target point cloud is performed according to the transformation relationship between the camera coordinate system and the robotic arm coordinate system, it can be ensured that the coordinates of the obtained trajectory points are more compatible with the robotic arm, thereby improving the accuracy of the subsequently obtained second motion trajectory.

[0152] In some embodiments, the above-mentioned matrix determination unit for grasping points is specifically used for:

[0153] For any two adjacent target points in the second target point cloud, the following steps are performed, wherein the sorting relationship of each target point in the second target point cloud is determined based on the sorting relationship of the pixel coordinates as described above:

[0154] Determine the line Lb containing the normal vector of the first-ranked target point on the point cloud map;

[0155] Determine the line La that passes through the target point after the sorting and is perpendicular to the above Lb;

[0156] Based on Lb and La mentioned above, determine the midpoint;

[0157] Determine the direction vector from the above intermediate point to the target point in the above order, wherein the direction vector is perpendicular to the above Lb;

[0158] The trajectory normal vector is determined based on the above normal vector and the above direction vector, and the above trajectory normal vector is perpendicular to the plane formed by the above Lb and the above La;

[0159] The transformation matrix is ​​determined based on the aforementioned direction vector, normal vector, upper trajectory normal vector, and the coordinates of the target points listed first.

[0160] Based on the transformation relationship between the camera coordinate system and the robotic arm coordinate system, and the aforementioned transformation matrix, the matrix of the grasping points corresponding to the two target points is determined.

[0161] In this embodiment, the transformation matrix is ​​determined based on the direction vector, normal vector, trajectory normal vector, and the coordinates of the first-order target points. Since the direction vector, normal vector, and trajectory normal vector are mutually perpendicular, and the orientation of the trajectory point requires three mutually perpendicular vectors to be determined, the transformation matrix includes both information for determining the orientation of the first-order target points and information for determining their coordinates. Therefore, after determining the grab point matrix `grabCordinate` based on the transformation relationship between the camera coordinate system and the robotic arm coordinate system and the transformation matrix, it can be guaranteed that `grabCordinate` includes both information for determining the orientation of the first-order target points and information for determining their coordinates. The coordinates of the first target point in the sequence are defined. Then, Euler angles can be calculated based on the direction vector, normal vector, and trajectory normal vector in `grabCordinate` to obtain the pose of the trajectory point corresponding to the first target point. The coordinates of the corresponding trajectory point can be determined based on the coordinates of the first target point in `grabCordinate`, such as by determining the coordinates of the corresponding trajectory point based on the last item of each row in `grabCordinate` (where `grabCordinate(0,3),` `grabCordinate(1,3),` and `grabCordinate(2,3)` are the x0, y0, and z0 of the trajectory point, respectively).

[0162] In some embodiments, the transformation relationship between the camera coordinate system and the robotic arm coordinate system can be determined in the following manner. In this case, the motion trajectory generation device 4 provided in this application embodiment further includes:

[0163] The two-dimensional image acquisition module is used to acquire the two-dimensional image and three-dimensional point cloud map corresponding to the space where the motion trajectory needs to be determined. At least four specified graphics are pre-placed in the space where the motion trajectory needs to be determined.

[0164] The two-dimensional coordinate recognition module for the specified graphic is used to recognize at least four specified graphics in the two-dimensional image corresponding to the space where the motion trajectory needs to be determined, and to obtain the two-dimensional coordinates of each specified graphic.

[0165] The 3D point cloud coordinate determination module is used to align the 2D image and 3D point cloud map corresponding to the space where the motion trajectory needs to be determined, and determine the 3D point cloud coordinates corresponding to the 2D coordinates of each of the specified graphics.

[0166] The coordinate acquisition module of the robotic arm is used to acquire the coordinates of the robotic arm when the end of the robotic arm points to the at least four specified graphics.

[0167] The transformation relationship determination module is used to determine the transformation relationship between the camera coordinate system and the robot arm coordinate system based on the coordinates of each of the above-mentioned 3D point cloud and the coordinates of each of the above-mentioned robotic arms.

[0168] In this embodiment of the application, since a specified graphic is placed in the space where the motion trajectory needs to be determined, and the specified graphic is easily identifiable, the robotic arm can be calibrated quickly and accurately based on the identification result.

[0169] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0170] Example 3:

[0171] Figure 5 This is a schematic diagram of the structure of a processing device provided in one embodiment of this application. Figure 5 As shown, the processing device 5 of this embodiment includes: at least one processor 50 ( Figure 5 The diagram shows only one processor, memory 51, and computer program 52 stored in the memory 51 and executable on at least one processor 50. When the processor 50 executes the computer program 52, it implements the steps in any of the above method embodiments.

[0172] The aforementioned processing device 5 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. This processing device may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will understand that... Figure 5 This is merely an example of processing device 5 and does not constitute a limitation on processing device 5. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0173] The processor 50 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0174] In some embodiments, the aforementioned memory 51 may be an internal storage unit of the processing device 5, such as a hard disk or memory of the processing device 5. In other embodiments, the aforementioned memory 51 may be an external storage device of the processing device 5, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the processing device 5. Furthermore, the aforementioned memory 51 may include both internal storage units and external storage devices of the processing device 5. The aforementioned memory 51 is used to store operating systems, applications, bootloaders, data, and other programs, such as the program code of the aforementioned computer programs. The aforementioned memory 51 may also be used to temporarily store data that has been output or will be output.

[0175] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0176] This application also provides a network device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor. When the processor executes the computer program, it implements the steps in any of the above method embodiments.

[0177] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the various method embodiments described above.

[0178] This application provides a computer program product that, when run on a processing device, enables the processing device to implement the steps described in the various method embodiments above.

[0179] If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographic device / processing device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0180] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0181] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0182] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0183] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0184] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for generating a motion trajectory, characterized in that, Controllers used in robotic arms include: Receive a 3D point cloud image of the target object sent by the image acquisition device; Receive a two-dimensional trajectory map, which is obtained by determining a first motion trajectory in a two-dimensional image of the target body, wherein the first motion trajectory is the two-dimensional motion trajectory corresponding to when the user wants the robotic arm to move on the target body; Based on the mapping relationship between the two-dimensional image and the three-dimensional point cloud map, the projection of the first motion trajectory in the two-dimensional trajectory map onto the three-dimensional point cloud map is determined to obtain the first target point cloud. The trajectory points of the robotic arm and the posture corresponding to each trajectory point are determined based on the first target point cloud. Based on the trajectory points of the robotic arm and the postures corresponding to each trajectory point, a second motion trajectory of the robotic arm is generated. The second motion trajectory is a three-dimensional motion trajectory.

2. The method for generating a motion trajectory as described in claim 1, characterized in that, Determining the projection of the first motion trajectory in the two-dimensional trajectory map onto the three-dimensional point cloud map based on the mapping relationship between the two-dimensional image and the three-dimensional point cloud map includes: The first motion trajectory in the two-dimensional trajectory graph is subjected to skeleton extraction processing to obtain the target skeleton; The pixel coordinates of the target skeleton are sorted to obtain the sorted pixel coordinates; Based on the mapping relationship between the two-dimensional image and the three-dimensional point cloud map, the sorted pixel coordinates are projected onto the point cloud of the three-dimensional point cloud map to obtain the first target point cloud.

3. The method for generating a motion trajectory as described in claim 2, characterized in that, The process of extracting the skeleton from the first motion trajectory in the two-dimensional trajectory graph to obtain the target skeleton includes: The two-dimensional trajectory image is converted into a grayscale image and then binarized. The first motion trajectory in the binarized two-dimensional trajectory graph is subjected to skeleton extraction processing to obtain the target skeleton.

4. The method for generating a motion trajectory as described in claim 2 or 3, characterized in that, The step of determining the trajectory points of the robotic arm and the corresponding postures of each trajectory point based on the first target point cloud includes: The first target point cloud is downsampled to obtain the second target point cloud; The trajectory points of the robotic arm and the postures corresponding to each trajectory point are determined based on the second target point cloud.

5. The method for generating a motion trajectory as described in claim 4, characterized in that, The step of determining the trajectory points of the robotic arm and the corresponding postures of each trajectory point based on the second target point cloud includes: Based on the transformation relationship between the camera coordinate system and the robotic arm coordinate system and the second target point cloud, a matrix of gripping points is determined. The matrix of gripping points includes the coordinates of the target points in the second target point cloud and information for determining the posture of the robotic arm at the trajectory points. The coordinates of the trajectory points of the robotic arm and the posture of the robotic arm at the trajectory points are determined based on the matrix of the gripping points.

6. The method for generating a motion trajectory as described in claim 5, characterized in that, The step of determining the matrix of grasping points based on the transformation relationship between the camera coordinate system and the robotic arm coordinate system and the second target point cloud includes: For any two adjacent target points in the second target point cloud, the following steps are performed, wherein the sorting relationship of each target point in the second target point cloud is determined based on the sorting relationship of the sorted pixel coordinates: Determine the first straight line on the point cloud map containing the normal vector of the target point that is ranked first; Determine a second line that passes through the target points in the order and is perpendicular to the first line; Determine the midpoint based on the first line and the second line; Determine the direction vector from the intermediate point to the next target point in the order, the direction vector being perpendicular to the first straight line; The trajectory normal vector is determined based on the normal vector and the direction vector, and the trajectory normal vector is perpendicular to the plane formed by the first line and the second line. The transformation matrix is ​​determined based on the direction vector, the normal vector, the trajectory normal vector, and the coordinates of the first-ordered target point. Based on the transformation relationship between the camera coordinate system and the robotic arm coordinate system, and the transformation matrix, the matrix of the grasping points corresponding to the two target points is determined.

7. The method for generating a motion trajectory as described in claim 6, characterized in that, Also includes: Acquire a two-dimensional image and a three-dimensional point cloud map corresponding to the space where the motion trajectory needs to be determined, wherein at least four specified graphics are pre-placed in the space where the motion trajectory needs to be determined. Identify at least four specified shapes in the two-dimensional image corresponding to the space where the motion trajectory needs to be determined, and obtain the two-dimensional coordinates of each specified shape; Align the two-dimensional image and three-dimensional point cloud map corresponding to the space where the motion trajectory needs to be determined, and determine the three-dimensional point cloud coordinates corresponding to the two-dimensional coordinates of each specified graphic. Obtain the coordinates of the robotic arm when its end point points to the at least four specified shapes; Based on the coordinates of each of the three-dimensional point clouds and the coordinates of each of the robotic arms, the transformation relationship between the camera coordinate system and the robotic arm coordinate system is determined.

8. A device for generating motion trajectories, characterized in that, Controllers used in robotic arms include: The 3D point cloud acquisition module is used to receive the 3D point cloud image of the target object sent by the image acquisition device; A two-dimensional trajectory acquisition module is used to receive a two-dimensional trajectory, which is obtained by determining a first motion trajectory in a two-dimensional image of the target body, wherein the first motion trajectory is the two-dimensional motion trajectory corresponding to when the user wants the robotic arm to move on the target body; The first target point cloud determination module is used to determine the projection of the first motion trajectory in the two-dimensional trajectory map onto the three-dimensional point cloud map based on the mapping relationship between the two-dimensional image and the three-dimensional point cloud map, so as to obtain the first target point cloud. The attitude determination module is used to determine the trajectory points of the robotic arm and the attitude corresponding to each trajectory point based on the first target point cloud. The second motion trajectory determination module is used to generate a second motion trajectory of the robotic arm based on the trajectory points of the robotic arm and the postures corresponding to each trajectory point. The second motion trajectory is a three-dimensional motion trajectory.

9. A processing apparatus comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images