Path planning method, electronic device, and storage medium

Abstract

The application discloses a path planning method, an electronic device and a storage medium. The path planning method is applied to a robot, and the path planning method comprises the following steps: acquiring first point cloud data of a traction device in a to-be-trucked shelf and a first pose of the robot for collecting the first point cloud data, the to-be-trucked shelf further comprises a shelf, the shelf is movably connected with the traction device, and the traction device comprises a traction ring; determining a first target pose of the traction ring relative to the robot according to the first point cloud data; and determining a first target path according to the first pose and the first target pose, the first target path being used for the robot to truck the shelf through the traction ring in the traction device. According to the above scheme, the accuracy of the planned first target path can be improved.

Description

Technical Field

[0001] This application relates to the field of robotics, and in particular to a path planning method, electronic device, and storage medium. Background Technology

[0002] Currently, autonomous driving and robotics are widely used in various scenarios, such as unmanned food delivery vehicles in restaurants / hotels and unmanned patrol vehicles. Current technical solutions for determining shelf location and orientation rely on feature points reflecting laser light from the traction device to determine if the robot is on the traction device. However, the traction device often has many interfering discrete points, which can be mistakenly identified as not being part of the traction device, leading to the misconception that the traction device has not been detected. Due to these interference points, the estimated center of the traction device deviates from the actual center, resulting in low accuracy in the robot's planned path to the shelf. This makes it impossible to guarantee that the robot can accurately move to the shelf and precisely dock with the traction ring, ultimately preventing the robot from successfully completing the shelf-traction task.

[0003] Therefore, a path planning method is urgently needed. Summary of the Invention

[0004] This application provides at least one path planning method, electronic device, and storage medium that can improve the accuracy of the planned first target path.

[0005] This application provides a path planning method applied to a robot. The path planning method includes: acquiring first point cloud data of a traction device in a shelf to be traction and a first pose of the robot acquiring the first point cloud data. The traction shelf also includes a shelf, which is movably connected to the traction device. The traction device includes a traction ring. The method includes: determining a first target pose of the traction ring relative to the robot based on the first point cloud data; and determining a first target path based on the first pose and the first target pose. The first target path is used by the robot to traction the shelf through the traction ring in the traction device.

[0006] This application provides a path planning device, including: an acquisition module, a first determination module, and a second determination module; the acquisition module is used to acquire first point cloud data of a traction device in a shelf to be tractioned and a first pose of the robot collecting the first point cloud data, the traction shelf also includes a shelf, the shelf is movably connected to the traction device, and the traction device includes a traction ring; the first determination module is used to determine a first target pose of the traction ring relative to the robot based on the first point cloud data; the second determination module is used to determine a first target path based on the first pose and the first target pose, the first target path being used by the robot to traction the shelf through the traction ring in the traction device.

[0007] This application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-described path planning method.

[0008] This application provides a computer-readable storage medium storing program instructions thereon, which, when executed by a processor, implement the above-described path planning method.

[0009] The above scheme acquires the first point cloud data of the traction device in the shelf to be traction and the first pose of the robot collecting the first point cloud data. The traction shelf also includes a shelf, which is movably connected to the traction device. The traction device includes a traction ring. The first target pose of the traction ring relative to the robot is determined based on the first point cloud data. The first target path is determined based on the first pose and the first target pose. The first target path is used by the robot to traction the shelf through the traction ring in the traction device. In this way, the accuracy of the first target pose can be improved by obtaining the first target pose of the traction ring through the first point cloud data of the traction device, thereby improving the accuracy of the first target path determined based on the first target pose and the first pose, and thus improving the efficiency of the robot traction of the shelf based on the first target path.

[0010] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0012] Figure 1 This is a flowchart illustrating an exemplary embodiment of the path planning method of this application; Figure 2 yes Figure 1 A schematic diagram of the sub-process of step S12; Figure 3a yes Figure 2 A schematic diagram of the sub-process of step S21; Figure 3b This is a schematic diagram of the rack to be pulled and the robot in an exemplary embodiment of the path planning method of this application; Figure 3c This is a schematic diagram of the preset attribute information of the traction device in an exemplary embodiment of the path planning method of this application; Figure 3d This is a schematic diagram of the index partition of the first point cloud data in an exemplary embodiment of the path planning method of this application; Figure 4a yes Figure 2 A schematic diagram of the sub-process of step S22; Figure 4b This is a schematic diagram of the left and right fitting lines in an exemplary embodiment of the path planning method of this application; Figure 5a yes Figure 2 A schematic diagram of another sub-process in step S22; Figure 5b This is a schematic diagram of invalid points in an exemplary embodiment of the path planning method of this application; Figure 5c This is a schematic diagram of the clustering results of invalid points in an exemplary embodiment of the path planning method of this application; Figure 5d This is a schematic diagram of the inner loop boundary points in an exemplary embodiment of the path planning method of this application; Figure 5e This is a schematic diagram of a preset line segment between the left and right fitting lines in an exemplary embodiment of the path planning method of this application; Figure 5f This is a schematic diagram of a first type of traction device in an exemplary embodiment of the path planning method of this application; Figure 5g This is a schematic diagram of a second type of traction device in an exemplary embodiment of the path planning method of this application; Figure 5h This is a schematic diagram of a third type of traction device in an exemplary embodiment of the path planning method of this application; Figure 5i This is a schematic diagram of a fourth type of traction device in an exemplary embodiment of the path planning method of this application; Figure 6a This is yet another flowchart illustrating an exemplary embodiment of the path planning method of this application; Figure 6b This is a schematic diagram of the identification area in an exemplary embodiment of the path planning method of this application. Figure 7 This is another flowchart illustrating an exemplary embodiment of the path planning method of this application; Figure 8 This is a schematic diagram of the structure of an embodiment of the path planning device of this application; Figure 9 This is a schematic diagram of the structure of an embodiment of the electronic device of this application; Figure 10 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0013] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0014] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0015] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0016] This application provides several path planning methods and devices. Applications of these path planning methods include, but are not limited to, docking robots with shelves to be towed. The execution entity of the path planning method can be a path planning device, such as a robot, a built-in module of a robot, or a server connected to the robot for communication. The path planning device can be located within a terminal device, server, or other processing device. The terminal device can be user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, etc. In some possible implementations, the path planning method can be implemented by a processor calling computer-readable instructions stored in memory.

[0017] Please see Figure 1 , Figure 1 This is a flowchart illustrating an exemplary embodiment of the path planning method of this application. Specifically, the path planning method is applied to a robot and may include the following steps: Step S11: Obtain the first point cloud data of the traction device in the shelf to be tractioned and the first pose of the robot collecting the first point cloud data.

[0018] For example, the robot can be a mobile robot. For instance, the robot can be an indoor robot. When performing a task (such as moving objects in a warehouse), the mobile robot needs to accurately locate the position of the object corresponding to the task in the current environment to accurately execute the task. The task the robot needs to perform can be a traction task to move objects on a shelf. The traction shelf also includes the shelf itself. The shelf is movably connected to the traction device. The traction device includes a traction ring. The shelf to be traction is the shelf that the robot needs to move when performing the task. The shelf to be traction includes both the shelf and the traction device. The shelf is movably connected to the traction device, and the traction device is detachably connected to the shelf. After the traction device is movably connected to the shelf, it can rotate horizontally and / or vertically. The robot includes a pin. The traction device includes at least a traction ring. When the robot performs the task, it inserts the pin into the traction ring to achieve docking between the robot and the shelf to be traction. After docking, the robot pulls the shelf via the traction ring in the traction device. The type of traction ring includes, but is not limited to, circular rings, rectangular rings, U-shaped rings, etc. The type of traction device varies depending on the type of traction rod. The type of tow bar can be A-type tow bar, rectangular tow bar, straight tow bar, or U-type tow bar, etc.

[0019] The first point cloud data refers to the point cloud data collected by the traction device in the rack to be tractioned. The acquisition time of the first point cloud data can be the first acquisition moment. The acquisition device for the first point cloud data can be a robot or a 3D camera connected to the robot. For example, the acquisition device for the first point cloud data can be a 3D camera on the robot, and the 3D camera includes, but is not limited to, a ToF sensor or a binocular camera. The first pose is the robot's pose in the current environment when the first point cloud data is acquired.

[0020] In some application scenarios, step S11 above may involve acquiring the first point cloud data collected by the robot from the traction device or receiving the first point cloud data collected by the traction device from a 3D camera connected to the robot. In other application scenarios, step S11 above may involve the robot receiving the first pose of the robot when collecting the first point cloud data from the positioning module, or retrieving the first pose of the robot when collecting the first point cloud data from a preset database.

[0021] Step S12: Determine the first target pose of the traction ring relative to the robot based on the first point cloud data.

[0022] The first target pose represents the pose of the traction ring in the robot's coordinate system when acquiring the first point cloud data. The first target pose includes the orientation angle of the traction ring and / or the coordinate information of the reference point corresponding to the traction ring when acquiring the first point cloud data. For example, the reference point can be the center point or the center of the traction ring.

[0023] Specifically, step S12 can involve directly fitting the first point cloud data to obtain the fitting result of the traction device, and then determining the first target pose based on the fitting result of the traction device. Specifically, the fitting result of the traction device includes the fitting loop of the traction ring and at least two fitted line segments about the traction ring. In some application scenarios, the coordinate information of the reference point in the fitting loop of the traction ring is directly used as the coordinate information of the reference point corresponding to the traction ring; the orientation angle matching the coordinate information of the reference point in the fitting loop of the traction ring is used as the orientation angle of the traction ring. In other application scenarios, the orientation angle of the traction ring is determined based on at least two fitted line segments; the coordinate information of the reference point matching the orientation angle of the traction ring is used as the coordinate information of the reference point corresponding to the traction ring. In still other application scenarios, the coordinate information of the reference point in the fitting loop of the traction ring is directly used as the coordinate information of the reference point corresponding to the traction ring; the orientation angle of the traction ring is determined based on at least two fitted line segments.

[0024] For example, in the case where the traction device includes only a traction ring, the traction ring is movably connected to the shelf. The edge connecting the shelf and the traction ring is taken as the target edge, and the direction of the target edge is taken as the target direction. At least two fitted line segments with respect to the traction ring can be the lines connecting the two intersection points of a preset straight line and the traction ring to preset points on the target edge in the traction device, where the preset straight line represents a straight line in the target direction that passes through the center of the traction ring.

[0025] For example, in a traction device that may include a traction ring and a traction rod, the traction ring is movably connected to the rack via the traction rod. At least two fitted line segments for the traction ring may be fitted line segments whose two endpoints are respectively located on the rack and the traction ring.

[0026] Step S13: Determine the first target path based on the first pose and the first target pose. The first target path is used by the robot to pull the shelf through the traction ring in the traction device.

[0027] The first target path represents the movement path of the robot from its initial pose position to its first target pose position in the same coordinate system. Once the robot has completed its movement along the first target path, its pin engages with the traction ring to complete the docking between the robot and the rack to be tractioned.

[0028] In some application scenarios, step S13 above can be used to input the first pose and the first target pose into the path planning model to obtain the first target path. In other application scenarios, step S13 above can be used to convert the first pose and the first target pose to map coordinates respectively to obtain the converted first pose and the converted first target pose; and to perform preset path planning on the converted first pose and the converted first target pose to obtain the first target path.

[0029] The above scheme acquires the first point cloud data of the traction device in the shelf to be traction and the first pose of the robot collecting the first point cloud data. The traction shelf also includes a shelf, which is movably connected to the traction device. The traction device includes a traction ring. The first target pose of the traction ring relative to the robot is determined based on the first point cloud data. The first target path is determined based on the first pose and the first target pose. The first target path is used by the robot to traction the shelf through the traction ring in the traction device. In this way, the accuracy of the first target pose can be improved by obtaining the first target pose of the traction ring through the first point cloud data of the traction device, thereby improving the accuracy of the first target path determined based on the first target pose and the first pose, and thus improving the efficiency of the robot traction of the shelf based on the first target path.

[0030] Please see Figure 2 , Figure 2 yes Figure 1 A schematic diagram of the sub-process of step S12.

[0031] In some embodiments, step S12 may include the following steps: Step S21: Determine the first target point cloud data of the traction device from the first point cloud data. Step S22: Perform fitting processing on the first target point cloud data to obtain the fitting result of the traction device. Step S23: Determine the first target pose based on the fitting result of the traction device.

[0032] The first target point cloud data is at least a portion of the point cloud data in the first point cloud data. The first target point cloud data represents the point cloud data of the region of interest in the first point cloud data.

[0033] In some application scenarios, step S21 above can be used to select a fixed-size point cloud data from the first point cloud data as the first target point cloud data. The fixed size can be dynamically set according to the accuracy of path planning.

[0034] Please see Figure 3a , Figure 3a yes Figure 2 A schematic diagram of the sub-process of step S21.

[0035] In some embodiments, step S21 above may include the following steps: Step S31: Obtain the preset attribute information of the traction device.

[0036] The preset attribute information of the traction device characterizes the attribute information of each component in the traction device, specifically the length, width, or preset pose of each component in the map coordinate system. For example, when the traction device includes a traction ring, the preset attribute information of the traction device includes the first preset attribute information of the traction ring. The first preset attribute information includes at least one of the following: the outer diameter of the traction ring, the inner diameter of the traction ring, the height of the traction ring from the ground, and the preset pose of the traction ring in the map coordinate system or robot coordinate system. Among them, the preset pose of the traction ring includes, but is not limited to, the preset position of the reference point corresponding to the traction ring and the preset orientation angle of the traction ring.

[0037] Specifically, step S31 above may involve retrieving preset attribute information of the traction device from a server associated with the robot or a preset database.

[0038] Step S32: Determine the first initial position of the traction ring based on the preset attribute information of the traction device and the first position.

[0039] The first initial pose of the traction ring is represented by the pose of the traction ring relative to the robot, calculated based on the preset attribute information of the traction device.

[0040] In some application scenarios, the preset attribute information of the traction device and the first pose are input into the preset pose determination module to obtain the first initial pose of the traction ring relative to the robot output by the preset pose determination module.

[0041] In other application scenarios, the first initial position of the traction ring is determined based on the preset attribute information of the traction device, the first position, and the preset transformation relationship.

[0042] Step S33: Determine the target clipping region in the first point cloud data based on the preset attribute information of the traction device and the first initial pose.

[0043] In some application scenarios, the cutting direction is determined based on the values ​​of each axis in the first initial pose in the robot coordinate system, and the area that is equal to the length, width and height of the traction device in the preset attribute information of the traction device is selected from each cutting direction as the target cutting area.

[0044] In other application scenarios, the cutting range in each cutting direction is determined based on the first initial pose and the preset attribute information of the traction device. The area defined by the cutting range in each cutting direction is used as the target cutting area.

[0045] Step S34: Determine the first target point cloud data based on the target cropping region and the first point cloud data.

[0046] In some application scenarios, the first point cloud data belonging to the target clipping region is directly used as the first target point cloud data.

[0047] For example, a robot can be as follows Figure 3b The tractor shown can be used to tow the rack as follows: Figure 3b The illustrated tow rack includes a rack and a towing device; the rack is used to place items such as... Figure 3b The cargo shown has a traction device including a traction rod and a traction ring. Sensors for collecting point cloud data are placed at preset positions within the robot, such as... Figure 3b The ToF sensor shown is used in the robot. A pin in the robot is used to dock with a traction ring to complete the docking between the robot and the rack to be tractioned.

[0048] See the diagram for the tractor unit docking with the traction ring. Figure 3b As shown, in some other application scenarios, the triggering condition for obtaining the first point cloud data in step S11 above can be that after the tractor arrives at the identification preparation point in front of the tractor-type shelf, the original point cloud data of the tractor-type shelf is collected using a ToF sensor and used as the first point cloud data. Step S21 above can be to extract the traction device area from the first point cloud data to obtain the first target point cloud data, calculate the pose of the traction ring in the map coordinate system, i.e., the center point and the orientation angle, to achieve step S22 above. The tractor plans the point path between the first pose of the robot when collecting the first point cloud data and the identification result (i.e., the first target pose) to perform docking with the traction ring.

[0049] This application enables adaptive cropping of the point cloud distribution of the traction device region from the original point cloud (i.e., the first point cloud data or the second point cloud data acquired after step S13 above, taking the first point cloud data as an example). When the traction vehicle arrives at the recognition preparation point, the first point cloud data acquired by the ToF sensor needs to be transformed from the camera coordinate system to the vehicle coordinate system. After obtaining the point cloud data in the vehicle coordinate system, the point cloud cropping range strongly depends on the size information in the preset attribute information of the traction device and the preset pose of the reference point (such as the center point) corresponding to the traction ring in the vehicle coordinate system. Step S21 is specifically implemented as follows: Step S31 is executed to obtain the size information in the preset attribute information of the traction device. For example, the basic size information in the preset attribute information of the traction device is shown below. Figure 3cThe following parameters are included: the length of the traction bar, which can be represented by TowingBarLength; the width of the traction bar, which can be represented by TowingBarWidth; the outer diameter of the traction ring, which can be represented by TowingRingOuterDiameter; the inner diameter of the traction ring, which can be represented by TowingRingInnerDiameter; the height of the traction ring from the ground, which can be represented by TowingRingHeight; and the angle between the traction device and the horizontal direction, which can be represented by TowingBarAngle. Specifically, the process of projecting the length of the traction device in the x-axis direction in the robot coordinate system can be referred to the following formula (1): projectLenth =A1×A2 formula (1); Where A1 represents (TowingBarLength + TowingRingInnerDiamete), specifically representing the length of the traction device in the x-axis direction in the robot coordinate system. A2 represents cos(TowingBarAngle). projectLenth represents the projected length of the traction device in the x-axis direction in the robot coordinate system. The remaining parameters in formula (1) can be found in the size information of the preset attribute information of the traction device mentioned above, and will not be repeated here.

[0050] There is a preset conversion relationship between the preset pose of the reference point of the traction ring in the vehicle coordinate system and the pose of the robot when collecting point cloud data.

[0051] Specifically, the process of determining the first initial pose of the traction ring in step S32 above based on the preset attribute information of the traction device, the first pose, and the preset conversion relationship can refer to the following formula (2): Formula (2); in, This represents multiplication. The initial pose of the traction ring in the vehicle coordinate system (i.e., the robot coordinate system) is denoted as `init_pose_car`. Since the preset pose of the traction ring in the map coordinate system, `init_pose_map`, can be directly obtained from the preset attribute information of the traction device, and the vehicle pose, `car_pose`, at the recognition point (i.e., the robot's first pose when acquiring the first point cloud data) can be calculated from the odometry information, the poses are defined by quaternions, according to the preset transformation relationship (the preset pose is equal to the product of the robot's pose when acquiring the point cloud data and the traction ring's initial pose relative to the robot): The first initial pose of the traction ring can be calculated.

[0052] This requires ensuring that `car_pose` is reversible. The `x` value from `init_pose_car` (`init_pose_car_x`) is used to determine the clipping range in the x-direction. This is understandable given that in robot coordinates... Figure 3c Similarly, the y-axis, y-axis, and z-axis are obtained from init_pose_car. The y-value in init_pose_car_y is used to determine the clipping range in the y-direction, and the z-value in init_pose_car_z is used to determine the clipping range in the z-direction.

[0053] In other application scenarios, the method for determining the target clipping region in step S33 above may include the following: When clipping the point cloud range, additional clipping thresholds are set in the x, y, and z directions according to the first initial pose, to more accurately extract the distribution area of ​​the traction device and reduce interference from non-target areas. The extra length threshold extraLengthThreshold is used to constrain the clipping range in the x direction, the extra width threshold extraWidthThreshold is used to constrain the clipping range in the y direction, and the extra height threshold extraHeightThreshold is used to constrain the clipping range in the z direction. The setting of the point cloud clipping range is as follows: Specifically, the process of determining the target clipping region can refer to the following formulas (3) and (4): Formula (3); Formula (4); In formula (3), the specific values ​​of x, y, and z represent the cutting ranges set in the three directions of the x-axis, y-axis, and z-axis in the robot coordinate system. and These represent the minimum and maximum values ​​of the target clipping region along the x-axis in the robot coordinate system, respectively. and These represent the minimum and maximum values ​​of the target clipping region along the y-axis in the robot coordinate system, respectively. and These represent the minimum and maximum values ​​of the target clipping region along the z-axis in the robot coordinate system, respectively. , as well as These represent the additional clipping thresholds set in the x, y, and z directions based on the first initial pose. These thresholds can be preset values ​​or determined based on the first initial pose; there is no limitation here. The specific meanings of the other parameters in formulas (3) and (4) are as described above and will not be repeated here.

[0054] In some application scenarios, step S34 above may include the following steps: First point cloud data belonging to the target clipping region is used as candidate point cloud data. Clustering is performed on the candidate point cloud data to obtain the point cloud data corresponding to the target cluster. The first target point cloud data is determined based on the point cloud data corresponding to the target cluster.

[0055] Candidate point cloud data represents the point cloud data belonging to the target clipping region in the first point cloud data.

[0056] For example, step S34 above may include the following: retaining valid points (i.e., candidate point cloud data) within the point cloud clipping range for subsequent calculations to select the first target point cloud data from the candidate point cloud data. Clustering the clipped point cloud (candidate point cloud data) and filtering out noisy points in non-traction device areas. Using a depth-first search, starting point by point from the currently valid point cloud index list, checking its neighboring points in the four directions (up, down, left, right). If a neighboring point has not been visited, is valid, and meets the clustering conditions (by judging distance, etc.), it is added to the current clustering result. By continuously expanding neighboring points, a complete cluster is eventually formed. All valid points are traversed until all points have been visited or assigned to a cluster, thereby achieving clustering and segmentation of the candidate point cloud data. The clustering results are filtered, and the category with the largest number of points in the clustering results is the target cluster block (i.e., target cluster) for the traction device, and the point cloud data corresponding to this cluster block is directly used as the first target point cloud data.

[0057] The steps described above for determining the first target point cloud data based on the point cloud data corresponding to the target cluster may include the following steps: projecting the point cloud data corresponding to the target cluster onto a preset plane to obtain advanced point cloud data. The preset plane is either a horizontal plane or a ground plane. Then, filtering the advanced point cloud data to obtain the first target point cloud data.

[0058] It is understandable that projecting the 3D point cloud data corresponding to the target cluster at the traction device onto the ground to form a planar point cloud can be achieved by setting the z-coordinate value to 0 in the 3D point. During the projection process, although the traction device may have an angle with the horizontal direction, this angle will not affect the calculation of the yaw angle of the traction device in the vehicle coordinate system. In addition, for the acquisition of the reference point (x, y, z) of the traction ring, only (x, y) needs to be calculated in the map coordinate system. The traction vehicle will plan the path to the point to complete the docking based on the pose [x, y, yaw] of the traction ring in the map coordinate system. Therefore, although the projection process will lose the z-coordinate value, it will not affect the final destination. Because, after the traction ring is projected onto the ground, the existence of the angle will make the circle become an ellipse, but it will not affect the result of the center point fitting of the circle with a fixed radius. Therefore, this projection process will not affect the acquisition of the final center pose result of the traction ring, but it greatly facilitates the subsequent implementation of the preset fitting processing (i.e., circle fitting and / or line fitting) of the first target point cloud data, saving computing resources.

[0059] Specifically, the process of filtering advanced point cloud data to obtain the first target point cloud data may include the following: truncating the advanced point cloud data to obtain the first target point cloud data.

[0060] For example, point cloud data is truncated to subdivide the point cloud data corresponding to the traction device into a tow bar region and a tow ring region. In the planar point cloud (i.e., the advanced point cloud data) extracted in the above steps, the tip of the tow ring is identified, which is the point with the largest x-coordinate value in the vehicle coordinate system, and its point index and corresponding row index are recorded as the maximum row index. Next, the minimum row index in the planar point cloud is identified, which is the row index corresponding to the end point of the tow bar. Within a specified range (e.g., approximately the outer ring diameter from the tip of the tow ring), a group of boundary row indices is searched, and the final boundary row index is determined from this group. Based on the boundary row index, the maximum row index, and the minimum row index, the traction device region is subdivided into the tow bar and tow ring regions, as shown in [reference needed]. Figure 3d This allows for the extraction of effective index points corresponding to the traction ring region and traction rod region from the advanced point cloud data. In other application scenarios, this index truncation process can be applied to the first point cloud data (i.e., candidate point cloud data) belonging to the target clipping region or the point cloud data corresponding to the target cluster to obtain the first target point cloud data.

[0061] The fitting result of the traction device characterizes the attribute information of the fitted line segments and / or fitted loops obtained by performing a pre-defined fitting process on the point cloud data of the traction device. The attribute information of the fitted loop characterizes the coordinate information of the reference point corresponding to the fitted loop. The reference point corresponding to the fitted loop is the center point of the fitted loop.

[0062] The traction device includes a traction ring and a traction rod; before step S22 above, the first target point cloud data is divided into several point cloud data to be fitted according to the component types in the traction device; step S22 above may be to perform different preset fitting processes on each point cloud data to be fitted to obtain the fitting result of the traction device; the first target pose is determined according to the fitting result of the traction device.

[0063] In some application scenarios, the traction device also includes a traction rod, and the traction ring is movably connected to the shelf through the traction rod. The fitting result of the traction device includes the target fitted line segment of the traction rod and / or the target fitted coordinates of the reference point corresponding to the traction ring. Step S22 above may include the following steps: performing a first preset fitting process on the first target point cloud data belonging to the traction rod to obtain the target fitted line segment of the traction rod. Wherein, the first preset fitting process represents a preset straight line detection process, and this application does not limit the specific straight line detection algorithm. And / or, performing a second preset fitting process on the first target point cloud data belonging to the traction ring to obtain the target fitted coordinates of the reference point corresponding to the traction ring. Wherein, the second preset fitting process represents at least one circle fitting process, and this application does not limit the specific circle fitting algorithm.

[0064] In other application scenarios, step S22 may include the following steps: performing a first preset fitting process on the first target point cloud data belonging to the traction rod to obtain the target fitting line segment of the traction rod; and performing a second preset fitting process on the first target point cloud data belonging to the traction ring to obtain the target fitting coordinates of the reference point corresponding to the traction ring.

[0065] The first target point cloud data is a collection of point cloud data of the traction ring and point cloud data of the preset edge region of the traction rod in the first point cloud data.

[0066] Please see Figure 4a , Figure 4a yes Figure 2 A schematic diagram of the sub-process of step S22.

[0067] In some embodiments, the traction device further includes a traction rod, and the traction ring is movably connected to the shelf via the traction rod. The fitting result of the traction device includes the target fitted line segment corresponding to the traction rod. Step S22 above may include the following steps: Step S41: Use the first target point cloud data belonging to the traction rod as the second point cloud data to be fitted.

[0068] The second point cloud data to be fitted represents the first target point cloud data belonging to the traction rod in the first point cloud data.

[0069] In some application scenarios, step S41 can be used to directly use all point cloud data belonging to the traction rod in the first point cloud data as the second point cloud data to be fitted. In other application scenarios, step S41 can be used to use point cloud data belonging to a preset edge region of the traction rod in the first point cloud data as the second point cloud data to be fitted. The preset edge region represents the region to which a preset component of the traction rod belongs. The preset component is a part of the traction rod, and the two endpoints of the preset component are the connection point between the traction rod and the shelf and the connection point between the traction rod and the traction ring, respectively.

[0070] Step S42: Perform linear fitting on the second point cloud data to be fitted to obtain the initial fitted line segment.

[0071] The initial fitted line segment is the fitting result obtained by straight-line fitting of the second point cloud number to be fitted. The initial fitted line segment includes several already fitted line segments.

[0072] In some application scenarios, step S42 may involve performing at least one preset line detection process on the second point cloud to be fitted to obtain a detection line; and then performing a fusion process on the detection lines obtained from each preset line detection process to obtain an initial fitted line segment.

[0073] Step S43: Evaluate the initial fitted line segment to obtain the target fitted line segment.

[0074] The target fitted line segment represents the final fitting result of the traction rod.

[0075] In some application scenarios, the evaluation process in step S43 above can be as follows: if the line segment attribute information in the initially fitted line segment meets the preset evaluation requirements, the initially fitted line segment is used as the target fitted line segment; if the line segment attribute information of each fitted line segment in the initially fitted line segment does not meet the preset evaluation requirements, it is determined that the straight line fitting process of the second point cloud data to be fitted has an anomaly, and step S41 or step S42 above is re-executed. The line segment attribute information includes at least one of the following: the line segment length, line segment width, and angle with a preset direction of the fitted line segment. The preset direction can be the x-axis direction in the robot coordinate system. Specifically, if the line segment length of the fitted line segment is within a preset length range, the fitted line segment is determined to meet the preset evaluation requirements; and / or, if the line segment width of the fitted line segment is within a preset width range, the fitted line segment is determined to meet the preset evaluation requirements; and / or, if the angle between the fitted line segment and the preset direction is within a preset angle range, the fitted line segment is determined to meet the preset evaluation requirements. The preset length range, preset width range, and preset angle range can be dynamically set according to requirements. For example, the specific values ​​of the preset length range, preset width range, and preset angle range can be set according to the preset attribute information of the traction device, such as the length and width of the traction rod and the angle between the traction rod and the preset direction.

[0076] In other application scenarios, the evaluation process of step S43 above includes: determining whether to use the fitted line segment as the target fitted line segment based on whether the angle between the direction vectors of each fitted line segment in the initial fitted line segment meets the preset angle threshold. For example, the initial fitted line segments include a fitted left line segment and a fitted right line segment; the direction vector of the fitted left line segment is a unit vector pointing from the starting point to the ending point of the fitted left line segment, and the direction vector of the fitted right line segment is a unit vector pointing from the starting point to the ending point of the fitted right line segment; in response to the angle between the direction vectors of the fitted left line segment and the fitted right line segment being less than or equal to a preset angle threshold, and the fitted left line segment and the fitted right line segment having a minimum distance within a preset range in space and / or having a reasonable intersection point after extension, the fitted left line segment and the fitted right line segment are taken as target fitted line segments; in response to the angle between the direction vectors of the fitted left line segment and the fitted right line segment being greater than a preset angle threshold, and / or the spatial distribution of the fitted left line segment and the fitted right line segment not conforming to the preset structural characteristics, it is determined that the straight line fitting process of the second point cloud data to be fitted has an anomaly, and step S41 or S42 is re-executed.

[0077] In some application scenarios, the fitting result of the traction device includes the target fitted line segment of the traction rod, and the first target pose includes the first orientation angle of the traction ring. Step S23 above may include the following steps: determining the first target pose based on the fitting result of the traction device, including: taking the angle of the line segment belonging to a preset component in the traction rod as the target line segment angle, where the preset component represents the first component in the traction rod connected to the traction ring; determining the first orientation angle of the traction ring based on the target line segment angle; wherein the target fitted line segment includes two fitted line segments; and taking the average of the line segment angles of the two fitted line segments in the target fitted line segment as the first orientation angle of the traction ring.

[0078] For example, step S22 above can be used to calculate the orientation (i.e., the first orientation angle) of the traction ring when the first point cloud data is collected, using the first target point cloud data belonging to the traction rod region. This step is used to extract the left and right boundary index points of the traction rod, which are used to fit two straight lines to calculate the orientation angle. To reduce the error of directly using the point cloud of the traction ring region to calculate the orientation of the traction ring, this application uses the traction rod connected to the traction ring to assist in calculating the orientation of the traction ring, specifically including the following: First, step S41 above can be used to obtain the left and right boundary points of the traction rod. Specifically, the point indexes of the traction rod region in the first target point cloud data are grouped by row and stored in the row index group; for each row, the point with the smallest index in the row is found as the left boundary point, and the point with the largest index is found as the right boundary point, and stored in the left boundary point index group and the right boundary point index group respectively, where the left boundary point index group is used to fit the left straight line, and the right boundary point index group is used to fit the right straight line. Then, step S42 above is performed on the left boundary point and the right boundary point respectively. Line fitting can be based on RANSAC (Random Sample Consensus) fitting line algorithm to fit two straight lines on the left and right boundaries of the traction rod: extract target point cloud data according to the input point index group, and then randomly select a certain number of sample points in these points to estimate the parameters of the straight line model (the straight line equation Ax+By+C=0); calculate the distance from all points to the fitted line, count the number of consistency points that meet the error threshold, and evaluate the model based on the number of consistency points and the average error; in multiple iterations, continuously update the best model parameters until the best fitted line that meets the conditions is found; finally, calculate the slope, intercept and angle with the x-axis of the straight line based on the best model parameters; after calculating the angles Angle1 and Angle2 between the fitted lines of the left and right boundaries of the traction rod and the positive direction of the x-axis respectively using this method, calculate the average value to obtain the first orientation angle yaw of the traction ring, that is, the calculation process of the first orientation angle can refer to the following formula (5): Formula (5); Where Angle1 and Angle2 represent the angles between the fitting lines of the left and right boundaries of the traction rod and the positive x-axis, respectively. yaw represents the first orientation angle of the traction ring. See Figure 4b The sign of the included angle is specified when solving the problem. For example, if the slope of the left fitted line is positive, the included angle Angle1 is positive. If the slope of the right fitted line is negative, the included angle Angle2 is negative. The range of the included angle is [-90°, 90°].

[0079] In some application scenarios, the fitting result of the traction device includes the target fitted coordinates of the reference point corresponding to the traction ring, and the first target pose includes the position information of the traction ring. Step S23 above may include the following step: using the target fitted coordinates of the reference point corresponding to the traction ring as the position information of the traction ring.

[0080] Please see Figure 5a , Figure 5a yes Figure 2 A schematic diagram of another sub-process in step S22.

[0081] In some embodiments, the fitting result of the traction device includes the target fitting coordinates of the reference point corresponding to the traction ring.

[0082] Step S22 above may include the following steps: Step S51: Use the first target point cloud data belonging to the traction ring as the first point cloud data to be fitted.

[0083] The first point cloud data to be fitted represents the first target point cloud data belonging to the traction ring in the first point cloud data.

[0084] In some application scenarios, step S51 can be used to directly use all point cloud data belonging to the traction ring in the first point cloud data as the first point cloud data to be fitted. In other application scenarios, step S51 can be used to use point cloud data belonging to a preset edge region of the traction ring in the first point cloud data as the first point cloud data to be fitted. The preset edge region represents the region to which a preset side of the traction ring belongs, and the preset side is the inner ring side or the outer ring side of the traction ring.

[0085] Step S52: Select the inner ring boundary points of the traction ring from the first point cloud data to be fitted.

[0086] The inner ring boundary points represent the point cloud data belonging to the inner ring side of the traction ring in the first point cloud data to be fitted.

[0087] In some application scenarios, step S52 above can involve traversing the row and / or column indices of the first point cloud data to be fitted, selecting invalid points in the inner loop region. In some application scenarios, the edge points among the invalid points in the inner loop region are used as the inner loop boundary points of the traction loop. In other application scenarios, the invalid points in the inner loop region are clustered to obtain the target clusters corresponding to the invalid points, and the edge points in the target clusters corresponding to the invalid points are used as the inner loop boundary points of the traction loop. In still other application scenarios, the first point cloud data is filtered according to preset rules based on the target clusters corresponding to the invalid points to obtain the inner loop boundary points.

[0088] Step S53: Perform at least one preset circle fitting process on the inner ring boundary points to obtain the initial fitted coordinates.

[0089] The initial fitted coordinates represent the fitting results of the reference points obtained by performing at least one preset circle fitting process on the inner ring boundary points.

[0090] In some application scenarios, step S53 above can involve performing various preset circle fitting processes on the inner ring boundary points to obtain the fitting coordinates output by each preset circle fitting process, and using the fitting coordinates output by each preset circle fitting process as the initial fitting coordinates. In other application scenarios, step S53 above can also involve directly using the fusion result obtained by weighted fusion processing of the fitting coordinates output by each preset circle fitting process as the initial fitting coordinates.

[0091] Step S54: Determine the target fitting coordinates based on the initial fitting coordinates.

[0092] The target fitting coordinates characterize the final fitting result of the reference points obtained by performing various preset circle fitting processes on the inner ring boundary points.

[0093] In other application scenarios, step S54 above can be implemented by using the initial fitted coordinates as the target fitted coordinates in response to a preset condition being met by the difference between the initial fitted coordinates and the preset coordinates. If the difference between the initial fitted coordinates and the preset coordinates does not meet the preset condition, the initial fitted coordinates are corrected to obtain the target fitted coordinates. Correcting the initial fitted coordinates to obtain the target fitted coordinates can be achieved by re-executing steps S51, S52, or S53 above.

[0094] In other application scenarios, step S22 above can be the fitting result of performing circle fitting on the extracted inner ring boundary points of the first target point cloud data to calculate the reference points of the traction ring. Considering that the outer ring area of ​​the traction ring is connected to the traction rod, the accuracy of its outer ring boundary points is low, so the focus is placed on the extraction of the inner ring boundary points. Step S52 above can be mainly as follows: extracting invalid index points of the inner ring of the traction ring to obtain invalid points in the inner ring area.

[0095] The traction loop region is essentially a connected and closed area, with a hollow interior. Since the first point cloud data or the first target point cloud data is ordered, the hollow area inside the loop has corresponding index points. These index points are called invalid points, meaning their coordinate values ​​are all 0. Based on the row distribution range and valid index points of the traction loop region obtained through the above truncation process, by traversing each row and extracting the maximum and minimum point indices for that row, and incrementing the index values ​​by 1 in each row, the invalid point indices for each row can be extracted. After traversing all rows, the invalid point indices of the inner loop extracted from the row distribution can be obtained. Similarly, invalid points are extracted in each column. First, the column distribution range of the valid index points of the traction ring is calculated. Each column is traversed, and the maximum and minimum point indices are extracted. Since the index values ​​in the column direction increment sequentially along the "point cloud width" (which is determined based on the parameters of the 3D camera or ToF sensor), but the row index increments by 1, it needs to be converted to a "row" for traversal. Then, the invalid point indices corresponding to that column are extracted. After traversing all columns, the invalid point indices extracted from the column distribution are obtained. The invalid point indices extracted from the row and column distributions are queried. If the invalid point index extracted from the row distribution also exists in the invalid point index extracted from the column distribution, this invalid point index is recorded as an invalid point in the inner ring region for subsequent calculations; otherwise, the invalid point index is discarded. It can be understood that invalid points in the first target point cloud data can be referenced... Figure 5b As shown. (Through) Figure 5b The row and / or column distributions shown identify invalid points in the inner loop region.

[0096] After extracting the invalid point indices from the invalid points in the inner ring region, clustering and filtering of the invalid point indices are performed. To reduce the adverse effects of missing point clouds and to make the extracted invalid points more consistent with the target, i.e., distributed in the hollow area of ​​the inner ring, it is necessary to cluster and filter the invalid point indices. Here, a depth-first traversal is used to visit each invalid point index, checking its neighboring points in the four directions of up, down, left, and right. If the neighboring points have not been visited, they are added to the current category. Figure 5c An example is given. If a gap in the point cloud appears, there may be multiple clustering results. After extracting all cluster blocks, the cluster block with the most invalid points is retained as the target cluster corresponding to the invalid points, i.e., the target inner ring hollow region cluster block. This will be as follows: Figure 5c Cluster block 1 is shown as the target cluster corresponding to the invalid point, and the edge points in the target cluster corresponding to the invalid point are taken as the inner ring boundary points of the traction ring.

[0097] In other application scenarios, after extracting the invalid point indices of the hollow inner ring region, the valid point indices of the traction ring are used to extract the inner ring boundary points. The specific method is as follows: traverse each invalid point index, accessing its four elements (up, down, left, and right). If the element belongs to a valid point index and has not been accessed previously, record that valid point index. This allows obtaining all the boundary points of the inner ring. For example, the inner ring boundary points could be like... Figure 5d The point cloud data corresponding to the point index shown.

[0098] In some embodiments, step S53 may include the following steps: performing a first circle fitting process on the inner ring boundary points to obtain the first fitted coordinates of the reference points; performing a second circle fitting process on the inner ring boundary points based on the first fitted coordinates and the preset radius of the traction ring in the preset attribute information of the traction device to obtain the second fitted coordinates of the reference points; and using the second fitted coordinates as the initial fitted coordinates.

[0099] The first and second circle fitting processes are different. The first fitting coordinates represent the fitting result of the reference points obtained by performing the first circle fitting process on the inner ring boundary points. The second fitting coordinates represent the fitting result of the reference points obtained by performing the second circle fitting process on the inner ring boundary points.

[0100] For example, step S53 above may include the following: This application takes the first circle fitting process as a circle fitting process based on the least squares method, and the second circle fitting process as a circle fitting method based on the Ceres optimization library as an example. Step S53 above can combine the least squares method and the circle fitting method of the Ceres optimization library to perform high-precision fitting of point cloud data (i.e., inner ring boundary points), and calculate the average distance between the fitted circle and the point cloud to evaluate the fitting effect. This method first performs preliminary fitting of the point cloud using the least squares method to obtain the initial estimate of the circle center, and then uses this initial value as the starting point of Ceres optimization to further optimize the circle center coordinates to improve the fitting accuracy, specifically as follows: First, the inner ring boundary points are fitted using the least squares method. By constructing a data matrix and using singular value decomposition (SVD) to solve the least squares problem, the initial coordinates of the circle center (i.e., the first fitted coordinates of the reference point) are obtained. Specifically, for each inner ring boundary point, a data matrix is ​​constructed, and an objective equation is constructed according to a fixed radius. The least squares problem is solved by SVD decomposition to obtain the initial estimate of the circle center, which is used as the first fitted coordinates of the reference point.

[0101] Next, a least-squares optimization problem is constructed using the Ceres optimization library to obtain the second fitted coordinates from the first fitted coordinates of the benchmark point. The center coordinates obtained by the least-squares method are used as initial values, and the center position is further adjusted through the optimization process. For each inner ring boundary point, a residual term is constructed and added to the optimization problem. During the optimization process, the Gauss-Newton method or other optimization algorithms are used to iteratively minimize the sum of squared residuals, thereby obtaining the optimal center coordinates. Based on the optimized center coordinates and the known radius (i.e., the preset radius of the traction ring in the preset attribute information of the traction device), the final fitted circle point cloud is generated and used as the second fitted coordinates of the benchmark point for visualization. The distance difference between the fitted circle and each point in the point cloud is calculated, and the average value is obtained to obtain the average distance between the fitted circle and the point cloud, which serves as an evaluation index of the fitting effect.

[0102] It can be considered that the accuracy of the fitted coordinates of the benchmark points obtained by this application through various circle fitting processes is relatively high. Specifically, by combining the least squares method with the Ceres optimization library, the accuracy of circle fitting can be significantly improved while ensuring computational efficiency. The least squares method provides good initial values ​​for the optimization process (i.e., the accuracy of the first fitted coordinates is relatively high), while Ceres optimization further optimizes the first fitted coordinates to improve the accuracy of the second fitted coordinates, thereby improving the accuracy of the target fitted coordinates determined based on the second fitted coordinates, ensuring the accuracy and robustness of the benchmark point fitting results.

[0103] In some application scenarios, step S54 above can be used to directly use the initial fitted coordinates as the target fitted coordinates.

[0104] In some embodiments, step S54 may include the following steps: First verification process: if the difference between the initial fitted coordinates and the preset position coordinates of the traction ring in the preset attribute information of the traction device is less than a first preset difference, the initial fitted coordinates are used as the target fitted coordinates. And / or, second verification process: in response to the difference between the initial fitted coordinates and the mean coordinates corresponding to the first point cloud data to be fitted being less than a second preset difference, the initial fitted coordinates are used as the target fitted coordinates. And / or, third verification process: in response to the distance between the initial fitted coordinates and the candidate line segment corresponding to the traction ring being less than a threshold distance, the initial fitted coordinates are used as the target fitted coordinates.

[0105] Step S54 above may include the following steps: performing a preset verification process on the initial fitted coordinates; and, in response to the initial fitted coordinates meeting the preset verification conditions, using the initial fitted coordinates as the target fitted coordinates. The preset verification process includes any one, any two, or all three of the first verification process, second verification process, and third verification process. When the preset verification process includes any two or all three of the first verification process, second verification process, and third verification process, each verification process may be executed in parallel or sequentially. In the case of sequential execution of each verification process, this application does not limit the order in which the verification processes are executed. In the case of parallel execution of each verification process, if any verification process fails, the circle fitting process is confirmed to have failed, and steps S51, S52, or S53 above are re-executed. If each verification process passes in the case of parallel execution of each verification process, the circle fitting process is determined to be successful, and the initial fitted coordinates are used as the target fitted coordinates.

[0106] For illustrative purposes, the processing of step S54 in this application is exemplified by the sequential execution of three verification processes, and the specific order of execution is not limited. The three verification processes performed on the initial fitted coordinates of the traction ring may include the following: Step S54 is used to determine whether the initial fitted coordinates of the fitted traction ring are reliable and accurate, and the result is verified in three stages in sequence. First, the first verification process is performed based on the reference point of the traction ring in the vehicle coordinate system with a preset pose (x1, y1). By setting a verification threshold, the difference between the fitted traction ring center point (x0, y0) and the initial pose (x1, y1) is compared to see if it meets the verification threshold. If it does, the next verification process (i.e., the second verification process) is performed; otherwise, the identification fails. Secondly, a second verification process is performed on the initial fitted coordinates. The average value of all point cloud data in the first point cloud dataset is directly used as the mean coordinates. Specifically, the mean coordinates (x2, y2) are calculated based on all valid points in the traction ring region. A verification threshold is set to determine if the difference between (x0, y0) and (x2, y2) meets the threshold. If it does, the process proceeds to the next verification step (i.e., the third verification process); otherwise, the recognition fails. Finally, a third verification process is performed on the initial fitted coordinates. This requires performing a third verification based on the preset line segments obtained from the fitted left and right lines of the traction rod in the previous steps. Figure 5e As shown, the preset line segment can be the angle bisector obtained by the left and right fitted lines. The left and right fitted lines of the traction rod will intersect at a point (x3, y3). The orientation angle yaw can be used to calculate the slope k = tan(yaw) of the angle bisector. Given (x3, y3) and the slope k, the equation of the angle bisector can be determined: kx+y+(kx3 Since y3)=0, theoretically (x0, y0) is distributed along the angle bisector. However, in practice, due to various errors, it does not strictly pass through this line. Therefore, it is necessary to calculate the distance from (x0, y0) to the angle bisector and determine whether it meets the set verification threshold. If it does, the third verification process passes; otherwise, the recognition fails. In other application scenarios, depending on the type of traction device and its placement angle, the type of preset line segment between the left and right fitted lines is different. The angle between the direction vectors of the left and right fitted lines is used as the angle to be compared. When the angle to be compared is within the first preset angle range or the slopes of the left and right fitted lines are equal, the left and right fitted lines are considered to be approximately parallel or completely parallel. The preset line segment can also be an equidistant line segment with a preset slope between the left and right fitted lines. The preset slope can be the slope of the left fitted line, the slope of the right fitted line, or the average of the slopes of the left and right fitted lines. It is understandable that when the angle to be compared is within the first preset angle range, the extended direction vectors of the left and right fitted lines intersect, and the difference between the angle between the left fitted line segment and the preset direction and the angle between the right fitted line segment and the preset direction is less than or equal to the threshold angle. The type of traction device is as follows: Figures 5f to 5i As shown. Understandably, before performing the third verification process, it is necessary to obtain the preset line segment between the left and right fitted lines based on the type and placement angle of the traction device.

[0107] For example, the type of traction rod in the traction device of this application can be a preset type. For example, the preset type can be a type A traction rod or other types, such as rectangular traction rods, straight traction rods, and U-shaped traction rods. When the type of traction rod is one of the above-mentioned other types, RANSAC fitting can be performed based on the left and right boundaries respectively, and the angles with the x-axis direction can be calculated separately. Unlike the type A traction rod, the angles between the left and right fitting lines of other types of traction rods and the x-axis direction in the robot coordinate system are basically the same. The average of the line segment angles of the left and right fitting lines is taken as the first orientation angle of the traction rod. For example, the line segment angle of the left fitting line, i.e., the angle between the left fitting line and the x-axis, is 10°, and the line segment angle of the right fitting line, i.e., the angle between the right fitting line and the x-axis, is 9.6°. The final calculated first orientation angle is 9.8°. At this time, there is an angle between the left and right fitting lines and this angle is within the first preset angle range. It is considered that the left and right fitting lines are parallel, and the parallel lines with equal distances between the left and right fitting lines are determined as the above-mentioned preset line segments.

[0108] In other application scenarios, the reliability of the left and right fitted lines is determined based on the type of the tow bar and the angle between the direction vectors of the left and right fitted lines. The angle between the direction vectors of the left and right fitted lines is used as the angle to be compared. If the slopes of the left and right fitted lines are not equal, and the tow bar type belongs to the target type and the angle to be compared is not within the second preset angle range, the reliability of the left and right fitted lines is confirmed to be low, the fitting of the left and right fitted lines fails, and the reliability of the first orientation angle determined based on the left and right fitted lines is low. It is understood that when the angle to be compared is within the second preset angle range, the left and right fitted lines conform to the structural characteristics of the tow bar. For example, in the case of a rectangular tow bar... Figure 5g The middle traction rod area includes the left traction rod area and the right traction rod area. The right traction rod area includes traction rods belonging to areas N1 and N2, and the left traction rod area includes traction rods belonging to areas N3 and N4. If the angle to be compared is not within the second preset angle range, it may be due to a significant offset of the rack or traction device to be tractioned. The right fitted line may be incorrect. Figure 5g The N1 region in the model is determined to the fitted line, and the left fitted line may be the... Figure 5g The N4 region in the model is identified as the fitted line, which leads to lower reliability of the left and right fitted lines, resulting in lower accuracy of the subsequently calculated orientation angle. The target type can be a rectangular or U-shaped traction rod. In response to the angle to be compared being within the second preset angle range, the reliability of the left and right fitted lines is confirmed to be high, and the fitting of the left and right fitted lines is successful.

[0109] Please see Figure 6a , Figure 6a This is another flowchart illustrating an exemplary embodiment of the path planning method of this application.

[0110] In some embodiments, after step S13 described above, the path planning method may further include the following steps: Step S61: Acquire the second point cloud data of the traction device and the second pose of the robot for acquiring the second point cloud data. The acquisition time of the second point cloud data is later than the acquisition time of the first point cloud data.

[0111] The second point cloud data consists of point cloud data collected by the traction device within a preset time period after the robot has traveled along the first target path. The triggering condition for collecting the second point cloud data can be either collecting it at a preset frequency or collecting it after a preset travel distance.

[0112] The acquisition methods for the second point cloud data and the second position pose are the same as those for the first point cloud data and the first position pose, only the timing of the acquisition is different, which will not be elaborated here.

[0113] Step S62: Determine the second target pose of the traction ring relative to the robot based on the second point cloud data.

[0114] The second target pose characterizes the pose of the traction ring in the robot's coordinate system when acquiring the second point cloud data. Step S62 is the same as step S12, and will not be repeated here.

[0115] Step S63: Correct the first target path based on the second pose and the second target pose to obtain the second target path. The second target path is used by the robot to pull the shelf via the traction ring in the traction device.

[0116] In some application scenarios, step S63 above can be used to obtain a second planned path by performing a preset path planning process based on the second pose and the second target pose; the second planned path can be directly used as the second target path; or the second target path can be weighted and fused with the untraveled paths in the first target path to obtain a fusion result, and the fusion result can be used as the second target path. The weights of each path in the weighted fusion are flexibly set according to the path length and / or the accuracy of the first point cloud data and the second point cloud data.

[0117] For example, after step S13 above, this application can also implement on-the-go identification and correction of the first target path, thereby achieving docking between the robot and the traction ring. Specifically, after extracting the first target pose and / or the second target pose of the traction ring from the vehicle coordinate system, it is converted into the traction ring pose [x, y, yaw] in the map coordinate system. Subsequently, the robot plans a path based on the first pose corresponding to the first target pose and / or the second pose corresponding to the second target pose to achieve accurate docking with the traction ring. Since the first point cloud data and / or the second point cloud data cropping process is highly dependent on the initial pose of the traction ring in the vehicle coordinate system, this identification process can be effectively extended to an on-the-go identification scheme, see Figure 6b Because the ToF sensor is installed at an angle downwards, it is necessary to pre-estimate the distribution of the recognition area that can completely capture the point cloud of the traction device. When the robot reaches the recognition area, it starts the walking and recognizing mode, and dynamically adjusts the path of the first target based on the real-time recognition results (i.e., the pose of the second target), thereby reducing unnecessary adjustment steps and significantly improving recognition efficiency and accuracy.

[0118] Please see Figure 7 , Figure 7 This is another flowchart illustrating an exemplary embodiment of the path planning method of this application.

[0119] In other application scenarios, this application can sequentially execute steps S71 to S75. Step S71: Determine the target clipping region in the first point cloud data based on the preset attribute information of the traction device and the first initial pose. Step S72: Determine the first target point cloud data based on the target clipping region and the first point cloud data. The first target point cloud data includes first point cloud data to be fitted belonging to the traction ring region after truncation processing and second point cloud data to be fitted belonging to the traction rod region. Step S73: Perform at least one preset circle fitting process on the first point cloud data to be fitted to obtain the target fitting coordinates of the reference point corresponding to the traction ring. Step S74: Perform at least one preset straight line fitting process on the second point cloud data to be fitted to obtain the first orientation angle of the traction ring. Step S75: Perform path planning based on the target fitting coordinates, the first orientation angle, and the robot's first pose to obtain the first target path.

[0120] It can be considered that the present application executes an adaptive point cloud pruning strategy based on the first pose and the preset pose of the traction device to extract the first target point cloud data in the traction device area, which can improve the accuracy of the first target point cloud data.

[0121] It can be considered that the method of extracting the traction ring and traction rod in this application realizes the partitioning processing of the first target point cloud data. The orientation angle of the traction ring is calculated using the first target point cloud data belonging to the traction rod area, and the target fitting coordinates of the reference point of the traction ring are calculated using the first target point cloud data belonging to the traction ring area. There is no need to paste additional auxiliary identification markers, so as to improve the efficiency and accuracy of determining the pose of the first target.

[0122] It can be considered that this application uses the distribution characteristics of the traction ring point cloud in the first target point cloud data and a refined inner ring boundary point extraction method to calculate the center point of the circle fitting, which can improve the accuracy of the determined target fitting coordinates.

[0123] It can be considered that after determining the initial fitted coordinates corresponding to the traction ring, performing at least one verification process on the initial fitted coordinates can determine the accuracy and reliability of the initial fitted coordinates, thereby improving the accuracy of the target fitted coordinates.

[0124] It can be considered that in step S13 above, after the robot determines the first target path and travels along the first target path, it acquires the second point cloud data and determines the pose of the second target to correct the first target path. This enables the robot to identify the target path while moving, reduce the path adjustment time, adjust the docking path in real time, and improve the efficiency and accuracy of the planned second target path.

[0125] The above scheme acquires the first point cloud data of the traction device in the shelf to be traction and the first pose of the robot collecting the first point cloud data. The traction shelf also includes a shelf, which is movably connected to the traction device. The traction device includes a traction ring. The first target pose of the traction ring relative to the robot is determined based on the first point cloud data. The first target path is determined based on the first pose and the first target pose. The first target path is used by the robot to traction the shelf through the traction ring in the traction device. In this way, the accuracy of the first target pose can be improved by obtaining the first target pose of the traction ring through the first point cloud data of the traction device, thereby improving the accuracy of the first target path determined based on the first target pose and the first pose, and thus improving the efficiency of the robot traction of the shelf based on the first target path.

[0126] Please see Figure 8 , Figure 8 This is a schematic diagram of an embodiment of the path planning device of this application. The path planning device 80 includes an acquisition module 81, a first determination module 82, and a second determination module 83; the acquisition module 81 is used to acquire first point cloud data of the traction device in the shelf to be tractioned and the first pose of the robot collecting the first point cloud data. The traction shelf also includes a shelf, which is movably connected to the traction device. The traction device includes a traction ring; the first determination module 82 is used to determine a first target pose of the traction ring relative to the robot based on the first point cloud data; the second determination module 83 is used to determine a first target path based on the first pose and the first target pose. The first target path is used by the robot to traction the shelf through the traction ring in the traction device.

[0127] The above scheme acquires the first point cloud data of the traction device in the shelf to be traction and the first pose of the robot collecting the first point cloud data. The traction shelf also includes a shelf, which is movably connected to the traction device. The traction device includes a traction ring. The first target pose of the traction ring relative to the robot is determined based on the first point cloud data. The first target path is determined based on the first pose and the first target pose. The first target path is used by the robot to traction the shelf through the traction ring in the traction device. In this way, the accuracy of the first target pose can be improved by obtaining the first target pose of the traction ring through the first point cloud data of the traction device, thereby improving the accuracy of the first target path determined based on the first target pose and the first pose, and thus improving the efficiency of the robot traction of the shelf based on the first target path.

[0128] Please refer to the path planning method for the functions performed by each module; they will not be elaborated here.

[0129] Please see Figure 9 , Figure 9This is a schematic diagram of the structure of an embodiment of the electronic device of this application. The electronic device 90 includes a memory 91 and a processor 92. The processor 92 is used to execute program instructions stored in the memory 91 to implement the steps in the above-described path planning method embodiment. In a specific implementation scenario, the electronic device 90 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 90 may also include mobile devices such as laptops and tablets, which are not limited here.

[0130] Specifically, processor 92 controls itself and memory 91 to implement the steps in the path planning method embodiments described above. Processor 92 can also be referred to as a CPU (Central Processing Unit). Processor 92 may be an integrated circuit chip with signal processing capabilities. Processor 92 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 92 can be implemented using integrated circuit chips.

[0131] The above scheme acquires the first point cloud data of the traction device in the shelf to be traction and the first pose of the robot collecting the first point cloud data. The traction shelf also includes a shelf, which is movably connected to the traction device. The traction device includes a traction ring. The first target pose of the traction ring relative to the robot is determined based on the first point cloud data. The first target path is determined based on the first pose and the first target pose. The first target path is used by the robot to traction the shelf through the traction ring in the traction device. In this way, the accuracy of the first target pose can be improved by obtaining the first target pose of the traction ring through the first point cloud data of the traction device, thereby improving the accuracy of the first target path determined based on the first target pose and the first pose, and thus improving the efficiency of the robot traction of the shelf based on the first target path.

[0132] Please see Figure 10 , Figure 10 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 100 stores program instructions 1001 thereon, which, when executed by a processor, implement the steps in any of the above-described path planning method embodiments.

[0133] The above scheme acquires the first point cloud data of the traction device in the shelf to be traction and the first pose of the robot collecting the first point cloud data. The traction shelf also includes a shelf, which is movably connected to the traction device. The traction device includes a traction ring. The first target pose of the traction ring relative to the robot is determined based on the first point cloud data. The first target path is determined based on the first pose and the first target pose. The first target path is used by the robot to traction the shelf through the traction ring in the traction device. In this way, the accuracy of the first target pose can be improved by obtaining the first target pose of the traction ring through the first point cloud data of the traction device, thereby improving the accuracy of the first target path determined based on the first target pose and the first pose, and thus improving the efficiency of the robot traction of the shelf based on the first target path.

[0134] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0135] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0136] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0137] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0138] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0139] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

Claims

1. A path planning method, characterized in that, The path planning method is applied to a robot, and the method includes: The robot acquires the first point cloud data of the traction device in the traction shelf and the first pose of the first point cloud data collected by the robot. The traction shelf also includes a shelf, which is movably connected to the traction device. The traction device includes a traction ring. Determining the first target pose of the traction ring relative to the robot based on the first point cloud data includes: determining the first target point cloud data of the traction device from the first point cloud data; performing fitting processing on the first target point cloud data to obtain the fitting result of the traction device; and determining the first target pose based on the fitting result of the traction device. A first target path is determined based on the first pose and the first target pose, and the first target path is used by the robot to pull the shelf through the traction ring in the traction device. The fitting result of the traction device includes the target fitting coordinates of the reference point corresponding to the traction ring; the step of fitting the first target point cloud data to obtain the fitting result of the traction device includes: taking the first target point cloud data belonging to the traction ring as the first point cloud data to be fitted; selecting the inner ring boundary point of the traction ring from the first point cloud data to be fitted; performing at least one preset circle fitting process on the inner ring boundary point to obtain the initial fitting coordinates; and determining the target fitting coordinates based on the initial fitting coordinates.

2. The method according to claim 1, characterized in that, The step of performing at least one preset circle fitting process on the inner ring boundary points to obtain the initial fitted coordinates includes: The first circle fitting process is performed on the inner ring boundary points to obtain the first fitted coordinates of the reference points; Based on the first fitted coordinates and the preset radius of the traction ring in the preset attribute information of the traction device, the inner ring boundary point is subjected to a second circle fitting process to obtain the second fitted coordinates of the reference point; The second fitted coordinates are used as the initial fitted coordinates.

3. The method according to claim 1, characterized in that, The step of determining the target fitting coordinates based on the initial fitting coordinates includes: In response to the initial fitted coordinates being less than a first preset difference in the difference between the initial fitted coordinates and the preset position coordinates of the traction ring in the preset attribute information of the traction device, the initial fitted coordinates are used as the target fitted coordinates; and / or, In response to the fact that the difference between the initial fitting coordinates and the mean coordinates corresponding to the first point cloud data to be fitted is less than a second preset difference, the initial fitting coordinates are used as the target fitting coordinates; and / or, In response to the initial fitted coordinates being less than a threshold distance between the initial fitted coordinates and the candidate line segment corresponding to the traction ring, the initial fitted coordinates are used as the target fitted coordinates.

4. The method according to claim 1, characterized in that, The traction device further includes a traction rod, and the traction ring is movably connected to the shelf through the traction rod. The fitting result of the traction device includes the target fitting line segment corresponding to the traction rod. The step of fitting the first target point cloud data to obtain the fitting result of the traction device includes: The first target point cloud data belonging to the traction rod is used as the second point cloud data to be fitted. The second point cloud data to be fitted is subjected to linear fitting to obtain an initial fitted line segment; The target fitted line segment is obtained by evaluating the initial fitted line segment.

5. The method according to claim 1, characterized in that, After the step of determining the first target path based on the first pose and the first target pose, the method further includes: Acquire the second point cloud data of the traction device and the second pose of the robot for collecting the second point cloud data, wherein the acquisition time of the second point cloud data is later than the acquisition time of the first point cloud data. The second target pose of the traction ring relative to the robot is determined based on the second point cloud data; The first target path is modified based on the second pose and the second target pose to obtain a second target path, which is used by the robot to pull the shelf through the traction ring in the traction device.

6. The method according to claim 1, characterized in that, The step of determining the first target point cloud data of the traction device from the first point cloud data includes: Obtain the preset attribute information of the traction device; Based on the preset attribute information of the traction device and the first pose, the first initial pose of the traction ring is determined; The target cropping region in the first point cloud data is determined based on the preset attribute information of the traction device and the first initial pose. Based on the target cropping region and the first point cloud data, the first target point cloud data is determined.

7. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to perform the method as claimed in any one of claims 1-6.

8. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they are used to implement the method as described in any one of claims 1-6.

Images