Object position detection method and apparatus, robot, readable storage medium, and program product

Abstract

The present application discloses an object position detection method, comprising: acquiring a priori position corresponding to a target object, and acquiring target point cloud data, collected by a robot, for the priori position; obtaining a plurality of candidate cluster centers corresponding to the target object; obtaining a plurality of candidate cluster center combinations; on the basis of geometric attributes respectively corresponding to the candidate cluster center combinations, determining, from among the candidate cluster center combinations, a target cluster center combination corresponding to the target object; and on the basis of the target cluster center combination, determining a target position of the target object with respect to the robot.

Description

Object position detection method and device, robot, readable storage medium and program product

[0001] Cross-reference to related applications

[0002] The present application claims priority to the Chinese patent application No. 202411079552.6, filed on August 6, 2024, entitled "Object position detection method and device, robot, readable storage medium and program product", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] The present application relates to the technical field of computer, in particular to an object position detection method and device, robot, computer readable storage medium and computer program product. BACKGROUND

[0004] With the development of robot technology, object detection and positioning technology appears. In the running process of the robot, the surrounding environment information is collected through the sensor, and then the position relationship between the surrounding object and itself is detected according to the environment information, which provides important data support for the positioning and path planning of the robot.

[0005] In the traditional technology, a graphic code with a unique code is pasted on the object in the environment, and the position and attribute information of the object are determined by scanning the graphic code by the robot. When the position of the object changes, the position information recognized according to the graphic code cannot indicate the actual position of the object, and there is a problem of low position detection accuracy. SUMMARY

[0006] According to various embodiments of the present application, an object position detection method and device, robot, readable storage medium and program product are provided.

[0007] An object position detection method, the method comprising:

[0008] obtaining a prior position corresponding to a target object, and obtaining target point cloud data collected by a robot for the prior position;

[0009] clustering the target point cloud data to obtain a plurality of candidate cluster centers corresponding to the target object;

[0010] grouping each candidate cluster center to obtain a plurality of candidate cluster center combinations;

[0011] calculating the geometric properties between each candidate cluster center included in the candidate cluster center combination;

[0012] determine, in each of the candidate cluster center combinations, a target cluster center combination corresponding to the target object based on the geometric attributes respectively corresponding to each of the candidate cluster center combinations;

[0013] determine a target position of the target object with respect to the robot based on the target cluster center combination.

[0014] An object position detection apparatus, the apparatus comprising:

[0015] a data acquisition module configured to acquire a prior position corresponding to a target object and acquire target point cloud data collected by a robot with respect to the prior position;

[0016] a point cloud clustering module configured to cluster the target point cloud data to obtain a plurality of candidate cluster centers corresponding to the target object;

[0017] a center combination module configured to group each of the candidate cluster centers to obtain a plurality of candidate cluster center combinations;

[0018] an attribute calculation module configured to calculate geometric attributes between each of the candidate cluster centers included in the candidate cluster center combinations;

[0019] a combination determination module configured to determine, in each of the candidate cluster center combinations, a target cluster center combination corresponding to the target object based on the geometric attributes respectively corresponding to each of the candidate cluster center combinations;

[0020] a position determination module configured to determine a target position of the target object with respect to the robot based on the target cluster center combination.

[0021] A robot comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the object position detection method when executing the computer program.

[0022] A computer readable storage medium having stored thereon a computer program, the computer program being executed by a processor to implement the steps of the object position detection method.

[0023] Details of one or more embodiments of the application are set forth in the accompanying drawings and the description below. Other features and advantages of the application will become apparent from the description, the drawings, and the claims. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced. Obviously, the accompanying drawings in the following description only only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0025] FIG. 1 is a diagram of an application environment of an object position detection method in an embodiment;

[0026] FIG. 2 is a flow diagram of an object position detection method in an embodiment;

[0027] FIG. 3 is a flow diagram of a step of collecting target point cloud data in an embodiment;

[0028] FIG. 4 is a diagram of a geometric attribute in an embodiment;

[0029] FIG. 5 is a flow diagram of an object position detection method in another embodiment;

[0030] FIG. 6 is a structural block diagram of an object position detection device in an embodiment;

[0031] FIG. 7 is an internal structural diagram of a robot in an embodiment. DETAILED DESCRIPTION

[0032] In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the accompanying drawings. The preferred embodiments of the present application are shown in the accompanying drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the application belongs. The terminology used in the description of the application herein is only for the purpose of describing the specific embodiments and is not intended to limit the present application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0034] The object position detection method provided by the embodiments of the present application can be applied to an application environment as shown in FIG. 1. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data required to be processed by the server 104. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. The terminal 102 obtains the prior position corresponding to the target object, and obtains the target point cloud data collected by the robot for the prior position. The terminal 102 clusters the target point cloud data to obtain a plurality of candidate cluster centers corresponding to the target object. The terminal 102 groups each candidate cluster center to obtain a plurality of candidate cluster center combinations. The terminal 102 calculates the geometric properties between each candidate cluster center included in the candidate cluster center combination. The terminal 102 determines the target cluster center combination corresponding to the target object in each candidate cluster center combination based on the geometric properties corresponding to each candidate cluster center combination respectively. The terminal 102 determines the target position of the target object for the robot based on the target cluster center combination. Among them, the terminal 102 can be, but is not limited to, various robots, personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle device, a projection device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The robot can be various industrial robots (such as handling robots, palletizing robots, spraying robots, etc.) that need to move autonomously, service robots (such as cleaning robots, delivery robots, mowing robots, etc.) or special robots (firefighting robots, underwater robots, security robots, etc.). The server 104 can be a stand-alone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0035] In an exemplary embodiment, as shown in FIG. 2, an object position detection method is provided, which is described below by taking a robot as an example. The object position detection method includes the following steps 202 to 212. Among them:

[0036] Step 202, obtaining a prior position corresponding to a target object, and obtaining target point cloud data collected by a robot for the prior position.

[0037] The target object refers to an object that needs to be detected in position in the robot operation area. For example, in the intelligent warehousing scenario, the carrying robot needs to carry the movable carrying platform (such as a shelf or a pallet) from one position to another position. At this time, the target object is the movable carrying platform. Before lifting the movable carrying platform, the center position of the movable carrying platform needs to be detected, and then the robot is controlled to travel directly below the movable carrying platform and lift and carry the movable carrying platform, thereby ensuring the stability during the carrying process. When the robot performs an obstacle avoidance task, it needs to monitor and identify obstacles in real time. At this time, the target object is the obstacle in the working environment. And so on.

[0038] The prior position refers to the approximate position of the target object. For example, the prior position corresponding to the target object can be obtained from the position information of the target object stored in the prior map.

[0039] The target point cloud data refers to the environment point cloud data collected by the robot through the sensor. In actual implementation, the target point cloud data can be two-dimensional laser point cloud data collected by a laser radar sensor.

[0040] Exemplarily, when the robot performs a task for the target object, the prior position corresponding to the target object is first obtained. Then, the target point cloud data for the prior position is obtained. Specifically, according to the prior position corresponding to the target object, the pose of the robot is adjusted so that the target object enters the sensing range of the sensor of the robot. The initial point cloud data for the prior position is collected through the sensor. Then, the point cloud data outside the prior position in the initial point cloud data is filtered to obtain the target point cloud data for the prior position.

[0041] In step 204, the target point cloud data is clustered to obtain a plurality of candidate cluster centers corresponding to the target object.

[0042] The candidate cluster center refers to a cluster center corresponding to a point cloud cluster that may be a positioning reference part of the target object extracted from the target point cloud data. The positioning reference part refers to a key part of each part of the target object for assisting in positioning the target object. For example, when the target object is a table, the positioning reference part can be a table leg. When the target object is a shelf, the positioning reference part can be a shelf leg. And so on.

[0043] Exemplarily, the robot clusters the target point cloud data based on distances between data points in the target point cloud data to obtain a plurality of initial point cloud clusters. The robot filters each initial point cloud cluster based on a size of the initial point cloud cluster to obtain a plurality of candidate point cloud clusters. A clustering center corresponding to each candidate point cloud cluster is taken as a candidate clustering center corresponding to the target object. In actual implementation, each initial point cloud cluster can be filtered based on at least one of a size, a shape, and a density of the initial point cloud cluster to obtain a plurality of candidate point cloud clusters. Filtering irrelevant point cloud data based on multiple factors can reduce interference caused by environmental factors, improve the accuracy of position detection, greatly reduce the amount of data calculation, save computer resources, and improve the efficiency of position detection.

[0044] In step 206, the candidate clustering centers are grouped to obtain a plurality of candidate clustering center combinations.

[0045] The candidate clustering center combination refers to a set containing a plurality of candidate clustering centers.

[0046] Exemplarily, the robot determines a minimum number of clustering centers required for positioning the target object according to a structural feature of the target object. Each candidate clustering center is grouped based on the minimum number of clustering centers corresponding to the target object to obtain a plurality of candidate clustering center combinations. For example, when the target object is a rectangular shelf with four shelf legs, the positions of three shelf legs are sufficient to determine the position of the rectangular shelf, and therefore, the minimum number of clustering centers corresponding to the rectangular shelf is 3. Each time, three different candidate clustering centers are selected from the candidate clustering centers to form a candidate clustering center combination, and all possible combinations are traversed to obtain a plurality of candidate clustering center combinations.

[0047] In step 208, geometric properties between the candidate clustering centers included in the candidate clustering center combination are calculated.

[0048] The geometric property refers to a geometric feature of a geometric shape formed by the candidate clustering centers, and includes at least one of a line, an angle, and a surface.

[0049] Exemplarily, for any candidate clustering center combination, the robot calculates geometric properties between the candidate clustering centers in the candidate clustering center combination. The geometric properties can include at least one of a length of a line between the candidate clustering centers, an angle between the lines, and a plane on which each candidate clustering center is located.

[0050] In step 210, a target clustering center combination corresponding to the target object is determined in each candidate clustering center combination based on respective geometric properties of the candidate clustering center combinations.

[0051] The target cluster center combination refers to a set composed of cluster centers corresponding to the positioning reference parts of the target object.

[0052] Exemplarily, the robot compares the geometric attributes corresponding to the candidate cluster center combination with the standard geometric attributes corresponding to the target object, and determines the candidate cluster center combination that is successfully compared as the target cluster center combination corresponding to the target object.

[0053] In some embodiments, the robot calculates the geometric shape type and the geometric size corresponding to the candidate cluster center combination based on the geometric attributes corresponding to the candidate cluster center combination. The candidate cluster center combination that has the same geometric shape type and geometric size as the target object is determined as the target cluster center combination corresponding to the target object. The geometric size refers to the size of the geometric shape, for example, when the geometric shape is a square, the geometric size can be the side length of the square; when the geometric shape is a rectangle, the geometric size can be the length and width of the rectangle; and so on.

[0054] In step 212, the target position of the target object with respect to the robot is determined based on the target cluster center combination.

[0055] Exemplarily, the robot calculates the target position of the target object with respect to the robot according to the positions of the cluster centers in the target cluster center combination and the structural features of the target object, which include the geometric shape type. The geometric shape type is a shape such as a rectangle, a regular pentagon, or a circle. The work task for the target object is performed according to the target position of the target object.

[0056] In some embodiments, the target cluster center combination can be input into a center position prediction model to obtain the target position of the target object with respect to the robot. The center position prediction model is obtained by supervised training based on a plurality of object samples respectively corresponding to cluster center combinations and the known target positions of each object sample with respect to the robot.

[0057] In some embodiments, the positioning reference parts corresponding to the target object can hinder the robot from traveling, for example, when the target object is a shelf and the positioning reference parts are shelf legs supporting the shelf, the shelf legs are obstacles that hinder the robot from traveling. Therefore, when the positioning reference parts corresponding to the target object hinder the robot from traveling, the positions of all the positioning reference parts corresponding to the target object are calculated based on the target cluster center combination corresponding to the target object. The obstacle avoidance travel for the target object is performed based on the positions of all the positioning reference parts corresponding to the target object and the target position of the target object, so as to perform the work task for the target object.

[0058] In the object position detection method, the prior position corresponding to the target object is obtained, target point cloud data of the prior position collected by the robot is obtained, and the target point cloud data is clustered to obtain a plurality of candidate cluster centers corresponding to the target object. Then, the candidate cluster centers are grouped to obtain a plurality of candidate cluster center combinations, and the geometric properties between the candidate cluster centers included in each candidate cluster center combination are calculated. Based on the geometric properties corresponding to each candidate cluster center combination, the target cluster center combination corresponding to the target object is determined in each candidate cluster center combination. Finally, the target position of the target object with respect to the robot is determined based on the target cluster center combination. In this way, the candidate cluster centers are grouped to obtain a plurality of candidate cluster center combinations to be identified, and it is determined whether each candidate cluster center combination is the target cluster center combination corresponding to the target object. Compared with directly finding the cluster center corresponding to the target object from all candidate cluster center combinations, the search range can be greatly reduced, the calculation amount can be significantly reduced, and the efficiency of position detection can be improved. Furthermore, since the geometric properties corresponding to the candidate cluster center combination can clearly and intuitively reflect the characteristics of the geometric shape formed by the candidate cluster centers in the combination, the target cluster center combination corresponding to the target object can be quickly and efficiently determined from the candidate cluster center combinations based on the geometric properties, thereby reducing the complexity of position detection, improving the efficiency of position detection, and improving the accuracy of position detection.

[0059] In an exemplary embodiment, as shown in FIG. 3, the target point cloud data of the prior position collected by the robot includes steps 302 to 306. Among them:

[0060] Step 302, obtaining initial point cloud data of the prior position collected by the robot.

[0061] Step 304, determining the relative position of the target object with respect to the robot based on the robot pose corresponding to the robot and the prior position.

[0062] Step 306, filtering the initial point cloud data based on the relative position to obtain the target point cloud data of the prior position.

[0063] Among them, the initial point cloud data refers to the initial environment point cloud data collected by the robot through the sensor. The robot pose refers to the specific position and orientation of the robot in space when the robot collects the initial point cloud data.

[0064] Exemplarily, the robot obtains initial point cloud data of the prior position of the target object collected by the sensor. Based on the robot pose corresponding to the robot when the initial point cloud data is collected, the prior position of the target object is converted into a relative position corresponding to the robot. Then, based on the relative position and the size information of the target object, a point cloud filtering range corresponding to the target object is determined. In actual implementation, the point cloud filtering range can be a range determined based on the relative position and slightly larger than the size of the target object. The data points outside the point cloud filtering range in the initial point cloud data are filtered to obtain target point cloud data of the prior position.

[0065] In the above embodiment, according to the robot pose and the prior position of the target object, a part of the data points in the initial point cloud data is filtered, and only the data points around the prior position are retained, which can effectively reduce the interference caused by irrelevant point cloud data, improve the accuracy of position detection, greatly reduce the data calculation amount, save computer resources, and improve the calculation efficiency.

[0066] In an exemplary embodiment, the target point cloud data is clustered to obtain a plurality of candidate cluster centers corresponding to the target object, including:

[0067] Based on the size of the positioning reference part of the target object, a reference cluster size is determined.

[0068] The target point cloud data is clustered to obtain a plurality of initial point cloud clusters; each initial point cloud cluster is filtered based on the reference cluster size to obtain a plurality of candidate point cloud clusters; and the cluster center corresponding to each candidate point cloud cluster is taken as a candidate cluster center corresponding to the target object.

[0069] The positioning reference part refers to a key part of each part of the target object used to assist in positioning the target object.

[0070] Exemplarily, the robot obtains the size of the positioning reference part of the target object, enlarges the size of the positioning reference part by a certain ratio to obtain a reference cluster size. For example, the size of the positioning reference part can be enlarged by 1.5 times to obtain the reference cluster size. The target point cloud data is clustered to obtain a plurality of initial point cloud clusters. In each initial point cloud data, the initial point cloud cluster with a size larger than the reference cluster size is filtered according to the reference cluster size to obtain a plurality of candidate point cloud clusters. Specifically, the initial point cloud cluster with a minimum circumscribed rectangle length greater than the reference cluster size can be filtered. Then, the cluster center corresponding to each candidate point cloud cluster is taken as a candidate cluster center corresponding to the target object.

[0071] In the above embodiment, irrelevant point cloud data is filtered according to the size of the positioning reference part, which can reduce the interference caused by environmental factors, improve the accuracy of position detection, greatly reduce the data calculation amount, save computer resources, and improve the efficiency of object position detection.

[0072] In an example embodiment, the geometric properties between the candidate cluster centers included in the candidate cluster center combination are calculated, including:

[0073] For any candidate cluster center combination, the lengths of the lines connecting the candidate cluster centers in the candidate cluster center combination are calculated, and the angles between the lines are calculated to obtain the geometric properties between the candidate cluster centers included in the candidate cluster center combination.

[0074] For example, for any candidate cluster center combination, the robot calculates the lengths of the lines connecting the candidate cluster centers in the candidate cluster center combination, and calculates the angles between the lines. The lengths of the lines connecting the candidate cluster centers and the angles between the lines are taken as the geometric properties between the candidate cluster centers. For example, when the candidate cluster center combination includes three candidate cluster centers, the geometric properties between the candidate cluster centers can be as shown in FIG. 4. The candidate cluster center combination includes three candidate cluster centers A, B, and C. The line segments AC, AB, and BC are the lines connecting the candidate cluster centers, and the three angles of the triangle ABC are the angles between the lines.

[0075] In the above embodiment, for each candidate cluster center combination, the lengths of the lines connecting the candidate cluster centers and the angles between the lines are taken as the geometric properties between the candidate cluster centers. The lengths of the lines and the angles reflect the key set characteristics of the candidate cluster center combination from different angles, providing comprehensive and powerful data support for subsequent determination of the target cluster center combination corresponding to the target object based on the geometric properties. Taking the lengths of the lines and the angles between the lines as the geometric properties can reduce the complexity of the position detection, improve the efficiency of the position detection, and ensure the accuracy of the position detection.

[0076] In an example embodiment, the object position detection method further includes:

[0077] When the number of candidate cluster centers is greater than or equal to the number threshold, the step of grouping the candidate cluster centers to obtain multiple candidate cluster center combinations is performed. When the number of candidate cluster centers is less than the number threshold, the supplementary point cloud data for the prior position collected by the robot is obtained, the target point cloud data and the supplementary point cloud data are fused to obtain updated point cloud data, the updated point cloud data is taken as the target point cloud data, and the step of clustering the target point cloud data to obtain multiple candidate cluster centers corresponding to the target object is performed.

[0078] The number threshold refers to the minimum number of cluster centers required for positioning the target object.

[0079] Exemplarily, when the number of the candidate cluster centers obtained by clustering the target point cloud data is greater than or equal to the number threshold, the robot enters the step of grouping the candidate cluster centers to obtain a plurality of candidate cluster center combinations. When the number of the candidate cluster centers is less than the number threshold, the robot acquires supplementary point cloud data for the prior position. In actual implementation, the robot can be equipped with a plurality of sensors, each of which is used to acquire environmental point cloud data in different directions. When the number of the candidate cluster centers is less than the number threshold, the robot can acquire environmental point cloud data in another direction as the supplementary point cloud data. Then, the target point cloud data and the supplementary point cloud data are spliced to obtain updated point cloud data. The updated point cloud data is taken as the target point cloud data, and the step of obtaining a plurality of candidate cluster centers corresponding to the target object by clustering the target point cloud data is executed again.

[0080] In some embodiments, for a robot with only one sensor, the robot can acquire the supplementary point cloud data by rotating or driving to another position. Specifically, the robot can obtain the geometric size corresponding to the target object, and determine the shortest adjustment path of the robot based on the geometric size corresponding to the target object. In this way, the shortest adjustment path is determined according to the geometric size of the target object, the path length of the robot is shortened, and thus the position detection cost is reduced and the position detection efficiency is improved. After the robot is controlled to move to another position along the shortest adjustment path, environmental point cloud data for the prior position is acquired again. The robot can also rotate by a preset angle and acquire environmental point cloud data for the prior position again. Then, the environmental point cloud data acquired again is projected into the coordinate system corresponding to the target point cloud data, i.e., the environmental point cloud data is transformed into the coordinate system before the movement, to obtain the supplementary point cloud data. The updated point cloud data is formed based on the supplementary point cloud data and the target point cloud data.

[0081] In the above embodiments, when the number of the candidate cluster centers obtained by clustering the target point cloud data is less than the number threshold, it indicates that the target object contained in the target point cloud data has too little information, and it is difficult to determine the position of the target object according to the target point cloud data. Therefore, the environmental point cloud data acquired by the sensor in another direction is acquired as the supplementary point cloud data. The environmental point cloud data acquired by the sensors in different directions is fused to obtain updated point cloud data containing more comprehensive environmental information, and then object position detection is performed again based on the updated point cloud data, which can effectively improve the efficiency and success rate of position detection.

[0082] In an exemplary embodiment, the geometric properties include line lengths and line angles. Based on the geometric properties corresponding to each candidate cluster center combination, the target cluster center combination corresponding to the target object is determined in each candidate cluster center combination, including:

[0083] obtaining a reference length set and a reference angle corresponding to the target object; combining the candidate cluster center combination in which the line length and the reference length set match successfully and the line angle and the reference angle also match successfully as an intermediate cluster center combination; and determining the target cluster center combination corresponding to the target object from each intermediate cluster center combination.

[0084] The reference length set refers to a set containing the known standard side length corresponding to the target object. The reference angle refers to the known standard angle corresponding to the target object. For example, when the target object is a rectangular shelf supported by four shelf legs at four corners of the shelf, the reference length set contains the length and width of the shelf determined based on the distance between the shelf legs, and the reference angle is 90°.

[0085] Exemplarily, the robot obtains the reference length set and the reference angle corresponding to the target object. The line length of each line in each candidate cluster center combination is compared with each standard side length in the reference length set. The target cluster center combination corresponding to the target object is determined from the candidate cluster center combination in which the line length and the reference length set match successfully and the line angle between the lines that match successfully and the reference angle also match successfully. Specifically, when the line length of each line in the candidate cluster center combination can find a matching side length in the reference length set, the angle between the lines that match successfully is compared with the reference angle between the side lengths in the reference length set that correspond to the matching lines, and when the angle between the lines and the reference angle also match successfully, the candidate cluster center combination is determined as the intermediate cluster center combination corresponding to the target object. The matching successfully refers to that the difference between the two is less than a preset threshold. When there is only one intermediate cluster center combination, the intermediate cluster center combination is determined as the target cluster center combination. When there are multiple intermediate cluster center combinations, the difference between the line length in each intermediate cluster center combination and the corresponding reference length, and the difference between the angle between the lines that match successfully and the corresponding reference angle are fused to obtain the comprehensive difference value corresponding to each intermediate cluster center combination respectively. The intermediate cluster center combination with the smallest comprehensive difference value is taken as the target cluster center combination corresponding to the target object.

[0086] In the above embodiments, by comparing the line length and the angle between the lines of each candidate cluster center, the target candidate cluster center combination corresponding to the target object can be quickly and accurately determined, the calculation amount of the position detection is reduced, and the efficiency and accuracy of the position detection are improved.

[0087] In an exemplary embodiment, determining the target position of the target object with respect to the robot based on the target cluster center combination includes:

[0088] The center position corresponding to the target object is calculated based on the geometric shape type corresponding to the target object and the target cluster center combination. The target position of the target object relative to the robot is determined based on the robot pose corresponding to the robot and the center position.

[0089] Exemplarily, the robot obtains the geometric shape type corresponding to the target object. The geometric shape type refers to the shape formed by each positioning reference part corresponding to the target object, which can be a rectangular shape, a trapezoidal shape, a hexagonal shape, etc. The center position refers to the geometric center of the shape formed by each positioning reference part corresponding to the target object. The center position corresponding to the target object is calculated based on each target cluster center included in the target cluster center combination and the geometric shape type corresponding to the target object. For example, as shown in FIG. 4, the target cluster center combination includes three candidate cluster centers A, B, and C, and when the geometric shape type corresponding to the target object is a rectangular shape, the midpoint D of the diagonal AC is taken as the center position of the target object. Then, the robot pose corresponding to the robot at the current time and the historical robot pose corresponding to the robot when collecting the target point cloud data are obtained. The center position corresponding to the target object is adjusted based on the pose change between the current robot pose and the historical robot pose, to obtain the target position of the target object relative to the current robot pose of the robot. The robot performs a work task on the target object based on the target position corresponding to the target object.

[0090] In actual implementation, the robot obtains the odometer data between the time when the target point cloud data is collected and the current time, and determines the robot pose corresponding to the robot at the current time based on the historical robot pose corresponding to the robot when collecting the target point cloud data and the odometer data. The odometer data can be collected by an odometry (ODOM). The odometer data is data for estimating the position and direction change of the robot relative to the initial position during movement by measuring the movement components (such as wheels) of the robot.

[0091] In the above embodiments, the center position corresponding to the target object can be quickly and accurately determined based on the geometric shape type corresponding to the target object and the target cluster center combination, thereby improving the efficiency of position detection. Furthermore, the center position corresponding to the target object is adjusted according to the pose change of the robot, to obtain the target position of the target object relative to the robot pose at the current time, thereby eliminating the error of the target position of the target object relative to the robot caused by the movement of the robot, and improving the accuracy and precision of position detection.

[0092] In one specific embodiment, the object position detection method proposed in the present application can be applied to a carrying robot in an intelligent warehousing system. The carrying robot calculates the position of the rack to be jacked up by accurately identifying the position of the rack leg of the rack to be jacked up, and performs the task of jacking up and carrying the rack based on the rack position. As shown in FIG. 5, the object position detection method includes the following steps:

[0093] 1. Data acquisition

[0094] The carrying robot obtains the environment point cloud data of the rack to be jacked up as initial point cloud data. The odometer data and the prior position of the rack to be jacked up are obtained. The initial robot pose corresponding to the robot when the environment point cloud data is collected is obtained.

[0095] 2. Filter data points

[0096] The carrying robot filters a part of the data points in the initial point cloud data based on the initial robot pose and the prior position of the rack to be jacked up, and obtains target point cloud data. The target point cloud data only contains data points around the prior position of the rack to be jacked up.

[0097] 3. Filter point cloud clusters

[0098] The carrying robot clusters the target point cloud data according to the distance between each data point in the target point cloud data, and obtains a plurality of initial point cloud clusters. The size threshold is determined according to the rack leg size of the rack to be jacked up, the initial point cloud clusters with a minimum circumscribed rectangle length greater than the size threshold are filtered, a plurality of candidate point cloud clusters are obtained, and the candidate cluster centers corresponding to each candidate point cloud cluster are calculated.

[0099] 4. Detect rack position

[0100] When the number of candidate cluster centers obtained by screening is less than or equal to two, the environment point cloud data collected by another sensor is obtained as supplementary point cloud data. The target point cloud data and the supplementary point cloud data are fused to obtain updated point cloud data, the updated point cloud data is taken as the target point cloud data, and the step of filtering point cloud is returned to be executed.

[0101] When the number of the candidate cluster centers screened is greater than two, the candidate cluster centers are combined to obtain a plurality of candidate cluster center combinations. Each candidate cluster center combination contains three different candidate cluster centers. For any candidate cluster center combination, the length of the line connecting each candidate cluster center and the included angle between the lines are calculated. In each candidate cluster center combination, the candidate cluster center combination with the length of the line closest to the length of the shelf of the to-be-jacked shelf and the included angle closest to 90° is screened as a target cluster center combination. The center position of the to-be-jacked shelf is calculated based on each cluster center in the target cluster center combination. Then, the current robot pose corresponding to the current time of the carrying robot is calculated according to the odometer data and the initial robot pose. The center position of the to-be-jacked shelf is adjusted based on the pose change between the current robot pose and the initial robot pose, to obtain the target position of the to-be-jacked shelf for the current robot pose.

[0102] In actual implementation, as the sensor data is often disturbed by various noises, a noise covariance matrix can be introduced by an extended Kalman filter (EKF) to filter the target position and output the filtered target position, so as to suppress the interference of the noises on the object position detection to a certain extent. The carrying robot travels to the center position of the to-be-jacked shelf based on the target position of the to-be-jacked shelf finally obtained, and jacks up and carries the to-be-jacked shelf. Specifically, the carrying robot can calculate the positions of all the shelf legs corresponding to the to-be-jacked shelf based on the target cluster center combination, avoid obstacles based on the positions of the shelf legs and the target position, travel to the center position below the to-be-jacked shelf, and then jack up and carry the to-be-jacked shelf.

[0103] In the above embodiment, a part of the point cloud data is filtered according to the prior position of the shelf, and the position detection is performed based on the filtered point cloud data, so as to save the computing resources and improve the computing efficiency. The target point cloud data filtered is clustered to obtain a plurality of initial point cloud clusters, and the larger initial point cloud clusters are filtered according to the size of the shelf leg, so as to further save the computing resources, improve the computing efficiency, and improve the accuracy of the position detection. The candidate cluster centers corresponding to the candidate point cloud clusters filtered are combined, the length of the line connecting each candidate cluster center in each candidate cluster center combination and the included angle between the lines are determined, the target cluster center combination corresponding to the to-be-jacked shelf is determined, and then the center position of the to-be-jacked shelf is determined according to the target cluster center combination, so as to reduce the complexity of the position detection, improve the efficiency of the position detection, and improve the accuracy of the position detection.

[0104] It should be understood that although the steps in the flowcharts involved in the embodiments described above are shown in sequence according to the arrows, the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of the steps is not strictly limited in sequence, and the steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the embodiments described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of the steps or stages is not necessarily sequential, but can be alternately or alternately executed with at least part of other steps or steps or stages in other steps.

[0105] Based on the same inventive concept, the embodiments of the present application also provide an object position detection device for implementing the object position detection method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more object position detection device embodiments provided below can refer to the limitations of the object position detection method described above, which will not be repeated here.

[0106] In an exemplary embodiment, as shown in FIG. 6, an object position detection device is provided, comprising: a data acquisition module 602, a point cloud clustering module 604, a center combination module 606, an attribute calculation module 608, a combination determination module 610 and a position determination module 612, wherein:

[0107] The data acquisition module 602 is configured to acquire a prior position corresponding to a target object, and acquire target point cloud data for the prior position collected by a robot.

[0108] The point cloud clustering module 604 is configured to cluster the target point cloud data to obtain a plurality of candidate cluster centers corresponding to the target object.

[0109] The center combination module 606 is configured to group the candidate cluster centers to obtain a plurality of candidate cluster center combinations.

[0110] The attribute calculation module 608 is configured to calculate the geometric properties between the candidate cluster centers included in the candidate cluster center combinations.

[0111] The combination determination module 610 is configured to determine a target cluster center combination corresponding to the target object from the candidate cluster center combinations based on the geometric properties corresponding to each candidate cluster center combination.

[0112] The position determination module 612 is configured to determine a target position of the target object with respect to the robot based on the target cluster center combination.

[0113] In an embodiment, the data acquisition module 602 is further configured to:

[0114] acquire initial point cloud data for the prior position collected by the robot; determine a relative position of the target object with respect to the robot based on a robot pose of the robot corresponding to the prior position; filter the initial point cloud data based on the relative position to obtain target point cloud data for the prior position.

[0115] In an embodiment, the point cloud clustering module 604 is further configured to:

[0116] determine a reference cluster size based on a size of the positioning reference part of the target object; cluster the target point cloud data to obtain a plurality of initial point cloud clusters; filter each initial point cloud cluster based on the reference cluster size to obtain a plurality of candidate point cloud clusters; and take a cluster center corresponding to each candidate point cloud cluster as a candidate cluster center corresponding to the target object.

[0117] In an embodiment, the attribute calculation module 608 is further configured to:

[0118] for any candidate cluster center combination, calculate a line length of a line connecting two candidate cluster centers in the candidate cluster center combination, calculate a line angle between the two lines, and obtain a geometric attribute between the candidate cluster centers included in the candidate cluster center combination.

[0119] In an embodiment, the data acquisition module 602 is further configured to:

[0120] when the number of candidate cluster centers is greater than or equal to the number threshold, execute the step of grouping the candidate cluster centers to obtain a plurality of candidate cluster center combinations; and when the number of candidate cluster centers is less than the number threshold, acquire supplementary point cloud data for the prior position collected by the robot, fuse the target point cloud data and the supplementary point cloud data to obtain updated point cloud data, take the updated point cloud data as the target point cloud data, and return to execute the step of clustering the target point cloud data to obtain a plurality of candidate cluster centers corresponding to the target object.

[0121] In an embodiment, the geometric attribute includes the line length and the line angle; based on the respective geometric attributes of each candidate cluster center combination, the attribute calculation module 608 is further configured to:

[0122] acquire a reference length set and a reference angle corresponding to the target object; take a candidate cluster center combination that successfully matches the line length with the reference length set and successfully matches the line angle with the reference angle as an intermediate cluster center combination; and determine a target cluster center combination corresponding to the target object from the intermediate cluster center combinations.

[0123] In an embodiment, the position determination module 612 is further configured to:

[0124] The center position corresponding to the target object is calculated based on the geometric shape type corresponding to the target object and each target cluster center in the target cluster center combination, and the target position of the target object with respect to the robot is determined based on the robot pose corresponding to the robot and the center position.

[0125] Each module in the object position detection apparatus described above can be implemented wholly or partially by software, hardware, and combinations thereof. Each module described above can be embedded in or independent of a processor in the robot in hardware form, or can be stored in a memory in the robot in software form, so as to be called and executed by the processor to perform operations corresponding to each module.

[0126] In an exemplary embodiment, a robot is provided, and an internal structure diagram of the robot can be as shown in FIG. 7. The robot includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. The processor of the robot is configured to provide computing and control capabilities. The memory of the robot includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the robot is configured to exchange information between the processor and external devices. The communication interface of the robot is configured to perform wired or wireless communication with external terminals, and the wireless communication can be achieved through WIFI, mobile cellular network, near field communication (NFC), or other technologies. The computer program is executed by the processor to implement an object position detection method. The display unit of the robot is configured to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the robot can be a touch layer overlaid on the display screen, or can be a key, a trackball, or a touchpad arranged on the robot shell, or can be an external keyboard, touchpad, or mouse, etc.

[0127] Those skilled in the art can understand that the structure shown in FIG. 7 is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the robot to which the scheme of the present application is applied. Specifically, the robot can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0128] In an embodiment, a robot is also provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in each method embodiment described above.

[0129] In an embodiment, a computer readable storage medium is provided, having stored thereon a computer program which, when executed by a processor, implements the steps of any of the above method embodiments.

[0130] In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of any of the above method embodiments.

[0131] A person of ordinary skill in the art can understand that all or part of the processes in the above method embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium and can include the processes of the above method embodiments when executed. Any reference to a memory, database or other medium used in the embodiments provided in the present application can include at least one of a non-volatile memory and a volatile memory. The non-volatile memory can include a read-only memory (ROM), a magnetic tape, a floppy disk, a flash memory, an optical storage, a high-density embedded non-volatile memory, a resistive random access memory (ReRAM), a magnetoresistive random access memory (MRAM), a ferroelectric random access memory (FRAM), a phase change memory (PCM), a graphene memory, etc. The volatile memory can include a random access memory (RAM) or an external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms such as a static random access memory (SRAM) or a dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0132] Any combination of the technical features in the above-described embodiments can be made, and for the sake of brevity, not all possible combinations are described, however, it is to be understood that the application embraces all such possible combinations.

[0133] The above-described embodiments only express several implementation manners of the application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the patent scope of the application. It should be pointed out that, for ordinary skilled persons in the art, some modifications and improvements can be made without departing from the concept of the application, and these all belong to the protection scope of the application. Therefore, the patent protection scope of the application should be subject to the appended claims.

Claims

1. A method for detecting a position of an object, comprising: obtaining a prior position corresponding to a target object, and obtaining target point cloud data of the prior position collected by a robot; clustering the target point cloud data to obtain a plurality of candidate cluster centers corresponding to the target object; grouping each of the candidate cluster centers to obtain a plurality of candidate cluster center combinations; calculating geometric properties between each of the candidate cluster centers included in the candidate cluster center combinations; determining a target cluster center combination corresponding to the target object from the candidate cluster center combinations based on the geometric properties corresponding to each of the candidate cluster center combinations, respectively; determining a target position of the target object with respect to the robot based on the target cluster center combination.

2. The method of claim 1, wherein, The obtaining of the target point cloud data of the prior position collected by the robot comprises: obtaining initial point cloud data of the prior position collected by the robot; determining a relative position of the target object with respect to the robot based on a robot pose corresponding to the robot and the prior position; filtering the initial point cloud data based on the relative position to obtain the target point cloud data of the prior position collected by the robot.

3. The method of claim 1, wherein, The clustering of the target point cloud data to obtain a plurality of candidate cluster centers corresponding to the target object comprises: determining a reference cluster size based on a size of a positioning reference part of the target object; clustering the target point cloud data to obtain a plurality of initial point cloud clusters; filtering each of the initial point cloud clusters based on the reference cluster size to obtain a plurality of candidate point cloud clusters; taking a cluster center corresponding to each of the candidate point cloud clusters as a candidate cluster center corresponding to the target object.

4. The method of claim 1, wherein, The calculating of the geometric properties between each of the candidate cluster centers included in the candidate cluster center combinations comprises: for any one of the candidate cluster center combinations, calculating a line length of a line connecting each of the candidate cluster centers in the candidate cluster center combination with each other, and calculating an included angle between each of the lines to obtain the geometric properties between each of the candidate cluster centers included in the candidate cluster center combination.

5. The method of claim 1, wherein, The method further comprises: determining a minimum number of cluster centers required for positioning the target object according to a structural feature of the target object; grouping each of the candidate cluster centers based on the minimum number of cluster centers corresponding to the target object to obtain a plurality of the candidate cluster center combinations.

6. The method of claim 1, wherein, The method further comprises: when the number of the candidate cluster centers is greater than or equal to a number threshold, performing the grouping of each of the candidate cluster centers to obtain a plurality of candidate cluster center combinations; when the number of the candidate cluster centers is less than the number threshold, obtaining supplementary point cloud data of the prior position collected by the robot, fusing the target point cloud data and the supplementary point cloud data to obtain updated point cloud data, taking the updated point cloud data as target point cloud data, and returning to the clustering of the target point cloud data to obtain a plurality of candidate cluster centers corresponding to the target object.

7. The method of claim 6, wherein, The fusion of the target point cloud data and the supplementary point cloud data obtains updated point cloud data, and the updated point cloud data is taken as target point cloud data, comprising: acquiring the geometric size corresponding to the target object; determining the shortest adjustment path based on the geometric size corresponding to the target object; moving to another position along the shortest adjustment path; again collecting the target point cloud data for the prior position; projecting the target point cloud data collected again to the coordinate system corresponding to the target point cloud data to obtain the supplementary point cloud data; forming the updated point cloud data based on the supplementary point cloud data and the target point cloud data.

8. The method of claim 6, the fusion of the target point cloud data and the supplementary point cloud data obtains updated point cloud data, and the updated point cloud data is taken as target point cloud data, comprising: rotating a preset angle, and again collecting the target point cloud data for the prior position; projecting the target point cloud data collected again to the coordinate system corresponding to the target point cloud data to obtain the supplementary point cloud data; forming the updated point cloud data based on the supplementary point cloud data and the target point cloud data.

9. The method of claim 1, wherein, The geometric properties include line length and line angle; and the determination of the target cluster center combination corresponding to the target object in each candidate cluster center combination based on the geometric properties respectively corresponding to each candidate cluster center combination, comprises: acquiring a reference length set and a reference angle corresponding to the target object; taking a candidate cluster center combination, in which the line length is successfully matched with the reference length set and the line angle is also successfully matched with the reference angle, as an intermediate cluster center combination; determining the target cluster center combination corresponding to the target object from each intermediate cluster center combination.

10. The method of claim 9, wherein, The successful matching of the line length with the reference length set means that the difference between the line length and the reference length set is less than a preset threshold value; The successful matching of the line angle with the reference angle means that the difference between the line angle and the reference angle is less than a preset threshold value.

11. The method of claim 10, wherein, The determination of the target cluster center combination corresponding to the target object from each intermediate cluster center combination, comprises: fusing the difference between the line length and the corresponding reference length in each intermediate cluster center combination and the difference between the line angle successfully matched and the corresponding reference angle to obtain a comprehensive difference value respectively corresponding to each intermediate cluster center combination; taking the intermediate cluster center combination with the minimum comprehensive difference value as the target cluster center combination corresponding to the target object.

12. The method of claim 1, wherein, The determination of the target position of the target object for the robot based on the target cluster center combination, comprises: calculating a center position corresponding to the target object based on the geometric shape type corresponding to the target object and each target cluster center in the target cluster center combination; determining the target position of the target object for the robot based on the robot pose corresponding to the robot and the center position.

13. The method of claim 1, wherein, The method further comprises: The center position prediction model is obtained by supervised training of a plurality of object samples respectively corresponding to a plurality of cluster center combinations and each object sample respectively corresponding to a known target position of the robot; The target cluster center combination is input into the center position prediction model to obtain a target position of the target object corresponding to the robot.

14. The method of claim 1, wherein, The method further comprises: When the positioning reference part corresponding to the target object hinders driving, positions of all positioning reference parts corresponding to the target object are calculated based on the target cluster center combination corresponding to the target object; Based on the positions of all positioning reference parts corresponding to the target object and the target position, obstacle avoidance driving for the target object is performed to execute a work task for the target object.

15. The method of claim 1, wherein, The method further comprises: A noise covariance matrix is introduced through an extended Kalman filter; The target position is filtered; The filtered target position is output.

16. An object position detection device, comprising: a data acquisition module configured to acquire a prior position corresponding to a target object and acquire target point cloud data corresponding to the prior position collected by a robot; a point cloud clustering module configured to cluster the target point cloud data to obtain a plurality of candidate cluster centers corresponding to the target object; a center combination module configured to group each candidate cluster center to obtain a plurality of candidate cluster center combinations; an attribute calculation module configured to calculate geometric attributes between each candidate cluster center included in the candidate cluster center combination; a combination determination module configured to determine a target cluster center combination corresponding to the target object in each candidate cluster center combination based on the geometric attributes corresponding to each candidate cluster center combination; a position determination module configured to determine a target position of the target object corresponding to the robot based on the target cluster center combination.

17. The apparatus of claim 16, wherein, The data acquisition module is further configured to: acquire initial point cloud data corresponding to the prior position collected by the robot; determine a relative position of the target object corresponding to the robot based on a robot pose corresponding to the robot and the prior position; filter the initial point cloud data based on the relative position to obtain the target point cloud data corresponding to the prior position collected by the robot.

18. The apparatus of claim 16, wherein, The point cloud clustering module is further configured to: determine a reference clustering size based on a size of a positioning reference part of the target object; cluster the target point cloud data to obtain a plurality of initial point cloud clusters; filter each initial point cloud cluster based on the reference clustering size to obtain a plurality of candidate point cloud clusters; use cluster centers corresponding to the candidate point cloud clusters as candidate cluster centers corresponding to the target object.

19. A robot comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1 to 15.

20. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 15.

Images