Inspection method and handling apparatus
By generating a template point cloud and the first preset pose of the target object, acquiring and registering the point cloud, and calculating the translation vector and rotation matrix, the problem of low detection accuracy caused by noisy point clouds and asymmetric objects is solved, and high-precision detection of the target object pose is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- VISIONNAV ROBOTICS SHENZHEN LTD
- Filing Date
- 2022-11-10
- Publication Date
- 2026-08-04
AI Technical Summary
Existing technologies have low accuracy in detecting the pose of target objects, especially when dealing with noisy point clouds and asymmetric target objects, making it difficult to accurately obtain the pose of the target objects.
By generating a template point cloud and the first preset pose of the target object, acquiring the first point cloud and the second point cloud, registering them, and then using a solution function to calculate the translation vector and rotation matrix, the pose of the target object is transformed, reducing the influence of noisy point clouds and asymmetric objects.
It improves the accuracy of target object pose detection, and can accurately calculate translation vectors and rotation matrices in the presence of noisy point clouds or asymmetric objects, thus ensuring the accuracy of pose detection.
Smart Images
Figure CN115880359B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of testing technology, and in particular to a testing method and a handling device. Background Technology
[0002] Existing technologies for target object pose detection typically extract point clouds from the tray, such as using Random Sample Consensus (RANSAC) or clustering algorithms to extract the target object's pose. However, these methods are usually limited to detecting regular target objects and cannot detect target objects with a lot of noisy point clouds due to interference. Furthermore, they are also difficult to detect asymmetric target objects, resulting in low detection accuracy. Summary of the Invention
[0003] In view of this, the embodiments of this application provide a detection method and a handling device, which avoids the influence of noisy point clouds and the target object being an asymmetric object on the detection of the target pose of the target object, and improves the detection accuracy of the target pose of the target object.
[0004] The detection method of this application is applied to a handling equipment, which includes a detection device. The detection method includes generating a first point cloud based on a first preset pose of a target object and a preset template point cloud, wherein the template point cloud is point cloud information of a preset object that is the same type and specification as the target object; controlling the handling equipment to move according to the first preset pose so that the detection device can collect a second point cloud of the target object; registering the first point cloud of the first point cloud and the second point cloud of the second point cloud to obtain a second point cloud that matches the first point cloud; generating a third point cloud and a fourth point cloud based on multiple sets of matched first and second point clouds; calculating a translation vector and a rotation matrix based on the third point cloud, the fourth point cloud, and a preset solution function; and transforming the first preset pose according to the translation vector and the rotation matrix to obtain the target pose of the target object.
[0005] The handling device of this application includes a processor and a handling device, the handling device including a detection device. The processor is used to generate a first point cloud based on a first preset pose of a target object and a preset template point cloud, the template point cloud being point cloud information of a preset object with the same type and specifications as the target object; control the handling device to move according to the first preset pose so that the detection device can collect a second point cloud of the target object; register the first point cloud of the first point cloud and the second point cloud of the second point cloud to obtain a second point cloud matching the first point cloud; generate a third point cloud and a fourth point cloud based on multiple sets of matched first and second point clouds respectively; calculate a translation vector and a rotation matrix based on the third point cloud, the fourth point cloud and a preset solution function; and transform the first preset pose according to the translation vector and the rotation matrix to obtain the target pose of the target object.
[0006] The detection method and handling equipment of this application can transform the direct acquisition of the target pose of the target object into a first point cloud generated by first matching the first point cloud and the first preset pose of the target object, and the second point cloud acquired from the target object. The first and second point clouds are then used to calculate translation vectors and rotation matrices. The first preset pose is then converted into the target pose of the target object using the translation vectors and rotation matrices. Since the acquired second point cloud of the target object necessarily contains accurate second point clouds, by acquiring the second point cloud registered with the first point cloud, the matched first point cloud can be obtained. The second point cloud is used to generate a third and fourth point cloud set based on the matched first and second point clouds, respectively, and a preset solution function to accurately calculate the translation vector and rotation matrix. This allows the transformation relationship between the first preset pose and the target pose of the target object to be obtained. Then, the first preset pose is converted into the target pose of the target object according to the translation vector and rotation matrix, thus realizing the detection of the target pose of the target object. Even if there are noisy point clouds in the second point cloud set or the target object is an asymmetric object, it will not affect the accuracy of the translation vector and rotation matrix, thereby improving the accuracy of the target pose detection of the target object.
[0007] Additional aspects and advantages of the embodiments of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0008] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein:
[0009] Figure 1 This is a flowchart illustrating the detection method of some embodiments of this application;
[0010] Figure 2 This is a schematic diagram of the structure of a conveying device according to certain embodiments of this application;
[0011] Figure 3 This is a schematic diagram of a detection method according to certain embodiments of this application;
[0012] Figure 4 This is a flowchart illustrating the detection method of some embodiments of this application;
[0013] Figure 5 This is a flowchart illustrating the detection method of some embodiments of this application;
[0014] Figure 6 This is a flowchart illustrating the detection method of some embodiments of this application;
[0015] Figure 7 This is a flowchart illustrating the detection method of some embodiments of this application;
[0016] Figure 8 This is a flowchart illustrating the detection method of certain embodiments of this application; and
[0017] Figure 9 This is a schematic diagram illustrating the interaction between a computer-readable storage medium and a processor according to certain embodiments of this application. Detailed Implementation
[0018] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of this application, and should not be construed as limiting the embodiments of this application.
[0019] Please see Figure 1 and Figure 2 The detection method of this application includes:
[0020] Step 011: Based on the first preset pose of the target object and the preset template point cloud set, generate the first point cloud set. The template point cloud set is the point cloud information of a preset object that is the same type and specification as the target object.
[0021] Specifically, goods in a warehouse are generally stored on pallets. During inbound or outbound operations, the handling equipment 100 loads pallets to transport the goods carried on them, thus completing the inbound or outbound process. This application uses a pallet used to carry goods as an example for illustration, but it is understood that the target object of this application is not limited to the pallet.
[0022] It is understandable that the location of goods stored in a warehouse is determined after the goods are received. Generally, the handling equipment 100 receives goods in a preset posture, therefore the posture of the stored goods is also fixed. When the handling equipment 100 receives an inbound or outbound request, it can directly obtain the first preset pose of the target object to be inbound or outbound. The first preset pose includes the first position coordinates and first posture of the target object. Alternatively, the warehouse may be equipped with image acquisition equipment capable of acquiring images of the target object, thereby identifying the first preset pose of the target object through the image.
[0023] Then, the handling equipment 100 can also acquire template information of a preset object with the same specifications as the target object. Taking the target object as the target pallet and the preset object as the preset pallet as an example, the template information of the preset pallet includes the width of the pallet piers, the width of the holes, the length, and the height of the preset pallet. The preset pallet includes an upper pallet and a lower pallet. The template information of the preset pallet also includes the height of the upper pallet and the height of the lower pallet, etc. Based on the template information of the preset pallet, a template point cloud can be generated. For example, if the target object is a target pallet of specification A, then the template point cloud is the point cloud information generated based on the template information of the preset pallet of specification A, ensuring that the object corresponding to the template point cloud is completely consistent with the target object, thereby improving the accuracy of subsequent detection of the target object's posture.
[0024] The processor 20 of the handling device 100 can process the first preset pose and the template point cloud to generate the first point cloud. Specifically, the first preset pose is the pose of the target object at a preset position (such as the center position). Therefore, the center point cloud located at the center position in the template point cloud can be obtained. According to the mapping relationship between the center point cloud and the first preset pose, all point clouds in the template point cloud are transformed to generate the first point cloud.
[0025] Step 012: Control the movement of the handling equipment 100 according to the first preset pose so that the detection device 30 can collect the second point aggregation of the target object.
[0026] Specifically, please combine Figure 3The handling equipment 100 includes a processor 20, a detection device 30, and a cargo moving device 40. The detection device 30 is installed on the cargo moving device 40 (e.g., at the root of the forks of the cargo moving device 40). After the processor 20 acquires the first preset pose of the target object, it can control the movement of the cargo moving device 40 according to the first preset pose, so that the detection device 30 can collect the second point cloud of the target object P0. For example, the cargo moving device 40 is controlled to move according to the first position coordinates of the first preset pose, so that the cargo moving device 40 moves to the position where the target object P0 is located (e.g., directly in front of the target object P0). Then, the attitude of the cargo moving device 40 is adjusted according to the first attitude (e.g., the yaw angle of the cargo moving device 40), so that the cargo moving device 40 can face the target object P0, thereby making the target object P0 within the field of view of the detection device 30. At this time, the detection device 30 can be controlled to collect the second point cloud of the target object.
[0027] Of course, when the handling equipment 100 transports the target object P0 into the warehouse, there may be placement errors, resulting in a certain deviation between the first preset pose of the target object P0 after placement and the actual pose of the target object P0. After controlling the movement of the handling equipment 100 according to the first preset pose, the handling equipment 100 may not be able to be completely aligned with the target object P0. However, since the field of view of the detection device 30 is generally larger than that of the target object P0, it will basically ensure that the target object P0 is within the field of view of the detection device 30, ensuring that the detection device 30 can obtain the second point aggregation of the target object P0, thereby ensuring the accuracy of subsequent detection of the target pose of the target object P0.
[0028] Step 013: Register the first point cloud of the first point cloud set and the second point cloud of the second point cloud set to obtain the second point cloud that matches the first point cloud.
[0029] Specifically, after obtaining the first and second point clouds, they can be registered to find the second point cloud in the second point cloud that matches the first point cloud in the first point cloud. It can be understood that when two point clouds match, the distance between them is generally minimized. Therefore, the distance between the first point cloud in the first point cloud and the second point cloud in the matched second point cloud should be the smallest among all the distances between the first point cloud in the first point cloud and all the second point clouds in the second point cloud; that is, finding the second point cloud in the second point cloud that has the smallest distance to the first point cloud in the first point cloud. Alternatively, a preset nearest neighbor algorithm (K-Nearest Neighbor, KNN algorithm) can be used to find the second point cloud in the second point cloud that matches the first point cloud in the first point cloud. This is not a restriction.
[0030] Step 014: Generate the third and fourth point cloud sets based on the first and second point clouds matched from multiple sets.
[0031] Specifically, after finding the second point cloud in the second point cloud that matches the first point cloud in each first point cloud, the corresponding point sets, namely the third point cloud and the fourth point cloud, can be obtained.
[0032] For example, if we first generate a third point cloud set based on all the first point clouds in multiple sets of matched first and second point clouds, it can be understood that, in order to ensure the integrity of the point cloud, the registration is performed based on the first point cloud of the first point cloud set. That is, the first point cloud of each first point cloud set has a matching second point cloud. The first point cloud of the third point cloud set is generated one-to-one with the first point cloud of the first point cloud set. At this time, the third point cloud set is the same as the first point cloud set, and the first point cloud in the third point cloud set is the first point cloud of the first point cloud set.
[0033] Then, based on all the second point clouds in the multiple sets of matched first and second point clouds, a fourth point cloud set is generated. It can be understood that the fourth point cloud set is different from the second point cloud set. The fourth point cloud set is generated from multiple second point clouds selected from the second point cloud set that match the first point cloud in the first point cloud set. Since the first point cloud in each third point cloud set has a matching second point cloud in the fourth point cloud set, that is, the first point cloud in the third point cloud set and the second point cloud in the fourth point cloud set correspond one-to-one. Therefore, the number of second point clouds in the fourth point cloud set is the same as the number of first point clouds in the third point cloud set.
[0034] It is understandable that, due to the limitations of the sampling accuracy of the detection device 30, the number of second point clouds in the second point cloud set is generally less than the number of first point clouds in the first point cloud set. However, the number of second point clouds in the fourth point cloud set is the same as the number of first point clouds in the third point cloud set, and the third point cloud set is the same as the first point cloud set. Therefore, the number of second point clouds in the fourth point cloud set is generally greater than the number of second point clouds in the second point cloud set.
[0035] Furthermore, since different first point clouds within the first point cloud set may correspond to the same second point cloud within the second point cloud set, there are multiple second point clouds with the same pose in the fourth point cloud set.
[0036] Step 015: Based on the third point cloud, the fourth point cloud, and the preset solution function, calculate the translation vector and rotation matrix.
[0037] Specifically, given the corresponding point sets (i.e., the third point cloud and the fourth point cloud), the translation vector and rotation matrix can be calculated based on the third point cloud, the fourth point cloud, and the preset solution function. The preset solution function can calculate the translation vector and rotation matrix based on the first point cloud and the second point cloud matched in the third point cloud and the fourth point cloud.
[0038] The process of calculating the translation vector and rotation matrix is as follows:
[0039] First, based on the third point set, the fourth point set, and the solution function, the minimum value of the dependent variable F(t) of the solution function (as shown in formula (1)) is calculated. The solution function is shown in formula (1) below:
[0040]
[0041] Here, F(t) can be considered the loss function, where F(t) represents all p i After rotation and the corresponding q i The least squares distance between them, R is the rotation matrix, t is the translation matrix, p i For the first point cloud in the third point cloud set, q i p is the second point cloud in the fourth point cloud set. i and q i Matching, w i For each matching p i and q i The corresponding weight, n, is equal to the number of first point clouds within the third point cloud set.
[0042] Formula (2) is obtained by calculating the minimum value of F(t) from formula (1):
[0043]
[0044] Secondly, a weighted average is applied to the first point cloud where the third point cloud converges to obtain the first centroid. Then, a weighted average is applied to the second point cloud of the fourth point cloud to obtain the second centroid. The following formula (3) is used. In this formula, the first centroid and the second centroid are both points. The first centroid is the weighted average coordinate of the coordinates of all the first point clouds in the third point cloud set, and the second centroid is the weighted average coordinate of the coordinates of all the second point clouds in the fourth point cloud set.
[0045]
[0046] Then, based on the first mass center Second center of mass Find the minimum value and calculate the translation vector t. Substituting formula (3) into formula (2) yields the following formula (4):
[0047]
[0048] Then, based on the third point set, the fourth point set, the translation vector t, and the solution function, the objective function is calculated. For example, substituting formula (4) into formula (1) yields the following formulas (5) and (6).
[0049]
[0050]
[0051] Finally, the rotation matrix R is calculated based on the objective function, the first centroid, and the second centroid (as shown in formula (7)). If formula (6) is substituted into formula (5), the following formula (7) can be obtained.
[0052]
[0053] Where d is latitude and SO(d) represents the rotation group.
[0054] Step 016: Transform the first preset pose according to the translation vector t and the rotation matrix R to obtain the target pose of the target object.
[0055] Specifically, the rotation matrix and translation vector describe the motion process of a rigid body in three-dimensional space. By calculating the translation vector and rotation matrix of the third point set and the fourth point set, the mapping relationship between the first preset pose of the target object and the target pose of the target object can be obtained. Thus, the target pose of the target object can be calculated based on the first preset pose, translation vector and rotation matrix of the target object.
[0056] The detection method of this application can transform the direct acquisition of the target pose of the target object into a process of first generating a first point cloud using a template point cloud and the first preset pose of the target object, and then registering the first and second point clouds in the acquired second point cloud of the target object to calculate translation vectors and rotation matrices. The first preset pose is then converted into the target pose of the target object using the translation vectors and rotation matrices. Since the acquired second point cloud of the target object necessarily contains accurate second point clouds, by acquiring the second point cloud registered with the first point cloud, the matched first and second point clouds can be obtained. The point cloud is used to generate a third and fourth point cloud set based on the matched first and second point clouds, respectively, and a preset solution function to accurately calculate the translation vector and rotation matrix. This allows the transformation relationship between the first preset pose and the target pose of the target object to be obtained. Then, the first preset pose is converted into the target pose of the target object according to the translation vector and rotation matrix, thus realizing the detection of the target pose of the target object. Even if there are noisy point clouds in the second point cloud set or the target object is an asymmetric object, it will not affect the accuracy of the translation vector and rotation matrix, thereby improving the accuracy of the target pose detection of the target object.
[0057] Please see Figure 4 In some implementations, step 011: generating a first point cloud based on the first preset pose of the target object and a preset template point cloud includes:
[0058] Step 0111: Randomly sample the template point cloud set to obtain multiple target template point clouds;
[0059] Step 0112: Generate the first point cloud set based on the first preset pose and multiple target template point clouds.
[0060] Specifically, it can be understood that when a template point cloud set is established for the target object at a high resolution (such as a template point cloud set containing 10,000 or 20,000 point clouds), registering all point clouds within the template point cloud set would result in excessive computation and reduced detection speed. In actual target pose detection, it is necessary to quickly obtain the target pose. Therefore, the template point cloud set can be randomly sampled to obtain a preset number (such as 100 or 500) of target template point clouds. The larger the number of target template point clouds, the better it is for improving the accuracy of the target pose, but the slower the detection speed. Therefore, the preset number (i.e., the number of randomly sampled target template point clouds) can be set according to the actual detection speed requirements to maximize the accuracy of target pose detection while ensuring detection speed.
[0061] After sampling the target template point cloud, the target template point cloud can be transformed according to the first preset pose of the target object to obtain multiple first point clouds, which can be used as the first point cloud set.
[0062] It is understandable that, taking the first preset pose of the target object as the center pose of the target object as an example, a mapping relationship can be determined based on the first preset pose and the center point cloud of the template point cloud set; then, based on this mapping relationship, all target template point clouds are transformed to obtain multiple first point clouds, which serve as the first point cloud set. For example, the first preset pose is (x0, y0, z0, th0), where x0, y0, and z0 are the three-dimensional position coordinates of the handling equipment 100, th0 is the attitude (such as yaw angle) of the handling equipment 100, and the center point cloud of the template point cloud set is (0, 0, 0, 0). Let the target template point cloud be (x1, y1, z1, th1), and the first point cloud be (x2, y2, z2, th2), then the mapping relationship between the target template point cloud and the first point cloud is x 2＝ x 1+ x0, y 2＝ y1+y0, z 2＝ z 1+ z0,th 2＝ th 1+ th0. Thus, based on the mapping relationship, the target template point cloud can be converted into the first point cloud, which serves as the first point cloud set.
[0063] Please see Figure 5 In some embodiments, before transforming the first preset pose according to the translation vector and rotation matrix, the detection method further includes:
[0064] Step 017: Transform the fourth point cloud according to the translation vector and rotation matrix to obtain the fifth point cloud;
[0065] Step 018: Calculate the variance of the distance between the first point cloud of the matched third point cloud and the third point cloud of the fifth point cloud in the third point cloud and the fifth point cloud.
[0066] If the variance is less than the preset threshold, proceed to step 016.
[0067] Specifically, to ensure the accuracy of the translation vector and rotation matrix, they need to be verified. It can be understood that after transforming the fourth point cloud set according to the translation vector and rotation matrix, the fifth point cloud set can be obtained. The third point cloud in the fifth point cloud set is obtained by transforming the corresponding second point cloud in the fourth point cloud set using the translation vector t and rotation matrix R. Therefore, the third point cloud in the fifth point cloud set and the second point cloud in the fourth point cloud set are in one-to-one correspondence. Furthermore, the second point cloud in the fourth point cloud set and the first point cloud in the third point cloud set are also in one-to-one correspondence. Therefore, the first point cloud of the third point cloud set corresponding to each third point cloud in the fifth point cloud set can be determined.
[0068] It is understandable that, assuming the translation vector and rotation matrix are accurate, the distance between the third point cloud in the fifth point cloud set and the matching first point cloud in the third point cloud set should be small. By calculating the variance of the distance between the first point cloud in the matching third point cloud set and the third point cloud in the fifth point cloud set, the accuracy of the translation vector and rotation matrix can be accurately assessed. If the variance is less than or equal to a preset threshold (the preset threshold can be determined based on the actual situation and is an empirical value), it can be determined that the translation vector and rotation matrix are accurate. At this point, the first preset pose can be transformed according to the translation vector and rotation matrix to obtain the target pose of the target object.
[0069] If the variance is greater than the preset threshold, it indicates that the accuracy of the translation vector and rotation matrix obtained from the first and second point sets is low. This means that when randomly sampling the template point set, the first point cloud in the first point set obtained after transforming the target template point cloud through the first preset pose may have a poor matching degree with the second point cloud in the second point set (e.g., the distance between the first point cloud in the first point set and the matching second point cloud in the second point set is too large). This may be due to the low accuracy or even absence of point cloud acquisition in some areas of the target object when the detection device 30 acquires the second point set of the target object.
[0070] Therefore, it is necessary to resample the template point cloud set to obtain multiple target template point clouds; and based on the first preset pose and multiple target template point clouds, the first point cloud set is regenerated. Then, based on the regenerated first and second point cloud sets, the translation vector t and rotation matrix R are calculated. If the variance is still greater than the preset threshold, the template point cloud set is resampled randomly again to obtain multiple target template point clouds, and the first point cloud set is regenerated based on the first preset pose and multiple target template point clouds. This process is iterated until a translation vector and rotation matrix with a variance less than or equal to the preset threshold are found. Thus, by continuously iterating the translation vector and rotation matrix based on variance evaluation, the accuracy of the translation vector and rotation matrix is ensured.
[0071] Please see Figure 6 In some implementations, before calculating the translation vector and rotation matrix based on the third point set, the fourth point set, and a preset solution function, the detection method further includes:
[0072] Step 019: Determine the first and second point clouds whose target distance is greater than a preset distance as the target corresponding point set, and the target distance is the distance between the matched first and second point clouds;
[0073] Step 020: Delete the target corresponding point set in the third point cloud and the fourth point cloud.
[0074] Specifically, after obtaining multiple sets of matching first and second point clouds, the accuracy of the second point cloud matching the first point cloud may be low due to missing points or noisy points. It's understandable that when the first and second point clouds match accurately, their distance is generally close. If the distance between the matched first and second point clouds is large, it indicates that when the detection device 30 acquires the second point cloud set of the target object, it failed to acquire the point cloud at the position corresponding to the first point cloud within the target object, or that it was replaced by noisy points. Inaccurate matching of the first and second point clouds is obviously detrimental to the subsequent calculation of translation vectors and rotation matrices.
[0075] Therefore, the distance between the first and second point clouds in each matching pair can be calculated first as the target distance. Then, the first and second point clouds with a target distance greater than a preset distance are found and identified as the target corresponding point set. Thus, the target object point set is deleted from the third and fourth point cloud sets. Subsequently, when improving the translation vector and rotation matrix based on the third and fourth point cloud sets with the target object point set deleted, it is beneficial to improve the accuracy of the translation vector and rotation matrix, reduce the number of iterations of the translation vector and rotation matrix, and thus improve the detection efficiency of the target pose.
[0076] Please see Figure 7 In some implementations, the detection method further includes:
[0077] Step 021: Obtain the second point cloud within the preset coordinate range of the second point cloud set to generate the sixth point cloud set. The preset coordinate range is determined based on the first preset pose and the preset size of the target object.
[0078] Step 013: Register the first point cloud of the first point cloud set and the second point cloud of the second point cloud set, including:
[0079] Step 0131: Register the first point cloud of the first point cloud set and the second point cloud of the sixth point cloud set.
[0080] Specifically, when the detection device 30 acquires the second point aggregation of the target object, the field of view of the detection device 30 may not only contain the target object, but also other objects. For example... Figure 3 As shown, apart from the target object P0 being completely within the field of view of the detection device 30, objects P1 and P2 are partially within the field of view of the detection device 30. After the cargo moving device 40 moves according to the first preset pose, it will generally face the target object directly, or there will be a small pose deviation. That is to say, the target object is within a preset area of the field of view of the detection device 30. The preset area can be determined according to the size of the target object. The preset area is generally slightly larger than the size of the target object to ensure that the preset area can contain the entire target object.
[0081] Therefore, after obtaining the second point cloud set of the target object, in order to reduce the computational load of registration, point cloud information of a preset area within the field of view of the detection device 30 can be obtained, so that the second point cloud set basically only contains the second point cloud related to the target object. When obtaining the point cloud information of the preset area, a preset coordinate range can be determined according to the preset area and the first preset pose, so as to quickly obtain the second point cloud within the preset coordinate range from the second point cloud set to obtain the sixth point cloud set. When registering with the first point cloud of the first point cloud set in the subsequent process, it is only necessary to register the first point cloud of the first point cloud set and the second point cloud of the sixth point cloud set to obtain the second point cloud that matches the first point cloud, thereby reducing the computational load of registration while ensuring the accuracy of registration.
[0082] Please see Figure 8 In some implementations, the detection method further includes:
[0083] Step 022: Filter the sixth point cloud based on the preset filtering radius to generate the seventh point cloud;
[0084] Step 013: Register the first point cloud of the first point cloud set and the second point cloud of the second point cloud set, including:
[0085] Step 0132: Register the first point cloud of the first point cloud set and the second point cloud of the seventh point cloud set.
[0086] Specifically, the second point cloud collected by the detection device 30 may contain a large number of noisy point clouds. Noisy point clouds are generally isolated, while normal point clouds are generally dense because the target object is a single entity. Based on the characteristics of the noisy point clouds, the sixth point cloud or the second point cloud can be filtered based on a preset filtering radius to remove isolated noisy point clouds from the sixth point cloud or the second point cloud, thereby generating a seventh point cloud. This allows for the registration of the first point cloud of the first point cloud and the second point cloud of the seventh point cloud, improving the matching accuracy of the first and second point clouds.
[0087] Taking filtering the sixth point cloud as an example, during filtering, the number of second point clouds in the sixth point cloud within a preset filtering radius can be obtained. If this number is less than a preset threshold (such as 10, 20, etc.), it can be identified as a noise point cloud. Thus, the second point clouds in the sixth point cloud within the preset filtering radius that have a number less than the preset threshold (such as 10, 20, etc.) are filtered out to generate the seventh point cloud, removing the influence of noise point clouds on registration.
[0088] Please refer to it again. Figure 2 The conveying device 100 of this application includes a processor 20 and a conveying device 100. The conveying device 100 includes a detection device 30. The processor 20 is used to generate a first point cloud based on a first preset pose of the target object and a preset template point cloud; control the conveying device 100 to move according to the first preset pose so that the detection device 30 can collect a second point cloud of the target object; register the first point cloud of the first point cloud and the second point cloud of the second point cloud to obtain a second point cloud that matches the first point cloud; generate a third point cloud and a fourth point cloud based on multiple sets of matched first point clouds and second point clouds respectively; calculate a translation vector and a rotation matrix based on the third point cloud, the fourth point cloud and a preset solution function; and transform the first preset pose according to the translation vector and the rotation matrix to obtain the target pose of the target object.
[0089] Optionally, the processor 20 is used to execute the detection method of any of the above embodiments, which will not be described in detail here for the sake of brevity.
[0090] Among them, the handling equipment 100 can be a mobile device such as an automated guided vehicle, a clamping vehicle, a tractor, a forklift, a reach stacker, or a warehouse handling equipment 100.
[0091] Automated Guided Vehicles (AGVs) are transport vehicles equipped with electromagnetic or optical automatic navigation devices, capable of traveling along a predetermined navigation path, and possessing safety protection and various transfer functions. In industrial applications, these driverless transport vehicles are powered by rechargeable batteries. Their movement and behavior are typically controlled by a computer, or their path is established using electromagnetic tracks attached to the floor. The AGV moves and operates based on the signals transmitted through these tracks.
[0092] Please see Figure 9This application also provides a computer-readable storage medium 300 storing a computer program 310. When the computer program 310 is executed by the processor 20, it implements the steps of the detection method of any of the above embodiments. For the sake of brevity, these steps will not be repeated here.
[0093] It is understood that a computer program 310 includes computer program code. Computer program code can be in the form of source code, object code, executable files, or certain intermediate forms. Computer-readable storage media can be non-volatile computer-readable storage media such as any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc.
[0094] In the description of this specification, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with the described embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0095] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.
[0096] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.
Claims
1. A method of detection, characterized in that, Applied to handling equipment, the handling equipment includes a detection device, and the detection method includes: Based on the first preset pose of the target object and the preset template point cloud set, a first point cloud set is generated, wherein the template point cloud set is the point cloud information of a preset object that is the same as the target object in terms of type and specifications; The transport device is moved according to the first preset pose so that the detection device can collect the second point cluster of the target object; The first point cloud of the first point cloud set and the second point cloud of the second point cloud set are registered to obtain the second point cloud that matches the first point cloud; A third point cloud set and a fourth point cloud set are generated based on multiple sets of matched first and second point clouds, respectively. Based on the third point set, the fourth point set, and the preset solution function, calculate the translation vector and rotation matrix; The first preset pose is transformed according to the translation vector and the rotation matrix to obtain the target pose of the target object.
2. The detection method according to claim 1, characterized in that, The process of generating a first point cloud based on a first preset pose of the target object and a preset template point cloud includes: Random sampling is performed on the template point cloud set to obtain multiple target template point clouds; Based on the first preset pose and multiple target template point clouds, the first point cloud set is generated.
3. The detection method according to claim 2, characterized in that, Before transforming the first preset pose according to the translation vector and the rotation matrix, the method further includes: The fourth point set is transformed according to the translation vector and the rotation matrix to obtain the fifth point set; Calculate the variance of the distance between the first point cloud of the third point cloud set and the third point cloud set of the fifth point cloud set. If the variance is less than a preset threshold, proceed to the step of transforming the first preset pose according to the translation vector and the rotation matrix.
4. The detection method according to claim 3, characterized in that, Before transforming the first preset pose according to the translation vector and the rotation matrix, the method further includes: If the variance is greater than the preset threshold, the process of randomly sampling the template point cloud set to obtain multiple target template point clouds is repeated.
5. The method of claim 1, wherein, The calculation of the translation vector and rotation matrix based on the third point cloud, the fourth point cloud, and a preset solution function includes: Based on the third point cloud, the fourth point cloud, and the solution function, the minimum value of the dependent variable of the solution function is calculated. A weighted average is taken of the first point cloud of the third point cloud set to obtain the first centroid, and a weighted average is taken of the second point cloud of the fourth point cloud set to obtain the second centroid; The translation vector is calculated based on the first centroid, the second centroid, and the minimum value; The objective function is calculated based on the third point cloud, the fourth point cloud, the translation vector, and the solution function. The rotation matrix is calculated based on the objective function, the first centroid, and the second centroid.
6. The method of claim 1, wherein, Before calculating the translation vector and rotation matrix based on the third point set, the fourth point set, and the preset solution function, the detection method further includes: The first point cloud and the second point cloud that match the target distance are determined to be the target corresponding point set, wherein the target distance is the distance between the first point cloud and the second point cloud that match; Delete the target corresponding point set from the third point cloud set and the fourth point cloud set.
7. The method of claim 1, wherein, The detection method further includes: The second point cloud within a preset coordinate range of the second point cloud set is obtained to generate the sixth point cloud set. The preset coordinate range is determined based on the first preset pose and the preset size of the target object. The registration of the first point cloud of the first point cloud set and the second point cloud of the second point cloud set includes: The first point cloud of the first point cloud set and the second point cloud of the sixth point cloud set are registered.
8. The detection method according to claim 7, characterized in that, Also includes: The sixth point cloud is filtered based on a preset filtering radius to generate the seventh point cloud; The registration of the first point cloud of the first point cloud set and the second point cloud of the second point cloud set includes: The first point cloud of the first point cloud set and the second point cloud of the seventh point cloud set are registered.
9. The detection method according to claim 8, characterized in that, The step of filtering the sixth point cloud based on a preset filtering radius to generate the seventh point cloud includes: Obtain the number of second point clouds located within the preset filtering radius of the second point cloud in the sixth point cloud set; The second point cloud of the sixth point cloud set whose quantity is less than a preset threshold is filtered out to generate the seventh point cloud set.
10. A handling apparatus, characterized by It includes a processor and a handling device, the handling device including a detection device, the processor being used to generate a first point cloud based on a first preset pose of the target object and a preset template point cloud, the template point cloud being point cloud information of a preset object that is the same type and specification as the target object; The transport device is moved according to the first preset pose so that the detection device can collect the second point cloud of the target object; the first point cloud of the first point cloud and the second point cloud of the second point cloud are registered to obtain the second point cloud that matches the first point cloud; A third point cloud set and a fourth point cloud set are generated based on multiple sets of matched first and second point clouds, respectively; based on the third point cloud set, the fourth point cloud set, and a preset solution function, the translation vector and rotation matrix are calculated; The first preset pose is transformed according to the translation vector and the rotation matrix to obtain the target pose of the target object.