Object point cloud overlap identification method, device, equipment, robot and storage medium

Abstract

The application relates to the technical field of robot vision, and provides an object point cloud overlap identification method, device, equipment, robot and storage medium, the method comprises the following steps: obtaining a first object point cloud of a first object and a second object point cloud of a second object; the point number of the first object point cloud is less than or equal to the point number of the second object point cloud; determining the identification distance between each point in the first object point cloud and all points in the second object point cloud; comparing each identification distance with a first preset distance respectively to determine the number of overlapping points according to a first comparison result; obtaining an overlap rate according to the number of overlapping points and the point number of the first object point cloud; comparing the overlap rate with a first preset threshold to determine an overlap identification result according to a second comparison result. The application provides a method for judging whether objects overlap according to a point cloud overlap rate, improves the accuracy of object overlap judgment, and is helpful for a robot to understand object placement in a complex environment and improves the intelligent degree of the robot.

Description

Technical Field

[0001] This application relates to the field of robot vision technology, and in particular to methods, apparatus, devices, robots, and storage media for recognizing overlapping point clouds of objects. Background Technology

[0002] With the development of science and technology, robots have gradually entered all areas of human society. Autonomous movement and localization of robots require the use of environmental maps. Traditional robot environmental maps only contain geometric information, thus limiting robots to simple movements and hindering their ability to understand their environment and accurately identify overlapping objects in complex semantic grid map scenes. Summary of the Invention

[0003] This application aims to address at least one of the technical problems existing in related technologies. To this end, this application proposes a method for recognizing overlapping point clouds of objects, which improves the accuracy of object overlap judgment, helps robots understand the placement of objects in complex environments, and enhances the intelligence level of robots.

[0004] This application also proposes an object point cloud overlap recognition device, electronic device, robot, and storage medium.

[0005] The method for overlapping object point clouds according to an embodiment of the first aspect of this application includes: acquiring a first object point cloud of a first object and a second object point cloud of a second object; wherein the number of points in the first object point cloud is less than or equal to the number of points in the second object point cloud; determining the recognition distance between each point in the first object point cloud and the nearest point of all points in the second object point cloud; comparing each recognition distance with a first preset distance to determine the number of overlapping points based on a first comparison result; obtaining an overlap rate based on the number of overlapping points and the number of points in the first object point cloud; and comparing the overlap rate with a first preset threshold to determine the overlapping recognition result of the first object and the second object based on a second comparison result.

[0006] The object point cloud overlap recognition method according to the embodiments of this application determines the overlap rate of two object point clouds based on the distance between them, and judges whether the object point clouds overlap based on the overlap rate, thereby improving the accuracy of object overlap judgment; it helps robots understand the placement of objects in complex environments and improves the intelligence level of robots.

[0007] According to one embodiment of this application, the overlap rate is compared with a first preset threshold to determine the overlap recognition result of the first object and the second object based on the second comparison result, including: when the overlap rate is greater than or equal to the first preset threshold, it is determined that the first object and the second object overlap; when the overlap rate is less than the first preset threshold, it is determined that the first object and the second object do not overlap.

[0008] According to one embodiment of this application, after determining that the first object and the second object overlap, the method includes: when the number of points in the point cloud of the first object is less than the number of points in the point cloud of the second object, the point cloud of the first object is marked for deletion; when the number of points in the point cloud of the first object is equal to the number of points in the point cloud of the second object, either the point cloud of the first object or the point cloud of the second object is marked for deletion.

[0009] According to one embodiment of this application, each identification distance is compared with a first preset distance to determine the number of overlapping points based on the first comparison result, including: comparing each identification distance with the first preset distance to obtain a first comparison result; wherein the first comparison result includes identification distances that are less than or equal to the first preset distance; based on the first comparison result, determining the number of all identification distances that are less than or equal to the first preset distance, and using the number as the number of overlapping points.

[0010] According to one embodiment of this application, obtaining a first object point cloud of a first object and a second object point cloud of a second object includes: extracting object point clouds based on a semantic raster map to obtain an object point cloud set; obtaining two object point clouds that have not been marked for deletion from the object point cloud set; comparing the number of points in the two object point clouds that have not been marked for deletion; and determining the first object point cloud and the second object point cloud from the two object point clouds that have not been marked for deletion based on the comparison result.

[0011] According to one embodiment of this application, object point cloud extraction based on a semantic raster map to obtain an object point cloud set includes: extracting multiple semantic point clouds from the semantic raster map; performing clustering processing on the multiple semantic point clouds to obtain multiple object point clouds; and determining the object point cloud set based on the multiple object point clouds.

[0012] According to one embodiment of this application, determining an object point cloud set based on multiple object point clouds includes: using a point cloud classification model to identify the correctly classified object point cloud among the multiple object point clouds as the target object point cloud; and determining the object point cloud set based on the target object point cloud.

[0013] An object point cloud overlap recognition device according to a second aspect embodiment of this application includes: an acquisition module for acquiring a first object point cloud of a first object and a second object point cloud of a second object; wherein the number of points in the first object point cloud is less than or equal to the number of points in the second object point cloud; a distance determination module for determining the recognition distance between each point in the first object point cloud and the nearest point of all points in the second object point cloud; a distance comparison module for comparing each recognition distance with a first preset distance to determine the number of overlapping points based on a first comparison result; an overlap rate determination module for obtaining an overlap rate based on the number of overlapping points and the number of points in the first object point cloud; and a threshold comparison module for comparing the overlap rate with a first preset threshold to determine the overlap recognition result of the first object and the second object based on a second comparison result.

[0014] An electronic device according to a third aspect of this application includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements any of the object point cloud overlap recognition methods described above.

[0015] The robot according to a fourth aspect of this application includes a control unit; the control unit is used to perform any of the above-described object point cloud overlap recognition methods.

[0016] According to a fifth aspect embodiment of the present application, a non-transitory computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements any of the object point cloud overlap recognition methods described above.

[0017] The above-described one or more technical solutions in the embodiments of this application have at least one of the following technical effects:

[0018] The overlap rate of two object point clouds is determined by the distance between them, and the overlap rate is used to determine whether the object point clouds overlap, thus improving the accuracy of object overlap judgment.

[0019] It helps robots understand the placement of objects in complex environments, better comprehend environmental information at the semantic level, improve the intelligence level of robots, and provide information support for robots' complex behavioral decisions, human-computer interaction and other tasks.

[0020] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is one of the flowcharts illustrating the overlapping recognition method for object point clouds provided in the embodiments of this application;

[0023] Figure 2 This is a second schematic flowchart of the overlapping recognition method for object point clouds provided in the embodiments of this application;

[0024] Figure 3 This is a schematic diagram of the structure of the object point cloud overlap recognition device provided in the embodiments of this application;

[0025] Figure 4This is a schematic diagram of the physical structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0026] The embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but should not be used to limit the scope of this application.

[0027] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0028] In complex semantic raster map scenarios, objects may overlap, which can interfere with the robot and prevent it from accurately recognizing the image.

[0029] Based on this, embodiments of this application provide a method for overlapping object point clouds. Please refer to [link to relevant documentation]. Figure 1 , Figure 1 This is one of the flowcharts illustrating the overlapping recognition method for object point clouds provided in this application embodiment. In this embodiment, the overlapping recognition method for object point clouds may include steps 110 to 150, each step being as follows:

[0030] Step 110: Obtain the first object point cloud of the first object and the second object point cloud of the second object.

[0031] Point cloud data is a three-dimensional data set composed of a large number of points, each of which contains spatial location information and possible other attributes.

[0032] For example, each point in the point cloud data may include spatial location information and color information. The spatial location coordinates may be XYZ coordinate values ​​in the world coordinate system, and the color information may be RGB color values.

[0033] In some embodiments, point cloud data may also include other attributes, such as normal vectors, texture coordinates, and intensity values, to describe the shape and properties of the points.

[0034] There are several ways to acquire point cloud data, such as through sensor scanning or computer model conversion.

[0035] Sensor scanning: Sensors such as LiDAR or depth cameras can be used to acquire point cloud data by scanning the surrounding environment. LiDAR calculates distances by emitting laser beams and measuring the time it takes for them to reflect back, thus obtaining point cloud data. Depth cameras use infrared radiation and image sensors to measure the distances of objects in the scene, thereby generating point cloud data.

[0036] Computer model conversion: Using computer vision and image processing techniques, two-dimensional images or three-dimensional models can be converted into point cloud data. For example, point cloud data can be recovered from images through structured light, stereo vision, or multi-view image capture.

[0037] Object point clouds are a type of point cloud data, and the point cloud data in each object point cloud are of the same category. In this embodiment, the number of points in the first object point cloud is less than or equal to the number of points in the second object point cloud.

[0038] Step 120: Determine the recognition distance between each point in the first object point cloud and the nearest point in the second object point cloud.

[0039] The first object point cloud is an object point cloud with fewer points, while the second object point cloud is an object point cloud with more points. Calculating the distance between the two object point clouds based on the first object point cloud can save computational resources.

[0040] For example, the number of points in the first object point cloud is determined, and then the distance between each point in the first object point cloud and the nearest point in the second object point cloud is used as the recognition distance of that point.

[0041] Step 130: Compare each recognition distance with the first preset distance to determine the number of overlapping points based on the first comparison result.

[0042] The first preset distance can be set according to the actual situation. It should be noted that the larger the first preset distance, the more lenient the standard for judging object overlap; the smaller the first preset distance, the more stringent the standard for judging object overlap.

[0043] It should be noted that there is no order restriction between steps 120 and 130. In some embodiments, the recognition distance between each point in the first object point cloud and the nearest point of all points in the second object point cloud can be determined. After summarizing the recognition distances of all points in the first object point cloud, each recognition distance in the summarized data is compared with the first preset distance.

[0044] In other embodiments, the identification distance between a point in the first object point cloud and the nearest point in all points in the second object point cloud can also be determined, without having to aggregate all identification distances, and the identification distance can be compared with a first preset distance.

[0045] Step 140: Obtain the overlap rate based on the number of overlapping points and the number of points in the first object's point cloud.

[0046] The overlap rate is related to the number of overlapping points and the number of points in the first object's point cloud. Specifically, the overlap rate is calculated by dividing the number of overlapping points by the number of points in the first object's point cloud.

[0047] For example, assuming the first object point cloud is A and the number of points in the first object point cloud is N, and the second object point cloud is B, calculate the recognition distance between each point in the first object point cloud and the second object point cloud B, and count the number of overlapping points in the first object point cloud A as K, then the overlap rate is K / N.

[0048] Step 150: Compare the overlap rate with the first preset threshold to determine the overlap recognition result of the first object and the second object based on the second comparison result.

[0049] The first preset threshold can be set according to the actual situation. The larger the first preset threshold, the more lenient the standard for judging object overlap; the smaller the first preset distance, the more stringent the standard for judging object overlap.

[0050] The overlap rate is compared with the first preset threshold, and the overlap recognition result can be obtained based on the second comparison result.

[0051] In some embodiments, the first preset threshold value ranges from 0 to 1. For example, the first preset threshold value can be set to 0.4, 0.5, 0.6, etc.

[0052] This embodiment provides a method for overlapping object point clouds. It acquires a first object point cloud with a smaller number of points and an object point cloud with a larger number of points, and determines the recognition distance between each point in the first object point cloud and the nearest point in the second object point cloud. Each recognition distance is compared with a first preset distance to determine the number of overlapping points based on the first comparison result. The overlap rate is obtained based on the number of overlapping points and the number of points in the first object point cloud. The overlap rate is compared with a first preset threshold to determine the overlapping recognition result of the first and second objects based on the second comparison result. Through this method, this embodiment can determine the overlap rate of two object point clouds based on the distance between them, and judge whether the object point clouds overlap based on the overlap rate, thus improving the accuracy of object overlap judgment. This helps robots understand the placement of objects in complex environments and improves the robot's intelligence level.

[0053] Based on the above embodiments, the step of comparing the overlap rate with a first preset threshold to determine the overlap recognition result of the first object and the second object according to the second comparison result may specifically include:

[0054] Step 151: When the overlap rate is greater than or equal to the first preset threshold, it is determined that the first object and the second object overlap.

[0055] Step 152: When the overlap rate is less than the first preset threshold, it is determined that the first object and the second object do not overlap.

[0056] In this embodiment, the numerical comparison between the overlap rate and the first preset threshold can include three cases: greater than, equal to, and less than.

[0057] When the overlap rate is greater than or equal to the first preset threshold, it is determined that the first object point cloud and the second object point cloud overlap, and the first object represented by the first object point cloud and the second object represented by the second object point cloud are considered to be overlapping.

[0058] When the overlap rate is less than the first preset threshold, it is determined that the first object point cloud and the second object point cloud do not overlap, and the first object represented by the first object point cloud and the second object represented by the second object point cloud are considered to be non-overlapping.

[0059] This embodiment uses the overlap rate to determine whether object point clouds overlap, thus improving the accuracy of object overlap determination.

[0060] Based on the above embodiments, the step of determining the overlap of the first object and the second object may specifically include:

[0061] Step 161: When the number of points in the first object point cloud is less than the number of points in the second object point cloud, the first object point cloud is marked for deletion.

[0062] Step 162: When the number of points in the first object point cloud is equal to the number of points in the second object point cloud, either the first object point cloud or the second object point cloud is marked for deletion.

[0063] In this embodiment, when it is considered that the first object represented by the first object point cloud and the second object represented by the second object point cloud are overlapping, further processing can be performed on the overlapping objects, and the object point cloud corresponding to one of the two overlapping objects can be marked and deleted.

[0064] Since objects seen more comprehensively by the camera are less likely to be misidentified, and a larger number of points in an object's point cloud indicates a more comprehensive view of the object, when there is a high overlap, object point clouds with fewer points can be prioritized for deletion.

[0065] For example, the number of points in the first object point cloud can be less than or equal to the number of points in the second object point cloud; therefore, when the number of points in the first object point cloud is less than the number of points in the second object point cloud, the first object point cloud with the smaller number of points can be directly marked for deletion.

[0066] When the number of points in the first object point cloud is equal to the number of points in the second object point cloud, either the first object point cloud or the second object point cloud can be selected for deletion; or, considering the influence of other parameters, either the first object point cloud or the second object point cloud can be deleted.

[0067] In this embodiment, after determining that objects overlap, a method for processing the overlapping objects is provided, namely, deleting one of them; furthermore, a method for determining the point cloud of the object to be deleted is given, deleting the point cloud of the object with the smaller number of points based on the overlap rate. This approach enables the robot to understand the placement of objects in complex environments, improving object recognition accuracy.

[0068] Based on the above embodiments, the step of comparing each identification distance with a first preset distance to determine the number of overlapping points according to the first comparison result may specifically include:

[0069] Step 131: Compare each recognition distance with the first preset distance to obtain the first comparison result. Step 132: Based on the first comparison result, determine the number of recognition distances that are less than or equal to the first preset distance, and use this number as the number of overlapping points.

[0070] The first comparison result includes recognition distances that are less than or equal to a first preset distance. In this embodiment, the number of overlapping points is determined by comparing the recognition distance and the first preset distance. For example, each recognition distance is compared with the first preset distance, and points whose recognition distance is greater than or equal to the first preset distance are taken as overlapping points. The total number of overlapping points is the sum of all overlapping points.

[0071] Based on the above embodiments, the steps for obtaining the first object point cloud of the first object and the second object point cloud of the second object may specifically include:

[0072] Step 111: Extract object point clouds based on semantic raster maps to obtain object point cloud sets.

[0073] Step 112: Obtain two object point clouds that have not been marked for deletion from the object point cloud set.

[0074] Step 113: Compare the point counts of the two unmarked object point clouds.

[0075] Step 114: Based on the comparison results, determine the first object point cloud and the second object point cloud from the two object point clouds that were not marked for deletion.

[0076] In this embodiment, the first object point cloud and the second object point cloud can be obtained from a semantic raster map. A semantic raster map refers to a 3D semantic raster, which can use multiple cubic raster cells to represent real-world shape information. Each raster cell has unique raster information, including semantic information and an index value.

[0077] Object point cloud extraction from semantic raster maps yields a set of object point clouds containing multiple object point clouds. Then, two object point clouds that have not been marked for deletion are extracted from the object point cloud set and their values ​​are compared. Based on the comparison results, the object point cloud with the smaller number of points is selected as the first object point cloud, and the object point cloud with the larger number of points is selected as the second object point cloud, thus completing the extraction of the first and second object point clouds.

[0078] For example, the object point cloud overlap recognition method of this embodiment can be applied to semantic navigation of indoor robots. The semantic grid map can be stored in the indoor robot, and the robot can extract environmental information about the surrounding environment based on the semantic grid map. Furthermore, combining the above-described object point cloud overlap recognition method can improve the object recognition rate on the semantic grid map, helping the robot understand surrounding objects and thus perform more complex tasks.

[0079] Based on the above embodiments, the step of extracting object point clouds from semantic raster maps to obtain an object point cloud set may specifically include:

[0080] Step 1111: Extract multiple semantic point clouds from the semantic raster map.

[0081] Step 1112: Cluster the multiple semantic point clouds to obtain multiple object point clouds.

[0082] Step 1113: Determine the object point cloud set based on multiple object point clouds.

[0083] In this embodiment, multiple semantic point clouds can be extracted from the semantic raster map, which is a complete point cloud containing all the objects to be identified. Therefore, the semantic point clouds need to be clustered before the object point clouds can be obtained.

[0084] For example, the semantic point cloud Euclidean distance clustering algorithm can be used to cluster all points in the semantic point cloud according to distance and category, and the object point cloud of each object can be extracted from the clustering results, with the points in each object point cloud belonging to the same category.

[0085] Understandably, the Euclidean distance clustering algorithm for point clouds is equivalent to finding connected components in three-dimensional space.

[0086] Based on the above embodiments, the step of determining an object point cloud set based on multiple object point clouds may specifically include:

[0087] Step 11131: Use a point cloud classification model to identify the point cloud that is correctly classified from multiple object point clouds as the target object point cloud.

[0088] Step 11132: Determine the object point cloud set based on the target object point cloud.

[0089] After obtaining multiple object point clouds through clustering, the categories of each point may not be correct when fused into the semantic raster, and each raster extracts points of multiple different categories. Therefore, these object point clouds may contain point clouds with incorrect categories and some object overlaps due to category errors.

[0090] Given this situation, in this embodiment, the target object point cloud can also be determined by a point cloud classification model, that is, the correctly classified object point cloud is selected from multiple object point clouds as the target object point cloud.

[0091] In some embodiments, the step of using a point cloud classification model to identify correctly classified object point clouds from multiple object point clouds as the target object point cloud may specifically include:

[0092] Each object point cloud is input into the point cloud classification model, and the first category output by the point cloud classification model is obtained. It is then determined whether the category of the object point cloud is the same as the first category output by the model. If the category of the object point cloud is the same as the first category output by the model, the object point cloud is considered to be correctly classified, and the object point cloud is retained.

[0093] If the category of the object point cloud is different from the first category output by the model, the object point cloud is considered to be classified incorrectly and should be deleted.

[0094] The object point cloud that remains after the point cloud classification model is the target object point cloud. By summing up all the target object point clouds, we can obtain the object point cloud set.

[0095] In some embodiments, the point cloud classification model can be based on network structures such as PointNet and PointNet++. PointNet++ includes a multi-level feature extraction structure, which can effectively extract local and global features, resulting in more accurate point cloud-based classification results.

[0096] In this embodiment, multiple semantic point clouds are extracted from the semantic raster map, and clustering is performed to obtain multiple object point clouds. A point cloud classification model is used to determine whether the object point cloud category is correct, thereby filtering out object point clouds with incorrect categories. Finally, a set of object point clouds is obtained, which improves the accuracy of object category recognition in the semantic raster map.

[0097] Please see Figure 2 , Figure 2This is a second schematic flowchart of the object point cloud overlap recognition method provided in this application embodiment. In this embodiment, the object point cloud overlap recognition method may include steps 210 to 290, each step being as follows:

[0098] Step 210: Calculate the 3D bounding box of the point cloud for each object.

[0099] A bounding box, also known as a bounding box, is a spatial computational geometry structure that effectively represents the boundaries and shape of an object as a set of points within a bounding box. Using bounding box algorithms makes it highly efficient for computers to process relationships between objects within a large dataset or perform distance checks, and it is widely used in fields such as object detection and computer vision.

[0100] 3D bounding boxes are three-dimensional bounding boxes. The overlap of objects can be determined by whether their 3D bounding boxes overlap; however, this method is prone to false positives. For example, the bounding boxes of a coffee table and an L-shaped sofa typically overlap, but in reality, they do not. Therefore, combining the object point cloud overlap recognition method of this embodiment with the calculation of 3D bounding boxes can improve the accuracy of object overlap determination.

[0101] Step 220: Randomly select two object point clouds from the object point cloud set.

[0102] Step 230: Determine whether the point clouds of the two objects contain point clouds that have been marked for deletion.

[0103] Determine whether the two object point clouds contain point clouds that have been marked as deleted. If the two object point clouds contain at least one point cloud that has been marked as deleted, return to step 220 and re-extract the two object point clouds from the object point cloud set for overlap recognition. If neither object point cloud contains a point cloud that has been marked as deleted, continue to step 240.

[0104] Step 240: Determine whether the 3D bounding boxes of the point clouds of the two objects overlap.

[0105] Determine whether the 3D bounding boxes of two object point clouds overlap. If the 3D bounding boxes of the two object point clouds overlap, proceed to step 250 and use the object point cloud overlap recognition method of this embodiment to further calculate the overlap rate of the two object point clouds. If the 3D bounding boxes of the two object point clouds do not overlap, there is no need to continue to perform the overlap recognition of the two object point clouds. Return to step 220 and continue to extract the remaining object point clouds from the object point cloud set for overlap recognition.

[0106] Step 250: Calculate the overlap rate of the point clouds of the two objects.

[0107] In this embodiment, the overlap rate of two object point clouds is determined by calculating the distance between them. The object point cloud with fewer points is selected as the first object point cloud, and the other object point cloud is selected as the second object point cloud. The number of points in the first object point cloud is obtained. The distance between each point in the first object point cloud and the nearest point in the second object point cloud is calculated. The number of overlapping points in the first object point cloud whose corresponding distance is less than a first preset distance is counted. Finally, the overlap rate of the two object point clouds is obtained based on the number of overlapping points and the number of points in the first object point cloud.

[0108] Step 260: Determine whether the overlap rate is greater than the first preset threshold.

[0109] If the overlap rate is greater than the first preset threshold, then proceed to step 270 to mark the object point cloud with the smaller number of points as deleted; if the overlap rate is less than or equal to the first preset threshold, then there is no need to continue to identify the overlap between the two object point clouds, and return to step 220 to continue to extract the remaining object point clouds from the object point cloud set for overlap identification.

[0110] Step 270: Mark the point clouds of objects with smaller point counts as to be deleted.

[0111] Step 280: Determine whether all combinations have been processed.

[0112] If any combination of two object point clouds in the object point cloud set has been processed, then proceed to step 290 to extract object point clouds that have not been marked as deleted from all object point clouds; if there are still combinations of two object point clouds that have not been processed in the object point cloud set, then return to step 220 to extract combinations of two object point clouds that have not been processed from the object point cloud set for overlap recognition.

[0113] Step 290: Extract the object point clouds that were not marked as deleted from all object point clouds.

[0114] The object point cloud overlap recognition method provided in this embodiment, combined with the application of 3D bounding boxes, includes two object overlap judgments: the first judgment is based on the 3D bounding box, and the second judgment is based on the overlap rate, which improves the accuracy of object overlap judgment; it helps robots understand the placement of objects in complex environments, better understand environmental information at the semantic level, improve the robot's intelligence level, and provide information support for the robot's complex behavioral decisions, human-computer interaction and other tasks.

[0115] On the other hand, embodiments of this application provide an overlap recognition device for object point clouds. Please refer to [link to relevant documentation]. Figure 3 , Figure 3This is a schematic diagram of the structure of the object point cloud overlap recognition device provided in an embodiment of this application. In this embodiment, the object point cloud overlap recognition device may include an acquisition module 310, a distance determination module 320, a distance comparison module 330, an overlap rate determination module 340, and a threshold comparison module 350. Specifically:

[0116] The acquisition module 310 is used to acquire the first object point cloud of the first object and the second object point cloud of the second object; wherein the number of points in the first object point cloud is less than or equal to the number of points in the second object point cloud.

[0117] The distance determination module 320 is used to determine the recognition distance between each point in the first object point cloud and the nearest point in the second object point cloud.

[0118] The distance comparison module 330 is used to compare each identification distance with a first preset distance to determine the number of overlapping points based on the first comparison result.

[0119] The overlap rate determination module 340 is used to obtain the overlap rate based on the number of overlapping points and the number of points in the first object's point cloud.

[0120] The threshold comparison module 350 is used to compare the overlap rate with a first preset threshold to determine the overlap recognition result of the first object and the second object based on the second comparison result.

[0121] The object point cloud overlap recognition device provided in this embodiment acquires a first object point cloud of a first object and a second object point cloud of a second object through an acquisition module, wherein the number of points in the first object point cloud is less than or equal to the number of points in the second object point cloud; a distance determination module determines the recognition distance between each point in the first object point cloud and the nearest point in all points in the second object point cloud; a distance comparison module compares each recognition distance with a first preset distance to determine the number of overlapping points based on the first comparison result; an overlap rate determination module obtains the overlap rate based on the number of overlapping points and the number of points in the first object point cloud; and a threshold comparison module compares the overlap rate with a first preset threshold to determine the overlap recognition result of the first and second objects based on the second comparison result. Therefore, the overlap rate of two object point clouds can be determined based on the distance between them, and the overlap rate can be used to determine whether the object point clouds overlap. This object point cloud overlap recognition device improves the accuracy of object overlap judgment, helps robots understand the placement of objects in complex environments, and improves the intelligence level of robots.

[0122] Based on the above embodiments, the threshold comparison module 350 is specifically used for:

[0123] When the overlap rate is greater than or equal to the first preset threshold, the first object and the second object are determined to overlap; when the overlap rate is less than the first preset threshold, the first object and the second object are determined not to overlap.

[0124] Based on the above embodiments, the object point cloud overlap recognition device further includes a marker deletion module, which is specifically used for:

[0125] When the number of points in the first object point cloud is less than the number of points in the second object point cloud, the first object point cloud is marked for deletion; when the number of points in the first object point cloud is equal to the number of points in the second object point cloud, either the first object point cloud or the second object point cloud is marked for deletion.

[0126] Based on the above embodiments, the distance comparison module 330 is specifically used for:

[0127] Each recognition distance is compared with a first preset distance to obtain a first comparison result; wherein the first comparison result includes recognition distances that are less than or equal to the first preset distance; based on the first comparison result, the number of recognition distances that are less than or equal to the first preset distance is determined, and the number is used as the number of overlapping points.

[0128] Based on the above embodiments, the acquisition module 310 is specifically used for:

[0129] Object point cloud extraction is performed based on semantic raster map to obtain object point cloud set; two object point clouds that have not been marked for deletion are obtained from the object point cloud set; the number of points in the two object point clouds that have not been marked for deletion is compared numerically; based on the comparison result, the first object point cloud and the second object point cloud are determined from the two object point clouds that have not been marked for deletion.

[0130] Based on the above embodiments, the acquisition module 310 can be a point cloud set determination module, which can be used for:

[0131] Extract multiple semantic point clouds from the semantic raster map; perform clustering on the multiple semantic point clouds to obtain multiple object point clouds; determine the object point cloud set based on the multiple object point clouds.

[0132] Based on the above embodiments, the point cloud set determination module is specifically used for:

[0133] The point cloud classification model identifies the correctly classified object point cloud from multiple object point clouds as the target object point cloud; the object point cloud set is then determined based on the target object point cloud.

[0134] Furthermore, embodiments of this application also provide an electronic device. Please refer to [link / reference]. Figure 4 , Figure 4This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of this application. The electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440. The processor 410, communication interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions stored in the memory 430 to execute the following method:

[0135] Acquire a first object point cloud of a first object and a second object point cloud of a second object; wherein the number of points in the first object point cloud is less than or equal to the number of points in the second object point cloud; determine the recognition distance between each point in the first object point cloud and the nearest point in all points in the second object point cloud; compare each recognition distance with a first preset distance to determine the number of overlapping points based on the first comparison result; obtain the overlap rate based on the number of overlapping points and the number of points in the first object point cloud; compare the overlap rate with a first preset threshold to determine the overlap recognition result of the first object and the second object based on the second comparison result.

[0136] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0137] In another aspect, embodiments of this application disclose a robot, which includes a control unit for executing the methods provided in the above-described method embodiments, such as including:

[0138] Acquire a first object point cloud of a first object and a second object point cloud of a second object; wherein the number of points in the first object point cloud is less than or equal to the number of points in the second object point cloud; determine the recognition distance between each point in the first object point cloud and the nearest point in all points in the second object point cloud; compare each recognition distance with a first preset distance to determine the number of overlapping points based on the first comparison result; obtain the overlap rate based on the number of overlapping points and the number of points in the first object point cloud; compare the overlap rate with a first preset threshold to determine the overlap recognition result of the first object and the second object based on the second comparison result.

[0139] In another aspect, embodiments of this application also provide a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments, including, for example:

[0140] Acquire a first object point cloud of a first object and a second object point cloud of a second object; wherein the number of points in the first object point cloud is less than or equal to the number of points in the second object point cloud; determine the recognition distance between each point in the first object point cloud and the nearest point in all points in the second object point cloud; compare each recognition distance with a first preset distance to determine the number of overlapping points based on the first comparison result; obtain the overlap rate based on the number of overlapping points and the number of points in the first object point cloud; compare the overlap rate with a first preset threshold to determine the overlap recognition result of the first object and the second object based on the second comparison result.

[0141] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0142] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate this application and are not intended to limit this application. Although this application has been described in detail with reference to the embodiments, those skilled in the art should understand that various combinations, modifications, or equivalent substitutions of the technical solutions of this application do not depart from the spirit and scope of the technical solutions of this application and should be covered within the scope of the claims of this application.

Claims

1. A method for recognizing overlapping point clouds of objects, characterized in that, include: Obtain the point cloud of the first object and the point cloud of the second object. The number of points in the first object point cloud is less than or equal to the number of points in the second object point cloud; Determine the recognition distance between each point in the first object point cloud and the nearest point in all points in the second object point cloud; Each recognition distance is compared with the first preset distance to determine the number of overlapping points based on the first comparison result; The overlap rate is obtained based on the number of overlapping points and the number of points in the point cloud of the first object; The overlap rate is compared with a first preset threshold to determine the overlap recognition result of the first object and the second object based on the second comparison result.

2. The method for overlapping point clouds of objects according to claim 1, characterized in that, The step of comparing the overlap rate with a first preset threshold to determine the overlap recognition result of the first object and the second object based on the second comparison result includes: When the overlap rate is greater than or equal to the first preset threshold, it is determined that the first object and the second object overlap. When the overlap rate is less than the first preset threshold, it is determined that the first object and the second object do not overlap.

3. The method for overlapping point clouds of objects according to claim 2, characterized in that, After determining that the first object and the second object overlap, the following steps are included: When the number of points in the first object point cloud is less than the number of points in the second object point cloud, the first object point cloud is marked for deletion; When the number of points in the first object point cloud is equal to the number of points in the second object point cloud, either the first object point cloud or the second object point cloud is marked for deletion.

4. The method for overlapping point clouds of objects according to claim 1, characterized in that, The step of comparing each identification distance with a first preset distance to determine the number of overlapping points based on the first comparison result includes: Each recognition distance is compared with a first preset distance to obtain the first comparison result; wherein the first comparison result includes recognition distances that are less than or equal to the first preset distance; Based on the first comparison result, determine the number of all identification distances that are less than or equal to the first preset distance, and use the number as the number of overlapping points.

5. The method for overlapping point clouds of objects according to any one of claims 1 to 4, characterized in that, The acquisition of the first object point cloud of the first object and the second object point cloud of the second object includes: Object point cloud extraction is performed based on semantic raster maps to obtain a set of object point clouds; Obtain two object point clouds that have not been marked for deletion from the object point cloud set; Compare the number of points in the point clouds of the two objects that were not marked for deletion. Based on the comparison results, the first object point cloud and the second object point cloud are determined from the two object point clouds that were not marked for deletion.

6. The method for overlapping point clouds of objects according to claim 5, characterized in that, The object point cloud extraction based on semantic raster maps yields an object point cloud set, including: Extract multiple semantic point clouds from the semantic raster map; Clustering is performed on the multiple semantic point clouds to obtain multiple object point clouds; The object point cloud set is determined based on the multiple object point clouds.

7. The method for overlapping point clouds of objects according to claim 6, characterized in that, Determining the object point cloud set based on the multiple object point clouds includes: The point cloud of the object that is correctly classified among the multiple object point clouds is used as the target object point cloud by the point cloud classification model. The object point cloud set is determined based on the target object point cloud.

8. A device for recognizing overlapping point clouds of objects, characterized in that, include: The acquisition module is used to acquire the first object point cloud of the first object and the second object point cloud of the second object. The number of points in the first object point cloud is less than or equal to the number of points in the second object point cloud; The distance determination module is used to determine the recognition distance between each point in the first object point cloud and the nearest point in all points in the second object point cloud; The distance comparison module is used to compare each identification distance with a first preset distance to determine the number of overlapping points based on the first comparison result; The overlap rate determination module is used to obtain the overlap rate based on the number of overlapping points and the number of points in the first object point cloud; The threshold comparison module is used to compare the overlap rate with a first preset threshold to determine the overlap recognition result of the first object and the second object based on the second comparison result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method for overlapping object point clouds as described in any one of claims 1 to 7.

10. A robot, characterized in that, Includes a control unit; the control unit is used to perform the overlapping recognition method for object point clouds as described in any one of claims 1 to 7.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the overlapping recognition method for object point clouds as described in any one of claims 1 to 7.

Images