A detection method, device, computer equipment and storage medium

Abstract

The present disclosure provides a detection method, apparatus, computer equipment and storage medium, wherein the method includes acquiring first point cloud data in a target environment and target geometric features of a target object; based on the target geometric features, filtering first target point cloud data belonging to the target object from the first point cloud data, and performing object segmentation on the first point cloud data other than the first target point cloud data to obtain second target point cloud data of other objects other than the target object; based on the second target point cloud data and the first target point cloud data, determining the positional relationship information between each other object and the target object; based on the positional relationship information, filtering a first obstacle located on the target object and at least a preset distance beyond the target object from other objects. The embodiments of the present disclosure utilize the target geometric features to accurately segment the first target point cloud data of the target object and the second target point cloud data of other objects, thereby accurately distinguishing the first obstacle.

Description

Technical Field

[0001] The present disclosure relates to the technical field of point cloud processing and object detection, and in particular to a detection method, apparatus, computer equipment and storage medium. Background Art

[0002] When the robot is running in a narrow shelf aisle, some of the goods on the shelves on both sides of the robot that extend beyond the shelves will collide with the robot, affecting the normal operation of the robot. Usually, the volume of the goods that extend beyond the shelves is small and closely connected to the shelves. By clustering the collected point cloud data, it is impossible to segment the small volume of the goods that are closely connected to the shelves. It is only considered that the goods that extend beyond the shelves are integrated, so accurate segmentation cannot be achieved, resulting in reduced environmental perception of the robot, inability to complete accurate obstacle avoidance, and easy damage to the robot in the event of a collision, increasing the robot maintenance cost. Summary of the invention

[0003] The embodiments of the present disclosure at least provide a detection method, apparatus, computer device and storage medium.

[0004] In a first aspect, an embodiment of the present disclosure provides a detection method, comprising:

[0005] Acquire first point cloud data in a target environment and target geometric features of a target object;

[0006] Based on the target geometric features, first target point cloud data belonging to the target object is selected from the first point cloud data, and object segmentation is performed on the first point cloud data other than the first target point cloud data to obtain second target point cloud data of other objects other than the target object;

[0007] Determine positional relationship information between each other object and the target object based on the second target point cloud data and the first target point cloud data;

[0008] Based on the position relationship information, a first obstacle located on the target object and beyond the target object by at least a preset distance is screened from the other objects.

[0009] In an optional implementation, the filtering of first target point cloud data belonging to the target object from the first point cloud data based on the target geometric features includes:

[0010] Obtaining a preset fitting range of the target geometric feature;

[0011] Based on the fitting range and the target geometric features, first point cloud data belonging to the target object is fitted to determine the first target point cloud data.

[0012] In an optional implementation manner, the other objects include objects located between two adjacent target objects;

[0013] The performing object segmentation on the first point cloud data except the first target point cloud data to obtain second target point cloud data of other objects except the target object includes:

[0014] Based on the first target point cloud data, screening second point cloud data located between two adjacent target objects from the first point cloud data;

[0015] The second point cloud data is segmented to obtain second target point cloud data of objects other than the target object.

[0016] In an optional implementation manner, the performing object segmentation on the first point cloud data except the first target point cloud data to obtain second target point cloud data of other objects except the target object includes:

[0017] Determining edge position information of the target object based on the first target point cloud data;

[0018] Based on the edge position information, point cloud data that is not located between two adjacent target objects is removed from the first point cloud data, and the remaining point cloud data after the removal is used as the third point cloud data;

[0019] The third point cloud data is subjected to object segmentation to obtain second target point cloud data of objects other than the target object.

[0020] In an optional implementation, the position relationship information includes at least one of the following: the other object is on the target object; the other object is located on the target object and exceeds the target object; the other object is not on the target object.

[0021] In an optional implementation manner, the selecting, based on the position relationship information, from the other objects, a first obstacle located on the target object and at least a preset distance beyond the target object, comprises:

[0022] When the position relationship information indicates that the other object is located on the target object and exceeds the target object, determining a length by which the other object exceeds the target object in a preset direction;

[0023] When the length is greater than the preset distance, the other object is regarded as the first obstacle.

[0024] In an optional implementation manner, after determining the positional relationship information between each other object and the target object based on the second target point cloud data and the first target point cloud data, the method further includes:

[0025] When the position relationship information indicates that the other object is not on the target object, the other object outside the target object is regarded as a second obstacle.

[0026] In an optional embodiment, the target object includes a shelf; the target geometric feature includes a feature of a preset plane of the shelf; and the preset plane includes a plane of the shelf that is perpendicular to the goods import and export direction and / or the horizontal plane.

[0027] In an optional implementation, the target object includes the ground; the target geometric feature includes a ground plane feature; and the detection method further includes:

[0028] Based on the positional relationship information between the other objects and the ground, a second obstacle located on the ground is screened from the other objects.

[0029] In an optional implementation manner, after selecting, based on the position relationship information, a first obstacle located on the target object and beyond a preset distance from the target object from the other objects, the method further includes:

[0030] Determine, based on the second target point cloud data of the first obstacle, first position information of the first obstacle in the target environment, so that the mobile device avoids the first obstacle based on the first position information; and / or

[0031] Based on the second target point cloud data of the second obstacle, second position information of the second obstacle in the target environment is determined, so that the mobile device avoids the second obstacle based on the second position information.

[0032] In a second aspect, the present disclosure also provides a detection device, including:

[0033] A data acquisition module, used to acquire first point cloud data in a target environment and target geometric features of a target object;

[0034] a screening and segmentation module, configured to screen first target point cloud data belonging to the target object from the first point cloud data based on the target geometric features, and perform object segmentation on the first point cloud data other than the first target point cloud data to obtain second target point cloud data of other objects other than the target object;

[0035] An information determination module, configured to determine positional relationship information between each other object and the target object based on the second target point cloud data and the first target point cloud data;

[0036] The object screening module is used to screen a first obstacle located on the target object and beyond the target object by at least a preset distance from the other objects based on the position relationship information.

[0037] In an optional embodiment, the screening and segmentation module is used to obtain a preset fitting range of the target geometric features; based on the fitting range and the target geometric features, fit the first point cloud data belonging to the target object to determine the first target point cloud data.

[0038] In an optional implementation manner, the other objects include objects located between two adjacent target objects;

[0039] The screening and segmentation module is used to screen second point cloud data located between two adjacent target objects from the first point cloud data based on the first target point cloud data; and perform object segmentation on the second point cloud data to obtain second target point cloud data of other objects except the target object.

[0040] In an optional embodiment, the screening and segmentation module is used to determine the edge position information of the target object based on the first target point cloud data; based on the edge position information, eliminate the point cloud data that is not located between two adjacent target objects from the first point cloud data, and use the remaining point cloud data after elimination as the third point cloud data; perform object segmentation on the third point cloud data to obtain the second target point cloud data of other objects except the target object.

[0041] In an optional implementation, the position relationship information includes at least one of the following: the other object is on the target object; the other object is located on the target object and exceeds the target object; the other object is not on the target object.

[0042] In an optional embodiment, the object screening module is used to determine the length of the other object exceeding the target object in a preset direction when the position relationship information indicates that the other object is located on the target object and exceeds the target object; when the length is greater than the preset distance, the other object is regarded as the first obstacle.

[0043] In an optional embodiment, the object screening module is also used to, after determining the positional relationship information of each other object with the target object based on the second target point cloud data and the first target point cloud data, treat other objects outside the target object as second obstacles when the positional relationship information indicates that the other object is not on the target object.

[0044] In an optional embodiment, the target object includes a shelf; the target geometric feature includes a feature of a preset plane of the shelf; and the preset plane includes a plane of the shelf that is perpendicular to the goods import and export direction and / or the horizontal plane.

[0045] In an optional embodiment, the target object includes the ground; the target geometric features include ground plane features; and the object screening module is further used to screen a second obstacle located on the ground from the other objects based on positional relationship information between the other objects and the ground.

[0046] In an optional embodiment, the detection device also includes an obstacle avoidance processing module, which is used to determine the first position information of the first obstacle in the target environment based on the second target point cloud data of the first obstacle after screening out a first obstacle located on the target object and beyond a preset distance from the target object from the other objects based on the position relationship information, so that the mobile device avoids the first obstacle based on the first position information; and / or, determine the second position information of the second obstacle in the target environment based on the second target point cloud data of the second obstacle, so that the mobile device avoids the second obstacle based on the second position information.

[0047] In a third aspect, an optional implementation of the present disclosure further provides a computer device, a processor, and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the machine-readable instructions perform the steps of the above-mentioned first aspect, or any possible implementation of the first aspect.

[0048] In a fourth aspect, an optional implementation of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are executed.

[0049] For a description of the effects of the above-mentioned detection device, computer equipment, and computer-readable storage medium, please refer to the description of the above-mentioned detection method, which will not be repeated here.

[0050] The detection method, device, computer equipment and storage medium provided by the embodiment of the present disclosure obtain the first point cloud data and the target geometric features of the target object in the target environment; based on the target geometric features, the first target point cloud data belonging to the target object is selected from the first point cloud data, and the first point cloud data other than the first target point cloud data is segmented to obtain the second target point cloud data of other objects other than the target object; based on the second target point cloud data and the first target point cloud data, the position relationship information of each other object and the target object is determined; based on the position relationship information, the first obstacle located on the target object and exceeding the target object by at least a preset distance is selected from the other objects. Compared with the prior art in which the excess part of the goods with a smaller volume that is closely connected to the shelf cannot be segmented, the target geometric features are used, and the target geometric features are the only geometric features of the target object, and do not include features that do not belong to the target object, for example, the shelf plane features do not include features of the goods with a smaller volume that exceed the shelf. Using the target geometric features of the target object, the first target point cloud data belonging to the target object and the second target point cloud data belonging to other objects can be accurately segmented, and then, using the position relationship information of each other object and the target object, it is possible to accurately distinguish which other objects belong to the first obstacle.

[0051] Furthermore, the detection method, apparatus, computer device and storage medium provided by the embodiments of the present disclosure obtain a pre-set fitting range of the target geometric features; based on the fitting range and the target geometric features, fit the first point cloud data belonging to the target object to determine the first target point cloud data. Fitting the first point cloud data belonging to the target object within the fitting range can further improve the accuracy of the determined first target point cloud data.

[0052] Furthermore, the detection method, apparatus, computer device and storage medium provided by the embodiments of the present disclosure, when other objects include an object located between two adjacent target objects, based on the first target point cloud data, filter the second point cloud data located between two adjacent target objects from the first point cloud data; perform object segmentation on the second point cloud data to obtain second target point cloud data of other objects except the target objects. The second point cloud data is a subset of the first point cloud data except the first target point cloud data, and object segmentation is performed only on the filtered second point cloud data, which can reduce the amount of object segmentation and improve segmentation efficiency compared with performing object segmentation processing on the first point cloud data except the first target point cloud data.

[0053] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0055] Figure 1 A flow chart of a detection method provided by an embodiment of the present disclosure is shown;

[0056] Figure 2 A schematic diagram showing a target environment provided by an embodiment of the present disclosure is shown;

[0057] Figure 3 A flowchart of fitting first point cloud data and determining first target point cloud data provided by an embodiment of the present disclosure is shown;

[0058] Figure 4 A schematic diagram of the process of the robot performing environment perception and obstacle avoidance provided by the embodiment of the present disclosure is shown;

[0059] Figure 5 A schematic diagram of a detection device provided by an embodiment of the present disclosure is shown;

[0060] Figure 6 A schematic diagram of the structure of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the present disclosure claimed for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0062] In addition, the terms "first", "second", etc. in the description and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0063] The "multiple or several" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0064] Research has found that when the robot is running in a narrow shelf aisle, some of the goods on the shelves on both sides of the robot that extend beyond the shelves will collide with the robot, affecting the normal operation of the robot. Usually, the volume of the goods that extend beyond the shelves is small and closely connected to the shelves. By clustering the collected point cloud data, it is impossible to segment the small volume of the goods that are closely connected to the shelves. It is only considered that the goods that extend beyond the shelves are integrated, so accurate segmentation cannot be achieved, resulting in reduced environmental perception of the robot, and the inability to complete accurate obstacle avoidance. In the event of a collision, the robot is easily damaged, increasing the maintenance cost of the robot.

[0065] Based on the above research, the present disclosure provides a detection method, device, computer equipment and storage medium, which utilizes the target geometric feature, which is the only geometric feature of the target object and does not include features that do not belong to the target object, for example, the shelf plane feature does not include features of smaller volume goods that exceed the shelf. Using the target geometric feature of the target object, the first target point cloud data belonging to the target object and the second target point cloud data belonging to other objects can be accurately segmented, and then, using the positional relationship information of each other object and the target object, it is possible to accurately distinguish which other objects belong to the first obstacle.

[0066] The defects existing in the above solutions are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the present disclosure for the above problems below should be the contributions made by the inventor to the present disclosure during the disclosure process.

[0067] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0068] It should be noted that the specific terms mentioned in the embodiments of the present disclosure include:

[0069] 1. k-means algorithm (k-means clustering algorithm) is an iterative clustering analysis algorithm. The so-called clustering, its steps are, pre-dividing the data into K groups, then randomly selecting K objects as the initial cluster centers, and then calculating the distance between each object and each seed cluster center, and assigning each object to the cluster center closest to it.

[0070] 2. DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is a representative density-based clustering algorithm. Different from partitioning and hierarchical clustering methods, it defines a cluster as the largest set of density-connected points, can divide areas with sufficiently high density into clusters, and can find clusters of any shape in noisy spatial databases.

[0071] 3. RANSAC is the abbreviation of Random Sample Consensus. It is an algorithm that calculates the mathematical model parameters of the data based on a set of sample data sets containing abnormal data to obtain valid sample data.

[0072] To facilitate understanding of this embodiment, the application scenario of a detection method disclosed in the embodiment of the present disclosure is first introduced in detail. A detection method disclosed in the embodiment of the present disclosure can be applied to a scenario in which a robot performs environmental perception and obstacle avoidance. Specifically, the robot runs in some narrow alleys, accurately distinguishes between the shelves on both sides and the goods beyond the shelves, and treats the goods beyond the shelves as obstacles. A sensor can be installed on the robot to detect environmental information in the robot's operating space, including the cargo status information of the shelves on both sides of the robot in the narrow alley and the obstacle information suspended between the shelves on both sides in the narrow alley, wherein the cargo status information includes at least one of the following: the cargo is on the inside of the shelf (not beyond the shelf), the cargo is beyond the shelf, and the cargo is on the outside of the shelf (such as the ground; or, the cargo is in the process of falling off the shelf, and is above the ground between the shelves on both sides in the narrow alley).

[0073] The executor of the detection method provided in the embodiments of the present disclosure is generally a computer device with certain computing capabilities. In some possible implementations, the detection method can be implemented by a processor calling computer-readable instructions stored in a memory.

[0074] The detection method provided in the embodiment of the present disclosure is described below by taking a computer device as an example of an execution subject.

[0075] See the above introduction to the application scenario of a detection method disclosed in the embodiment of the present disclosure, such as Figure 1 As shown, it is a flow chart of a detection method provided by an embodiment of the present disclosure, the method includes steps S101 to S104, wherein:

[0076] S101: Acquire first point cloud data in a target environment and target geometric features of a target object.

[0077] In this step, the target environment may include the environment where the target object is located. For example, in the field of warehousing and logistics, taking the scenario where a robot runs in a warehouse as an example, the target environment includes robots, shelves, goods, other devices and equipment, etc. In addition, the target object may include shelves and / or the ground.

[0078] The first point cloud data may be point cloud data of the target environment collected by the sensor installed on the robot, wherein each first point cloud data has a unique position coordinate in the robot coordinate system. Here, the sensor may be installed on the top of the robot's gantry to collect information about the target environment in the robot's operating space. Specifically, it may collect environmental information in front of the robot and cargo status information on the shelves on both sides of the robot. Here, the sensor used may be a three-dimensional perception sensor, including but not limited to a time-of-flight-based camera, a structured light-based camera, a binocular camera, a lidar sensor, and a millimeter-wave radar sensor.

[0079] In this step, the target geometric features may include features of a preset plane of the shelf and / or features of a ground plane, wherein the preset plane may include a plane of the shelf that is perpendicular to the direction of the goods inlet and outlet and / or the horizontal plane, which may be referred to in Figure 2 , which is a schematic diagram of the target environment, which is a top view. 21 represents the robot, 22 represents the positive direction of the robot, 23 represents the shelf on the left side of the robot, 24 represents the shelf on the right side of the robot, 25 represents the direction of the import and export of goods, and 26 represents the preset plane of the shelf, which can be the right plane of the shelf 23 on the left side of the robot, that is, the preset plane 26-A, or the left plane of the shelf 24 on the right side of the robot, that is, the preset plane 26-B.

[0080] S102: Filter first target point cloud data belonging to a target object from the first point cloud data based on target geometric features, and perform object segmentation on the first point cloud data other than the first target point cloud data to obtain second target point cloud data of other objects other than the target object.

[0081] First, based on the target geometric features, an equation about the target geometric features of the target object can be established, and the first point cloud data belonging to the target object and satisfying the target geometric features can be selected from the first point cloud data and fitted to determine the first target point cloud data. Afterwards, the first target point cloud data can be clustered and segmented and saved separately.

[0082] Continuing with the above example, first, we can establish the plane equation in the robot coordinate system o-xyz, that is,

[0083]

[0084] Wherein, a, b, c represent coefficients on X, Y, and Z, respectively, X represents the coordinate value on the x-axis direction, Y represents the coordinate value on the y-axis direction, Z represents the coordinate value on the z-axis direction, and M represents the distance between the preset plane 26-A and the preset plane 26-B. It should be noted that, in general, the robot running path is the center line between the preset plane 26-A and the preset plane 26-B. Wherein, see Figure 2 As shown, the right side of the robot is the positive direction of the x-axis in the robot coordinate system o-xyz, the forward direction of the robot is the positive direction of the y-axis in the robot coordinate system o-xyz, and the vertical upward direction is the positive direction of the z-axis in the robot coordinate system o-xyz.

[0085] Afterwards, based on the characteristics of the preset plane 26-A on the left shelf of the robot, the plane equation belonging to the preset plane 26-A can be established in the robot coordinate system o-xyz. Since the preset plane 26-A is parallel to the yoz plane and the preset plane 26-A passes through the point o-xyz of the robot coordinate system Therefore, the plane equation of the preset plane 26-A is:

[0086]

[0087] Alternatively, based on the characteristics of the preset plane 26-B of the shelf on the right side of the robot, a plane equation belonging to the preset plane 26-B can be established in the robot coordinate system o-xyz. Since the preset plane 26-B is parallel to the yoz plane and the preset plane 26-B passes through the point o-xyz of the robot coordinate system Therefore, the plane equation of the preset plane 26-B is:

[0088]

[0089] Taking the preset plane 26-A and the preset plane 26-B as an example, in the specific implementation, based on the position coordinates of the first point cloud data in the robot coordinate system, according to the plane equation of the preset plane 26-A and the plane equation of the preset plane 26-B, the first target point cloud data belonging to the preset plane 26-A and the first target point cloud data belonging to the preset plane 26-B are screened from the first point cloud data, and the first target point cloud data belonging to the preset plane 26-A is separated from the first point cloud data and saved separately, and the first target point cloud data belonging to the preset plane 26-B is separated from the first point cloud data and saved separately.

[0090] After the first target point cloud data has been segmented out, it is detected whether the distance between the first point cloud data and the first target point cloud data exceeds at least the preset distance from two adjacent target objects in the first point cloud data remaining after the first target point cloud data has been segmented out, and the first point cloud data exceeding at least the preset distance is clustered, and the object is segmented to obtain the second target point cloud data of the obstacle. Specifically, the KMeans algorithm or DBSCAN can be used to analyze the first point cloud data exceeding the preset distance, and according to the similarity principle, the first point cloud data with a higher similarity is divided into the same cluster to obtain the clustered object, and the object is segmented to obtain the second target point cloud data belonging to the obstacle.

[0091] It should be noted that the position coordinates of the separately saved first target point cloud data in the robot coordinate system are known. Based on the known position coordinates of the first target point cloud data, the distance from the first point cloud data between two adjacent target objects in the first point cloud data remaining after the first target point cloud data has been segmented can be calculated.

[0092] Exemplarily, other objects include all objects except the target object. For example, when the target object is a shelf and / or the ground, other objects may include goods on the shelf, or goods on the ground, or other obstacles on the ground, or obstacles suspended above the lanes between adjacent shelves, etc. The second target point cloud data may include point cloud data of goods on the shelf, or point cloud data of goods on the ground, or point cloud data of other obstacles on the ground, or point cloud data of obstacles suspended above the lanes between adjacent shelves, etc.

[0093] S103: Determine positional relationship information between each other object and the target object based on the second target point cloud data and the first target point cloud data.

[0094] In this step, the position relationship information may include at least one of the following: other objects are on the target object; other objects are located on the target object and beyond the target object; other objects are not on the target object, etc.

[0095] Exemplarily, the first target point cloud data may be the point cloud data of the preset plane 26-A of the shelf, and the second target point cloud data may be the point cloud data belonging to the goods 231 on the left shelf 23. Based on the position coordinates of the first target point cloud data and the second target point cloud data in the robot coordinate system, the position coordinates of the point cloud data of each goods on the left shelf 23 in the robot coordinate system are determined, the position coordinates of the preset plane 26-A are determined, and then the position relationship information of each goods and the left shelf 23 is determined, including goods on the left shelf 23, such as goods 231-A; goods located on the left shelf 23 and beyond the left shelf 23, such as goods 231-B.

[0096] S104: Based on the position relationship information, a first obstacle located on the target object and beyond the target object by at least a preset distance is selected from other objects.

[0097] In a specific implementation, when the position relationship information indicates that other objects are located on the target object and exceed the target object, the length of the other objects exceeding the target object in the preset direction is determined; when the length is greater than the preset distance, the other objects are regarded as the first obstacle.

[0098] Here, the preset distance can be determined according to a specific application example, and the embodiments of the present disclosure do not specifically limit it. Without departing from the scope of the present disclosure, those skilled in the art can make various parameter modifications to the preset distance, and these modifications should fall within the scope of the present disclosure.

[0099] For example, see Figure 2 As shown, the preset direction may include the direction of the import and export of goods, wherein for the preset plane 26-A, the preset direction may be the direction indicated by 27, and for the preset plane 26-B, the preset direction may be the direction indicated by 28. Taking the preset plane 26-A as a reference, the goods on the left shelf 23 and exceeding the preset plane 26-A by at least a preset distance are regarded as the first obstacle, such as the part of the goods 231-B that exceeds the preset plane 26-A. Here, the preset distance can be determined based on the distance M between the preset plane 26-A and the preset plane 26-B and the distance L occupied by the robot in the lane between the preset plane 26-A and the preset plane 26-B. For example, the preset distance can be

[0100] Afterwards, based on the second target point cloud data of the first obstacle, first position information of the first obstacle in the target environment is determined, so that the mobile device avoids the first obstacle based on the first position information.

[0101] Exemplarily, the robot may perform obstacle avoidance processing according to the first position information of the first obstacle in the operating environment, stop operating in time, and send a signal indicating the appearance of the first obstacle and the position coordinates of the first obstacle in the target environment to the service station.

[0102] In some embodiments, when the position relationship information indicates that other objects are not on the target object, other objects outside the target object are regarded as second obstacles. Afterwards, second position information of the second obstacle in the target environment can be determined based on the second target point cloud data of the second obstacle, so that the mobile device avoids the second obstacle based on the second position information.

[0103] Exemplarily, the situation where the position relationship information indicates that the other object is not on the target object may include the situation where the other object is located on the ground between the preset plane 26-A and the preset plane 26-B, or the situation where the other object is suspended between the preset plane 26-A and the preset plane 26-B, and the other object in this situation is regarded as the second obstacle. At this time, the robot can perform obstacle avoidance processing according to the second position information of the second obstacle in the operating environment, stop running in time, and send a signal of the appearance of the second obstacle and the position coordinates of the second obstacle in the target environment to the service desk.

[0104] In some embodiments, when the target object includes the ground and the target geometric features include ground plane features, a second obstacle located on the ground may be screened from other objects based on positional relationship information between the other objects and the ground.

[0105] Here, the method of selecting the second obstacle located on the ground from other objects can refer to the method of selecting the first obstacle in the above step S104. In specific implementation, based on the ground plane feature, a plane equation belonging to the ground can be established in the robot coordinate system o-xyz. Since the ground is parallel to the xoy plane and the ground passes through the origin (0,0,0) of the robot coordinate system o-xyz, the plane equation of the ground is

[0106] Z=0

[0107] Afterwards, based on the position coordinates of the second target point cloud data in the robot coordinate system and the plane equation of the ground Z=0, the second target point cloud data located on the ground can be determined, that is, there is point cloud data in the second target point cloud data whose distance to the ground is zero, and other objects corresponding to the second target point cloud data located on the ground can be regarded as second obstacles.

[0108] The above steps S101 to S104 utilize the target geometric feature, which is the only geometric feature of the target object and does not include features that do not belong to the target object. For example, the shelf plane feature does not include features of smaller volume goods that exceed the shelf. Using the target geometric feature of the target object, the first target point cloud data belonging to the target object and the second target point cloud data belonging to other objects can be accurately segmented. Afterwards, using the positional relationship information between each other object and the target object, it is possible to accurately distinguish which other objects belong to the first obstacle.

[0109] Based on the above step S102, first target point cloud data belonging to the target object is selected from the first point cloud data based on the target geometric features. In a possible implementation, the first point cloud data belonging to the target object and satisfying the target geometric features is fit processed. For details, see Figure 3 As shown, it is a flow chart of fitting the first point cloud data and determining the first target point cloud data, including steps S301 to S302:

[0110] S301: Obtaining a pre-set fitting range of a target geometric feature;

[0111] S302: Fitting first point cloud data belonging to the target object based on the fitting range and the target geometric features to determine first target point cloud data.

[0112] Here, the fitting range is the range defined by the RANSAC algorithm. By fitting the first point cloud data belonging to the target object and satisfying the target geometric features through the RANSAC algorithm, the accuracy of the determined first target point cloud data can be further improved by fitting the first point cloud data belonging to the target object within the fitting range using the RANSAC algorithm.

[0113] Based on the object segmentation of the first point cloud data except the first target point cloud data in step S102 above, the second target point cloud data of other objects except the target object is obtained. In a possible implementation, only the first point cloud data at the required position can be segmented for other objects, which can reduce the amount of object segmentation and improve segmentation efficiency. In specific implementation, first, based on the first target point cloud data, the second point cloud data located between two adjacent target objects can be screened from the first point cloud data; then, the second point cloud data can be segmented to obtain the second target point cloud data of other objects except the target object.

[0114] Exemplarily, it is detected whether the distance between the second point cloud data and the first target point cloud data exceeds at least a preset distance, the second point cloud data exceeding at least the preset distance is clustered, and object segmentation is performed to obtain second target point cloud data of the obstacle.

[0115] Here, the second point cloud data belongs to a subset of the first point cloud data other than the first target point cloud data. Exemplarily, when the first target point cloud data of the preset plane 26-A and the first target point cloud data of the preset plane 26-B are known, the position coordinates of the preset plane 26-A and the position coordinates of the preset plane 26-B can be determined, and the second point cloud data located between the preset plane 26-A and the preset plane 26-B can be screened out from the first point cloud data, and the second point cloud data is clustered to determine the second target point cloud data belonging to other objects, and the second target point cloud data is segmented from the second point cloud data, so as to prepare for further determining the positional relationship information of other objects and the target object.

[0116] Here, object segmentation is performed only on the filtered second point cloud data, which can reduce the object segmentation amount and improve the segmentation efficiency compared with performing object segmentation processing on the first point cloud data except the first target point cloud data.

[0117] Based on the object segmentation performed on the first point cloud data except the first target point cloud data in the above step S102, second target point cloud data of objects other than the target object is obtained. In another possible implementation, unnecessary obstacle point cloud data in the first point cloud data can be first eliminated, and then the retained first point cloud data is segmented. In a specific implementation, the edge position information of the target object is determined based on the first target point cloud data; based on the edge position information, the point cloud data that is not located between two adjacent target objects is eliminated from the first point cloud data, and the remaining point cloud data after the elimination is used as the third point cloud data; the third point cloud data is segmented to obtain second target point cloud data of objects other than the target object.

[0118] Here, the obstacle point cloud data may be point cloud data that is not located between two adjacent target objects, for example, Figure 2 As shown, the obstacle point cloud data may be point cloud data other than the point cloud data between the preset plane 26 -A and the preset plane 26 -B (including the point cloud data of the preset plane 26 -A and the point cloud data of the preset plane 26 -B).

[0119] The edge position information of the target object includes the position information of the preset plane 26-A and the position information of the preset plane 26-B. The third point cloud data includes the point cloud data between the preset plane 26-A and the preset plane 26-B.

[0120] The above method of first eliminating and then segmenting can avoid the influence of the obstacle point cloud data on the object segmentation of the third point cloud data, and can improve the accuracy of object segmentation.

[0121] The following uses the application scenario of robot environment perception and obstacle avoidance as an example to elaborate on the above steps S101 to S104. Figure 4 As shown, it is a schematic diagram of the process of the robot performing environment perception and obstacle avoidance, including steps S401 to S406:

[0122] S401: The robot collects first point cloud data through a sensor installed on the top of the robot gantry.

[0123] S402: Based on the plane features of the shelf, first target point cloud data of a preset plane of the shelf is fitted and segmented in the first point cloud data.

[0124] S403: Determine whether the distance from the first point cloud data to the preset plane between the preset planes of two adjacent shelves exceeds at least the preset distance; if yes, execute step S404; if not, loop through step S401.

[0125] S404: Clustering the first point cloud data that exceeds at least a preset distance, and performing object segmentation to obtain second target point cloud data of other objects.

[0126] S405: other objects that are beyond the preset distance but less than the preset threshold are regarded as the first obstacles on the shelf; other objects that are beyond the preset distance but greater than or equal to the preset threshold are regarded as the second obstacles on the ground and / or suspended above the aisle between adjacent shelves.

[0127] S406: Sending the position information of the first obstacle and / or the second obstacle in the target environment to the robot, so that the robot performs obstacle avoidance processing based on the position information of the obstacles.

[0128] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0129] Based on the same inventive concept, a detection device corresponding to the detection method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the detection device in the embodiment of the present disclosure is similar to the above-mentioned detection method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0130] Reference Figure 5 , which is a schematic diagram of a detection device provided by an embodiment of the present disclosure, the device comprises: a data acquisition module 501, a screening and segmentation module 502, an information determination module 503 and an object screening module 504; wherein,

[0131] A data acquisition module 501 is used to acquire first point cloud data in a target environment and target geometric features of a target object;

[0132] A screening and segmentation module 502 is used to screen first target point cloud data belonging to the target object from the first point cloud data based on the target geometric features, and perform object segmentation on the first point cloud data other than the first target point cloud data to obtain second target point cloud data of other objects other than the target object;

[0133] An information determination module 503, configured to determine positional relationship information between each other object and the target object based on the second target point cloud data and the first target point cloud data;

[0134] The object screening module 504 is configured to screen, based on the position relationship information, a first obstacle located on the target object and beyond the target object by at least a preset distance from the other objects.

[0135] In an optional implementation, the screening and segmentation module 502 is used to obtain a pre-set fitting range of the target geometric features; based on the fitting range and the target geometric features, fit the first point cloud data belonging to the target object to determine the first target point cloud data.

[0136] In an optional implementation manner, the other objects include objects located between two adjacent target objects;

[0137] The screening and segmentation module 502 is used to screen second point cloud data located between two adjacent target objects from the first point cloud data based on the first target point cloud data; and perform object segmentation on the second point cloud data to obtain second target point cloud data of other objects except the target object.

[0138] In an optional implementation, the screening and segmentation module 502 is used to determine the edge position information of the target object based on the first target point cloud data; based on the edge position information, eliminate the point cloud data that is not located between two adjacent target objects from the first point cloud data, and use the remaining point cloud data after elimination as the third point cloud data; perform object segmentation on the third point cloud data to obtain the second target point cloud data of other objects except the target object.

[0139] In an optional implementation, the position relationship information includes at least one of the following: the other object is on the target object; the other object is located on the target object and exceeds the target object; the other object is not on the target object.

[0140] In an optional embodiment, the object screening module 504 is used to determine the length of the other object exceeding the target object in a preset direction when the position relationship information indicates that the other object is located on the target object and exceeds the target object; when the length is greater than the preset distance, the other object is regarded as the first obstacle.

[0141] In an optional embodiment, the object screening module 504 is also used to, after determining the positional relationship information of each other object with the target object based on the second target point cloud data and the first target point cloud data, treat other objects outside the target object as second obstacles when the positional relationship information indicates that the other object is not on the target object.

[0142] In an optional embodiment, the target object includes a shelf; the target geometric feature includes a feature of a preset plane of the shelf; and the preset plane includes a plane of the shelf that is perpendicular to the goods import and export direction and / or the horizontal plane.

[0143] In an optional implementation, the target object includes the ground; the target geometric features include ground plane features; the object screening module 504 is further used to screen a second obstacle located on the ground from the other objects based on positional relationship information between the other objects and the ground.

[0144] In an optional embodiment, the detection device also includes an obstacle avoidance processing module 505, which is used to determine the first position information of the first obstacle in the target environment based on the second target point cloud data of the first obstacle after screening out the first obstacle located on the target object and beyond a preset distance from the target object from the other objects based on the position relationship information, so that the mobile device avoids the first obstacle based on the first position information; and / or, based on the second target point cloud data of the second obstacle, determine the second position information of the second obstacle in the target environment, so that the mobile device avoids the second obstacle based on the second position information.

[0145] For descriptions of the processing flow of each module in the detection device and the interaction flow between each module, reference may be made to the relevant descriptions in the above detection method embodiment, which will not be described in detail here.

[0146] The present disclosure also provides a computer device, such as Figure 6 FIG. 1 is a schematic diagram of a computer device structure provided by an embodiment of the present disclosure, including:

[0147] A processor 61 and a memory 62; the memory 62 stores machine-readable instructions executable by the processor 61, and the processor 61 is used to execute the machine-readable instructions stored in the memory 62. When the machine-readable instructions are executed by the processor 61, the processor 61 performs the following steps:

[0148] S101: Acquire first point cloud data in a target environment and target geometric features of a target object.

[0149] S102: Filter first target point cloud data belonging to a target object from the first point cloud data based on target geometric features, and perform object segmentation on the first point cloud data other than the first target point cloud data to obtain second target point cloud data of other objects other than the target object.

[0150] S103: Determine positional relationship information between each other object and the target object based on the second target point cloud data and the first target point cloud data.

[0151] S104: Based on the position relationship information, a first obstacle located on the target object and beyond the target object by at least a preset distance is selected from other objects.

[0152] The above-mentioned memory 62 includes internal memory 621 and external memory 622; the memory 621 here is also called internal memory, which is used to temporarily store the calculation data in the processor 61, as well as the data exchanged with the external memory 622 such as the hard disk. The processor 61 exchanges data with the external memory 622 through the internal memory 621.

[0153] The specific execution process of the above instructions can refer to the steps of the detection method described in the embodiment of the present disclosure, which will not be repeated here.

[0154] The present disclosure also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the detection method described in the above method embodiment are executed. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0155] The computer program product of the detection method provided in the embodiments of the present disclosure includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the steps of the detection method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0156] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0157] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0158] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0159] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0160] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0161] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A detection method, characterized in that: include: Acquire first point cloud data in a target environment and target geometric features of a target object; Based on the target geometric features, first target point cloud data belonging to the target object is selected from the first point cloud data, and object segmentation is performed on the first point cloud data other than the first target point cloud data to obtain second target point cloud data of other objects other than the target object; Based on the second target point cloud data and the first target point cloud data, determining positional relationship information between each other object and the target object; the positional relationship information includes at least one of the following: the other object is on the target object; The other object is located on the target object and exceeds the target object; the other object is not on the target object; Based on the position relationship information, a first obstacle located on the target object and beyond the target object by at least a preset distance is screened from the other objects.

2. The method according to claim 1, characterized in that The step of screening first target point cloud data belonging to the target object from the first point cloud data based on the target geometric features includes: Obtaining a preset fitting range of the target geometric feature; Based on the fitting range and the target geometric features, first point cloud data belonging to the target object is fitted to determine the first target point cloud data.

3. The method according to claim 1 or 2, characterized in that: The other objects include objects located between two adjacent target objects; The performing object segmentation on the first point cloud data except the first target point cloud data to obtain second target point cloud data of other objects except the target object includes: Based on the first target point cloud data, screening second point cloud data located between two adjacent target objects from the first point cloud data; The second point cloud data is segmented to obtain second target point cloud data of objects other than the target object.

4. The method according to claim 1 or 2, characterized in that: The performing object segmentation on the first point cloud data except the first target point cloud data to obtain second target point cloud data of other objects except the target object includes: Determining edge position information of the target object based on the first target point cloud data; Based on the edge position information, point cloud data that is not located between two adjacent target objects is removed from the first point cloud data, and the remaining point cloud data after the removal is used as the third point cloud data; The third point cloud data is subjected to object segmentation to obtain second target point cloud data of objects other than the target object.

5. The method according to claim 1, characterized in that The step of screening, based on the position relationship information, a first obstacle located on the target object and at least a preset distance beyond the target object from the other objects comprises: When the position relationship information indicates that the other object is located on the target object and exceeds the target object, determining a length by which the other object exceeds the target object in a preset direction; When the length is greater than the preset distance, the other object is regarded as the first obstacle.

6. The method according to claim 1, characterized in that After determining the positional relationship information between each other object and the target object based on the second target point cloud data and the first target point cloud data, the method further includes: When the position relationship information indicates that the other object is not on the target object, the other object outside the target object is regarded as a second obstacle.

7. The method according to claim 1 or 2, characterized in that: The target object includes a shelf; the target geometric feature includes a feature of a preset plane of the shelf; the preset plane includes a plane of the shelf that is perpendicular to a cargo import and export direction and / or a horizontal plane.

8. The method according to claim 7, characterized in that The target object includes the ground; the target geometric features include ground plane features; The detection method further comprises: Based on the positional relationship information between the other objects and the ground, a second obstacle located on the ground is screened from the other objects.

9. The method according to claim 6, characterized in that After selecting a first obstacle located on the target object and beyond a preset distance from the target object from the other objects based on the position relationship information, the method further includes: Determine, based on the second target point cloud data of the first obstacle, first position information of the first obstacle in the target environment, so that the mobile device avoids the first obstacle based on the first position information; and / or Based on the second target point cloud data of the second obstacle, second position information of the second obstacle in the target environment is determined, so that the mobile device avoids the second obstacle based on the second position information.

10. A detection device, characterized in that: include: A data acquisition module, used to acquire first point cloud data in a target environment and target geometric features of a target object; a screening and segmentation module, configured to screen first target point cloud data belonging to the target object from the first point cloud data based on the target geometric features, and perform object segmentation on the first point cloud data other than the first target point cloud data to obtain second target point cloud data of other objects other than the target object; An information determination module, configured to determine positional relationship information between each other object and the target object based on the second target point cloud data and the first target point cloud data; The position relationship information includes at least one of the following: the other object is on the target object; The other object is located on the target object and exceeds the target object; the other object is not on the target object; The object screening module is used to screen a first obstacle located on the target object and beyond the target object by at least a preset distance from the other objects based on the position relationship information.

11. A computer device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory. When the machine-readable instructions are executed by the processor, the processor executes the steps of the detection method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program. When the computer program is executed by a computer device, the computer device executes the steps of the detection method according to any one of claims 1 to 9.

Images