Target detection methods, equipment and storage media

Abstract

This application discloses a target object detection method, device, and storage medium. The method includes: acquiring a target point cloud, which is obtained by using a detection device to collect data from a detection area; performing dimensionality reduction mapping on the target point cloud to obtain two-dimensional point data, wherein the two-dimensional points in the two-dimensional point data are distributed in rows and columns; clustering the two-dimensional points in the two-dimensional point data to obtain clustering results; and determining the point cloud belonging to the target object based on the clustering results. Through the above methods, this application can improve the detection efficiency of target objects.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus and storage medium for detecting a target object. Background Technology

[0002] In certain scenarios, especially in vehicle driving, it is necessary to detect obstacles or signs on the road to ensure driving safety, enabling the vehicle to be controlled safely based on the detected information. Specifically, when a target obstacle is present on the road, quickly detecting the obstacle and its location can promptly alert the driver or provide more time for effective obstacle avoidance.

[0003] Therefore, the ability to quickly detect target objects is crucial for safe driving. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide a target object detection method, device, and storage medium that can improve the efficiency of target object detection.

[0005] To address the aforementioned technical problems, this application provides a target object detection method, comprising: acquiring a target point cloud, wherein the target point cloud is obtained by using a detection device to collect data from a detection area; performing dimensionality reduction mapping on the target point cloud to obtain two-dimensional point data, wherein the two-dimensional points contained in the two-dimensional point data are distributed in rows and columns; clustering each two-dimensional point in the two-dimensional point data to obtain a clustering result; and determining the point cloud belonging to the target object in the target point cloud based on the clustering result.

[0006] Among them, the two-dimensional point data is based on the horizontal and vertical viewpoints of the detection device as coordinate axes; and / or, the two-dimensional point data is a two-dimensional image.

[0007] The process of dimensionality reduction mapping of the target point cloud to obtain two-dimensional point data includes: for each target point in the target point cloud, using the first direction viewpoint and first direction resolution of the detection device, determining the target point mapping column, and using the second direction viewpoint and second direction resolution of the detection device, determining the target point mapping row, where one of the first and second directions is the horizontal direction and the other is the vertical direction, and the intersection of the target point mapping row and mapping column is defined as a two-dimensional point; for each two-dimensional point, based on the detection data of the target point corresponding to the two-dimensional point, obtaining the data value corresponding to the two-dimensional point in the two-dimensional point data, where the detection data of the target point comes from the target point cloud.

[0008] Before determining the target point mapping column using the first direction viewpoint and first direction resolution of the detection device, and determining the target point mapping row using the second direction viewpoint and second direction resolution of the detection device, the method further includes: obtaining the target point coordinates from the target point cloud, the target point coordinates including first axis coordinates, second axis coordinates, and third axis coordinates, the third axis being parallel to the second direction; determining the target point mapping column using the first direction viewpoint and first direction resolution of the detection device includes: determining the target point mapping column using the first axis coordinates, second axis coordinates, first direction viewpoint, and first direction resolution of the target point; determining the target point mapping column using the second direction viewpoint and second direction resolution of the detection device includes: determining the target point mapping column using the third axis coordinates, second direction viewpoint, and second direction resolution of the target point.

[0009] Specifically, the mapping column of the target point is determined using the first axis coordinate, the second axis coordinate, the viewing angle in the first direction, and the resolution in the first direction. This includes: subtracting the ratio between the first axis coordinate and the second axis coordinate of the target point from the maximum viewing angle in the first direction to obtain a first difference; and determining the first ratio or a neighboring integer of the first ratio between the first difference and the resolution in the first direction as the mapping column of the target point. The mapping row of the target point is determined using the third axis coordinate, the viewing angle in the second direction, and the resolution in the second direction. This includes: subtracting the ratio between the third axis coordinate and the distance in the first direction of the target point from the maximum viewing angle in the second direction to obtain a second difference; and determining the second ratio or a neighboring integer of the second ratio between the second difference and the resolution in the second direction as the mapping row of the target point.

[0010] Wherein, the first direction is the horizontal direction, and the second direction is the vertical direction; and / or, based on the detection data of the target point corresponding to the two-dimensional point, the data value corresponding to the two-dimensional point in the two-dimensional point data is obtained, including: selecting a target point that meets the preset distance requirement from at least one target point corresponding to the two-dimensional point, and using the detection data of the selected target point as the data value corresponding to the two-dimensional point.

[0011] The clustering results include several clusters of two-dimensional points. Based on the clustering results, the point clouds belonging to the target objects in the target point cloud are determined, including: for each cluster, the target points corresponding to each two-dimensional point in the cluster are found from the target point cloud as candidate point clouds corresponding to the cluster, and each cluster's candidate point cloud corresponds to an object; from each cluster's candidate point cloud, candidate point clouds whose feature factors match the characteristics of the target object are found as the target object's point cloud, and the feature factors are used to characterize the features of the objects corresponding to the candidate point clouds.

[0012] The feature factors include at least one of the height of the object corresponding to the candidate point cloud, the height of the object above the ground, and the verticality of the candidate point cloud; and / or, before finding candidate point clouds whose feature factors match the characteristics of the target object from the candidate point clouds corresponding to each cluster, the method further includes at least one of the following steps to obtain the feature factors: using the candidate point cloud to determine the bounding box of the corresponding object, obtaining the height of the bounding box as the height of the object corresponding to the candidate point cloud; obtaining the distance between the bottom surface of the bounding box and the ground to obtain the height of the object corresponding to the candidate point cloud above the ground; obtaining the point cloud covariance matrix of the candidate point cloud, performing eigenvalue decomposition on the point cloud covariance matrix to obtain the eigenvector, and obtaining the verticality of the candidate point cloud based on the eigenvector.

[0013] Among them, the target point cloud is a point cloud that is suspected to belong to the target object, which is extracted from the original point cloud collected by the detection device on the detection area. The target object includes the ground, and the target point cloud includes the ground point cloud that belongs to the ground. The distance between the bottom surface of the bounding box and the ground is obtained, including: determining the ground plane parameters using the ground point cloud, which is the point cloud that belongs to the ground in the original point cloud; and calculating the distance between the bottom surface and the ground using the coordinates of the preset points in the bottom surface and the ground plane parameters.

[0014] Before determining the ground plane parameters using the ground point cloud, the method further includes: segmenting the ground point cloud in a preset direction to obtain sub-ground point clouds corresponding to several sub-regions; determining the ground plane parameters using the ground point cloud, including: for each sub-region, determining the ground plane parameters of the sub-region based on the sub-ground point cloud in the sub-region; calculating the distance between the bottom surface and the ground using the coordinates of a preset point in the bottom surface and the ground plane parameters, including: taking the sub-region to which the preset point belongs as a reference sub-region, and calculating the distance between the bottom surface and the ground using the coordinates of the preset point and the ground plane parameters of the reference sub-region.

[0015] Wherein, the target object is a cone; and / or, acquiring the target point cloud includes: acquiring the original point cloud collected by the detection device on the detection area; and extracting the target point cloud suspected to belong to the target object from the original point cloud.

[0016] The process of extracting target point clouds that are suspected to belong to the target object from the original point cloud includes: performing semantic segmentation on the original point cloud using a semantic segmentation model to obtain several categories of point clouds, including ground and target objects; and selecting point clouds belonging to the target object category from the several categories of point clouds as target point clouds.

[0017] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide an electronic device, including a memory and a processor coupled to each other, wherein the memory stores program instructions; and the processor is used to execute the program instructions stored in the memory to implement the above-mentioned method.

[0018] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide a computer-readable storage medium for storing program instructions that can be executed to implement the above-mentioned method.

[0019] The above-described scheme, after acquiring the target point cloud, does not directly cluster the target point cloud. Instead, it first performs dimensionality reduction mapping on the target point cloud to obtain two-dimensional point data with row and column distribution. Then, it clusters the two-dimensional points in the two-dimensional point data to obtain the clustering results. Compared to directly clustering the target point cloud, the two-dimensional point clustering method described in this application can effectively reduce the number of points to be clustered, thus improving the clustering efficiency and consequently the target detection efficiency. Attached Figure Description

[0020] Figure 1 This is a schematic flowchart of an embodiment of the target object detection method provided in this application;

[0021] Figure 2 yes Figure 1 The flowchart of step S12 shown is a schematic diagram of one embodiment;

[0022] Figure 3 yes Figure 1 The flowchart of an embodiment of step S14 is shown below;

[0023] Figure 4 This is a schematic diagram of the framework of an embodiment of the electronic device provided in this application;

[0024] Figure 5 This is a schematic diagram of the framework of the computer-readable storage medium provided in this application. Detailed Implementation

[0025] To make the purpose, technical solution and effects of this application clearer and more explicit, the following describes this application in further detail with reference to the accompanying drawings and embodiments.

[0026] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0027] Please see Figure 1 , Figure 1 This is a schematic flowchart of an embodiment of the target object detection method provided in this application. It should be noted that if substantially the same result is obtained, this embodiment does not necessarily reflect that outcome. Figure 1 The illustrated process sequence is limited. For example... Figure 1 As shown, this embodiment includes:

[0028] S11: Acquire the target point cloud, which is obtained by using a detection device to collect data from the detection area.

[0029] This embodiment is used to perform dimensionality reduction mapping on the collected target point cloud, reducing the three-dimensional point cloud to two-dimensional points, so as to quickly obtain the clustering results by clustering a small number of points, and then quickly determine the point cloud belonging to the target object.

[0030] The target object in this paper can be any object to be detected. For example, in a vehicle driving scenario, the target object can be traffic signs on the road, such as traffic cones; of course, it can also be other obstacles on the road, such as other vehicles, pedestrians, etc. The detection area in this paper can be the maximum detection area of ​​the detection device, or it can be a region of interest (ROI) selected from the maximum detection area where the target object may appear. The detection device in this paper can be a lidar, or it can be a millimeter-wave radar, etc. Lidar can accurately measure the geometric features of an object, and when the geometric features of the target object are significantly different from other non-target objects, lidar can be preferred for detection.

[0031] In some embodiments, acquiring the target point cloud includes: first acquiring the original point cloud collected by the detection device from the detection area; and then extracting the target point cloud suspected to belong to the target object from the acquired original point cloud. The original point cloud is the point cloud corresponding to each object existing in the detection area collected by the detection device.

[0032] In one specific implementation, extracting target point clouds suspected of belonging to the target object from the acquired original point cloud includes: firstly, performing semantic segmentation on the original point cloud using a semantic segmentation model to obtain several categories of point clouds, where the categories include ground and target objects; then, selecting point clouds belonging to the target object category from the several categories of point clouds as target point clouds. That is, firstly, performing semantic segmentation on the original point cloud using a semantic segmentation model to determine point clouds belonging to the ground and point clouds suspected of being target objects, and then selecting target point clouds belonging to the target object category from the point clouds belonging to the ground and the point clouds suspected of being target objects. In this implementation, the semantic segmentation model can quickly eliminate point clouds that clearly do not belong to the target object, or can quickly eliminate point clouds that are unfavorable to determining the target object, which is beneficial for quickly identifying point clouds belonging to the target object from the target point cloud subsequently.

[0033] In one specific embodiment, the detection device is a lidar. In order to make the semantic segmentation model perform more accurate semantic segmentation on the original point cloud and obtain accurate segmentation results (semantic segmentation model), the coordinate information of the original point cloud and the reflectivity of each point in the original point cloud can be input into the semantic segmentation model. Since different objects reflect different light waves when the laser hits an object, different objects can be distinguished by reflectivity. Therefore, when the semantic segmentation model processes the original point cloud containing reflectivity, it is easy to obtain accurate segmentation results.

[0034] In another specific implementation, a random sampling consensus algorithm can be used to segment the original point cloud into ground points, resulting in ground point clouds. The remaining points in the original point cloud are considered obstacle point clouds. Then, based on the height information of the target object point cloud, points in the obstacle point cloud that meet the target object's height are extracted as potential target point clouds. That is, points with heights similar to the target object's height are extracted from the obstacle point cloud and included in the potential target point cloud.

[0035] It should be noted that in some embodiments, the target point cloud only includes point clouds that are suspected to be target objects; in other embodiments, the target point cloud includes not only point clouds that are suspected to be target objects, but also ground point clouds belonging to the ground. The specific target point cloud can be determined according to actual needs. For example, in some scenarios, it is necessary to filter target objects from obstacles based on the distance between obstacles and the ground. In this case, the obtained target point cloud may include point clouds that are suspected to be target objects and ground point clouds belonging to the ground.

[0036] S12: Perform dimensionality reduction mapping on the target point cloud to obtain two-dimensional point data, in which the two-dimensional points are distributed in rows and columns.

[0037] It should be noted that before performing dimensionality reduction mapping on the target point cloud to obtain two-dimensional point data, a two-dimensional mapping space should be determined first. Then, the target point cloud is mapped to this two-dimensional mapping space to obtain the corresponding two-dimensional point data. In this embodiment, the two-dimensional points contained in the two-dimensional point data are distributed in rows and columns in the two-dimensional mapping space, that is, the two-dimensional points have row and column information in the two-dimensional mapping space.

[0038] In one embodiment, a two-dimensional mapping space can be obtained by using the horizontal and vertical viewing angles of the detection device as coordinate axes, respectively. That is, the two-dimensional point data is obtained by using the horizontal and vertical viewing angles of the detection device as coordinate axes. Of course, in other embodiments, other reference objects can be used to determine the coordinate axes in two directions to obtain the two-dimensional mapping space. For example, in a vehicle driving scenario, the vehicle can be used as a reference object, with the vehicle's direction of travel and the direction perpendicular to the direction of travel as coordinate axes to obtain the two-dimensional mapping space.

[0039] In one embodiment, the obtained two-dimensional point data is a two-dimensional image. That is, the target point cloud is mapped to a two-dimensional image. It can be understood that when the target point cloud is mapped to a two-dimensional image, the subsequent step is to cluster each two-dimensional point in the two-dimensional image to obtain the corresponding clustering results.

[0040] Taking two-dimensional point data as a two-dimensional image as an example, in some embodiments, the size of the two-dimensional image can be determined according to the horizontal and vertical viewing angles of the detection device. Further, the size of each pixel in the two-dimensional image can be determined according to the horizontal and vertical resolutions of the detection device. After determining the size of the two-dimensional image and the size of the pixels, for each target point, the mapping rows and columns of the dimensionality-reduced two-dimensional points (i.e., the positions of the two-dimensional points in the two-dimensional image) can be determined according to the target point, the horizontal viewing angle and corresponding resolution of the detection device, and the vertical viewing angle and corresponding resolution of the detection device. After determining the mapping rows and columns of the two-dimensional points, the position corresponding to the intersection of any mapping row and column may contain multiple target points in the target point cloud. In order to facilitate subsequent clustering using fewer target points, target points that meet the preset requirements can be selected from the detection data of the target points corresponding to the two-dimensional points. Then, the detection data corresponding to the selected target points can be used as the data values ​​corresponding to the two-dimensional points.

[0041] Specifically, please refer to Figure 2 , Figure 2 yes Figure 1 The flowchart shown is a schematic diagram of one embodiment of step S12. It should be noted that if substantially the same result is achieved, this embodiment does not necessarily follow the same pattern. Figure 2 The illustrated process sequence is limited. For example... Figure 2 As shown, this embodiment includes:

[0042] S21: For each target point in the target point cloud, the mapping column of the target point is determined by using the first direction view and the first direction resolution of the detection device, and the mapping row of the target point is determined by using the second direction view and the second direction resolution of the detection device. One of the first direction and the second direction is the horizontal direction and the other is the vertical direction. The intersection of the mapping row and the mapping column of the target point is defined as a two-dimensional point.

[0043] In summary, this embodiment determines the mapping rows and columns of the dimensionality-reduced two-dimensional points based on the coordinates of the target point, the horizontal viewing angle and corresponding resolution of the detection device, and the vertical viewing angle and corresponding resolution of the detection device. Before determining the mapping rows and columns of the target point, the coordinates of the target point should be obtained from the target point cloud. The coordinates of the target point are three-dimensional coordinates, including a first axis coordinate, a second axis coordinate, and a third axis coordinate, wherein the third axis is parallel to the second direction. In a specific embodiment, the first direction of the detection device can be pre-set as horizontal, the second direction as vertical, and the third axis coordinate of the target point as the coordinates of the point in the vertical direction.

[0044] In this embodiment, determining the target point mapping column using the first direction viewing angle and the first direction resolution of the detection device includes: determining the target point mapping column using the first axis coordinates, the second axis coordinates, the first direction viewing angle, and the first direction resolution of the target point.

[0045] Specifically, the mapping column for determining the target point can be found in the following formula:

[0046] Col = (Fov left -P y / P x ) / H ratio

[0047] In the formula, Col represents the target point mapping column, and Fov left P represents the maximum field of view in the first direction. y / P x H represents the ratio between the first axis coordinate and the second axis coordinate of the target point. ratio This indicates the resolution in the first direction.

[0048] In this embodiment, the first difference is obtained by subtracting the ratio between the first axis coordinate and the second axis coordinate of the target point from the maximum viewing angle in the first direction. Then, the first ratio between the first difference and the resolution in the first direction, or the nearest integer of the first ratio, is determined as the mapping column of the target point.

[0049] In addition, the mapping line of the target point is determined by using the second-direction viewing angle and the second-direction resolution of the detection device, including: determining the mapping line of the target point by using the third-axis coordinates of the target point, the second-direction viewing angle, and the second-direction resolution.

[0050] Specifically, the mapping row for the target point can be determined using the following formula:

[0051] Row = (Fov) up -P z / P range ) / V ratio

[0052] In the formula, Row represents the mapping row of the target point, and Fov up P represents the maximum field of view in the second direction. z P represents the third axis coordinate of the target point. range V represents the distance in the first direction from the target point. ratio Resolution in the second direction.

[0053] In this embodiment, the maximum viewing angle in the second direction is first subtracted from the ratio between the third axis coordinate of the target point and the distance in the first direction of the target point to obtain the second difference. Then, the second ratio between the second difference and the resolution in the second direction, or the nearest integer of the second ratio, is determined as the mapping row of the target point.

[0054] S22: For each two-dimensional point, based on the detection data of the target point corresponding to the two-dimensional point, the data value corresponding to the two-dimensional point in the two-dimensional point data is obtained. The detection data of the target point comes from the target point cloud.

[0055] In this embodiment, after obtaining the mapping row and mapping column of the target point, the intersection area of ​​the same mapping row and mapping column may include multiple target points. In order to facilitate subsequent clustering using fewer target points, in this embodiment, the target point that meets the preset distance requirement can be selected from at least one target point corresponding to the two-dimensional point, and the detection data of the selected target point can be used as the data value corresponding to the two-dimensional point.

[0056] In this system, each target point in the intersecting area has a corresponding distance value from the center of the intersecting area. In one embodiment, the target point that meets the preset distance requirement is the one with the largest distance value among the target points; in another embodiment, the target point that meets the preset distance requirement is the one with the smallest distance value among the target points; of course, in other embodiments, the target point that meets the preset distance requirement can also be the one whose distance value is closest to the average distance value of all target points, and the specific distance requirement can be set according to actual needs. In addition, the detection data of the target point can be determined according to the specific detection device. For example, if the detection device is a lidar, the detection data includes point cloud coordinates and point cloud reflectivity, and the data value corresponding to a two-dimensional point includes two-dimensional point coordinates and reflectivity.

[0057] S13: Cluster the two-dimensional points in the two-dimensional point data to obtain the clustering results.

[0058] In one embodiment, a breadth-first search algorithm can be used to cluster the two-dimensional points in the two-dimensional point data to obtain clustering results; of course, other related clustering algorithms can also be used to cluster the two-dimensional points in the two-dimensional point data to obtain clustering results. The obtained clustering results include several clusters of two-dimensional points, and the two-dimensional points contained in each cluster belong to the same object.

[0059] In other embodiments, other clustering algorithms (such as the DBSCAN algorithm) can also be used to cluster the two-dimensional points in the two-dimensional point data to obtain the clustering results.

[0060] S14: Based on the clustering results, determine the point clouds in the target point cloud that belong to the target object.

[0061] In one embodiment, the clustering result obtained by clustering includes several clusters of two-dimensional points, and each cluster of two-dimensional points belongs to the same object. After obtaining several clusters of two-dimensional points, for each cluster, the target points corresponding to each two-dimensional point in the cluster can be found from the target point cloud, and the found target points can be used as the candidate point cloud corresponding to the cluster. Then, from the candidate point clouds corresponding to each cluster, candidate point clouds whose feature factors match the characteristics of the target object can be found as the point cloud of the target object, so as to use the characteristics of the target object to find the target object whose feature factors match the characteristics of the target object.

[0062] Specifically, please refer to Figure 3 , Figure 3 yes Figure 1 The diagram shows a flowchart of one embodiment of step S14. It should be noted that if substantially the same result is achieved, this embodiment does not necessarily follow the same pattern. Figure 3 The illustrated process sequence is limited. For example... Figure 3 As shown, this embodiment includes:

[0063] S31: For each cluster, find the target point corresponding to each two-dimensional point in the target point cloud and use it as the candidate point cloud for that cluster. Each candidate point cloud of a cluster corresponds to an object.

[0064] In this embodiment, the clustering results obtained by clustering include several clusters of two-dimensional points, with each cluster's two-dimensional points belonging to the same object. After obtaining the two-dimensional points of each cluster, the target points corresponding to each two-dimensional point in the cluster are found from the target point cloud, and the found target points are used as candidate point clouds corresponding to that cluster. This facilitates the subsequent determination of the target object using the feature factors of the candidate point clouds of each cluster. Each cluster's candidate point cloud corresponds to one object; for example, the candidate point cloud of cluster A corresponds to object A, and the candidate point cloud of cluster B corresponds to object B.

[0065] S32: From the candidate point clouds corresponding to each cluster, find the candidate point clouds whose feature factors match the features of the target object, and use them as the point clouds of the target object. The feature factors are used to characterize the features of the object corresponding to the candidate point clouds.

[0066] In this embodiment, feature factors are used to characterize the features of the object corresponding to the candidate point cloud. These feature factors include at least one of the following: the height of the object corresponding to the candidate point cloud, the height of the object above the ground, and the verticality of the candidate point cloud. The height of the object represents the height of the object itself; the height of the object above the ground represents the distance between the bottom surface of the object and the ground; and the verticality of the candidate point cloud represents the distribution of the candidate point cloud along different axes.

[0067] In one embodiment, the feature factor includes the height of the object corresponding to the candidate point cloud. In this embodiment, objects whose height matches the height of the target object can be filtered based on the object's height information, and the target object can then be determined from the filtered objects. In another embodiment, the feature factor includes the height of the object above the ground. In this embodiment, objects whose height above the ground matches the height of the target object above the ground can be filtered based on the object's height above the ground, and the target object can then be determined from the filtered objects. In yet another embodiment, the feature factor includes the verticality of the candidate point cloud. In this embodiment, objects whose verticality matches the verticality of the target object can be filtered based on the object's verticality along the target axis, and the target object can then be determined from the filtered objects.

[0068] In one specific embodiment, the feature factors include the height of the object corresponding to the candidate point cloud, the height of the object above the ground, and the verticality of the candidate point cloud. Candidate point clouds whose feature factors match the characteristics of the target object can be sequentially selected based on a preset priority order of object height, object height above the ground, and candidate point cloud verticality, and thus used as the target object's point cloud. For example, preliminary selection can be performed using the object's height above the ground, followed by further selection using the object's height, and finally, the target object can be determined using the verticality of the candidate point cloud.

[0069] The above-described scheme, after acquiring the target point cloud, does not directly cluster the target point cloud. Instead, it first performs dimensionality reduction mapping on the target point cloud to obtain two-dimensional point data with row and column distribution. Then, it clusters the two-dimensional points in the two-dimensional point data to obtain the clustering results. Compared to directly clustering the target point cloud, the two-dimensional point clustering method described in this application can effectively reduce the number of points to be clustered, thus improving the clustering efficiency and consequently the target detection efficiency.

[0070] In one embodiment, before selecting candidate point clouds whose feature factors match the characteristics of the target object from the candidate point clouds corresponding to each cluster, the feature factors must be obtained. Obtaining the feature factors includes at least one of the following steps:

[0071] First, the bounding box of the corresponding object is determined using the candidate point cloud, and the height of the bounding box is obtained. The height of the bounding box is the height of the object corresponding to the candidate point cloud.

[0072] After determining the candidate point cloud, the bounding box of the corresponding object and the height of the bounding box can be determined based on the three-dimensional coordinates of each point in the candidate point cloud. The height of the bounding box is the height of the object corresponding to the candidate point cloud.

[0073] Second, obtain the distance between the bottom of the bounding box and the ground to get the height of the object corresponding to the candidate point cloud above the ground.

[0074] In one embodiment, the target point cloud is a point cloud suspected to belong to a target object, extracted from the original point cloud collected by the detection device from the detection area. The target object includes the ground, and the target point cloud includes ground point clouds belonging to the ground. For example, a semantic segmentation model can be used to perform semantic segmentation on the original point cloud to obtain ground point clouds belonging to the ground.

[0075] After acquiring the ground point cloud, the ground plane parameters are determined using the ground point cloud. The distance between the bottom surface of the object's bounding box and the ground plane parameters is calculated using the coordinates of a preset point on the bottom surface of the object's bounding box. The preset point on the bottom surface of the object's bounding box can be any point on the bottom surface of the object's bounding box, or a point adjacent to the centroid coordinates of the bottom surface point; there are no specific restrictions. The ground plane parameters are represented by the equation of the ground plane. After determining the ground plane parameters and the preset point on the bottom surface of the object's bounding box, the distance between the bottom surface of the object's bounding box and the ground can be obtained by calculating the distance from the preset point to the plane.

[0076] Optionally, after obtaining the ground point cloud belonging to the ground, the ground plane parameters can be determined using the ground point cloud. The coordinates of the centroid coordinates and the height of the object bounding box can be used to determine the coordinates of the center point of the bottom surface of the object bounding box. The center point of the bottom surface is used as a preset point of the bottom surface. By calculating the distance from the preset point to the plane, the distance between the bottom surface of the object bounding box and the ground can be obtained.

[0077] In one implementation, the ground plane parameters can be determined by filtering the ground point cloud using a random sample consensus algorithm. For example, three ground points can be extracted at any time to construct a ground plane model Ax + By + Cz + D = 0. The proportion of ground points distributed within the model within a certain threshold is statistically analyzed using the ground plane model. This process is iterated multiple times to find the optimal ground plane model as the ground plane parameters. Alternatively, other correlation fitting algorithms can be used to fit the ground point cloud to obtain the ground plane parameters.

[0078] In some embodiments, considering that the ground has a slope, in order to accurately obtain the distance between the object and the ground, the ground point cloud can be segmented in a preset direction before using the ground point cloud to determine the ground plane parameters, resulting in sub-ground point clouds corresponding to several sub-regions. The preset direction is the vehicle's forward direction.

[0079] In this embodiment, determining ground plane parameters using ground point clouds includes: for each sub-region, determining the ground plane parameters of the sub-region based on the sub-ground point clouds within the sub-region. The ground plane parameters of each sub-region can be determined using the method described above.

[0080] In addition, after determining multiple sub-regions, the sub-region to which the preset point in the bottom surface of the object's bounding box belongs can be used as a reference sub-region, and the distance between the bottom surface and the ground can be calculated using the coordinates of the preset point and the ground plane parameters of the reference sub-region.

[0081] Third, obtain the point cloud covariance matrix of the candidate point cloud, perform eigenvalue decomposition on the point cloud covariance matrix to obtain eigenvectors, and obtain the verticality of the candidate point cloud based on the eigenvectors.

[0082] In this embodiment, after determining the candidate point cloud, the point cloud covariance matrix is ​​calculated based on the coordinates of each point in the candidate point cloud. Then, eigenvalue decomposition is performed on the point cloud covariance matrix to obtain eigenvectors. The obtained eigenvectors are vectors composed of eigenvalues ​​along each axis, that is, the obtained eigenvectors contain the eigenvalues ​​of the candidate point cloud along each axis (including the vertical, horizontal, and longitudinal axes). The magnitude of the eigenvalues ​​along each axis indicates the distribution of the candidate point cloud along that axis. For example, if the eigenvalues ​​along the vertical axis are greater than the eigenvalues ​​along the two horizontal axes, it indicates that the candidate point cloud is more distributed near the vertical axis and less distributed near the horizontal axis, and the object is a slender object. Conversely, if the eigenvalues ​​along the vertical axis are less than the eigenvalues ​​along the horizontal axes, it indicates that the candidate point cloud is less distributed near the vertical axis and more distributed along the horizontal axis, and the object is a wide object.

[0083] In this embodiment, the perpendicularity of the candidate point cloud along the target axis can be obtained based on the feature vector. The perpendicularity along the target axis is the proportion of the feature value along the target axis to the sum of the feature values ​​along all axes. The target axis can be selected according to the actual situation of the target object. For example, if the target object is a long and slender object, the target axis can be determined as the vertical axis, and then the perpendicularity of the candidate point cloud along the target axis can be obtained.

[0084] In one specific implementation, the point cloud covariance matrix of the candidate point cloud is eigenvalued to obtain the eigenvector Vec = (E x E y E z The verticality of the candidate point cloud on the vertical axis is:

[0085]

[0086] In the formula, Ver represents verticality, and E z E represents the eigenvalues ​​on the vertical axis.x E represents the eigenvalues ​​on the horizontal axis. y The eigenvalues ​​are represented on the vertical axis.

[0087] It should be noted that after obtaining the feature factors using the above method, candidate point clouds whose feature factors match the characteristics of the target object are selected from the candidate point clouds corresponding to each cluster, and these candidate point clouds are used as the point clouds of the target object.

[0088] In this scenario, when the feature factors include the height of the objects corresponding to the candidate point clouds, candidate point clouds whose heights match the target object's height can be selected from the candidate point clouds corresponding to each object, based on the target object's actual height. This serves as the target object's point cloud, thus filtering out objects whose heights do not match the target object's height. Specifically, a height range can be set based on the target object's actual height, and objects whose heights are not within this range can be filtered out. For example, objects that are significantly taller or shorter than the target object can be filtered out, ultimately yielding the target object and its point cloud.

[0089] When the feature factor includes the object's height above the ground, candidate point clouds that match the target object's height can be selected from the candidate point clouds corresponding to each object, and these selected point clouds can be used as the target object's point cloud. In this way, objects whose height above the ground does not match the target object's height can be filtered out. For example, if the target object is a traffic cone placed close to the ground, other objects that are at a height distance from the ground, such as road fences, can be filtered out based on their height above the ground.

[0090] When the feature factor includes the verticality of the candidate point cloud, the candidate point cloud whose verticality meets the target object's verticality can be selected from the candidate point clouds corresponding to each object, based on the target object's verticality along the target axis. This serves as the target object's point cloud, thus filtering out objects whose verticality does not meet the target object's verticality requirement. Specifically, a verticality threshold can be set based on the target object's actual verticality, and objects whose verticality threshold does not meet this threshold can be filtered out to finally obtain the target object and its point cloud.

[0091] The specific factors included in the aforementioned feature factors can be set according to the actual detection scenario. For example, in autonomous driving scenarios, autonomous vehicles need to have the ability to recognize road traffic signs to cope with complex traffic situations. Road repair and maintenance construction sites are common scenarios, and these scenarios usually require the use of traffic cones to cordon off the construction area to provide safety warnings and advance notice to passing vehicles and pedestrians. Therefore, autonomous vehicles need to be able to accurately recognize signs such as traffic cones to ensure safe driving. In this scenario, to improve driving safety, the vehicle needs to be able to accurately recognize traffic cones.

[0092] In one embodiment, the target object is a cone. After acquiring the original point cloud collected by the detection device from the detection area, the original point cloud can be semantically segmented using a semantic segmentation model to obtain the target point cloud. The target point cloud includes point clouds that are suspected to belong to the cone. After obtaining the target point cloud, the target point clouds that are suspected to belong to the cone are subjected to dimensionality reduction mapping to obtain two-dimensional point data, which is a two-dimensional image. Then, each two-dimensional point in the two-dimensional image is clustered to obtain the clustering result, which includes several clusters of two-dimensional points. For each cluster, the target points corresponding to each two-dimensional point in the cluster are found from the target point cloud as candidate point clouds corresponding to the cluster. Each cluster's candidate point cloud corresponds to an object (each object suspected to be a cone), that is, the clustering result includes the result of each object suspected to be a cone. After finding the candidate point clouds corresponding to each cluster (each object), the candidate point clouds whose feature factors meet the cone features are found from the candidate point clouds corresponding to each cluster as the cone point cloud.

[0093] In one specific embodiment, the feature factors include the height of the object, the height of the object above the ground, and the verticality of the candidate point cloud. Although the cones have various heights, their heights are generally within a certain range. In addition, the cones used for safety warnings and announcements are generally placed on the ground, meaning there is no height difference between the cones and the ground. Furthermore, the cones are elongated objects, and their verticality in the vertical axis is greater than or equal to a preset verticality threshold.

[0094] In this embodiment, the first target object whose height is within the height range can be selected from the objects first through the above height range. Then, the second target object whose height distance from the ground is less than a preset threshold (a threshold set here to take into account calculation error) can be selected from the selected objects. Then, the objects with a verticality greater than or equal to the preset verticality threshold in the vertical axis of the second target object can be selected as cones.

[0095] Please see Figure 4 , Figure 4 This is a schematic diagram of a framework of an embodiment of the electronic device provided in this application. In this embodiment, the electronic device 40 includes a memory 41 and a processor 42 coupled to each other.

[0096] The memory 41 stores program instructions, and the processor 42 executes the program instructions stored in the memory 41 to implement the steps of any of the above-described method implementations. In a specific implementation scenario, the electronic device 40 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 40 may also include mobile devices such as laptops and tablets, which are not limited here.

[0097] Specifically, processor 42 controls itself and memory 41 to implement the steps of any of the above embodiments. Processor 42 may also be referred to as a CPU (Central Processing Unit). Processor 42 may be an integrated circuit chip with signal processing capabilities. Processor 42 may also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 42 may be implemented using integrated circuit chips.

[0098] Please see Figure 5 , Figure 5 This is a schematic diagram of the framework of the computer-readable storage medium provided in this application. The computer-readable storage medium 50 of this application embodiment stores program instructions 51, which, when executed, implement the methods provided in any embodiment and any non-conflicting combination of the above methods. The program instructions 51 can form a program file and be stored in the computer-readable storage medium 50 in the form of a software product, so that a computer device (which may be a personal computer, server, or network device, etc.) can execute all or part of the steps of the methods of various embodiments of this application. The aforementioned computer-readable storage medium 50 includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or terminal devices such as computers, servers, mobile phones, and tablets.

[0099] The above-described scheme, after acquiring the target point cloud, does not directly cluster the target point cloud. Instead, it first performs dimensionality reduction mapping on the target point cloud to obtain two-dimensional point data with row and column distribution. Then, it clusters the two-dimensional points in the two-dimensional point data to obtain the clustering results. Compared to directly clustering the target point cloud, the two-dimensional point clustering method described in this application can effectively reduce the number of points to be clustered, thus improving the clustering efficiency and consequently the target detection efficiency.

[0100] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0101] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for detecting a target object, characterized in that, The method includes: Acquire a target point cloud, which is obtained by using a detection device to collect data from the detection area; The target point cloud is subjected to dimensionality reduction mapping to obtain two-dimensional point data, wherein the two-dimensional points contained in the two-dimensional point data are distributed in rows and columns; Cluster the two-dimensional points in the two-dimensional point data to obtain the clustering results; Based on the clustering results, the point clouds belonging to the target object are determined in the target point cloud; The process of dimensionality reduction mapping of the target point cloud to obtain two-dimensional point data includes: For each target point in the target point cloud, the mapping column of the target point is determined using the first axis coordinates and the second axis coordinates of the target point, the viewing angle and the resolution of the first direction of the detection device, and the mapping row of the target point is determined using the third axis coordinates of the target point, the viewing angle and the resolution of the second direction of the detection device, where one of the first direction and the second direction is horizontal and the other is vertical, the third axis is parallel to the second direction, and the intersection of the mapping row and the mapping column of the target point is defined as the two-dimensional point. For each of the two-dimensional points, based on the detection data of the target point corresponding to the two-dimensional point, the data value corresponding to the two-dimensional point in the two-dimensional point data is obtained, and the detection data of the target point comes from the target point cloud; Wherein, the mapping column of the target point is a first ratio or a neighboring integer of the first ratio, the first ratio being the ratio between a first difference and the resolution in the first direction, the first difference being obtained by subtracting the ratio between the first axis coordinate and the second axis coordinate of the target point from the maximum viewing angle in the first direction; the mapping column of the target point is a second ratio or a neighboring integer of the second ratio, the second ratio being the ratio between a second difference and the resolution in the second direction, the second difference being obtained by subtracting the ratio between the third axis coordinate of the target point and the distance in the first direction of the target point from the maximum viewing angle in the second direction.

2. The method according to claim 1, characterized in that, The two-dimensional point data is based on the horizontal and vertical viewpoints of the detection device as coordinate axes, respectively. And / or, the two-dimensional point data is a two-dimensional image.

3. The method according to claim 1, characterized in that, Before determining the mapping column of the target point using the viewing angle and resolution of the first direction of the detection device, and determining the mapping row of the target point using the viewing angle and resolution of the second direction of the detection device, the method further includes: The coordinates of the target point are obtained from the target point cloud, and the coordinates of the target point include the first axis coordinate, the second axis coordinate, and the third axis coordinate.

4. The method according to claim 1, characterized in that, The first direction is horizontal, and the second direction is vertical; And / or, obtaining the data value corresponding to the two-dimensional point in the two-dimensional point data based on the detection data of the target point corresponding to the two-dimensional point includes: From at least one target point corresponding to the two-dimensional point, select the target point that meets the preset distance requirement, and use the detection data of the selected target point as the data value corresponding to the two-dimensional point.

5. The method according to claim 1, characterized in that, The clustering result includes several clusters of the two-dimensional points; the step of determining the point cloud belonging to the target object in the target point cloud based on the clustering result includes: For each cluster, the target point corresponding to each two-dimensional point in the cluster is found from the target point cloud and used as the candidate point cloud corresponding to the cluster. Each candidate point cloud of the cluster corresponds to an object. From the candidate point clouds corresponding to each cluster, the candidate point clouds whose feature factors match the features of the target object are selected as the point clouds of the target object. The feature factors are used to characterize the features of the object corresponding to the candidate point clouds.

6. The method according to claim 5, characterized in that, The feature factors include at least one of the following: the height of the object corresponding to the candidate point cloud, the height of the object above the ground, and the verticality of the candidate point cloud. And / or, before finding the candidate point cloud whose feature factors match the target object features from the candidate point clouds corresponding to each of the clusters, and using it as the point cloud of the target object, the method further includes at least one of the following steps to obtain the feature factors: The bounding box of the corresponding object is determined using the candidate point cloud, and the height of the bounding box is obtained as the height of the object corresponding to the candidate point cloud. Obtain the distance between the bottom surface of the bounding box and the ground to obtain the height of the object corresponding to the candidate point cloud above the ground; Obtain the point cloud covariance matrix of the candidate point cloud, perform eigenvalue decomposition on the point cloud covariance matrix to obtain eigenvectors, and obtain the verticality of the candidate point cloud based on the eigenvectors.

7. The method according to claim 6, characterized in that, The target point cloud is a point cloud that is suspected to belong to the target object, extracted from the original point cloud collected by the detection device from the detection area. The target object includes the ground, and the target point cloud includes ground point clouds belonging to the ground. The step of obtaining the distance between the bottom surface of the bounding box and the ground includes: Ground plane parameters are determined using ground point clouds, where the ground point clouds are the point clouds belonging to the ground in the original point cloud; The distance between the bottom surface and the ground is calculated using the coordinates of a preset point on the bottom surface and the ground plane parameters.

8. The method according to claim 7, characterized in that, Before determining the ground plane parameters using ground point clouds, the method further includes: The ground point cloud is segmented in a preset direction to obtain sub-ground point clouds corresponding to several sub-regions; The method of determining ground plane parameters using ground point clouds includes: For each sub-region, the ground plane parameters of the sub-region are determined based on the sub-ground point cloud in the sub-region; The step of calculating the distance between the bottom surface and the ground surface using the coordinates of a preset point on the bottom surface and the ground plane parameters includes: Using the sub-region to which the preset point belongs as a reference sub-region, the distance between the bottom surface and the ground is calculated using the coordinates of the preset point and the ground plane parameters of the reference sub-region.

9. The method according to claim 1, characterized in that, The target object is a cone-shaped container; And / or, the acquisition of the target point cloud includes: Acquire the raw point cloud data collected by the detection device on the detection area; Extract target point clouds that are suspected to belong to the target object from the original point cloud.

10. The method according to claim 9, characterized in that, Extracting target point cloud data suspected to belong to the target object from the original point cloud includes: The original point cloud is semantically segmented using a semantic segmentation model to obtain point clouds of several categories, including the ground and the target object. Select the point cloud belonging to the target object category from the aforementioned point clouds, and use it as the target point cloud.

11. An electronic device, characterized in that, Including interconnected memory and processor, The memory stores program instructions; The processor is used to execute program instructions stored in the memory to implement the method according to any one of claims 1-10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that can be executed by a processor to implement the method according to any one of claims 1-10.

Images