A clustering method and apparatus for point cloud data

Abstract

This application discloses a method and apparatus for clustering point cloud data. This method can be used in the field of point cloud data processing within the field of artificial intelligence. The method includes: rasterizing multiple points in the point cloud data to be processed based on the location information of each point, obtaining raster data corresponding to the point cloud data to be processed; obtaining the predicted category of each point in the point cloud data to be processed; determining the predicted category of a first raster based on the predicted category of each point located in a first raster included in the raster data to be processed; and performing a clustering operation based on the predicted category of each raster in the raster data to be processed, to obtain at least one point cloud cluster corresponding to the point cloud data to be processed. By utilizing both the location information and the predicted category of each point in the point cloud data to be processed, the entire clustering process can be completed, which helps to improve the accuracy of the entire clustering process.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more particularly to a method and apparatus for clustering point cloud data. Background Technology

[0002] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0003] Object detection is a common application in artificial intelligence. Point cloud data-based object detection algorithms are frequently used in intelligent systems such as autonomous driving and smart security. Specifically, after obtaining the point cloud data to be processed, clustering is performed on the point cloud data to obtain at least one point cloud cluster, and then a classification operation is performed based on at least one point cloud cluster.

[0004] Because the current clustering method is not very accurate, the overall accuracy of target detection is low. Therefore, a more accurate clustering method is urgently needed. Summary of the Invention

[0005] This application provides a method and apparatus for clustering point cloud data. It utilizes the location information of each point in the point cloud data to be processed and the predicted category of each point to complete the entire clustering process, which helps to improve the accuracy of the entire clustering process.

[0006] To address the aforementioned technical problems, this application provides the following technical solutions:

[0007] Firstly, embodiments of this application provide a clustering method for point cloud data, applicable to the processing of point cloud data in the field of artificial intelligence. The method may include: an electronic device performing rasterization processing on multiple points in the point cloud data to be processed based on the position information of each point in the point cloud data to be processed, obtaining raster data to be processed corresponding to the point cloud data to be processed; wherein, the point cloud data to be processed includes the position information of each of N points corresponding to the environment at a target time, i.e., the point cloud data to be processed is used to describe the environment at the target time; further, the point cloud data to be processed may be point cloud data collected by a single sensor; or it may be point cloud data obtained by fusing point cloud data collected by multiple sensors; the raster data to be processed includes the raster coordinates of each point in the point cloud data to be processed in a target raster map, and may also include the correspondence between the coordinates of each point in the point cloud data to be processed and the raster coordinates of each point in the target raster map. The electronic device obtains the predicted category of each point in the point cloud data to be processed, and determines the predicted category of the first raster based on the predicted category of each point located in the first raster, wherein the first raster is any one of the multiple rasteres included in the raster data to be processed. The electronic device performs a clustering operation based on the predicted category of each grid cell in the raster data to be processed, so as to obtain at least one point cloud cluster corresponding to the point cloud data to be processed. Specifically, the electronic device can perform a clustering operation based on the predicted category of each grid cell in the raster data to be processed, so as to obtain the clustering results of multiple grid cells in the raster data to be processed. Based on the correspondence between the coordinates of each point in the point cloud data to be processed and the grid coordinates of each point in the target raster map, and the clustering results of multiple grid cells in the raster data to be processed, at least one point cloud cluster corresponding to the point cloud data to be processed is generated.

[0008] In this implementation, during the clustering of multiple points in the point cloud data to be processed, the predicted category of each point is also obtained. Based on the location information of each point in the point cloud data to be processed, the corresponding raster data to be processed is obtained. Based on the predicted category of the points located in the raster, the predicted category of the aforementioned raster is generated. Then, based on the predicted category of each raster in the raster data to be processed, the raster data to be processed is clustered, thereby obtaining at least one point cloud cluster corresponding to the point cloud data to be processed. Since the clustering operation no longer relies solely on the location information of each point in the point cloud data to be processed, but simultaneously utilizes the location information and the predicted category of each point in the point cloud data to be processed to complete the entire clustering process, it is beneficial to improve the accuracy of the entire clustering process.

[0009] In one possible implementation of the first aspect, for any one of the N points in the point cloud data to be processed (hereinafter referred to as the "target point" for convenience), the position information of the target point can be three-dimensional spatial coordinates or two-dimensional coordinates. Correspondingly, the target raster map corresponding to the point cloud data to be processed can be a raster map established in a three-dimensional coordinate system or a raster map established in a two-dimensional coordinate system. Further, the three-dimensional spatial coordinates include, but are not limited to, a three-dimensional rectangular coordinate system or a spherical coordinate system, etc. The position of any target point in the spherical coordinate system can be represented by three indices: radial distance, zenith angle, and azimuth angle; the two-dimensional coordinate system includes, but is not limited to, a two-dimensional rectangular coordinate system or a polar coordinate system, etc. The position of any target point in the polar coordinate system can be represented by two indices: polar radius and polar angle.

[0010] In one possible implementation of the first aspect, the electronic device acquiring the predicted category of each point in the point cloud data to be processed may include: the electronic device inputting the point cloud data to be processed into a first neural network to generate the predicted category of each point among the N points included in the point cloud data to be processed through the first neural network, wherein the first neural network is a neural network for semantic segmentation.

[0011] In one possible implementation of the first aspect, the electronic device determines the predicted category of the first grid based on the predicted category of each point in the first grid. This may include: the electronic device determining an initial predicted category of the first grid based on the predicted category of each point in the first grid; and the electronic device traversing each grid in the grid data to be processed to generate an initial predicted category for each grid in the grid data to be processed. The electronic device acquires at least one target neighboring grid corresponding to the first grid, and determines the final predicted category of the first grid based on the initial predicted category of each target neighboring grid in the at least one target neighboring grid. Here, at least one target neighboring grid corresponding to the first grid refers to a grid located around the first grid in the target grid map corresponding to the point cloud data to be processed. The at least one target neighboring grid corresponding to the first grid may include grids directly adjacent to the first grid, or grids indirectly adjacent to the first grid, etc., without limitation here. In this implementation, the predicted category of the first grid refers to the final predicted category of the first grid. The electronic device performs a clustering operation based on the predicted category of each raster in the raster data to be processed to obtain at least one point cloud cluster, including: the electronic device performs a clustering operation based on the final predicted category of each raster in the raster data to be processed to obtain at least one point cloud cluster.

[0012] In this implementation, noise may be introduced during the acquisition and processing of point cloud data, which may lead to errors when determining the category of the first grid based solely on the predicted category of each point in the first grid. Therefore, when determining the final predicted category of the first grid, not only the predicted category of each point within the first grid is considered, but also the initial predicted categories of the neighboring grids of the first grid are taken into account. This is to minimize the errors caused by noise in the grid category determination process, thereby improving the accuracy of the grid category determination process and ultimately enhancing the accuracy of the entire clustering process of the point cloud data to be processed.

[0013] In one possible implementation of the first aspect, the electronic device determines the initial predicted category of the first grid based on the predicted category of each point located in the first grid. This may include: the electronic device acquiring the predicted category of each point located in the first grid to count the correspondences between S points in the first grid and n+1 categories, and the number of points in the first grid corresponding to each of the n+1 categories, where n is an integer greater than or equal to 0. The electronic device determines a third category from the n+1 categories and determines the initial predicted category of the first grid based on the third category; wherein the third category is one of the n+1 categories, and the first point of the third category has the largest number of points among the S points located in the first grid.

[0014] In one possible implementation of the first aspect, the electronic device may further acquire a first total number of points located in the first grid and a first number of points in the first grid whose predicted category is the third category, and generate a first probability value, which is obtained based on the ratio between the first number and the first total number. When the category pointed to by the third category has semantic meaning, the electronic device determines whether the first probability value corresponding to the third category is greater than or equal to a third threshold corresponding to the third category. If the first probability value corresponding to the third category is greater than or equal to the third threshold, the electronic device determines the final predicted category of the first grid as the third category; if the first probability value corresponding to the third category is less than the third threshold corresponding to the third category, the electronic device may determine the final predicted category of the first grid as having no semantic meaning, or it may determine the final predicted category of the first grid as background.

[0015] In one possible implementation of the first aspect, the electronic device can directly determine the final predicted category of the first grid based on the predicted category of each point located in the first grid (that is, the initial predicted category of the first grid is the final predicted category of the first grid); in this implementation, the predicted category of the first grid can refer to the final predicted category of the first grid or the initial predicted category of the first grid.

[0016] In one possible implementation of the first aspect, when the initial predicted category of the first grid is a first category, the number of at least one target neighbor grid corresponding to the first grid is a first value; when the initial predicted category of the first grid is a second category, the number of at least one target neighbor grid corresponding to the first grid is a second value. The first and second values ​​differ if the size of objects in the first category is different from that of objects in the second category. In this implementation, since the sizes of objects of different categories may differ in the actual environment, selecting different numbers of neighbor grids for objects of different sizes helps to make the selected neighbor grids closely resemble the shapes of objects in the actual environment, thereby improving the accuracy of the grid's predicted category.

[0017] In one possible implementation of the first aspect, the method may further include: the electronic device determining at least one boundary raster from a plurality of rasters included in the raster data to be processed based on the predicted category of each raster in the raster data to be processed; wherein, the boundary raster refers to a raster located on the boundary of an object in the target raster image, that is, a boundary raster refers to a raster located between different objects in the target raster image. The electronic device performing a clustering operation based on the predicted category of each raster in the raster data to be processed may include: the electronic device performing a clustering operation based on the predicted category of each raster in the raster data to be processed and at least one boundary raster in the raster data to be processed.

[0018] In this implementation, at least one boundary raster is first determined from the multiple rasteres included in the raster data to be processed based on the predicted category of each raster in the raster data to be processed. Then, a clustering operation is performed based on the predicted category of each raster in the raster data to be processed and at least one boundary raster in the raster data to be processed, so as to avoid determining the boundary raster as the cluster center (also known as the "seed raster") during the clustering process, which helps to improve the accuracy of the clustering process.

[0019] In one possible implementation of the first aspect, the electronic device determines at least one boundary grid from a plurality of grids included in the grid data to be processed based on the predicted category of each grid in the grid data to be processed. This may include: the electronic device acquiring a target probability value corresponding to a second grid; wherein the second grid is any one of at least one target neighboring grids of the first grid, the target probability value corresponding to the second grid indicates the ratio between a third value corresponding to the second grid and a fourth value corresponding to the second grid, the third value being the number of grids in the neighboring grids of the second grid that have the same predicted category as the second grid, and the fourth value being the total number of neighboring grids of the second grid; the predicted category of the second grid may be obtained directly based on the predicted categories of multiple points within the second grid, or the predicted category of the second grid may be obtained by performing secondary processing based on the predicted categories of multiple points within each grid (i.e., the predicted category of the second grid may be the final predicted category of the second grid). The electronic device traverses each target neighboring grid of the first grid to generate a target probability value corresponding to each target neighboring grid of the first grid. Based on the target probability value corresponding to each target neighboring grid, and if a preset condition is met, the electronic device determines the first grid as a boundary grid.

[0020] In this implementation, since the point cloud data to be processed may contain noise, the predicted category of the first grid may be incorrect. Therefore, the target probability value corresponding to each target neighbor grid of the first grid is used to determine whether the first grid is a boundary grid. The target neighborhood value of the second grid (that is, any target neighbor grid of the first grid) represents the ratio between the number of grids in the neighbor grids of the second grid that have the same predicted category as the second grid and the total number of neighbor grids of the second grid. Compared with judging whether the first grid is a boundary grid based solely on the target probability value of the first grid, this method is beneficial to improving the accuracy of the boundary grid judgment process.

[0021] In one possible implementation of the first aspect, the preset conditions include: the average value of multiple target probability values ​​corresponding to multiple target neighboring grids of the first grid is less than or equal to a first threshold, and the variance of multiple target probability values ​​corresponding to multiple target neighboring grids of the first grid is greater than or equal to a second threshold.

[0022] In this implementation, if a grid is a boundary grid, since a boundary grid refers to a grid located between different objects, the predicted categories of its multiple neighboring grids may differ. Consequently, the target probability values ​​corresponding to the boundary grid and its neighboring grids may be small. When the number of grids corresponding to different object types in the neighboring grids of a boundary grid is different, the variance of the multiple target probability values ​​corresponding to the multiple neighboring grids of the boundary grid will also be large. Since boundary grids will not be reclassified as non-boundary grids during subsequent clustering based on the predicted category of each grid, and grids not yet identified as boundaries during clustering may also be identified as object boundaries, this solution uses the average and variance of the multiple target probability values ​​corresponding to the multiple target neighboring grids of the first grid to determine whether the first grid is a boundary grid. This reduces the probability of classifying non-boundary grids as boundary grids, which helps improve the accuracy of the clustering process for the point cloud data to be processed.

[0023] In one possible implementation of the first aspect, both the first threshold and the second threshold are greater than 0 and less than 1. When the predicted category of the first grid (i.e., the third category of the first grid) is specifically represented as the fourth category, the first threshold can be a1 and the second threshold can be b1. When the predicted category of the first grid (i.e., the third category of the first grid) is specifically represented as the fourth category, the first threshold can be a2 and the second threshold can be b2. When the third and fourth categories are different categories, the values ​​of a1 and a2 can be different, and / or the values ​​of b1 and b2 can be different.

[0024] Secondly, embodiments of this application provide a clustering device for point cloud data, which can be used in the field of point cloud data processing in the field of artificial intelligence. The device may include: a rasterization module, used to rasterize multiple points in the point cloud data to be processed according to the position information of each point in the point cloud data to be processed, to obtain raster data to be processed corresponding to the point cloud data to be processed; an acquisition module, used to acquire the predicted category of each point in the point cloud data to be processed; a determination module, used to determine the predicted category of a first raster according to the predicted category of each point located in the first raster, wherein the first raster is any one of the multiple rasteres included in the raster data to be processed; and a clustering module, used to perform a clustering operation according to the predicted category of each raster in the raster data to be processed, so as to obtain at least one point cloud cluster corresponding to the point cloud data to be processed.

[0025] The point cloud data clustering device provided in the second aspect of the embodiments of this application can also perform the steps executed by the electronic device in the various possible implementations of the first aspect. For the specific implementation steps of the second aspect of the embodiments of this application and the various possible implementations of the second aspect, as well as the beneficial effects brought by each possible implementation, please refer to the description in the various possible implementations of the first aspect, which will not be repeated here.

[0026] Thirdly, embodiments of this application provide a computer program product including computer-readable instructions that, when executed on a computer, cause the computer to perform the point cloud data clustering method described in the first aspect.

[0027] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to execute the point cloud data clustering method described in the first aspect.

[0028] Fifthly, embodiments of this application provide an electronic device that may include a processor and a memory coupled together. The memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the point cloud data clustering method described in the first aspect is implemented.

[0029] In a sixth aspect, embodiments of this application provide a circuit system including a processing circuit configured to perform the point cloud data clustering method described in the first aspect above.

[0030] Seventhly, embodiments of this application provide a chip system including a processor for implementing the functions involved in the foregoing aspects, such as transmitting or processing data and / or information involved in the foregoing methods. In one possible design, the chip system further includes a memory for storing program instructions and data necessary for a server or communication device. The chip system may be composed of chips or may include chips and other discrete devices. Attached Figure Description

[0031] Figure 1 A schematic diagram of the main framework of artificial intelligence provided in the embodiments of this application;

[0032] Figure 2 A flowchart of a method for clustering point cloud data provided in the embodiments of this application;

[0033] Figure 3 A flowchart illustrating another method for clustering point cloud data provided in this application embodiment;

[0034] Figure 4This is a schematic diagram of the target raster map corresponding to the point cloud data to be processed in the point cloud data clustering method provided in the embodiments of this application;

[0035] Figure 5 This is a schematic diagram illustrating the method for clustering point cloud data provided in this application to obtain the first probability value corresponding to the first grid cell.

[0036] Figure 6 A schematic diagram illustrating at least one target neighborhood grid of the first grid obtained in the clustering method of point cloud data provided in the embodiments of this application;

[0037] Figure 7 Another schematic diagram illustrating the method for clustering point cloud data provided in this application to obtain at least one target neighborhood grid of the first grid;

[0038] Figure 8 This is a schematic diagram of the second probability value corresponding to the first grid in the point cloud data clustering method provided in the embodiments of this application;

[0039] Figure 9 A schematic diagram of a point cloud data clustering device provided in an embodiment of this application;

[0040] Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0041] Figure 11 This is a schematic diagram of the structure of an autonomous vehicle provided in an embodiment of this application;

[0042] Figure 12 This is a schematic diagram of a chip structure provided in an embodiment of this application. Detailed Implementation

[0043] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0044] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0045] First, the overall workflow of the artificial intelligence system is described; please refer to [link / reference]. Figure 1 , Figure 1 The diagram illustrates a structural framework for artificial intelligence (AI). The framework is further elaborated below along two dimensions: the "Intelligent Information Chain" (horizontal axis) and the "IT Value Chain" (vertical axis). The "Intelligent Information Chain" reflects a series of processes from data acquisition to processing. For example, it could be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensation process of "data—information—knowledge—wisdom." The "IT Value Chain" reflects the value that AI brings to the information technology industry, from the underlying infrastructure of human intelligence and information (provided and processed through technological means) to the industrial ecosystem of the system.

[0046] (1) Infrastructure

[0047] The infrastructure provides computing power to support artificial intelligence systems, enabling communication with the external world and providing support through a basic platform. Communication with the outside world is achieved through sensors; computing power is provided by intelligent chips, which can specifically employ hardware acceleration chips such as central processing units (CPUs), embedded neural network processing units (NPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs). The basic platform includes distributed computing frameworks and related platform guarantees and support, which may include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to acquire data, and this data is provided to intelligent chips in the distributed computing system provided by the basic platform for computation.

[0048] (2) Data

[0049] The data at the next layer of infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data from traditional devices, including business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0050] (3) Data processing

[0051] Data processing typically includes methods such as data training, machine learning, deep learning, search, reasoning, and decision-making.

[0052] Among them, machine learning and deep learning can perform intelligent information modeling, extraction, preprocessing, and training on data, including symbolization and formalization.

[0053] Reasoning refers to the process in which, in a computer or intelligent system, the machine thinks and solves problems by simulating human intelligent reasoning, based on reasoning control strategies and using formalized information. Typical functions include search and matching.

[0054] Decision-making refers to the process of making decisions based on intelligent information after reasoning, and it typically provides functions such as classification, sorting, and prediction.

[0055] (4) General ability

[0056] After the data processing mentioned above, the results of the data processing can be used to form some general capabilities, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0057] (5) Smart Products and Industry Applications

[0058] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields. They encapsulate overall artificial intelligence solutions, productize intelligent information decision-making, and realize practical applications. Their application areas mainly include: intelligent terminals, intelligent manufacturing, intelligent transportation, smart homes, intelligent healthcare, intelligent security, autonomous driving, and smart cities.

[0059] This application can be applied to the processing of point cloud data in the field of artificial intelligence, specifically to applications that utilize point cloud data for target detection. For example, in the field of autonomous driving, autonomous vehicles collect point cloud data corresponding to their surrounding environment through sensors, and cluster the collected point cloud data to obtain at least one point cloud cluster. Then, a classification operation is performed based on at least one point cloud cluster, thus completing the target detection operation based on the collected point cloud data. As another example, in the field of intelligent security, surveillance systems can collect point cloud data corresponding to the monitored environment through sensors and perform target detection operations based on the collected point cloud data.

[0060] It should be noted that the examples above are only for the convenience of understanding the application scenarios of this solution and are not intended to limit this solution. Not all application scenarios of the embodiments of this application are listed here. In all the above scenarios, the clustering methods used suffer from poor accuracy; therefore, a more accurate clustering method is needed.

[0061] To address the aforementioned issues, this application provides a clustering method for point cloud data. For details, please refer to [link to relevant documentation]. Figure 2 , Figure 2 This is a flowchart of a method for clustering point cloud data provided in an embodiment of this application. A1. Based on the location information of each point in the point cloud data to be processed, multiple points in the point cloud data to be processed are rasterized to obtain raster data corresponding to the point cloud data to be processed. The point cloud data to be processed includes the location information of each of N points corresponding to the environment at a target time; that is, the point cloud data to be processed is used to describe the environment at the target time, which can be any time, depending on the actual situation. A2. Obtain the predicted category of each point in the point cloud data to be processed. A3. Based on the predicted category of each point located in the first raster, determine the predicted category of the first raster, which is any one of the multiple rasteres included in the raster data to be processed. A4. Based on the predicted category of each raster in the raster data to be processed, perform a clustering operation to obtain at least one point cloud cluster corresponding to the point cloud data to be processed.

[0062] The following describes the point cloud data clustering method provided in the embodiments of this application. For details, please refer to [link to relevant documentation]. Figure 3 , Figure 3 This is a flowchart illustrating a point cloud data clustering method provided in an embodiment of this application. The point cloud data clustering method provided in an embodiment of this application may include:

[0063] 301. Obtain the point cloud data to be processed.

[0064] In this embodiment of the application, the electronic device needs to acquire point cloud data to be processed. The meaning of point cloud data to be processed can be found in the above description, and will not be repeated here.

[0065] Specifically, in some application scenarios, electronic devices can directly collect point cloud data corresponding to the environment around the electronic device through sensors, including but not limited to lidar, millimeter-wave radar, or other sensors used to collect point cloud data. In other application scenarios, electronic devices can receive point cloud data to be processed sent by other electronic devices, meaning that the point cloud data to be processed does not necessarily describe the environment around the electronic device.

[0066] In other application scenarios, electronic devices can also obtain point cloud data to be processed from multiple point cloud data stored locally. The method by which electronic devices obtain point cloud data to be processed is not limited here.

[0067] Furthermore, the point cloud data to be processed can be point cloud data collected by a single sensor; or it can be point cloud data obtained by fusing point cloud data collected by multiple sensors; the aforementioned sensor can specifically be a radar.

[0068] The target point is any one of the N points in the point cloud data to be processed (hereinafter referred to as the "target point" for ease of description). The position information of the target point can be three-dimensional spatial coordinates or two-dimensional coordinates. If the coordinates of each target point are in three-dimensional spatial coordinates, then the coordinates of each target point can be in a three-dimensional Cartesian coordinate system, a spherical coordinate system, or other types of three-dimensional coordinate systems. The position of any target point in the spherical coordinate system can be represented by three indices: radial distance, zenith angle, and azimuth angle.

[0069] To further understand the data format of the target point cloud data to be processed, as an example, an electronic device uses a lidar to collect surrounding environmental data at a certain scanning rate to acquire target point cloud data corresponding to the environment at the target time. Specifically, the target point cloud data to be processed can be represented as {p j}, j = 1 to N, where N represents the number of target points included in the target point cloud data to be processed; p j p represents any target point included in the target point cloud data to be processed. j = [x, y, z].

[0070] If the coordinates of each target point are in two-dimensional coordinates, then the coordinates of each target point can be in a two-dimensional rectangular coordinate system, a polar coordinate system, or other types of two-dimensional coordinate systems. In a polar coordinate system, the position of any target point can be represented by two indices: the polar radius and the polar angle. It should be noted that the example of target point coordinates here is only for the convenience of understanding this scheme. The specific type of coordinate used needs to be flexibly determined based on the actual application scenario, and no limitation is made here.

[0071] 302. Based on the position information of each point in the point cloud data to be processed, perform rasterization processing on multiple points in the point cloud data to be processed to obtain the raster data to be processed corresponding to the point cloud data to be processed.

[0072] In this embodiment, after acquiring the point cloud data to be processed, the electronic device obtains the position information of each point in the point cloud data, and can then perform rasterization processing on multiple points in the point cloud data to obtain raster data to be processed corresponding to the point cloud data. The raster data to be processed includes the raster coordinates of each point in the point cloud data in the target raster image, and may also include the correspondence between the coordinates of each point in the point cloud data and the raster coordinates of each point in the target raster image.

[0073] Furthermore, if each point in the point cloud data to be processed uses three-dimensional spatial coordinates, then the target raster can be a target raster built in a three-dimensional coordinate system. For example, if the coordinates of each point are in a three-dimensional Cartesian coordinate system, then the target raster can be a target raster built in a three-dimensional Cartesian coordinate system; as another example, if the coordinates of each point are in a spherical coordinate system, then the target raster can be a target raster built in a spherical coordinate system, and so on.

[0074] If each point in the point cloud data to be processed uses two-dimensional coordinates, then the target raster can be a target raster built in a two-dimensional coordinate system. For example, if the coordinates of each point are in a two-dimensional Cartesian coordinate system, then the target raster can be a target raster built in a two-dimensional Cartesian coordinate system; as another example, if the coordinates of each point are in a polar coordinate system, then the target raster can be a target raster built in a polar coordinate system, and so on. The specific form of the target raster needs to be determined based on the type of coordinate system corresponding to each point in the point cloud data to be processed, and is not limited here.

[0075] Specifically, after acquiring the point cloud data to be processed, the electronic device can generate a target raster map corresponding to the point cloud data to be processed, that is, determine the origin of the target raster map and the unit length of the raster in the target raster map. The origin of the target raster map can also be called the pole of the target raster map.

[0076] Next, the electronic device needs to acquire the raster coordinates of each point in the point cloud data to be processed. For any one of the N points in the point cloud data to be processed (i.e., the "target point"), the electronic device acquires the location information of the target point, and generates the raster coordinates of the target point in the target raster map according to the location information of the target point and the preset mapping function, and obtains the correspondence between the location information of the target point and the raster coordinates of the target point.

[0077] To better understand this solution, as an example, we will use the target point's location information as its three-dimensional spatial coordinates in a three-dimensional Cartesian coordinate system. The three-dimensional Cartesian coordinate system includes the x-axis, y-axis, and z-axis. The following is an example of a preset mapping function:

[0078]

[0079] Where (x′, y′, z′) represents the grid coordinates of the target point in the target grid map in a three-dimensional Cartesian coordinate system, (x, y, z) represents the coordinates of the target point in a three-dimensional Cartesian coordinate system, F represents the preset mapping function, and factor x This represents the length of one grid cell along the x-axis, also known as the factor. x The factor represents the unit length of the grid along the x-axis. y This represents the length of one grid cell along the y-axis, also known as the factor. y The factor represents the unit length of the grid along the y-axis. z This represents the length of one grid cell along the z-axis, also known as the factor. z The start represents the unit length of the grid along the z-axis. x The value of corresponds to the range of values ​​of the point cloud data to be processed in the x-axis direction, start y The value of corresponds to the range of values ​​of the point cloud data to be processed in the y-axis direction, start z The value of `start` corresponds to the range of values ​​in the z-axis direction of the point cloud data to be processed. For example, if the range of all points in the entire point cloud data to be processed is [-98.9, 98.9] in the x-axis direction, [-39.8, 39.8] in the y-axis direction, and [-1.5, 6.7] in the z-axis direction, then `start`... x It can be set to -100, start y It can be set to -40, start z It can be set to -2. Optionally, if the electronic device only wants to process the point cloud data of the portion where x ≥ 10, that is, the point cloud data to be processed only includes the portion where x ≥ 10, start x It can be set to 10, start y It can be set to -40, start z It can be set to -2. It should be understood that the examples in equation (1) and the foregoing examples are only for the convenience of understanding this scheme and are not intended to limit this scheme.

[0080] As another example, taking the target point's location information as its three-dimensional spatial coordinates in a spherical coordinate system as an example, the spherical coordinate system includes three dimensions: radial distance, zenith angle, and azimuth angle. The following is another example of a preset mapping function:

[0081]

[0082] in, This represents the grid coordinates of the target point in the target grid map in spherical coordinates. The factor represents the coordinates of the target point in spherical coordinates. r The start represents the unit length of the grid in the radial direction in spherical coordinates. r The value of corresponds to the range of radial distance values ​​in the point cloud data to be processed. For example, start r Setting it to 5 means that the radial distance between all points in the point cloud data to be processed and the origin is greater than or equal to 5. θ The value of factor corresponds to the range of values ​​of the point cloud data to be processed on the zenith angle axis. θ This represents the size of the grid on the zenith angle axis, for example, the factor. θ Setting it to 0.5° means that the size of each grid cell on the zenith angle axis is 0.5°. The value of corresponds to the range of values ​​of the point cloud data to be processed on the azimuth axis. The grid size represents the azimuth axis. It should be understood that the examples and illustrations in Equation (2) are only for the convenience of understanding this scheme and are not intended to limit this scheme.

[0083] To understand this solution more intuitively, please refer to [link / reference]. Figure 4 , Figure 4 This is a schematic diagram of the target raster map corresponding to the point cloud data to be processed in the point cloud data clustering method provided in the embodiments of this application. Figure 4 Taking the points in the point cloud data to be processed as an example, which uses a three-dimensional rectangular coordinate system, Figure 4 The image shows point cloud data in a three-dimensional Cartesian coordinate system (i.e., the X, Y, and Z axes in the illustration), and multiple grids in the plane formed by the X and Y axes. Figure 4 The correspondence between point cloud data and raster images can be observed. It should be understood that in practice, the points in the point cloud data to be processed can also be represented in other coordinate systems; the specific situation should be determined based on the actual circumstances. Figure 4 The examples in the examples are only for the purpose of understanding the correspondence between points in point cloud data and graticles in raster maps, and are not intended to limit this scheme.

[0084] 303. Obtain the predicted category for each point in the point cloud data to be processed.

[0085] In this embodiment, the electronic device also acquires the predicted category of each point in the point cloud data to be processed. Specifically, in one implementation, the electronic device is configured with a first neural network for semantic segmentation. The electronic device inputs the point cloud data to be processed into the first neural network to generate a predicted category for each of the N points included in the point cloud data to be processed. In another implementation, the electronic device acquires the predicted category of each of the N points included in the point cloud data to be processed sent by other electronic devices.

[0086] The first neural network is a mature neural network that has undergone training. For example, the first neural network used for semantic segmentation can be a convolutional neural network or other neural networks used for semantic segmentation.

[0087] The predicted category of each point can be the category of objects in the environment at the target time. For example, if the electronic device is an autonomous vehicle and the point cloud data to be processed is the point cloud data corresponding to the surrounding environment collected by the vehicle, then the predicted category of each point can specifically be people, vehicles, bicycles, background points, etc., which will not be exhaustively listed here.

[0088] The predicted category of each point output by the first neural network can be represented as a number, character, array, or other form. As an example, this example uses the category number of each point output by the first neural network to indicate the predicted category. For instance, if the category number of the target point output by the first neural network is 0, it means the predicted category of the target point is a point without semantic meaning; "a point without semantic meaning" can also be understood as a "background point." As another example, if the category number of the target point output by the first neural network is 1, it means the predicted category of the target point is a person. It should be understood that these examples are only for ease of understanding of this scheme and are not intended to limit this scheme.

[0089] It should be noted that the execution order of steps 302 and 303 is not limited in the embodiments of this application. Step 302 can be executed first and then step 303 can be executed; or step 303 can be executed first and then step 302 can be executed.

[0090] 304. Determine the predicted category of the first grid cell based on the predicted category of each point located in the first grid cell.

[0091] In this embodiment, the electronic device can determine the predicted category of each grid in the point cloud data to be processed based on the predicted category of each point in the point cloud data to be processed and the grid coordinates of each point in the point cloud data to be processed. The first grid is any one of the multiple grids included in the grid data to be processed.

[0092] In one implementation, step 304 may include: the electronic device can directly determine the final predicted category of the first grid based on the predicted category of each point located in the first grid (that is, the initial predicted category of the first grid is the final predicted category of the first grid).

[0093] Specifically, after determining the first grid, the electronic device can determine the S points included in the first grid based on the coordinates of each point in the point cloud data to be processed (i.e., the position information of each point) and the correspondence between the coordinates of each point in the point cloud data to be processed and the grid coordinates of each point in the target grid map. S is an integer greater than or equal to 1.

[0094] The electronic device obtains the predicted category of each point in the first grid. That is, the electronic device can count the number of points in the first grid that correspond to the n+1 categories, and can also count the number of points in the first grid that correspond to each of the n+1 categories, where n is an integer greater than or equal to 0.

[0095] The electronic device determines a third category from n+1 categories and, based on this third category, determines the final predicted category of the first grid. The third category is one of the n+1 categories, and the first point in the first grid, among the S points, has the largest number belonging to the third category. Furthermore, the electronic device can represent the final predicted category of a grid using numbers, characters, arrays, or other forms of data. The electronic device can use the same or different types of information to indicate the predicted category of each point and the predicted category of each grid.

[0096] Optionally, the electronic device can also obtain the value of S (hereinafter referred to as "first total quantity" for ease of description), and the first quantity of the first point in the first grid that is predicted to be of the third category, and generate a first probability value, which is obtained based on the ratio between the first quantity and the first total quantity.

[0097] To further understand this scheme, the following is an example of a formula for the electronic device to obtain the first probability value corresponding to the third category:

[0098]

[0099] Where P1 represents a first probability value corresponding to the third category among the n+1 categories, and i represents the category number corresponding to each of the S first points located in the first grid. The value of i can be any value from 0 to n. In equation (3), the category pointed to when the value of i is 0 is a point with no semantic meaning (or a background point). As shown in equation (3) above, if the category number corresponding to the first point with the largest number of points among the S points located in the first grid (that is, the category number of the third category) is not equal to 0, then the ratio between the total number of first points in the first grid and the total number of all points in the first grid is obtained, and the aforementioned ratio is determined as the first probability value.

[0100] If the category number corresponding to the most numerous point among the S points in the first grid is equal to 0, then we can first determine whether the ratio between the number of points with category number 0 in the first grid and the total number of all points in the first grid is greater than or equal to the fourth threshold (i.e., threshold1). If the result of the above determination is yes, then we further determine whether the ratio between the number of points with category number 0 in the first grid and the total number of all points in the first grid is greater than the fifth threshold (i.e., threshold2). If the aforementioned ratio is greater than the fifth threshold, then the aforementioned ratio is determined as the first probability value; if the aforementioned ratio is less than or equal to the fifth threshold, then the fifth threshold is determined as the first probability value. Here, threshold1 is a value greater than or equal to 0, threshold2 is a value greater than 0, and the values ​​of threshold1 and threshold2 can be the same or different, which is not limited here.

[0101] If the category number corresponding to the most numerous first point among the S points in the first grid is equal to 0, and the ratio between the number of points with category number equal to 0 in the first grid and the total number of all points in the first grid is less than the fourth threshold, then the first probability value can be determined as 0. When the first probability value corresponding to the first grid is 0, it can be regarded as that there is no point with semantic meaning in the first grid. Then the final predicted category of the first grid can be determined as having no semantic meaning, or the final predicted category of the first grid can be determined as background. It should be understood that the example in equation (3) is only for the convenience of understanding this scheme and is not used to limit this scheme.

[0102] To understand this solution more intuitively, please refer to [link / reference]. Figure 5 , Figure 5 This is a schematic diagram illustrating the method for clustering point cloud data provided in this application to obtain the first probability value corresponding to the first grid. Figure 5The image shows multiple points within the first grid and the predicted category for each point. The total number of points in the first grid is 51. 42 points are predicted as "vehicles," 3 as "background," and 6 as "vulnerable road users" (VRUs). VRUs include, but are not limited to, people, bicycles, and other road users besides vehicles. The first grid has the most points predicted as "vehicles," and the ratio of these points to the total number of points in the first grid is 0.82. Since "vehicles" are not considered "background," the first probability value corresponding to the first grid can be 0.82. It should be noted that in reality, the shape of each point in the point cloud data to be processed is identical. Figure 5 Different shapes are used to distinguish different categories of points. Figure 5 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.

[0103] This describes the process by which an electronic device determines the final predicted category of a first grid cell based on a third category. More specifically, in one implementation, the electronic device can directly determine the final predicted category of the first grid cell as the third category. In another implementation, the electronic device can determine whether a first probability value is greater than or equal to a third threshold corresponding to the third category. If the first probability value corresponding to the third category is greater than or equal to the third threshold, the electronic device determines the final predicted category of the first grid cell as the third category; if the first probability value corresponding to the third category is less than the third threshold, the electronic device can determine the final predicted category of the first grid cell as having no semantic meaning, or it can determine the final predicted category of the first grid cell as background.

[0104] In another implementation, the electronic device employs different processing methods when the third category manifests as different categories. For example, when the category pointed to by the third category has semantic meaning, such as vehicles, people, traffic lights, bus stops, etc., it should be understood that this example is only for ease of understanding of this solution, and the specific manifestation of the third category can be flexibly determined based on the actual application scenario and is not intended to limit this solution. The electronic device determines whether the first probability value corresponding to the third category is greater than or equal to the third threshold corresponding to the third category. If the first probability value corresponding to the third category is greater than or equal to the third threshold, the electronic device determines the final predicted category of the first grid as the third category; if the first probability value corresponding to the third category is less than the third threshold, the electronic device can determine the final predicted category of the first grid as having no semantic meaning, or it can determine the final predicted category of the first grid as background.

[0105] If the category pointed to by the third category has no semantic meaning, that is, if the number of points without a semantic category is the largest among the S points in the first grid, the electronic device can determine the final predicted category of the first grid as having no semantic meaning, or the electronic device can determine the final predicted category of the first grid as background.

[0106] Furthermore, the third threshold corresponding to different categories can be the same or different. For example, when the third category points to the category of "person," the third threshold value can be 0.3; when the third category points to the category of "bicycle," the third threshold value can be 0.4, etc. These examples are only for ease of understanding and are not intended to limit the scope of this scheme. Since the density of objects of different categories varies, the number of points corresponding to objects of different categories within a unit space will also differ. Different third thresholds for different categories are beneficial for improving the accuracy of the grid's category determination process.

[0107] In another implementation, the electronic device determines an initial predicted category for the first grid based on the predicted category of each point located in the first grid; it then acquires at least one target neighboring grid corresponding to the first grid, and determines the final predicted category of the first grid based on the initial predicted category of each of the at least one target neighboring grid. The electronic device traverses each grid in the grid data to be processed, performing the aforementioned operations on each grid in the grid data to be processed, thereby obtaining the final predicted category of each grid in the grid data to be processed.

[0108] In this embodiment, noise may be introduced during the acquisition and processing of point cloud data, which may lead to errors when determining the category of the first grid based solely on the predicted category of each point in the first grid. Therefore, when determining the final predicted category of the first grid, not only the predicted category of each point within the first grid is considered, but also the initial predicted categories of the neighboring grids of the first grid are considered. This is to minimize the errors caused by noise in the grid category determination process, thereby further improving the accuracy of the grid category determination process and ultimately improving the accuracy of the clustering process of the entire point cloud data to be processed.

[0109] Specifically, the implementation of the step "the electronic device determines the initial predicted category of the first grid based on the predicted category of each point in the first grid" can be found in the description of the implementation "directly determining the final predicted category of the first grid based on the predicted category of each point in the first grid." The only difference is that "final predicted category" in the previous implementation is replaced with "initial predicted category" in this implementation. The electronic device traverses each grid in the target grid map and obtains the initial predicted category of each grid in the target grid map using the above method.

[0110] The electronic device acquires at least one target neighboring raster corresponding to a first raster, and acquires the initial predicted category of each target neighboring raster in the at least one target neighboring raster. It also acquires a second total number of all target neighboring rasteres corresponding to the first raster, and a second number of rasteres in the at least one target neighboring raster whose initial predicted category is a third category (i.e., the initial predicted category of the first raster). A second probability value is generated, which is the ratio between the second number and the second total number. If the second probability value is greater than or equal to a sixth threshold, the initial predicted category of the first raster is determined as the final predicted category of the first raster; if the second probability value is less than the sixth threshold, the final predicted category of the first raster is determined as background, or the final predicted category of the first raster is determined as having no semantic meaning.

[0111] The sixth threshold is greater than 0 and less than 1. For example, the value of the sixth threshold can be 0.2, 0.25 or other values. The value of the sixth threshold can be determined by combining factors such as the resolution of the sensor used to collect the point cloud data to be processed and the accuracy of the neural network used for semantic segmentation. This example is only for the convenience of understanding this scheme.

[0112] At least one target neighboring grid cell corresponding to the first grid cell refers to the grid cell located around the first grid cell in the target grid cell map. The at least one target neighboring grid cell corresponding to the first grid cell may include grid cells that are directly adjacent to the first grid cell, or grid cells that are indirectly adjacent to the first grid cell, etc., without limitation here.

[0113] Furthermore, in one implementation, when the initial predicted category of the first grid cell specifically represents different categories, the number of at least one target neighboring grid cell corresponding to the first grid cell is always the same. For a more intuitive understanding of this scheme, please refer to [link to relevant documentation]. Figure 6 , Figure 6 This is a schematic diagram illustrating at least one target neighborhood grid of a first grid obtained in a point cloud data clustering method provided in an embodiment of this application. Figure 6 Includes two sub-diagrams, (a) and (b). Figure 6 (a) Sub-diagram and Figure 6 The sub-illustrative diagrams in (b) all assume the target raster image is built in a two-dimensional Cartesian coordinate system. B1 represents the first raster, B2 represents the target neighbor raster of the first raster, and the initial predicted category corresponding to the first raster pointed to by B1 is vehicle. C1 represents the first raster, C2 represents the target neighbor raster of the first raster, and the initial predicted category corresponding to the first raster pointed to by C1 is person. Figure 6 As shown, when the initial predicted category of the first grid cell is different, the number of at least one target neighboring grid cell corresponding to the first grid cell is the same. It should be understood that... Figure 6The examples in the text are only for the purpose of understanding the concept of target neighborhood grid and are not intended to limit this scheme.

[0114] In another implementation, when the initial predicted category of the first grid is a first category, the number of at least one target neighbor grid corresponding to the first grid is a first value; when the initial predicted category of the first grid is a second category, the number of at least one target neighbor grid corresponding to the first grid is a second value. If the size of objects in the first category is different from that of objects in the second category, then the first value and the second value are different.

[0115] In this embodiment of the application, since the size of objects of different categories may be different in the actual environment, selecting different numbers of neighborhood grids for objects of different sizes is beneficial to make the selected neighborhood grids close to the shape of objects in the actual environment, so as to improve the accuracy of the grid's predicted category.

[0116] To understand this solution more intuitively, please refer to [link / reference]. Figure 7 , Figure 7 This is a schematic diagram illustrating at least one target neighborhood grid of a first grid obtained in a point cloud data clustering method provided in an embodiment of this application. Figure 7 Includes two sub-diagrams, (a) and (b). Figure 7 (a) Sub-diagram and Figure 7 (b) The sub-illustrative diagrams all assume that the target grid is built in a two-dimensional Cartesian coordinate system. D1 represents the first grid, D2 represents the target neighbor grid of the first grid, and the initial predicted category corresponding to the first grid pointed to by D1 is vehicle, that is... Figure 7 The sub-illustrative diagram (a) uses the initial predicted category of the first grid cell as a vehicle as an example. E1 represents the first grid cell, E2 represents the target neighbor grid cell of the first grid cell, and the initial predicted category corresponding to the first grid cell pointed to by E1 is human, that is... Figure 7 The sub-diagram (b) uses the initial predicted category of the first grid cell as an example. For instance... Figure 7 As shown, the number of target neighbor grids for the first grid differs depending on whether the initial predicted category of the first grid is "vehicle" or "human". This should be understood. Figure 7 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.

[0117] To further understand this scheme, the following is an example of a formula for the electronic device to generate the second probability value:

[0118]

[0119] Where SA represents the set of position vectors of the target neighboring rasters of the first raster, CurPosVec represents the position vector corresponding to the first raster, and StdSize t StdSize represents the standard size of objects in the third category (i.e., the initial predicted category of the first grid) in vector form. If the target grid is built on a two-dimensional coordinate system, the size of the third category object is described by values ​​in two dimensions; if the target grid is built on a three-dimensional coordinate system, the size of the three categories of objects is described by values ​​in three dimensions. When the third category uses different categories, StdSize... t The values ​​of the corresponding vectors can be different. GridSize represents the size of a grid cell in the target grid map in the form of a vector. The division in equation (4) is the division of the scalars at corresponding positions in the vector. This represents vector addition and the union of the results. Counter is a counting operator. Counter(P) t (P in SA) represents the number of grid cells in the first grid cell whose initial predicted class is the third class. Counter(P in SA) represents the total number of grid cells in the first grid cell. It should be understood that the example in Equation (4) is only for the convenience of understanding this scheme and is not intended to limit this scheme.

[0120] To understand this solution more intuitively, please refer to [link / reference]. Figure 8 , Figure 8 This is a schematic diagram of the second probability value corresponding to the first grid in the point cloud data clustering method provided in the embodiments of this application. Figure 8 Taking the target raster map in China and Israel as an example, which is a raster map built in a two-dimensional Cartesian coordinate system, the initial predicted category corresponding to the first raster pointed to by F1 is vehicle. Figure 8 The grid within the dashed box represents all target neighboring grids of the first grid, such as... Figure 8 As shown, the number of grid cells in the target neighborhood of the first grid that share the same initial predicted class as the first grid (i.e., the grid cell pointed to by F1) is 14. The total number of target neighborhood grid cells in the first grid is 20. Therefore, the second probability value corresponding to the first grid is 0.7. It should be understood that... Figure 8 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.

[0121] 305. Based on the predicted category of each raster in the raster data to be processed, determine at least one boundary raster from the multiple raster data to be processed.

[0122] In some embodiments of this application, the electronic device can determine at least one boundary grid from a plurality of grids included in the grid data to be processed based on the predicted category of each grid in the grid data to be processed. A boundary grid refers to a grid located on the boundary of an object in the target grid image, that is, a boundary grid refers to a grid located between different objects in the target grid image.

[0123] In step 304, if the electronic device directly determines the final predicted category of the first grid based on the predicted category of each point in the first grid, then the predicted category of each grid in step 305 refers to the final predicted category of each grid. If in step 304, the electronic device first generates the initial predicted category of each grid and then generates the final predicted category of each grid, then the predicted category of each grid in the grid data to be processed can refer to either the initial predicted category or the final predicted category of each grid. That is, the predicted category of each grid can be obtained directly based on the predicted categories of multiple points within each grid (hereinafter referred to as the "first predicted category" for ease of description); or, the predicted category of each grid can be obtained by performing secondary processing based on the predicted categories of multiple points within each grid (hereinafter referred to as the "second predicted category" for ease of description).

[0124] Specifically, regarding the process of determining whether any one of the multiple rasters included in the raster data to be processed (hereinafter referred to as the "first raster" for convenience) is a boundary raster, in one implementation, the electronic device acquires at least one target neighboring raster corresponding to the first raster. For any one of the at least one target neighboring raster (hereinafter referred to as the "second raster" for convenience), the electronic device can acquire at least one neighboring raster corresponding to the second raster, and determine the number of rasteres in the neighboring raster of the second raster that have the same predicted category as the second raster as a third value, determine the total number of neighboring rasteres of the second raster as a fourth value, and determine the ratio between the third value and the fourth value as the target probability value.

[0125] If the electronic device determines that the first preset condition is met based on the target probability value corresponding to each target neighboring grid in at least one target neighboring grid, the first grid is determined to be a boundary grid; if the electronic device determines that the first preset condition is not met based on the target probability value corresponding to each target neighboring grid in at least one target neighboring grid, the first grid is determined to be a non-boundary grid, that is, the first grid is determined to be a non-boundary grid.

[0126] In this embodiment, since noise may exist in the point cloud data to be processed, the predicted category of the first grid may be incorrect. Therefore, the target probability value corresponding to each target neighbor grid of the first grid is used to determine whether the first grid is a boundary grid. The target neighborhood value of the second grid (that is, any target neighbor grid of the first grid) represents the ratio between the number of grids in the neighbor grids of the second grid that have the same predicted category as the second grid and the total number of neighbor grids of the second grid. Compared with judging whether the first grid is a boundary grid based solely on the target probability value of the first grid, this method is beneficial to improving the accuracy of the boundary grid judgment process.

[0127] It's important to clarify that the concepts of "target probability value" and "second probability value" are very similar. The difference lies in the following: For a target raster, the "second probability value" refers to the ratio between the second quantity and the second total quantity. The second quantity is the number of rasters in at least one neighboring raster of the target raster whose initial predicted class matches the initial predicted class of the target raster. The second total quantity is the total number of at least one neighboring raster of the target raster. Conversely, the "target probability value" for a target raster refers to the ratio between the third value and the fourth value. The third value is the number of rasters in the target raster's neighboring raster that share the same predicted class as the target raster. The fourth value is the total number of neighboring raster of the target raster. The difference lies in the fact that the initial predicted category of the target raster specifically refers to the category obtained directly based on the predicted categories of multiple points located within the target raster (that is, specifically the first predicted category of the target raster); while the predicted category of the target raster can refer to either the category obtained directly based on the predicted categories of multiple points located within the target raster, or the category obtained after secondary processing based on the predicted categories of multiple points within the target raster (that is, it can be the first predicted category of the target raster or the second predicted category of the target raster).

[0128] The electronic device traverses each target neighboring grid of the first grid to obtain the target probability value corresponding to each target neighboring grid. For example, the search vector used may be a vector group with a unit size of 1 grid and angles of [0, 45, 90, 135]. That is, if the first grid is used as the starting point, the target neighboring grids of the first grid are searched in the directions of 0 degrees, 45 degrees, 90 degrees, and 135 degrees respectively. Each search moves one grid. It should be noted that this example is only to demonstrate the feasibility of this scheme; other search methods can be used in practice, which are not listed here.

[0129] Specifically, this refers to the process by which an electronic device acquires the target probability value corresponding to a target neighboring grid. The specific implementation of "the electronic device acquiring at least one target neighboring grid corresponding to the first grid" can be found in the description of step 304 above, and will not be repeated here.

[0130] The electronic device can determine whether the categories of a second grid and a certain neighboring grid of the second grid are the same based on a first predicted category of the second grid and its neighboring grids, or it can determine whether the categories of a second grid and a certain neighboring grid of the second grid are the same based on a second predicted category of the second grid and its neighboring grids. The electronic device traverses all neighboring grids of the second grid, thereby calculating the target probability value corresponding to the second grid. The electronic device performs the aforementioned operation on each target neighboring grid of the first grid, thereby obtaining the target probability value corresponding to each target neighboring grid of the first grid.

[0131] Regarding the specific meaning of "first preset condition": In one case, the first preset condition includes: the average value of multiple target probability values ​​corresponding to multiple target neighboring grids of the first grid is less than or equal to a first threshold, and the variance of multiple target probability values ​​corresponding to multiple target neighboring grids of the first grid is greater than or equal to a second threshold.

[0132] Specifically, after obtaining the target probability value corresponding to each target neighboring grid in at least one target neighboring grid of the first grid, the electronic device generates the average and variance of multiple target probability values ​​corresponding to multiple target neighboring grids of the first grid, determines whether the average is less than or equal to a first threshold, and determines whether the variance is greater than or equal to a second threshold. If the average is less than or equal to the first threshold and the variance is greater than or equal to the second threshold, the first grid is determined as a boundary grid; if the average is greater than the first threshold or the variance is less than the second threshold, the first grid is determined as a non-boundary grid.

[0133] In this embodiment, if a certain grid is a boundary grid, since a boundary grid refers to a grid located between different objects, the predicted categories of multiple neighboring grids of the boundary grid may be different. Correspondingly, the target probability values ​​corresponding to the boundary grid and its neighboring grids may be small. When the number of grids corresponding to different types of objects in the neighboring grids of the boundary grid is different, the variance of the multiple target probability values ​​corresponding to the multiple neighboring grids of the boundary grid will also be large. Since the boundary grid will not be reclassified as a non-boundary grid during the subsequent clustering process based on the predicted category of each grid, and grids that have not yet been identified as boundaries during the clustering process may also be identified as object boundaries, this solution uses the average and variance of the multiple target probability values ​​corresponding to the multiple target neighboring grids of the first grid to determine whether the first grid is a boundary grid. This reduces the probability of classifying a non-boundary grid as a boundary grid, which is beneficial to improving the accuracy of the clustering process of the point cloud data to be processed.

[0134] Furthermore, both the first threshold and the second threshold are greater than 0 and less than 1. When the predicted category of the first grid (i.e., the third category of the first grid) is specifically represented as the fourth category, the first threshold can be a1 and the second threshold can be b1. When the predicted category of the first grid (i.e., the third category of the first grid) is specifically represented as the fourth category, the first threshold can be a2 and the second threshold can be b2.

[0135] In one implementation, regardless of whether the third and fourth categories are the same, a1 equals a2 and b1 equals b2. That is, regardless of the specific category predicted by the first grid, the same first and second thresholds are used. As an example, the first threshold could be 0.3 and the second threshold could be 0.5, etc. It should be understood that this example is only for the convenience of understanding this scheme and is not intended to limit this scheme.

[0136] In another implementation, if the first threshold is set very high and the second threshold is set very low, the first preset condition can be easily met, leading to many rasters in the raster data being identified as boundary rasters, thus exacerbating oversegmentation. Conversely, if the first threshold is set very low and the second threshold is set very high, the first preset condition can be difficult to meet, resulting in very few rasters in the raster data being identified as boundary rasters, thus exacerbating undersegmentation. Therefore, technicians can set different first and second thresholds for different categories based on the actual application scenario. That is, when the third and fourth categories are different categories, the values ​​of a1 and a2 can be different, and / or, the values ​​of b1 and b2 can be different.

[0137] As an example, if the predicted category of the first grid is "vehicle," a lower first threshold and a higher second threshold can be set to suppress oversegmentation when clustering vehicles. As another example, if the predicted category of the first grid is "person," a higher first threshold and a lower second threshold can be set to improve undersegmentation when clustering people. It should be understood that these examples are only for illustrative purposes and are not intended to limit the scope of the solution. In this embodiment, by adjusting the values ​​of the first and second thresholds, oversegmentation of larger objects can be suppressed, and underfitting of smaller objects can be improved, thereby further enhancing the accuracy of the clustering process and ultimately improving the overall accuracy of the target detection result.

[0138] In another scenario, the first preset condition can be that the variance of multiple target probability values ​​corresponding to multiple target neighboring rasters of the first raster is greater than or equal to a first preset threshold, wherein the first preset threshold is an integer greater than 0, and the value of the first preset threshold can be the same as or different from the value of the second threshold. Furthermore, when the predicted category of the first raster is different, the value of the first preset threshold can be the same as or different from the value of the second threshold.

[0139] Specifically, the electronic device generates the variance of multiple target probability values ​​corresponding to multiple target neighboring grids of the first grid, and determines whether the variance is greater than or equal to a first preset threshold. If the variance is greater than or equal to the first preset threshold, the first grid is determined as a boundary grid; if the variance is less than the first preset threshold, the first grid is determined as a non-boundary grid.

[0140] In another scenario, the first preset condition can be that the average of multiple target probability values ​​corresponding to multiple target neighboring rasters of the first raster is less than or equal to a second preset threshold, wherein the second preset threshold is an integer greater than 0, and the value of the second preset threshold can be the same as or different from the value of the first threshold. Furthermore, when the predicted category of the first raster is different, the value of the second preset threshold can be the same as or different from the value of the first threshold.

[0141] In another implementation, the electronic device can also obtain the target probability value corresponding to the first grid according to the predicted category of each grid in the grid data to be processed, and determine whether the target probability value corresponding to the first grid is greater than or equal to a third preset threshold. If the aforementioned determination result is yes, then the first grid is determined to be a non-boundary grid; if the aforementioned determination result is no, then the first grid is determined to be a boundary grid.

[0142] The electronic device traverses each of the multiple grids included in the raster data to be processed, and determines one by one whether each grid in the raster data to be processed is a boundary grid, and then selects one or more boundary grids from the multiple grids included in the raster data to be processed.

[0143] 306. Perform clustering operations based on the predicted category of each raster in the raster data to be processed, so as to obtain at least one point cloud cluster corresponding to the point cloud data to be processed.

[0144] In this embodiment, step 305 is optional. If step 305 is executed, step 306 may specifically include: the electronic device performing a clustering operation based on the predicted category of each grid in the grid data to be processed obtained in step 304 and at least one boundary grid obtained in step 305 to obtain a first clustering result, the first clustering result indicating the clustering result of multiple grids in the grid data to be processed; the electronic device generating at least one point cloud cluster corresponding to the point cloud data to be processed based on the correspondence between the coordinates of each point in the point cloud data to be processed and the grid coordinates of each point in the target grid map and the first clustering result. A point cloud cluster can be understood as a set consisting of at least one point, and a point cloud cluster includes the position information (i.e., the coordinates of one or more points) of one or more points in the point cloud data to be processed.

[0145] In this embodiment, at least one boundary raster is first determined from the multiple rasteres included in the raster data to be processed based on the predicted category of each raster in the raster data to be processed. Then, a clustering operation is performed based on the predicted category of each raster in the raster data to be processed and at least one boundary raster in the raster data to be processed, so as to avoid determining the boundary raster as the cluster center (also known as the "seed raster") during the clustering process, which helps to improve the accuracy of the clustering process.

[0146] Specifically, the process for the first clustering result of the electronic device is as follows: The electronic device can select a first seed grid from multiple grids included in the target grid map and initialize the point cloud cluster corresponding to the first seed grid. The first seed grid is also the grid that serves as the cluster center. The first seed grid must be a non-boundary grid that has not yet undergone clustering operations, and the predicted category corresponding to the first seed grid cannot be semantically meaningless; in other words, the predicted category corresponding to the first seed grid cannot be the background. Optionally, the first probability value corresponding to the first seed grid needs to be greater than or equal to a fourth preset threshold, and / or, the second probability value corresponding to the first seed grid needs to be greater than or equal to a fifth preset threshold. The value of the fourth preset threshold is greater than the third threshold to improve the accuracy of the first seed grid.

[0147] The electronic device acquires at least one neighborhood grid of the first seed grid. For any neighboring grid (hereinafter referred to as the "third grid") in the at least one neighborhood grid of the first seed grid, the electronic device can determine whether the predicted category of the third grid is the same as the predicted category of the first seed grid. If the predicted category of the third grid is the same as the predicted category of the first seed grid, the third grid is added to the point cloud cluster corresponding to the first seed grid. If the predicted category of the third grid is different from the predicted category of the first seed grid, the next neighborhood grid of the first seed grid is acquired. It should be noted that the specific implementation of the step of the electronic device acquiring at least one neighborhood grid of the first seed grid can be found in the description of the above steps, and will not be repeated here.

[0148] Optionally, the electronic device can determine whether the predicted category of the third grid is the same as the predicted category of the first seed grid, and whether the third grid meets a second preset condition. If both of the aforementioned determinations are yes, the third grid is added to the point cloud cluster corresponding to the first seed grid; otherwise, the next neighborhood grid of the first seed grid is acquired. The second preset condition includes whether the first probability value corresponding to the third grid is greater than or equal to a fourth preset threshold, and / or, the second probability value corresponding to the third grid needs to be greater than or equal to a fifth preset threshold.

[0149] After the electronic device traverses all neighboring grates of the first seed grates, that is, after performing the above judgment operation on each neighboring grates of the first seed grates, a point cloud cluster corresponding to the first seed grates is obtained. The next seed grates can be selected from the multiple grates included in the target grating to generate a point cloud cluster corresponding to the next seed grating. The electronic device repeats the above steps until clustering operations have been performed on each grating in the target grating to generate the first clustering result.

[0150] Optionally, after the electronic device performs a clustering operation once on each grid cell in the target grid image, since some grid cells in the target grid image may not be assigned to a cluster, the electronic device can repeat the above clustering operation H times to re-cluster the grid cells that have not yet been added to a cluster. The value of H can be an integer greater than or equal to 1. For example, the value of H can be 1, 2, or other values, etc. Furthermore, in different batches of these H times, the values ​​of the fourth preset threshold and the fifth preset threshold can become progressively smaller.

[0151] If step 305 is not executed, then step 306 may specifically include the electronic device directly performing a clustering operation on multiple rasters in the raster data to be processed based on the predicted category of each raster in the raster data to be processed obtained through step 304, so as to obtain a second clustering result, the second clustering result indicating the clustering result of multiple rasters in the raster data to be processed; the electronic device generating at least one point cloud cluster corresponding to the point cloud data to be processed based on the correspondence between the coordinates of each point in the point cloud data to be processed and the raster coordinates of each point in the target raster map and the second clustering result.

[0152] Specifically, the electronic device performs clustering operations directly on multiple rasters in the raster data to be processed based on the predicted category of each raster in the raster data to be processed. For the specific implementation method, please refer to the description of the previous implementation method. The difference is that the seed raster in this implementation method is a raster that has not yet undergone clustering operations, and the predicted category corresponding to the first seed raster cannot be semantically meaningless. That is, the seed raster in this implementation method does not need to be a non-boundary raster.

[0153] 307. Perform a classification operation based on at least one point cloud cluster to obtain the target detection result corresponding to the point cloud data to be processed.

[0154] In this embodiment, after obtaining at least one point cloud cluster corresponding to the point cloud data to be processed, the electronic device can further perform a classification operation through a second neural network to obtain an object detection result corresponding to the point cloud data to be processed. The object detection result indicates the classification category of each point cloud cluster in the at least one point cloud cluster, and can also indicate the position of each point cloud cluster in the at least one point cloud cluster. The second neural network is a neural network that has undergone training operations; for example, the second neural network can specifically be a fully connected neural network or other types of neural networks.

[0155] In this embodiment, during the clustering of multiple points in the point cloud data to be processed, the predicted category of each point is also obtained. Based on the location information of each point in the point cloud data to be processed, the raster data to be processed corresponding to the point cloud data to be processed is obtained. Based on the predicted category of the points located in the raster, the predicted category of the aforementioned raster is generated. Then, based on the predicted category of each raster in the raster data to be processed, the raster data to be processed is clustered, thereby obtaining at least one point cloud cluster corresponding to the point cloud data to be processed. Since the clustering operation no longer relies solely on the location information of each point in the point cloud data to be processed, but simultaneously utilizes the location information and the predicted category of each point in the point cloud data to be processed to complete the entire clustering process, it is beneficial to improve the accuracy of the entire clustering process.

[0156] To provide a more intuitive understanding of the beneficial effects of the embodiments of this application, experiments were conducted on the publicly available dataset KITTI Tracking in the embodiments of this application. The experimental results will be presented below in conjunction with Table 1.

[0157] Table 1

[0158] European distance DBSCAN point density Examples of this application vehicle 86.5 87.4 88.0 VRU 64.7 60.5 73.6

[0159] Table 1 shows the target detection operations performed by Euclidean distance-based spatial clustering of applications with noise (DBSCAN), density-based spatial clustering of applications with noise (DBSCAN), and the scheme provided in the embodiments of this application. The first row of Table 1 shows the accuracy of vehicle detection using the three schemes mentioned above, and the second row of Table 1 shows the accuracy of Vulnerable Road User (VRU) detection using the three schemes mentioned above. As can be seen from Table 1, the scheme provided in the embodiments of this application can effectively improve the accuracy of target detection results.

[0160] exist Figures 2 to 8 Based on the corresponding embodiments, in order to better implement the above-described solutions of this application, related equipment for implementing the above solutions is also provided below. See details. Figure 9 , Figure 9 This is a schematic diagram of a point cloud data clustering device provided in an embodiment of this application. The point cloud data clustering device 900 may include: a rasterization module 901, used to rasterize multiple points in the point cloud data to be processed according to the position information of each point in the point cloud data to be processed, to obtain raster data to be processed corresponding to the point cloud data to be processed; an acquisition module 902, used to acquire the predicted category of each point in the point cloud data to be processed; a determination module 903, used to determine the predicted category of a first raster according to the predicted category of each point located in the first raster, wherein the first raster is any one of the multiple rasteres included in the raster data to be processed; and a clustering module 904, used to perform a clustering operation according to the predicted category of each raster in the raster data to be processed, to obtain at least one point cloud cluster corresponding to the point cloud data to be processed.

[0161] In this embodiment, during the clustering of multiple points in the point cloud data to be processed, the predicted category of each point is obtained by the acquisition module 902. The rasterization module 901 obtains the raster data to be processed corresponding to the point cloud data based on the position information of each point in the point cloud data to be processed. The determination module 903 generates the predicted category of the aforementioned raster based on the predicted category of the points located in the raster. Then, based on the predicted category of each raster in the raster data to be processed, the raster data to be processed is clustered, thereby obtaining at least one point cloud cluster corresponding to the point cloud data to be processed. Since the clustering operation no longer relies solely on the position information of each point in the point cloud data to be processed, but simultaneously utilizes the position information and the predicted category of each point in the point cloud data to be processed to complete the entire clustering process, it is beneficial to improve the accuracy of the entire clustering process.

[0162] In one possible design, the determining module 903 is specifically used to: determine the initial predicted category of the first grid based on the predicted category of each point located in the first grid; obtain at least one target neighboring grid corresponding to the first grid, and determine the final predicted category of the first grid based on the initial predicted category of each target neighboring grid in the at least one target neighboring grid; and the clustering module 904 is specifically used to perform a clustering operation based on the final predicted category of each grid in the grid data to be processed, so as to obtain at least one point cloud cluster.

[0163] In one possible design, when the initial predicted category of the first grid is a first category, the number of at least one target neighbor grid corresponding to the first grid is a first value; when the initial predicted category of the first grid is a second category, the number of at least one target neighbor grid corresponding to the first grid is a second value. If the size of objects of the first category is different from that of objects of the second category, then the first value and the second value are different.

[0164] In one possible design, the determining module 903 is further configured to determine at least one boundary raster from the plurality of rasters included in the raster data to be processed based on the predicted class of each raster in the raster data to be processed; the clustering module 904 is specifically configured to perform a clustering operation based on the predicted class of each raster in the raster data to be processed and at least one boundary raster in the raster data to be processed.

[0165] In one possible design, the clustering module 904 is specifically used to: obtain the target probability value corresponding to the second grid, wherein the second grid is any grid among at least one target neighbor grid of the first grid, the target probability value corresponding to the second grid indicates the ratio between a third value corresponding to the second grid and a fourth value corresponding to the second grid, the third value being the number of grids in the neighbor grids of the second grid that have the same predicted category as the second grid, and the fourth value being the total number of neighbor grids of the second grid; and determine the first grid as a boundary grid if a preset condition is met based on the target probability value corresponding to each target neighbor grid.

[0166] In one possible design, the preset conditions include: the average of multiple target probability values ​​corresponding to multiple target neighboring grids of the first grid is less than or equal to a first threshold, and the variance of multiple target probability values ​​corresponding to multiple target neighboring grids of the first grid is greater than or equal to a second threshold.

[0167] It should be noted that the information interaction and execution process between the modules / units in the point cloud data clustering device 900 are different from those in this application. Figures 2 to 8 The various method embodiments are based on the same concept, and the details can be found in the descriptions of the method embodiments shown above in this application, which will not be repeated here.

[0168] The following describes an electronic device provided by an embodiment of this application. Please refer to [link / reference]. Figure 10 , Figure 10 This is a schematic diagram of an electronic device provided in an embodiment of this application. The electronic device 1000 can specifically be a vehicle, radar, a monitoring system, or a data processing device within a monitoring system, etc., and is not limited thereto. The electronic device 1000 may be equipped with... Figure 9 The point cloud data clustering device 900 described in the corresponding embodiment is used to implement Figures 2 to 8 The electronic device in the corresponding embodiment has the following functions. Specifically, the electronic device 1000 may include: a receiver 1001, a transmitter 1002, a processor 1003, and a memory 1004 (wherein the electronic device 1000 may have one or more processors 1003). Figure 10 (Taking a processor as an example), processor 1003 may include application processor 10031 and communication processor 10032. In some embodiments of this application, receiver 1001, transmitter 1002, processor 1003 and memory 1004 may be connected via bus or other means.

[0169] Memory 1004 may include read-only memory and random access memory, and provides instructions and data to processor 1003. A portion of memory 1004 may also include non-volatile random access memory (NVRAM). Memory 1004 stores processor and operation instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations.

[0170] Processor 1003 controls the operation of electronic devices. In specific applications, the various components of electronic devices are coupled together through a bus system, which may include not only data buses but also power buses, control buses, and status signal buses. However, for clarity, all buses in the diagram are referred to as a bus system.

[0171] The methods disclosed in the embodiments of this application can be applied to or implemented by the processor 1003. The processor 1003 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 1003 or by instructions in software form. The processor 1003 can be a general-purpose processor, a digital signal processor (DSP), a microprocessor, or a microcontroller, and may further include an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The processor 1003 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 1004. Processor 1003 reads the information in memory 1004 and, in conjunction with its hardware, completes the steps of the above method.

[0172] Receiver 1001 can be used to receive input digital or character information, and to generate signal inputs related to the settings and function control of electronic devices. Transmitter 1002 can be used to output digital or character information through the first interface; transmitter 1002 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; transmitter 1002 may also include a display device such as a display screen.

[0173] In this embodiment of the application, the application processor 10031 in the processor 1003 is used to execute... Figures 2 to 8 In the point cloud data clustering method provided in the corresponding embodiment, the steps executed by the electronic device are as follows: Specifically, the application processor 10031 in the processor 1003 is used to execute the following steps:

[0174] Based on the location information of each point in the point cloud data to be processed, multiple points in the point cloud data to be processed are rasterized to obtain the raster data to be processed corresponding to the point cloud data to be processed.

[0175] Obtain the predicted category for each point in the point cloud data to be processed;

[0176] The predicted category of the first grid is determined based on the predicted category of each point located in the first grid. The first grid is any one of the multiple grids included in the grid data to be processed.

[0177] Clustering is performed based on the predicted category of each raster in the raster data to be processed to obtain at least one point cloud cluster corresponding to the point cloud data to be processed.

[0178] It should be noted that the specific manner in which the application processor 10031 executes the above steps is different from that in this application. Figures 2 to 8 The various method embodiments are based on the same concept, and the technical effects they bring are the same as those in this application. Figures 2 to 8 The corresponding method embodiments are the same, and for details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0179] This application also provides an autonomous driving vehicle; please refer to [link / reference]. Figure 11 , Figure 11This is a schematic diagram of an autonomous vehicle provided in an embodiment of this application. The autonomous vehicle 100 is configured in a fully or partially autonomous driving mode. For example, the autonomous vehicle 100 can control itself while in autonomous driving mode, and can determine the current state of the vehicle and its surrounding environment through human operation, determine the possible behavior of at least one other vehicle in the surrounding environment, and determine the confidence level corresponding to the probability of other vehicles performing possible behaviors, and control the autonomous vehicle 100 based on the determined information. When the autonomous vehicle 100 is in autonomous driving mode, it can also be set to operate without human interaction.

[0180] The autonomous vehicle 100 may include various subsystems, such as a mobility system 102, a sensor system 104, a control system 106, one or more peripheral devices 108, a power supply 110, a computer system 112, and a user interface 116. Optionally, the autonomous vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of the autonomous vehicle 100 may be interconnected via wired or wireless means.

[0181] The mobility system 102 may include components that provide powered motion for the autonomous vehicle 100. In one embodiment, the mobility system 102 may include an engine 118, an energy source 119, a transmission 120, and wheels (also referred to as tires) 121.

[0182] Engine 118 can be an internal combustion engine, an electric motor, an air-compressed engine, or other combinations of engines, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air-compressed engine. Engine 118 converts energy source 119 into mechanical energy. Examples of energy source 119 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. Energy source 119 can also provide energy to other systems of the autonomous vehicle 100. Transmission 120 transmits mechanical power from engine 118 to wheels 121. Transmission 120 may include a gearbox, a differential, and a drive shaft. In one embodiment, transmission 120 may also include other components, such as a clutch. The drive shaft may include one or more axles that can be coupled to one or more wheels 121.

[0183] Sensor system 104 may include several sensors for sensing information about the environment surrounding autonomous vehicle 100. For example, sensor system 104 may include a positioning system 122 (which may be a Global Positioning System (GPS), a BeiDou Navigation Satellite System, or another positioning system), an inertial measurement unit (IMU) 124, a radar 126, a laser rangefinder 128, and a camera 130. Sensor system 104 may also include sensors for the internal systems of the autonomous vehicle 100 being monitored (e.g., an in-vehicle air quality monitor, fuel gauge, oil temperature gauge, etc.). Sensing data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, speed, etc.). This detection and identification is a key function for the safe operation of autonomous vehicle 100.

[0184] The positioning system 122 can be used to estimate the geographical location of the autonomous vehicle 100. An IMU 124 is used to sense changes in the position and orientation of the autonomous vehicle 100 based on inertial acceleration. In one embodiment, the IMU 124 can be a combination of an accelerometer and a gyroscope. A radar 126 can use radio signals to sense objects in the surrounding environment of the autonomous vehicle 100, specifically millimeter-wave radar or lidar. In some embodiments, in addition to sensing objects, the radar 126 can also be used to sense the speed and / or direction of travel of objects. A laser rangefinder 128 can use lasers to sense objects in the environment in which the autonomous vehicle 100 is located. In some embodiments, the laser rangefinder 128 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components. A camera 130 can be used to capture multiple images of the surrounding environment of the autonomous vehicle 100. The camera 130 can be a still camera or a video camera.

[0185] The control system 106 controls the operation of the autonomous vehicle 100 and its components. The control system 106 may include various components, including a steering system 132, a throttle 134, a braking unit 136, a computer vision system 140, a path control system 142, and an obstacle avoidance system 144.

[0186] The steering system 132 is operable to adjust the forward direction of the autonomous vehicle 100. For example, in one embodiment, it may be a steering wheel system. The throttle 134 controls the operating speed of the engine 118 and thus the speed of the autonomous vehicle 100. The braking unit 136 controls the deceleration of the autonomous vehicle 100. The braking unit 136 may use friction to slow down the wheels 121. In other embodiments, the braking unit 136 may convert the kinetic energy of the wheels 121 into electrical current. The braking unit 136 may also take other forms to slow down the rotational speed of the wheels 121 to control the speed of the autonomous vehicle 100. The computer vision system 140 is operable to process and analyze images captured by the camera 130 to identify objects and / or features in the environment surrounding the autonomous vehicle 100. The objects and / or features may include traffic signals, road boundaries, and obstacles. The computer vision system 140 may use object recognition algorithms, Structure from Motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 140 may be used to map the environment, track objects, estimate the speed of objects, etc. The route control system 142 is used to determine the driving route and speed of the autonomous vehicle 100. In some embodiments, the route control system 142 may include a lateral planning module 1421 and a longitudinal planning module 1422, which are respectively used to combine data from the obstacle avoidance system 144, GPS 122, and one or more predetermined maps to determine the driving route and speed for the autonomous vehicle 100. The obstacle avoidance system 144 is used to identify, evaluate, and avoid or otherwise traverse obstacles in the environment of the autonomous vehicle 100, which may specifically be physical obstacles and virtual moving bodies that may collide with the autonomous vehicle 100. In one example, the control system 106 may add or replace components other than those shown and described. Alternatively, some of the components shown above may be reduced.

[0187] The autonomous vehicle 100 interacts with external sensors, other vehicles, other computer systems, or users via peripheral devices 108. Peripheral devices 108 may include a wireless communication system 146, an onboard computer 148, a microphone 150, and / or a speaker 152. In some embodiments, peripheral devices 108 provide a means for the user of the autonomous vehicle 100 to interact with a user interface 116. For example, the onboard computer 148 may provide information to the user of the autonomous vehicle 100. The user interface 116 may also operate the onboard computer 148 to receive user input. The onboard computer 148 may be operated via a touchscreen. In other cases, peripheral devices 108 may provide a means for the autonomous vehicle 100 to communicate with other devices located within the vehicle. For example, the microphone 150 may receive audio (e.g., voice commands or other audio input) from the user of the autonomous vehicle 100. Similarly, the speaker 152 may output audio to the user of the autonomous vehicle 100. The wireless communication system 146 may communicate wirelessly with one or more devices, either directly or via a communication network. For example, the wireless communication system 146 may use 3G cellular communication, such as code division multiple access (CDMA), EVDO, Global System for Mobile Communications (GSM) / General Packet Radio Service (GPRS), or 4G cellular communication, such as long term evolution (LTE). Alternatively, it may use 5G cellular communication. The wireless communication system 146 may utilize a wireless local area network (WLAN) for communication. In some embodiments, the wireless communication system 146 may utilize an infrared link, Bluetooth, or ZigBee to communicate directly with devices. Other wireless protocols, such as various vehicle communication systems, may also be used. For example, the wireless communication system 146 may include one or more dedicated short range communications (DSRC) devices that may enable public and / or private data communication between the vehicle and / or a roadside station.

[0188] Power source 110 can provide power to various components of autonomous vehicle 100. In one embodiment, power source 110 can be a rechargeable lithium-ion or lead-acid battery. One or more such battery packs can be configured to provide power to various components of autonomous vehicle 100. In some embodiments, power source 110 and energy source 119 can be implemented together, as is the case in some fully electric vehicles.

[0189] Some or all of the functions of the autonomous vehicle 100 are controlled by a computer system 112. The computer system 112 may include at least one processor 113, which executes instructions 115 stored in a non-transitory computer-readable medium such as memory 114. The computer system 112 may also be multiple computing devices controlling individual components or subsystems of the autonomous vehicle 100 in a distributed manner. The processor 113 may be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor 113 may be a dedicated device such as an application-specific integrated circuit (ASIC) or other hardware-based processor. Although... Figure 1 The processor, memory, and other components of computer system 112 within the same block are functionally illustrated; however, those skilled in the art will understand that the processor or memory may actually include multiple processors or memories not stored in the same physical housing. For example, memory 114 may be a hard disk drive or other storage media located in a housing different from that of computer system 112. Therefore, references to processor 113 or memory 114 will be understood to include a collection of processors or memories that may or may not operate in parallel. Unlike using a single processor to perform the steps described herein, some components, such as steering and deceleration components, may each have their own processor that performs calculations only related to the component's specific function.

[0190] In all the aspects described herein, processor 113 may be located remotely from autonomous vehicle 100 and may communicate wirelessly with autonomous vehicle 100. In other aspects, some of the processes described herein are executed on processor 113 located within autonomous vehicle 100, while others are executed by remote processor 113, including taking the necessary steps to perform a single operation.

[0191] In some embodiments, memory 114 may contain instructions 115 (e.g., program logic) that can be executed by processor 113 to perform various functions of the autonomous vehicle 100, including those described above. Memory 114 may also contain additional instructions, including instructions to send data to, receive data from, interact with, and / or control one or more of the mobility system 102, sensor system 104, control system 106, and peripheral devices 108. In addition to instructions 115, memory 114 may also store data such as road maps, route information, vehicle position, direction, speed, and other such vehicle data, as well as other information. This information may be used by the autonomous vehicle 100 and computer system 112 during operation of the autonomous, semi-autonomous, and / or manual modes. A user interface 116 is provided to or receives information from a user of the autonomous vehicle 100. Optionally, user interface 116 may include one or more input / output devices within the set of peripheral devices 108, such as wireless communication system 146, onboard computer 148, microphone 150, and speaker 152.

[0192] Computer system 112 can control the functions of autonomous vehicle 100 based on input received from various subsystems (e.g., driving system 102, sensor system 104, and control system 106) and from user interface 116. For example, computer system 112 can utilize input from control system 106 to control steering system 132 to avoid obstacles detected by sensor system 104 and obstacle avoidance system 144. In some embodiments, computer system 112 is operable to provide control over many aspects of autonomous vehicle 100 and its subsystems.

[0193] Alternatively, one or more of these components may be installed separately from or associated with the autonomous vehicle 100. For example, the memory 114 may exist partially or completely separately from the autonomous vehicle 100. The components may be communicatively coupled together in a wired and / or wireless manner.

[0194] Optionally, the components described above are merely examples. In actual applications, components in each of the above modules may be added or removed as needed. Figure 1 This should not be construed as a limitation on the embodiments of this application. An autonomous vehicle traveling on a road, such as the autonomous vehicle 100 described above, can identify objects in its surrounding environment to determine adjustments to its current speed. These objects can be other vehicles, traffic control equipment, or other types of objects. In some examples, each identified object can be considered independently, and based on the object's individual characteristics, such as its current speed, acceleration, and distance from the vehicle, the speed adjustment to be made by the autonomous vehicle can be determined.

[0195] Optionally, the autonomous vehicle 100 or the computing device associated with the autonomous vehicle 100, such as Figure 1 The computer system 112, computer vision system 140, and memory 114 can predict the behavior of the identified objects based on the characteristics of the identified objects and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of each other, so all identified objects can be considered together to predict the behavior of a single identified object. The autonomous vehicle 100 can adjust its speed based on the predicted behavior of the identified objects. In other words, the autonomous vehicle 100 can determine what steady state the vehicle will need to adjust to (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the objects. In this process, other factors can also be considered in determining the speed of the autonomous vehicle 100, such as the lateral position of the autonomous vehicle 100 on the road it is traveling on, the curvature of the road, the proximity of static and dynamic objects, etc. In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device can also provide instructions to modify the steering angle of the autonomous vehicle 100 so that the autonomous vehicle 100 follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the autonomous vehicle 100 (e.g., cars in adjacent lanes on the road).

[0196] The aforementioned autonomous vehicle 100 can be a car, truck, motorcycle, bus, ship, airplane, helicopter, lawnmower, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, train, and handcart, etc., and this application embodiment does not impose any special limitations.

[0197] In this embodiment of the application, the processor 113 in the autonomous vehicle 100 is used to execute Figures 2 to 8 The specific manner in which the processor 113 executes each of the above steps in the point cloud data clustering method provided in the corresponding embodiment is the same as that in this application. Figures 2 to 8 The various method embodiments are based on the same concept, and the technical effects they bring are the same as those in this application. Figures 2 to 8 The corresponding method embodiments are the same, and for details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0198] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned actions. Figures 2 to 8 The steps performed by the electronic device in the method described in the illustrated embodiment.

[0199] This application also provides a computer-readable storage medium storing a program that, when run on a computer, causes the computer to perform the aforementioned actions. Figures 2 to 8 The steps performed by the electronic device in the method described in the illustrated embodiment.

[0200] The point cloud data clustering device and electronic device provided in this application embodiment can specifically be a chip. The chip includes a processing unit and a communication unit. The processing unit can be, for example, a processor, and the communication unit can be, for example, an input / output interface, pins, or circuits. The processing unit can execute computer execution instructions stored in the storage unit to cause the chip to perform the above-mentioned operations. Figures 2 to 8 The illustrated embodiment describes a clustering method for point cloud data. Optionally, the storage unit is a storage unit within the chip, such as a register or cache. Alternatively, the storage unit can be a storage unit located outside the chip within the wireless access device, such as a read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, such as random access memory (RAM).

[0201] For details, please refer to Figure 12 , Figure 12 This is a schematic diagram of a chip provided in an embodiment of this application. The chip can be represented as a neural network processor (NPU) 120. The NPU 120 is mounted as a coprocessor on the host CPU, and tasks are assigned by the host CPU. The core part of the NPU is the arithmetic circuit 1203, which is controlled by the controller 1204 to extract matrix data from the memory and perform multiplication operations.

[0202] In some implementations, the arithmetic circuit 1203 internally includes multiple processing engines (PEs). In some implementations, the arithmetic circuit 1203 is a two-dimensional pulsating array. The arithmetic circuit 1203 can also be a one-dimensional pulsating array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 1203 is a general-purpose matrix processor.

[0203] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from the weight memory 1202 and caches it in each PE of the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from the input memory 1201 and performs matrix operations with matrix B. The partial result or the final result of the obtained matrix is ​​stored in the accumulator 1208.

[0204] Unified memory 1206 is used to store input and output data. Weight data is directly transferred to weight memory 1202 via Direct Memory Access Controller (DMAC) 1205. Input data is also transferred to unified memory 1206 via DMAC.

[0205] BIU stands for Bus Interface Unit, which is used for interaction between the AXI bus and the DMAC and the Instruction Fetch Buffer (IFB) 1209.

[0206] The Bus Interface Unit (BIU) 1210 is used by the instruction fetch memory 1209 to fetch instructions from external memory, and also by the memory access controller 1205 to fetch the original data of the input matrix A or the weight matrix B from external memory.

[0207] The DMAC is mainly used to move input data from external memory DDR to unified memory 1206, or to weight data to weight memory 1202, or to input data to input memory 1201.

[0208] The vector computation unit 1207 includes multiple arithmetic processing units that, when needed, further process the output of the computation circuit, such as vector multiplication, vector addition, exponential operations, logarithmic operations, size comparisons, etc. It is mainly used for computation in non-convolutional / fully connected layers of neural networks, such as Batch Normalization, pixel-level summation, and upsampling of feature planes.

[0209] In some implementations, the vector computation unit 1207 can store the processed output vector in the unified memory 1206. For example, the vector computation unit 1207 can apply linear and / or nonlinear functions to the output of the computation circuit 1203, such as performing linear interpolation on the feature planes extracted by the convolutional layer, or, for example, accumulating a vector of values ​​to generate activation values. In some implementations, the vector computation unit 1207 generates normalized values, pixel-level summed values, or both. In some implementations, the processed output vector can be used as an activation input to the computation circuit 1203, for example, for use in subsequent layers of the neural network.

[0210] The instruction fetch buffer 1209 connected to the controller 1204 is used to store the instructions used by the controller 1204;

[0211] Unified memory 1206, input memory 1201, weight memory 1202, and instruction fetch memory 1209 are all on-chip memories. External memory is proprietary to this NPU hardware architecture.

[0212] in, Figures 3 to 9 The operations of each layer in the target neural network shown can be performed by the operation circuit 1203 or the vector calculation unit 1207.

[0213] The processor mentioned above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits used to control the execution of a program in the first aspect of the method.

[0214] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0215] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0216] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0217] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A clustering method for point cloud data, characterized in that, The method includes: Based on the location information of each point in the point cloud data to be processed, multiple points in the point cloud data to be processed are rasterized to obtain raster data to be processed corresponding to the point cloud data to be processed. Obtain the predicted category of each point in the point cloud data to be processed; The predicted category of the first grid is determined based on the predicted category of each point located in the first grid, wherein the first grid is any one of the multiple grids included in the grid data to be processed; Based on the predicted category of each grid cell in the grid data to be processed, determining at least one boundary grid cell from a plurality of grid cells included in the grid data to be processed includes: obtaining a target probability value corresponding to a second grid cell, wherein the second grid cell is any one of at least one target neighbor grid cells of the first grid cell, the target probability value indicating the ratio between a third value corresponding to the second grid cell and a fourth value corresponding to the second grid cell, the third value being the number of grid cells in the neighbor grid cells of the second grid cell that have the same predicted category as the second grid cell, and the fourth value being the total number of neighbor grid cells of the second grid cell; and determining the first grid cell as a boundary grid cell if a preset condition is met based on the target probability value corresponding to each target neighbor grid cell. Based on the predicted category of each raster in the raster data to be processed and at least one boundary raster in the raster data to be processed, a clustering operation is performed to obtain at least one point cloud cluster corresponding to the point cloud data to be processed.

2. The method according to claim 1, characterized in that, Determining the predicted category of the first grid cell based on the predicted category of each point located in the first grid cell includes: The initial predicted category of the first grid is determined based on the predicted category of each point located in the first grid. Obtain at least one target neighboring grid corresponding to the first grid, and determine the final predicted category of the first grid based on the initial predicted category of each target neighboring grid in the at least one target neighboring grid; The step of performing a clustering operation based on the predicted category of each raster in the raster data to be processed to obtain at least one point cloud cluster includes: Based on the final predicted category of each raster in the raster data to be processed, a clustering operation is performed to obtain at least one point cloud cluster.

3. The method according to claim 2, characterized in that, When the initial predicted category of the first grid is a first category, the number of at least one target neighbor grid corresponding to the first grid is a first value. When the initial predicted category of the first grid is a second category, the number of at least one target neighbor grid corresponding to the first grid is a second value. If the size of the object in the first category is different from that of the object in the second category, then the first value and the second value are different.

4. The method according to claim 1, characterized in that, The preset conditions include: the average value of multiple target probability values ​​corresponding to multiple target neighbor grids of the first grid is less than or equal to a first threshold, and the variance of multiple target probability values ​​corresponding to multiple target neighbor grids of the first grid is greater than or equal to a second threshold.

5. A clustering device for point cloud data, characterized in that, The device includes: The rasterization module is used to perform rasterization processing on multiple points in the point cloud data to be processed based on the position information of each point in the point cloud data to be processed, so as to obtain the raster data to be processed corresponding to the point cloud data to be processed. The acquisition module is used to acquire the predicted category of each point in the point cloud data to be processed; The determination module is used to determine the predicted category of the first grid based on the predicted category of each point located in the first grid, wherein the first grid is any one of the multiple grids included in the grid data to be processed; The clustering module is used to perform clustering operations based on the predicted category of each raster in the raster data to be processed, so as to obtain at least one point cloud cluster corresponding to the point cloud data to be processed; The determining module is further configured to determine at least one boundary grid cell from the plurality of grid cells included in the raster data to be processed based on the predicted category of each grid cell in the raster data to be processed. The clustering module is specifically used to perform clustering operations based on the predicted category of each raster in the raster data to be processed and at least one boundary raster in the raster data to be processed; The clustering module is specifically used to: obtain the target probability value corresponding to the second grid, wherein the second grid is any one of the at least one target neighbor grids of the first grid, the target probability value corresponding to the second grid indicates the ratio between a third value corresponding to the second grid and a fourth value corresponding to the second grid, the third value being the number of grids in the neighbor grids of the second grid that have the same predicted category as the second grid, and the fourth value being the total number of neighbor grids of the second grid; and determine the first grid as a boundary grid if a preset condition is met based on the target probability value corresponding to each target neighbor grid.

6. The apparatus according to claim 5, characterized in that, The determining module is specifically used for: The initial predicted category of the first grid is determined based on the predicted category of each point located in the first grid. Obtain at least one target neighboring grid corresponding to the first grid, and determine the final predicted category of the first grid based on the initial predicted category of each target neighboring grid in the at least one target neighboring grid; The clustering module is specifically used to perform clustering operations based on the final predicted category of each raster in the raster data to be processed, so as to obtain at least one point cloud cluster.

7. The apparatus according to claim 6, characterized in that, When the initial predicted category of the first grid is a first category, the number of at least one target neighbor grid corresponding to the first grid is a first value. When the initial predicted category of the first grid is a second category, the number of at least one target neighbor grid corresponding to the first grid is a second value. If the size of the object in the first category is different from that of the object in the second category, then the first value and the second value are different.

8. The apparatus according to claim 5, characterized in that, The preset conditions include: the average value of multiple target probability values ​​corresponding to multiple target neighbor grids of the first grid is less than or equal to a first threshold, and the variance of multiple target probability values ​​corresponding to multiple target neighbor grids of the first grid is greater than or equal to a second threshold.

9. A computer program product, characterized in that, The computer program product includes computer-readable instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that, Includes a program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 4.

11. An electronic device, characterized in that, It includes a processor and a memory, wherein the processor is coupled to the memory. The memory is used to store programs; The processor is configured to execute a program in the memory, causing the electronic device to perform the method as described in any one of claims 1 to 4.

12. The electronic device according to claim 11, characterized in that, The electronic device is a vehicle, radar, monitoring equipment, or data processing equipment in a monitoring system.

Images