Point cloud filtering method and device for removing background point cloud, and vehicle

Abstract

The application provides a point cloud filtering method and device for removing background point clouds, and a vehicle. A plurality of point cloud columns and an initial index are obtained. A plurality of shallow feature maps, a plurality of point cloud column features corresponding to the plurality of shallow feature maps, and a first index are determined according to the plurality of point cloud columns and the initial index. The plurality of shallow feature maps and the first index are input into a preset convolutional backbone network to obtain a plurality of splicing features corresponding to the plurality of point cloud columns, wherein the shallow feature maps and the splicing features correspond one by one. The plurality of splicing features are input into a full connection layer to obtain a plurality of classification labels corresponding to the plurality of point cloud columns, wherein the full connection layer is contained in the preset convolutional backbone network, and the classification label is used to represent an object corresponding to a point cloud in the point cloud column. A target point cloud is determined according to the plurality of classification labels, thereby solving the technical problem that the precision of a 3D target detection small model on a laser radar for detecting a target object is not high in a complex urban scene in the prior art.

Description

Technical Field

[0001] This application relates to the field of intelligent driving, and more specifically, to a point cloud filtering method and apparatus, and a vehicle for removing background point clouds. Background Technology

[0002] In recent years, LiDAR has become increasingly popular in the autonomous driving industry. With the rapid development of deep learning algorithms, it is gradually becoming the main perception method for autonomous vehicles. Currently, the laser-based perception scheme mainly achieves laser perception through 3D target detection algorithms. However, due to the limited computing resources on the vehicle, it is impossible to run a large 3D target detection model. Therefore, small 3D target detection models are usually deployed on the vehicle for laser perception. Although the small model has good target perception effect in simple and limited scenarios such as highways and parks, it has poor detection effect in complex scenarios such as urban areas. The buildings in urban scenes are varied, the green belts of different streets are quite different, and pedestrians, bicycles and electric vehicles frequently appear. These factors all increase the difficulty of detection. The reason why the small model misdetects in complex scenarios is that the small model has a large gap in accuracy, generalization ability and other aspects compared with the large model. The small model PointPillars[1], which is commonly used in the field of autonomous driving perception, ranks 235th in the KITTI 3D target detection list[2], with an average accuracy of 74.31 on medium-difficulty targets. The large model SE-SSD, currently ranked 25th on the KITTI3D object detection leaderboard, has an average accuracy of 82.54 for medium-difficulty targets, showing a significant performance difference. Therefore, existing technologies suffer from technical problems such as false detections by small object detection models due to interference from green belts, buildings, and other factors in complex scenes, resulting in low accuracy in object recognition.

[0003] No effective solutions have yet been proposed for the aforementioned technical problems in the existing technology. Summary of the Invention

[0004] The main purpose of this application is to provide a point cloud filtering method, device, and vehicle for removing background point clouds, so as to solve the technical problem that the accuracy of target detection by the small 3D target detection model on the lidar is not high in the complex urban scene.

[0005] According to one aspect of the present invention, a point cloud filtering method for removing background point clouds is provided, comprising: acquiring a plurality of point cloud pillars and an initial index, wherein the initial index is a coordinate matrix containing the coordinates of a plurality of point cloud pillars corresponding to the plurality of point cloud pillars, and each point cloud pillar contains a plurality of point clouds; determining a plurality of shallow feature maps, a plurality of point cloud pillar features corresponding to the plurality of shallow feature maps, and a first index based on the plurality of point cloud pillars and the initial index, wherein the point cloud pillars and shallow feature maps are correlated. Figure OneA corresponding, the coordinate value in the first index is half of the point cloud column coordinate value in the initial index; control multiple shallow feature maps and the first index input into a preset convolutional backbone network to obtain multiple splicing features corresponding to multiple point cloud columns, wherein the shallow feature map and the splicing feature one-to-one correspondence; control multiple splicing features input into a full connection layer to obtain multiple classification labels corresponding to multiple point cloud columns, wherein the full connection layer is contained in the preset convolutional backbone network, and the classification label is used to represent the object corresponding to the point cloud in the point cloud column; determine the target point cloud according to the multiple classification labels.

[0006] Further, obtaining multiple point cloud columns and an initial index comprises: obtaining an original point cloud; segmenting the original point cloud into multiple point cloud columns with the same shape; obtaining multiple coordinates corresponding to the multiple point cloud columns in a preset bird's eye view plane coordinate system; determining a coordinate matrix composed of the multiple coordinates as the initial index.

[0007] Further, determining multiple shallow feature maps, multiple point cloud column features corresponding to the multiple shallow feature maps, and a first index according to the multiple point cloud columns and the initial index comprises: controlling the point cloud column to input into a point cloud column feature extraction model, performing sparse processing on the point cloud column through the initial index to obtain multiple point cloud column features corresponding to the point cloud column; splicing the multiple point cloud column features to obtain a shallow feature map corresponding to the point cloud column, and obtain multiple shallow feature maps corresponding to the multiple point cloud columns; and performing convolution processing on the initial index through the point cloud column feature extraction model to obtain the first index.

[0008] Further, the preset convolution backbone network comprises a plurality of 2D convolution modules, and the plurality of shallow feature maps and the first index are input into the preset convolution backbone network to obtain a plurality of splicing features corresponding to the plurality of point cloud columns, comprising: step 401: according to the first index, the first feature map is processed by the 2D convolution module to obtain a sparse feature map, wherein the length and width of the first feature map are half of the length and width of the shallow feature map; step 402: the first index is processed by the 2D convolution module to obtain a target index, wherein the coordinate values contained in the target index are half of the coordinate values contained in the first index; step 403: according to the sparse feature map, a first sparse feature is generated; step 404: the number of replication and expansion is determined, and the first sparse feature is replicated and expanded according to the number of replication and expansion to obtain at least one replicated and expanded sparse feature; step 405: the first sparse feature and the at least one replicated and expanded sparse feature are spliced to obtain a target sparse feature; step 406: after each convolution process is completed, the sparse feature map and the target index obtained are input into the next 2D convolution module, and steps 401 to 405 are repeatedly executed to obtain a plurality of target sparse features corresponding to a plurality of convolution processes, wherein one 2D convolution module corresponds to one convolution process; step 407: a plurality of target sparse features obtained after a plurality of convolution processes are completed and point cloud column features corresponding to the shallow feature map are spliced to obtain a splicing feature; step 408: whenever a shallow feature map and a target index are input into the preset convolution backbone network, steps 401 to 407 are repeatedly executed to obtain a plurality of splicing features.

[0009] Further, according to the first index, the first feature map is processed by the 2D convolution module to obtain a sparse feature map, wherein the length and width of the first feature map are half of the length and width of the shallow feature map, comprising: according to the first index, the shallow feature map is screened to retain a plurality of target features with the same coordinates as the coordinates in the first index, and non-target features are assigned a value of 0; the feature map composed of the plurality of target features is determined as the sparse feature map.

[0010] Further, according to the sparse feature map, a first sparse feature is generated, comprising: extracting a plurality of sparse features contained in the sparse feature map; splicing the plurality of sparse features to obtain the first sparse feature.

[0011] Further, the number of replication and expansion is determined, comprising: determining the number of times corresponding to the current convolution process, and determining the number of times corresponding to the current convolution process as the number of replication and expansion.

[0012] Further, in a case that the current convolution process is the first convolution process, the first sparse feature is copied and expanded according to the number of times of copying and expanding to obtain at least one copied and expanded sparse feature, including: obtaining n coordinate values in the first sparse feature, wherein n is a natural number starting from 1, and n is less than or equal to 2n-1; obtaining a preset relationship table, and searching for 2n target coordinate values corresponding to the n coordinate values, wherein in the preset relationship table, one coordinate value corresponds to two target coordinate values; and placing the 2n target coordinate values into corresponding spaces to obtain the copied and expanded sparse feature.

[0013] Further, according to the plurality of classification identifiers, the target point cloud is determined: in a case that the classification identifier output by the full connection layer is a first identifier, the point cloud in the point cloud column is determined as the point cloud corresponding to the target object; in a case that the classification identifier output by the full connection layer is a second identifier, the point cloud in the point cloud column is determined as the point cloud corresponding to the background; and in a case that the classification identifier output by the full connection layer is a third identifier, the point cloud in the point cloud column is determined as the point cloud corresponding to the ground. The point cloud corresponding to the target object and the point cloud corresponding to the ground are determined as the target point cloud.

[0014] According to another aspect of the embodiment of the present application, a point cloud filtering device for removing background point clouds is also provided, including: a first obtaining unit configured to obtain a plurality of point cloud columns and an initial index, wherein the initial index is a coordinate matrix containing a plurality of point cloud column coordinates corresponding to the plurality of point cloud columns, and each point cloud column contains a plurality of point clouds; a first determining unit configured to determine, according to the plurality of point cloud columns and the initial index, a plurality of shallow feature maps, a plurality of point cloud column features corresponding to the plurality of shallow feature maps, and a first index, wherein the point cloud column and the shallow feature map correspond to each other, and the coordinate value in the first index is half of the point cloud column coordinate value in the initial index; a first control unit configured to control the plurality of shallow feature maps and the first index to be input into a preset convolution backbone network to obtain a plurality of splicing features corresponding to the plurality of point cloud columns, wherein the shallow feature map and the splicing feature correspond to each other; a second control unit configured to control the plurality of splicing features to be input into a full connection layer to obtain a plurality of classification identifiers corresponding to the plurality of point cloud columns, wherein the full connection layer is contained in the preset convolution backbone network, and the classification identifier is used to represent the object corresponding to the point cloud in the point cloud column; and a second determining unit configured to determine the target point cloud according to the plurality of classification identifiers. Figure One

[0015] According to another aspect of the embodiment of the present application, a vehicle is also provided, including a laser radar and a point cloud filtering device for removing background point clouds, and the point cloud filtering device for removing background point clouds is used to execute the point cloud filtering method for removing background point clouds.

[0016] ​According to another aspect of the present invention, an electronic device is also provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include methods for performing point cloud filtering for removing background point clouds.

[0017] In this embodiment of the invention, multiple point cloud pillars and an initial index are obtained, wherein the initial index is a coordinate matrix containing the coordinates of multiple point cloud pillars corresponding to the multiple point cloud pillars, and each point cloud pillar contains multiple point clouds; based on the multiple point cloud pillars and the initial index, multiple shallow feature maps, multiple point cloud pillar features corresponding to the multiple shallow feature maps, and a first index are determined, wherein the point cloud pillars and shallow feature maps are... Figure One In a one-to-one correspondence, the coordinate value in the first index is half the coordinate value of the point cloud pillar in the initial index. Multiple shallow feature maps and the first index are input into a preset convolutional backbone network to obtain multiple stitched features corresponding to multiple point cloud pillars. The shallow feature maps and stitched features correspond one-to-one. Multiple stitched features are input into a fully connected layer to obtain multiple classification labels corresponding to multiple point cloud pillars. The fully connected layer is included in the preset convolutional backbone network, and the classification labels are used to characterize the objects corresponding to the point clouds in the point cloud pillars. Based on multiple classification labels, the target point cloud is determined, which solves the technical problem of low accuracy in detecting targets by small 3D target detection models on LiDAR in complex urban scenes. By classifying the point cloud, the technical effect of improving the detection accuracy of the target object is achieved. Attached Figure Description

[0018] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0019] Figure 1 This is a flowchart of a point cloud filtering method for removing background point clouds according to an embodiment of this application;

[0020] Figure 2 A schematic diagram illustrating the process of the point cloud filtering method for removing background point clouds provided in this application;

[0021] Figure 3 A schematic diagram illustrating the replication-extended sparse features provided in this application;

[0022] Figure 4 This is a schematic diagram of a point cloud filtering device for removing background point clouds according to an embodiment of this application. Detailed Implementation

[0023] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other in the case of no conflict. The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0024] In order for those skilled in the art to better understand the scheme of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0026] It should be understood that when an element (such as a layer, film, region, or substrate) is described as "on" another element, it can be directly on the other element, or there can be an intermediate element. Also, in the specification and claims, when it is described that an element is "connected" to another element, the element can be "directly connected" to the other element, or "connected" to the other element through a third element.

[0027] PointNet: is a model proposed by Charles R. Qi et al. of Stanford University, which can directly process point cloud. For each point in the input point cloud, learn its corresponding spatial code, and then use the features of all points to get a global point cloud feature.

[0028] As mentioned in the background, the existing technology has the technical problem that the precision of the 3D target detection small model on the laser radar for detecting target objects is not high in the complex urban scene. In order to solve the above problem, in a typical embodiment of the present application, a point cloud filtering method and device for removing background point cloud and a vehicle are provided.

[0029] According to the embodiments of the present application, a point cloud filtering method for removing background point cloud is provided.

[0030] In order to improve the detection accuracy of small models in urban scenes, the patent proposes a point cloud filtering method for removing background point clouds to filter out point clouds in background areas such as buildings and green belts in urban scenes, and then send the filtered point clouds to the small model 3D target detection algorithm for perception. Since the background point cloud is filtered out, there is no problem of false detection in the green belt area.

[0031] Figure 1 is a flowchart of a point cloud filtering method for removing background point clouds according to an embodiment of the present application. As shown in Figure 1 , the method comprises the following steps:

[0032] Step S101, obtaining a plurality of point cloud columns and an initial index, wherein the initial index is a coordinate matrix containing a plurality of point cloud column coordinates corresponding to the plurality of point cloud columns, and each point cloud column contains a plurality of point clouds.

[0033] Therefore, the present application proposes a semantic segmentation network, which obtains regular point cloud columns and initial indexes corresponding to the point cloud columns by segmenting the point clouds obtained by the laser radar, wherein the initial index contains the coordinates corresponding to the segmented point cloud columns.

[0034] Step S102, determining a plurality of shallow feature maps, a plurality of point cloud column features corresponding to the plurality of shallow feature maps, and a first index according to the plurality of point cloud columns and the initial index, wherein the point cloud column corresponds to the shallow feature map, and the coordinate value in the first index is half of the point cloud column coordinate value in the initial index. The point cloud column and the initial index are input into PointNet to obtain the shallow feature map (the same as the shallow feature extraction method of PointPillars). Figure One

[0035] Step S103, controlling the plurality of shallow feature maps and the first index to be input into a preset convolutional backbone network to obtain a plurality of splicing features corresponding to the plurality of point cloud columns, wherein the shallow feature map corresponds to the splicing feature one by one.

[0036] Step S104, controlling the plurality of splicing features to be input into a full connection layer to obtain a plurality of classification identifiers corresponding to the plurality of point cloud columns, wherein the full connection layer is contained in the preset convolutional backbone network, and the classification identifier is used to represent the object corresponding to the point cloud in the point cloud column.

[0037] Step S105, determining a target point cloud according to the plurality of classification identifiers.

[0038] ​Therefore, the point cloud filtering method for removing background point clouds provided in the application proposes a new point cloud column semantic segmentation algorithm, classifies the point cloud columns, and filters out the background point clouds using the classification results. The filtered point clouds are input into an existing 3D target detection small model for laser perception. Since the background point clouds have been removed, the phenomenon of false detection in the background point cloud area will not occur. Meanwhile, the 2D convolution backbone network proposed in the application uses an index to sparsify the dense feature map, and then performs convolution on the sparsified feature map. This avoids the influence of the zero-value point cloud column expansion feature on the point cloud column feature, so that the point cloud column feature has stronger representation ability and can improve the classification accuracy. Since the model structure is simple (one PointNet, three ordinary convolution layers, and one fully connected layer), the inference speed is fast, and it is very suitable for point cloud preprocessing.

[0039] In an optional embodiment, the obtaining the plurality of point cloud columns and the initial index comprises: obtaining an original point cloud; segmenting the original point cloud into a plurality of point cloud columns of the same shape; obtaining a plurality of coordinates corresponding to the plurality of point cloud columns in a preset bird's eye view plane coordinate system; and determining a coordinate matrix composed of the plurality of coordinates as the initial index. Wherein, Figure 2 The process diagram of the point cloud filtering method for removing background point clouds provided in the application is shown in Figure 2 The initial index is {P, 2}.

[0040] Further, according to the plurality of point cloud columns and the initial index, the plurality of shallow layer feature maps, the plurality of point cloud column features corresponding to the plurality of shallow layer feature maps, and the first index are determined, comprising: controlling the point cloud column to be input into a point cloud column feature extraction model, and sparsifying the point cloud column through the initial index to obtain a plurality of point cloud column features corresponding to the point cloud column; splicing the plurality of point cloud column features to obtain a shallow layer feature map corresponding to the point cloud column, and obtaining a plurality of shallow layer feature maps corresponding to the plurality of point cloud columns; and performing convolution processing on the initial index through the point cloud column feature extraction model to obtain the first index.

[0041] In the above embodiment, the point cloud column is sparsified by the point cloud column feature extraction layer of PointPillars, and the specific process diagram is shown in Figure 2 The dense point cloud column is sparsified through the initial index {P, 2} to obtain the point cloud column feature (the point cloud column feature is obtained by sparsifying the point cloud column through the initial index), and the point cloud column feature is spliced into a shallow layer feature map {H, W, C}. Meanwhile, it should be noted that the coordinate values in the initial index are all halved after the initial index passes through the point cloud column extraction model, and the first index is obtained. That is, the initial index is also sparsified to obtain the first index.

[0042] After obtaining the shallow feature map {H, W, C}, the shallow feature map and the first index are input into a 2D convolution backbone network. The 2D backbone convolution network proposed in the application comprises three 2D convolution layers, and a plurality of spliced features corresponding to a plurality of point cloud columns are obtained. The specific steps are as follows:

[0043] Step 401: According to the first index, the first feature map is processed by a 2D convolution module to obtain a sparse feature map, wherein the length and width of the first feature map are half of the length and width of the shallow feature map.

[0044] Specifically, according to the first index, the first feature map is processed by a 2D convolution module to obtain a sparse feature map, wherein the length and width of the first feature map are half of the length and width of the shallow feature map, including: according to the first index, the shallow feature map is filtered to retain a plurality of target features with the same coordinates as the coordinates in the first index, and non-target features are assigned a value of 0; the feature map composed of the plurality of target features is determined as the sparse feature map.

[0045] The above is an example of the convolution process of the first 2D convolution module in the 2D convolution backbone network. Through the first index {P, 2}, the first feature map {H / 2, W / 2, C} is sparse filtered, only features with coordinates appearing in the first index {P1, 2} are retained, other features are set to 0, to obtain a sparse feature map {H / 2, W / 2, C}, and the features are extracted from the sparse feature map to obtain the first sparse feature {P1, C} obtained after the convolution process of the first 2D convolution module.

[0046] Step 402: The first index is processed by a 2D convolution module to obtain a target index, wherein the coordinate values contained in the target index are half of the coordinate values contained in the first index.

[0047] Then, when the first feature map {H / 2, W / 2, C} is processed by the first 2D convolution module for the first time, the first feature map {H / 2, W / 2, C} is sparse processed by the first index to obtain a sparse feature map {H / 2, W / 2, C}, and at the same time, the coordinate values in the first index are also halved to obtain a target index {P1, 2}.

[0048] Step 403: According to the sparse feature map, a first sparse feature is generated.

[0049] Specifically, a plurality of sparse features contained in the sparse feature map are extracted, and the plurality of sparse features are spliced to obtain a first sparse feature.

[0050] Step 404: determining the number of times of copying and expanding, and copying and expanding the first sparse feature according to the number of times of copying and expanding to obtain at least one copied and expanded sparse feature;

[0051] Step 405: splicing the first sparse feature and the at least one copied and expanded sparse feature to obtain a target sparse feature;

[0052] Since the convolution module contained in the 2D backbone convolution network provided in the present application is 3, the feature map input into the backbone convolution network needs to be convolved three times, and the first sparse feature obtained in the first convolution process needs to be copied and expanded once to obtain the first copied and expanded sparse feature, and so on. In the second convolution process, the first sparse feature needs to be copied and expanded twice, and in the third convolution process, the first sparse feature needs to be copied and expanded three times. The first sparse feature and the at least one copied and expanded sparse feature obtained by copying and expanding are spliced to obtain a target sparse feature. It should be noted that a target sparse feature is obtained after each convolution process is completed, and multiple target sparse features are obtained after multiple convolution processes are completed.

[0053] It should be noted that after each convolution process is completed, the index input into the corresponding convolution module is sparsified, and the coordinate value in the index after sparsification is half of the coordinate value corresponding to the index input into the convolution module.

[0054] Step 406: after the sparse feature map and the target index obtained after each convolution process are input into the next 2D convolution module, steps 401 to 405 are repeatedly executed to obtain multiple target sparse features corresponding to multiple convolution processes, wherein one 2D convolution module corresponds to one convolution process.

[0055] Step 407: splicing the multiple target sparse features obtained after multiple convolution processes are completed and the point cloud column feature corresponding to the shallow feature map to obtain a spliced feature;

[0056] As described above, since the 2D convolution backbone network provided in the present application provides three convolution modules, the target sparse features obtained in the three convolution processes of the three convolution modules and the point cloud column features extracted by the point cloud column feature extraction layer are spliced to obtain the spliced feature sought in the present application.

[0057] It should be noted that several target sparse features are obtained after several convolutions of the 2D convolution backbone network, and the last obtained target sparse feature and the point cloud column feature are spliced to obtain the spliced feature.

[0058] Step 408: whenever there is a shallow feature map and a target index, input into the preset convolutional backbone network, repeat steps 401-407 to obtain a plurality of spliced features. In each convolutional module, repeat the above steps to obtain the target sparse feature corresponding to each convolutional module, and splice the target sparse feature obtained after each convolution process with the point cloud column feature obtained at the beginning to obtain the final spliced feature.

[0059] In an optional embodiment, determining the number of replication expansions comprises: determining the number of times corresponding to the current convolution process, and determining the number of times corresponding to the current convolution process as the number of replication expansions.

[0060] As described above, Figure 2 As shown in the first sparse feature corresponding to each convolution process is obtained after each convolution module completes the corresponding convolution process, the first sparse feature obtained after the first convolution process is {P1, C}, the replicated and expanded sparse feature {P, C} is obtained after one replication expansion, the first sparse feature obtained after the second convolution process is {P2, 2C}, two replicated and expanded sparse features {P1, 2C} and {P, 2C} are obtained after two replication expansions, and correspondingly, the first sparse feature obtained after the third convolution process is {P3, 4C}, three first sparse features {P2, 4C}, {P1, 4C}, and {P, 4C} are obtained after three replication expansions.

[0061] In the case where the current convolution process is the first convolution process, replicating and expanding the first sparse feature according to the number of replication expansions to obtain at least one replicated and expanded sparse feature comprises: obtaining n coordinate values in the first sparse feature, wherein n is a natural number starting from 1; obtaining a preset relationship table, and finding 2n target coordinate values according to the n coordinate values, wherein in the preset relationship table, one coordinate value corresponds to two target coordinate values; and placing the 2n target coordinate values into the corresponding space to obtain the replicated and expanded sparse feature.

[0062] Specifically, a schematic diagram of the replicated and expanded sparse feature is shown in Figure 3 , Figure 3 Taking the first feature map {H / 2, W / 2, C} as an example in the first sparse feature corresponding to each convolution process is obtained after each convolution module completes the corresponding convolution process, the first sparse feature obtained after the first convolution process is {P1, C}, the replicated and expanded sparse feature {P, C} is obtained after one replication expansion, the first sparse feature obtained after the second convolution process is {P2, 2C}, two replicated and expanded sparse features {P1, 2C} and {P, 2C} are obtained after two replication expansions, and correspondingly, the first sparse feature obtained after the third convolution process is {P3, 4C}, three first sparse features {P2, 4C}, {P1, 4C}, and {P, 4C} are obtained after three replication expansions.

[0063]

[0064] Further, the first sparse feature {P1, C} is subjected to a convolution operation of the 2D module to obtain a target index {P2, 2} for screening features in a second convolution module, as shown in Figure 3 {P2, 2} is a coordinate matrix containing (0, 0, 0), (0, 1, 0), and (0, 1, 1), and the extension of {P2, 2} is determined by copying and extending the relationship coordinates in the preset relationship table to determine the two target coordinate values corresponding to each coordinate value, and then the extension of {P2, 2} is performed, and the corresponding relationship table between {P1, 2} and {P2, 2} in Figure 3 , (0, 1, 0)-(0, 2, 1) and (0, 3, 0), (0, 1, 1)-(0, 2, 3) and (0, 3, 3), so that the coordinate values contained in the copied and extended sparse feature are determined through a one-to-two relationship, that is, the coordinate values contained in {P1, C}. As can be seen from the above description, the copied and extended sparse feature is {P1, C}.

[0065] Further, through the corresponding relationship between the first index {P1, 2} and the target index {P2, 2}, the feature {P2, 2C} can be copied and extended to {P1, 2C}. The effect of sparsifying the dense feature map is to prevent zero-value point cloud from participating in subsequent convolution calculation. The point cloud column on the bird's eye view is sparse, and the point cloud column where no point cloud exists is filled with zero. The zero-value point cloud column participates in the convolution calculation to make the activation space expand and increase the number of active points (point cloud column indexes) in the space. These increased active points will weaken the representation ability of the point cloud column feature, so it is not desirable for the zero-value point cloud column to participate in the subsequent convolution. This application imitates the sub-flow type sparse convolution, first sparsifies the dense feature map through the point cloud column index, and then inputs the sparsified feature into the ordinary convolution, so that the zero-value point cloud column does not participate in the subsequent convolution operation. After copying and extending, the length of all features becomes P. Then, the four features obtained by splicing the three features obtained by the 2D convolution backbone network and the feature obtained by the point cloud column feature extraction layer are obtained. The feature {P, 8C} is input into the point cloud column classification network (only one fully connected layer) to obtain the classification result, that is, the classification identifier.

[0066] In an optional embodiment, the target point cloud is determined according to the classification labels: in a case where the classification label output by the full connection layer is a first label, the point cloud in the point cloud column is determined as the point cloud corresponding to the target object; in a case where the classification label output by the full connection layer is a second label, the point cloud in the point cloud column is determined as the point cloud corresponding to the background; in a case where the classification label output by the full connection layer is a third label, the point cloud in the point cloud column is determined as the point cloud corresponding to the ground. The point cloud corresponding to the target object and the point cloud corresponding to the ground are determined as the target point cloud. In the present application, the classification label is divided into three categories, 0, 1 and 2. 0 represents that the point cloud column is the target object, for example, a target vehicle; 1 represents that the point cloud column is the ground; and 2 represents that the point cloud column is the background (buildings, green belts, etc.). The point cloud column is screened using the classification label. The point clouds in the point cloud columns of 0 and 1 are retained, so as to remove the background point cloud. The target point cloud obtained after classification is input into the 3D target detection small model for perception, which can greatly reduce the probability of false detection.

[0067] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0068] The embodiment of the present application also provides a point cloud filtering device for removing background point clouds. It should be noted that the point cloud filtering device for removing background point clouds provided by the embodiment of the present application can be used to execute the point cloud filtering method for removing background point clouds provided by the embodiment of the present application. Hereinafter, the point cloud filtering device for removing background point clouds provided by the embodiment of the present application is introduced.

[0069] Figure 4 is a schematic diagram of a point cloud filtering device for removing background point clouds according to the embodiment of the present application. As shown in Figure 4 the device comprises: a first acquisition unit 401, configured to acquire a plurality of point cloud columns and an initial index, wherein the initial index is a coordinate matrix containing a plurality of point cloud column coordinates corresponding to the plurality of point cloud columns, and each point cloud column contains a plurality of point clouds; a first determination unit 402, configured to determine a plurality of shallow feature maps, a plurality of point cloud column features corresponding to the plurality of shallow feature maps and a first index according to the plurality of point cloud columns and the initial index, wherein the point cloud column and the shallow feature Figure Onecorresponding to the plurality of point cloud columns, wherein the plurality of shallow feature maps and the plurality of spliced features are one-to-one corresponding; the second control unit 404 is configured to control the plurality of spliced features to be input into a full connection layer to obtain a plurality of classification labels corresponding to the plurality of point cloud columns, wherein the full connection layer is contained in the preset convolutional backbone network, and the classification label is used to represent an object corresponding to the point cloud in the point cloud column; and the second determination unit 405 is configured to determine the target point cloud according to the plurality of classification labels.

[0070] In an optional embodiment, the first acquisition unit 401 comprises: a first acquisition subunit configured to acquire an original point cloud; a first segmentation subunit configured to segment the original point cloud into a plurality of point cloud columns with the same shape; a second acquisition subunit configured to acquire a plurality of coordinates corresponding to the plurality of point cloud columns in a preset bird's eye view plane coordinate system; and a first determination subunit configured to determine a coordinate matrix composed of the plurality of coordinates as an initial index.

[0071] In an optional embodiment, the first determination unit 402 comprises: a first control subunit configured to control the point cloud column to be input into a point cloud column feature extraction model, and perform sparse processing on the point cloud column through the initial index to obtain a plurality of point cloud column features corresponding to the point cloud column; a first splicing subunit configured to splice the plurality of point cloud column features to obtain a shallow feature map corresponding to the point cloud column, and obtain a plurality of shallow feature maps corresponding to the plurality of point cloud columns; and a first convolution processing subunit configured to perform convolution processing on the initial index through the point cloud column feature extraction model to obtain a first index.

[0072] In an optional embodiment, the preset convolution backbone network comprises a plurality of 2D convolution modules, and the first control unit 403 comprises: a second convolution processing subunit configured to perform convolution processing on the first feature map by the 2D convolution module according to the first index to obtain a sparse feature map, wherein the length and width of the first feature map are half of the length and width of the shallow layer feature map; a third convolution processing subunit configured to perform convolution processing on the first index by the 2D convolution module to obtain a target index, wherein the coordinate values contained in the target index are half of the coordinate values contained in the first index; a generation subunit configured to generate a first sparse feature according to the sparse feature map; a copy and expansion subunit configured to determine a copy and expansion number and copy and expand the first sparse feature according to the copy and expansion number to obtain at least one copied and expanded sparse feature; a second splicing subunit configured to splice the first sparse feature and the at least one copied and expanded sparse feature to obtain a target sparse feature; a first input subunit configured to, after inputting the sparse feature map and the target index obtained after each convolution process into the next 2D convolution module, repeatedly execute steps 401 to 405 to obtain a plurality of target sparse features corresponding to a plurality of convolution processes, wherein one 2D convolution module corresponds to one convolution process; a third splicing subunit configured to splice the plurality of target sparse features obtained after the plurality of convolution processes and the point cloud column feature corresponding to the shallow layer feature map to obtain a spliced feature; and a second input subunit configured to, whenever a shallow layer feature map and a target index are input into the preset convolution backbone network, repeatedly execute the above steps to obtain a plurality of spliced features.

[0073] In an optional embodiment, the second convolution processing subunit comprises: a screening module configured to screen the shallow layer feature map according to the first index to retain a plurality of target features whose coordinates are the same as the coordinates in the first index and assign a non-target feature a value of 0; and a first determination module configured to determine a feature map constituted by the plurality of target features as the sparse feature map.

[0074] In an optional embodiment, the generation subunit comprises: an extraction module configured to extract a plurality of sparse features contained in the sparse feature map; and a splicing module configured to splice the plurality of sparse features to obtain the first sparse feature.

[0075] In an optional embodiment, the copy and expansion subunit comprises: a second determination module configured to determine a number corresponding to the current convolution process and determine the number corresponding to the current convolution process as the copy and expansion number.

[0076] In an optional embodiment, in the case that the current convolution process is the first convolution process, the copying and expanding subunit comprises: a first obtaining module configured to obtain n coordinate values in the first sparse feature; a searching module configured to obtain a preset relationship table and search for 2n target coordinate values according to the n coordinate values, wherein in the preset relationship table, one coordinate value corresponds to two target coordinate values; and a second obtaining module configured to place the 2n target coordinate values into corresponding spaces to obtain a copied and expanded sparse feature.

[0077] In an optional embodiment, the second control unit 404 comprises: a second determining subunit configured to determine the point cloud in the point cloud column as the point cloud corresponding to the target object in the case that the classification label output by the full connection layer is the first label; a third determining subunit configured to determine the point cloud in the point cloud column as the point cloud corresponding to the background in the case that the classification label output by the full connection layer is the second label; a fourth determining subunit configured to determine the point cloud in the point cloud column as the point cloud corresponding to the ground in the case that the classification label output by the full connection layer is the third label; and a fifth determining subunit configured to determine the point cloud corresponding to the target object and the point cloud corresponding to the ground as the target point cloud.

[0078] A point cloud filtering device for removing background point clouds comprises a processor and a memory, and the above-mentioned first obtaining unit and the like are stored in the memory as program units, and the corresponding functions are realized by the processor executing the above-mentioned program units stored in the memory.

[0079] The processor comprises a core, and the core retrieves the corresponding program units from the memory. The core can be one or more, and the technical problem that the precision of the 3D target detection small model on the laser radar for detecting the target object is not high in the prior art can be solved by adjusting the core parameters.

[0080] The memory can comprise a non-permanent memory in a computer readable medium, a random access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory comprises at least one memory chip.

[0081] The embodiment of the present application provides a computer readable storage medium, which stores a program, and the program is executed by a processor to realize a point cloud filtering method for removing background point clouds.

[0082] The embodiment of the present application provides a processor, which is used for running a program, and the program is executed to realize a point cloud filtering method for removing background point clouds.

[0083] The embodiment of the present application provides a device, the device comprises a processor, a memory and a program stored on the memory and executable on the processor, and at least the following steps are realized when the processor executes the program: a plurality of point cloud columns and an initial index are acquired, wherein the initial index is a coordinate matrix containing a plurality of point cloud column coordinates corresponding to the plurality of point cloud columns, and each point cloud column contains a plurality of point clouds; according to the plurality of point cloud columns and the initial index, a plurality of shallow feature maps, a plurality of point cloud column features corresponding to the plurality of shallow feature maps and a first index are determined, wherein the point cloud column and the shallow feature map are one-to-one correspondence, and the coordinate value in the first index is one-half of the point cloud column coordinate value in the initial index; the plurality of shallow feature maps and the first index are controlled to be input into a preset convolutional backbone network to obtain a plurality of splicing features corresponding to the plurality of point cloud columns, wherein the shallow feature map and the splicing feature are one-to-one correspondence; the plurality of splicing features are controlled to be input into a full connection layer to obtain a plurality of classification labels corresponding to the plurality of point cloud columns, wherein the full connection layer is contained in the preset convolutional backbone network, and the classification label is used to represent the object corresponding to the point cloud in the point cloud column; and a target point cloud is determined according to the plurality of classification labels. Figure One

[0084] Optionally, the plurality of point cloud columns and the initial index are acquired, comprising: acquiring an original point cloud; the original point cloud is segmented into a plurality of point cloud columns with the same shape; a plurality of coordinates corresponding to the plurality of point cloud columns in a preset bird's eye view plane coordinate system are acquired; and the coordinate matrix composed of the plurality of coordinates is determined as the initial index.

[0085] Optionally, according to the plurality of point cloud columns and the initial index, the plurality of shallow feature maps, the plurality of point cloud column features corresponding to the plurality of shallow feature maps and the first index are determined, comprising: the point cloud column is controlled to be input into a point cloud column feature extraction model, the point cloud column is sparsified through the initial index to obtain a plurality of point cloud column features corresponding to the point cloud column; the plurality of point cloud column features are spliced to obtain a shallow feature map corresponding to the point cloud column, and a plurality of shallow feature maps corresponding to the plurality of point cloud columns are obtained; and the initial index is convoluted through the point cloud column feature extraction model to obtain the first index.

[0086] ​Optionally, the preset convolution backbone network comprises a plurality of 2D convolution modules, and the plurality of shallow feature maps and the first index are input into the preset convolution backbone network to obtain a plurality of splicing features corresponding to a plurality of point cloud columns, comprising: step 401: performing convolution processing on the first feature map by the 2D convolution module according to the first index to obtain a sparse feature map, wherein the length and width of the first feature map are half of the length and width of the shallow feature map; step 402: performing convolution processing on the first index by the 2D convolution module to obtain a target index, wherein the coordinate values contained in the target index are half of the coordinate values contained in the first index; step 403: generating a first sparse feature according to the sparse feature map; step 404: determining a replication expansion number and replicating and expanding the first sparse feature according to the replication expansion number to obtain at least one replicated and expanded sparse feature; step 405: splicing the first sparse feature and the at least one replicated and expanded sparse feature to obtain a target sparse feature; step 406: after the sparse feature map and the target index obtained after each convolution process are input into the next 2D convolution module, steps 401 to 405 are repeatedly executed to obtain a plurality of target sparse features corresponding to a plurality of convolution processes, wherein one 2D convolution module corresponds to one convolution process; step 407: splicing the plurality of target sparse features obtained after the plurality of convolution processes are completed and the point cloud column features corresponding to the shallow feature map to obtain a splicing feature; step 408: whenever a shallow feature map and a target index are input into the preset convolution backbone network, steps 401 to 407 are repeatedly executed to obtain a plurality of splicing features.

[0087] Optionally, the first feature map is convoluted by the 2D convolution module according to the first index to obtain a sparse feature map, wherein the length and width of the first feature map are half of the length and width of the shallow feature map, comprising: screening the shallow feature map according to the first index to retain a plurality of target features whose coordinates are the same as the coordinates in the first index, and assigning non-target features a value of 0; the feature map composed of the plurality of target features is determined as the sparse feature map.

[0088] Optionally, the first sparse feature is generated according to the sparse feature map, comprising: extracting a plurality of sparse features contained in the sparse feature map; splicing the plurality of sparse features to obtain the first sparse feature.

[0089] Optionally, the replication expansion number is determined, comprising: determining a number corresponding to the current convolution process, and determining the number corresponding to the current convolution process as the replication expansion number.

[0090] Optionally, when the current convolution process is the first convolution process, the first sparse feature is copied and expanded according to the number of copy expansions to obtain at least one copied and expanded sparse feature, including: obtaining n coordinate values ​​from the first sparse feature, where n is a natural number starting from 1; obtaining a preset relation table, and finding 2n target coordinate values ​​corresponding to the n coordinate values, where in the preset relation table, one coordinate value corresponds to two target coordinate values; and putting the 2n target coordinate values ​​into the corresponding space to obtain the copied and expanded sparse feature.

[0091] Optionally, the target point cloud is determined based on multiple classification identifiers: if the classification identifier output by the fully connected layer is the first identifier, the point cloud in the point cloud column is determined to be the point cloud corresponding to the target object; if the classification identifier output by the fully connected layer is the second identifier, the point cloud in the point cloud column is determined to be the point cloud corresponding to the background; if the classification identifier output by the fully connected layer is the third identifier, the point cloud in the point cloud column is determined to be the point cloud corresponding to the ground; the point cloud corresponding to the target object and the point cloud corresponding to the ground are determined as the target point cloud. The device mentioned in this paper can be a server, PC, PAD, mobile phone, etc.

[0092] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having at least the following method steps: obtaining multiple point cloud pillars and an initial index, wherein the initial index is a coordinate matrix containing the coordinates of multiple point cloud pillars corresponding to the multiple point cloud pillars, and each point cloud pillar contains multiple point clouds; determining multiple shallow feature maps, multiple point cloud pillar features corresponding to the multiple shallow feature maps, and a first index based on the multiple point cloud pillars and the initial index, wherein the point cloud pillars and shallow feature maps are... Figure One In a one-to-one correspondence, the coordinate value in the first index is half of the coordinate value of the point cloud pillar in the initial index; multiple shallow feature maps and the first index are input into a preset convolutional backbone network to obtain multiple spliced ​​features corresponding to multiple point cloud pillars, wherein the shallow feature maps and spliced ​​features correspond one-to-one; multiple spliced ​​features are input into a fully connected layer to obtain multiple classification labels corresponding to multiple point cloud pillars, wherein the fully connected layer is included in the preset convolutional backbone network, and the classification labels are used to characterize the objects corresponding to the point clouds in the point cloud pillars; based on the multiple classification labels, the target point cloud is determined.

[0093] Optionally, obtaining multiple point cloud pillars and an initial index includes: obtaining the original point cloud; dividing the original point cloud into multiple point cloud pillars of the same shape; obtaining multiple coordinates corresponding to the multiple point cloud pillars in a preset bird's-eye view plane coordinate system; and determining the coordinate matrix formed by the multiple coordinates as the initial index.

[0094] Optionally, according to the plurality of point cloud columns and the initial index, the plurality of shallow feature maps, the plurality of point cloud column features corresponding to the plurality of shallow feature maps, and the first index are determined, including: controlling the point cloud column to be input into the point cloud column feature extraction model, and performing sparse processing on the point cloud column through the initial index to obtain a plurality of point cloud column features corresponding to the point cloud column; splicing the plurality of point cloud column features to obtain a shallow feature map corresponding to the point cloud column, and obtain a plurality of shallow feature maps corresponding to the plurality of point cloud columns; and performing convolution processing on the initial index through the point cloud column feature extraction model to obtain the first index.

[0095] Optionally, the preset convolutional backbone network includes a plurality of 2D convolutional modules, and the plurality of shallow feature maps and the first index are input into the preset convolutional backbone network to obtain a plurality of spliced features corresponding to the plurality of point cloud columns, including: step 401: according to the first index, performing convolution processing on the first feature map through the 2D convolutional module to obtain a sparse feature map, wherein the length and width of the first feature map are half of the length and width of the shallow feature map; step 402: performing convolution processing on the first index through the 2D convolutional module to obtain a target index, wherein the coordinate values contained in the target index are half of the coordinate values contained in the first index; step 403: generating a first sparse feature according to the sparse feature map; step 404: determining a replication expansion number, and replicating and expanding the first sparse feature according to the replication expansion number to obtain at least one replicated and expanded sparse feature; step 405: splicing the first sparse feature and the at least one replicated and expanded sparse feature to obtain a target sparse feature; step 406: after the sparse feature map and the target index obtained after each convolution process are input into the next 2D convolutional module, steps 401 to 405 are repeatedly executed to obtain a plurality of target sparse features corresponding to a plurality of convolution processes, wherein one 2D convolutional module corresponds to one convolution process; step 407: splicing the plurality of target sparse features obtained after the plurality of convolution processes and the point cloud column features corresponding to the shallow feature map to obtain the spliced feature; and step 408: whenever a shallow feature map and a target index are input into the preset convolutional backbone network, steps 401 to 407 are repeatedly executed to obtain a plurality of spliced features.

[0096] Optionally, according to the first index, the first feature map is convoluted through the 2D convolutional module to obtain a sparse feature map, wherein the length and width of the first feature map are half of the length and width of the shallow feature map, including: according to the first index, the shallow feature map is screened to retain a plurality of target features with the same coordinates as the coordinates in the first index, and non-target features are assigned a value of 0; and the feature map composed of the plurality of target features is determined as the sparse feature map.

[0097] Optionally, the first sparse feature is generated according to the sparse feature map, including: extracting a plurality of sparse features contained in the sparse feature map; and splicing the plurality of sparse features to obtain the first sparse feature.

[0098] Optionally, the number of replication expansions is determined, including: determining a number of times corresponding to the current convolution process, and determining the number of times corresponding to the current convolution process as the number of replication expansions.

[0099] Optionally, in a case where the current convolution process is the first convolution process, the first sparse feature is replicated and expanded according to the number of replication expansions to obtain at least one replicated and expanded sparse feature, including: obtaining n coordinate values in the first sparse feature, where n is a natural number starting from 1; obtaining a preset relationship table, and finding 2n target coordinate values according to the n coordinate values, where in the preset relationship table, one coordinate value corresponds to two target coordinate values; and placing the 2n target coordinate values into corresponding spaces to obtain the replicated and expanded sparse feature.

[0100] Optionally, the target point cloud is determined according to the plurality of classification labels, including: in a case where the classification label output by the full connection layer is a first label, determining the point cloud in the point cloud column as a point cloud corresponding to a target object; in a case where the classification label output by the full connection layer is a second label, determining the point cloud in the point cloud column as a point cloud corresponding to a background; in a case where the classification label output by the full connection layer is a third label, determining the point cloud in the point cloud column as a point cloud corresponding to a ground; and determining the point cloud corresponding to the target object and the point cloud corresponding to the ground as the target point cloud.

[0101] In the above embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0102] In several embodiments provided in the present application, it should be understood that the disclosed technical contents can be implemented by other ways. Among them, the device embodiments described above are only schematic, for example, the division of units can be a logical function division, and actual implementation can have another division way, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or modules shown or discussed can be indirect coupling or communication connection through some interfaces, units or modules, which can be electrical or other forms.

[0103] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on a plurality of units. According to actual needs, part or all of the units can be selected to achieve the purpose of the present embodiment scheme.

[0104] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0105] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in the embodiments of the present application. The foregoing storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, and various other media that can store program codes.

[0106] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A point cloud filtering method for removing background point clouds, characterized in that, The method comprises the following steps: obtaining a plurality of point cloud columns and an initial index, wherein the initial index is a coordinate matrix containing a plurality of point cloud column coordinates corresponding to a plurality of point cloud columns, and each of the point cloud columns contains a plurality of point clouds; determining a plurality of shallow feature maps, a plurality of point cloud column features corresponding to the plurality of shallow feature maps, and a first index according to the plurality of point cloud columns and the initial index, wherein the point cloud column corresponds to the shallow feature map one by one, and the coordinate value in the first index is half of the point cloud column coordinate value in the initial index; controlling the plurality of shallow feature maps and the first index to be input into a preset convolutional backbone network to obtain a plurality of splicing features corresponding to the plurality of point cloud columns, wherein the shallow feature map corresponds to the splicing feature one by one; controlling the plurality of splicing features to be input into a full connection layer to obtain a plurality of classification labels corresponding to the plurality of point cloud columns, wherein the full connection layer is contained in the preset convolutional backbone network, and the classification label is used to represent the object corresponding to the point cloud in the point cloud column; determining a target point cloud according to the plurality of classification labels, The preset convolutional backbone network comprises a plurality of 2D convolutional modules, and controlling the plurality of shallow feature maps and the first index to be input into the preset convolutional backbone network to obtain a plurality of splicing features corresponding to the plurality of point cloud columns comprises: Step 401: performing convolutional processing on a first feature map through the 2D convolutional module according to the first index to obtain a sparse feature map, wherein the length and width of the first feature map are half of the length and width of the shallow feature map; Step 402: performing convolutional processing on the first index through the 2D convolutional module to obtain a target index, wherein the coordinate value contained in the target index is half of the coordinate value contained in the first index; Step 403: generating a first sparse feature according to the sparse feature map; Step 404: determining a replication expansion number and replicating and expanding the first sparse feature according to the replication expansion number to obtain at least one replicated and expanded sparse feature; Step 405: splicing the first sparse feature and at least one of the replicated and expanded sparse features to obtain a target sparse feature; Step 406: after the sparse feature map and the target index obtained after each convolutional process are input into the next 2D convolutional module, the steps 401 to 405 are repeatedly executed to obtain a plurality of target sparse features corresponding to a plurality of convolutional processes, wherein one 2D convolutional module corresponds to one convolutional process; Step 407: splicing the point cloud column features corresponding to the plurality of target sparse features obtained after a plurality of convolutional processes and the shallow feature map to obtain the splicing feature; Step 408: whenever one shallow feature map and one target index are input into the preset convolutional backbone network, the steps 401 to 407 are repeatedly executed to obtain a plurality of splicing features.

2. The method of claim 1, wherein, Obtaining a plurality of point cloud columns and an initial index comprises: obtaining an original point cloud; segmenting the original point cloud into a plurality of point cloud columns of the same shape; obtaining a plurality of coordinates corresponding to the plurality of point cloud columns in a preset bird's eye view plane coordinate system; determining a coordinate matrix composed of the plurality of coordinates as the initial index.

3. The method of claim 1, wherein, According to the plurality of point cloud columns and the initial index, a plurality of shallow feature maps, a plurality of point cloud column features corresponding to the plurality of shallow feature maps, and a first index are determined, including: controlling the point cloud column to be input into a point cloud column feature extraction model, and performing sparse processing on the point cloud column through the initial index to obtain a plurality of point cloud column features corresponding to the point cloud column; splicing a plurality of point cloud column features to obtain the shallow feature map corresponding to the point cloud column, and obtaining a plurality of shallow feature maps corresponding to a plurality of point cloud columns; performing convolution processing on the initial index through the point cloud column feature extraction model to obtain the first index.

4. The method of claim 1, wherein, According to the first index, the 2D convolution module is used to perform convolution processing on the first feature map to obtain a sparse feature map, wherein the length and width of the first feature map are half of the length and width of the shallow feature map, including: According to the first index, the shallow feature map is screened to retain a plurality of target features with the same coordinates as the coordinates in the first index, and non-target features are assigned a value of 0; determining the feature map composed of a plurality of target features as the sparse feature map.

5. The method of claim 1, wherein, According to the sparse feature map, a first sparse feature is generated, including: extracting a plurality of sparse features contained in the sparse feature map; splicing a plurality of sparse features to obtain the first sparse feature.

6. The method of claim 1, wherein, determining the number of replication and expansion, including: determining the number corresponding to the current convolution process, and determining the number corresponding to the current convolution process as the number of replication and expansion.

7. The method of claim 1, wherein, In the case where the current convolution process is the first convolution process, the first sparse feature is replicated and expanded according to the number of replication and expansion to obtain at least one replicated and expanded sparse feature, including: obtaining n coordinate values in the first sparse feature, wherein n is a natural number starting from 1; obtaining a preset relationship table, and finding 2n target coordinate values according to n coordinate values, wherein in the preset relationship table, one coordinate value corresponds to two target coordinate values; putting 2n target coordinate values into the corresponding space to obtain the replicated and expanded sparse feature.

8. The method of claim 1, wherein, According to a plurality of classification identifiers, a target point cloud is determined: in the case where the classification identifier output by the full connection layer is a first identifier, the point cloud in the point cloud column is determined as a point cloud corresponding to a target object; in the case where the classification identifier output by the full connection layer is a second identifier, the point cloud in the point cloud column is determined as a point cloud corresponding to a background; in the case where the classification identifier output by the full connection layer is a third identifier, the point cloud in the point cloud column is determined as a point cloud corresponding to a ground; determining the point cloud corresponding to the target object and the point cloud corresponding to the ground as the target point cloud.

9. A point cloud filtering apparatus for removing background point clouds, characterized in that, including: The first acquisition unit is configured to acquire a plurality of point cloud columns and an initial index, wherein the initial index is a coordinate matrix containing a plurality of point cloud column coordinates corresponding to the plurality of point cloud columns, and each of the point cloud columns contains a plurality of point clouds; The first determination unit is configured to determine, according to the plurality of point cloud columns and the initial index, a plurality of shallow feature maps, a plurality of point cloud column features corresponding to the plurality of shallow feature maps, and a first index, wherein the point cloud columns and the shallow feature maps correspond to each other one by one, and a coordinate value in the first index is one half of a point cloud column coordinate value in the initial index; The first control unit is configured to control the plurality of shallow feature maps and the first index to be input into a preset convolutional backbone network, so as to obtain a plurality of splicing features corresponding to the plurality of point cloud columns, wherein the shallow feature maps and the splicing features correspond to each other one by one; The second control unit is configured to control the plurality of splicing features to be input into a full connection layer, so as to obtain a plurality of classification labels corresponding to the plurality of point cloud columns, wherein the full connection layer is contained in the preset convolutional backbone network, and the classification labels are used to represent objects corresponding to the point clouds in the point cloud columns; The second determination unit is configured to determine, according to the plurality of classification labels, a target point cloud, The preset convolutional backbone network comprises a plurality of 2D convolution modules, and the first control unit comprises: The second convolution processing subunit is configured to perform convolution processing on a first feature map by the 2D convolution module according to the first index, so as to obtain a sparse feature map, wherein a length and a width of the first feature map are one half of a length and a width of the shallow feature map; The third convolution processing subunit is configured to perform convolution processing on the first index by the 2D convolution module, so as to obtain a target index, wherein a coordinate value contained in the target index is one half of a coordinate value contained in the first index; The generation subunit is configured to generate a first sparse feature according to the sparse feature map; The copying and expanding subunit is configured to determine a copying and expanding number of times, and copy and expand the first sparse feature according to the copying and expanding number of times, so as to obtain at least one copied and expanded sparse feature; The second splicing subunit is configured to splice the first sparse feature and at least one of the copied and expanded sparse features, so as to obtain a target sparse feature; The first input subunit is configured to repeatedly execute steps 401 to 405 after the sparse feature map and the target index obtained after each convolution process are input into a next 2D convolution module, so as to obtain a plurality of target sparse features corresponding to a plurality of convolution processes, wherein one 2D convolution module corresponds to one convolution process; The third splicing subunit is configured to splice a plurality of target sparse features obtained after a plurality of convolution processes and point cloud column features corresponding to the shallow feature maps, so as to obtain the splicing features; The second input subunit is configured to repeatedly execute the steps whenever one shallow feature map and the target index are input into the preset convolutional backbone network, so as to obtain a plurality of splicing features.

10. A vehicle characterized by comprising: A laser radar and a point cloud filtering device for removing background point clouds for performing a point cloud filtering method for removing background point clouds according to any one of claims 1 to 8.

11. An electronic device, comprising: comprising: one or more processors, memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including a point cloud filtering method for removing background point clouds according to any one of claims 1 to 8.

Images