Three-dimensional (3D) object detection method and device, controller, vehicle and medium

Abstract

The embodiment of the invention relates to a method and device for three-dimensional (3D) object detection, a controller, a vehicle and a medium. The method comprises the following steps: acquiring a detection point cloud for a target 3D scene, wherein the detection point cloud comprises a plurality of detection points for a plurality of object points in the target 3D scene; the method further includes generating at least one shadow point based on an offset operation on the plurality of detection points. The method further includes detecting an object in the target 3D scene based on the plurality of detection points and the at least one shadow point. According to the method disclosed by the embodiment of the invention, the shadow point can be generated through the offset operation based on the original detection point to prevent or reduce information loss in 3D object detection, so that the accuracy of 3D object detection is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of data processing, and more particularly, to methods, apparatuses, controllers, vehicles, and media for three-dimensional (3D) object detection. Background Art

[0002] With the development of intelligent devices, the demand for object detection in a scene is increasing, and in more and more applications, it is necessary to detect objects in a three-dimensional (3D) scene (e.g., roads, office areas, or playgrounds, etc.). For example, during the autonomous driving or assisted driving of a vehicle, it is necessary to detect objects (e.g., people or other vehicles) in a 3D driving scene to determine the driving operations to be performed. For such applications, the accuracy of object detection may directly affect the safety of users. Therefore, the accuracy of object detection is very important. Summary of the Invention

[0003] Embodiments of the present disclosure provide a method, an apparatus, a controller, a vehicle, and a medium for three-dimensional (3D) object detection.

[0004] According to a first aspect of the present disclosure, a method for three-dimensional (3D) object detection is provided. The method includes obtaining a detection point cloud for a target 3D scene, where the detection point cloud includes a plurality of detection points for a plurality of object points in the target 3D scene. The method further includes generating at least one shadow point based on an offset operation on the plurality of detection points. The method further includes detecting an object in the target 3D scene based on the plurality of detection points and the at least one shadow point.

[0005] According to a second aspect of the present disclosure, an apparatus for 3D object detection is provided. The apparatus includes an obtaining unit configured to obtain a detection point cloud for a target 3D scene, where the detection point cloud includes a plurality of detection points for a plurality of object points in the target 3D scene. The apparatus further includes a generating unit configured to generate at least one shadow point based on an offset operation on the plurality of detection points. The apparatus further includes a detecting unit configured to detect an object in the target 3D scene based on the plurality of detection points and the at least one shadow point.

[0006] According to a third aspect of the present disclosure, a controller is provided. The controller includes at least one processor; and a memory coupled to the at least one processor and having instructions stored thereon, the instructions, when executed by the at least one processor, cause the controller to implement the method according to the first aspect of the present disclosure.

[0007] According to a fourth aspect of the present disclosure, a vehicle is provided. The vehicle includes the controller according to the third aspect of the present disclosure.

[0008] In a fifth aspect of the present disclosure, a computer-readable storage medium is provided. Computer-executable instructions are stored on the computer-readable storage medium, and the computer-executable instructions are executed by a processor to implement the method according to the first aspect of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] By describing the exemplary embodiments of the present disclosure in more detail in conjunction with the accompanying drawings, the above and other objects, features, and advantages of the present disclosure will become more apparent. In the exemplary embodiments of the present disclosure, the same reference numerals generally represent the same components.

[0010] Figure 1 FIG. is a schematic diagram of an exemplary environment in which the device and / or method according to the embodiments of the present disclosure may be implemented.

[0011] Figure 2 FIG. is a flowchart of a method for three-dimensional (3D) object detection according to an embodiment of the present disclosure.

[0012] Figure 3 FIG. is a flowchart of a process for generating shadow points according to an embodiment of the present disclosure.

[0013] Figure 4 FIG. is a schematic diagram of an example of generating shadow points according to an embodiment of the present disclosure.

[0014] Figure 5 FIG. is a schematic diagram of another example of generating shadow points according to an embodiment of the present disclosure.

[0015] Figure 6 FIG. is a schematic diagram of an example of projecting a two-dimensional (2D) pseudo-image according to an embodiment of the present disclosure.

[0016] Figure 7 FIG. is a schematic diagram of a neural network model according to an embodiment of the present disclosure.

[0017] Figure 8 FIG. is a schematic block diagram of a device for 3D object detection according to an embodiment of the present disclosure.

[0018] Figure 9 FIG. is a schematic block diagram of an example of an exemplary device suitable for implementing embodiments of the present disclosure.

[0019] In each of the drawings, the same or corresponding reference numerals indicate the same or corresponding parts. DETAILED DESCRIPTION

[0020] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0021] In the description of the embodiments of the present disclosure, the term "including" and its like should be understood as an open inclusion, that is, "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "an embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. There may also be other explicit and implicit definitions hereinafter.

[0022] With the development of intelligent devices, the demand for object detection of objects in a three-dimensional (3D) scene is increasing. To facilitate 3D object detection, the detection point cloud for the 3D scene is usually projected into a two-dimensional (2D) pseudo-image from a predetermined perspective, so that the detection method of 2D images can be used to perform 3D object detection. For example, common projection methods may include a bird's-eye view (BEV, also known as "top view") projection method, a range view (RV) projection method, and a multi-view fusion projection method.

[0023] During the projection process, the detection space corresponding to the 3D scene (where multiple detection points of the detection point cloud are distributed) is usually divided into multiple voxels, and then 2D projection is performed in units of voxels. Therefore, corresponding spatial information will be lost during the projection process. For example, information in a corresponding dimension will be lost. For example, for the top view (BEV) projection method, in the same voxel, only the information of the detection point with a higher height in the vertical direction can be effectively mapped during the projection process, while the information of the detection point with a lower height may be lost. For example, a person in a 3D scene may only be projected as the outline of the head and shoulders, and the side information will be lost. In other words, the BEV projection method will lose height information, and there is a competitive relationship among detection points in terms of height. Similarly, the RV projection method will lose depth information. These information losses may lead to inaccurate detection results of 3D object detection. To improve the accuracy of the detection results, a multi-view fusion projection method can be used, which is a combination of the above two. However, the multi-view fusion projection method requires a large amount of computing resources, resulting in the inability of ordinary electronic devices to support this projection method and a too high cost.

[0024] To address at least the above and other potential issues, embodiments of the present disclosure provide a method for 3D object detection. The method includes obtaining a detection point cloud for a target 3D scene, where the detection point cloud includes a plurality of detection points for a plurality of object points in the target 3D scene. The method further includes generating at least one shadow point based on an offset operation on the plurality of detection points. The method also includes detecting an object in the target 3D scene based on the plurality of detection points and the at least one shadow point. According to the method of embodiments of the present disclosure, information loss in 3D object detection can be prevented or reduced by generating shadow points based on an offset operation of the original detection points, thereby improving the accuracy of 3D object detection.

[0025] Embodiments of the present disclosure will be described in further detail below with reference to the accompanying drawings. Figure 1 FIG. shows a schematic diagram of an example environment 100 in which an apparatus and / or method according to an embodiment of the present disclosure may be implemented. As Figure 1 shown, a 3D detection space 110 corresponding to a target 3D scene is shown in the example environment 100. The 3D detection space 110 may be a virtual space for representing the target 3D scene. The target 3D scene may be any scene, such as a road, an office area, or a playground, etc. In some embodiments, the length L, width W, and height H of the 3D detection space may be set according to the length, width, and height of the target 3D scene. For example, the length L, width W, and height H of the 3D detection space may be set to be the same as the length, width, and height of the target 3D scene, respectively.

[0026] The detection point cloud 120 for the target 3D scene may be distributed in the 3D detection space 110. In some embodiments, the detection point cloud 120 may be a lidar point cloud obtained by a lidar. For example, a lidar point cloud obtained by a lidar on a vehicle. The detection point cloud 120 may include a plurality of detection points for a plurality of object points in the target 3D scene. For example, the plurality of object points may be points on a person or a vehicle in a road scene (e.g., a large number of points on each pedestrian or a large number of points on each vehicle). Figure 1 The detection points in the detection point cloud 120 are represented by solid dots in, and Figure 1 one of the detection points 120-i (i is an integer greater than or equal to 1) is labeled in. Hereinafter, the detection point 120-i will be used as an example for description, but it should be understood that the description for the detection point 120-i may be applicable to any other detection point. It should also be understood that the detection point cloud 120 may include more or fewer detection points than Figure 1 shown, and may have a different distribution from Figure 1 shown.

[0027] After obtaining as Figure 1After detecting the detection point cloud 120 shown, according to an embodiment of the present disclosure, the detection point cloud 120 can be projected into a two-dimensional (2D) pseudo-image in a predetermined projection manner without losing the information of the corresponding dimension (for example, the height information for the top view projection manner) or reducing the information loss of the corresponding dimension. This can make object detection using the 2D pseudo-image more accurate. In the following examples, the predetermined projection manner will be the top view (i.e., BEV) projection manner as an example for further description.

[0028] Figure 2 FIG. illustrates a flowchart of a method 200 for 3D object detection according to an embodiment of the present disclosure. The method 200 can be executed by any controller or electronic device, or can be executed by a trained neural network model, or can be executed in combination by a controller or electronic device and a trained neural network model. As Figure 2 shown, at block 202, a detection point cloud for a target 3D scene is obtained (for example, Figure 1 the detection point cloud 120 shown), and the detection point cloud 120 includes a plurality of detection points for a plurality of object points in the target 3D scene. The target 3D scene can be any scene, for example, a road, an office area, or a playground, etc. The object points in the target 3D scene can be any object points, for example, points on a person, a vehicle, or any other object in the scene.

[0029] In some embodiments, the detection point cloud 120 can be a lidar point cloud obtained by a lidar (for example, on a vehicle). In some embodiments, the plurality of detection points in the detection point cloud 120 correspond one-to-one to the plurality of object points. That is, for each object point, there is a detection point. In some embodiments, the plurality of detection points are distributed in a 3D detection space representing the target 3D scene (for example, Figure 1 the 3D detection space 110 shown). In some embodiments, each detection point in the plurality of detection points (for example, Figure 1 the detection point 120-i shown) corresponds in position in the 3D detection space 110 to the corresponding object point in the target 3D scene.

[0030] At block 204, at least one shadow point is generated based on an offset operation on the plurality of detection points. In some embodiments, at least one shadow point can be generated by performing an offset operation on the plurality of detection points in the horizontal direction. This can copy the information of the original detection point 120-i to another position in the 3D detection space, so that the information of its specific dimension (for example, height) can be retained according to the offset degree of the detection point (which will be further described in the following examples).

[0031] At block 206, an object in a target 3D scene is detected based on a plurality of detection points and at least one shadow point. For example, a 2D pseudo-image can be generated by projecting based on the original plurality of detection points and the at least one generated shadow point to detect an object in the target 3D scene. This can enable obtaining a 2D pseudo-image with more sufficient information of the object under test through the specific dimensional information indicated by the shadow points, thereby making the 3D object detection using the 2D pseudo-image more accurate. According to method 200 of an embodiment of the present disclosure, shadow points can be generated through an offset operation based on the original detection points to prevent or reduce information loss in 3D object detection, thereby improving the accuracy of 3D object detection.

[0032] Figure 3 FIG. shows a flowchart of a process 300 for generating shadow points according to an embodiment of the present disclosure. Figure 3 The process can correspond to Figure 2 block 204 in Figure 2 In some embodiments, each detection point 120-i in the detection point cloud obtained at block 202 of i , y i , z i ) can have detection point information, and the detection point information can include: a detection point position, and a detection point feature. In some embodiments, the detection point position can include: a horizontal position, and a height. For example, the detection point position of each detection point 120-i can be represented by position coordinates (x

[0033] After obtaining a plurality of detection points with the above detection point information, in some embodiments, at Figure 3At the frame 302, at least one detection point among multiple detection points can be selected as the target detection point. For example, the detection point 120-i can be selected as one of the target detection points. At the frame 304, for each target detection point among at least one target detection point (e.g., the target detection point 120-i), by offsetting the horizontal position of the target detection point based on the height of the target detection point, a corresponding shadow point that is horizontally offset relative to the target detection point 120-i can be generated. In some embodiments, the corresponding shadow point has shadow point information, and the shadow point information can include: the shadow point position, and the detection point features of the target detection point 120-i. In some embodiments, the shadow point position can include: the offset horizontal position obtained through the offset, and the height of the target detection point 120-i. In other words, the difference between the shadow point information of the generated corresponding shadow point and the detection point information of the target detection point 120-i can only be: the horizontal position is different. In addition, it should be understood that according to actual needs, other different information can also be added to the shadow point information and the detection point information, such as flag information for distinguishing the corresponding shadow point and the target detection point, etc., which will be further described in the examples below.

[0034] Figure 4 FIG. 400 is a schematic diagram illustrating an example of generating a shadow point according to an embodiment of the present disclosure. In Figure 4 it, solid dots represent the original detection points (which include the target detection points), hollow circles represent the shadow points, and dashed arrows represent the offsets in the horizontal direction. Figure 4 Three detection points 120-i, 120-j, and 120-k selected as the target detection points are shown, and three corresponding shadow points 120-i', 120-j', and 120-k' generated by horizontally offsetting these three detection points 120-i, 120-j, and 120-k are shown. It can be Figure 4 seen that relative to the original detection points (target detection points) 120-i, 120-j, and 120-k, the corresponding shadow points 120-i', 120-j', and 120-k' only have an offset in the horizontal position, while the height remains unchanged. In this embodiment, i, j, and k can be distinct integers greater than or equal to 1.

[0035] In some embodiments, according to the following equations (1), (2), and (3), the corresponding shadow point 120-i' of the target detection point 120-i with position coordinates (x i , y i , z i ) can be generated (it should be understood that this also applies to other target detection points).

[0036] x i’ = x i + σ × zi (1)

[0037] y i’ = y i + σ × z i (2)

[0038] z i’ = z i (3)

[0039] In the above equations (1)-(3), σ is an offset constant, which can be set according to actual needs, or can be determined by a trained neural network model. x i’ , y i’ and z i’ are the position coordinates (x i’ , y i’ , z i’ ) corresponding to the shadow point 120-i'. It can be seen from the above equations (1)-(3) that the corresponding shadow point 120-i' has only an offset in the horizontal direction (i.e., in the (x, y) plane) relative to the target detection point 120-i, while the height remains unchanged. In addition, the degree of offset of the corresponding shadow point 120-i' in the horizontal direction is related to the height z i of the target detection point 120-i. Therefore, the height of the target detection point can be determined by the degree of offset. Thus, when projecting the detection point cloud into a 2D pseudo-image, the height information of the detection points can be retained, so that 3D object detection can be more accurate.

[0040] It should be understood that although Figure 4 only shows 3 detection points as target detection points, any number of detection points can be selected as target detection points according to actual needs. For example, in some embodiments, in order to most comprehensively retain the detection point information, all detection points in the detection point cloud 120 can be selected as target detection points. In other embodiments, for example, in order to consider the balance between computing resources and expected accuracy, at least one detection point in the detection point cloud 120 that meets a predetermined condition can be selected as a target detection point.

[0041] In some embodiments, the predetermined condition may include that the detection point position meets a predetermined position condition. For example, for each detection point (e.g., detection point 120-i), its position coordinates (x i , y i , z i ) can be determined, and x i , y i and z iWhether at least one of them is greater than the corresponding threshold value. In the case of being greater than the corresponding threshold value, the detection point 120-i can be determined as the target detection point. In some other embodiments, the predetermined condition may include that the detection point corresponds to a point in a predetermined area of a two-dimensional (2D) image of the target 3D scene. For example, while obtaining the above-mentioned detection point cloud 120 by the lidar of the vehicle, the 2D image of the same target 3D scene can also be acquired by the camera of the vehicle. Then, the predetermined area of interest can be determined by performing any type of feature extraction on the 2D image. For example, the foreground, background, or any boundary, or any other area can be extracted. Then, the detection points corresponding to the points in the predetermined area of interest can be used as the above-mentioned target detection points. In still some other embodiments, the predetermined condition may include that the type of the detection point belongs to a predetermined type. Here, the type of the detection point can be determined by any means. The following will refer to Figure 5 to describe an example of selecting a target detection point by the type of the detection point and then generating the corresponding shadow point.

[0042] Figure 5 FIG. 500 is a schematic diagram illustrating another example of generating a shadow point according to an embodiment of the present disclosure. In some embodiments, in order to perform the projection of the detection point cloud, the method 200 according to an embodiment of the present disclosure may further include: dividing the 3D detection space 110 into at least one voxel, and the at least one voxel may have the same shape and volume (for the top view projection method). As Figure 5 shown, the detection space 110 is divided into a plurality of voxel-shaped voxels, Figure 5 and two adjacent voxels Vi and Vi+1 are marked therein. In some embodiments, each detection point may be located in the corresponding voxel among the at least one divided voxels. It should be understood that there may be voxels without detection points.

[0043] After dividing the at least one voxel, in some embodiments, the method 200 according to an embodiment of the present disclosure may further include: determining the voxel information of each voxel in the at least one voxel based on at least one of the following: the position of the voxel in the 3D detection space and the detection point information of the detection points in the voxel. In some embodiments, the voxel information may be determined by a trained neural network model and may indicate the overall characteristics of all the detection points in the voxel.

[0044] After obtaining the voxel information of each voxel as described above, in some embodiments, method 200 according to the present disclosure may further include determining the type of each detection point 120-i as follows: by performing semantic segmentation on the corresponding voxel based on the detection point information of the detection point 120-i and the voxel information of the corresponding voxel where the detection point 120-i is located, to determine the type of the detection point 120-i. For example, in the case where the detection point 120-i is determined to be a detection point of a predetermined type, the detection point may be selected as a target detection point, thereby generating a corresponding shadow point.

[0045] For example, taking the Figure 5 voxel Vi shown in as an example for illustration. There are 5 detection points 501, 502, 503, 504, and 505 in the voxel Vi. Two of these 5 detection points, 501 and 505, are determined to be detection points belonging to a predetermined type (the two may be detection points belonging to different predetermined types), and thus are selected as target detection points. For these two target detection points 501 and 505, corresponding shadow points 501' and 505' are respectively generated. Here, in order to enable the shadow points to more effectively reflect information in a specific dimension (for example, height), in some embodiments, the target detection points 501 and 505 and the corresponding shadow points 501' and 505' may be located in different voxels Vi, Vi+1, and the shadow point information may further include the voxel information of the voxel Vi where the target detection points 501 and 505 are located. It should be understood that although Figure 5 shows that the corresponding shadow points 501' and 505' of the target detection points 501 and 505 in the voxel Vi are located in the same voxel Vi+1, it should be understood that the corresponding shadow points 501' and 505' of the target detection points 501 and 505 in the voxel Vi may be located in different voxels (for example, adjacent or non-adjacent voxels).

[0046] After generating at least one shadow point as described above, 3D object detection may be performed based on the original multiple detection points and the generated at least one shadow point, that is, execute Figure 2 block 206. In some embodiments, block 206 may include: based on a top view projection method, by, for example, Figure 5 at least one voxel divided in, projecting the multiple detection points and the at least one shadow point into a 2D pseudo-image. In some embodiments, this projection may map the voxel information of each voxel in the at least one voxel and the shadow point information of the shadow points located in the voxel into the pixel information of the corresponding pixels in the 2D pseudo-image.

[0047] Figure 6 FIG. 600 is a schematic diagram illustrating an example of projecting a 2D pseudo-image according to an embodiment of the present disclosure. As Figure 6As shown, the 3D detection space 110 is divided into a plurality of voxels, and each voxel is a cube with the same shape and volume. There may be no detection points and shadow points in some voxels. There may be detection points but no shadow points in some other voxels. There may be only shadow points of detection points from other voxels in some yet other voxels. There may be detection points and shadow points in some other voxels. Regardless of which voxel, the voxel information of the voxel and the shadow point information of the shadow points (if any) can be mapped to the pixel information of the corresponding pixel Pi in the 2D pseudo-image 610. It should be understood that Figure 6 the pixel Pi of the pseudo-image 610 shown in

[0048] is only for illustrative purposes and does not indicate the graphics in the actually mapped image. After obtaining the 2D pseudo-image 610 as above, in some embodiments, the frame 206 may further include: detecting an object in the target 3D scene based on the 2D pseudo-image 610. In some embodiments, the method 200 described above with reference to Figure 2-6 may be performed in whole or in part by a trained neural network model. Figure 7 FIG. 700 shows a schematic diagram of a neural network model according to an embodiment of the present disclosure. In some embodiments, the neural network model 700 may be obtained based on any 3D object detection neural network model for performing 3D object detection. For example, a multi-layer perception and max pooling neural network model, such as a pillar-based 3D object detection neural network model.

[0049] In the case where the method 200 according to the present disclosure is entirely performed by the neural network model 700, the trained neural network model 700 can be obtained by adding an algorithm for generating shadow points to a known 3D object detection neural network model and training it. During the training process of the neural network model 700, its input may be the detection points in the obtained detection point cloud, and the output may be the detected object. Thus, the training data of the original 3D object detection neural network model can be used to train the neural network model 700 without creating new training data. This makes it possible to simply and easily implement the method according to the present disclosure by utilizing the known 3D object detection neural network model and its training data.

[0050] In addition, in the case where part of the method 200 according to the present disclosure is executed by the neural network model 700, it is only necessary to train a relatively simple predetermined neural network model for this part of the operation to obtain the trained neural network model 700. In addition, in some embodiments, during the execution of the above method 200 by the trained neural network model 700, flag information (for example, a flag bit mask) can also be added to the original detection points and the corresponding shadow points to distinguish between the two. For example, for the original detection points, the flag bit mask can be equal to one of "0" and "1", while for the corresponding shadow points, the flag bit mask can be equal to the other of "0" and "1". It should be understood that this is only an example, and any other form of flag information can be used according to actual needs. In addition, in some embodiments, during the object detection process, the trained neural network model 700 can also select whether to use the shadow point information of some or all of the shadow points. For example, this can depend on the required object detection accuracy and the expected detection efficiency, etc.

[0051] Figure 8 FIG. schematically illustrates a block diagram of an apparatus 800 for 3D object detection according to an embodiment of the present disclosure. As Figure 8 shown, the apparatus 800 includes an acquisition unit 802 configured to acquire a detection point cloud for a target 3D scene, the detection point cloud including a plurality of detection points for a plurality of object points in the target 3D scene. The apparatus 800 further includes a generation unit 804 configured to generate at least one shadow point based on an offset operation of the plurality of detection points. The apparatus 800 further includes a detection unit 806 configured to detect an object in the target 3D scene based on the plurality of detection points and the at least one shadow point. In some embodiments, the apparatus 800 can be a trained neural network model.

[0052] In some embodiments, the detection point cloud can be a lidar point cloud obtained by a lidar. In some embodiments, the plurality of detection points can correspond one-to-one with the plurality of object points. In some embodiments, the plurality of detection points can be distributed in a 3D detection space representing the target 3D scene. In some embodiments, the position of each detection point in the 3D detection space among the plurality of detection points can correspond to the position of the corresponding object point in the target 3D scene among the plurality of object points. In some embodiments, the detection points can have detection point information, and the detection point information can include: the detection point position, and the detection point feature. In some embodiments, the detection point position can include: the horizontal position, and the height. In some embodiments, the detection point feature can indicate the feature of the corresponding object point.

[0053] In some embodiments, the generating unit 804 may be configured to select at least one detection point among a plurality of detection points as a target detection point. In some embodiments, the generating unit 804 may be configured to select all of the detection points among the plurality of detection points as target detection points. In some embodiments, the generating unit 804 may be configured to select at least one detection point that meets a predetermined condition among the plurality of detection points as a target detection point. In some embodiments, the predetermined condition may include that the position of the detection point meets a predetermined position condition. In some embodiments, the predetermined condition may include that the type of the detection point belongs to a predetermined type. In some embodiments, the predetermined condition may include that the detection point corresponds to a point in a predetermined area of a two-dimensional (2D) image of the target 3D scene.

[0054] In some embodiments, the generating unit 804 may further be configured to, for each target detection point among at least one target detection point, generate a corresponding shadow point that is horizontally offset relative to the target detection point by offsetting the horizontal position of the target detection point based on the height of the target detection point. In some embodiments, the corresponding shadow point may have shadow point information, and the shadow point information may include: the shadow point position, and the detection point feature of the target detection point. In some embodiments, the shadow point position may include: the offset horizontal position obtained through the offset, and the height of the target detection point.

[0055] In some embodiments, the apparatus 800 may further include a voxel unit, and the voxel unit may be configured to divide the 3D detection space into at least one voxel, and the at least one voxel has the same shape and volume. In some embodiments, the voxel unit may further be configured to determine the voxel information of each voxel among the at least one voxel based on at least one of the following: the position of the voxel in the 3D detection space, and the detection point information of all detection points within the voxel. In some embodiments, the apparatus 800 may further include a type determination unit, configured to determine the type of each detection point as follows: by performing semantic segmentation on the corresponding voxel based on the detection point information of the detection point and the voxel information of the corresponding voxel where the detection point is located, to determine the type of the detection point. In some embodiments, each detection point may be located in a corresponding voxel among the at least one voxel. In some embodiments, the target detection point and the corresponding shadow point may be located in different voxels. In some embodiments, the shadow point information may further include the voxel information of the voxel where the target detection point is located.

[0056] In some embodiments, the detection unit 806 may be configured to project a plurality of detection points and at least one shadow point into a 2D pseudo-image based on a top view projection manner through at least one voxel. In some embodiments, the projection may cause the voxel information of each voxel in at least one voxel and the shadow point information of the shadow point located in the voxel to be mapped to the pixel information of the corresponding pixel in the 2D pseudo-image. In some embodiments, the detection unit 806 may also be configured to detect an object in the target 3D scene based on the 2D pseudo-image.

[0057] The apparatus 800 according to an embodiment of the present disclosure can prevent or reduce information loss in 3D object detection by generating shadow points through an offset operation based on original detection points, thereby improving the accuracy of 3D object detection.

[0058] Figure 9 A schematic block diagram of an exemplary device 900 suitable for implementing the embodiments of the present disclosure is shown. The controller described above can be implemented using the device 900. As shown, the device 900 includes a processor 901, which can execute various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 902 and loaded into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The processor 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0059] The various processes and treatments described above, such as the method 200 and the process 300, can be executed by the processor 901. For example, in some embodiments, the 200 and the process 300 can be implemented as a computer software program, which is tangibly included in a machine-readable medium. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 900 via the ROM 902. When the computer program is loaded into the RAM 903 and executed by the processor 901, one or more actions of the method 200 and the process 300 described above can be executed.

[0060] The present disclosure may be a method, an apparatus, a system, and / or a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for performing various aspects of the present disclosure.

[0061] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example—but not limited to—an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punched card or raised structures in grooves having instructions stored thereon, and any suitable combination of the foregoing. The computer-readable storage medium as used herein is not construed to be a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0062] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to respective computing / processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include a copper transmission cable, an optical fiber transmission, a wireless transmission, a router, a firewall, a switch, a gateway computer, and / or an edge server. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0063] The computer program instructions for performing the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine - related instructions, microcode, firmware instructions, state - setting data, or source code or object code written in any combination of one or more programming languages, including object - oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer - readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand - alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or alternatively, may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, by using the state information of the computer - readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field - programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer - readable program instructions to implement various aspects of this disclosure.

Claims

1. A method for three-dimensional (3D) object detection, comprising: obtaining a detection point cloud for a target 3D scene, the detection point cloud including a plurality of detection points for a plurality of object points in the target 3D scene; generating at least one shadow point based on an offset operation on the plurality of detection points; and detecting an object in the target 3D scene based on the plurality of detection points and the at least one shadow point.

2. The method according to claim 1, wherein the plurality of detection points correspond one-to-one with the plurality of object points, wherein the plurality of detection points are distributed in a 3D detection space representing the target 3D scene, and wherein the position of each detection point in the 3D detection space corresponds to the position of the corresponding object point in the plurality of object points in the target 3D scene.

3. The method according to claim 2, wherein the detection point has detection point information, the detection point information comprising: a detection point position and a detection point feature, wherein the detection point position includes a horizontal position and a height, and wherein the detection point feature indicates the feature of the corresponding object point.

4. The method according to claim 3, wherein generating at least one shadow point based on the plurality of detection points comprising: selecting at least one detection point from the plurality of detection points as a target detection point; and for each target detection point in the at least one target detection point, generating a corresponding shadow point that is horizontally offset relative to the target detection point by offsetting the horizontal position of the target detection point based on the height of the target detection point, wherein the corresponding shadow point has shadow point information, the shadow point information including a shadow point position and the detection point feature of the target detection point, and wherein the shadow point position includes an offset horizontal position obtained through the offset and the height of the target detection point.

5. The method according to claim 4, wherein selecting at least one detection point from the plurality of detection points as a target detection point comprising: selecting all detection points from the plurality of detection points as target detection points, or selecting at least one detection point that meets a predetermined condition from the plurality of detection points as the target detection point.

6. The method according to claim 5, wherein the predetermined condition includes at least one of the following: the detection point position meets a predetermined position condition, the type of the detection point belongs to a predetermined type, and the detection point corresponds to a point in a predetermined area of a two-dimensional (2D) image of the target 3D scene.

7. The method according to claim 6, further comprising: dividing the 3D detection space into at least one voxel, the at least one voxel having the same shape and volume; and determining voxel information of each voxel in the at least one voxel based on at least one of the following: the position of the voxel in the 3D detection space and the detection point information of all detection points within the voxel.

8. The method according to claim 7, wherein each detection point is located in a corresponding voxel in the at least one voxel, and Wherein the target detection point and the corresponding shadow point are located in different voxels, and the shadow point information further includes the voxel information of the voxel in which the target detection point is located.

9. The method according to claim 8, wherein an object in the target 3D scene is detected based on the plurality of detection points and the at least one shadow point comprising: Based on a top view projection method, the plurality of detection points and the at least one shadow point are projected into a 2D pseudo-image through the at least one voxel; and Based on the 2D pseudo-image, an object in the target 3D scene is detected, wherein the projection maps the voxel information of each voxel in the at least one voxel and the shadow point information of the shadow point located in the voxel to the pixel information of the corresponding pixel in the 2D pseudo-image.

10. The method according to claim 7, further comprising determining the type of each detection point as follows: By performing semantic segmentation on the corresponding voxel based on the detection point information of the detection point and the voxel information of the corresponding voxel in which the detection point is located, the type of the detection point is determined.

11. The method according to any one of claims 1 to 10, wherein the detection point cloud is a lidar point cloud obtained by a lidar, and wherein the method is performed by a trained neural network model.

12. An apparatus for three-dimensional (3D) object detection, comprising: An acquisition unit configured to acquire a detection point cloud for a target 3D scene, the detection point cloud including a plurality of detection points for a plurality of object points in the target 3D scene; A generation unit configured to generate at least one shadow point based on an offset operation on the plurality of detection points; and A detection unit configured to detect an object in the target 3D scene based on the plurality of detection points and the at least one shadow point.

13. A controller, comprising: At least one processor; and A memory coupled to the at least one processor and having instructions stored thereon, the instructions causing the controller to perform the method according to any one of claims 1-11 when executed by the at least one processor.

14. A vehicle comprising the controller according to claim 13.

15. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions are executed by a processor to implement the method according to any one of claims 1 to 11.

Images