METHOD, DEVICE, CONTROL UNIT, VEHICLE AND MEDIUM FOR THREE-DIMENSIONAL (3D) OBJECT DETECTION

Abstract

The embodiments of the present disclosure relate to a method, an apparatus, a controller, a vehicle, and a medium for three-dimensional (3D) object detection. The method includes retrieving a detection point cloud for a target 3D scene, the detection point cloud including a plurality of detection points for a plurality of object points in the target 3D scene. The method further includes generating at least one shadow point based on an offset operation of the plurality of detection points. The method further includes detecting objects in the target 3D scene based on the plurality of detection points and at least one shadow point.According to the method of the embodiments of the present disclosure, it is possible to generate shadow points by an offset operation based on the original detection points in order to prevent or reduce information loss in 3D object detection and thus improve the accuracy of 3D object detection.

Description

FIELD OF THE INVENTIONThe embodiments of the present disclosure relate to the field of data processing and, more specifically, to a method, apparatus, controller, vehicle, and medium for three-dimensional (3D) object detection.PRIOR ARTAs smart devices develop, the need for object detection in scenes increases, and in more and more applications, objects are required to be detected in three-dimensional (3D) scenes (such as roads, office areas, or recreational parks). For example, during autonomous driving or assisted driving of vehicles, objects (such as persons or other vehicles) must be detected in 3D driving scenes in order to determine the driving maneuvers to be carried out. In such applications, the accuracy of object detection may directly affect the security of users, and thus the accuracy of object detection is of critical importance.DISCLOSURE OF THE INVENTIONThe embodiments of the present disclosure provide a method, apparatus, controller, vehicle, and medium for three-dimensional (3D) object detection.According to a first aspect of the present disclosure, a method for three-dimensional 3D object detection is provided. The method includes retrieving a detection point cloud for a target 3D scene, wherein the detection point cloud includes multiple detection points for multiple object points in the target 3D scene. The method further includes generating at least one shadow point based on an offset operation of the plurality of detection points. The method further includes detecting objects in the target 3D scene based on the plurality of detection points and at least one shadow point.According to a second aspect of the present disclosure, a 3D object detection apparatus is provided. The apparatus comprises a retrieving unit configured to retrieve a detection point cloud for a target 3D scene, wherein the detection point cloud comprises a plurality of detection points for a plurality of object points in the target 3D scene. The apparatus further includes a generation unit configured to generate at least one shadow point based on an offset operation of the plurality of detection points. The apparatus further comprises a detection unit configured to detect objects in the target 3D scene based on the plurality of detection points and the at least one shadow point.According to a third aspect of the present disclosure, a controller is provided. The controller includes at least one processor and a memory coupled to the at least one processor and including instructions stored thereon, the instructions, when executed by the at least one processor, cause the controller to implement the method according to the first aspect of the present disclosure.According to a fourth aspect of the present disclosure, a vehicle is provided. This vehicle includes a controller according to the third aspect of the present disclosure.In a fifth aspect of the present disclosure, a computer readable storage medium is provided. This computer readable storage medium has stored thereon computer executable instructions, the computer executable instructions being executed by a processor to implement the method according to the first aspect of the present disclosure.DESCRIPTION OF THE FIGURESBy describing the exemplary embodiments of the present disclosure in more detail in conjunction with the figures, the above and other objects, features, and advantages of the present disclosure will become more apparent, and in the exemplary embodiments, the same reference numerals typically represent the same components. FIG. 1 shows a schematic diagram of an example environment in which the apparatus and / or method according to the embodiments of the present disclosure may be implemented. FIG. 2 shows a flow diagram of a method for three-dimensional (3D) object detection according to the exemplary embodiments of the present disclosure. FIG. 3 is a flowchart of a process for generating shadow points according to embodiments of the present disclosure. FIG. 4 is a schematic diagram of an example of generating shadow points according to the embodiments of the present disclosure. FIG. 5 is a schematic diagram of another example of generating shadows points according to the embodiments of the present disclosure. FIG. 6 is a schematic diagram of an example of a projected two-dimensional (2D) pseudo image according to the embodiments of the present disclosure. FIG. 7 is a schematic diagram of a neural network model according to embodiments of the present disclosure. FIG. 8 shows a schematic block diagram of a 3D object detection apparatus according to embodiments of the present disclosure. FIG. 9 shows a schematic block diagram of an example device suitable for implementing embodiments of the content of the present disclosure.In the figures, like or corresponding reference numerals designate the same or corresponding parts.DETAILED DESCRIPTION OF THE EMBODIMENTSHereinafter, the embodiments of the disclosure will be described in more detail with reference to the figures. While some embodiments of the present disclosure are illustrated in the figures, it should be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but that these embodiments are provided to provide a thorough and complete understanding of the present disclosure. It should be understood that the figures and embodiments of the present disclosure are merely examples and are not intended to limit the scope of the disclosure.In the descriptions of the embodiments of the present disclosure, the term "comprise" and similar terms should be understood as open-ended inclusion, i.e., "comprise, but not be limited to.". The term "based on" should be understood to mean "at least partially based on.". The terms "an exemplary embodiment" or "this exemplary embodiment" are to be understood as "at least one exemplary embodiment". The terms "first / r / s", "second / r / s", etc. may represent different or same objects. Further explicit and implicit definitions may be included below.As smart devices develop, the need for object detection of objects in three-dimensional (3D) scenes is increasing. To facilitate 3D object detection, the detection point cloud of a 3D scene is usually projected from a predefined perspective into a two-dimensional (2D) pseudo-image in order to be able to use methods for detecting 2D images for 3D object detection. For example, common projection methods include bird's-eye view (BEV) projection method, distance view (RV) projection method, and multi-view fusion projection method.During the projection process, the detection space corresponding to the 3D scene (in which the multiple detection points of the detection point cloud are distributed) is usually divided into multiple voxels, and the 2D projection is then performed voxel-by-voxel. Therefore, during the projection process, corresponding spatial information is lost, for example information of the corresponding dimensions. Thus, in the top-view projection method (BEV), in the same voxel, only the information of the detection points having a higher height can be efficiently mapped in the vertical direction during the projection process, while the information of the detection points having a lower height can be lost. For example, a person in a 3D scene can only be projected as a head and shoulder outline while lateral information is lost. In other words, the BEV projection method results in loss of elevation information, thereby competing the detection points with respect to elevation. Similarly, depth information is lost in the RV projection method. These information losses may result in inaccurate detection results in 3D object detection. In order to improve the accuracy of the detection results, the multi-view fusion projection method, which is a combination of the above two methods, may be used. However, the multi-view fusion projection method requires a considerable amount of computation, which results in that conventional electronic devices cannot support this projection method and the cost is too high.To solve at least the foregoing and other potential problems, embodiments of the present disclosure provide a method for 3D object detection. The method includes retrieving a detection point cloud for a target 3D scene, wherein the detection point cloud includes multiple detection points for multiple object points in the target 3D scene. The method further includes generating at least one shadow point based on an offset operation of the plurality of detection points. The method further includes detecting objects in the target 3D scene based on the plurality of detection points and at least one shadow point. According to the method of the embodiment of the present disclosure, it is possible to generate shadow points by an offset operation based on the original detection points, to prevent or reduce loss of information in the 3D object detection, and thus improve the accuracy of the 3D object detection.In the following, the exemplary embodiments of the disclosure are additionally described in detail with reference to the figures. FIG. 1 shows a schematic diagram of an example environment 100 in which the apparatus and / or method according to the embodiments of the present disclosure may be implemented. As shown in FIG. 1, in the example environment 100, a 3D detection space 110 corresponding to a target 3D scene is presented. The 3D sensing space 110 may be a virtual space representing the target 3D scene. The target 3D scene may be any scene, such as a road, office area, or recreational park. In some embodiments, the length L, width W, and height H of the 3D sensing space may be adjusted according to the length, width, and height of the target 3D scene. For example, the length L, the width W, and the height H of the 3D detection space may be set to the same values as the length, width, and height of the target 3D scene.The detection point cloud 120 of the target 3D scene may be distributed in the 3D detection space 110. In some embodiments, the detection point cloud 120 may be a LiDAR point cloud generated by a LiDAR. For example, it may be a LiDAR point cloud captured by a LiDAR in a vehicle. The detection point cloud 120 may include multiple detection points for multiple object points in the target 3D scene. For example, the plurality of object points may be points on persons or vehicles in a road scene (e.g., a large number of points on each pedestrian or a large number of points on each vehicle). In FIG. 1, the detection points of the detection point cloud 120 are represented by solid circle points, and a detection point 120- i(where i is an integer greater than or equal to 1) is marked in FIG. 1. Hereinafter, description will be made with reference to the example of the detection point 120- i, but it should be understood that the description is applicable to the detection point 120- ito any other detection points. It should be further understood that the detection point cloud 120 may include more or fewer detection points than the detection points shown in FIG. 1 and may have a different distribution than that shown in FIG. 1.After retrieving the capture point cloud 120 illustrated in FIG. 1, the capture point cloud 120 may be projected into a two-dimensional (2D) pseudo image using a predefined projection method without losing corresponding dimensional information (e.g., height information in the top view projection method) or reducing the corresponding information losses. This can increase the accuracy of object detection using this 2D pseudo image. In the examples below, the predefined projection method is described in more detail by way of example using the top view projection method (i.e. BEV).FIG. 2 shows a flow diagram of a method 200 for 3D object detection according to the exemplary embodiments of the present disclosure. The method 200 may be performed by any controller or electronic device, or may be performed by a trained neural network model, or may be performed in combination by a controller or electronic device and a trained neural network model. As shown in FIG. 2, at block 202, a detection point cloud for a target 3D scene is retrieved (e.g., the detection point cloud 120 shown in FIG. 1 ), wherein the detection point cloud 120 includes multiple detection points for multiple object points in the target 3D scene. The target 3D scene may be any scene, such as a road, office area, or recreational park. The object points in the target 3D scene may be any object points, for example, points on persons, vehicles, or any other objects in the scene.In some embodiments, the detection point cloud 120 may be a LiDAR point cloud detected by a LiDAR (e.g., in a vehicle). In some embodiments, the plurality of detection points in the detection point cloud 120 correspond one-to-one to the plurality of object points. This means that a corresponding detection point exists for each object point. In some embodiments, the plurality of sensing points are distributed in the 3D sensing space (e.g., 3D sensing space 110 shown in FIG. 1 ) representing the target 3D scene. In some embodiments, the position of each detection point (e.g., detection point 120- ishown in FIG. 1 ) among the plurality of detection points in the 3D detection space 110 corresponds to the position of the corresponding object point among the plurality of object points in the target 3D scene.At block 204, at least one shadow point is generated based on an offset operation of the plurality of detection points. In some embodiments, at least one shadow point may be generated by an offset operation of the plurality of detection points in the horizontal direction. This allows the information of the original sensing point 120- ito be copied to another position in the 3D sensing space, whereby the information of a certain dimension (e.g., height) may be retained based on the degree of offset of the sensing point (this will be explained in more detail in the examples below).At block 206, objects in the target 3D scene are detected based on the plurality of detection points and at least one shadow point. For example, a 2D pseudo-image can be created by the projection based on the original multiple capture points and the generated at least one shadow point in order to capture objects in the target 3D scene. This makes it possible to obtain a 2D pseudo image with more comprehensive information on the objects to be detected by the specific dimension information displayed by the shadow points, thereby improving the accuracy of 3D object detection using this 2D pseudo image. With the method 200 according to the embodiments of the present disclosure, shadow points may be generated based on an offset operation of the original detection points in order to prevent or reduce information loss in the 3D object detection and thus increase the accuracy of the 3D object detection.FIG. 3 shows a flow diagram of a process 300 for generating shadows points according to embodiments of the present disclosure. The process illustrated in FIG. 3 may correspond to block 204 in FIG. 2. In some embodiments, each detection point 120- iin the detection point cloud retrieved in block 202 of FIG. 2 may contain detection point information, and this detection point information may include: the detection point position and the detection point features. In some embodiments, the detection point position may include: a horizontal position and a height. For example, the detection point position of each detection point 120- imay be represented by the position coordinates (x i, y i, z i) in a Cartesian coordinate system. These position coordinates may represent the position of the detection point 120- iin the 3D detection space 110 and may also represent the position of the corresponding object point in the target 3D scene, respectively. In some embodiments, the detection point features in the detection point information may indicate the features of the corresponding object point. For example, the detection point features may represent all other detection information other than the detection point position (e.g., the signal strength or spectral distribution of the detection), which detection information may be used to display the features of the object point.After multiple detection points are obtained with the above-mentioned detection point information, in some embodiments, at least one detection point among the multiple detection points may be selected as a target detection point in block 302 of FIG. 3. For example, the detection point 120- imay be selected as one of the target detection points. In block 304, for each target detection point among the at least one target detection point (e.g., target detection point 120- i), a corresponding shadow point may be generated that is horizontally shifted relative to the target detection point 120- iby shifting the horizontal position of the target detection point based on the height of the target detection point. In some embodiments, the corresponding shadow point has shadow point information, wherein the shadow point information may include: the shadow point position and the detection point features of the target detection point 120- i. In some embodiments, the shadow point position may include: the offset horizontal position obtained by the offset; and the height of the target detection point 120- i. In other words, the differences in the shadow point information of the generated corresponding shadow point and the detection point information of the target detection point 120- imay be only in the different horizontal position. Moreover, it should be understood that additional different information may be added to the shadow point information and the detection point information, for example, label information for distinguishing between the corresponding shadow point and the target detection point, as will be explained in more detail in the examples below, depending on actual need.FIG. 4 shows a schematic diagram 400 of an example for generating shadows points according to the embodiments of the present disclosure. In FIG. 4, the solid circle points represent the original detection points (including the target detection points), while the empty circles represent the shadow points, and the broken arrows indicate the offset in the horizontal direction. FIG. 4 shows three detection points 120-i, 120-j, and 120-k selected as target detection points, and the three corresponding shadow points 120-i', 120-j', and 120-k' generated by the horizontal offset of the positions of the three detection points 120-i, 120-j, and 120-k. As can be seen from FIG. 4, the corresponding shadow points 120-i', 120-j' and 120-k' have only one offset in the horizontal position compared to the original detection points (target detection points) 120-i, 120-j and 120-k, while the height remains unchanged. In this embodiment, i, j, and k may be different integers greater than or equal to 1.In some embodiments, the corresponding shadow points 120- i' of a target detection point 120- imay be generated with the position coordinates (x i, y i, z i) according to the following equation (1), equation (2), and equation (3) (it should be understood that this applies to other target detection points as well).In the above equations (1) to (3), σ represents a shift constant that can be set depending on actual needs or determined by a trained neural network model. x i', y i' and z i' are the position coordinates of the corresponding shadow point 120-i' (x i', y i', z i'). It is understood from the above equations (1) to (3) that the corresponding shadow point 120-i' is displaced only in the horizontal direction (i.e., in the (x, y) plane) as compared with the target detection point 120-i, while the height remains unchanged. Moreover, the degree of horizontal displacement of the corresponding shadow point 120-i' correlates with the height z i of the target detection point 120-i. Therefore, the height of the target detection point can be determined by the degree of offset. This makes it possible to preserve the height information of the detection points when projecting the detection point cloud into a 2D pseudo image, whereby the accuracy of the 3D object detection can be improved.It should be understood that although only 3 detection points are illustrated as target detection points in FIG. 4, any number of detection points may be selected as target detection points depending on actual needs. For example, in some embodiments, all of the detection points in the detection point cloud 120 may be selected as target detection points to preserve the detection point information as completely as possible. In other embodiments, for example, to take into account the balance between computing resources and the desired accuracy, at least one detection point that satisfies the predetermined conditions may be selected from the detection point cloud 120 as the target detection point.In some embodiments, the predetermined conditions may include the position of the sensing point corresponding to predetermined position conditions. For example, for each detection point (e.g., detection point 120- i), it may be determined whether at least one of the values of x i, y i and z i among the position coordinates (x i, y i, z i) is greater than a corresponding threshold, and when this value exceeds the threshold, the detection point 120- iis determined as a target detection point. In other embodiments, the predetermined conditions may include the capture point corresponding to a point in a predetermined range of a two-dimensional 2D image of the target 3D scene. For example, simultaneously with the detection of the detection point cloud 120 by a LiDAR of a vehicle, a 2D image of the same target 3D scene can also be recorded by a camera of the vehicle. Subsequently, a predetermined region of interest in the 2D image may be determined by any type of feature extraction, e.g., extraction of the foreground, background, any boundaries, or any other regions. The detection points corresponding to the points in the predetermined region of interest may then be selected as the target detection points described above. In further embodiments, the predetermined conditions may include the type of the detection points corresponding to a predetermined type. The type of the detection points can be determined in any desired manner. In the following, an example of how target detection points are selected based on the type of the detection points and the corresponding shadow points are generated will be described with reference to FIG. 5.FIG. 5 shows a schematic diagram 500 of another example for generating shadows points according to the embodiments of the present disclosure. In some embodiments, to perform the projection of the detection point cloud, the method 200 according to the embodiments of the present disclosure may further include: dividing the 3D detection space 110 into at least one voxel, wherein the at least one voxel (for the top view projection method) may have the same shape and volume. As shown in FIG. 5, the detection space 110 is divided into a plurality of cube-shaped voxels, with two adjacent voxels Vi and Vi+1 being identified in FIG. 5. In some embodiments, each detection point in the corresponding voxel may be below the divided at least one voxel. It should be understood that voxels may exist which do not have detection points.After splitting into at least one voxel, in some embodiments, the method 200 according to embodiments of the present disclosure may further comprise: determining voxel information for each voxel in the at least one voxel based on at least one of the position of the voxel in the 3D sensing space and the sensing point information of all sensing points within the voxel. In some embodiments, the voxel information may be determined by a trained neural network model and may indicate the overall characteristics of all the detection points in the voxel.After determining the voxel information of each voxel as described above, in some embodiments, the method 200 according to the embodiments of the present disclosure may further include: semantically dividing the corresponding voxel based on the detection point information of the respective detection point 120- i, and the voxel information of the voxel in which this detection point 120- iis located, to determine the type of the detection point 120- i. For example, if it is determined that the detection point 120- iis a detection point of a predetermined type, that detection point may be selected as a target detection point to generate a corresponding shadow point.By way of example, this is illustrated below with reference to the voxel Vi shown in FIG. 5. Voxel Vi contains 5 detection points 501, 502, 503, 504, and 505. Among these 5 detection points, the 2 detection points 501 and 505 have been identified as detection points of a predetermined type (both may belong to different predetermined types) and thus selected as target detection points. For these two target detection points 501 and 505, the corresponding shadow points 501' and 505' have been generated. In some embodiments, to ensure that the shadow points can more effectively represent information about particular dimensions (e.g., height), it may be arranged that the target detection points 501 and 505 and the corresponding shadow points 501' and 505' are located in different voxels Vi and Vi+1, and the shadow point information may further include the voxel information of the voxel Vi in which the target detection points 501 and 505 are located. It should be understood that although it is shown in FIG. 5 that the corresponding shadow points 501' and 505' of the target detection points 501 and 505 are located in the same voxel Vi+1, the corresponding shadow points 501' and 505' of the target detection points 501 and 505 in the voxel Vi may also be located in different voxels (e.g., in adjacent or non-adjacent voxels).After at least one shadow point has been generated as described above, the 3D object detection may be performed based on the original multiple detection points and the generated at least one shadow point, i.e., block 206 of FIG. 2 is executed. In some embodiments, block 206 may include: performing a projection based on the top view projection method by projecting the plurality of capture points and at least one shadow point into a 2D pseudo-image by the at least one voxel, such as those split in FIG. 5. In some embodiments, the projection may cause the voxel information of each voxel among the at least one voxels as well as the shadow point information of the shadow points in these voxels to be mapped as pixel information of the corresponding pixels in the 2D pseudo-image.FIG. 6 shows a schematic diagram 600 of an example of a 2D projected pseudo image according to embodiments of the present disclosure. As shown in FIG. 6, the 3D detection space 110 is divided into a plurality of voxels, each voxel being a cubic region having the same shape and volume. Some voxels may contain neither capture points nor shadow points. Other voxels may contain detection points but not shadow points. Still other voxels may contain only shadow points originating from other voxels. There are also voxels that may contain both detection points and shadow points. Regardless of the type of voxel, the voxel information of the voxel as well as the shadow point information of the shadow point (if present) may be mapped as pixel information of the corresponding pixel Pi in the 2D pseudo image 610. It should be understood that the pixels Piof the pseudo-image 610 shown in FIG. 6 are shown for illustration purposes only and are not intended to indicate the actual representation of the representations depicted in the projected image.After the 2D pseudo-image 610 is obtained as described above, block 206 may, in some embodiments, further comprise: detecting objects in the target 3D scene based on the 2D pseudo-image 610. In some embodiments, the method 200 described above with respect to FIGS. 2-6 may be performed in whole or in part by a trained neural network model. FIG. 7 shows a schematic diagram 700 of a neural network model according to embodiments of the present disclosure. In some embodiments, neural network model 700 may be created based on any neural network model for 3D object detection to perform 3D object detection. Examples thereof include multilayer perception and max pooling neural network models or a pillar-based 3D object detection neural network model.In the case that the method 200 according to the present disclosure is fully executed by the neural network model 700, the trained neural network model 700 may be obtained by adding an algorithm for generating shadow points to a known neural network model for 3D object detection and by training. During the training process of the neural network model 700, its inputs may be the detection points in the obtained detection point cloud, while the outputs may be the detected objects. This allows the neural network model 700 to be trained with the training data of an original neural network for 3D object detection without the need to create new training data. This enables known neural networks to be used for 3D object detection and the training data thereof in order to implement the method according to the present disclosure in a simple and efficient manner.Moreover, in the case that the method 200 according to the present disclosure is only partially executed by the neural network model 700, it is only necessary to train a relatively simple predefined neural network model for the corresponding part of the method in order to obtain the trained neural network model 700. Additionally, during execution of the method 200 described above, in some embodiments, flag information (e.g., flag bit mask) may be added to the original detection points and the corresponding shadow points by the trained neural network model 700 to distinguish between these two. For example, the flag bit mask for the original detection points may be "0" or "1" while the flag bit mask for the corresponding shadow points may be the other of "0" and "1". It should be understood that this is merely an example and any other form of flag information may be used depending on the actual need. Moreover, in some embodiments, the trained neural network model 700 may select during the process of object detection whether it uses the shadow point information of some or all of the shadow points, which may depend on, for example, the required object detection accuracy and the desired detection efficiency.FIG. 8 shows a schematic block diagram of a 3D object detection apparatus 800 according to embodiments of the present disclosure. As shown in FIG. 8, the apparatus 800 comprises a retrieving unit 802 configured to retrieve a detection point cloud for a target 3D scene, wherein the detection point cloud comprises a plurality of detection points for a plurality of object points in the target 3D scene. The apparatus 800 further includes a generation unit 804 configured to generate at least one shadow point based on an offset operation of the plurality of detection points. The apparatus 800 further comprises an acquisition unit 806 configured to acquire objects in the target 3D scene based on the plurality of acquisition points and the at least one shadow point. In some embodiments, device 800 may be a trained neural network model.In some embodiments, the acquisition point cloud may be a LiDAR point cloud generated by a LiDAR. In some embodiments, the plurality of detection points may correspond to the plurality of object points one by one. In some embodiments, the plurality of detection points may be distributed in the 3D detection space representing the target 3D scene. In some embodiments, the position of each detection point among the plurality of detection points in the 3D detection space may correspond to the position of the corresponding object point among the plurality of object points in the target 3D scene. In some embodiments, the detection points may have detection point information, and the detection point information may include: the detection point position and the detection point features. In some embodiments, the detection point position may include: a horizontal position and a height. In some embodiments, the capture point features may indicate the features of the corresponding object point.In some embodiments, the generation unit 804 may be configured to select at least one detection point among the plurality of detection points as a target detection point. In some embodiments, the generation unit 804 may be configured to select all of the detection points among the plurality of detection points as target detection points. In some embodiments, the generation unit 804 may be configured to select at least one detection point satisfying predetermined conditions among the plurality of detection points as a target detection point. In some embodiments, the predetermined conditions may include the position of the sensing point corresponding to predetermined position conditions. In some embodiments, the predetermined conditions may include the type of the sensing points corresponding to a predetermined type. In some embodiments, the predetermined conditions may include the capture point corresponding to a point in a predetermined region of a two-dimensional 2D image of the target 3D scene.In some embodiments, the generation unit 804 may be further configured to generate, for each target detection point among the at least one target detection point, a corresponding shadow point that is offset in the horizontal direction relative to the target detection point by shifting the horizontal position of the target detection point based on the height of the detection point. In some embodiments, the corresponding shadow point may have shadow point information, wherein the shadow point information may include: the shadow point position and the detection point features of the target detection point. In some embodiments, the shadow point position may include: the offset horizontal position obtained by the offset; and the height of the target detection point.In some embodiments, the apparatus 800 may further comprise a voxel unit, wherein the voxel unit is configured for dividing the 3D sensing space into at least one voxel, wherein each of the at least one voxels has the same shape and volume. In some embodiments, the voxel unit may be further configured to determine the voxel information for each voxel in the at least one voxel based on at least one of: the position of the voxel in the 3D detection space and the detection point information of all detection points within the voxel. In some embodiments, the apparatus 800 may further comprise a type determination unit configured to determine the type of each detection point in the following manner: semantically dividing the corresponding voxel based on the detection point information of the respective detection point and the voxel information of the voxel in which this detection point is located, in order to determine the type of the detection point. In some embodiments, each detection point in the corresponding voxel may be below the at least one voxel. In some embodiments, the target detection points and the corresponding shadow points may be located in different voxels. In some embodiments, the shadow point information may further include the voxel information of the voxel in which the target detection points are located.In some embodiments, the acquisition unit 806 may be configured to project the plurality of acquisition points and at least one shadow point through at least one voxel into a 2D pseudo-image based on the top view projection method. In some embodiments, the projection may cause the voxel information of each voxel among the at least one voxels as well as the shadow point information of the shadow points in these voxels to be mapped as pixel information of the corresponding pixels in the 2D pseudo-image. In some embodiments, the sensing unit 806 may be further configured to sense objects in the target 3D scene based on the 2D pseudo-image.With the apparatus 800 according to the embodiments of the present disclosure, shadow points may be generated based on an offset operation of the original detection points to prevent or reduce loss of information in the 3D object detection, thereby increasing the accuracy of the 3D object detection.FIG. 9 shows a schematic block diagram of an example device 900 suitable for implementing embodiments of the present disclosure. The above-mentioned control device can be realized by means of the device 900. As shown in the figure, the apparatus 900 comprises a processor 901 that can perform various suitable actions and processes in accordance with computer program instructions stored in a read only memory (ROM) 902 and loaded into a random access memory (RAM) 903. The RAM 903 may further store various programs and data required for the operation of the device 900. The processor 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input / output interface (I / O) 905 is also connected to bus 904.Any process described above and any processing, e.g., method 200 and process 300, may be performed by processor 901. In some embodiments, 200 and process 300 may be implemented, for example, as a computer software program that is physically embodied in a machine-readable medium. In some embodiments, the computer program may be fully or partially loaded and / or installed onto the device 900 via the ROM 802. When the computer program is loaded into the RAM 903 and executed by the processor 901, one or more actions of the method 200 and process 300 described above may be performed.The present disclosure may be a method, apparatus, system, and / or computer program product. The computer program product may include a computer readable storage medium having thereon computer readable program instructions for carrying out the various aspects of the present disclosure.A computer readable storage medium may be a tangible device that can receive and store instructions for use by an apparatus for instruction execution. The computer readable storage medium may be, for example, but not limited to, an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, or a suitable combination of the foregoing. More specific (but not exhaustive) examples of computer readable storage media include: portable computer diskettes, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or Flash memories), static random access memories (SRAM), portable compact disc read-only memories (CD-ROM), digital multi-function disks (DVD), memory sticks, floppy disks, mechanical coding devices such as punch-cards or raised structures in depressions with stored instructions, and any suitable combinations of the above media. The computer readable storage media used herein are not to be interpreted as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through an optical fiber cable), or electrical signals transmitted through wires.The computer readable program instructions described herein may be downloaded from a computer readable storage medium to various computer / processing devices or to an external computer or external storage device via a network such as the Internet, a local area network or a wide area network, and / or a wireless network. A network adapter card or network interface in each computer / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage on a computer readable storage medium in the respective computer / processing device.The computer program instructions for performing operations within the scope of the present disclosure may be assembler instructions, ISA (Instruction Set Architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, the programming languages including object oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on a computer, partly on a computer, as a stand-alone software package, partly on a computer and partly on a remote computer or entirely on a remote computer or server. In the latter case, the remote computer may be connected to a computer by any type of network (including a local area network (LAN) or a wide area network (WAN)), or may be connected to an external computer (for example, by an Internet Service Provider via the Internet). In some embodiments, the electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be adapted using state information of computer readable program instructions, wherein the electronic circuit may execute the computer readable program instructions to implement various aspects of the present disclosure.

Claims

A method for three-dimensional 3D object detection, comprising: retrieving a detection point cloud for a target 3D scene, wherein the detection point cloud comprises a plurality of detection points for a plurality of object points in the target 3D scene; generating at least one shadow point based on an offset operation of the plurality of detection points; and detecting objects in the target 3D scene based on the plurality of detection points and the at least one shadow point.The method of claim 1, wherein the plurality of detection points are associated one-to-one with the plurality of object points, wherein the plurality of detection points are distributed in the 3D detection space representing the target 3D scene, and wherein the position of each detection point among the plurality of detection points in the 3D detection space corresponds to the position of the associated object point among the plurality of object points in the target 3D scene.The method of claim 2, wherein the detection points have detection point information and the detection point information comprises: detection point position and detection point features, wherein the detection point position comprises: horizontal position and height, and wherein the detection point features indicate the features of the associated object point.The method of claim 3, wherein the generating at least one shadow point based on the plurality of detection points comprises: selecting at least one detection point among the plurality of detection points as a target detection point; and for each target detection point among the at least one target detection point, generating a corresponding shadow point that is offset relative to the target detection point in the horizontal direction by displacing the horizontal position of the target detection point based on the height of the detection point, wherein the corresponding shadow point has shadow point information, and the shadow point information comprises: the shadow point position and the detection point features of the target detection point, and wherein the shadow point position comprises: the horizontal offset position obtained by the offset and the height of the target detection point.The method according to claim 4, wherein the selecting at least one detection point among the plurality of detection points as a target detection point comprises: selecting all the detection points among the plurality of detection points as target detection points, or selecting at least one detection point satisfying predetermined conditions among the plurality of detection points as a target detection point.The method of claim 5, wherein the predetermined conditions comprise at least one of: the position of the sensing point satisfies predetermined position conditions, the type of the sensing point belongs to a predetermined type, and the sensing point corresponds to a point in a predetermined area of the two-dimensional 2D image of the target 3D scene.The method of claim 6, further comprising: dividing the 3D detection space into at least one voxel, each of the at least one voxels having the same shape and volume; and determining the voxel information for each voxel among the at least one voxel based on at least one of: the position of the voxel in the 3D detection space and the detection point information of all the detection points within the voxel.The method of claim 7, wherein each detection point is located in a corresponding voxel among the at least one voxels, and wherein the target detection point and the corresponding shadow point are located in different voxels, and the shadow point information further comprises the voxel information of the voxel in which the target detection point is located.The method of claim 8, wherein detecting the objects in the target 3D scene based on the plurality of detection points and the at least one shadow point comprises: projecting the plurality of detection points and the at least one shadow point into a 2D pseudo image by a top view projection method through the at least one voxel; and detecting the objects in the target 3D scene based on the 2D pseudo image, wherein the projection causes the voxel information of each voxel among the at least one voxels as well as the shadow point information of the shadow points located in the voxel to be mapped as pixel information of the corresponding pixels in the 2D pseudo image.The method of claim 7, further comprising determining the type of each detection point: semantically dividing the corresponding voxel based on the detection point information of the respective detection point as well as the voxel information of the voxel in which that detection point is located to determine the type of the detection point.The method of any one of claims 1 to 10, wherein the acquisition point cloud is a LiDAR point cloud acquired by a LiDAR, and wherein the method is performed by a trained neural network model.A three-dimensional 3D object detection apparatus comprising: a retrieving unit configured to retrieve a detection point cloud for a target 3D scene, wherein the detection point cloud comprises a plurality of detection points for a plurality of object points in the target 3D scene; a generating unit configured to generate at least one shadow point based on an offset operation of the plurality of detection points; and a detecting unit configured to detect objects in the target 3D scene based on the plurality of detection points and the at least one shadow point.A controller, comprising: at least one processor; and a memory coupled to the at least one processor and having instructions stored thereon that, when executed by the at least one processor, cause the controller to perform a method according to any one of claims 1 to 11.A vehicle comprising a controller according to claim 13.A computer readable storage medium having computer executable instructions stored thereon, the computer executable instructions being executed by a processor to implement a method according to any one of claims 1 to 11.

Images