Target object detection method and apparatus

Abstract

The application discloses a target object detection method and device, and relates to the technical field of vehicle detection, and comprises the following steps: acquiring point cloud data of an object in a target region of interest monitored by a 4D millimeter wave radar, wherein the point cloud data comprises object position coordinates and X-dimensional target data of the object; distributing the point cloud data in the form of a top view to a preset plane to obtain a plurality of cubical columns; performing feature conversion on the point cloud data in each cubical column to obtain a 2D feature tensor; constructing an image of each 2D feature tensor in the cubical column corresponding to the 2D feature tensor to obtain a 2D pseudo image; and detecting the object in the 2D pseudo image by using a two-dimensional convolution model to obtain a detection result of a target object. The application uses relative radial velocity data and / or intensity data in the 4D millimeter wave radar as additional input features to detect the target object, further improves the accuracy of vehicle detection of the target object, improves the safety of the automatic driving system, and reduces the cost.

Description

Technical Field

[0001] This application belongs to the field of vehicle inspection technology, and in particular relates to a method and apparatus for detecting target objects. Background Technology

[0002] To meet the demands of intelligent driving, the performance of automotive radar has significantly improved in recent years. As autonomous driving technology progresses from the demonstration phase to the implementation phase, higher requirements are being placed on perception capabilities. Mainstream autonomous driving systems rely on the fusion of cameras and LiDAR for perception, while millimeter-wave radar is mainly used in active safety functions such as Automatic Emergency Braking (AEB) and Forward Collision Warning (FCW).

[0003] Traditional millimeter-wave radar only provides information in three dimensions: horizontal angle, speed, and distance. If traditional millimeter-wave radar is used for vehicle detection, it lacks pitch angle information, resulting in low height resolution. This makes it impossible to measure altitude and determine whether a stationary object in front is on the ground or in the air, which may lead to phenomena such as ghost braking.

[0004] Therefore, new 4D millimeter-wave radar is now being used for detection. However, the 3D target detection model of 4D millimeter-wave radar point cloud is basically a transfer of some well-performing and mature models from LiDAR for training. There is still a lack of special design and optimization for the characteristics of 4D millimeter-wave radar, which makes the overall target detection accuracy of the deep learning model not high, and there is still a large gap from the detection accuracy of LiDAR point cloud. Summary of the Invention

[0005] This application provides a target object detection method and apparatus to improve the accuracy of target object detection.

[0006] According to a first aspect of this application, embodiments of this application provide a target object detection method, comprising:

[0007] Acquire point cloud data of objects within the region of interest detected by the 4D millimeter-wave radar. The point cloud data includes the object's position coordinates and X-dimensional target data. The X-dimensional target data is determined based on feature data, which includes the relative radial velocity data between the vehicle and the object and / or the class intensity data of the reflected wave received by the 4D millimeter-wave radar from the object. X is a positive integer.

[0008] The point cloud data is distributed onto a preset plane in the form of a top view, resulting in multiple cubic pillars with a size of (D, P, N). Here, D is the number of feature dimensions, which is determined according to the type of point cloud data, P is the number of cubic pillars with a non-zero number of point clouds, and N is the number of point clouds within each cubic pillar.

[0009] The point cloud data in each cubic column is subjected to feature transformation to obtain a 2D feature tensor.

[0010] The image is constructed by placing each 2D feature tensor into the corresponding cubic pillar of the 2D feature tensor to obtain a 2D pseudo image;

[0011] The detection results of the target object are obtained by detecting objects in 2D pseudo-images using a two-dimensional convolution model.

[0012] Optionally, the point cloud data is distributed onto a preset plane in the form of a top view, resulting in multiple cubic columns, including:

[0013] The point cloud data is distributed onto a preset plane in the form of a top view, resulting in multiple quasi-cubic columns;

[0014] Based on the point cloud data in the quasi-cubic column, determine the point position coordinates of each point in the quasi-cubic column;

[0015] Based on the coordinates of the points, calculate the average positional offset between the center point and all other points in the quasi-cubic column, where the center point is the point at the center of the column.

[0016] Determine the center position offset based on the object's position coordinates and center coordinates, where the center coordinates are the position coordinates of the center point;

[0017] Combine the object coordinates, average position offset, and center position offset to obtain Z-dimensional position data, where Z is a positive integer;

[0018] The number of feature dimensions D is determined based on the number of dimensions of the Z-dimensional location data and the number of dimensions of the X-dimensional target data.

[0019] Based on the point cloud data of multiple quasi-cubic cylinders, determine the number N of point clouds in each quasi-cubic cylinder and the number P of cubic cylinders with a non-zero number of point clouds.

[0020] Determine the cubic prism based on D, P, and N.

[0021] Optionally, the X-dimensional target data is X-dimensional relative radial velocity data.

[0022] Optionally, obtain X-dimensional target data, including:

[0023] Based on the object's position coordinates, determine the horizontal and pitch angles between the object and the vehicle;

[0024] Based on the horizontal angle, pitch angle, and relative radial velocity data, the absolute radial velocity data of the object and the vehicle are calculated through vector decomposition.

[0025] The absolute radial velocity data is defined as the X-dimensional target data.

[0026] Optionally, obtain X-dimensional target data, including:

[0027] Based on the object's position coordinates, determine the horizontal and pitch angles between the object and the vehicle;

[0028] Based on the horizontal angle, pitch angle, and relative radial velocity data, the absolute radial velocity data of the object and the vehicle are calculated through vector decomposition.

[0029] The absolute radial velocity data is decomposed along the first direction and the second direction to obtain the first velocity component data and the second velocity component data. The first direction is the x-axis direction of the vehicle coordinate system, and the second direction is the y-axis direction of the vehicle coordinate system.

[0030] Calculate the average velocity data of each point in the cubic column in the first and second directions respectively as the first average velocity data and the second average velocity data.

[0031] The difference between the first velocity component data and the first velocity average data is determined as the first target velocity difference data.

[0032] The difference between the second velocity component data and the second velocity average data is determined as the second target velocity difference data.

[0033] The first velocity component data, the second velocity component data, the first target velocity difference data, and the second target velocity difference data are spliced ​​and combined to generate X-dimensional target data.

[0034] Optionally, the X-dimensional target data is class intensity data;

[0035] Optionally, obtain X-dimensional target data, including:

[0036] Acquire multiple types of intensity data from the radar under different operating modes;

[0037] The maximum value among multiple class strength data is determined as the valid class strength data;

[0038] The effective field intensity data is determined as X-dimensional target data.

[0039] Optionally, obtain X-dimensional target data, including:

[0040] Acquire multiple types of intensity data from the radar under different operating modes;

[0041] The maximum value among multiple class strength data is determined as the valid class strength data;

[0042] Acquire distance data from the radar's range gate;

[0043] Based on the magnitude of the distance data, the difference between the effective class intensity distance data and the threshold is calculated to obtain the true reflection cross section;

[0044] Based on the preset relationship between the reflection cross section and the class intensity data, the real class intensity data corresponding to the real reflection cross section is determined as the X-dimensional target data.

[0045] Optionally, feature transformation is performed on the point cloud data in each cubic column to obtain a 2D feature tensor, including:

[0046] The point cloud data in each cubic column is used to extract features by an encoder to obtain a tensor;

[0047] Extract valid points from the points of the cubic column using pooling operations;

[0048] Based on the tensor, the effective points are compressed to obtain the 2D feature tensor.

[0049] According to a second aspect of this application, embodiments of this application provide a target object detection device, the device comprising:

[0050] The acquisition module acquires point cloud data of objects within the region of interest detected by the 4D millimeter-wave radar. The point cloud data includes the object's position coordinates and X-dimensional target data. The X-dimensional target data is determined by feature data, which includes the relative radial velocity data between the vehicle and the object and / or the class intensity data of the reflected wave received by the 4D millimeter-wave radar from the object, where X is a positive integer.

[0051] The allocation module distributes the point cloud data to a preset plane in the form of a top view, resulting in multiple cubic pillars. The size of each cubic pillar is (D, P, N), where D is the feature dimension, which is determined based on the point cloud data, P is the number of cubic pillars with a non-zero number of point clouds, and N is the number of point clouds within each cubic pillar.

[0052] The transformation module is used to perform feature transformation on the point cloud data in each cubic column to obtain a 2D feature tensor.

[0053] The construction module is used to construct images from the cubic pillars corresponding to each 2D feature tensor to obtain 2D pseudo-images.

[0054] The detection module is used to detect objects in 2D pseudo-images using a two-dimensional convolutional model, and obtain the detection results of the target objects.

[0055] According to a third aspect of this application, a target object detection device is provided, the device comprising: a processor and a memory storing computer program instructions;

[0056] When the processor executes computer program instructions, it implements the target object detection method of any one of the first aspects.

[0057] According to a fourth aspect of this application, embodiments of this application provide a computer storage medium storing computer program instructions, which, when executed by a processor, implement the target object detection method of any one of the first aspects.

[0058] According to a fifth aspect of this application, embodiments of this application provide a computer program product in which instructions, when executed by a processor of an electronic device, cause the electronic device to perform the target object detection method of any one of the first aspects.

[0059] The target object detection method and apparatus of this application acquire point cloud data containing object position coordinates and X-dimensional target data, generate cubic pillars from the point cloud data, perform feature transformation on the point cloud data during the cubic pillar process to generate a 2D feature tensor, and reconstruct a 2D pseudo-image from the 2D feature tensor. The object in the 2D pseudo-image can then be detected using a two-dimensional convolutional model to obtain the target object detection result. Based on this, by using target data from millimeter-wave radar as additional input features and performing target object detection based on the adjusted cubic pillars, the accuracy of vehicle target object detection is further improved, enhancing the safety of the autonomous driving system while reducing costs. Attached Figure Description

[0060] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a flowchart illustrating a target object detection method according to an exemplary embodiment;

[0062] Figure 2 This is another flowchart illustrating a target object detection method according to an exemplary embodiment;

[0063] Figure 3 This is yet another flowchart illustrating a target object detection method according to an exemplary embodiment;

[0064] Figure 4 This is a structural block diagram of a target object detection device according to an exemplary embodiment;

[0065] Figure 5 This is a structural block diagram of a target object detection device according to an exemplary embodiment. Detailed Implementation

[0066] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0067] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-external inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0068] As described in the background section, traditional millimeter-wave radar only provides information in three dimensions: horizontal angle, speed, and distance. If traditional millimeter-wave radar is used for vehicle detection, it lacks pitch angle information, resulting in low height resolution, making it impossible to measure height and determine whether a stationary object in front is on the ground or in the air. This may lead to phenomena such as ghost braking.

[0069] To address the problems of existing technologies, this application utilizes relative radial velocity data and / or class intensity data from 4D millimeter-wave radar as additional input features to detect target objects, thereby further improving the accuracy of vehicle target object detection, enhancing the safety of autonomous driving systems, and reducing costs.

[0070] Based on this, this application acquires point cloud data containing object location coordinates and X-dimensional target data, generates cubic pillars from the point cloud data, performs feature transformation on the point cloud data during the process of creating the cubic pillars, thereby generating a 2D feature tensor, and reconstructs the 2D feature tensor to obtain a 2D pseudo-image. This allows for the detection of objects in the 2D pseudo-image using a two-dimensional convolutional model, yielding the target object detection result. Furthermore, by using target data from millimeter-wave radar as additional input features and performing target object detection based on the adjusted cubic pillars, the accuracy of vehicle target object detection is further improved, enhancing the safety of the autonomous driving system while reducing costs.

[0071] Based on this, this application provides a target object detection method and apparatus. The target object detection method provided in the embodiments of this application will be described first below.

[0072] Figure 1 A flowchart illustrating a target object detection method according to an embodiment of this application is shown. Figure 1 As shown, it may include the following steps:

[0073] S101, acquire point cloud data of objects within the region of interest detected by the 4D millimeter-wave radar. The point cloud data includes the object's position coordinates and X-dimensional target data. The X-dimensional target data is determined based on feature data, which includes the relative radial velocity data between the vehicle and the object and / or the class intensity data of the reflected wave received by the 4D millimeter-wave radar from the object. X is a positive integer.

[0074] S102, the point cloud data is distributed to a preset plane in the form of a top view to obtain multiple cubic pillars. The size of the cubic pillars is (D, P, N), where D is the number of feature dimensions, which is determined according to the type of point cloud data, P is the number of cubic pillars with a non-zero number of point clouds, and N is the number of point clouds in the cubic pillars.

[0075] S103, Perform feature transformation on the point cloud data in each cubic column to obtain a 2D feature tensor;

[0076] S104, construct an image from each 2D feature tensor in the corresponding cubic pillar of the 2D feature tensor to obtain a 2D pseudo image;

[0077] S105 uses a two-dimensional convolution model to detect objects in a 2D pseudo-image and obtains the detection results of the target object.

[0078] Based on the above embodiments, after acquiring point cloud data containing object position coordinates and X-dimensional target data, a cubic prism is generated from the point cloud data. Then, feature transformation is performed on the point cloud data during the process of creating the cubic prism, thereby generating a 2D feature tensor. This 2D feature tensor is then used to reconstruct an image, resulting in a 2D pseudo-image. A two-dimensional convolutional model can then be used to detect objects in the 2D pseudo-image, yielding the target object detection result. Based on this, using target data from millimeter-wave radar as additional input features and performing target object detection based on the adjusted cubic prism further improves the accuracy of vehicle target object detection, enhances the safety of the autonomous driving system, and reduces costs.

[0079] In S101 above, the 4D millimeter-wave radar emits radar waves to detect the target region of interest, then receives the reflected radar waves and acquires point cloud data of all objects in the target region of interest. The objects may include dynamic or static objects such as obstacles and vehicles. The point cloud data may include the position coordinates of the objects and target data. The target data is determined based on feature data. Specifically, the feature data may include relative radial velocity data and / or intensity data of the reflected waves from the objects received by the 4D millimeter-wave radar.

[0080] When the feature data includes relative radial velocity data, determining the X-dimensional target data from the relative radial velocity data can include the following implementation methods:

[0081] In one example, the raw data output by the millimeter-wave radar includes relative radial velocity data, so the feature data can be directly relative radial velocity data. Therefore, the X-dimensional target data is directly relative radial velocity data. In this embodiment, X is 1.

[0082] In another example, since the relative radial velocity data is the velocity of the object relative to the vehicle, its value will change with the real-time change of the vehicle speed. For example, the relative radial velocity data corresponding to a stationary object is negative, and the value is unstable when the radar detects different frames. Therefore, self-vehicle motion compensation is considered. Based on this, the horizontal angle and pitch angle between the object and the vehicle can be determined according to the object's position coordinates.

[0083] Based on the horizontal angle, pitch angle, and relative radial velocity data, the absolute radial velocity data of the object and the vehicle are calculated by vector decomposition of the relative radial velocity data.

[0084] Therefore, the absolute radial velocity data can be used as feature data, that is, the absolute radial velocity data is X-dimensional target data, and in this embodiment, X is 1.

[0085] In another example, to construct the cubic pillars more accurately and ensure the accuracy of the point cloud arrangement within the cubic pillars, the feature data can also be considered in conjunction with the cubic pillars.

[0086] Specifically, based on the object's position coordinates, determine the horizontal angle and pitch angle between the object and the vehicle;

[0087] Based on the horizontal angle, pitch angle, and relative radial velocity data, the absolute radial velocity data of the object and the vehicle are calculated through vector decomposition.

[0088] The absolute radial velocity data is decomposed along the first direction and the second direction to obtain the first velocity component data and the second velocity component data. The first direction is the x-axis direction of the vehicle coordinate system, and the second direction is the y-axis direction of the vehicle coordinate system.

[0089] Calculate the average velocity data of each point in the cubic column in the first and second directions respectively as the first average velocity data and the second average velocity data.

[0090] The difference between the first velocity component data and the first velocity average data is determined as the first target velocity difference data.

[0091] The difference between the second velocity component data and the second velocity average data is determined as the second target velocity difference data.

[0092] The first velocity component data, the second velocity component data, the first target velocity difference data, and the second target velocity difference data are spliced ​​and combined to generate X-dimensional target data.

[0093] Based on the known relative radial velocity data of each point, the absolute radial velocity data of each point cloud in the cubic column can be calculated. The absolute radial velocity data is decomposed along the x and y axes based on the vehicle coordinate system to obtain the first velocity component data and the second velocity component data. Then, based on the cubic column, the average difference between the first velocity component data and the second velocity component data of each point and the first velocity component data and the second velocity component data of the center point is calculated as the first target velocity difference data and the second target velocity difference data. Therefore, the first velocity component data, the second velocity component data, the first target velocity difference data and the second target velocity difference data obtained after combining the cubic column can be used as feature data. The feature data can be directly determined as X-dimensional target data. In this embodiment, X is 4.

[0094] When the feature data includes class intensity data, determining the X-dimensional target data from the feature velocity can include the following implementation methods:

[0095] In one example, the millimeter-wave radar also outputs class intensity data, the specific value of which is related to the material of the object. Different objects have different materials, and their class intensity data are also different. In order to increase the accuracy of object detection, feature data can be directly determined based on class intensity data. Thus, the X-dimensional target data can also be directly the class intensity data. In this embodiment, X is 1.

[0096] In another embodiment, the class intensity data is a physical quantity related to the target's radar cross section. For a specific radar, the two decibel values ​​differ by a fixed value. However, for different radar operating modes, their transmit power and antenna gain are different, so it is necessary to calibrate for each operating mode. The maximum value of the class intensity data is relatively stable, and it shows a relatively good regular change at different positions and angles. Therefore, the maximum value is used to calibrate the target's true radar cross section value.

[0097] Therefore, it is necessary to acquire multiple types of intensity data of the radar under different operating modes;

[0098] The maximum value among multiple class strength data is determined as the valid class strength data;

[0099] The effective field intensity data is defined as X-dimensional target data; that is, the X-dimensional target data is the maximum value of the field intensity data of the radar in different operating modes. In this embodiment, X is 1.

[0100] In another embodiment, theoretically, the intensity class data has already calibrated against the range effect and is only related to the target's radar cross-section characteristics. The measured intensity class data at 10m and 15m are quite consistent, while the intensity class data measured at 5m is lower, mainly due to range gate suppression at the first few ranges and the antenna's near-field characteristics. The intensity class data also needs to consider the actual radar cross-section.

[0101] The specific method for obtaining the intensity of a land type field may include: determining the maximum value among multiple intensity data as the valid intensity data;

[0102] Acquire distance data from the radar's range gate;

[0103] Based on the magnitude of the distance data, the difference between the effective class intensity distance data and the threshold is calculated to obtain the true reflection cross section;

[0104] Based on the preset relationship between the reflection cross section and the class intensity data, the real class intensity data corresponding to the real reflection cross section is determined as the X-dimensional target data.

[0105] Specifically, the range gate refers to a specific time period in the echo signal received by the radar. It is an important parameter in the radar system used to determine the distance between the target object and the radar.

[0106] The true reflection cross section can be calculated using the following formula:

[0107]

[0108] If the actual reflection cross section of the object is selected as the characteristic data reflecting the object, then the actual reflection cross section is used as the X-dimensional target data. In this embodiment, X is 1.

[0109] By introducing field intensity data, the model's ability to distinguish objects of different materials is enhanced, thereby improving the accuracy of object detection.

[0110] When the feature data includes relative radial velocity data and intensity-type data, the feature data from any embodiment when the feature data is relative radial velocity data and the feature data from any embodiment when the feature data is intensity-type data are spliced ​​together to form new X-dimensional target data. Here, X is the target data, which includes feature data related to relative radial velocity data and feature data related to intensity-type data. This allows for full utilization of the different physical properties output by millimeter-wave radar point clouds to perform richer feature learning, further improving detection accuracy.

[0111] As an example, X, representing the target data, can be (first velocity component data, second velocity component data, first target velocity difference data, second target velocity difference data, and actual reflection cross section). In this embodiment, X is 5.

[0112] In step S102, the point cloud data is first divided into regions, and the point cloud data is distributed into equally sized cubic prisms on the xy plane in the form of a top view. Specifically, the size of the cubic prism is (D, P, N), where D is the number of feature dimensions, which is determined according to the type of point cloud data and is related to the X value; P is the number of cubic prisms among the multiple cubic prisms where the number of point clouds is not zero; and N is the number of point clouds within each cubic prism.

[0113] In S103, the point cloud data in the cubic column is transformed and compressed to generate a 2D feature tensor of size (C,P), thereby avoiding complex three-dimensional convolution calculations and simplifying the calculation and generation steps.

[0114] In S104, the 2D feature tensor is encoded and then placed back into its original cubic column position to generate a 2D pseudo-image of size (C,H,W). H and W can be regarded as the height and width of the feature map, P = H × W, and C is the number of channels.

[0115] In S105, after converting the 3D point cloud data into a 2D pseudo-image, the object is detected by applying some backbone networks and detection heads of the 2D image to the 2D convolution model. In other words, 2D convolution is used to reduce the amount of computation and complexity.

[0116] To improve the accuracy of target object detection, this application also provides another implementation of the target object detection method.

[0117] Figure 2 Another schematic flowchart of a target object detection method provided in one embodiment of this application is shown, as follows: Figure 2 As shown, the above S102 may include the following steps:

[0118] S201, Distribute the point cloud data to a preset plane in the form of a top view to obtain multiple quasi-cubic columns;

[0119] S202, determine the point position coordinates of each point in the quasi-cubic column based on the point cloud data in the quasi-cubic column;

[0120] S203, based on the coordinates of the points, calculate the average positional offset between the center point and other points in the quasi-cubic column, where the center point is the point at the center of the column;

[0121] S204, Determine the center position offset based on the object's position coordinates and center coordinates, where the center coordinates are the position coordinates of the center point;

[0122] S205, combine the object coordinates, average position offset, and center position offset to obtain Z-dimensional position data, where Z is a positive integer;

[0123] S206, Determine the number of feature dimensions D based on the number of dimensions of the Z-dimensional location data and the number of dimensions of the X-dimensional target data;

[0124] S207, Based on the point cloud data of multiple quasi-cubic cylinders, determine the number N of point clouds in each quasi-cubic cylinder and the number P of cubic cylinders with a non-zero number of point clouds.

[0125] S208, determine the cubic column based on D, P, and N.

[0126] Based on the above embodiments, by combining the object's position coordinates and target data, the number of feature dimensions of the cube pillar is determined, new feature data is added, and a cube pillar related to the feature data is generated, thereby improving the accuracy of object detection.

[0127] In S201, the point cloud data is distributed to a preset plane in the form of a top view. The height of the cubic column is related to the number of feature dimensions, so it is necessary to refer to the number of feature dimensions in the point cloud data. Therefore, the cubic column at this time is still a quasi-cubic column and has not been determined.

[0128] In S202, the points in the quasi-cubic column correspond to the points in the point cloud data, only the representation is different. After obtaining the quasi-cubic column, the coordinates of the center point of the center position of the cubic column can be found directly. It should be noted that each point in the quasi-cubic column is three-dimensional, that is, it needs to be represented by x, y, and z.

[0129] In S203, based on the position coordinates of each point in the cubic column, the average position offset between the center point and all other points in the quasi-cubic column is calculated. The average position offset is in three-dimensional coordinates.

[0130] In S204, based on the object's position coordinates and the center coordinates of the center point, the center position offset between the object and the center point is determined. The object's position coordinates are three-dimensional coordinates, and the center position offset is also a three-dimensional coordinate.

[0131] In S205, the object coordinates, average position offset, and center position offset are combined to obtain Z-dimensional position data. Since the object coordinates, average position offset, and center position offset are all three-dimensional coordinates, the final result is 9-dimensional position data, so Z = 9.

[0132] In S206, the number of feature dimensions D is determined based on the number of dimensions of the Z-dimensional location data and the number of dimensions of the X-dimensional target data, where D = Z + X. In an example, when the target data is 5-dimensional, D = 9 + 5 = 14.

[0133] In S207, based on the point cloud data of multiple quasi-cubic cylinders, the number N of point clouds in each quasi-cubic cylinder and the number P of cubic cylinders with a non-zero number of point clouds are determined respectively.

[0134] In S208, after obtaining D, N, and P, a cubic prism of size (D, P, N) can be determined.

[0135] To reduce computational complexity, this application also provides an alternative implementation of the target object detection method.

[0136] Figure 3 This illustration shows another flowchart of a target object detection method provided in one embodiment of this application, as follows: Figure 3 As shown, S103 may further include the following steps:

[0137] S301 uses an encoder to extract features from the point cloud data in each cubic column to obtain a tensor;

[0138] S302 extracts valid points from the points of the cubic column through pooling operations;

[0139] S303, based on the tensor, the effective points are compressed to obtain the 2D feature tensor.

[0140] Based on the above embodiments, by performing extraction and pooling operations on the cubic pillars through dimensionality increase, the points that best reflect the characteristics of the cubic pillars are easily extracted from the pillars. Then, compression is performed to obtain the feature tensor that best reflects the characteristics of the cubic pillars, thereby ensuring the accuracy of the feature tensor and thus ensuring the accuracy of subsequent calculations.

[0141] In S301, the encoder uses these stacked cubic pillars to extract features from the point cloud data (i.e., to increase the dimensionality of these points) and generate a tensor of size (C, P, N). Specifically, in one embodiment, if D = 9, the dimensionality can be increased to 24, i.e., C = 24.

[0142] In S302, max pooling is used to extract the most representative valid point of each cubic column.

[0143] In S303, this valid data is further compressed into a 2D feature tensor of size (C,P).

[0144] Based on the above embodiments, by adjusting the preset positions and / or preset connection lines of each electrical module of the vehicle chassis, a vehicle chassis that meets the requirements of electromagnetic compatibility is obtained, and the accuracy of the preset positions and / or preset connection lines is determined.

[0145] Based on the same inventive concept, this application also provides a target object detection device 400. (Specifically combined with...) Figure 4 Please provide a detailed explanation.

[0146] Figure 4 A schematic diagram of the hardware structure of the target object detection device 500 provided in an embodiment of the present invention is shown.

[0147] like Figure 4 As shown, the target object detection device 400 includes:

[0148] The acquisition module 410 acquires point cloud data of objects within the region of interest detected by the 4D millimeter-wave radar. The point cloud data includes the object's position coordinates and X-dimensional target data. The X-dimensional target data is determined by feature data, which includes the relative radial velocity data between the vehicle and the object and / or the class intensity data of the reflected wave received by the 4D millimeter-wave radar from the object, where X is a positive integer.

[0149] The allocation module 420 allocates the point cloud data to a preset plane in the form of a top view, resulting in multiple cubic pillars. The size of the cubic pillars is (D, P, N), where D is the feature dimension, which is determined according to the point cloud data, P is the number of cubic pillars with a non-zero number of point clouds, and N is the number of point clouds within each cubic pillar.

[0150] The conversion module 430 is used to perform feature conversion on the point cloud data in each cubic column to obtain a 2D feature tensor.

[0151] The construction module 440 is used to construct images from each 2D feature tensor in the cubic pillars corresponding to the 2D feature tensors to obtain a 2D pseudo-image.

[0152] The detection module 450 is used to detect objects in a 2D pseudo-image using a two-dimensional convolution model to obtain the detection results of the target object.

[0153] In the target object detection device 400 provided in this embodiment, after the acquisition module 410 acquires point cloud data containing object position coordinates and X-dimensional target data, the allocation module 420 generates a cubic prism based on the point cloud data, and the conversion module 430 performs feature conversion on the point cloud data in the process of converting the cubic prism to generate a 2D feature tensor. The construction module 440 then reconstructs the 2D feature tensor to obtain a 2D pseudo-image. The detection module 450 can then use a two-dimensional convolution model to detect objects in the 2D pseudo-image to obtain the target object detection result. Based on this, by using target data from millimeter-wave radar as additional input features and performing target object detection based on the adjusted cubic prism, the accuracy of vehicle target object detection is further improved, the safety of the autonomous driving system is enhanced, and costs are reduced.

[0154] Optionally, the allocation module 420 may include:

[0155] The allocation unit is used to allocate point cloud data to a preset plane in the form of a top view, resulting in multiple quasi-cubic columns;

[0156] The first determining unit is used to determine the point position coordinates of each point in the quasi-cubic column based on the point cloud data in the quasi-cubic column;

[0157] The calculation unit is used to calculate the average positional offset between the center point and other points in the quasi-cubic column, based on the coordinates of the point positions. The center point is the point at the center of the cubic column.

[0158] The second determining unit is used to determine the center position offset based on the object's position coordinates and center coordinates, where the center coordinates are the position coordinates of the center point;

[0159] The combination unit is used to combine the object coordinates, average position offset, and center position offset to obtain Z-dimensional position data, where Z is a positive integer;

[0160] The third determining unit is used to determine the number of feature dimensions D based on the number of dimensions of the Z-dimensional location data and the number of dimensions of the X-dimensional target data;

[0161] The fourth determining unit is used to determine the number N of point clouds in each quasi-cubic column and the number P of cubic columns with a non-zero number of point clouds, based on the point cloud data of multiple quasi-cubic columns.

[0162] The fifth determining unit is used to determine the cubic column based on D, P, and N.

[0163] Optionally, the conversion module 430 may include:

[0164] The first extraction unit is used to extract features from the point cloud data in each cubic column through the encoder to obtain a tensor;

[0165] The second extraction unit is used to extract valid points from the points of the cubic column through pooling operations;

[0166] The compression unit is used to compress the effective points of the tensor to obtain a 2D feature tensor.

[0167] The target object detection device 500 provided in this application embodiment can achieve... Figures 1 to 3 The various processes implemented in the method embodiments can achieve the same technical effect, and will not be described again here to avoid repetition.

[0168] Figure 5 A schematic diagram of the hardware structure of the target object detection device provided in an embodiment of the present invention is shown.

[0169] The target object detection device may include a processor 501 and a memory 502 storing computer program instructions.

[0170] Specifically, the processor 501 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of the present invention.

[0171] Memory 502 may include mass storage for data or instructions. For example, and not limitingly, memory 502 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 502 may include removable or non-removable (or fixed) media. Where appropriate, memory 502 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 502 is non-volatile solid-state memory.

[0172] In certain embodiments, memory 502 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, generally, memory 502 includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors 501), it is operable to perform the operations described with reference to the method according to one aspect of this application.

[0173] The processor 501 reads and executes computer program instructions stored in the memory 502 to implement any of the target object detection methods in the above embodiments.

[0174] In one example, the target object detection device may further include a communication interface 503 and a bus 504. As shown in the figure, the processor 501, memory 502, and communication interface 503 are connected via the bus 504 and communicate with each other.

[0175] The communication interface 503 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of the present invention.

[0176] Bus 504 may include hardware, software, or both. For example, and not as a limitation, bus 44 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, a Wireless Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Control Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 504 may include one or more buses 504. Although specific bus 504s are described and illustrated in embodiments of this application, this application contemplates any suitable bus 504 or interconnect.

[0177] This target object detection device can combine current target object detection methods to achieve [the desired result]. Figures 1-4 The described target object detection method and apparatus.

[0178] In addition, this application also provides a computer program product, including computer program instructions, which, when executed by processor 501, can implement the steps and corresponding content of the aforementioned method embodiments.

[0179] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0180] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0181] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0182] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0183] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for detecting a target object, characterized in that, The method includes: The point cloud data of objects within the region of interest detected by 4D millimeter-wave radar is acquired. The point cloud data includes the object's position coordinates and X-dimensional target data. The X-dimensional target data is determined based on feature data, which includes the relative radial velocity data between the vehicle and the object. X is a positive integer. The point cloud data is distributed onto a preset plane in the form of a top view to obtain multiple cubic pillars. The size of each cubic pillar is (D, P, N), where D is the number of feature dimensions, which is determined according to the type of point cloud data, P is the number of cubic pillars in which the number of point clouds is not 0, and N is the number of point clouds in each cubic pillar. The point cloud data in each of the cubic pillars is subjected to feature transformation to obtain a 2D feature tensor. Each of the 2D feature tensors is used to construct an image in the cubic pillar corresponding to the 2D feature tensor to obtain a 2D pseudo-image; The objects in the 2D pseudo-image are detected using a two-dimensional convolution model to obtain the detection results of the target objects; Acquiring the X-dimensional target data includes: determining the horizontal angle and pitch angle between the object and the vehicle based on the object's position coordinates; calculating the absolute radial velocity data between the object and the vehicle through vector decomposition based on the horizontal angle, the pitch angle, and the relative radial velocity data; decomposing the absolute radial velocity data along a first direction and a second direction to obtain first velocity component data and second velocity component data, where the first direction is the x-axis direction of the vehicle coordinate system and the second direction is the y-axis direction of the vehicle coordinate system; calculating the average velocity data of each point in the cubic column in the first direction and the second direction respectively to obtain first average velocity data and second average velocity data; determining the difference between the first velocity component data and the first average velocity data as the first target velocity difference data; determining the difference between the second velocity component data and the second average velocity data as the second target velocity difference data; and concatenating and combining the first velocity component data, the second velocity component data, the first target velocity difference data, and the second target velocity difference data to generate the X-dimensional target data.

2. The target object detection method as described in claim 1, characterized in that, The step of allocating the point cloud data to a preset plane in the form of a top view to obtain multiple cubic pillars includes: The point cloud data is distributed onto a preset plane in the form of a top view to obtain multiple quasi-cubic columns; Based on the point cloud data in the quasi-cubic column, determine the point position coordinates of each point in the quasi-cubic column; Based on the coordinates of the points, calculate the average positional offset between the center point and other points in the quasi-cubic column, where the center point is the point at the center of the quasi-cubic column. The center position offset is determined based on the object's position coordinates and center coordinates, where the center coordinates are the position coordinates of the center point. The object's position coordinates, the average position offset, and the center position offset are combined to obtain Z-dimensional position data, where Z is a positive integer; The number of feature dimensions D is determined based on the number of dimensions of the Z-dimensional position data and the number of dimensions of the X-dimensional target data; Based on the point cloud data of the plurality of quasi-cubic pillars, determine the number N of point clouds in each quasi-cubic pillar and the number P of the pillars whose point cloud count is not 0. The cubic column is determined based on D, P, and N.

3. The target object detection method as described in claim 2, characterized in that, The X-dimensional target data also includes class intensity data.

4. The target object detection method as described in claim 3, characterized in that, Obtaining X-dimensional target data also includes: Acquire multiple intensity data of the aforementioned type from the radar under different operating modes; The maximum value among the multiple class strength data is determined as the valid class strength data; The effective class intensity data and the relative radial velocity data are stitched together to obtain the X-dimensional target data.

5. The target object detection method as described in claim 3, characterized in that, Obtain X-dimensional target data, including: Acquire multiple intensity data of the aforementioned type from the radar under different operating modes; The maximum value among the multiple class strength data is determined as the valid class strength data; Obtain the distance data of the radar's range gate; Based on the magnitude of the distance data, the difference between the effective class intensity distance data and the threshold is calculated to obtain the true reflection cross section; Based on the preset relationship between the reflection cross section and the class intensity data, the real class intensity data corresponding to the real reflection cross section is determined; The real-class intensity data and the relative radial velocity data are stitched together to obtain the X-dimensional target data.

6. The target object detection method as described in claim 1, characterized in that, The step of performing feature transformation on the point cloud data in each of the cubic pillars to obtain a 2D feature tensor includes: The point cloud data in each of the cubic pillars is feature-extracted using an encoder to obtain a tensor. Valid points are extracted from the points of the cubic column through pooling operations; The 2D feature tensor is obtained by compressing the effective points based on the tensor.

7. A method for detecting a target object, characterized in that, The method includes: The point cloud data of objects within the region of interest detected by the 4D millimeter-wave radar is acquired. The point cloud data includes the object's position coordinates and X-dimensional target data. The X-dimensional target data is determined based on feature data, which includes the intensity data of the wave emitted by the object received by the 4D millimeter-wave radar. The point cloud data is distributed onto a preset plane in the form of a top view to obtain multiple cubic pillars. The size of each cubic pillar is (D, P, N), where D is the number of feature dimensions, which is determined according to the type of point cloud data, P is the number of cubic pillars in which the number of point clouds is not 0, and N is the number of point clouds in each cubic pillar. The point cloud data in each of the cubic pillars is subjected to feature transformation to obtain a 2D feature tensor. Each of the 2D feature tensors is used to construct an image in the cubic pillar corresponding to the 2D feature tensor to obtain a 2D pseudo-image; The objects in the 2D pseudo-image are detected using a two-dimensional convolution model to obtain the detection results of the target objects; Acquiring X-dimensional target data includes: acquiring multiple intensity data types of the radar under different operating modes; determining the maximum value among the multiple intensity data types as the effective intensity data type; acquiring the range data of the radar's range gate; calculating the difference between the effective intensity data type and the threshold value based on the magnitude of the range data to obtain the true reflection cross section; and determining the true intensity data type corresponding to the true reflection cross section as the X-dimensional target data based on a preset relationship between the reflection cross section and the intensity data type.

8. The target object detection method as described in claim 7, characterized in that, The step of allocating the point cloud data to a preset plane in the form of a top view to obtain multiple cubic pillars includes: The point cloud data is distributed onto a preset plane in the form of a top view to obtain multiple quasi-cubic columns; Based on the point cloud data in the quasi-cubic column, determine the point position coordinates of each point in the quasi-cubic column; Based on the coordinates of the points, calculate the average positional offset between the center point and other points in the quasi-cubic column, where the center point is the point at the center of the quasi-cubic column. The center position offset is determined based on the object's position coordinates and center coordinates, where the center coordinates are the position coordinates of the center point. The object's position coordinates, the average position offset, and the center position offset are combined to obtain Z-dimensional position data, where Z is a positive integer; The number of feature dimensions D is determined based on the number of dimensions of the Z-dimensional position data and the number of dimensions of the X-dimensional target data; Based on the point cloud data of the plurality of quasi-cubic pillars, determine the number N of point clouds in each quasi-cubic pillar and the number P of the pillars whose point cloud count is not 0. The cubic column is determined based on D, P, and N.

9. The target object detection method as described in claim 8, characterized in that, The X-dimensional target data also includes X-dimensional relative radial velocity data.

10. The target object detection method as described in claim 9, characterized in that, Obtaining the X-dimensional target data includes: Based on the object's position coordinates, determine the horizontal angle and pitch angle between the object and the vehicle; Based on the horizontal angle, the pitch angle, and the relative radial velocity data, the absolute radial velocity data of the object and the vehicle are calculated through vector decomposition. The absolute radial velocity data and the class intensity data are concatenated to obtain the X-dimensional target data.

11. The target object detection method as described in claim 7, characterized in that, The step of performing feature transformation on the point cloud data in each of the cubic pillars to obtain a 2D feature tensor includes: The point cloud data in each of the cubic pillars is feature-extracted using an encoder to obtain a tensor. Valid points are extracted from the points of the cubic column through pooling operations; The 2D feature tensor is obtained by compressing the effective points based on the tensor.

12. A target object detection device, characterized in that, The device includes: The acquisition module acquires point cloud data of objects within the region of interest detected by the 4D millimeter-wave radar. The point cloud data includes the object's position coordinates and X-dimensional target data. The X-dimensional target data is determined by feature data, which includes the relative radial velocity data between the vehicle and the object and / or the intensity data of the reflected wave received by the 4D millimeter-wave radar from the object, where X is a positive integer. The allocation module allocates the point cloud data to a preset plane in the form of a top view, resulting in multiple cubic pillars. The size of each cubic pillar is (D, P, N), where D is the feature dimension, which is determined based on the point cloud data, P is the number of cubic pillars in which the number of point clouds is not zero, and N is the number of point clouds within each cubic pillar. The conversion module is used to perform feature conversion on the point cloud data in each of the cubic pillars to obtain a 2D feature tensor. The construction module is used to construct an image from each of the 2D feature tensors in the cubic pillars corresponding to the 2D feature tensors, thereby obtaining a 2D pseudo-image. The detection module is used to detect objects in 2D pseudo-images using a two-dimensional convolutional model to obtain the detection results of the target objects; The acquisition module is used to determine the horizontal angle and pitch angle between the object and the vehicle based on the object's position coordinates; calculate the absolute radial velocity data between the object and the vehicle through vector decomposition based on the horizontal angle, the pitch angle, and the relative radial velocity data; decompose the absolute radial velocity data along a first direction and a second direction to obtain first velocity component data and second velocity component data, where the first direction is the x-axis direction of the vehicle coordinate system and the second direction is the y-axis direction of the vehicle coordinate system; calculate the average velocity data of each point in the cubic column in the first direction and the second direction respectively to obtain the first average velocity data and the second average velocity data; determine the difference between the first velocity component data and the first average velocity data as the first target velocity difference data; determine the difference between the second velocity component data and the second average velocity data as the second target velocity difference data; and concatenate and combine the first velocity component data, the second velocity component data, the first target velocity difference data, and the second target velocity difference data to generate the X-dimensional target data.

Images