Target object detection method and device

Abstract

The invention discloses a target object detection method and device, and relates to the technical field of vehicle detection, and the method comprises the steps: obtaining the point cloud data of an object in a target region of interest monitored by a 4D millimeter wave radar, the point cloud data comprises the object position coordinates of the object and X-dimensional target data, distributing the point cloud data to a preset plane in the form of a top view, a plurality of cubic columns are obtained, feature conversion is carried out on the point cloud data in each cubic column, and a 2D feature tensor is obtained; performing image construction on each 2D feature tensor in a cubic column corresponding to the 2D feature tensor to obtain a 2D pseudo image; and detecting an object in the 2D pseudo image through the two-dimensional convolution model to obtain a detection result of the target object. The relative radial speed data and / or similar intensity data in the 4D millimeter wave radar are / is used as additional input features to detect the target object, the target object detection precision of the vehicle is further improved, the safety of the automatic driving system is improved, and meanwhile the cost is reduced.

Description

Technical Field

[0001] The present application belongs to the field of vehicle detection technology, and in particular, relates to a target object detection method and device. Background Art

[0002] In order to meet the needs of intelligent driving, the performance of automotive radar has been greatly improved in recent years. As autonomous driving technology develops from the demonstration stage to the implementation stage, higher requirements are also placed on perception capabilities. Mainstream autonomous driving systems rely on the fusion of cameras and lidar for perception, and millimeter-wave radar is mainly used for active safety functions such as automatic emergency braking (AEB) and forward collision warning (FCW).

[0003] Traditional millimeter-wave radars only have information in three dimensions, namely horizontal angle, speed and distance. If traditional millimeter-wave radars are used to detect vehicles, since there is no pitch angle information, the height resolution is low and the height cannot be measured. It is difficult to determine whether the stationary object in front is on the ground or in the air, which may cause phenomena such as ghost braking.

[0004] Therefore, the new 4D millimeter-wave radar is now being used for detection. However, the 3D target detection model of the 4D millimeter-wave radar point cloud is basically trained by migrating some well-performing and mature models in the lidar. There is still a lack of special design and optimization for the characteristics of the 4D millimeter-wave radar, which makes the overall target detection accuracy of the deep learning model not high, and there is still a large gap from the detection accuracy of the lidar point cloud. Summary of the invention

[0005] The embodiments of the present application provide a method and device for detecting a target object, thereby improving the accuracy of detecting a target object.

[0006] According to a first aspect of the present application, an embodiment of the present application provides a target object detection method, which includes:

[0007] Acquire point cloud data of an object in a target area of ​​interest monitored by a 4D millimeter wave radar, the point cloud data including object position coordinates and X-dimensional target data of the object, the X-dimensional target data being determined according to feature data, the feature data including relative radial velocity data between the vehicle and the object and / or class intensity data of a reflected wave received by the 4D millimeter wave radar, where X is a positive integer;

[0008] The point cloud data is distributed to a preset plane in the form of a top view to obtain a plurality of cubic columns, and the size of the cubic columns is (D, P, N), where D is the number of feature dimensions, and the number of feature dimensions is determined according to the type of point cloud data, P is the number of cubic columns in which the number of point clouds is not 0, and N is the number of point clouds in the cubic columns;

[0009] Perform feature transformation on the point cloud data in each cubic column to obtain a 2D feature tensor;

[0010] Each 2D feature tensor is image-constructed in a cubic column corresponding to the 2D feature tensor to obtain a 2D pseudo image;

[0011] The object in the 2D pseudo image is detected through a two-dimensional convolution model to obtain the detection result of the target object.

[0012] Optionally, the point cloud data is distributed to a preset plane in the form of a top view to obtain a plurality of cubic columns, including:

[0013] The point cloud data is distributed to a preset plane in the form of a top view to obtain a plurality of quasi-cubic columns;

[0014] According to the point cloud data in the quasi-cubic column, the point position coordinates of each point in the quasi-cubic column are determined;

[0015] According to the point position coordinates, calculate the average position offset between the center point in the quasi-cubic column and other points except the center point, where the center point is the point at the center position of the cubic column;

[0016] Determine the center position offset according to the object position coordinates and the center coordinates, where the center coordinates are the position coordinates of the center point;

[0017] Combine the object coordinates, the average position offset, and the center position offset to obtain Z-dimensional position data, where Z is a positive integer;

[0018] Determine the number of feature dimensions D according to the number of dimensions of the Z-dimensional position data and the number of dimensions of the X-dimensional target data;

[0019] According to the point cloud data of the plurality of quasi-cubic columns, the number N of point clouds in each quasi-cubic column and the number P of cubic columns whose number of point clouds is not 0 are determined respectively;

[0020] Based on D, P and N, determine the cubic column.

[0021] Optionally, the X-dimensional target data is X-dimensional relative radial velocity data.

[0022] Optionally, obtaining X-dimensional target data includes:

[0023] According to the position coordinates of the object, determine the horizontal angle and the pitch angle between the object and the vehicle;

[0024] According to the horizontal angle, pitch angle and relative radial velocity data, the absolute radial velocity data of the object and the vehicle are obtained by vector decomposition calculation;

[0025] The absolute radial velocity data is determined as X-dimensional target data.

[0026] Optionally, obtaining X-dimensional target data includes:

[0027] According to the position coordinates of the object, determine the horizontal angle and the pitch angle between the object and the vehicle;

[0028] According to the horizontal angle, pitch angle and relative radial velocity data, the absolute radial velocity data of the object and the vehicle are obtained by vector decomposition calculation;

[0029] Decomposing the absolute radial velocity data along a first direction and a second direction to obtain first velocity component data and second velocity component data, the first direction being the x-axis direction of the vehicle coordinate system, and the second direction being the y-axis direction of the vehicle coordinate system;

[0030] Calculate the average velocity data of each point in the cubic column in the first direction and the second direction as the first average velocity data and the second average velocity data respectively;

[0031] determining a difference between the first velocity component data and the first velocity average data as first target velocity difference data;

[0032] Determine the difference data between the second speed component data and the second speed average data as the second target speed difference data;

[0033] The first velocity component data, the second velocity component data, the first target velocity difference data and the second target velocity difference data are concatenated and combined to generate X-dimensional target data.

[0034] Optionally, the X-dimensional target data is class intensity data;

[0035] Optionally, obtaining X-dimensional target data includes:

[0036] Obtain multiple class strength data of radar in different working modes;

[0037] determining a maximum value among the plurality of class strength data as the effective class strength data;

[0038] The effective class field intensity data is determined as X-dimensional target data.

[0039] Optionally, obtaining X-dimensional target data includes:

[0040] Obtain multiple class strength data of radar in different working modes;

[0041] determining a maximum value among the plurality of class strength data as the effective class strength data;

[0042] Get the distance data of the radar's range gate;

[0043] According to the size of the distance data, the difference between the effective class intensity distance data and the threshold is calculated to obtain the true reflection cross section;

[0044] According to the preset relationship between the reflection cross section and the class intensity data, the real class intensity data corresponding to the real reflection cross section is determined as the X-dimensional target data.

[0045] Optionally, feature transformation is performed on the point cloud data in each cubic column to obtain a 2D feature tensor, including:

[0046] The encoder is used to extract features from the point cloud data in each cubic column to obtain a tensor;

[0047] Extract valid points from the points of the cubic column through pooling operation;

[0048] According to the tensor, the valid points are compressed to obtain a 2D feature tensor.

[0049] According to a second aspect of the present application, an embodiment of the present application provides a target object detection device, the device comprising:

[0050] an acquisition module, which acquires point cloud data of an object within a target area of ​​interest monitored by a 4D millimeter wave radar, wherein the point cloud data includes object position coordinates and X-dimensional target data of the object, wherein the X-dimensional target data is determined by feature data, and the feature data includes relative radial velocity data between the vehicle and the object and / or class intensity data of a reflected wave received by the 4D millimeter wave radar from the object, wherein X is a positive integer;

[0051] An allocation module allocates the point cloud data to a preset plane in the form of a top view to obtain multiple cubic columns, and the size of the cubic columns is (D, P, N), where D is a feature dimension, which is determined according to the point cloud data, P is the number of cubic columns whose number of point clouds is not 0 among the multiple cubic columns, and N is the number of point clouds in the cubic columns;

[0052] The conversion module is used to perform feature conversion on the point cloud data in each cubic column to obtain a 2D feature tensor;

[0053] A construction module is used to construct an image of each 2D feature tensor in a cubic column corresponding to the 2D feature tensor to obtain a 2D pseudo image;

[0054] The detection module is used to detect objects in the 2D pseudo image through a two-dimensional convolution model to obtain detection results of the target objects.

[0055] According to a third aspect of the present application, a target object detection device is provided, the device comprising: a processor and a memory storing computer program instructions;

[0056] When the processor executes the computer program instructions, the target object detection method of any one of the first aspects is implemented.

[0057] According to the fourth aspect of the present application, an embodiment of the present application provides a computer storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the target object detection method of any one of the first aspects is implemented.

[0058] According to the fifth aspect of the present application, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the target object detection method of any one of the first aspects.

[0059] The target object detection method and device of the embodiment of the present application, after acquiring the point cloud data including the object position coordinates and the X-dimensional target data, generates a cubic column according to the point cloud data, and then performs feature conversion on the point cloud data in the process of the cubic column, thereby generating a 2D feature tensor, and reconstructs the image of the 2D feature tensor to obtain a 2D pseudo image; that is, the two-dimensional convolution model can be used to detect the object in the 2D pseudo image to obtain the detection result of the target object. Based on this, the target data in the millimeter wave radar is used as an additional input feature, and the target object detection is performed according to the adjusted cubic column, which further improves the accuracy of the vehicle's detection of the target object, improves the safety of the autonomous driving system, and reduces the cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0061] Figure 1 is a flow chart of a target object detection method according to an exemplary embodiment;

[0062] Figure 2 is another flow chart of a target object detection method according to an exemplary embodiment;

[0063] Figure 3 is another flow chart of a target object detection method according to an exemplary embodiment;

[0064] Figure 4 is a structural block diagram of a target object detection device according to an exemplary embodiment;

[0065] Figure 5 is a structural block diagram of a target object detection device according to an exemplary embodiment. DETAILED DESCRIPTION

[0066] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0067] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-otherwise inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0068] As described in the background technology section, traditional millimeter-wave radars only have information in three dimensions, namely horizontal angle, speed and distance. If traditional millimeter-wave radars are used to detect vehicles, since there is no pitch angle information, the height resolution is low and the height cannot be measured. It is difficult to determine whether the stationary object in front is on the ground or in the air, which may cause phenomena such as ghost braking.

[0069] In order to solve the problems of the prior art, the present application utilizes relative radial velocity data and / or class intensity data in 4D millimeter-wave radar as additional input features to detect target objects, further improve the vehicle's accuracy in detecting target objects, improve the safety of the autonomous driving system, and reduce costs.

[0070] Based on this, after acquiring the point cloud data containing the object position coordinates and X-dimensional target data, the present application generates a cubic column based on the point cloud data, and then performs feature conversion on the point cloud data in the cubic column, thereby generating a 2D feature tensor, and reconstructs the image of the 2D feature tensor to obtain a 2D pseudo image; that is, the two-dimensional convolution model can be used to detect the object in the 2D pseudo image to obtain the detection result of the target object. Based on this, the target data in the millimeter wave radar is used as an additional input feature, and the target object detection is performed based on the adjusted cubic column, which further improves the vehicle's detection accuracy of the target object, improves the safety of the autonomous driving system, and reduces costs.

[0071] Based on this, the present application provides a target object detection method and device. The target object detection method provided in the embodiment of the present application is first introduced below.

[0072] Figure 1 A schematic diagram of a process flow of a target object detection method provided by an embodiment of the present application is shown. Figure 1 As shown, it may include the following steps:

[0073] S101, acquiring point cloud data of an object in a target area of ​​interest monitored by a 4D millimeter wave radar, where the point cloud data includes object position coordinates and X-dimensional target data of the object, where the X-dimensional target data is determined according to feature data, where the feature data includes relative radial velocity data between the vehicle and the object and / or class intensity data of a reflected wave received by the 4D millimeter wave radar from the object, where X is a positive integer;

[0074] S102, allocating the point cloud data to a preset plane in the form of a top view to obtain a plurality of cubic columns, the size of the cubic columns being (D, P, N), wherein D is the number of feature dimensions, the number of feature dimensions being determined according to the type of the point cloud data, P is the number of cubic columns in which the number of point clouds is not 0 among the plurality of cubic columns, and N is the number of point clouds in the cubic columns;

[0075] S103, performing feature conversion on the point cloud data in each cubic column to obtain a 2D feature tensor;

[0076] S104, constructing an image of each 2D feature tensor in a cubic column corresponding to the 2D feature tensor to obtain a 2D pseudo image;

[0077] S105, detecting the object in the 2D pseudo image by using a two-dimensional convolution model to obtain a detection result of the target object.

[0078] Based on the above embodiment, after obtaining the point cloud data containing the object position coordinates and the X-dimensional target data, a cubic column is generated according to the point cloud data, and then the point cloud data in the cubic column is transformed to generate a 2D feature tensor, and the 2D feature tensor is reconstructed to obtain a 2D pseudo image; that is, the two-dimensional convolution model can be used to detect the object in the 2D pseudo image to obtain the detection result of the target object. Based on this, the target data in the millimeter wave radar is used as an additional input feature, and the target object is detected according to the adjusted cubic column, which further improves the accuracy of the vehicle's detection of the target object, improves the safety of the autonomous driving system, and reduces costs.

[0079] In the above S101, the 4D millimeter wave radar sends out radar waves to the target area of ​​interest for detection, then receives the reflected radar waves, and obtains point cloud data of all objects in the target area of ​​interest, wherein the objects may include dynamic objects such as obstacles and vehicles or static objects; the point cloud data may include the position coordinates of the objects and the target data; and the target data is determined based on the feature data, specifically, the feature data may include relative radial velocity data and / or class intensity data of the reflected waves received by the 4D millimeter wave radar from the objects.

[0080] When the characteristic data includes relative radial velocity data, determining the X-dimensional target data from the relative radial velocity data may include the following implementations:

[0081] In one example, the raw data output by the millimeter wave radar includes relative radial velocity data, so the feature data can be directly the relative radial velocity data, and the X-dimensional target data is directly the relative radial velocity data. In this embodiment, X is 1.

[0082] In another example, since the relative radial velocity data is the speed of the object relative to the vehicle, its value will change with the real-time change of the vehicle speed. For example, the relative radial velocity data corresponding to a stationary object is negative, and the value is unstable when the radar detects different frames. Therefore, it is considered to perform ego motion compensation. Based on this, the horizontal angle and pitch angle between the object and the vehicle can be determined according to the object position coordinates;

[0083] According to the horizontal angle, the pitch angle and the relative radial velocity data, the relative radial velocity data is calculated by vector decomposition to obtain the absolute radial velocity data of the object and the vehicle;

[0084] Therefore, the absolute radial velocity data may be used as feature data, that is, the absolute radial velocity data is X-dimensional target data. In this embodiment, X is 1.

[0085] In another example, in order to more accurately construct the cubic column and ensure the accuracy of the point cloud arrangement in the cubic column, the feature data may also be considered in combination with the cubic column.

[0086] Specifically, according to the position coordinates of the object, the horizontal angle and the pitch angle between the object and the vehicle are determined;

[0087] According to the horizontal angle, pitch angle and relative radial velocity data, the absolute radial velocity data of the object and the vehicle are obtained by vector decomposition calculation;

[0088] Decomposing the absolute radial velocity data along a first direction and a second direction to obtain first velocity component data and second velocity component data, the first direction being the x-axis direction of the vehicle coordinate system, and the second direction being the y-axis direction of the vehicle coordinate system;

[0089] Calculate the average velocity data of each point in the cubic column in the first direction and the second direction as the first average velocity data and the second average velocity data respectively;

[0090] determining a difference between the first velocity component data and the first velocity average data as first target velocity difference data;

[0091] determining a difference between the second velocity component data and the second velocity average data as second target velocity difference data;

[0092] The first velocity component data, the second velocity component data, the first target velocity difference data and the second target velocity difference data are concatenated and combined to generate X-dimensional target data.

[0093] The absolute radial velocity data of each point cloud in the cubic column can be calculated based on the known relative radial velocity data of each point; the absolute radial velocity data is decomposed based on the x and y axes of the vehicle coordinate system to obtain the first velocity component data and the second velocity component data, and then based on the cubic column, the average difference between the first velocity component data and the second velocity component data of each point and the first velocity component data and the second velocity component data of the center point is calculated as the first target velocity difference data and the second target velocity difference data. Therefore, the first velocity component data, the second velocity component data, the first target velocity difference data and the second target velocity difference data obtained after combining the cubic column can be regarded as feature data, and the feature data can be directly determined as X-dimensional target data. In this embodiment, X is 4.

[0094] When the feature data includes class intensity data, determining the X-dimensional target data by the feature velocity may include the following implementations:

[0095] In one example, the millimeter-wave radar also outputs class intensity data, and its specific value is related to the material of the object. Different objects have different materials, and their class intensity data are also different. In order to increase the accuracy of object detection, the feature data can be directly determined based on the class intensity data, and the X-dimensional target data can also be directly the class intensity data. In this embodiment, X is 1.

[0096] In another embodiment, the class intensity data is a physical quantity related to the radar cross section of the target. For a specific radar, the decibel values ​​of the two differ by a fixed value. However, for different working modes of the radar, its transmission power and antenna gain are different, so it is necessary to calibrate each working mode. The maximum value of the class intensity data is relatively stable, and different positions and angles show relatively good regular changes. Therefore, the maximum value is used to calibrate the real radar cross section value of the target.

[0097] Based on this, it is necessary to obtain multiple types of intensity data of the radar in different working modes;

[0098] determining a maximum value among the plurality of class strength data as the effective class strength data;

[0099] The effective field-like intensity data is determined as the X-dimensional target data; that is, the X-dimensional target data is the maximum value of the field-like intensity data of the radar under different working modes. In this embodiment, X is 1.

[0100] In another embodiment, the class intensity data has theoretically calibrated out the effect of distance and is only related to the radar cross-section characteristics of the target. The class intensity data values ​​measured at 10m and 15m are relatively consistent, while the class intensity data measured at 5m is smaller, mainly due to the suppression of the first few range gates and the near field of the antenna. The class field intensity data also needs to consider the actual radar cross-section.

[0101] Specifically, the method for obtaining the class field strength may include: determining the maximum value among a plurality of class strength data as the effective class strength data;

[0102] Get the distance data of the radar's range gate;

[0103] According to the size of the distance data, the difference between the effective class intensity distance data and the threshold is calculated to obtain the true reflection cross section;

[0104] According to the preset relationship between the reflection cross section and the class intensity data, the real class intensity data corresponding to the real reflection cross section is determined as the X-dimensional target data.

[0105] Specifically, the range gate refers to a specific time period in the echo signal received by the radar. It is an important parameter in the radar system and is used to determine the distance between the target object and the radar.

[0106] The true reflection cross section can be calculated by the following formula:

[0107]

[0108] The real reflection cross section of the object is selected as the characteristic data reflecting the object, and the real reflection cross section is used as the X-dimensional target data. In this embodiment, X is 1.

[0109] By introducing class field strength data, the model's ability to distinguish objects of different materials is enhanced, thereby improving the accuracy of object detection.

[0110] When the feature data includes relative radial velocity data and quasi-intensity data, the feature data in any one of the embodiments when the feature data is relative radial velocity data and the feature data in any one of the embodiments when the feature data is quasi-field intensity data are spliced ​​to form new X-dimensional target data, wherein X is the target data, i.e., it includes feature data related to the relative radial velocity data and feature data of the quasi-intensity data, thereby making full use of the different physical properties output by the millimeter-wave radar point cloud to perform richer feature learning, thereby further improving the detection accuracy.

[0111] As an example, X is the target data, which may be (first velocity component data, second velocity component data, first target velocity difference data, second target velocity difference data, real reflection cross section). In this embodiment, X is 5.

[0112] In S102, the point cloud data is first divided into regions, and the point cloud data is distributed into cubic columns of equal size in the xy plane in the form of a top view. Specifically, the size of the cubic column is (D, P, N), where D is the number of feature dimensions, which is determined according to the type of point cloud data and is related to the X value, P is the number of cubic columns in which the number of point clouds is not 0, and N is the number of point clouds in the cubic column.

[0113] In S103, the point cloud data in the cubic column is converted and compressed to generate a 2D feature tensor of size (C, P), thereby avoiding complex three-dimensional convolution calculations and simplifying the calculation generation steps.

[0114] In S104, the 2D feature tensor is encoded and placed back to the original cubic column position to generate a 2D pseudo image of size (C, H, W), where H and W can be regarded as the height and width of the feature map, P = H × W, and C is the number of channels.

[0115] In S105, after the three-dimensional point cloud data is converted into a 2D pseudo image, some backbone networks and detection heads of the two-dimensional image are applied to detect the object through a two-dimensional convolution model, that is, two-dimensional convolution is used to reduce the amount of calculation and complexity.

[0116] In order to improve the accuracy of target object detection, the present application also provides another implementation of the target object detection method.

[0117] Figure 2 Another schematic diagram of a target object detection method provided by an embodiment of the present application is shown in FIG. Figure 2 As shown, the above S102 may include the following steps:

[0118] S201, allocating the point cloud data to a preset plane in the form of a top view to obtain a plurality of quasi-cubic columns;

[0119] S202, determining the point position coordinates of each point in the quasi-cubic column according to the point cloud data in the quasi-cubic column;

[0120] S203, calculating the average position offset between the center point in the quasi-cubic column and other points except the center point according to the point position coordinates, where the center point is the point at the center position of the cubic column;

[0121] S204, determining a center position offset according to the object position coordinates and the center coordinates, where the center coordinates are the position coordinates of the center point;

[0122] S205, combining the object coordinates, the average position offset, and the center position offset to obtain Z-dimensional position data, where Z is a positive integer;

[0123] S206, determining the number D of feature dimensions according to the number of dimensions of the Z-dimensional position data and the number of dimensions of the X-dimensional target data;

[0124] S207, determining the number N of point clouds in each quasi-cubic column and the number P of cubic columns whose number of point clouds is not 0, respectively, according to the point cloud data of the plurality of quasi-cubic columns;

[0125] S208, determining the cubic column according to D, P and N.

[0126] Based on the above embodiment, by combining the object position coordinates and target data, the number of characteristic dimensions of the cubic column is determined, new characteristic data is added, and a cubic column related to the characteristic data is generated, thereby improving the accuracy of object detection.

[0127] In S201, the point cloud data is allocated to the preset plane in the form of a top view. The height of the cubic column is related to the number of feature dimensions, so it is necessary to refer to the number of feature dimensions in the point cloud data. Therefore, the cubic column at this time is only a quasi-cubic column and has not been determined.

[0128] In S202, the points in the quasi-cubic column correspond to the points in the point cloud data, but they are represented differently. After obtaining the quasi-cubic column, the coordinates of the center point of the center position of the cubic column are directly found. It should be noted that in the quasi-cubic column, each point is three-dimensional, that is, it needs to be represented by x, y, and z.

[0129] In S203, the average position offset between the center point and other points except the center point in the quasi-cubic column is calculated according to the position coordinates of each point in the cubic column, and the average position offset is a three-dimensional coordinate.

[0130] In S204, the center position offset between the object and the center point is determined according to the object position coordinates and the center coordinates of the center point. The object position coordinates are three-dimensional coordinates, and the center position offset is also three-dimensional coordinates.

[0131] In S205, the object coordinates, average position offset, and center position offset are combined to obtain Z-dimensional position data. Since the object coordinates, average position offset, and center position offset are all three-dimensional coordinates, the final result is 9-dimensional position data, and Z=9.

[0132] In S206, the number of feature dimensions D is determined according to the number of dimensions of the Z-dimensional position data and the number of dimensions of the X-dimensional target data, where D=Z+X. In one example, when the target data is 5-dimensional, D=9+5=14.

[0133] In S207, according to the point cloud data of the plurality of quasi-cubic columns, the number N of point clouds in each quasi-cubic column and the number P of cubic columns whose number of point clouds is not 0 are determined respectively;

[0134] In S208, after D, N and P are obtained, a cubic column with a size of (D, P, N) can be determined.

[0135] In order to reduce the computational difficulty, the present application also provides another implementation of the target object detection method.

[0136] Figure 3 Another schematic diagram of a target object detection method provided by an embodiment of the present application is shown in FIG. Figure 3 As shown, S103 may further include the following steps:

[0137] S301, extracting features from the point cloud data in each cubic column through an encoder to obtain a tensor;

[0138] S302, extracting valid points from the points of the cubic column through a pooling operation;

[0139] S303, according to the tensor, compress the valid points to obtain a 2D feature tensor.

[0140] Based on the above embodiment, by extracting and pooling the cubic column through dimensionality increase, it is convenient to extract the points that best reflect the characteristics of the cubic column from the cubic column, and then compress them to obtain the feature tensor that best reflects the characteristics of the cubic column, thereby ensuring the accuracy of the feature tensor and further ensuring the accuracy of subsequent calculations.

[0141] In S301, the encoder uses these stacked cubic columns to extract features from the point cloud data (i.e., these points are dimensionally upgraded) to generate a tensor of size (C, P, N). Specifically, in one embodiment, if D=9, it can be increased to 24 by dimensionally upgrading, i.e., C=24.

[0142] In S302, a maximum pooling operation is performed to extract the valid points in each cubic column that best represent the cubic column.

[0143] In S303, the effective feature tensor is compressed into a 2D feature tensor of size (C, P).

[0144] Based on the above embodiments, by adjusting the preset positions and / or preset connection lines of each electrical module of the vehicle chassis, a vehicle chassis that meets the electromagnetic compatibility requirements is obtained, and the accuracy of the preset positions and / or preset connection lines is determined.

[0145] Based on the same inventive concept, the present application also provides a target object detection device 400. Figure 4 Provide detailed explanation.

[0146] Figure 4 A schematic diagram of the hardware structure of a target object detection device 500 provided in an embodiment of the present invention is shown.

[0147] like Figure 4 As shown, the target object detection device 400 includes:

[0148] An acquisition module 410 acquires point cloud data of an object within a target region of interest monitored by a 4D millimeter wave radar, wherein the point cloud data includes object position coordinates and X-dimensional target data of the object, wherein the X-dimensional target data is determined by feature data, and the feature data includes relative radial velocity data between the vehicle and the object and / or class intensity data of a reflected wave received by the 4D millimeter wave radar from the object, wherein X is a positive integer;

[0149] The allocation module 420 allocates the point cloud data to a preset plane in the form of a top view to obtain a plurality of cubic columns, and the size of the cubic columns is (D, P, N), wherein D is a feature dimension, which is determined according to the point cloud data, P is the number of cubic columns in which the number of point clouds is not 0 among the plurality of cubic columns, and N is the number of point clouds in the cubic columns;

[0150] The conversion module 430 is used to perform feature conversion on the point cloud data in each cubic column to obtain a 2D feature tensor;

[0151] A construction module 440 is used to construct an image of each 2D feature tensor in a cubic column corresponding to the 2D feature tensor to obtain a 2D pseudo image;

[0152] The detection module 450 is used to detect the object in the 2D pseudo image through a two-dimensional convolution model to obtain a detection result of the target object.

[0153] In the target object detection device 400 provided in this embodiment, after the acquisition module 410 acquires the point cloud data including the object position coordinates and the X-dimensional target data, the allocation module 420 generates a cubic column according to the point cloud data, and the conversion module 430 performs feature conversion on the point cloud data in the cubic column to generate a 2D feature tensor, and the construction module 440 reconstructs the image of the 2D feature tensor to obtain a 2D pseudo image; the detection module 450 can use the two-dimensional convolution model to detect the object in the 2D pseudo image to obtain the detection result of the target object. Based on this, the target data in the millimeter wave radar is used as an additional input feature, and the target object detection is performed according to the adjusted cubic column, which further improves the accuracy of the vehicle's detection of the target object, improves the safety of the autonomous driving system, and reduces the cost.

[0154] Optionally, the allocation module 420 may include:

[0155] An allocating unit, used for allocating the point cloud data to a preset plane in the form of a top view to obtain a plurality of quasi-cubic columns;

[0156] A first determining unit is used to determine the point position coordinates of each point in the quasi-cubic column according to the point cloud data in the quasi-cubic column;

[0157] A calculation unit, used for calculating the average position offset between the center point in the quasi-cubic column and other points except the center point according to the point position coordinates, wherein the center point is the point at the center position of the cubic column;

[0158] A second determining unit is used to determine the center position offset according to the object position coordinates and the center coordinates, where the center coordinates are the position coordinates of the center point;

[0159] A combining unit, used for combining the object coordinates, the average position offset, and the center position offset to obtain Z-dimensional position data, where Z is a positive integer;

[0160] A third determining unit is used to determine the number D of feature dimensions according to the number of dimensions of the Z-dimensional position data and the number of dimensions of the X-dimensional target data;

[0161] A fourth determining unit, configured to determine, based on the point cloud data of the plurality of quasi-cubic columns, the number N of point clouds in each quasi-cubic column and the number P of cubic columns whose number of point clouds is not 0;

[0162] The fifth determining unit is used to determine the cubic column according to D, P and N.

[0163] Optionally, the conversion module 430 may include:

[0164] A first extraction unit is used to extract features from the point cloud data in each cubic column through an encoder to obtain a tensor;

[0165] A second extraction unit, used for extracting valid points from the points of the cubic column through a pooling operation;

[0166] The compression unit is used to compress the valid points according to the tensor to obtain a 2D feature tensor.

[0167] The target object detection device 500 provided in the embodiment of the present application can achieve Figures 1 to 3 The various processes implemented by the method embodiment can achieve the same technical effect, and to avoid repetition, they will not be described here.

[0168] Figure 5 A schematic diagram of the hardware structure of a target object detection device provided by an embodiment of the present invention is shown.

[0169] The target object detection device may include a processor 501 and a memory 502 storing computer program instructions.

[0170] Specifically, the processor 501 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0171] The memory 502 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 502 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 502 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 502 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 502 is a non-volatile solid-state memory.

[0172] In certain embodiments, the memory 502 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, typically, the memory 502 includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors 501), it is operable to perform the operations described with reference to the method according to an aspect of the present application.

[0173] The processor 501 implements any one of the target object detection methods in the above embodiments by reading and executing computer program instructions stored in the memory 502 .

[0174] In one example, the target object detection device may further include a communication interface 503 and a bus 504. As shown in the figure, the processor 501, the memory 502, and the communication interface 503 are connected via the bus 504 and communicate with each other.

[0175] The communication interface 503 is mainly used to implement the communication between the modules, devices, units and / or equipment in the embodiment of the present invention.

[0176] The bus 504 includes hardware, software or both. For example and not limitation, the bus 44 may include an accelerated graphics port (AGP) or other graphics bus, an enhanced industry standard architecture (EISA) bus, a front-side bus (FSB), a hypertransport (HT) interconnect, an industry standard architecture (ISA) bus, a wireless bandwidth interconnect, a low pin count (LPC) bus, a memory bus, a microchannel architecture (MCA) bus, a peripheral control interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standard association local (VLB) bus or other suitable bus or a combination of two or more of these. Where appropriate, the bus 504 may include one or more buses 504. Although the present application embodiment describes and shows a specific bus 504, the present application considers any suitable bus 504 or interconnection.

[0177] The target object detection device can be based on the current target object detection method to achieve the combination Figure 1-Figure 4 A method and apparatus for detecting a target object are described.

[0178] In addition, an embodiment of the present application further provides a computer program product, including computer program instructions, which can implement the steps and corresponding contents of the aforementioned method embodiment when the computer program product is executed by the processor 501.

[0179] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0180] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0181] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0182] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0183] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A target object detection method, characterized in that: The method comprises: Acquire point cloud data of an object within a target area of ​​interest monitored by a 4D millimeter wave radar, the point cloud data including object position coordinates and X-dimensional target data of the object, the X-dimensional target data being determined according to feature data, the feature data including relative radial velocity data between the vehicle and the object and / or class intensity data of a reflected wave received by the 4D millimeter wave radar from the object, where X is a positive integer; Distribute the point cloud data to a preset plane in the form of a top view to obtain a plurality of cubic columns, the size of the cubic columns being (D, P, N), wherein D is the number of feature dimensions, which is determined according to the type of the point cloud data, P is the number of the cubic columns whose number of point clouds is not 0 among the plurality of the cubic columns, and N is the number of point clouds in the cubic columns; Performing feature conversion on the point cloud data in each of the cubic columns to obtain a 2D feature tensor; Performing image construction on each of the 2D feature tensors in the cubic column corresponding to the 2D feature tensor to obtain a 2D pseudo image; The object in the 2D pseudo image is detected by using a two-dimensional convolution model to obtain a detection result of the target object.

2. The target object detection method according to claim 1, characterized in that: The step of allocating the point cloud data to a preset plane in the form of a top view to obtain a plurality of cubic columns comprises: Distributing the point cloud data to a preset plane in the form of a top view to obtain a plurality of quasi-cubic columns; Determine the point position coordinates of each point in the quasi-cubic column according to the point cloud data in the quasi-cubic column respectively; According to the point position coordinates, calculate the average position offset between the center point in the quasi-cubic column and other points except the center point, wherein the center point is the point at the center position in the quasi-cubic column; Determine the center position offset according to the object position coordinates and the center coordinates, wherein the center coordinates are the position coordinates of the center point; Combine the object position coordinates, the average position offset, and the center position offset to obtain Z-dimensional position data, wherein Z is a positive integer; Determine the number of feature dimensions D according to the number of dimensions of the Z-dimensional position data and the number of dimensions of the X-dimensional target data; According to the point cloud data of the plurality of quasi-cubic columns, respectively determining the number N of point clouds in each of the quasi-cubic columns and the number P of the cubic columns whose number of point clouds is not 0; According to the D, P and N, the cubic column is determined.

3. The target object detection method according to claim 2, characterized in that: The X-dimensional target data is X-dimensional relative radial velocity data.

4. The target object detection method according to claim 3, characterized in that: Acquiring the X-dimensional target data includes: Determine the horizontal angle and the pitch angle between the object and the vehicle according to the position coordinates of the object; Obtaining absolute radial velocity data of the object and the vehicle by vector decomposition calculation according to the horizontal angle, the pitch angle and the relative radial velocity data; The absolute radial velocity data is determined as the X-dimensional target data.

5. The target object detection method according to claim 3, characterized in that: Acquiring the X-dimensional target data includes: Determine the horizontal angle and the pitch angle between the object and the vehicle according to the position coordinates of the object; Obtaining absolute radial velocity data of the object and the vehicle by vector decomposition calculation according to the horizontal angle, the pitch angle and the relative radial velocity data; Decomposing the absolute radial velocity data along a first direction and a second direction to obtain first velocity component data and second velocity component data, wherein the first direction is an x-axis direction of a vehicle coordinate system, and the second direction is a y-axis direction of the vehicle coordinate system; Calculating the average speed data of each point in the cubic column in the first direction and the second direction as first average speed data and second average speed data respectively; determining a difference between the first speed component data and the first speed average data as first target speed difference data; determining a difference between the second speed component data and the second speed average data as second target speed difference data; The first speed component data, the second speed component data, the first target speed difference data and the second target speed difference data are concatenated and combined to generate the X-dimensional target data.

6. The target object detection method according to claim 2, characterized in that: The X-dimensional target data is class intensity data.

7. The target object detection method according to claim 6, characterized in that: Get X-dimensional target data, including: Acquire a plurality of class strength data of the radar in different working modes; determining a maximum value among the plurality of class strength data as valid class strength data; The effective class intensity data is determined as the X-dimensional target data.

8. The target object detection method according to claim 6, characterized in that: Get X-dimensional target data, including: Acquire a plurality of class strength data of the radar in different working modes; determining a maximum value among the plurality of class strength data as valid class strength data; Acquiring distance data of a range gate of the radar; According to the size of the distance data, the difference between the effective class intensity distance data and the threshold is calculated to obtain the real reflection cross section; According to a preset relationship between the reflection cross section and the class intensity data, the real class intensity data corresponding to the real reflection cross section is determined as the X-dimensional target data.

9. The target object detection method according to claim 1, characterized in that: The feature conversion is performed on the point cloud data in each of the cubic columns to obtain a 2D feature tensor, including: Extracting features from the point cloud data in each of the cubic columns through an encoder to obtain a tensor; Extracting valid points from the points of the cubic column by a pooling operation; According to the tensor, the valid points are compressed to obtain the 2D feature tensor.

10. A target object detection device, characterized in that: The device comprises: an acquisition module, which acquires point cloud data of an object within a target area of ​​interest monitored by a 4D millimeter wave radar, wherein the point cloud data includes object position coordinates of the object and X-dimensional target data, wherein the X-dimensional target data is determined by feature data, and the feature data includes relative radial velocity data between the vehicle and the object and / or class intensity data of a reflected wave received by the 4D millimeter wave radar from the object, wherein X is a positive integer; an allocation module, allocating the point cloud data to a preset plane in the form of a top view to obtain a plurality of cubic columns, the size of the cubic columns being (D, P, N), wherein D is a feature dimension, the feature dimension being determined according to the point cloud data, P is the number of the cubic columns whose number of point clouds is not 0 among the plurality of cubic columns, and N is the number of point clouds in the cubic columns; A conversion module, used to perform feature conversion on the point cloud data in each of the cubic columns to obtain a 2D feature tensor; A construction module, used for constructing an image of each of the 2D feature tensors in the cubic column corresponding to the 2D feature tensor to obtain a 2D pseudo image; The detection module is used to detect objects in the 2D pseudo image through a two-dimensional convolution model to obtain detection results of the target objects.

Images