Method and device for correcting Doppler information of point cloud data of millimeter wave radar

Abstract

The invention relates to a Doppler information correction method for point cloud data of a millimeter wave radar. The method comprises the following steps: calculating a vehicle speed V at a receiving moment of the point cloud data of the millimeter wave radar based on a vehicle state signal (S1); and correcting the Doppler velocity of the millimeter wave radar point cloud data based on the projection velocity of the vehicle velocity V in the Doppler direction of the millimeter wave radar point cloud data (S2). The application also relates to an apparatus for Doppler information correction of point cloud data of millimeter wave radar, a vehicle comprising said apparatus, and a computer program product. According to the method and the device, normalization processing of Doppler information of point cloud data of each frame of different millimeter-wave radars is realized, feature uniformity of the point cloud data provided for the deep neural network is ensured, and the requirement of the deep neural network for maximum extraction of speed feature information of the millimeter-wave radar point cloud data is met.

Description

Technical Field

[0001] The present application relates to the field of millimeter-wave radar, and in particular to a method for correcting Doppler information of point cloud data of millimeter-wave radar, a device for correcting Doppler information of point cloud data of millimeter-wave radar, a vehicle including a device according to the present application, and a computer program product for at least assisting in implementing the steps of the method according to the present application. Background Art

[0002] With the development of artificial intelligence technology, deep neural networks are now capable of processing millimeter-wave radar point cloud data and extracting physical information about objects in the surrounding environment. Vehicles typically have different millimeter-wave radars installed at different locations. The Doppler information generated by these different millimeter-wave radars for the same object in the point cloud data can vary, for example, when the object is overtaking a vehicle. This makes it difficult for deep neural networks to uniformly process the Doppler information from these different millimeter-wave radar point cloud data.

[0003] Therefore, there is room for improvement in the current processing method of Doppler information of millimeter wave radar point cloud data. Summary of the Invention

[0004] The purpose of the present application is to provide a method for correcting Doppler information of point cloud data of millimeter-wave radar, a device for correcting Doppler information of point cloud data of millimeter-wave radar, a vehicle including the device according to the present application, and a computer program product, so as to at least partially solve the problems in the prior art.

[0005] According to a first aspect of the present application, a method for correcting Doppler information of point cloud data of a millimeter-wave radar is provided, the method comprising:

[0006] - Step S1: Calculating the vehicle speed V at the moment of receiving the millimeter-wave radar point cloud data based on the vehicle state signal; and

[0007] - Step S2: Correcting the Doppler velocity of the millimeter-wave radar point cloud data based on the projection velocity of the vehicle velocity V in the Doppler direction of the millimeter-wave radar point cloud data.

[0008] The core concept of this application is to convert the Doppler information of the point cloud data of the millimeter-wave radars installed at various positions on the vehicle from the coordinate system of each radar to the vehicle coordinate system, and use the vehicle speed to compensate for the Doppler information of each frame of point cloud data, thereby achieving normalization of the Doppler information of each frame of point cloud data from different millimeter-wave radars, ensuring the feature uniformity of the point cloud data provided to the deep neural network, thereby meeting the deep neural network's requirement for maximizing the extraction of speed feature information from millimeter-wave radar point cloud data.

[0009] According to an optional embodiment of the present application, step S1 may include:

[0010] -Step S11: Based on the longitudinal velocity V of the vehicle x , wheelbase L and front wheel turning angle β to calculate the vehicle's lateral acceleration a y ;

[0011] -Step S12: Based on the lateral acceleration a of the vehicle y Calculate the lateral velocity V of the vehicle in the vehicle coordinate system at the moment of receiving the millimeter wave radar point cloud data y ;as well as

[0012] -Step S13: Based on the longitudinal velocity V of the vehicle x and lateral velocity V y Determine the vehicle speed V at the moment of receiving the millimeter-wave radar point cloud data.

[0013] According to another optional embodiment of the present application, the lateral acceleration a of the vehicle at each RoV information reception moment can be calculated based on the RoV information received between the first frame point cloud data and the second frame point cloud data. y , wherein the RoV information includes in particular the longitudinal speed V of the vehicle x and the front wheel steering angle β.

[0014] According to another optional embodiment of the present application, the lateral acceleration a of the vehicle can be calculated y The integration within the time interval between the first frame of point cloud data and the second frame of point cloud data is used to obtain the lateral velocity V of the vehicle in the vehicle coordinate system at the moment of receiving the second frame of point cloud data. y .

[0015] According to another optional embodiment of the present application, step S2 may include:

[0016] -Step S21: The lateral velocity V of the second frame of point cloud data and the vehicle can be calculated based on the point cloud position information of the second frame of point cloud data in the radar coordinate system and the displacement information of the radar coordinate system relative to the vehicle coordinate system. y A first angle θ1 between them;

[0017] -Step S22: The longitudinal velocity V of the vehicle may be x and lateral velocity V y Calculate the vehicle speed V and the vehicle's longitudinal speed V x a second angle θ2 between them; and

[0018] -Step S23: Calculating the projection speed V of the vehicle speed V in the Doppler direction of the second frame of point cloud data based on the vehicle speed V, the first angle θ1 and the second angle θ2 p And the Doppler velocity V of the second frame point cloud data d Using the calculated projection velocity V p Corrected to the Doppler correction velocity V of the second frame point cloud data c , its formula is, for example:

[0019] V c =|V·cos(90°-θ1+θ2)|-V d .

[0020] According to another optional embodiment of the present application, the velocity V can be corrected based on the Doppler c The velocity feature information of millimeter-wave radar point cloud data is extracted through a deep neural network, where the positive Doppler correction velocity V c Indicates that the target speed determined based on the second frame of point cloud data is greater than the vehicle speed V, and the negative Doppler correction speed V c It indicates that the target object speed determined based on the second frame of point cloud data is less than the vehicle speed V.

[0021] According to a second aspect of the present application, a device for correcting Doppler information of point cloud data of a millimeter-wave radar is provided, wherein the device may include at least one processor and a memory, wherein program instructions that can be executed by the at least one processor are stored in the memory, and when the program instructions are executed by the at least one processor, a method according to the present application is implemented.

[0022] According to another optional embodiment of the present application, the device may be integrated into a millimeter wave radar.

[0023] According to another optional embodiment of the present application, the apparatus may be configured as a domain controller.

[0024] According to a third aspect of the present application, a vehicle is provided, comprising the device according to the present application.

[0025] According to a third aspect of the present application, a computer program product, such as a computer-readable program carrier, is provided, which contains or stores computer program instructions, and when the computer program instructions are executed by a processor, at least assists in implementing the steps of the method described in the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The principles, features and advantages of the present invention will be better understood by describing the present invention in more detail below with reference to the accompanying drawings.

[0027] Figure 1 A flowchart showing a method for correcting Doppler information of point cloud data of a millimeter-wave radar according to an exemplary embodiment of the present application is provided;

[0028] Figure 2 A flowchart illustrating a method for correcting Doppler information of point cloud data of a millimeter-wave radar according to another exemplary embodiment of the present application is shown;

[0029] Figure 3 A timing diagram showing point cloud data of a millimeter wave radar according to an exemplary embodiment of the present application;

[0030] Figure 4 A vector diagram showing Doppler-corrected velocity of point cloud data of a millimeter-wave radar according to an exemplary embodiment of the present application;

[0031] Figure 5 A flowchart showing a method for correcting Doppler information of point cloud data for millimeter wave radar according to another exemplary embodiment of the present application; and

[0032] Figure 6 A schematic diagram illustrating a vehicle including an apparatus for correcting Doppler information of point cloud data of a millimeter-wave radar according to an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0033] In order to make the technical problems, technical solutions and beneficial technical effects to be solved by this application more clearly understood, this application will be further described in detail below with reference to the accompanying drawings and multiple exemplary embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit the scope of protection of this application.

[0034] Figure 1 A flowchart of a method for correcting Doppler information of point cloud data of a millimeter-wave radar according to an exemplary embodiment of the present application is shown. The following exemplary embodiments describe the method according to the present application in more detail.

[0035] like Figure 1As shown, the method may include steps S1 and S2. In step S1, the vehicle velocity V at the time of receiving the millimeter-wave radar point cloud data can be calculated based on the vehicle status signal. Millimeter-wave radars typically operate in the millimeter-wave band of 30 GHz to 300 GHz. They transmit electromagnetic waves with a wavelength of 1 mm to 10 mm into the surrounding environment and receive echoes reflected by targets in the surrounding environment. Millimeter-wave radar point cloud data can be obtained by signal processing of the transmitted and echo signals. Vehicles are typically equipped with different millimeter-wave radars at different locations. The Doppler information of the point cloud data of the same target from different millimeter-wave radars may vary. For example, when a dynamic target on the left side of the vehicle overtakes the vehicle from behind, the Doppler velocity of the dynamic target obtained by the millimeter-wave radar installed on the left rear side of the vehicle is negative, indicating that the dynamic target is approaching the vehicle, while the Doppler velocity of the dynamic target obtained by the millimeter-wave radar installed on the left front side of the vehicle is positive, indicating that the dynamic target is moving away from the vehicle. This makes it difficult for deep neural networks to uniformly process the Doppler information of the point cloud data from different millimeter-wave radars.

[0036] In the current embodiment of the present application, the Doppler information of the point cloud data of different millimeter-wave radars can be converted to the vehicle coordinate system, so that the availability of the Doppler information of the point cloud data can be independent of the dependence on a single millimeter-wave radar. To this end, it is necessary to first calculate the vehicle speed V at the moment of receiving the millimeter-wave radar point cloud data. The Doppler information of the radar point cloud data is a vector represented by a polar coordinate system, while the vehicle speed V is usually a vector represented by a Cartesian coordinate system. Since only the longitudinal speed V of the vehicle can be obtained from the vehicle's CAN bus signal, x , so we need to first calculate the vehicle's lateral velocity V y Next, combine Figure 2 The workflow diagram of the method for correcting Doppler information of point cloud data of a millimeter-wave radar according to another exemplary embodiment of the present application illustrates a calculation process of the vehicle speed V in detail.

[0037] like Figure 2 As shown, the step S1 may include steps S11 to S13. In step S11, the longitudinal speed V of the vehicle may be x , wheelbase L and front wheel turning angle β to calculate the vehicle's lateral acceleration a y , its formula is, for example:

[0038]

[0039] Where R is the turning radius of the vehicle and the wheelbase L is a predetermined fixed value. x and the front wheel steering angle β can be collected by the corresponding on-board sensors and obtained from the vehicle's CAN bus signal.

[0040] In step S12, the lateral acceleration a of the vehicle may be used to determine the vehicle's y Calculate the lateral velocity V of the vehicle in the vehicle coordinate system at the moment of receiving the millimeter wave radar point cloud data y .like Figure 3 A timing diagram of millimeter-wave radar point cloud data according to an exemplary embodiment of the present application is shown. The first frame of millimeter-wave radar point cloud data is received at time T0, and the second frame of millimeter-wave radar point cloud data is received at time T1. Both frames of point cloud data are marked with thick lines, and the reception time of the two frames of point cloud data is, for example, 50ms apart. According to the vehicle's communication architecture, a frame of RoV (Rest of Vehicle) information is received every 10ms between the two frames of point cloud data, which is marked with a thin line. The RoV information is used to describe the relationship between the millimeter-wave radar and other components of the vehicle during vehicle operation, and particularly includes the vehicle's motion state information, such as the vehicle's longitudinal velocity V. x and the front wheel turning angle β, etc., to perform motion compensation on the point cloud data of the millimeter-wave radar, thereby calculating the reception time of the vehicle at each RoV information based on the RoV information received between the first frame of point cloud data and the second frame of point cloud data - that is, Figure 3 T marked in 01 、T 02 、T 03 and T 04 ——lateral acceleration a y Next, the vehicle's lateral acceleration a can be calculated y The integral within the time interval between the first frame of point cloud data and the second frame of point cloud data received, that is, the time interval between T0 and T1, is based on the setting conditions of the vehicle coordinate system. At the initial moment, that is, the moment T0 of receiving the first frame of point cloud data, the vehicle lateral velocity is set to zero in the vehicle coordinate system. Therefore, the calculated integral is equal to the lateral velocity V of the vehicle in the vehicle coordinate system at the moment T1 of receiving the second frame of point cloud data. y To simplify the calculation, the integral can be calculated as 01 、T 01 To T 02 、T 02 To T 03 、T 03 To T 04 、T 04 The corresponding lateral acceleration a within T1 y The sum of the integrals of , for example, is calculated as:

[0041]

[0042] Among them, ayi represents the lateral acceleration at the i-th time interval between T0 and T1, where i is any integer between 1 and 5, and V xi represents the longitudinal velocity of the vehicle at the i-th time interval between T0 and T1.

[0043] In step S13, the longitudinal velocity V x and lateral velocity V y Determine the vehicle speed V at the time of receiving the millimeter wave radar point cloud data. Here, the lateral speed V of the vehicle at the time of receiving the second frame of point cloud data T1 can be y and longitudinal velocity V x like Figure 4 By performing vector synthesis as shown in the vector diagram, the vehicle speed V at the reception time T1 of the second frame point cloud data is obtained.

[0044] In step S2, the projection speed V of the vehicle speed V in the Doppler direction of the millimeter wave radar point cloud data can be used as the basis. p The Doppler velocity of the millimeter wave radar point cloud data is corrected. Here, the vehicle speed V can be projected on the Doppler direction of the millimeter wave radar point cloud data to obtain the projection speed V p , so that the projection speed V p and the Doppler velocity V of the millimeter-wave radar point cloud data d In the same direction, the projection velocity V p Doppler velocity V of millimeter wave radar point cloud data d Make a quick correction.

[0045] Next, combine Figure 5 The flowchart showing the method for correcting Doppler information of point cloud data of millimeter wave radar according to another exemplary embodiment of the present application details step S2. Figure 5 As shown, step S2 may include steps S21 to S23.

[0046] In step S21, the lateral velocity V of the second frame of point cloud data and the vehicle can be calculated based on the point cloud position information of the second frame of point cloud data in the radar coordinate system and the displacement information of the radar coordinate system relative to the vehicle coordinate system. y The first angle θ1 between them. Figure 4In the vector diagram of the Doppler corrected velocity of the millimeter wave radar point cloud data according to an exemplary embodiment of the present application, the hollow circle represents the position of the second frame of the millimeter wave radar point cloud data - its coordinates are (x0, y0), the solid circle represents the current position of the vehicle (i.e., the origin of the vehicle coordinate system), and the line between the hollow circle and the solid circle represents the displacement of the radar coordinate system relative to the vehicle coordinate system - its displacement is, for example, (Δx, Δy). The line is consistent with the lateral velocity V of the vehicle. y The angle θ1 between them is equal to the second frame point cloud data and the lateral speed V of the vehicle y The first included angle θ1 between them can be calculated by the following formula:

[0047]

[0048] In step S22, the longitudinal velocity V of the vehicle may be x and lateral velocity V y Calculate the vehicle speed V and the vehicle's longitudinal speed V x The second included angle θ2 between them can be calculated, for example, by the following formula:

[0049]

[0050] In step S23, the projection velocity V of the vehicle speed V in the Doppler direction of the second frame point cloud data can be calculated based on the vehicle speed V, the first angle θ1 and the second angle θ2. p And the Doppler velocity V of the second frame point cloud data d Using the calculated projection velocity V p Corrected to the Doppler correction velocity V of the second frame point cloud data c , its formula is, for example:

[0051] V c =V p -V d =|V·cos(90°-θ1+θ2)|-V d .

[0052] Here, since the projection velocity V p and the Doppler velocity V of the millimeter-wave radar point cloud data d They are all in the direction of the line connecting the point cloud data and the vehicle coordinate system, so the projection speed V p Doppler velocity V of millimeter wave radar point cloud data d Perform rapid corrections to eliminate the impact of vehicle speed on the Doppler information of point cloud data acquired by different millimeter-wave radars.

[0053] For each frame of point cloud data from millimeter-wave radars installed at different locations on the vehicle, including forward-facing millimeter-wave radars and millimeter-wave radars installed at the left front corner, left rear corner, right front corner, and / or right rear corner of the vehicle, the compensation algorithm can be used to convert the Doppler information of each millimeter-wave radar's point cloud data from the radar coordinate system of each millimeter-wave radar to the vehicle coordinate system, thereby eliminating the effect of the millimeter-wave radar's installation location on the Doppler information of the point cloud data and obtaining the normalized Doppler information of the millimeter-wave radar point cloud data. Based on the Doppler correction velocity V c The velocity feature information of millimeter-wave radar point cloud data can be extracted through deep neural networks, where the positive Doppler correction velocity V c Indicates that the target speed determined based on the second frame of point cloud data is greater than the vehicle speed V, and the negative Doppler correction speed V c It indicates that the target object speed determined based on the second frame of point cloud data is less than the vehicle speed V.

[0054] According to an embodiment of the present application, the Doppler information of the point cloud data of the millimeter-wave radars installed at various positions of the vehicle is converted from each radar coordinate system to the vehicle coordinate system, and the Doppler information of each frame of point cloud data is compensated using the vehicle speed, thereby achieving normalization processing of the Doppler information of each frame of point cloud data of different millimeter-wave radars, ensuring the feature uniformity of the point cloud data provided to the deep neural network, thereby meeting the requirement of the deep neural network for maximizing the extraction of speed feature information of millimeter-wave radar point cloud data.

[0055] In addition, it should be noted that the step numbers described herein do not necessarily represent a chronological order, but are merely a reference mark. The order can be changed according to specific circumstances as long as the technical purpose of this application can be achieved.

[0056] Figure 6 A schematic diagram of a vehicle including an apparatus for correcting Doppler information of point cloud data of a millimeter-wave radar according to an exemplary embodiment of the present application is shown.

[0057] like Figure 6 As shown, a vehicle 1 may include an apparatus 10 for correcting Doppler information of millimeter-wave radar point cloud data. The apparatus 10 may include at least one processor 11 and a memory 12. The memory 12 stores program instructions that can be executed by the at least one processor 11. When the program instructions are executed by the at least one processor 11, the method according to the present application is implemented.

[0058] Optionally, the device 10 may be integrated into a millimeter-wave radar, or may be configured as a domain controller.

[0059] It should be understood that, in this document, the expressions "first", "second", "third", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance, nor should they be understood as implicitly indicating the quantity of the indicated technical features.

[0060] If an embodiment includes an "and / or" relationship between a first feature and a second feature, it should be interpreted as follows: according to one embodiment, the embodiment has both the first feature and the second feature, and according to another embodiment, the embodiment has either only the first feature or only the second feature.

[0061] Although specific embodiments have been described above, these embodiments are not intended to limit the scope of the present disclosure, even when only a single embodiment is described with respect to specific features. The feature examples provided in the present disclosure are intended to be illustrative and not limiting, unless otherwise stated. In specific implementations, multiple features may be combined with each other, depending on actual needs, where technically feasible. Various substitutions, changes, and modifications may be contemplated without departing from the spirit and scope of the present application.

Claims

1. A method for correcting Doppler information of point cloud data of a millimeter-wave radar, the method comprising: Step S1: Calculating the vehicle speed V at the time of receiving the millimeter-wave radar point cloud data based on the vehicle state signal; as well as Step S2: Correcting the Doppler velocity of the millimeter-wave radar point cloud data based on the projection velocity of the vehicle velocity V in the Doppler direction of the millimeter-wave radar point cloud data.

2. The method according to claim 1, wherein The step S1 comprises: Step S11: Based on the longitudinal velocity V of the vehicle x , wheelbase L and front wheel turning angle β to calculate the vehicle's lateral acceleration a y ; Step S12: Based on the lateral acceleration a of the vehicle y Calculate the lateral velocity V of the vehicle in the vehicle coordinate system at the moment of receiving the millimeter wave radar point cloud data y ;as well as Step S13: Based on the longitudinal velocity V of the vehicle x and lateral velocity V y Determine the vehicle speed V at the moment of receiving the millimeter-wave radar point cloud data.

3. The method according to claim 2, wherein: Calculate the lateral acceleration a of the vehicle at each RoV information reception moment based on the RoV information received between the first frame point cloud data and the second frame point cloud data y , wherein the RoV information includes the longitudinal velocity V of the vehicle x and the front wheel steering angle β.

4. The method according to claim 3, wherein: Calculate the vehicle's lateral acceleration a y The integration within the time interval between the first frame of point cloud data and the second frame of point cloud data is used to obtain the lateral velocity V of the vehicle in the vehicle coordinate system at the moment of receiving the second frame of point cloud data. y .

5. The method according to any one of claims 1 to 4, wherein The step S2 comprises: Step S21: Calculate the lateral velocity V of the second frame of point cloud data and the vehicle based on the point cloud position information of the second frame of point cloud data in the radar coordinate system and the displacement information of the radar coordinate system relative to the vehicle coordinate system. y A first angle θ1 between them; Step S22: Based on the longitudinal velocity V of the vehicle x and lateral velocity V y Calculate the vehicle speed V and the vehicle's longitudinal speed V x a second angle θ2 between them; and Step S23: Calculate the projection velocity V of the vehicle speed V in the Doppler direction of the second frame point cloud data based on the vehicle speed V, the first angle θ1 and the second angle θ2 p And the Doppler velocity V of the second frame point cloud data d Using the calculated projection velocity V p Corrected to the Doppler correction velocity V of the second frame point cloud data c , its formula is, for example: V c =|V·cos(90°-θ1+θ2)|-V d 。 6. The method according to any one of claims 1 to 5, wherein Based on the Doppler-corrected velocity V c The velocity feature information of millimeter wave radar point cloud data is extracted through deep neural network, where the positive Doppler correction velocity V c Indicates that the target speed determined based on the second frame of point cloud data is greater than the vehicle speed V, and the negative Doppler correction speed V c It indicates that the target object speed determined based on the second frame of point cloud data is less than the vehicle speed V.

7. A device (10) for correcting Doppler information of point cloud data of millimeter wave radar, wherein: The device (10) includes at least one processor (11) and a memory (12), wherein program instructions that can be executed by the at least one processor (11) are stored in the memory (12), and when the program instructions are executed by the at least one processor (11), the method according to any one of claims 1 to 6 is implemented.

8. The device (10) according to claim 7, wherein The device (10) is integrated into a millimeter wave radar; and / or The device (10) is designed as a domain controller.

9. A vehicle (1) comprising a device (10) according to claim 7 or 8. 10 . A computer program product, such as a computer-readable program carrier, comprising or storing computer program instructions, which, when executed by a processor, at least assist in implementing the steps of the method according to claim 1 .

Images