Method and device based on 4D millimeter wave radar and V2X application fusion

Abstract

The invention discloses a 4D millimeter wave radar and V2X application fusion-based method and device, and the method comprises the steps: carrying out the active detection of a target object around an environment through a 4D millimeter wave radar, obtaining the distance, angle, speed and height information of the target object, and generating point cloud data; transmitting the point cloud data to a V2X device; a calculation unit of the V2X equipment analyzes the point cloud data by using a pre-training model, and identifies the type and position information of the target object; and a V2X communication unit of the V2X equipment broadcasts the identified type and position information of the target object to peripheral equipment supporting V2X communication through a V2X communication technology. According to the method, the reliability of vehicle-road cooperation application can be enhanced, and the road peering efficiency and the safe passing capacity are improved.

Description

Technical Field

[0001] The present invention relates to the fields of 4D millimeter wave radar perception technology, artificial intelligence technology and V2X communication technology, and more specifically to a method and device based on the fusion of 4D millimeter wave radar and V2X application, which is used to achieve vehicle-road collaboration and improve road efficiency and safe passage. Background Art

[0002] V2X (Vehicle to Everything) is a communication technology that connects vehicles to everything, including vehicles, infrastructure, networks, people, etc. It enables information exchange and data transmission through vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I) and vehicle-to-person (V2P). This technology aims to improve road safety, reduce traffic congestion, reduce energy consumption and improve driving experience.

[0003] Existing radars can only detect obstacles wirelessly at a long distance, but cannot transmit detection results wirelessly at a long distance; they must transmit results through Ethernet cables or RS485 buses, and must also pass through the intermediate link of MEC (Multi-Access Edge Computing) to transparently transmit signals, and then send them to the RSU device (the abbreviation of Road Side Unit, which literally means road side unit) through the Ethernet network, so that the radar detection results can be integrated and broadcasted through V2X signals (V2X (Vehicle to Everything) refers to the information exchange technology between vehicles and various devices in the surrounding environment in the field of autonomous driving). Such a system not only wastes resources and increases costs, but also increases the size of products, making it inconvenient to promote applications. Summary of the invention

[0004] In view of this, the present invention provides a method and device based on the integration of 4D millimeter-wave radar and V2X application. The present invention uses V2X communication technology to make up for the defect of short transmission distance of millimeter-wave radar perception results, and uses millimeter-wave radar all-weather high-precision active perception technology to make up for the lack of high-precision active perception capability of V2X equipment. Through the heterogeneous integration of the two, the complexity of equipment in the process of realizing vehicle-road collaboration is reduced, the reliability of vehicle-road collaboration applications is enhanced, and the efficiency of road travel and the ability of safe passage are improved.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] In a first aspect, an embodiment of the present invention provides a method based on fusion of 4D millimeter wave radar and V2X application, comprising the following steps:

[0007] S1. Use 4D millimeter-wave radar to actively detect surrounding targets, obtain the distance, angle, speed and height information of the target, and generate point cloud data;

[0008] S2. Transmitting the point cloud data to a V2X device;

[0009] S3. The computing unit of the V2X device uses a pre-trained model to analyze the point cloud data to identify the type and location information of the target object;

[0010] S4. The V2X communication unit of the V2X device broadcasts the type and location information of the identified target object to surrounding devices supporting V2X communication through V2X communication technology.

[0011] Furthermore, the step S1 comprises the following steps:

[0012] S11, using a 4D millimeter wave radar to transmit a linear frequency modulated continuous wave FMCW signal, and transmitting the signal into the environment through an antenna array;

[0013] S12, 4D millimeter wave radar receives the signal reflected by the target object and mixes it with the transmitted linear frequency modulated continuous wave FMCW signal to generate an intermediate frequency signal;

[0014] S13, performing frequency analysis on the intermediate frequency signal to extract target information, including: distance, speed, angle and height information;

[0015] S14, converting the extracted target object information into three-dimensional coordinates to generate point cloud data.

[0016] Furthermore, in step S13, the distance information is calculated by measuring the time delay or frequency offset of the intermediate frequency signal.

[0017] Furthermore, in step S13, the angle information is calculated by measuring the phase difference or beam shape between adjacent receiving channels.

[0018] Furthermore, in step S13, the speed information is calculated by measuring the Doppler frequency shift of the intermediate frequency signal.

[0019] Furthermore, in step S13, the height information is measured:

[0020] Calculate the height of the target in three-dimensional space based on angle and distance.

[0021] Furthermore, the step S3 comprises:

[0022] Use the pre-trained model to extract features from point cloud data and extract features related to the target type, including: the size, shape, texture and motion state of the target;

[0023] Match the extracted features with the target object point cloud model in the pre-trained model, and identify the type of the target object based on the matching results;

[0024] After identifying the type of the target, the location information of the target is extracted again, including the distance, angle, speed and height of the target;

[0025] Fusion calculation is performed based on the type and location information of the identified target to predict safety hazards.

[0026] In a second aspect, an embodiment of the present invention further provides a device based on the fusion of 4D millimeter wave radar and V2X application, using the method based on the fusion of 4D millimeter wave radar and V2X application as described in any one of the first aspects, the device comprising: a 4D millimeter wave radar and a V2X device connected to each other;

[0027] The V2X device is integrated onto the data processing board of the 4D millimeter wave radar through a B2B socket; the V2X device has a vehicle bus interface for communicating and interacting with other device units;

[0028] The 4D millimeter wave radar is used to actively detect surrounding targets and generate point cloud data;

[0029] The V2X device is used to analyze point cloud data and identify target objects, and broadcast the identified target object information to surrounding devices through V2X communication technology.

[0030] Furthermore, the V2X device has an OTA upgrade interface.

[0031] It can be seen from the above technical solutions that compared with the prior art, the present invention has the following technical advantages:

[0032] The present invention heterogeneously integrates the millimeter-wave radar sensing device with the V2X device. With the help of the all-weather high-precision sensing capability of the millimeter-wave radar, the sensed data is directly transmitted to the V2X device through the communication bus. With the help of the high-performance NPU (Neural network Processing Unit) processing capability of the V2X device, the model pre-trained by the radar data is loaded for target object matching, so that the information of traffic participants participating in road traffic can be quickly acquired. Then, the long-distance transmission of the sensing data can be realized through the communication frequency band in the V2X communication technology, which can realize single-point deployment and multi-point perception, thereby improving the active safety capability of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0034] Figure 1 Schematic diagram of the method based on the fusion of 4D millimeter wave radar and V2X application provided by the present invention.

[0035] Figure 2 A detailed process diagram of radar detection and V2X equipment analysis and calculation provided by the present invention.

[0036] Figure 3 A schematic diagram of a linear frequency modulation pulse signal with amplitude as a time function provided by the present invention.

[0037] Figure 4 A schematic diagram of a linear frequency modulation pulse signal with frequency as a time function provided by the present invention.

[0038] Figure 5 This is a schematic diagram of Chirp signal ranging provided by the present invention.

[0039] Figure 6 This is a schematic diagram of the intermediate frequency signal provided by the present invention.

[0040] Figure 7 A schematic diagram of the fast Fourier transform provided by the present invention.

[0041] Figure 8 This is a schematic diagram of a uniform linear array antenna provided by the present invention.

[0042] Fig. 9 This is a schematic diagram of the TDMA-MIMO signal provided by the present invention.

[0043] Fig.10 Schematic diagram of the radar matrix provided by the present invention.

[0044] Fig.11 This is a schematic diagram of the overall radar algorithm flow provided by the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] The embodiment of the present invention discloses a method based on the integration of 4D millimeter wave radar and V2X application, involving 4D millimeter wave radar and V2X equipment; wherein, the 4D millimeter wave radar is used as the front end to complete the active detection of environmental surrounding targets, and the V2X equipment is used as the back end to complete the analysis of perceived targets based on its computing unit, and identify traffic participants with different spatial positions, speeds, and directions of movement, as well as obstacles that hinder the safe operation of traffic, through a pre-trained model, and then broadcast the identified environmental targets to surrounding devices supporting V2X communication through a V2X communication unit, thereby realizing a high-precision and highly reliable vehicle-road collaborative safety scenario.

[0047] The overall processing flow is as follows Figure 1 As shown, the following steps are included:

[0048] S1. Use 4D millimeter-wave radar to actively detect surrounding targets, obtain the distance, angle, speed and height information of the target, and generate point cloud data;

[0049] S2. Transmitting the point cloud data to a V2X device;

[0050] S3. The computing unit of the V2X device uses a pre-trained model to analyze the point cloud data to identify the type and location information of the target object;

[0051] S4. The V2X communication unit of the V2X device broadcasts the type and location information of the identified target object to surrounding devices supporting V2X communication through V2X communication technology.

[0052] This method is based on the integration of 4D millimeter-wave radar and V2X application. 4D uses the all-weather high-precision perception capability of millimeter-wave radar and realizes the long-distance transmission and sharing of perception data through V2X communication technology, thereby realizing high-precision and high-reliability vehicle-road cooperative safety scenarios. And through the pre-trained model to identify the information of traffic participants, the driving safety warning and control functions are realized; this method is helpful to promote the development and application of vehicle-road cooperative technology.

[0053] The technical solution of the present invention is described in detail below. It is generally divided into: millimeter wave radar front-end radio frequency detection and V2X back-end model analysis and calculation. The detailed description is from these two parts. The detailed data processing flow is as follows: Figure 2 As shown in the figure, it mainly includes: the front-end radar actively detects the surrounding environment, the V2X equipment computing unit analyzes the targets detected by the radar, realizes target classification, and conducts safety analysis on key targets; the V2X communication unit realizes data sharing of perceived targets, and combines independent V2X equipment to realize driving safety warning and related control functions. The overall process is as follows: Figure 2 The process is explained, followed by a detailed explanation of each part.

[0054] 1. Radar front-end detailed description:

[0055] 1. Emitting radar waves

[0056] Signal generation: Millimeter wave radar generates radar waveforms through frequency modulated continuous wave (FMCW). It produces a linear frequency modulated (LFM) signal that performs frequency sweeps within a specific frequency band.

[0057] Signal Transmission: The generated frequency modulated or pulsed signal is transmitted into the environment through an antenna array, usually at a certain beam angle.

[0058] The method for generating the FMCW signal waveform is shown in the following formula:

[0059]

[0060] Among them, A: represents the amplitude of the signal and controls the strength of the signal. In actual systems, it is related to the transmit power and is usually a fixed value.

[0061] f 0 : The starting frequency of the signal (also called the carrier frequency), which is the frequency of the signal at t=0.

[0062] β: Linear frequency modulation rate (frequency change rate), in Hz / s or GHz / s, represents the rate of change of signal frequency over time, which determines the sweep range (bandwidth) and sweep period of the FMCW signal. The frequency change in one cycle is: linear frequency modulation rate × cycle time.

[0063] t: time variable, indicating the signal propagation time. In the formula, the frequency of the signal changes linearly with time t.

[0064] Describes the quadratic change in phase caused by a linear change in frequency, where is the accumulated phase change, whose first derivative is directly related to frequency.

[0065] Figure 3 This is a diagram with an amplitude of 1, an initial frequency of 1 Hz, an end frequency of 10 Hz, and a sampling rate of 1000 Hz for 1 second (it is difficult to view if the frequency is too high for diagramming, so this is just a brief illustration). The horizontal axis is Time, which represents time in seconds; the vertical axis is Amplitude, which represents amplitude; Linear Chirp Signal represents linear frequency modulation pulse signal.

[0066] Figure 4It is an image of a 77GHZ-81GHZ linear frequency modulated pulse signal (frequency as a time function); wherein, the B bandwidth is 4GHz; the horizontal axis is Time, which represents time in μs; the vertical axis is Frequency (GHZ), which represents frequency.

[0067] The phase calculation formula is:

[0068]

[0069] f start is the initial frequency, f end is the end frequency, k is the frequency modulation rate, (Signal duration T), t is time.

[0070] 2. Receive reflected signal

[0071] Reflected wave reception: When radar waves encounter target objects (such as vehicles, pedestrians, obstacles, etc.) during propagation, they will be reflected. The radar receiver receives these reflected signals through the antenna array.

[0072] Signal mixing: The received echo signal will be mixed with the transmitted signal to generate a mixed signal, which is further used to analyze the target's distance, speed and other information.

[0073] 3. Signal demodulation and frequency processing

[0074] Frequency offset calculation: By comparing the frequency difference between the transmitted signal and the received signal, the radar system can calculate the relative speed and distance of the target.

[0075] For FMCW radar, the frequency difference between the received signal and the transmitted signal (also called frequency offset) can be used to calculate the distance of the target. By measuring this frequency offset, the radar system can infer the distance information of the target.

[0076] Doppler effect calculation: Due to relative motion, the frequency of the reflected wave will shift (Doppler effect). By calculating the frequency change, the relative speed of the target can be inferred.

[0077] 4. Extraction of distance and speed

[0078] ① Distance calculation: For FMCW radar, the distance of the target can be calculated by measuring the time delay (or frequency offset) of the signal.

[0079] The 4D millimeter wave radar transmission signal is a linear frequency modulated sawtooth wave signal, that is, a Chirp signal, such as Figure 5 As shown, the ranging principle is as follows: where T C is the signal cycle time, T RF is the sweep time, TI The Chrip signal frequency is 76 GHz to 81 GHz. This embodiment uses the IWR1843 radar chip, whose ADC sampling frequency is 5 MHz, while the received signal frequency is 76 to 81 GHz. The received signal cannot be processed directly, and a mixer is required to mix the received signal and the transmitted signal. After mixing, the down-conversion is retained as the intermediate frequency signal, such as Figure 6 shown.

[0080] The intermediate frequency signal expression is:

[0081]

[0082] ω 1 and ω 2 : Indicates the angular frequency of the transmitted and received signals. For linear frequency modulated continuous wave radar, the difference between the received signal frequency and the transmitted signal frequency is usually related to the distance or speed of the target. For linear frequency modulated signals, the difference between the two is that the transmitted signal frequency changes linearly with time, while the received signal frequency will be offset by the relative speed of the target (Doppler effect) or the distance of the target (frequency shift caused by delay). Therefore, the frequency difference of the intermediate frequency signal is proportional to the distance or speed of the target.

[0083] t: time variable, representing the moment of the signal.

[0084] and Indicates the initial phase of the transmitted signal and the received signal. The phase difference between the two reflects the influence of the target distance or other environmental characteristics. The phase difference contains the distance information related to the target, which plays a key role in pulse compression or precise ranging.

[0085] IF signal x out It is a time-varying sinusoidal wave whose frequency difference and phase difference contain the target's motion and position information.

[0086] From the intermediate frequency signal diagram, we can get:

[0087]

[0088] S is the frequency sweep slope, τ is the receiving delay, and d is the target distance. The formula for measuring the target distance obtained by sorting out the previous formula is:

[0089]

[0090] c: represents the transmission speed of millimeter wave radar, that is, the speed of light;

[0091] f IF : represents the intermediate frequency signal obtained by formula (4);

[0092] S: represents the sweep slope, that is, the relationship between frequency change and time.

[0093] From formula (5), we can know that the intermediate frequency signal frequency f IF Proportional to the target distance. The intermediate frequency signal is sampled to obtain a time domain sequence, and after adding the Hanning window, a fast Fourier transform (Range Fast Fourier Transform, Range-FFT) of the distance dimension is performed, such as Figure 7 As shown:

[0094] Taking the number of sampling points N as the number of FFT points, we can get:

[0095]

[0096] f s is the sampling rate of the intermediate frequency signal;

[0097] k n ∈[0,N-1] is the sampling point number of N-point FFT;

[0098] S: represents the sweep slope, that is, the relationship between frequency change and time.

[0099] The sampling period T s The relationship between the bandwidth B is as follows:

[0100]

[0101] B=ST s (8)

[0102] Substituting formula (7) into formula (6) and formula (8) yields:

[0103]

[0104] From formula (9), we can see that when B is constant, d and k n That is, it is proportional to the distance gate, so the distance resolution can be obtained:

[0105]

[0106] The maximum measurement distance of the radar is d max Limited by the IF signal frequency:

[0107]

[0108] Among them, f IFmax is the maximum intermediate frequency signal. In Complex 2x and real sampling mode, f IFmax ≤0.9f s / 2,f s is the sampling rate of the intermediate frequency signal.

[0109] ② Angle calculation: In a radar system with multiple antenna arrays, the angle (azimuth and elevation angle) of a target can be estimated through phase difference or beamforming technology, which helps to locate the precise position of the target.

[0110] The angle measurement principle is as Figure 8 shown; after performing Range-FFT on the sampled intermediate-frequency signal, the distance information of the target is obtained. However, the specific azimuth of the target relative to the radar is still unknown from the detected radar distance value, and there is a range ambiguity problem. Further signal processing is required to obtain the azimuth angle information of the radar. In this embodiment, the phase method is used for angle measurement. For uniformly distributed antenna array elements, as Fig. 9 shown, when the echo signal is incident on the receiving antenna, the path difference ΔR between adjacent echo signals caused by the spacing of the receiving antennas can be expressed as:

[0111] ΔR = d a sinθ (12)

[0112] where d a is the array spacing and θ is the incident angle.

[0113] Since a small path difference will cause a phase change in the signals received by different arrays. And the phase difference ΔΦ between adjacent receiving channels is:

[0114]

[0115] where f 0 is the initial frequency of the chrip signal, c is the signal speed, taking the value of the speed of light, d a is the array spacing, and θ is the incident angle.

[0116] After rearrangement, we get:

[0117]

[0118] where d a is the array spacing, θ is the incident angle, f 0 is the initial frequency of the chrip signal, and ΔΦ is the phase difference derived from formula 13.

[0119] The signal transmission adopts the TMDA-MIMO mode. Three transmitting antennas are activated at different time intervals, reducing the mutual interference caused by multiple antennas working simultaneously at the same moment, thereby improving the signal-to-noise ratio and detectability of the system. At the same time, the effect of N d = N t *N r virtual antennas is achieved on the physical antenna, avoiding the high cost and design complexity brought by using a large number of physical antennas, which enables the radar system to achieve high-performance target detection at a low cost.

[0120] Performing angle-dimensional FFT (green part) on the received data of different virtual channels to obtain the radar matrix diagram, such as the radar matrix Fig.10 As shown, the two-dimensional matrix is ​​the distance-angle matrix. After Angle-FFT, the phase difference of adjacent receiving channels can be obtained, and then the azimuth information of the target can be obtained. The corresponding actual ΔΦ expression is:

[0121]

[0122] ΔΦ: The phase difference between the signals received by different virtual channels in the antenna array.

[0123] k a : FFT index, related to the antenna channel.

[0124] N d : The total number of channels of the antenna array.

[0125] Substituting formula (15) into formula (14) yields the angular resolution:

[0126]

[0127] θ res : Radar angular resolution, which indicates the minimum interval that can distinguish target angles under the physical configuration of the array.

[0128] c: speed of light.

[0129] f 0 : Initial frequency of the FM signal.

[0130] N d : The number of channels of the antenna array.

[0131] d d : The distance between adjacent antennas in the antenna array.

[0132] When the phase difference exceeds ΔΦ<π, angle ambiguity will occur. Substituting the maximum unambiguous phase difference into formula (14) yields the maximum unambiguous angle θ M expression:

[0133]

[0134] θ M : Maximum unambiguous angle range; when the phase difference ΔΦ exceeds 2π, the angle measurement will be blurred. This formula defines the upper limit range of the angle measurement.

[0135] ③Speed ​​calculation: Use the Doppler frequency shift principle to calculate the speed of the target relative to the radar.

[0136] Principle of measuring target radial velocity:

[0137] 3.1 Doppler Effect

[0138] When the target object of the radar moves relatively, the speed v of the target object will cause the frequency shift of the echo signal, which is called the Doppler shift f d .

[0139] The formula for Doppler shift is:

[0140]

[0141] v: radial velocity of the target (the velocity component along the direction of radar wave propagation, unit: m / s).

[0142] λ: The wavelength of the millimeter wave signal (unit: m), which is related to the carrier frequency f c The relationship is

[0143] c: speed of light, approximately 3*10 8 m / s.

[0144] 3.2 Linear Frequency Modulated Continuous Wave (Chirp Signal)

[0145] FMCW radar acquires the distance and speed information of the target simultaneously by sending a linear frequency modulation signal. The frequency of the Chirp signal increases or decreases linearly with time, and its frequency expression is:

[0146] f(t)=f c +k*t (19)

[0147] Among them, f c : Carrier frequency (center frequency, usually 77GHz or 79GHz);

[0148] FM slope (unit: Hz / s), which indicates the rate at which the signal frequency changes;

[0149] B: bandwidth, usually 4 GHz;

[0150] T c : Modulation period, usually tens of microseconds.

[0151] 3.3 Target echo signal

[0152] When the signal sent by the radar is reflected by the target, the echo signal received by the radar will also contain the delay time t d and Doppler frequency shift f d The reflected signal can be expressed as:

[0153]

[0154] in, The delay time of the signal going to and from the target is related to the target distance R;

[0155] c is the speed of light, the specific value is given above;

[0156] f d : Doppler shift caused by target velocity.

[0157] 3.4. Intermediate frequency signal analysis (mixing processing)

[0158] FMCW radar mixes the received signal with the transmitted signal (difference frequency) to obtain an intermediate frequency signal (beat signal), whose frequency is f b ,The intermediate frequency signal consists of two parts:

[0159] f b =f range +f d (twenty one)

[0160] Distance frequency f range : Caused by signal delay time, indicating target distance: in

[0161] Doppler frequency f d : Caused by the relative motion of the target, indicating the target speed.

[0162] 3.5. Separation of distance and speed

[0163] In order to change the target distance frequency f range With Doppler frequency f d Separation, FMCW radar usually uses positive and negative slope frequency modulation (bidirectional Chirp):

[0164] Up-chirp: frequency from low to high;

[0165] Down-chirp: frequency from high to low;

[0166] The intermediate frequencies of the echo signals of the two sweeps are:

[0167]

[0168] By calculating the sum and difference of the uplink and downlink sweep frequencies, the distance and speed of the target can be obtained respectively:

[0169] Target distance R:

[0170]

[0171] Target speed v:

[0172]

[0173] The target speed information is obtained through the above processing.

[0174] ④Height measurement

[0175] Height measurement principle: After obtaining the pitch angle (elevation) of the target, the angle can be combined with the distance (range) of the target to calculate the height of the target in three-dimensional space.

[0176] Assume that the radar is located at the origin, and the coordinates of the target object are (x, y, z), where z is the height. The height of the target can be calculated using the following formula:

[0177] z=R*sin(θ elevation ) (26)

[0178] Where R is the distance between the target and the radar;

[0179] θ elevation It is the elevation angle of the target relative to the radar.

[0180] Based on the above explanations of the principles ①, ②, ③, and ④, the overall algorithm flow of the radar front end is condensed as follows: Fig.11 shown.

[0181] 5. Object detection and recognition

[0182] Target detection: After obtaining information such as distance, speed, angle, etc., the radar system will process the data and detect multiple targets in the environment (such as other vehicles, pedestrians, obstacles, etc.). Target detection algorithms include cluster analysis, threshold setting, signal filtering, etc.

[0183] Data denoising and filtering: The signal may contain noise or interference, especially in complex environments (such as multi-target environments). Therefore, the system will apply filtering technology (such as Kalman filtering, wavelet transform, etc.) to suppress noise and retain valid target information.

[0184] 6. Data fusion and point cloud generation

[0185] Data fusion: In order to improve detection accuracy and robustness, information from multiple radar sensors (multi-point radar) or other sensors (such as lidar, cameras, etc.) is usually fused.

[0186] Point cloud generation: The distance, speed, angle and other information of the target are synthesized into coordinate points in space, which constitute the so-called "point cloud". Each point in the point cloud represents the position of a target, usually with (X, Y, Z) coordinates and other attributes (such as speed, reflection intensity, etc.).

[0187] 2. Analysis and calculation of V2X equipment back-end model

[0188] The V2X backend model analysis and calculation module is mainly composed of the V2X calculation unit and the fusion data part:

[0189] ①V2X computing unit

[0190] This part obtains the point cloud data detected by the radar front end through the bus, which includes the distance, angle, speed and height information of the target object. However, due to the heterogeneous design, the radar data processing part adopts a low-power high-performance processor containing NPU for processing. Therefore, the device does not have powerful computing capabilities like NVIDIA's graphics processor. Therefore, this embodiment adopts a cluster graphics computing unit and learns based on massive millimeter-wave radar data to generate models of most of the target radar echo point cloud data that need to be used, including: adults (including lying flat, standing, squatting in different positions), children (including lying flat, standing, squatting in different positions), small passenger cars, large buses, large trucks, small trains, bicycles, motorcycles, cones, water horses, municipal engineering vehicles and other models. In this way, not only can the computing efficiency be greatly improved, but also the required new target models can be updated according to subsequent needs.

[0191] The model identification process is as follows:

[0192] Feature extraction: The pre-trained model first performs feature extraction on the point cloud data to extract features related to the target type, such as the size, shape, texture, motion state, etc. of the target.

[0193] Model matching: Match the extracted features with the target object model in the pre-trained model, and identify the type of the target object based on the matching results.

[0194] Location information extraction: After identifying the target type, the model further extracts the target’s location information, including the target’s distance, angle, speed, and height.

[0195] Through the computing unit, the position information, angle information, and speed information of the target object of the received point cloud data are fused and calculated with the carrier information of the equipment. Its main purpose is to identify whether there are predictable safety hazards based on various information of the carrier where the equipment is located. If there are relevant hazards, the effective data will be fused. Through the communication method between the equipment and the carrier, the personnel operating the carrier can be informed more accurately and timely, thereby improving the safety of road traffic and the efficiency of road travel.

[0196] The following is a simple scenario describing the principle of calculating the security risk of the carrier where the device is located based on the V2X device data and the point cloud data detected by the 4D millimeter wave radar front end:

[0197] A. Previous collision warning description

[0198] By calculating the relative position, speed, deceleration and other parameters between the vehicle and the preceding vehicle in real time, the potential collision risk can be predicted. The key steps are as follows:

[0199] a. Obtain the carrier-related data of the device, including the speed v s , acceleration a s , location information x s ;

[0200] b. Sequentially extract the velocity v of a target object in the point cloud data provided by the radar front end f , acceleration a f , location information x f ;

[0201] c. Calculate relative distance:

[0202] d rel =x f -x s (27)

[0203] d. Calculate the collision time TTC:

[0204]

[0205] When TTC is small (e.g. TTC < 5s), the early warning system triggers a warning; when TTC = ∞, it means that no collision will occur (i.e. the speed of the device's own carrier is less than or equal to the speed of the target in front);

[0206] e. Collision deceleration calculation: In an emergency, to avoid a collision, the device's own carrier needs to brake at a speed of:

[0207]

[0208] If the current braking acceleration speed of the carrier where the device is located is as Greater than a req , (the braking acceleration is a negative number, so it is greater than), triggering early warning, etc.

[0209] f. If steps d and e do not trigger an alert, jump to step b and continue running.

[0210] B. Description of blind spot / lane change warning

[0211] The risk of collision can be determined by the distance and relative speed between the vehicle and vehicles in the blind spot or adjacent lane.

[0212] a. Obtain the carrier-related data of the device, including the speed v s , device carrier location information x s ,y s , the width and length W of the device carrier s , L s ;

[0213] b. Sequentially extract the relevant data of a target in the point cloud data provided by the radar front end, including the speed v t , device carrier location information x t ,y t , the width and length W of the device carrier t , L t ;

[0214] c. Blind spot calculation: The blind spot is a fixed area of ​​the vehicle itself, usually defined as:

[0215] Horizontal position: starting from the rear of the vehicle and extending to a certain distance behind the vehicle (usually 3 to 5 meters);

[0216] Vertical position: Extends a certain width on both sides of the vehicle (usually 1 to 2 meters);

[0217] The judgment conditions for the target vehicle to be in the blind spot are:

[0218] x s -L s ≤x t ≤x s # (30)

[0219]

[0220] Where: ΔW represents the width expansion of the blind spot, which is generally 0.5 to 1 meter.

[0221] d. Lane change collision prediction: If the vehicle plans to change lanes, the collision risk of vehicles in adjacent lanes must be considered. Judgment conditions include:

[0222] Relative distance:

[0223] d rel =|x t -x s | (32)

[0224] Relative speed:

[0225] v rel =|v s -x t | (33)

[0226] Calculate TTC for collision events:

[0227]

[0228] If the target vehicle is in the planned lane change area of ​​the device carrier (usually defined as a range extending 5 meters front and rear), and the TTC is small (for example, TTC < 5s), the warning system triggers the lane change warning;

[0229] e. If steps c and d do not trigger an alert, jump to step b and continue running.

[0230] In actual implementation, this device can have dozens of built-in safety warning scenarios. Due to limited space, we will not describe them one by one. We will only briefly describe the calculation logic based on the above two scenarios.

[0231] ② Fusion data

[0232] In the data fusion part, the target object calculated by the computing unit will hinder the safe data extraction. Through V2X wireless communication, the corresponding target object can be shared with other surrounding V2X devices through the communication frequency band to enhance traffic efficiency and enhance traffic safety.

[0233] In the present invention, the core implementation includes:

[0234] 1. Enhance the active perception capability of V2X devices (equipment) of surrounding objects. Traditional V2X devices or equipment do not have active perception capability.

[0235] 2. Use cluster computing to extract key target models, replace traditional traffic detection and identification equipment with relatively low-cost hardware components, and enhance low-carbon energy-saving effects;

[0236] 3. Significantly reduce the transmission time consumption of traditional V2X network perception targets;

[0237] 4. Data fusion and sharing: The target acquired by the millimeter-wave radar sensor at the front end of the device is directly encrypted and sent out by the V2X device at the back end of the device, enhancing information sharing capabilities.

[0238] 5. Trajectory prediction: predict the future position of the target vehicle and calculate the relative position and speed.

[0239] 6. Real-time calculation: Through the vehicle kinematic equation and dynamic update algorithm, the collision time TTC and relative distance d are calculated in real time. rel , relative speed v rel , Braking speed a rel And other parameters.

[0240] 7. Early warning decision: trigger different levels of early warning information based on calculation results and thresholds.

[0241] The method based on the integration of 4D millimeter-wave radar and V2X application provided by the present invention can not only realize all-weather high-precision perception by combining 4D millimeter-wave radar with V2X technology, but also make full use of V2X communication technology to broadcast the millimeter-wave radar perception capability in a range, thereby realizing exploration from a higher starting point.

[0242] Based on the same inventive concept, an embodiment of the present invention further provides a device based on the fusion of 4D millimeter wave radar and V2X application, using the method based on the fusion of 4D millimeter wave radar and V2X application of the above embodiment, the device includes: a 4D millimeter wave radar and a V2X device connected to each other;

[0243] The V2X module is integrated under the data processing board of the millimeter-wave radar through the B2B socket, and communicates and interacts with other equipment units through the on-board bus. This can effectively avoid interference and save space layout, achieving the goal of perfect integration in structure.

[0244] The radar antenna and V2X antenna are arranged on two different structural surfaces, which effectively avoids the possibility of crosstalk.

[0245] In terms of data flow, the radar transmits position, speed, direction, and highly structured data to V2X for calculation and analysis, and then broadcasts the fused data to other vehicles or RSUs.

[0246] 4D millimeter-wave radar, used to actively detect surrounding objects and generate point cloud data;

[0247] V2X devices are used to analyze point cloud data and identify targets, and broadcast the identified target information to surrounding devices through V2X communication technology. V2X devices have OTA upgrade interfaces, which can update corresponding recognition models and programs according to different needs to handle dynamic changes of vehicles and target objects.

[0248] The device of the present invention aims to solve the problem that V2X equipment cannot actively detect traffic participants in the surrounding environment during vehicle operation. By combining 4D millimeter-wave radar with V2X equipment, the active detection capability of V2X equipment is improved, the detection cost of traffic participants in traditional V2X communication and the time-consuming data processing flow are reduced, and the safety of vehicles and traffic participants in vehicle-road collaboration is achieved with higher accuracy and higher reliability.

[0249] The device has the following technical advantages:

[0250] Improve perception accuracy: 4D millimeter-wave radar can provide high-precision target position, speed and direction information, while V2X communication can obtain information about the surrounding environment in real time. By integrating the two, the environment around the vehicle can be perceived more accurately and the perception accuracy can be improved.

[0251] Enhanced perception capabilities: Both 4D millimeter-wave radar and V2X communication have a large perception range. By integrating the two, the perception range can be expanded, target objects can be better detected and tracked, and perception capabilities can be enhanced.

[0252] Improve the accuracy of decision-making and control: Based on more accurate environmental perception information, 4D millimeter-wave radar and V2X fusion technology can make more accurate driving decisions and control strategies, improving the safety and stability of vehicle driving.

[0253] Enhanced traffic efficiency: V2X communication can realize real-time communication between vehicles, vehicles and infrastructure, and vehicles and pedestrians. Through information sharing and collaborative decision-making, it can optimize traffic flow and improve traffic efficiency.

[0254] Reduce the risk of accidents: Through more accurate perception and smarter decision-making and control, 4D millimeter-wave radar and V2X fusion technology can reduce the risk of vehicle accidents and improve road traffic safety.

[0255] Reduce overall cost: The integration of 4D millimeter-wave radar and V2X functions can effectively reduce overall cost, size, and expand its application areas.

[0256] In summary, the fusion device of 4D millimeter-wave radar and V2X can improve perception accuracy, enhance perception capabilities, improve the accuracy of decision-making and control, enhance traffic efficiency and reduce accident risks.

[0257] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0258] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method based on the fusion of 4D millimeter wave radar and V2X application, characterized in that: The steps include: S1. Use 4D millimeter-wave radar to actively detect surrounding targets, obtain the distance, angle, speed and height information of the target, and generate point cloud data; S2. Transmitting the point cloud data to a V2X device; S3. The computing unit of the V2X device uses a pre-trained model to analyze the point cloud data to identify the type and location information of the target object; S4. The V2X communication unit of the V2X device broadcasts the type and location information of the identified target object to surrounding devices supporting V2X communication through V2X communication technology.

2. The method based on 4D millimeter wave radar and V2X application fusion according to claim 1 is characterized in that: The step S1 comprises the following steps: S11, using a 4D millimeter wave radar to transmit a linear frequency modulated continuous wave FMCW signal, and transmitting the signal into the environment through an antenna array; S12, 4D millimeter wave radar receives the signal reflected by the target object and mixes it with the transmitted linear frequency modulated continuous wave FMCW signal to generate an intermediate frequency signal; S13, performing frequency analysis on the intermediate frequency signal to extract target information, including: distance, speed, angle and height information; S14, converting the extracted target object information into three-dimensional coordinates to generate point cloud data.

3. The method based on 4D millimeter wave radar and V2X application fusion according to claim 2 is characterized in that: In step S13, the distance information is calculated by measuring the time delay or frequency offset of the intermediate frequency signal.

4. The method based on 4D millimeter wave radar and V2X application fusion according to claim 2 is characterized in that: In step S13, the angle information is obtained by measuring the phase difference or beam shape between adjacent receiving channels.

5. The method based on 4D millimeter wave radar and V2X application fusion according to claim 2 is characterized in that: In step S13, the speed information is calculated by measuring the Doppler frequency shift of the intermediate frequency signal.

6. The method based on 4D millimeter wave radar and V2X application fusion according to claim 2 is characterized in that: In step S13, the height information is measured: Calculate the height of the target in three-dimensional space based on angle and distance.

7. The method based on 4D millimeter wave radar and V2X application fusion according to claim 1 is characterized in that: The step S3 comprises: Use the pre-trained model to extract features from point cloud data and extract features related to the target type, including: the size, shape, texture and motion state of the target; Match the extracted features with the target object point cloud model in the pre-trained model, and identify the type of the target object based on the matching results; After identifying the type of the target, the location information of the target is extracted again, including the distance, angle, speed and height of the target; Fusion calculation is performed based on the type and location information of the identified target to predict safety hazards.

8. A device based on 4D millimeter wave radar and V2X application fusion, characterized in that: Using the method based on 4D millimeter wave radar and V2X application fusion as described in any one of claims 1 to 7, the device includes: a 4D millimeter wave radar and a V2X device connected to each other; The V2X device is integrated onto the data processing board of the 4D millimeter wave radar through a B2B socket; the V2X device has an on-board bus interface for communicating and interacting with other device units; The 4D millimeter wave radar is used to actively detect surrounding targets and generate point cloud data; The V2X device is used to analyze point cloud data and identify target objects, and broadcast the identified target object information to surrounding devices through V2X communication technology.

9. The device based on 4D millimeter wave radar and V2X application fusion according to claim 8, characterized in that: The V2X device has an OTA upgrade interface.

Images