Millimeter wave radar device based on intelligent AI learning

Abstract

The invention relates to the technical field of millimeter-wave radars, in particular to a millimeter-wave radar device based on intelligent AI learning, which comprises a multi-mode interface used for accessing weather information, vehicle speed information and scene information; the control module is used for weather classification, scene classification and millimeter wave radar wave sending parameter determination; the radio frequency front end and the antenna array are used for generating and sending wireless radio frequency signals based on the wave sending parameters; according to the millimeter-wave radar architecture, environment perception, dynamic adjustment and AI learning are deeply combined, compared with a traditional millimeter-wave radar, the detection precision in a complex environment is improved by more than 40%, and the detection confidence of the millimeter-wave radar in the complex environment is improved. And the requirements of automatic driving on sensor functions are met.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of millimeter wave radars, in particular to a millimeter wave radar device based on intelligent AI learning. BACKGROUND

[0002] The basic architecture of an existing millimeter wave radar generally comprises an antenna, a radio frequency front end, a signal processing unit, a power management, a control module, a whole vehicle interaction module and the like. The control module informs the radio frequency front end to emit millimeter waves. Frequency modulated continuous wave (FMCW) is generally adopted, that is, the frequency of the emitted signal changes linearly with time (such as sawtooth wave or triangular wave). A kind of wave emission model is conventionally configured, such as long distance, short distance or long and short distance alternation. Different wave emission models have different advantages in distance and angle resolution.

[0003] However, the current traditional millimeter wave radar and 4D millimeter wave radar can only emit waves for detection according to the set wave emission mode. Different wave emission models have their own advantages and disadvantages, and in actual application, the environment such as weather, vehicle speed and scene is variable, and the set wave emission mode cannot simultaneously consider environmental changes and cannot be adjusted to the best wave emission mode at all times. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a millimeter wave radar device based on intelligent AI learning to solve the problems in the background art.

[0005] In order to achieve the above-mentioned purpose, the present application adopts the following technical scheme:

[0006] The millimeter wave radar device based on intelligent AI learning of the present application comprises:

[0007] A multi-modal interface is used to access weather information, vehicle speed information and scene information, wherein the weather information is a weather type obtained through an Internet API interface or hardware sensor data obtained by a vehicle-mounted sensor, and the scene information is radar point cloud data obtained by a vehicle-mounted radar;

[0008] A control module is connected with the multi-modal interface, and is used to classify the weather information based on a pre-constructed weather classification model when the weather information is the hardware sensor data, to obtain a weather type; identify the scene information based on a pre-scene classification model, to obtain a scene category; and match millimeter wave radar parameters based on the weather type, the vehicle speed information or the scene category, to obtain wave emission parameters of the millimeter wave radar, wherein the wave emission parameters include one or a combination of multiple of the following: transmission power, waveform modulation mode, filtering algorithm, scanning frequency, constant false alarm detection threshold, update frequency, detection radius and target type of interest of the radar.

[0009] A radio frequency front end and an antenna array are connected to the control module and used to generate and send wireless radio frequency signals based on the wave emission parameters.

[0010] In an embodiment of the present application, access weather information, including:

[0011] When the vehicle is in online mode, send weather type request information to the API interface, and when receiving return data from the API interface, extract the weather type from the return data and cache it, wherein the weather type request information includes location information and API key;

[0012] When the vehicle is in offline mode, obtain the refresh frequency collected by the wiper sensor and the echo signal of the vehicle-mounted radar to obtain radar point cloud data.

[0013] In an embodiment of the present application, the hardware sensor data includes the refresh speed collected by the wiper sensor and the echo signal obtained by the vehicle-mounted radar, wherein the weather information is classified based on a pre-constructed weather classification model to obtain the weather type, including:

[0014] Extract the time-domain waveform features, Doppler spectrum and signal-to-noise ratio attenuation of the echo signal;

[0015] Based on the pre-constructed weather classification model, the weather is classified based on the time-domain waveform features, the Doppler spectrum and the signal-to-noise ratio attenuation to obtain the weather type and the intensity quantization value of the weather type, wherein the weather classification model is obtained by training a lightweight convolutional neural network or a Transformer neural network.

[0016] In an embodiment of the present application, the scene information is identified based on a pre-scene classification model to obtain the scene category, including:

[0017] Extract the spatial distribution features of the radar point cloud data, wherein the spatial distribution features include point cloud density and velocity distribution features;

[0018] Based on the pre-constructed scene classification model, the scene where the vehicle is located is classified based on the spatial distribution features to obtain the classification result, wherein the scene classification model is obtained by training and clustering a Gaussian mixture model.

[0019] In an embodiment of the present application, the wave emission parameters of the millimeter wave radar are matched based on the weather type to obtain the wave emission parameters of the millimeter wave radar, including:

[0020] When the weather type is rainy, foggy or snowy, the current transmission power P of the millimeter wave radar is obtained now, based on the intensity quantization value R of rainy, foggy or snowy days and the pre-built intensity-power relationship, the optimal power P is determined target , based on the current transmit power P now and the optimal power P target Determine the power increase ΔP = P target -P now , and increasing the transmit power based on the power increase ΔP to compensate for signal attenuation, wherein the intensity-power relationship is obtained based on sample data fitting, and the mathematical expression of the intensity-power relationship is:

[0021] P target =a·R b

[0022] Where a and b are empirical coefficients;

[0023] When the weather type is snowy, the frequency hopping continuous wave anti-interference mode is adopted, and the filtering algorithm of the millimeter wave radar is switched to adaptive Kalman filtering or wavelet noise reduction based on the weather type.

[0024] In one embodiment of the present application, the millimeter-wave radar parameters are matched based on the vehicle speed information to obtain the transmission parameters of the millimeter-wave radar, including:

[0025] Extract the vehicle speed v from the vehicle speed information now , and obtain the current frequency f of the millimeter wave radar now , and at the vehicle speed v now When the vehicle speed v is less than the preset first speed threshold, now and the pre-built vehicle speed frequency reference curve to determine the vehicle speed v now The corresponding reference wave frequency f target , based on the current wave frequency f now and the reference wave frequency f target Calculate the frequency increase value Δf=f target -f now , and increasing the frequency of the wave based on the frequency increase value Δf;

[0026] When the vehicle speed is greater than a preset second speed threshold, the power of the millimeter-wave radar is increased by looking up a table to extend the detection distance, wherein the first speed threshold is less than the second speed threshold.

[0027] In one embodiment of the present application, the method for constructing the vehicle speed frequency reference curve includes:

[0028] Acquire sample data obtained from the test, wherein the sample data includes a vehicle speed and a transmission frequency of a millimeter-wave radar, and the vehicle speed of the sample data is less than a preset first speed threshold;

[0029] Clustering the sample data based on the vehicle speed to obtain a plurality of sample clusters;

[0030] Calculating the average value and variance of the wave emission frequency in each sample cluster, calculating the deviation rate of the wave emission frequency of the sample data in each sample cluster from the corresponding average value, and comparing the variance with a preset variance threshold;

[0031] When the variance is greater than the variance threshold, removing the sample data with the largest deviation rate, and returning to calculating the average value and variance of the wave emission frequency in each sample cluster until the variance is less than or equal to the variance threshold; when the variance is less than or equal to the variance threshold, taking the average value in the cluster as a typical value;

[0032] Calculating the vehicle speed average value and vehicle speed standard deviation of the sample data in the sample cluster, and constructing a vehicle speed reference range conforming to three standard deviations based on the vehicle speed average value and the vehicle speed standard deviation; and mapping the vehicle speed reference range and the typical value of the wave emission frequency of the sample cluster to a two-dimensional coordinate system for fitting to obtain a vehicle speed frequency reference curve.

[0033] In an embodiment of the present application, the millimeter wave radar parameters are matched based on the scene category to obtain the wave emission parameters of the millimeter wave radar, including:

[0034] When the scene category is an urban road, the target type to which the radar pays attention is set as pedestrian / bicycle detection priority, the constant false alarm detection threshold is set as a first threshold value, the update frequency is set as a first frequency value, and the detection radius is set as a first detection range; when the scene category is a highway, the target type to which the radar pays attention is set as vehicle detection priority, the constant false alarm detection threshold is set as a second threshold value, the update frequency is set as a second frequency value, and the detection radius is set as a second detection range; when the scene category is a parking lot, the target type to which the radar pays attention is set as static obstacle detection priority, the constant false alarm detection threshold is set as a third threshold value, the update frequency is set as a third frequency value, and the detection radius is set as a third detection range.

[0035] The first threshold value is less than the second threshold value, the second threshold value is less than the third threshold value; the third frequency value is less than the first frequency value, the first frequency value is less than the second frequency value; the third detection range is within the first detection range, and the second detection range is outside the first detection range.

[0036] In an embodiment of the present application, it further includes:

[0037] Compensating the Doppler shift of the vehicle-mounted radar based on the vehicle speed information to obtain a compensated target.

[0038] In an embodiment of the present application, the Doppler shift of the vehicle-mounted radar is compensated based on the vehicle speed information, and a compensated target is obtained, comprising:

[0039] The Doppler shift caused by the vehicle movement is calculated based on the vehicle speed information, and the Doppler shift of the target relative to the vehicle is calculated;

[0040] The Doppler shift caused by the vehicle movement is eliminated from the Doppler shift of the target relative to the vehicle, and the Doppler shift of the target relative to the ground is obtained;

[0041] The target is tracked and positioned based on the Doppler shift of the target relative to the ground.

[0042] The beneficial effects of the present application are: a millimeter wave radar device based on intelligent AI learning, comprising a multi-modal interface for accessing weather information, vehicle speed information and scene information; when the weather information is the hardware sensor data, the weather information is classified based on a pre-constructed weather classification model, and the weather type is obtained; the scene information is identified based on a pre-scene classification model, and the scene category is obtained; the millimeter wave radar parameters are matched based on the weather type, the vehicle speed information and the scene category, and the control module of the millimeter wave radar wave parameter is obtained; and the radio frequency front end and the antenna array for generating and sending radio frequency signals based on the wave parameter; the millimeter wave radar architecture in the present application, which combines environmental perception, dynamic adjustment and AI learning in depth, has a detection accuracy improvement of more than 40% compared with traditional millimeter wave radar in complex environment, and improves the detection confidence of millimeter wave radar in complex environment. It meets the needs of higher automatic driving for sensor functions than current. BRIEF DESCRIPTION OF DRAWINGS

[0043] The present application will be further described below in conjunction with the drawings and embodiments:

[0044] Figure 1 is a hardware framework diagram of a millimeter wave radar device based on intelligent AI learning in an embodiment of the present application;

[0045] Figure 2 is a system framework diagram of a millimeter wave radar device based on intelligent AI learning in the present application;

[0046] Figure 3 is a flow chart of the operation method of the millimeter wave radar device based on intelligent AI learning in an embodiment of the present application. DETAILED DESCRIPTION

[0047] Following, the embodiments of the present application are illustrated by specific examples, and other advantages and effects of the present application can be easily understood by those skilled in the art from the disclosure of the specification. The present application can also be implemented or applied by other different specific embodiments, and various modifications or changes can be made to the details in the specification based on different views and applications without departing from the spirit of the present application. It should be noted that the following examples and features in the examples can be combined with each other without conflict.

[0048] It should be noted that the diagrams provided in the following examples only illustrate the basic concept of the present application in a schematic manner, and only the layers related to the present application are shown in the diagrams, not drawn according to the number, shape and size of the layers in actual implementation. The actual implementation of each layer may be a random change in pattern, number and proportion, and the layer layout pattern may also be more complex.

[0049] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application, however, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details.

[0050] Figure 1 is a hardware framework diagram of a millimeter wave radar device based on intelligent AI learning shown in an embodiment of the present application, as shown in Figure 1 A millimeter wave radar device based on intelligent AI learning of the present embodiment includes a multi-modal interface, a control module, a radio frequency front end, and an antenna array. The functions and principles of each functional module are introduced as follows:

[0051] (1) Multi-modal interface, used for accessing weather information, vehicle speed information and scene information;

[0052] Among them, the weather information is a weather type obtained through an Internet API interface or hardware sensor data obtained by a vehicle-mounted sensor, and the scene information is radar point cloud data obtained by a vehicle-mounted radar;

[0053] There are two ways to obtain weather information, online input and discrete input.

[0054] Online input: When the vehicle is in online mode, obtain weather information through Internet API, ensure that the vehicle-mounted system has 4G / 5G module or connect through mobile phone hotspot.

[0055] API selection: integrate third-party weather API (such as OpenWeatherMap, Weather.com or Gaode / Baidu map API). Use vehicle-mounted GPS to obtain real-time location, or input destination by user.

[0056] Then the weather type request information (HTTP request) is sent to the API interface, and when receiving the return data from the API interface, the weather type is extracted from the return data (for example, JSON / XML data) and cached, wherein the weather type request information includes location information and API key;

[0057] Offline input: when the vehicle is in offline mode, the refresh frequency collected by the wiper sensor and the echo signal of the vehicle-mounted radar are obtained to obtain hardware sensor data.

[0058] The vehicle speed information is obtained through the CAN (Controller Area Network) bus of the vehicle.

[0059] (2) Control module, connected with the multi-modal interface, for weather classification, scene classification and millimeter wave radar parameter determination.

[0060] (2-1) Weather classification: when the weather information is the hardware sensor data, the weather information is classified based on a pre-constructed weather classification model to obtain the weather type.

[0061] In this application, the influence mode of the attenuation and noise of the vehicle-mounted radar echo signal is different under different weather types (sunny, rainy, foggy, snowy). Based on this principle, an AI model is constructed to classify the hardware sensor data to obtain the current weather type.

[0062] The AI model in this application is constructed based on a lightweight CNN (Convolutional Neural Network) or a Transformer neural network. The artificial neural network is trained by collecting sample data to obtain a classification model.

[0063] The sample data includes the refresh frequency collected by the wiper sensor and the time domain interference features, Doppler interference features and SNR (Signal-to-Noise Ratio) attenuation features extracted from the radar echo signal. After labeling the sample data, the training data can be obtained. The artificial neural network is trained based on the training data to obtain a classification model. The following table is an example of training data in this application:

[0064] Table 1. Training data example table

[0065] Weather type Time-domain interference Doppler interference SNR degradation Rain Heavy rain clutter Spectrum broadening (±100 Hz) 0.2-0.5 dB / km Snow Basal noise uplift Low frequency interference (0-50 Hz) 0.1-1.2 dB / km Hail Sudden high-amplitude pulses Random frequency shift (velocity > 10 m / s) Local severe attenuation

[0066] In the above table, the determination of time domain interference, Doppler interference and SNR (Signal-to-Noise Ratio) attenuation is based on the determination of time domain features, Doppler spectrum and SNR data of the echo signal. The extraction of time domain features, Doppler spectrum and SNR data of the echo signal is a prior art, which will not be described here.

[0067] In addition, the intensity of the weather type can be labeled, such as light rain / heavy rain, thin fog / thick fog, etc., to train a more refined model.

[0068] After obtaining the weather classification model, the time domain waveform feature, Doppler spectrum and signal-to-noise ratio attenuation of the echo signal are extracted; then, based on the pre-constructed weather classification model, the time domain waveform feature, Doppler spectrum and signal-to-noise ratio attenuation are combined to perform weather classification, and the weather type is obtained.

[0069] (2-2) Scene classification: identifying the scene information based on a pre-scene classification model to obtain a scene category;

[0070] In this embodiment, scene recognition is performed based on the radar point cloud data returned by the millimeter wave radar, and an AI model is also used.

[0071] The scene classification model is based on a Gaussian mixture model, which is trained and clustered by collecting a large amount of sample data (the sample data contains spatial distribution features of point cloud data) to obtain the scene classification model.

[0072] The Gaussian mixture model (Gaussian Mixture Model, GMM) is a probability-based statistical model used to describe the distribution of a data set as a weighted combination of multiple Gaussian distributions (normal distributions). It assumes that data points are generated by several Gaussian distributions, each representing a "subgroup" or "cluster", and the entire model describes the overall distribution characteristics of the data through the weighted sum of multiple Gaussian distributions.

[0073] In addition, the scene classification model in this application adopts edge device deployment optimization: a large model is compressed into a TinyML version using knowledge distillation.

[0074] After obtaining the scene classification model, the spatial distribution features of the radar point cloud data are extracted, wherein the spatial distribution features include point cloud density and velocity distribution features; then, based on the pre-constructed scene classification model combined with the spatial distribution features, the scene where the vehicle is located is classified to obtain a classification result, wherein the scene classification model is obtained by training and clustering a Gaussian mixture model.

[0075] (2-3) Matching the millimeter wave radar parameters based on the weather type, the vehicle speed information or the scene category to obtain the millimeter wave radar transmission parameter, wherein the transmission parameter includes one or a combination of multiple of the transmission power, the waveform modulation mode, the filtering algorithm, the scanning frequency, the constant false alarm detection threshold, the update frequency, the detection radius and the target type of radar attention;

[0076] In this embodiment, the millimeter wave radar wave emission strategy is dynamically adjusted through the weather type, vehicle speed and scene category to improve the accuracy of the millimeter wave radar echo signal.

[0077] (2-3-1) Determining the wave emission parameter based on the weather type includes:

[0078] When the weather type is rain, fog or snow, the transmission power is increased to compensate for signal attenuation, specifically including:

[0079] When the weather type is rain, fog or snow, the current transmission power P now of the millimeter wave radar is obtained. The value of the RF transmission power register is read through the radar controller. For example, the current transmission power is 15 dBm.

[0080] Based on the intensity quantization value R of rain, fog or snow and the pre-constructed intensity-power relationship formula, the best power f target is determined. The intensity quantization value R is calibrated through an AI model. The AI model in the foregoing is used to identify the intensity when identifying the weather, for example, the intensity of rain is 3 (indicating heavy rain).

[0081] Based on the current transmission power P now and the best power P target , the power increase amount ΔP = P target -P now is determined, and the transmission power is increased based on the power increase amount ΔP to compensate for signal attenuation, wherein the intensity-power relationship formula is obtained by fitting sample data, and the mathematical expression of the intensity-power relationship formula is:

[0082] f target =a·R b

[0083] In the formula, a and b are both empirical coefficients.

[0084] In this application, the intensity-power relationship formula is fitted with the sample data prepared in advance to obtain the values of the empirical coefficients a and b, so that the best power value can be obtained by substituting the current rain intensity R into the intensity-power relationship formula. The sample data records the best working power of the millimeter wave radar under different intensities of different weather. In addition, the empirical coefficients a and b corresponding to each weather are different and need to be fitted and determined respectively.

[0085] When the weather type is snow, the frequency modulation continuous wave (FMCW) anti-interference mode is adopted.

[0086] And switch wavelet noise reduction or adaptive Kalman filtering based on weather type. For example, when the weather type is sunny, foggy or snowy, the filtering algorithm of the millimeter wave radar is switched to adaptive Kalman filtering; when the weather type is rainy, dusty or foggy, the filtering algorithm of the millimeter wave radar is switched to wavelet noise reduction.

[0087] (2-3-2) Determining the wave parameters based on vehicle speed includes:

[0088] The vehicle speed is extracted from the vehicle speed information, and when the vehicle speed is less than a preset first speed threshold, the frequency of the wave is increased; when the vehicle speed is greater than a preset second speed threshold, the detection distance is extended. In this application, the process of adaptive adjustment includes:

[0089] (2-3-2-1) Extract the vehicle speed v from the vehicle speed information now , and obtain the current frequency f of the millimeter wave radar now , and at the vehicle speed v now When the vehicle speed v is less than the preset first speed threshold, now and the pre-built vehicle speed frequency reference curve to determine the vehicle speed v now The corresponding reference wave frequency f target , based on the current wave frequency f now and the reference wave frequency f target Calculate the frequency increase value Δf=f target -f now , and increasing the frequency of the wave based on the frequency increase value Δf;

[0090] (2-3-2-2) When the vehicle speed is greater than a preset second speed threshold, the power of the millimeter-wave radar is increased by looking up a table to extend the detection distance, wherein the first speed threshold is less than the second speed threshold.

[0091] For the aforementioned domestically produced vehicles, the key to determining the corresponding transmission frequencies at different speeds is to obtain a speed-frequency reference curve based on the test sample data. This speed-frequency reference curve records the optimal transmission frequencies corresponding to different speed values ​​at lower speeds. These values ​​are obtained through experiments or manual annotation. The process of fitting the curve based on the sample data is as follows:

[0092] Obtain sample data obtained from the experiment, wherein the sample data includes the vehicle speed and the transmission frequency of the millimeter-wave radar, and the vehicle speed of the sample data is less than a preset first speed threshold; the sample data includes two columns: vehicle speed v (unit: km / h) and transmission frequency f (unit: Hz), and all samples satisfy that the vehicle speed v is less than the first speed threshold.

[0093] The sample data are clustered based on vehicle speed to obtain multiple sample clusters; based on the two-dimensional features of vehicle speed and wave frequency, the K-means algorithm is used to divide the samples into k clusters.

[0094] Calculate the mean value and variance of the wave frequency within each sample cluster, calculate the deviation rate of the wave frequency of the sample data within each sample cluster from the corresponding mean value, and compare the variance with a preset variance threshold; when the variance is greater than the variance threshold, remove the sample data with the largest deviation rate, and return to calculating the mean value and variance of the wave frequency within each sample cluster until the variance is less than or equal to the variance threshold; when the variance is less than or equal to the variance threshold, use the mean value within the cluster as the typical value; perform statistical filtering on the samples within the cluster by calculating the mean value and variance to remove outliers within the cluster and ensure the reliability of the typical value.

[0095] The average speed and standard deviation of the sample data within the sample cluster are calculated. Based on these average and standard deviations, a speed reference range corresponding to three times the standard deviation is constructed. The speed reference range and typical values ​​of the transmission frequency for this sample cluster are mapped to a two-dimensional coordinate system for fitting, resulting in a speed-frequency reference curve. Finally, a speed range defined based on three times the standard deviation covers most scenarios, and a speed-frequency reference curve is generated for dynamic frequency adjustment in real-world systems.

[0096] Based on the above curve, the scanning frequency can be adapted according to the vehicle speed signal, for example:

[0097] Low speed (<30km / h): Increase the transmission frequency (15Hz→30Hz) to enhance close-range target resolution.

[0098] High speed (>80km / h): Extended detection range (200m→300m) to reduce false alarm rate.

[0099] (2-3-3) Determining the wave parameters based on the scene category includes:

[0100] When the scene category is urban road, the target type of the radar is set to pedestrian / bicycle detection priority, the constant false alarm detection threshold is set to a first threshold value, the update frequency is set to a first frequency value, and the detection radius is set to a first detection range;

[0101] When the scene category is a highway, the target type focused on by the radar is set to vehicle detection priority, the constant false alarm detection threshold is set to a second threshold value, the update frequency is set to a second frequency value, and the detection radius is set to a second detection range;

[0102] When the scene category is a parking lot, the target type to which the radar pays attention is set to static obstacle detection priority, the constant false alarm detection threshold is set to a third threshold value, the update frequency is set to a third frequency value, and the detection radius is set to a third detection range;

[0103] The first threshold value is less than the second threshold value, the second threshold value is less than the third threshold value, the third frequency value is less than the first frequency value, the first frequency value is less than the second frequency value, the third detection range is located in the first detection range, and the second detection range is located outside the first detection range.

[0104] The threshold value, the frequency value, and the detection range can be determined by constructing a parameter template library and a scene-parameter mapping table, so that the millimeter wave radar wave emission parameters can be quickly determined according to the scene-parameter mapping table. The following table is an example of the scene-parameter mapping table in the embodiment:

[0105] Table 2. Scene-parameter mapping table

[0106]

[0107] According to scene recognition, a preset parameter template (such as adjusting the CFAR detection threshold) is automatically loaded. The constant false alarm detection threshold (CFAR Threshold) is a core technology for adaptively adjusting the detection threshold in a radar, communication, and other signal processing systems. The core goal is to maintain a constant false alarm probability (False Alarm Rate, PFA) under a dynamically changing noise or clutter background, thereby stabilizing the detection performance.

[0108] Urban road: pedestrian / bicycle detection priority, high-density point cloud processing.

[0109] Highway: long-distance vehicle tracking, low delay requirement.

[0110] Parking lot: static obstacle identification (height limit bar, road stake).

[0111] (3) The radio frequency front end and the antenna array are connected with the control module, and are used for generating and sending wireless radio frequency signals based on the wave emission parameters.

[0112] In addition, based on the vehicle speed information collected in the CAN bus, the application can also compensate the Doppler frequency shift of the millimeter wave radar, so that the millimeter wave radar is more accurate when positioning and tracking the target. Specifically, it includes:

[0113] The Doppler frequency shift caused by the movement of the vehicle is calculated based on the vehicle speed information, and the Doppler frequency shift of the target relative to the vehicle is calculated.

[0114] eliminate the Doppler shift caused by the vehicle movement from the Doppler shift of the target relative to the vehicle to obtain the Doppler shift of the target relative to the ground;

[0115] track and locate the target based on the Doppler shift of the target relative to the ground.

[0116] The core of locating and tracking the target by Doppler shift is to establish a positioning equation by combining the Doppler shift with the motion state of the observer / target, to update the target state in real time by a filtering algorithm (such as Kalman filtering), and to improve the accuracy by combining the Doppler shift with other observation parameters (such as angle, distance). The process belongs to the prior art, and will not be described here.

[0117] Figure 2 The system framework diagram of the millimeter wave radar device based on intelligent AI learning in the application is shown in FIG. 1, and the application includes the following in another embodiment from the perspective of the system framework diagram: Figure 2

[0118] The feature input layer includes a weather classification model, a vehicle speed signal input module, a scene classification model, and an adaptive parameter adjustment engine of the millimeter wave radar. The adaptive parameter adjustment engine automatically adjusts the wave emission parameters of the millimeter wave radar based on the weather type, vehicle speed information, and scene category.

[0119] The execution layer controls the millimeter wave radar array (77 / 79 GHz) of the millimeter wave radar to emit waves according to the wave emission parameters, and the parameters are matched, such as transmission power, waveform modulation, filtering algorithm, wave emission frequency, and loading of preset parameter templates.

[0120] The preprocessing layer filters and denoises the echo signal after receiving the echo of the millimeter wave radar and extracts time-frequency domain features (FFT / STFT). The millimeter wave radar uses millimeter wave electromagnetic waves, and the echo signal received by the millimeter wave radar needs to be processed by DSP to extract useful information. Time-frequency domain feature extraction is to analyze the signal from the time domain and the frequency domain to extract features, such as the speed, distance, and angle of the target. FFT is fast Fourier transform, which is used to convert time-domain signals to frequency domain. For example, the signal received by the radar is in the time domain, and the frequency components of the signal can be obtained by FFT, which is used to measure the speed in the radar because the Doppler effect will cause the frequency to change. STFT, short-time Fourier transform. It is to process the signal in windows, and do FFT in each window, so that the frequency information changing with time can be obtained. In this way, the signal can be analyzed in time and frequency, which is more suitable for processing dynamic changing targets, such as signals when a vehicle accelerates or decelerates.

[0121] ​A post-processing layer for algorithms related to star target detection and tracking based on extracted signal features (DBSCAN / Kalman filtering), multi-target association, and trajectory prediction.

[0122] An output execution layer for inputting the results obtained by the post-processing layer to the decision system of the vehicle, such as the early warning system, through the vehicle control interface (CAN / FlexRay).

[0123] Figure 3 For the running method flowchart of the millimeter wave radar device based on intelligent AI learning in an embodiment of the present application, as shown in Figure 3 The running method of the millimeter wave radar device based on intelligent AI learning in the present application includes:

[0124] S1, access weather information, vehicle speed information, and scene information, wherein the weather information is a weather type obtained through an Internet API interface or hardware sensor data obtained by a vehicle-mounted sensor, and the scene information is radar point cloud data obtained by a vehicle-mounted radar;

[0125] S2, when the weather information is the hardware sensor data, classifying the weather information based on a pre-constructed weather classification model to obtain a weather type; identifying the scene information based on a pre-scene classification model to obtain a scene category; and matching millimeter wave radar parameters based on the weather type, the vehicle speed information, and the scene category to obtain millimeter wave radar transmission parameters, wherein the transmission parameters include one or a combination of transmission power, waveform modulation mode, filtering algorithm, scanning frequency, constant false alarm detection threshold, update frequency, detection radius, and target type of interest of the radar;

[0126] S3, generating and sending a wireless radio frequency signal based on the transmission parameters.

[0127] The hardware architecture and system architecture in the present application are both based on the above-mentioned method logic. In other embodiments of the present application, other hardware architectures or system architectures can also be constructed, as long as they comply with the above-mentioned method logic, which should be considered as covered by the claims of the present application.

[0128] The application discloses a millimeter wave radar device based on intelligent AI learning, which comprises a multi-modal interface for accessing weather information, vehicle speed information and scene information; a control module for matching millimeter wave radar parameters based on the weather type, the vehicle speed information and the scene category, wherein when the weather information is the hardware sensor data, the weather information is classified based on a pre-constructed weather classification model to obtain a weather type; the scene information is identified based on a pre-scene classification model to obtain a scene category; and the millimeter wave radar parameter is matched based on the weather type, the vehicle speed information and the scene category; a radio frequency front end and an antenna array for generating and sending a wireless radio frequency signal based on the wave emission parameter; the millimeter wave radar architecture in the application combines environment perception, dynamic adjustment and AI learning in depth, and compared with a traditional millimeter wave radar, the detection precision in a complex environment is improved by more than 40%, and the detection confidence of the millimeter wave radar in the complex environment is improved; and the application meets the demand of a higher automatic driving sensor function than the current.

[0129] In the above-described embodiments, although the application has been described in conjunction with specific embodiments thereof, a number of alternatives, modifications and variations will be apparent to those skilled in the art in the light of the foregoing description. The embodiments of the application are intended to embrace all such alternatives, modifications and variations as fall within the scope of the appended claims.

[0130] The above-described embodiments are merely illustrative of the principles of the application and its efficacy, and are not intended to limit the application. Any modification or change which falls within the spirit and scope of the application should be considered as equivalent to the application. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and technical concept of the application should be covered by the claims of the application.

Claims

1. A millimeter wave radar device based on intelligent AI learning, characterized in that: include: A multimodal interface for accessing weather information, vehicle speed information, and scene information. The weather information is weather type acquired through an Internet API or hardware sensor data acquired by on-board sensors. The scene information is radar point cloud data acquired by on-board radar. a control module connected to the multimodal interface, configured to, when the weather information is the hardware sensor data, classify the weather information based on a pre-built weather classification model to obtain a weather type; identify the scene information based on a pre-built scene classification model to obtain a scene category; and match millimeter-wave radar parameters based on the weather type, the vehicle speed information, or the scene category to obtain transmission parameters of the millimeter-wave radar, wherein the transmission parameters include one or a combination of transmit power, waveform modulation mode, filtering algorithm, scanning frequency, constant false alarm detection threshold, update frequency, detection radius, and target type of radar interest; The radio frequency front end and the antenna array are connected to the control module and are used to generate and send wireless radio frequency signals based on the wave transmission parameters.

2. The millimeter wave radar device based on intelligent AI learning according to claim 1, characterized in that: Access weather information, including: When the vehicle is in online mode, sending weather type request information to the API interface, and upon receiving return data from the API interface, extracting the weather type from the return data and caching it, wherein the weather type request information includes location information and an API key; When the vehicle is in offline mode, the refresh rate collected by the wiper sensor and the echo signal of the vehicle-mounted radar are obtained to obtain radar point cloud data.

3. The millimeter wave radar device based on intelligent AI learning according to claim 1, characterized in that: The hardware sensor data includes the refresh rate collected by the wiper sensor and the echo signal obtained by the vehicle-mounted radar. The weather information is classified based on a pre-built weather classification model to obtain the weather type, including: Extracting the time domain waveform characteristics, Doppler spectrum and signal-to-noise ratio attenuation of the echo signal; Weather classification is performed based on a pre-built weather classification model in combination with the time domain waveform characteristics, the Doppler spectrum, and the signal-to-noise ratio attenuation to obtain a weather type and a quantized intensity value of the weather type, wherein the weather classification model is obtained by training a lightweight convolutional neural network or a Transformer neural network.

4. The millimeter wave radar device based on intelligent AI learning according to claim 1, characterized in that: The scene information is identified based on a pre-set scene classification model to obtain a scene category, including: Extracting spatial distribution features of the radar point cloud data, wherein the spatial distribution features include point cloud density and velocity distribution features; The scene in which the vehicle is located is classified based on a pre-built scene classification model combined with the spatial distribution characteristics to obtain a classification result, wherein the scene classification model is obtained by training and clustering a Gaussian mixture model.

5. The millimeter wave radar device based on intelligent AI learning according to claim 1, characterized in that: Matching millimeter-wave radar parameters based on the weather type to obtain millimeter-wave radar transmission parameters includes: When the weather type is rainy, foggy or snowy, obtain the current transmit power P of the millimeter wave radar now , based on the intensity quantization value R of rainy, foggy or snowy days and the pre-built intensity-power relationship, the optimal power P is determined target , based on the current transmit power P now and the optimal power P target Determine the power increase ΔP = P target -P now , and increasing the transmit power based on the power increase ΔP to compensate for signal attenuation, wherein the intensity-power relationship is obtained based on sample data fitting, and the mathematical expression of the intensity-power relationship is: P target =a·R b Where a and b are empirical coefficients; When the weather type is snowy, the frequency hopping continuous wave anti-interference mode is adopted, and the filtering algorithm of the millimeter wave radar is switched to adaptive Kalman filtering or wavelet noise reduction based on the weather type.

6. The millimeter wave radar device based on intelligent AI learning according to claim 1, characterized in that: Matching millimeter-wave radar parameters based on the vehicle speed information to obtain millimeter-wave radar transmission parameters includes: Extract the vehicle speed v from the vehicle speed information now , and obtain the current frequency f of the millimeter wave radar now , and at the vehicle speed v now When the vehicle speed v is less than the preset first speed threshold, now and the pre-built vehicle speed frequency reference curve to determine the vehicle speed v now The corresponding reference wave frequency f target , based on the current wave frequency f now and the reference wave frequency P target Calculate the frequency increase value Δf=f target -f now , and increasing the frequency of the wave based on the frequency increase value Δf; When the vehicle speed is greater than a preset second speed threshold, the power of the millimeter-wave radar is increased by looking up a table to extend the detection distance, wherein the first speed threshold is less than the second speed threshold.

7. The millimeter wave radar device based on intelligent AI learning according to claim 6, characterized in that: The method for constructing the vehicle speed frequency reference curve includes: Acquire sample data obtained from the test, wherein the sample data includes a vehicle speed and a transmission frequency of a millimeter-wave radar, and the vehicle speed of the sample data is less than a preset first speed threshold; Clustering the sample data based on vehicle speed to obtain multiple sample clusters; Calculating the average and variance of the wave frequency in each sample cluster, calculating the deviation rate between the wave frequency of the sample data in each sample cluster and the corresponding average value, and comparing the variance with a preset variance threshold; When the variance is greater than the variance threshold, remove the sample data with the largest deviation rate, and return to calculating the average value and variance of the wave frequency in each sample cluster until the variance is less than or equal to the variance threshold; when the variance is less than or equal to the variance threshold, take the average value in the cluster as the typical value; The vehicle speed average and vehicle speed standard deviation of the sample data in the sample cluster are calculated, and a vehicle speed reference range that conforms to three times the standard deviation is constructed based on the vehicle speed average and vehicle speed standard deviation; and the vehicle speed reference range of the sample cluster and the typical value of the wave frequency are mapped to a two-dimensional coordinate system for fitting to obtain a vehicle speed frequency reference curve.

8. The millimeter wave radar device based on intelligent AI learning according to claim 1, characterized in that: Matching millimeter-wave radar parameters based on the scene category to obtain millimeter-wave radar transmission parameters includes: When the scene category is an urban road, the target type that the radar focuses on is set to pedestrian / bicycle detection priority, the constant false alarm detection threshold is set to a first threshold value, the update frequency is set to a first frequency value, and the detection radius is set to a first detection range; when the scene category is an expressway, the target type that the radar focuses on is set to vehicle detection priority, the constant false alarm detection threshold is set to a second threshold value, the update frequency is set to a second frequency value, and the detection radius is set to a second detection range; when the scene category is a parking lot, the target type that the radar focuses on is set to static obstacle detection priority, the constant false alarm detection threshold is set to a third threshold value, the update frequency is set to a third frequency value, and the detection radius is set to a third detection range; Among them, the first threshold value is smaller than the second threshold value, and the second threshold value is smaller than the third threshold value; the third frequency value is smaller than the first frequency value, and the first frequency value is smaller than the second frequency value; the third detection range is within the first detection range, and the second detection range is outside the first detection range.

9. The millimeter wave radar device based on intelligent AI learning according to claim 1, characterized in that: Also includes: The Doppler frequency shift of the vehicle-mounted radar is compensated based on the vehicle speed information to obtain a compensated target.

10. The millimeter wave radar device based on intelligent AI learning according to claim 9, characterized in that: Compensating for the Doppler frequency shift of the vehicle-mounted radar based on the vehicle speed information to obtain a compensated target includes: Calculating the Doppler shift caused by vehicle movement based on the vehicle speed information, and calculating the Doppler shift of the target relative to the vehicle; Eliminating a Doppler frequency shift caused by the movement of the vehicle from a Doppler frequency shift of the target relative to the vehicle to obtain a Doppler frequency shift of the target relative to the ground; The target is tracked and located based on the Doppler frequency shift of the target relative to the ground.

Images