Vehicle-mounted environment sensing system based on ultrasonic radar and millimeter wave radar

Abstract

The invention discloses a vehicle-mounted environment sensing system based on an ultrasonic radar and a millimeter wave radar, and relates to the technical field of intelligent driving. The problem that in the prior art, extra environment information is obtained by depending on communication between roadside equipment and vehicle-mounted equipment, the complexity of multi-sensor fusion is not fully considered, the advantages of different sensors are difficult to fully exert, and consequently the precision and reliability of environment sensing are limited is solved. According to the system, the surrounding environment of the vehicle can be comprehensively and accurately sensed in a mode of fusing multiple sensors, the domain controller carries out fusion processing on multi-source data, vehicle-mounted environment sensing information is generated, intelligent vehicle control is achieved accordingly, the states and the data of the sensors are monitored and processed through the communication management module, and the system is convenient to use. And functions of working mode management, time synchronization management and the like are realized, so that stable and reliable operation of the system under various working conditions is guaranteed, and the performance and practicability of the whole vehicle-mounted environment sensing system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent driving, and particularly relates to an in-vehicle environment perception system based on ultrasonic radar and millimeter-wave radar. Background Art

[0002] The in-vehicle environment perception system has become a key part of intelligent driving. The Chinese patent application with the publication number CN111768642A discloses a road environment perception and vehicle control method, system, device and vehicle for a vehicle, including roadside equipment and in-vehicle equipment arranged on the vehicle; the roadside equipment includes roadside sensors for sensing the road environment; in-vehicle sensors for performing environment perception are also arranged on the vehicle; one or more communication devices are arranged between the roadside equipment and the in-vehicle equipment, and the roadside equipment can communicate with the in-vehicle equipment through one or more communication devices, and the communicable range for the roadside equipment to communicate with the in-vehicle equipment is greater than the perception range of the in-vehicle sensors; the roadside equipment obtains road environment information through the roadside sensors according to a pre-set acquisition strategy; the roadside equipment transmits the road environment information to the in-vehicle equipment through one or more communication devices; the in-vehicle equipment generates path planning information according to the road environment information and controls the vehicle to travel.

[0003] Although the above patent provides richer data support for intelligent driving, there are still the following problems:

[0004] The prior art relies on the communication between roadside equipment and in-vehicle equipment to obtain additional environment information, does not fully consider the complexity of multi-sensor fusion, and is difficult to give full play to the advantages of different sensors, resulting in limited accuracy and reliability of environment perception and being unable to meet the growing needs of intelligent driving. Summary of the Invention

[0005] The purpose of the present invention is to provide an in-vehicle environment perception system based on ultrasonic radar and millimeter-wave radar. By adopting a multi-sensor fusion method, it can comprehensively and accurately perceive the environment around the vehicle. The domain controller performs fusion processing and intelligent analysis on multi-source data, generates in-vehicle environment perception information, and accordingly realizes intelligent vehicle control, improving the performance and practicality of the entire in-vehicle environment perception system, so as to solve the problems proposed in the above background art.

[0006] To achieve the above purpose, the present invention provides the following technical solutions:

[0007] An in-vehicle environment perception system based on ultrasonic radar and millimeter-wave radar, including:

[0008] A radar detection unit, configured to detect obstacle information around the vehicle in real time based on an ultrasonic radar sensor, detect object information in the middle and long distances of the vehicle in real time based on a millimeter-wave radar sensor, and generate radar data based on the real-time detection results;

[0009] A domain controller, configured to receive and process the acquired radar data based on a communication link, generate vehicle environment perception information, match vehicle control instructions according to the generated vehicle environment perception information, and control the driving state of the vehicle according to the vehicle control instructions;

[0010] Meanwhile, when receiving the radar data, the domain controller performs anomaly detection and processing on the data, and filters and judges the radar data according to preset standard conditions.

[0011] Furthermore, the radar detection unit further includes: capturing image data during the vehicle driving process based on multiple cameras with different perspectives carried thereon, transmitting and receiving laser pulses through a lidar to obtain laser data of the vehicle surrounding environment, and meanwhile, fusing the positioning technologies of inertial navigation and GPS to obtain the positioning data of the vehicle.

[0012] Furthermore, the domain controller includes:

[0013] A communication management module, configured to establish a communication link between the radar detection unit and the domain controller, and realize data interaction between the ultrasonic radar sensor, the millimeter-wave radar sensor and the domain controller based on the communication link;

[0014] A data processing module, configured to perform fusion processing on the acquired radar data, image data, laser data and positioning data, identify target objects around the vehicle, and generate vehicle environment perception information;

[0015] A control module, configured to judge the distance and relative motion state between the vehicle and surrounding objects based on the vehicle environment perception information, and control the braking system of the vehicle according to the distance and relative motion state, and dynamically adjust the driving state of the vehicle;

[0016] A power management module, configured to provide power output and serve as a wake-up source, and perform mode conversion on the domain controller in combination with the operating state and actual requirements of the vehicle.

[0017] Furthermore, the communication management module further includes:

[0018] Initializing the communication interfaces of the ultrasonic radar sensor and the millimeter-wave radar sensor, acquiring radar data based on the communication link, and judging the power states of the sensors. If the power is abnormal, an alarm is immediately issued and the fault time point is recorded;

[0019] During the data reception process, traverse according to the characteristics of the acquired radar data itself, extract abnormal data that does not meet its own requirements, and obtain the acquisition time point of the abnormal data, and judge whether the time interval and the number of time points of the acquisition time point meet the minimum normal standard;

[0020] If the condition is met, abnormal data is removed from the radar data to obtain normal radar data; otherwise, it is determined that the radar detection unit has an abnormal acquisition, and an alarm reminder is given.

[0021] After the sensor fault alarm is issued, the fault information is displayed on the HMI through the in-vehicle network, a sound alarm is issued, and at the same time, the fault information is uploaded to the background server through the TBOX.

[0022] Furthermore, abnormal data that does not meet its own requirements is extracted, and the acquisition time point of the abnormal data is obtained. It is determined whether the time interval and the number of time points of the acquisition time point meet the minimum normal standard, including setting the standard conditions corresponding to the minimum normal standard. The setting process is as follows:

[0023] Extract abnormal data that does not meet its own requirements;

[0024] Compare the abnormal data that does not meet its own requirements with a preset abnormal data reference value;

[0025] Screen out abnormal data that is not lower than the abnormal data reference value as the first abnormal data;

[0026] Screen out abnormal data that is lower than the abnormal data reference value as the second abnormal data;

[0027] Extract the data moment corresponding to the first abnormal data as the first data moment data;

[0028] Extract the data moment corresponding to the second abnormal data as the second data moment data;

[0029] Obtain the standard deviation of the first abnormal data according to the first abnormal data, and perform normalization processing on the standard deviation of the first abnormal data to obtain the normalized standard deviation of the first abnormal data;

[0030] Obtain the standard deviation of the time interval corresponding to the first data moment data according to the first data moment data, and perform normalization processing on the standard deviation of the time interval corresponding to the first data moment data to obtain the normalized first time standard deviation;

[0031] Obtain the standard deviation of the second abnormal data according to the second abnormal data, and perform normalization processing on the standard deviation of the second abnormal data to obtain the normalized standard deviation of the second abnormal data;

[0032] Obtain the standard deviation of the time interval corresponding to the second data moment data according to the second data moment data, and perform normalization processing on the standard deviation of the time interval corresponding to the second data moment data to obtain the normalized second time standard deviation;

[0033] Use the standard deviation of the first abnormal data, the standard deviation of the second abnormal data, the standard deviation of the first time, and the standard deviation of the second time to determine whether the time interval and the number of time points of the acquisition time points meet the minimum normal standard.

[0034] Further, using the standard deviation of the first abnormal data, the standard deviation of the second abnormal data, the standard deviation of the first time, and the standard deviation of the second time to determine whether the time interval and the number of time points of the acquisition time points meet the minimum normal standard, including:

[0035] Retrieve the standard deviation of the first abnormal data, the standard deviation of the second abnormal data, the standard deviation of the first time, and the standard deviation of the second time;

[0036] Compare the standard deviation of the first abnormal data, the standard deviation of the second abnormal data, the standard deviation of the first time, and the standard deviation of the second time with their corresponding threshold parameters respectively;

[0037] When the standard deviation of the first abnormal data and the standard deviation of the second abnormal data do not exceed their corresponding threshold parameters, and the standard deviation of the first time and the standard deviation of the second time do not exceed their corresponding threshold parameters, it is determined that the time interval and the number of time points of the acquisition time points meet the minimum normal standard;

[0038] When the standard deviation of the first abnormal data and the standard deviation of the second abnormal data both exceed their corresponding threshold parameters, or the standard deviation of the first time and the standard deviation of the second time both exceed their corresponding threshold parameters, it is determined that the time interval and the number of time points of the acquisition time points do not meet the minimum normal standard;

[0039] When any one of the standard deviation of the first abnormal data and the standard deviation of the second abnormal data exceeds its corresponding threshold parameter, or any one of the standard deviation of the first time and the standard deviation of the second time exceeds its corresponding threshold parameter, extract the abnormal data standard deviation that exceeds its corresponding threshold parameter as the observed abnormal data standard deviation, and extract the time standard deviation that exceeds its corresponding threshold parameter as the observed time standard deviation;

[0040] Use the observed abnormal data standard deviation and the observed time standard deviation to obtain the standard deviation coefficient;

[0041] Wherein, the standard deviation coefficient is obtained through the following formula:

[0042]

[0043] Wherein, U represents the standard deviation coefficient; σ represents the observed abnormal data standard deviation; σ 0 represents the threshold parameter corresponding to the observed abnormal data standard deviation; τ represents the observed time standard deviation; τ 0 represents the threshold parameter corresponding to the observed time standard deviation; σ yDenote the standard deviation values in the first abnormal data standard deviation and the second abnormal data standard deviation that do not exceed their corresponding threshold parameters; σ y0 Denote the threshold parameters corresponding to the standard deviation values in the first abnormal data standard deviation and the second abnormal data standard deviation that do not exceed their corresponding threshold parameters; τ y Denote the standard deviation values in the first time standard deviation and the second time standard deviation that do not exceed their corresponding threshold parameters; τ y0 Denote the threshold parameters corresponding to the standard deviation values in the first time standard deviation and the second time standard deviation that do not exceed their corresponding threshold parameters;

[0044] Compare the standard deviation coefficient with a preset coefficient reference value;

[0045] When the standard deviation coefficient is not lower than the preset coefficient reference value, it is determined that the time interval and the number of time points of the acquisition time point do not meet the minimum normal standard;

[0046] When the standard deviation coefficient is lower than the preset coefficient reference value, it is determined that the time interval and the number of time points of the acquisition time point meet the minimum normal standard.

[0047] Further, the domain controller further includes:

[0048] A working mode management module configured to control the switching of the domain controller between different working modes according to different operating states and usage scenarios of the vehicle, where the working modes of the domain controller include: an operating mode, a low power consumption mode, a sleep mode, and a power-off mode;

[0049] A time synchronization management module configured to achieve time synchronization between each module in the domain controller and between each external device in the radar detection unit.

[0050] Further, the data processing module performs fusion processing, specifically:

[0051] Classify and integrate the received radar data, image data, laser data, and positioning data, and perform preprocessing;

[0052] Based on the synchronization time of the time synchronization management module, align the preprocessed various types of data, extract the feature information in each type of data, and associate the feature information about the same target in different types of data;

[0053] Obtain multi-sensor fusion data containing different target objects as data samples, determine the feature information and data types of the target objects, and determine the classification identifiers of the target objects based on the feature information and data types of the target objects;

[0054] Inputting the classification identifier of the target object into a preset neural network for learning, determining the classification expression of the data classification identifier, and constructing a target recognition model of the target object based on the classification expression;

[0055] The target object is classified and identified through the target recognition model to determine the category of the target object. At the same time, the associated feature information is fused to generate an information data set of the target object;

[0056] Based on the recognition results, a tracking track is established for each target object, and the parameter data of the position, speed and direction of each target object at different times are recorded;

[0057] According to the historical tracking trajectory and current parameter data of the target object, the future motion trajectory of the target object is predicted;

[0058] Based on the tracking trajectory and prediction results of each target object, the vehicle environment perception information is generated.

[0059] Furthermore, a tracking track is established for each target object based on the recognition result, specifically including:

[0060] According to the classification identification of the target object, a corresponding target storage space is established to store the information data set of the target object;

[0061] Construct a time axis according to the time series of data collection, map the collected data related to the target object onto the time axis according to the time series, and generate dynamic data of the target object;

[0062] Extract the positioning data of each target object at different times from the dynamic data, combine the initial position of each target object, construct the moving trajectory of the target object, and determine the moving characteristics of the target object;

[0063] At the same time, the trajectory change frequency of the target object in different time periods is analyzed to generate the pause characteristics of the target object;

[0064] Determine the target time segment where each target object appears most frequently or stays the longest in the dynamic data, and generate position attributes based on the order and position of the target time segment in the entire dynamic data;

[0065] When multiple target objects exist at the same time, the motion state and interaction behavior of each target object in a similar time period and position are analyzed to extract the interaction features between different target objects;

[0066] The movement characteristics, pause characteristics, position attributes and interaction characteristics of the target objects are integrated to establish a movement data set corresponding to each target object, and the movement range of each target object is determined in combination with the environmental information of the scene.

[0067] Further, the control module dynamically adjusts the driving state of the vehicle, specifically as follows:

[0068] Based on the generated vehicle-mounted environment perception information, combined with the current position and destination information of the vehicle, plan the driving path of the vehicle;

[0069] And adjust the driving path based on the moving trajectory and moving range of the target object to avoid areas with potential dangers or areas that may cause driving obstacles;

[0070] According to the path planning result, the distance and relative motion state between the vehicle and surrounding target objects, as well as traffic rules and road conditions, determine the driving speed of the vehicle, and when the vehicle approaches a vehicle or obstacle moving slowly ahead, reduce the speed and maintain a safe distance.

[0071] Compared with the prior art, the beneficial effects of the present invention are:

[0072] By adopting the method of multi-sensor fusion, it can comprehensively and accurately perceive the vehicle's surrounding environment. Whether it is a short-range obstacle, a medium- or long-range object, or the vehicle's positioning information, etc., can be accurately obtained. The domain controller performs fusion processing and intelligent analysis on multi-source data to generate vehicle-mounted environment perception information, and accordingly realizes intelligent vehicle control. And through functions such as monitoring and processing the sensor status and data by the communication management module, as well as working mode management and time synchronization management, etc., it ensures the stable and reliable operation of the system under various working conditions, and improves the performance and practicability of the entire vehicle-mounted environment perception system. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 It is a module diagram of the vehicle-mounted environment perception system of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0074] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0075] In order to solve the technical problem that the prior art relies on the communication between roadside devices and in-vehicle devices to obtain additional environment information, does not fully consider the complexity of multi-sensor fusion, is difficult to fully utilize the advantages of different sensors, resulting in limited accuracy and reliability of environment perception, and cannot meet the growing needs of intelligent driving, please refer to Figure 1 , the present embodiment provides the following technical solutions:

[0076] A vehicle-mounted environment perception system based on ultrasonic radar and millimeter-wave radar, comprising:

[0077] A radar detection unit, configured to detect obstacle information around the vehicle in real time based on ultrasonic radar sensors installed on the front and rear bumpers of the vehicle, detect object information in the middle and long distances of the vehicle based on millimeter-wave radar sensors installed on the front windshield of the vehicle, generate radar data based on the real-time detection results, and further includes:

[0078] Capture image data during the vehicle's driving process based on multiple cameras with different perspectives carried, transmit it to the domain controller, identify targets such as road signs, traffic signals, pedestrians, and vehicles based on computer vision technology, and through the processing of consecutive frame images, can also predict the movement trajectories of the targets, providing richer information for the vehicle's decision-making control; and emit and receive laser pulses through a lidar to obtain laser data of the vehicle's surrounding environment, fuse it with ultrasonic radar and millimeter-wave radar data to create a three-dimensional map of the surrounding environment, measure the distance, shape, and speed of objects, improving the accuracy and reliability of environmental perception; at the same time, fuse the positioning technology of inertial navigation and GPS to obtain the positioning data of the vehicle;

[0079] A domain controller, configured to receive and process the obtained radar data based on a communication link, generate vehicle-mounted environment perception information, match vehicle control instructions according to the generated vehicle-mounted environment perception information, control the driving state of the vehicle according to the vehicle control instructions. At the same time, when the domain controller receives radar data, it performs anomaly detection and processing on the data, and filters and judges the radar data according to preset standard conditions.

[0080] In this embodiment, by detecting obstacle information around the vehicle in real time through ultrasonic radar sensors, detecting object information in the middle and long distances through millimeter-wave radar sensors, and further fusing camera image data, lidar laser data, and positioning data, it can comprehensively and accurately perceive the vehicle-mounted environment, providing rich and accurate information for the vehicle's driving. By fusing multiple sensors, it can more comprehensively and accurately perceive the surrounding environment. The domain controller processes sensor data in real time, generates vehicle-mounted environment perception information, and controls the vehicle's driving state according to the information, which can avoid collisions, assist driving, and can be used as the core component of an autonomous driving system, improving driving safety and convenience.

[0081] In this embodiment, the domain controller includes:

[0082] A communication management module, configured to establish a communication link between the radar detection unit and the domain controller, and realize data interaction between the ultrasonic radar sensors, millimeter-wave radar sensors, and the domain controller based on the communication link;

[0083] A data processing module, configured to perform fusion processing on the obtained radar data, image data, laser data, and positioning data, identify target objects around the vehicle, and generate vehicle-mounted environment perception information;

[0084] A control module, configured to judge the distance and relative motion state between the vehicle and surrounding objects based on vehicle-mounted environment perception information, and control the braking system of the vehicle according to the distance and relative motion state, and dynamically adjust the driving state of the vehicle. Specifically:

[0085] Based on the generated vehicle-mounted environment perception information, combined with the current position and destination information of the vehicle, plan the driving path of the vehicle;

[0086] And adjust the driving path based on the moving trajectory and moving range of the target object to avoid areas with potential dangers or areas that may cause driving obstacles;

[0087] Determine the driving speed of the vehicle according to the path planning result, the distance and relative motion state between the vehicle and surrounding target objects, as well as traffic rules and road conditions, and reduce the vehicle speed and maintain a safe distance when the vehicle approaches a vehicle or obstacle moving slowly ahead;

[0088] A power management module, configured to provide stable and multi-specification power output, and serve as a wake-up source. Combine the operating state and actual needs of the vehicle to perform mode conversion on the domain controller, such as recovering from a sleep or low-power state to an operating state, and switching between an operating mode, a low-power mode, a sleep mode, and a power-off mode;

[0089] A working mode management module, configured to control the switching of the domain controller between different working modes according to different operating states and usage scenarios of the vehicle. Among them, the working modes of the domain controller include: an operating mode, a low-power mode, a sleep mode, and a power-off mode;

[0090] A time synchronization management module, configured to achieve time synchronization between each module in the domain controller and each external device in the radar detection unit. With the help of the network general precise time protocol, provide timing for the application processor and the micro control unit as the management time, and use the master-slave setting of this protocol supported by the switch to ensure accurate time synchronization of each internal module. The domain controller synchronously triggers the camera, and the micro control unit transmits the data plane time to the ultrasonic and millimeter wave radar related modules through a specific interface and the CAN bus. When external timing cannot be used, the external real-time clock provides timing for the internal real-time clock.

[0091] In this embodiment, the data processing module integrates multiple types of data to accurately identify the target object. The power control module intelligently plans the driving route based on the vehicle-mounted environmental perception information and dynamically adjusts the vehicle speed according to the status of surrounding objects and traffic conditions to ensure the safe driving of the vehicle in all aspects. The power management module provides stable power output and flexibly switches the domain controller working mode according to the actual needs of the vehicle to achieve high efficiency and energy saving. The working mode management module further optimizes the domain controller working mode according to the different operating states and usage scenarios of the vehicle to improve the system adaptability. The time synchronization management module ensures the consistency of time among various modules in the system and external devices, enhances the system coordination, greatly improves the overall operating efficiency and reliability, and provides strong support for the intelligent operation of the vehicle.

[0092] In this embodiment, the communication management module further includes:

[0093] Initialize the communication interface of the ultrasonic radar sensor and the millimeter wave radar sensor, obtain radar data based on the communication link, and determine the power status of each sensor. If the power supply is abnormal, immediately issue an alarm and record the fault time point;

[0094] During the data receiving process, the radar data is traversed according to its own characteristics, abnormal data that does not meet its own requirements are extracted, and the collection time points of the abnormal data are obtained to determine whether the time interval and number of the collection time points meet the minimum normal standard;

[0095] If the conditions are met, the abnormal data is removed from the radar data to obtain normal radar data. Otherwise, the radar detection unit is judged to have abnormal data collection, and an alarm is issued to notify relevant personnel to check the radar equipment and record the fault time and type.

[0096] When a sensor fault alarm is issued, the fault information is displayed on the HMI through the in-vehicle network, and an audible alarm is issued. At the same time, the fault information (fault time, type, sensor number, etc.) is uploaded to the background server through TBOX, which facilitates remote monitoring and maintenance and provides data support for subsequent fault analysis and system optimization.

[0097] In this embodiment, the HMI is used to display key driving information (such as speed, warning signals) or control emergency functions (such as emergency brake assist);

[0098] In this embodiment, TBOX is used for network access, OTA, remote control, location query / vehicle tracking, battery management, location reminder, eCa ll, remote diagnosis, platform monitoring / national supervision, etc.;

[0099] In this embodiment, during the data receiving process, the parameter self-checking unit traverses the data according to the ultrasonic radar data characteristics, checks whether the distance data is within the range of 15cm-5.5m, whether the frequency is 57.5kHz, etc.; extracts abnormal data and obtains the acquisition time point, and determines whether the time interval and quantity meet the minimum standard (such as the time interval does not exceed 100ms, the number of consecutive abnormal points does not exceed 5, or whether it meets the preset standard conditions). If so, the abnormal data is removed; otherwise, it is determined that the ultrasonic radar acquisition is abnormal, the communication management module issues an alarm, records the fault time and type, and uploads the fault information to the background.

[0100] Specifically, the abnormal data that does not meet the requirements is extracted, and the collection time points of the abnormal data are obtained. It is determined whether the time interval and the number of time points of the collection time points meet the minimum normal standard, including setting the standard conditions corresponding to the minimum normal standard. The setting process is as follows:

[0101] Extract abnormal data that does not meet your requirements;

[0102] Compare the abnormal data that does not meet the requirements with a preset abnormal data reference value;

[0103] Screening out abnormal data that is not lower than the abnormal data reference value as first abnormal data;

[0104] Screening out abnormal data lower than the abnormal data reference value as second abnormal data;

[0105] Extracting the data time corresponding to the first abnormal data as the first data time data;

[0106] Extracting the data time corresponding to the second abnormal data as second data time data;

[0107] Obtaining a standard deviation of the first abnormal data according to the first abnormal data, and normalizing the standard deviation of the first abnormal data to obtain the first abnormal data standard deviation after normalization;

[0108] Acquire a time interval standard deviation corresponding to the first data moment data according to the first data moment data, and perform normalization processing on the time interval standard deviation corresponding to the first data moment data to acquire a first time standard deviation after normalization processing;

[0109] Obtaining a standard deviation of the second abnormal data according to the second abnormal data, and normalizing the standard deviation of the second abnormal data to obtain the standardized standard deviation of the second abnormal data;

[0110] Obtain the standard deviation of the time interval corresponding to the data at the second data moment according to the data at the second data moment, and perform normalization processing on the standard deviation of the time interval corresponding to the data at the second data moment to obtain the second time standard deviation after the standardization process;

[0111] Use the first abnormal data standard deviation, the second abnormal data standard deviation, the first time standard deviation, and the second time standard deviation to determine whether the time interval and the number of time points of the acquisition time point meet the minimum normal standard.

[0112] The technical effects of the above technical solutions are as follows: By subdividing the abnormal data that does not meet the requirements into the first abnormal data (not lower than the abnormal data reference value) and the second abnormal data (lower than the abnormal data reference value), the refined classification of the abnormal data is realized. This classification helps to more accurately understand the abnormal situation of the data and provides a basis for subsequent analysis and processing. Normalize the standard deviations of the first abnormal data and the second abnormal data, as well as the corresponding standard deviation of the time interval, so that data with different dimensions and scales become comparable. This standardization process helps to eliminate the differences between the data and makes the subsequent analysis more accurate and reliable. By comprehensively considering the first abnormal data standard deviation, the second abnormal data standard deviation, the first time standard deviation, and the second time standard deviation, this technical solution can evaluate the time interval and the number of time points of the acquisition time point from multiple dimensions. This multi-dimensional evaluation can more comprehensively reflect the performance status of the data and helps to discover potential problems. This technical solution allows setting the minimum normal standard according to actual needs and provides a corresponding standard condition setting process. This flexibility enables this technical solution to adapt to different application scenarios and data characteristics and improves its practicality and applicability. By extracting the abnormal data that does not meet the requirements and analyzing whether the time interval and the number of time points of its acquisition time point meet the minimum normal standard, this technical solution helps to timely discover and handle the abnormalities and problems in the data. This helps to improve the quality and reliability of the data and provides strong support for subsequent data analysis and applications. The process-based processing method of the technical solution of this embodiment above helps to optimize the data processing process and improve work efficiency and accuracy.

[0113] Specifically, using the first abnormal data standard deviation, the second abnormal data standard deviation, the first time standard deviation, and the second time standard deviation to determine whether the time interval and the number of time points of the acquisition time point meet the minimum normal standard includes:

[0114] Retrieve the first abnormal data standard deviation, the second abnormal data standard deviation, the first time standard deviation, and the second time standard deviation;

[0115] Compare the first abnormal data standard deviation, the second abnormal data standard deviation, the first time standard deviation, and the second time standard deviation with their corresponding threshold parameters respectively;

[0116] When the standard deviation of the first abnormal data and the standard deviation of the second abnormal data do not exceed their corresponding threshold parameters, and the standard deviation of the first time and the standard deviation of the second time do not exceed their corresponding threshold parameters, it is determined that the time interval and the number of time points of the acquisition time points meet the minimum normal standard;

[0117] When the standard deviation of the first abnormal data and the standard deviation of the second abnormal data both exceed their corresponding threshold parameters, or the standard deviation of the first time and the standard deviation of the second time both exceed their corresponding threshold parameters, it is determined that the time interval and the number of time points of the acquisition time points do not meet the minimum normal standard;

[0118] When any one of the standard deviation of the first abnormal data and the standard deviation of the second abnormal data exceeds its corresponding threshold parameter, or any one of the standard deviation of the first time and the standard deviation of the second time exceeds its corresponding threshold parameter, the abnormal data standard deviation that exceeds its corresponding threshold parameter is extracted as the observed abnormal data standard deviation, and the time standard deviation that exceeds its corresponding threshold parameter is extracted as the observed time standard deviation;

[0119] The standard deviation coefficient is obtained by using the observed abnormal data standard deviation and the observed time standard deviation;

[0120] Among them, the standard deviation coefficient is obtained through the following formula:

[0121]

[0122] Among them, U represents the standard deviation coefficient; σ represents the observed abnormal data standard deviation; σ 0 represents the threshold parameter corresponding to the observed abnormal data standard deviation; τ represents the observed time standard deviation; τ 0 represents the threshold parameter corresponding to the observed time standard deviation; σ y represents the standard deviation value of the standard deviation of the first abnormal data and the second abnormal data that does not exceed its corresponding threshold parameter; σ y0 represents the threshold parameter corresponding to the standard deviation value of the standard deviation of the first abnormal data and the second abnormal data that does not exceed its corresponding threshold parameter; τ y represents the standard deviation value of the standard deviation of the first time and the second time that does not exceed its corresponding threshold parameter; τ y0 represents the threshold parameter corresponding to the standard deviation value of the standard deviation of the first time and the second time that does not exceed its corresponding threshold parameter;

[0123] Compare the standard deviation coefficient with a preset coefficient reference value;

[0124] When the standard deviation coefficient is not lower than the preset coefficient reference value, it is determined that the time interval and the number of time points of the acquisition time points do not meet the minimum normal standard;

[0125] When the coefficient of standard deviation is lower than the preset reference coefficient value, it is determined that the time interval and the number of time points at the acquisition time point meet the minimum normal standard.

[0126] The technical effects of the above technical solution are as follows: By comparing various standard deviations with the corresponding threshold parameters, it can effectively determine whether the time interval and the number of acquisition time points are appropriate, ensure the reliability and stability of the collected data, avoid data anomalies caused by unreasonable time intervals or the number of points, and improve the quality of the data acquisition link. For complex situations where some indicators exceed the threshold, by calculating the coefficient of standard deviation and comparing it with the reference value, it can more accurately determine whether the minimum normal standard is met, which helps to timely discover potential data acquisition problems, provides a reliable basis for subsequent data processing and analysis, and improves the system's ability to identify abnormal situations. Among them, where τ - τ 0 represents the difference between the standard deviation of the observation time and its threshold parameter, reflecting the degree to which the time standard deviation deviates from the normal range; σ - σ 0 represents the difference between the standard deviation of the observed abnormal data and its threshold parameter, reflecting the degree to which the standard deviation of the abnormal data deviates from the normal. Dividing the two and taking the square root comprehensively considers the deviation degrees of the time and the standard deviation of the abnormal data relative to their respective thresholds, and measures the relative relationship between the time and the fluctuations of the abnormal data. where τ y -τ y0 is the difference between the time standard deviation that does not exceed the threshold and its corresponding threshold parameter, and σ y0 -σ y is the difference between the standard deviation of the abnormal data that does not exceed the threshold and its corresponding threshold parameter. Similarly, dividing the two and taking the square root represents the deviation of the time and the standard deviation of the abnormal data that do not exceed the threshold relative to their respective thresholds, serving as a reference benchmark. The U (coefficient of standard deviation) calculated by the overall formula of the above technical solution quantifies the comparison between the observed abnormal situations (the part exceeding the threshold) and the normal situations (the part not exceeding the threshold), reflecting the comprehensive deviation degree of the time and the standard deviation of the abnormal data exceeding the threshold relative to the part not exceeding the threshold. When the coefficient of standard deviation is not lower than the preset reference coefficient value, it indicates that the comprehensive deviation degree of the time and the standard deviation of the abnormal data exceeding the threshold relative to the part not exceeding the threshold is large, that is, there are significant problems with the time interval or the number of time points at the acquisition time point, resulting in abnormal data fluctuations, and it is determined that the minimum normal standard is not met. When the coefficient of standard deviation is lower than the preset reference coefficient value, it indicates that the comprehensive deviation degree of the time and the standard deviation of the abnormal data exceeding the threshold relative to the part not exceeding the threshold is small, and the time interval and the number of time points at the acquisition time point are within an acceptable range, and it is determined that the minimum normal standard is met.

[0127] On the other hand, by introducing four key indicators, namely the standard deviation of the first abnormal data, the standard deviation of the second abnormal data, the standard deviation of the first time, and the standard deviation of the second time, this technical solution can comprehensively evaluate the rationality of the time interval and the number of time points of the acquisition time points. This comprehensive evaluation method is more accurate and comprehensive than the single-indicator evaluation. The technical solution provides a clear judgment criterion, that is, when all standard deviations do not exceed their corresponding threshold parameters, it is determined that the acquisition time points meet the minimum normal standard; when all standard deviations exceed the threshold or any one exceeds, further analysis is carried out according to the specific situation. This judgment criterion is both strict and flexible, can adapt to different data situations and application scenarios, and can reduce the misjudgment rate, thereby effectively improving the accuracy and timeliness of the standard compliance determination. On the other hand, the above technical solution of this embodiment can complete the standard determination with high accuracy and high efficiency only by using the standard deviation of the abnormal data and the time interval, greatly omitting the types of physical quantities used in the standard determination, improving the determination accuracy and efficiency in the case of omitting elements, and effectively reducing the occupancy rate of computing power resources while improving the accuracy and efficiency. When a standard deviation exceeds the threshold, the technical solution can accurately locate the abnormal data standard deviation and the time standard deviation that exceed the threshold, providing a clear direction for subsequent problem analysis and solution. By calculating the coefficient of standard deviation, this technical solution provides a more refined and quantitative basis for judging whether the time interval and the number of time points of the acquisition time points meet the minimum normal standard. The coefficient of standard deviation comprehensively considers the standard deviation of the observed abnormal data, the standard deviation of the observed time, and the standard deviation values that do not exceed the threshold, making the judgment result more accurate and reliable. The technical solution provides clear steps and processes, including retrieving the standard deviation, comparing the threshold, calculating the coefficient of standard deviation, etc., making the data processing process more standardized and efficient. This helps to reduce human errors and improve work efficiency. By comprehensively evaluating the time interval and the number of time points of the acquisition time points, this technical solution helps to discover and correct anomalies and errors in the data, thereby improving the quality and reliability of the data. This is of great significance for subsequent data analysis and applications. The threshold parameters and coefficient reference values in the technical solution can be adjusted and optimized according to the actual application scenario and data, making this technical solution have strong adaptability and flexibility. This helps to meet the data processing needs of different fields and industries.

[0128] In this embodiment, the millimeter wave radar self-check specifically includes: when receiving data, the parameter self-check unit checks the rationality of the data, such as the ranging range (0.1m-200m for the front radar, 0.1m-160m for the corner radar), frequency (100Hz), etc. Monitor the CAN communication status, if there are abnormalities such as data loss, error frame, or the message sending time after CANO wake-up exceeds 400ms, extract the abnormal data and time point, and judge whether the time-related indicators meet the standards (such as no more than 10 error frames within 1 minute). If the abnormal situation does not exceed the standard, process the abnormal data; otherwise, it is determined that the millimeter wave radar collection is abnormal, the communication management module issues an alarm, and records and uploads the fault information.

[0129] In this embodiment, the data processing module performs fusion processing, specifically:

[0130] Classify and integrate the received radar data, image data, laser data and positioning data, and perform preprocessing to remove interference noise in radar data, grayscale and reduce noise on image data, check and remove abnormal measurement points in laser data, evaluate the accuracy and reliability of positioning data and correct low-precision data;

[0131] Based on the synchronization time of the time synchronization management module, the pre-processed data is aligned, and the feature information in each type of data is extracted, and the feature information about the same target in different categories of data is associated; for example, the distance, speed, angle and other features of the target are extracted from the radar data; the shape, color, texture and other features of the target are extracted from the image data using the deep learning algorithm; the three-dimensional contour features of the target are extracted from the laser data, and the features about the same target in different sensors are associated through the feature matching algorithm to determine that these features describe the same target object;

[0132] Acquire multi-sensor fusion data containing different target objects as data samples, determine feature information and data types of the target objects, and determine a classification identification of the target objects based on the feature information and data types of the target objects;

[0133] Inputting the classification identifier of the target object into a preset neural network for learning, determining the classification expression of the data classification identifier, and constructing a target recognition model of the target object based on the classification expression;

[0134] The target object is classified and identified through the target recognition model to determine the category of the target object, including pedestrians, vehicles, road signs, traffic lights or other obstacles; at the same time, the associated feature information is fused to generate an information data set of the target object;

[0135] Based on the recognition results, a tracking track is established for each target object, and the parameter data of the position, speed and direction of each target object at different times are recorded;

[0136] Predict the future motion trajectory of the target object based on its historical tracking trajectory and current parameter data, taking into account the motion patterns of the target (such as uniform linear motion, accelerating motion, turning, etc.) and surrounding environmental factors (such as road curvature, traffic rules, etc.), to improve the accuracy of prediction, provide more forward-looking information for the decision-making control of the vehicle, enable the vehicle to make reasonable driving decisions in advance, and avoid dangerous situations such as collisions;

[0137] Generate vehicle-mounted environment perception information based on the tracking trajectory and prediction results of each target object, including:

[0138] List of target objects: List information such as the category, location, speed, shape, etc. of all surrounding target objects.

[0139] Trajectory of target objects: Display the historical trajectory and predicted trajectory of each target object.

[0140] Interaction of target objects: Display the interaction behaviors between different target objects, such as the change in following distance between vehicles, the avoidance behaviors of pedestrians and vehicles, etc.

[0141] Environmental information: Include information such as road conditions, traffic signal status, weather conditions, etc.

[0142] In this embodiment, establish a tracking trajectory for each target object based on the recognition result, specifically including:

[0143] Establish a corresponding target storage space according to the classification identifier of the target object for storing the information data set of the target object;

[0144] Construct a time axis according to the time series of data collection, map the data collected related to the target object onto the time axis according to the time series, and generate the dynamic data of the target object;

[0145] Extract the positioning data of each target object at different times from the dynamic data, combine with the initial position of each target object, construct the moving trajectory of the target object, and determine the moving characteristics such as the moving speed, acceleration, turning radius, etc. of the target object;

[0146] At the same time, analyze the trajectory change frequency of the target object in different time periods to generate the pause characteristics of the target object to indicate whether the target object is in a relatively stationary or slow moving state at certain moments;

[0147] Determine the target time period with the highest frequency of occurrence or the longest residence time of each target object in the dynamic data, and generate a position attribute based on the order and position of the target time period in the entire dynamic data, which is used to describe the important position or activity area of the target object in the scene;

[0148] For the situation where multiple target objects exist simultaneously, analyze the motion states and interaction behaviors of each target object in a similar time period and position, and extract the interaction features between different target objects, such as the change in following distance between vehicles, the avoidance behavior between pedestrians and vehicles, etc.;

[0149] Integrate the movement features, pause features, position attributes, and interaction features of the target objects to establish a movement data set corresponding to each target object, and determine the movement range of each target object in combination with the environmental information of the scene, including not only the spatial area actually reached by the target object, but also considering its possible movement trends and potential activity ranges.

[0150] In this embodiment, based on the synchronous time correlation feature information, the same target object can be accurately identified, the constructed target recognition model can accurately classify the target objects, through the information data set generated by fusing the correlation feature information, predict the future movement trajectory of the target object, and improve the accuracy by combining multiple factors, enabling the vehicle to make reasonable decisions in advance to avoid danger. When establishing the tracking trajectory, establish a storage space according to the classification identifier and generate dynamic data based on the time axis to clearly present the target movement. Adopt multiple features to establish a movement data set, determine the movement range in combination with the environment, master the movement characteristics and laws of the target, and finally generate rich, accurate, and forward-looking vehicle-mounted environment perception information, effectively improving the driving safety and intelligence level.

[0151] As described above, it is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent substitutions or changes, and should be covered by the protection scope of the present invention.

Claims

1. The vehicle-mounted environment perception system based on ultrasonic radar and millimeter wave radar is characterized by: include: A radar detection unit, configured to detect obstacle information around the vehicle in real time based on an ultrasonic radar sensor, detect object information in a medium and long distance from the vehicle in real time based on a millimeter wave radar sensor, and generate radar data based on the real-time detection results; A domain controller is configured to receive and process acquired radar data based on a communication link, generate vehicle environment perception information, match a vehicle control instruction according to the generated vehicle environment perception information, and control a driving state of the vehicle according to the vehicle control instruction; At the same time, when receiving radar data, the domain controller detects and processes the data for anomalies, and filters and judges the radar data according to preset standard conditions.

2. The vehicle-mounted environment perception system based on ultrasonic radar and millimeter wave radar as claimed in claim 1, characterized in that: The radar detection unit also includes: capturing image data of the vehicle during driving based on multiple cameras with different perspectives, and acquiring laser data of the vehicle's surrounding environment by transmitting and receiving laser pulses through a laser radar, and at the same time, integrating inertial navigation and GPS positioning technology to acquire the positioning data of the vehicle.

3. The vehicle-mounted environment perception system based on ultrasonic radar and millimeter wave radar as claimed in claim 2, characterized in that: Domain controllers, including: A communication management module is configured to establish a communication link between the radar detection unit and the domain controller, and realize data interaction between the ultrasonic radar sensor, the millimeter wave radar sensor and the domain controller based on the communication link; A data processing module configured to fuse the acquired radar data, image data, laser data and positioning data, identify target objects around the vehicle, and generate vehicle-mounted environment perception information; A control module configured to determine the distance and relative motion state between the vehicle and surrounding objects based on the vehicle-mounted environmental perception information, and to control the vehicle's braking system according to the distance and relative motion state to dynamically adjust the vehicle's driving state; The power management module is configured to provide power output and serve as a wake-up source to perform mode conversion on the domain controller in combination with the vehicle's operating status and actual needs.

4. The vehicle-mounted environment perception system based on ultrasonic radar and millimeter wave radar as claimed in claim 3, characterized in that: The communication management module also includes: Initialize the communication interface of the ultrasonic radar sensor and the millimeter wave radar sensor, obtain radar data based on the communication link, and determine the power status of each sensor. If the power supply is abnormal, immediately issue an alarm and record the fault time point; During the data receiving process, the radar data is traversed according to its own characteristics, abnormal data that does not meet its own requirements are extracted, and the collection time points of the abnormal data are obtained to determine whether the time interval and number of the collection time points meet the minimum normal standard; If the conditions are met, the abnormal data is removed from the radar data to obtain normal radar data. Otherwise, the radar detection unit is judged to have abnormal data collection and an alarm is issued; When a sensor fault alarm is issued, the fault information is displayed on the HMI through the in-vehicle network, an audible alarm is issued, and the fault information is uploaded to the backend server through TBOX.

5. The vehicle-mounted environment perception system based on ultrasonic radar and millimeter wave radar as claimed in claim 4, characterized in that: Extract the abnormal data that does not meet the requirements, obtain the collection time points of the abnormal data, and determine whether the time interval and number of the collection time points meet the minimum normal standard, including setting the standard conditions corresponding to the minimum normal standard. The setting process is as follows: Extract abnormal data that does not meet your requirements; Compare the abnormal data that does not meet the requirements with a preset abnormal data reference value; Screening out abnormal data that is not lower than the abnormal data reference value as first abnormal data; Screening out abnormal data below the abnormal data reference value as second abnormal data; Extracting the data time corresponding to the first abnormal data as the first data time data; Extracting the data time corresponding to the second abnormal data as second data time data; Obtaining a standard deviation of the first abnormal data according to the first abnormal data, and normalizing the standard deviation of the first abnormal data to obtain the first abnormal data standard deviation after normalization; Acquire a time interval standard deviation corresponding to the first data moment data according to the first data moment data, and perform normalization processing on the time interval standard deviation corresponding to the first data moment data to acquire a first time standard deviation after normalization processing; Obtaining a standard deviation of the second abnormal data according to the second abnormal data, and normalizing the standard deviation of the second abnormal data to obtain the standardized standard deviation of the second abnormal data; Acquire a time interval standard deviation corresponding to the second data moment data according to the second data moment data, and perform normalization processing on the time interval standard deviation corresponding to the second data moment data to acquire a second time standard deviation after normalization processing; The first abnormal data standard deviation, the second abnormal data standard deviation, the first time standard deviation and the second time standard deviation are used to determine whether the time interval of the acquisition time points and the number of time points meet the minimum normal standard.

6. The vehicle-mounted environment perception system based on ultrasonic radar and millimeter wave radar as claimed in claim 5, characterized in that: Using the first abnormal data standard deviation, the second abnormal data standard deviation, the first time standard deviation, and the second time standard deviation to determine whether the time interval and the number of time points of the acquisition time points meet the minimum normal standard includes: Retrieve the first abnormal data standard deviation, the second abnormal data standard deviation, the first time standard deviation, and the second time standard deviation; Compare the first abnormal data standard deviation, the second abnormal data standard deviation, the first time standard deviation and the second time standard deviation with their corresponding threshold parameters respectively; When the first abnormal data standard deviation and the second abnormal data standard deviation do not exceed their corresponding threshold parameters, and the first time standard deviation and the second time standard deviation do not exceed their corresponding threshold parameters, it is determined that the time interval and the number of time points of the acquisition time points meet the minimum normal standard; When both the first abnormal data standard deviation and the second abnormal data standard deviation exceed their corresponding threshold parameters, or both the first time standard deviation and the second time standard deviation exceed their corresponding threshold parameters, it is determined that the time interval and the number of time points of the acquisition time points do not meet the minimum normal standard; When any one of the first abnormal data standard deviation and the second abnormal data standard deviation exceeds its corresponding threshold parameter, or any one of the first time standard deviation and the second time standard deviation exceeds its corresponding threshold parameter, the abnormal data standard deviation exceeding its corresponding threshold parameter is extracted as the observed abnormal data standard deviation, and the time standard deviation exceeding its corresponding threshold parameter is extracted as the observed time standard deviation; Obtaining a standard deviation coefficient using the observed abnormal data standard deviation and the observed time standard deviation; Comparing the standard deviation coefficient with a preset coefficient reference value; When the standard deviation coefficient is not lower than the preset coefficient reference value, it is determined that the time interval and the number of time points of the acquisition time points do not meet the minimum normal standard; When the standard deviation coefficient is lower than a preset coefficient reference value, it is determined that the time interval and the number of time points of the acquisition time points meet the minimum normal standard.

7. The vehicle-mounted environment perception system based on ultrasonic radar and millimeter wave radar as claimed in claim 4, characterized in that: The domain controller further includes: A working mode management module is configured to control the switching of the domain controller between different working modes according to different operating states and usage scenarios of the vehicle, wherein the working modes of the domain controller include: operating mode, low power mode, sleep mode and power-off mode; The time synchronization management module is configured to achieve time synchronization between the modules in the domain controller and the external devices in the radar detection unit.

8. The vehicle-mounted environment perception system based on ultrasonic radar and millimeter wave radar as claimed in claim 7, characterized in that: The data processing module performs fusion processing, specifically: Classify and integrate the received radar data, image data, laser data and positioning data, and perform pre-processing; Based on the synchronization time of the time synchronization management module, the pre-processed data is aligned, and the feature information in each type of data is extracted, and the feature information about the same target in different types of data is associated; Acquire multi-sensor fusion data containing different target objects as data samples, determine feature information and data types of the target objects, and determine a classification identification of the target objects based on the feature information and data types of the target objects; Inputting the classification identifier of the target object into a preset neural network for learning, determining the classification expression of the data classification identifier, and constructing a target recognition model of the target object based on the classification expression; The target object is classified and identified through the target recognition model to determine the category of the target object. At the same time, the associated feature information is fused to generate an information data set of the target object; Based on the recognition results, a tracking track is established for each target object, and the parameter data of the position, speed and direction of each target object at different times are recorded; According to the historical tracking trajectory and current parameter data of the target object, the future motion trajectory of the target object is predicted; Based on the tracking trajectory and prediction results of each target object, the vehicle environment perception information is generated.

9. The vehicle-mounted environment perception system based on ultrasonic radar and millimeter wave radar as claimed in claim 8, characterized in that: Based on the recognition results, a tracking trajectory is established for each target object, including: According to the classification identification of the target object, a corresponding target storage space is established to store the information data set of the target object; Construct a time axis according to the time series of data collection, map the collected data related to the target object onto the time axis according to the time series, and generate dynamic data of the target object; Extract the positioning data of each target object at different times from the dynamic data, combine the initial position of each target object, construct the moving trajectory of the target object, and determine the moving characteristics of the target object; At the same time, the trajectory change frequency of the target object in different time periods is analyzed to generate the pause characteristics of the target object; Determine the target time segment where each target object appears most frequently or stays the longest in the dynamic data, and generate position attributes based on the order and position of the target time segment in the entire dynamic data; When multiple target objects exist at the same time, the motion state and interaction behavior of each target object in a similar time period and position are analyzed to extract the interaction features between different target objects; The movement characteristics, pause characteristics, position attributes and interaction characteristics of the target objects are integrated to establish a movement data set corresponding to each target object, and the movement range of each target object is determined in combination with the environmental information of the scene.

10. The vehicle-mounted environment perception system based on ultrasonic radar and millimeter wave radar as claimed in claim 9, characterized in that: The control module dynamically adjusts the vehicle's driving state, specifically: Based on the generated vehicle environment perception information, combined with the vehicle's current location and destination information, the vehicle's driving path is planned; The driving path is adjusted based on the moving trajectory and range of the target object to avoid areas with potential dangers or areas that may cause driving obstruction; The vehicle's speed is determined based on the path planning results, the distance and relative motion status between the vehicle and surrounding target objects, as well as traffic regulations and road conditions. When the vehicle approaches a slow-moving vehicle or obstacle in front, the speed is reduced to maintain a safe distance.

Images