Vehicle surroundings sensing system

Abstract

The invention relates to a vehicle surroundings perception system. The proposed innovative vehicle surroundings perception system provides visual information to the driver of the vehicle based on radar sensing by using two radar modules, where the maximum perception field is greater than 300 degrees, typically superior to 330 degrees, the overall system cost is affordable, where each radar module has a field of view of 180 degrees, and the processing is performed on the radar modules. There is no explicit need for processing on the proposed domain controller or vehicle processing unit for automatic driving. The proposed system utilizes an artificial intelligence approach that uses radar point cloud data to classify objects around the vehicle. Classification supported by artificial intelligence is executed, and training is performed through an actual radar measurement labeling data set in advance. Further, the proposed system may alert the driver, provide information to the vehicle control system to make autonomous actions, and wirelessly transmit ambient data to the cloud. In addition, the proposed system may provide parking support.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a vehicle surround perception system consisting of two millimeter wave radar sensor modules, each having a field of view of 180 degrees and the ability to process sensor data. The proposed system provides surround vehicle perception while the vehicle is in motion. The proposed system utilizes AI methods for data processing to classify objects around the vehicle in question using radar point cloud data. BACKGROUND

[0002] The state of the art blind spot detection and rear radar monitoring systems are summarized in the following patents. If based on radar sensor systems, the current state of the art vehicle surround perception systems use one of two state of the art radar arrangements: two corner radar sensors plus a rear radar system in the middle of the rear side of the vehicle, or a simple two corner radar sensor system without a radar system in the middle of the vehicle, where each radar system covers a field of view of 120 degrees. Thus, the maximum perception field in the direction of travel is less than 300 degrees, typically 240 degrees. The proposed innovative vehicle surround perception system provides a maximum perception field of more than 300 degrees, or typically 330 degrees, with a total system cost comparable to the state of the art systems.

[0003] US7552012B2 Apparatus for detecting objects in a blind spot of a vehicle, introduces a two sensor approach for blind spot detection on one corner of a vehicle. The proposed two sensors have a field of view of less than or equal to 120 degrees in a constellation diagram.

[0004] US8527151B2 Side impact safety system with blind spot detection radar data fusion, introduces a motor vehicle blind spot detection system with a long range sensor covering a rear quadrant blind spot sensing area and a front quadrant side impact sensing area. The proposed blind spot detection sensor has a field of view of less than or equal to 120 degrees.

[0005] US20110291874A1 Vehicle radar system and method for detecting objects, introduces a vehicle radar system comprising a side radar sensor providing a detection area at a side of the vehicle and a rear radar sensor providing a detection area at a rear of the vehicle. Each of the side radar sensor and the rear radar sensor comprises a monopulse radar device having a field of view of less than or equal to 120 degrees.

[0006] US6680689B1 Method for determining object classification from side looking sensor data, introduces an object type of a three-dimensional object sensed by a long range sensor for detecting objects to the side of a transport vehicle. The object is classified as either a stationary object or a moving vehicle.

[0007] KR102065626B1 Vehicle radar system with blind spot detection, introduces a vehicle radar system for monitoring a blind spot of a vehicle. The system includes a radar transmitter mounted on the vehicle and a transmitting antenna coupled with the radar transmitter. The transmitting antenna transmits a patterned radiation to an area adjacent to the vehicle, the pattern including a first lobe and a second lobe. The proposed vehicle radar system has a field of view less than or equal to 120 degrees.

[0008] US20110025548A1 System and method for vehicle sensor fusion, introduces a system and method for tracking and evaluating targets sensed by one or more active safety sensors in a motor vehicle. The system and method tracks detected targets from one or more sensors as fusion tracks and determines the maturity and reasonableness of these fusion tracks for determining an appropriate response.

[0009] US20090167514A1 Combination radar backup assist and blind spot detector and method, introduces a system and method for detecting objects in a blind spot and behind a vehicle. According to the disclosed embodiment, the system includes a first sensor and a second sensor having a combined long / narrow detection antenna and a short / wide detection antenna. The proposed vehicle radar system has a field of view less than or equal to 120 degrees.

[0010] US20160252610A1 Blind spot radar system with improved semi-trailer tracking, introduces steps to provide a system configured to detect objects in proximity to a vehicle using radar. The system detects an object within a first portion of a blind spot area and is reconfigured to detect an object within a second portion of the blind spot area different from the first portion. The proposed vehicle radar system has a field of view less than or equal to 120 degrees.

[0011] US10732262B2 Apparatus and method for detecting sensor alignment in automotive detection systems, introduces a first sensor that transmits a first transmission signal into an area and receives a first reflection signal and generates a first reception signal. A second sensor transmits a second transmission signal into the area and receives a second reflection signal and generates a second reception signal. The proposed vehicle radar system has a field of view less than or equal to 120 degrees.

[0012] US11854397B2 Rear side blind spot warning system and method for a vehicle, introduces a rear side blind spot warning system for a vehicle, the system comprising: a sensor configured to sense position information and movement information of an external obstacle; a determiner configured to determine a type of the external obstacle located in a rear blind spot. The proposed vehicle radar system has a field of view less than or equal to 120 degrees.

[0013] US8552848B2 System and method for combining blind spot detection and rear cross path collision warning, comprising a vehicle equipped with a blind spot detection and rear cross path collision warning system. The proposed vehicle radar system has a field of view less than or equal to 120 degrees.

[0014] EP2972467B1 Vehicle radar system with blind spot detection, introduces a special position of the corner radar with a special orientation angle with respect to the symmetry axis of the vehicle. The proposed vehicle radar system has a field of view less than or equal to 120 degrees.

[0015] EP 3418768A1 Vehicle radar system comprising two radar sensor arrangements, introduces two radar sensor arrangements: a first radar sensor arrangement and a second radar sensor arrangement spaced apart by a distance (d). The proposed vehicle radar system has a field of view less than or equal to 120 degrees.

[0016] US7504986B2 Blind spot sensor system, introduces two devices: a first device for emitting a first radar beam and a second device for emitting a second radar beam. The radial field of view of the first radar beam is tilted opposite to the direction of travel of the motor vehicle, and the radial field of view of the second radar beam (beam 11) is oriented in a substantially vertical manner with respect to the direction of travel, so that the field of view of the radar beams at least partially overlaps and substantially covers the monitored area. The proposed vehicle radar system has a field of view less than or equal to 120 degrees. SUMMARY

[0017] The basic motivation of the present invention is to provide a new generation of vehicle surround perception system, providing a maximum perception field of more than 300 degrees, typically 330 degrees, with minimal system cost, good power consumption, and additional features compared to prior art solutions. It is proposed to use only two radar sensor modules, each with a 180 degrees field of view (FoV). Their arrangement, with a special positioning angle symmetric to the symmetry line of the vehicle, results in a surround perception system with more than 300 degrees coverage. Advantageously, the related signal processing for classifying moving objects (including object position and object dynamics) on the middle and side of the rear of the vehicle is performed for each half of the observation area within one 180 degrees radar sensor and for the second half of the observation area on the second part of the 180 degrees radar sensor, respectively, both being part of the proposed system. When compared to standard solutions using two blind spot detection radar sensors, the proposed vehicle surround perception system providing a maximum perception field of more than 300 degrees is implemented in an affordable way by providing better performance (larger perception). Prior art solutions use a domain controller and a central vehicle computing unit for signal processing, which makes it necessary to transfer radar sensor point cloud data using a high speed automotive communication protocol (like automotive Ethernet), which requires a specific wiring harness solution, and a more expensive corner radar connector with 4 or more pins. This has to be assumed for prior art corner radar sensors (same as rear radar sensors) used to provide features for autonomous driving, which is not necessary for driver perception. The proposed solution uses an affordable, low throughput automotive communication protocol, such as the CAN protocol, with a low cost 4 pin connector. Furthermore, the proposed two radar sensors with 180 degrees field of view perform more processing than prior art blind spot detection radar sensors, including full processing of point cloud data information, whereas prior art blind spot radars mainly generate radar point cloud data, which is then externally processed.

[0018] Thus, the proposed innovative solution for vehicle perception provides a greater maximum perception field than the state-of-the-art solutions (more than 300 degrees, typically up to 330 degrees) and its goal is the surrounding perception as the main application without the need for expensive external processing subsystems, which are usually required for autonomous driving. This innovative solution provides more safety and comfort for the driver without imposing the requirement for more expensive processing parts of the autonomous driving features (like domain controllers or vehicle AI sensor computing units). The proposed system uses predefined artificial pictures to provide visual information to the driver, which are dedicated to a specific set of classification categories. The proposed system can advantageously alert the driver. The proposed system can advantageously send information to the vehicle control system in order to initialize autonomous dynamic changes of the vehicle. The proposed system can advantageously send the surrounding perception information wirelessly to the cloud for evaluation by other systems or personnel. Moreover, the proposed system can have optional features related to parking application support. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 A state-of-the-art solution is shown, which can use state-of-the-art corner radars for blind spot detection for vehicle surrounding perception.

[0020] Figure 2 A proposed system for vehicle surrounding perception using 180-degree field of view radar sensors with their own digital processing capabilities is shown.

[0021] Figure 3 A comparison of the perception coverage of the proposed system for vehicle surrounding perception using 180-degree field of view compared to the state-of-the-art system is shown, with the same minimum detection distance in the rear area.

[0022] Figure 4 A classification of processing functions performed on 180-degree field of view radar sensor modules, which are a constituent part of the proposed system, is shown.

[0023] Figure 5 A possible implementation option of the proposed system is shown, showing the 180-degree field of view system part integrated in the vehicle environment.

[0024] Figure 6 A possible example of the perception information communication from the system to the driver on the display side is shown. DETAILED DESCRIPTION

[0025] Vehicle surround perception systems of the prior art are implemented using video sensing and video processing. These systems have certain drawbacks associated with video sensing as compared to radar sensing, such as visibility in fog, rain, or intense sunlight. Implementing a vehicle surround perception system by radar as a system can be done using corner radar sensors of the prior art, which are used for prior art autonomous driving features and as blind spot detectors. They are not used to classify targets around the vehicle using their own processing power, they transmit pre-processed data to a related domain controller or directly to the vehicle computing processing unit, where the processing is done and used with other video sensors, or in the simplest version, the blind spot radar detector makes a simple alert to the driver that something is in the blind angle. There is a need to provide radar-based vehicle surround perception systems to the driver, but also to provide a system that is affordable and for the driver's perception without the need to integrate into a more complex autonomous driving vehicle infrastructure. In other words, a vehicle surround perception system that benefits the driver, where those benefits can be provided to the driver without the need for a more expensive relationship with processors that support autonomous driving, and without the need to fuse video and radar, but still provide use in all weather conditions (e.g., fog, heavy rain, complete darkness, or very intense sunlight in certain directions that blinds the video sensors).

[0026] Figure 1a) shows a possible vehicle surround perception system with state of the art corner radars. The vehicle 1 has corner radars 10 located at the rear of the vehicle 1, each corner radar 10 has a field of view (FoV) of 120 degrees. The state of the art radar corner system 10 has a 12 degree angle between the printed circuit board with the planar antenna for the radiation of the corner radar 10 and the vehicle 1. The corner 11 is the field of view of the radiation pattern of the corner radar with approximately 120 degrees. In this constellation the left border of the field of view of the left corner radar 10 and the right border of the right corner radar 10 build the corner 13. The corner 13 shows the area that cannot be covered by the observation of the state of the art solution for vehicle perception. In this arrangement, if the corner 12 is symmetrical with 45 degrees, then the corner 13 is 150 degrees. Therefore, in an area of 150 degrees out of 360 degrees, it is not possible to have vehicle perception. If the corner 12 is minimized to 30 degrees, then the vehicle 1 has no surround perception in an area of 120 degrees and has perception in an area of 240 degrees. However, in the latter case there is a significant gap of 20 degrees in the sensing directly behind the vehicle and objects very close to the rear of the vehicle 1 cannot be detected. Therefore, using state of the art radar sensing technology to minimize the corner 13 requires a sacrifice of minimal direct detection in the actual direction of the vehicle 1. Another way to achieve additional radar sensors in the middle of the rear of the vehicle 1 is to fill the gap 20. However, this increases the total system cost. Furthermore, the state of the art corner radar sensor 11 performs pre-processing and provides point cloud radar data to a domain controller 18 and also to a main vehicle processing unit 19. The unit 19 makes the final classification of the objects by using state of the art methods. Therefore, the state of the art vehicle surround perception system with object classification requires two versions of the external processing unit, either with a domain controller 18 and a main processing unit 19 or only with a main processing unit 19 which is below 10 Mbit / s. The main processing unit usually performs radar video fusion and is essentially expensive and supports a certain level of vehicle autonomous driving. If the state of the art corner radar 11 is used without the processing unit 19, then in this case they can only detect objects using the processing power of each corner radar 11 itself, without providing a classification of the objects in a certain class. Figure 1b) summarizes the construction of the functional parts of the prior art corner radar sensor 10. The prior art corner radar sensor 10 has an RF part 14 comprising an antenna system and a millimeter wave RF chip and a radar point cloud processing unit 15. As a specific implementation option of the entity 15, the radar point cloud processing unit 15 can be part of an integrated radar integrated circuit with both the RF chip part and the entity 15 in one silicon. The radar point cloud processing unit 15 advantageously has parts of the radar sensor point cloud computation computed by hardwired processing and parts using soft processing. The prior art corner radar sensor 10 has a more complex implementation solution digital interface of the entity 16 enabling high speed digital data connection at more than 10 Mbit / s. In many cases, the entity 16 is issued as a car Ethernet interface allowing hundreds of Mbit / s in data transition. The entity 17 has a low throughput car digital interface of less than 10 Mbit / s, in many cases implemented with the CAN protocol family or in the simplest form by the LIN protocol family. Usually, the support of the entity 16 and the entity 17 is supported by the prior art corner radar sensor 10 in a way that it uses a 10 pin connector. If the prior art corner blind spot detection sensor 10 has only a 4 pin connector supporting only the low speed interface 17, usually the CAN protocol, then this prior art corner blind spot detection sensor 10 does not provide a classification of the objects on its module. It simply sends on the entity 17 the information that there is an object in the blind spot detection area, which area covers a 120 degree field of view with a detection of up to 50 meters, without getting information about what kind of object is detected.

[0027] Figure 2 The proposed vehicle surrounding perception system 100 is shown, comprising two radar sensor systems 101 located symmetrically at the rear of the vehicle 1 with an angle 102 to the side of the vehicle 1, wherein each radar sensor system 101 has an observation field of view of 180 degrees. With respect to the non-observation area determined by the angle 103, the observation area will be 270 degrees in case we have an angle 102 of 45 degrees. In the same constellation, in case of an angle 102 of 90 degrees, the observation area will be 180 degrees. Figure 1 The prior art method described in has a non-observation area of 150 degrees and an observation area of 210 degrees. Furthermore, Figure 2 The proposed solution of has a significantly lower blind area in the rear of the vehicle. The gap 200 is smaller than the gap 20 in case of the prior art solution, therefore, in case of the proposed solution of the ability to detect objects in the middle of the rear of the vehicle 1 is greater. If the observation quality quality is related to the minimum detection distance to the center of the vehicle 1 and if it is set to 50 meters for the proposed solution of then the observation quality quality is 100% in the proposed solution of and 50% in the prior art solution of. Figure 2 Figure 1 Figure 2 ​​The newly proposed system is the same as the prior art system, then the observation area of the prior art solution will be 120 degrees worse than the proposed solution in this case, as can be observed in Figure 3 The proposed system 100 has clear operational advantages over the prior art solution. Figure 2 b) outlines the functional blocks of the 180-degree millimeter wave sensor 101. It can be observed that the functional entities 14, 15 and 17 are identical in the prior art corner sensor 11 and the 180-degree millimeter wave sensor 101 by their multiple implementation options. However, the 180-degree millimeter wave sensor 101 uses a functional entity 104 to provide a classification of the targets using the input of entity 15. Instead of sensing a radar point cloud to be processed for classification in entities 18 and 19, the 180-degree millimeter wave sensor 101 advantageously performs the processing for classification on its own module, so that entities 18 and 19 are obsolete for the use of applications for vehicle surrounding perception. In other words, this means that the proposed vehicle surrounding system 100 can be used in vehicles that do not necessarily have an autonomous driving function, which is required by the expensive entity 19, but still keep providing important information to the driver, improving overall safety and overall comfort in an affordable way. These application features enable the proposed vehicle surrounding system 100 to be deployed independently of the autonomous driving environment. The 180-degree millimeter wave sensor 101 does not necessarily need to have a 10-pin connector to support high-speed data exchange on entity 16, advantageously using a simple 4-pin connector and the prior art established vehicle network using the CAN protocol family. This requires less wiring harness and less overall system cost compared to the prior art solution, combined with lower maintenance costs. As Figure 4 shown, the classification entity 104 performs classification of the detected objects in different classification implementation options. The classification process is a signal processing process using artificial intelligence (AI) methods, arbitrary algorithmic methods and multi-stage (AI) methods, which are advantageously set to use minimal processing resources with minimal memory and processing power requirements. In the AI method, training information for different classification levels is fed to the system, which allows the AI module in entity 104 to perform target classification. Using a neural network method, machine learning with optional deep learning is deployed. The training information is twofold: a set of information defined as actual cloud information set using numerical data: relating to the length range of the object, the speed range of the object and the size range of the object, where the point cloud data processing estimates these values. The point cloud data clustering is acquired by actual, intentional radar sensor measurements of the objects as part of a specific classification class. Figure 4a) shows the basic classification categories, allowing for a minimum of artificial intelligence (AI) computing resources and minimum system costs by still providing the driver with a sufficient minimum set of information of the proposed vehicle surroundings perception system 100. The entity 104 provides three groups of classifications using the classification entity 105 for passenger cars, the classification entity 106 for commercial vehicles (involving medium-sized commercial vehicles like trucks) and the classification entity 107 for motorcycle detection. These basic classification features are proposed as a basic selection of perception, capable of giving the driver the most basic information while driving: what is around the vehicle 1. Figure 4 b) shows an extended classification set to the basic classifications, including an additional classification by the entity 108 for bicycle detection and an additional classification by the entity 109 for classification of very long commercial vehicles with trailers. Figure 4 b) shows an extended classification set to the basic classifications, including an additional classification by the entity 108 for bicycle detection and an additional classification by the entity 109 for classification of very long commercial vehicles with trailers. Figure 3 b) shows an extended classification set to the basic classifications, including an additional classification by the entity 108 for bicycle detection and an additional classification by the entity 109 for classification of very long commercial vehicles with trailers. The classification entity 110 for detecting trikes and the classification entity 111 for classifying small commercial vehicles with a size smaller than trucks. The classification of objects around the vehicle 1 is performed by pre-computed point cloud data, wherein an artificial intelligence method is used. In addition to the radar point cloud data, the artificial intelligence method also utilizes the detected speed of the objects. In the entity 15, the computed radar point cloud data passes through an artificial intelligence function. The artificial intelligence function is pre-trained by labeled radar system data related to the classification categories. The labeled radar system data is pre-recorded and used for the artificial intelligence function, wherein the artificial intelligence function uses state-of-the-art computing methods and state-of-the-art algorithms. With minimal computing and memory usage, the selection of state-of-the-art computing methods and state-of-the-art algorithms is advantageously made, thereby intentionally dealing with detection accuracy trade-offs. Specifically, the artificial intelligence method is a combination of more than one state-of-the-art algorithmic method: Support Vector Machine (SVM) with decision trees, Multi-Layer Perceptron (MLP), Convolutional Neural Network (CNN) and Vision Transformer (ViT), all of which were invented for video processing, wherein they are applied only on radar point cloud signals and not on video signals.

[0028] The proposed 180-degree field of view radar sensor 101 has an entity 114 providing tracking of objects. All information received from the entity 101 about the perception of the surroundings of the vehicle is continuously transmitted, advantageously through the entity 17 to a display 113 in front of the driver. Figure 5 a) shows a front view of one actual implementation option of the millimeter wave radar sensor 101 with a 180-degree field of view as part of the proposed vehicle surroundings perception system 100. Figure 5 b) shows a possible position of the millimeter wave radar sensor 101 with a 180-degree field of view on the rear vehicle corner, wherein the CAN is connected to a display 113 in front of the driver.Figure 5 c) shows a millimeter wave radar sensor 101 with a 180 degrees field of view from the top of the corner of the vehicle, showing a 180 degrees field of view 102. The millimeter wave radar sensor 101 with a 180 degrees field of view is able to detect and classify objects at 50 meters away by using printed antenna strings with more than 5 patches in the antenna string, by multiple radiating patches implementation options.

[0029] Figure 6 A possible example of the communication of the perception information from the system to the driver on the display side is shown. The driver gets visual information of the artificial pictures related to the classification results of the system. The artificial pictures are positioned with their moving direction and relative speed at a specific distance and angle relative to the vehicle 1. The proposed surround perception system 100 can also advantageously be used to alert the driver in the case where the distance of a moving object to the vehicle 1 is less than a prescribed value. An alarm can also be emitted in the case where the proposed system 100 has predicted, based on the real-time computed position, speed and moving direction, that any vehicle around the vehicle 1 will hit the vehicle 1 or will come very close to the vehicle 1 in the future. Instead of or in parallel to sending the alarm related to the application scenario to the driver, the proposed system 100 can send the alarm to the control infrastructure of the vehicle 1. This control infrastructure can use the alarm to enforce a change of the vehicle dynamics. Changing the dynamics of the vehicle 1 can be a combination of one or more than one of the following activities: braking, acceleration, change of speed from one value to another, lane change, and general change of direction without changing speed and acceleration.

[0030] The information about the surround perception of the vehicle 1 acquired by the proposed system 100 can advantageously be transmitted to the cloud using the wireless equipment of the vehicle 1. Using the vehicle 1, the information of the cloud can be further processed and used for overall traffic monitoring or driving behavior analysis. This information on the cloud can also be used for commercial purposes, allowing the introduction of various commercial processes using the data provided.

[0031] Advantageously and in addition to providing a vehicle surround perception system, the proposed system 100 with the 180 degrees radar sensor 101 can provide parking support for the vehicle 1. The 180 degrees radar sensor 111 can track objects close to the vehicle 1, for example position, speed, angle relative to the vehicle 1, so that parking of the vehicle 1 can be performed automatically. Advantageously, the parking support uses the processing on the digital processing unit 114 on one or both of the entities 101.

[0032] In another aspect, a method for providing vehicle surround perception using the above system is provided, implemented in a millimeter wave sensor signal processing hardware (HW). It should be noted that all aspects and technical features related to the system reflect the functionality of the method. It should be noted here that this processing does not take place on a central unit, for example a computer of the vehicle. Rather, the processing takes place on the system itself, in particular on the millimeter wave sensor modules.

[0033] Further, a computer program product is provided, the computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method according to the previous aspect.

[0034] Further aspects and examples are found in the following numbered clauses:

[0035] 1. A system for providing vehicle surround perception, comprising:

[0036] two millimeter wave radar sensor modules, each millimeter wave radar sensor module having an operational field of 180 degrees, wherein the operational field is defined by a 6dB antenna radiation diagram drop, each millimeter wave radar sensor module having the same hardware components and the same respective hardware arrangement inside the millimeter wave sensor module, and each millimeter wave radar sensor module accessing a display through a communication network of the vehicle, wherein the display is located within the driver’s field of view;

[0037] wherein each millimeter wave radar sensor module is located on a rear corner of the vehicle, wherein the rear side of the vehicle is defined as opposite to the main driving direction side;

[0038] wherein the radiation pattern in an elevation plane of the millimeter wave radar sensor module has a maximum radiation in a horizontal plane parallel to the ground plane;

[0039] wherein the radiation pattern in an azimuth plane of the millimeter wave radar sensor module has a maximum radiation in a plane perpendicular to the back side of the millimeter wave radar sensor module;

[0040] wherein the back side of one millimeter wave radar sensor module builds an angle of more than 60 degrees with the back side of the other millimeter wave radar sensor module;

[0041] wherein at least one millimeter wave radar sensor module has a digital processing hardware functionality, using radar sensor point cloud data pre-generated by other functionalities within the millimeter wave radar sensor module, performing classification of objects detected by the millimeter wave radar sensor module;

[0042] wherein the at least one millimeter wave radar sensor module transmits classification information about objects surrounding the vehicle, distance of objects to the vehicle, relative position of objects to the vehicle and relative speed of the vehicle; wherein the basic classification categories are: passenger cars, large commercial vehicles with a length of not more than 10 meters and motorcycles;

[0043] wherein the classification information is physically transmitted by the at least one millimeter wave radar sensor module through a 4-pin connector;

[0044] wherein a display located within the driver's field of view uses the classification information and shows the dynamic position of the objects, wherein the objects are represented by artificial pictures and are related to the basic classification categories, wherein the total field of view in the azimuthal plane covers more than 300 degrees.

[0045] 2. System according to clause 1, wherein the millimeter wave sensor module has a connector with more than 4 pins.

[0046] 3. System according to clause 2, wherein the basic classification categories are extended to the detection of bicycles.

[0047] 4. System according to clause 3, wherein the basic classification categories are extended to the detection of long commercial vehicles with a length of more than 10 meters.

[0048] 5. System according to clause 4, wherein the basic classification categories are extended to the detection of three-wheeled commercial vehicles with a length of not more than 3 meters.

[0049] 6. System according to the preceding clauses,

[0050] wherein the classification of objects surrounding the vehicle with the system is performed by an artificial intelligence function using pre-computed point cloud data,

[0051] wherein the artificial intelligence function makes the classification decision, the artificial intelligence function is pre-trained by radar system labeled data, wherein the radar system labeled data is related to the classification categories,

[0052] wherein the speed of the objects surrounding the vehicle is considered in the course of the artificial intelligence computation in combination with the point cloud radar data.

[0053] 7. System according to the preceding clauses, wherein the surround perception information is continuously updated on a display and is visible to the driver.

[0054] 8. System according to the preceding clauses, wherein the surround perception information is used to issue a warning to the driver.

[0055] 9. System according to clause 7, wherein the surround perception information is transmitted to the cloud by a wireless connection of the vehicle for access by other systems.

[0056] 10. The system according to the preceding clause, wherein the surrounding perception information is used by the vehicle infrastructure to automatically perform an action, wherein the action changes the dynamic behavior of the vehicle.

[0057] 11. The system according to the preceding clause, wherein the surrounding perception information is additionally used for parking support and provides distance warnings to nearby objects of the vehicle, without performing a classification of the nearby objects.

[0058] 12. A method implemented in a millimeter wave sensor signal processing hardware for providing vehicle surrounding perception using the system according to any of the preceding clauses, the method comprising:

[0059] performing a classification of objects surrounding the vehicle to obtain classification information; and

[0060] using the classification information to show the dynamic position of the objects relative to the vehicle.

[0061] 13. The method according to clause 12, wherein the classification of the objects surrounding the vehicle is performed by an artificial intelligence function using pre-computed point cloud data,

[0062] wherein the artificial intelligence function makes the classification decision, the artificial intelligence function being pre-trained by radar system labeled data, wherein the radar system labeled data is related to the classification categories,

[0063] wherein the speed of the objects surrounding the vehicle is considered in combination with the point cloud radar data in the course of the artificial intelligence computation.

[0064] 14. The method according to any of clauses 12 to 13,

[0065] wherein the artificial intelligence function uses more than one algorithmic method applied on the radar point cloud data: support vector machine (SVM) with decision tree, multi-layer perception (MLP), convolutional neural network (CNN) and vision transformer (ViT).

[0066] 15. A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any of clauses 12 to 14.

Claims

1. A system for providing vehicle surroundings perception, comprising: Two millimeter-wave radar sensor modules, each with a 180-degree operating field defined by a 6dB antenna radiation pattern drop, each having identical hardware components and an identical corresponding hardware arrangement within the millimeter-wave sensor module, and each accessing a display unit via the vehicle's communication network, wherein the display unit is located within the driver's field of vision. Each of the millimeter-wave radar sensor modules is located on the rear corner of the vehicle, wherein the rear side of the vehicle is defined as opposite to the main driving direction side; The radiation pattern of the millimeter-wave radar sensor module in the elevation plane has the maximum radiation in the horizontal plane parallel to the ground plane; The radiation pattern of the millimeter-wave radar sensor module in the azimuth plane has the maximum radiation in the plane perpendicular to the back side of the millimeter-wave radar sensor module. In this configuration, the back side of one of the millimeter-wave radar sensors forms an angle greater than 60 degrees with the back side of another millimeter-wave radar sensor module. At least one of the millimeter-wave radar sensor modules has digital processing hardware functions to perform classification on objects detected by the millimeter-wave radar sensor module using radar sensor point cloud data pre-generated by other functions within the millimeter-wave radar sensor module. At least one of the millimeter-wave radar sensor modules transmits classification information about objects around the vehicle, the distance of the objects from the vehicle, the relative position of the objects to the vehicle, and the relative speed of the vehicle; wherein the basic classification categories are: passenger cars, large commercial vehicles with a length not exceeding 10 meters, and motorcycles; The classification information is physically transmitted from at least one of the millimeter-wave radar sensors via a 4-pin connector. The display unit located within the driver's field of vision uses the classification information and displays the dynamic position of the object, wherein the object is represented by an artificial image and is associated with the basic classification category, and wherein the total field of vision coverage in the orientation plane is greater than 300 degrees.

2. The system according to claim 1, wherein, The millimeter-wave sensor module has a connector with more than four pins.

3. The system according to claim 2, wherein, The basic classification categories were extended to include bicycle detection.

4. The system according to claim 3, wherein, The basic classification categories have been expanded to include the detection of long commercial vehicles exceeding 10 meters in length.

5. The system according to claim 4, wherein, The basic classification category has been expanded to include the detection of three-wheeled commercial vehicles with a length not exceeding 3 meters.

6. The system according to the preceding claims, in, The artificial intelligence function uses pre-computed point cloud data to perform classification on objects around the vehicle equipped with the system. The artificial intelligence function makes classification decisions, and this function is pre-trained using labeled data from the radar system, wherein the labeled data from the radar system is related to the classification category. In the process of artificial intelligence calculation, the speed of objects around the vehicle is considered in conjunction with point cloud radar data.

7. The system according to the preceding claims, wherein, The surrounding perception information is continuously updated on the display and is visible to the driver.

8. The system according to the preceding claims, wherein, Surrounding awareness information is used to warn the driver.

9. The system according to claim 7, wherein, The surrounding perception information is transmitted to the cloud via the vehicle's wireless connection for access by other systems.

10. The system according to the preceding claims, wherein, The vehicle infrastructure uses surrounding perception information to automatically perform actions, wherein the actions alter the dynamic behavior of the vehicle.

11. The system according to the preceding claims, wherein, The surrounding perception information is also used for parking support and provides distance warnings to nearby objects to the vehicle without the need to classify those nearby objects.

12. A method implemented in millimeter-wave sensor signal processing hardware for providing vehicle surroundings perception using a system according to any one of the preceding claims, the method comprising: Perform classification on the objects around the vehicle to obtain classification information; as well as The classification information is used to indicate the dynamic position of the object relative to the vehicle.

13. The method according to claim 12, wherein, The classification of objects around the vehicle is performed using pre-computed point cloud data by artificial intelligence functions. The artificial intelligence function makes classification decisions, and this function is pre-trained using labeled data from the radar system, wherein the labeled data from the radar system is related to the classification category. In the process of artificial intelligence calculation, the speed of objects around the vehicle is considered in conjunction with point cloud radar data.

14. The method according to any one of claims 12 to 13, in, The artificial intelligence function uses more than one algorithmic approach applied to the radar point cloud data: Support Vector Machine (SVM) with decision tree, Multilayer Perception (MLP), Convolutional Neural Network (CNN), and Vision Transformer (ViT).

15. A computer program product comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 12 to 14.

Images