A road perception method and system based on millimeter wave radar

Abstract

The present invention discloses a road perception method and system based on millimeter wave radar, comprising the following steps: S1, vehicle detection of whether there is a base station in the surrounding environment; S2, judging whether networking is required; S3, performing signal analysis on the network formed in step S2, judging the networking status, and if the networking is normal, performing target detection processing, and if the networking is abnormal, continuing to detect the surrounding base stations that can be networked; S4, classifying the detected targets, and performing behavior analysis and status analysis on the classified targets respectively; S5, judging whether the target will affect driving, and continuing to detect if it will not affect normal driving. The present invention adopts the above-mentioned road perception method and system based on millimeter wave radar, which can not only detect road conditions, but also use radio frequency networking communication between vehicles, use multiple base stations to perform networking detection, and share communication results, thereby improving the accuracy of detection and expanding the detection range.

Description

Technical Field

[0001] The present invention relates to the field of road perception technology, and in particular to a road perception method and system based on millimeter wave radar. Background Art

[0002] As the trend of intelligentization in the automotive industry grows, the demand for intelligent road driving is increasing. The most commonly used technology in automotive communications is road condition analysis based on image or laser technology. Image technology has many disadvantages, such as many blind spots, only processing matters within the visible range, and being greatly affected by extreme weather. Laser technology is expensive, has weak color and image recognition capabilities, and is greatly affected by extreme weather. Against this background, a road perception system based on millimeter-wave radar came into being. The system mainly uses millimeter-wave radio frequency signals for detection. Millimeter-wave radio frequency signals have significant advantages such as high frequency, large-capacity transmission, fast processing, strong anti-interference ability, strong penetration, and are less affected by extreme weather. Summary of the invention

[0003] The purpose of the present invention is to provide a road perception method and system based on millimeter-wave radar. By deploying base stations on vehicles, more data can be collected when there are more vehicles, and the estimation results of road conditions will be more accurate. Deploying fixed base stations in specific areas can effectively solve the problems of data collection, result prediction and network establishment when there is only one vehicle.

[0004] To achieve the above object, the present invention provides a road perception method based on millimeter wave radar, comprising the following steps:

[0005] S1. The vehicle detects whether there is a base station in the surrounding environment;

[0006] S2, determine whether networking is needed;

[0007] S3, perform signal analysis on the network formed in step S2 to determine the networking status. If the networking is normal, perform target detection processing. If the networking is abnormal, continue to detect surrounding base stations that can be networked;

[0008] S4, classifying the detected targets, and performing behavior analysis and status analysis on the classified targets;

[0009] S5. Determine whether the target will affect driving. If it will not affect normal driving, continue to test. If it will affect driving operation, feedback will be given to the driving system and operation prompts will be given to the driver.

[0010] Preferably, in step S2, the specific method for determining whether networking is needed is as follows: first, the vehicle detects whether there is a base station in the surrounding environment. If not, continue to detect. If yes, determine the type of base station. If it is a vehicle mobile base station, establish an ad hoc network between vehicles. If there is a ground base station, establish an ad hoc network between vehicles. If there is neither a vehicle nor a ground base station nearby, establish a single vehicle ad hoc network through radio frequency reflection.

[0011] Preferably, the establishment of the single-vehicle ad hoc network is specifically as follows: the vehicle generates a radio frequency signal through a radio frequency signal transceiver, and when there are no ground base stations and other vehicles around, a self-organizing network is formed based on the radio frequency signals reflected back from surrounding objects, and then network analysis and network detection are performed.

[0012] Preferably, in step S3, the target detection process includes radio frequency-based motor vehicle detection, radio frequency-based pedestrian detection and radio frequency-based obstacle detection. The radio frequency signal motor vehicle detection is based on the radio frequency signal field formed by the vehicle and the ground base station and the radio frequency signal field formed by the vehicle and the vehicle, and the vehicle in the signal field is detected, and then the vehicle status is analyzed. According to the analysis result, it is determined whether there will be a conflict with the vehicle. If there will be no conflict, the vehicle detection will continue. If there will be a conflict, timely feedback will be given to the driving system for driving operation prompts;

[0013] The pedestrian detection and pedestrian behavior analysis based on radio frequency signals is based on the radio frequency signal field formed by the vehicle and the ground base station and the radio frequency signal field formed by the vehicle and the vehicle, and the pedestrians in the signal field are detected, and then the pedestrian status is analyzed. According to the analysis result, it is determined whether it will affect driving. If it will not affect driving, pedestrian detection is continued. If it will affect driving, driving operation prompts are given;

[0014] The RF-based obstacle detection specifically detects all objects in the range according to the RF signal, then regards all objects except vehicles and pedestrians as road obstacles, and then analyzes the status of the road obstacles, and determines whether there will be a conflict with the vehicle based on the analysis results. If there will be no conflict, obstacle detection will continue. If there will be a conflict, timely feedback will be given to the driving system for driving operation prompts.

[0015] Preferably, in step S3, before target detection, it also includes collecting information on all objects in the radio frequency signal field, and then mining the collected data, extracting feature information for object recognition and classification, system prediction and estimation, and the system prompts the driver to complete the vehicle's estimation of the road condition.

[0016] Preferably, the vehicle status analysis includes vehicle movement direction judgment, distance judgment from the vehicle, vehicle type judgment and danger judgment; the pedestrian status analysis includes pedestrian movement state judgment, pedestrian position direction judgment, pedestrian behavior analysis, pedestrian danger judgment during vehicle driving and person detection under vehicle-to-vehicle and vehicle-to-ground interconnection; the road obstacle status analysis includes obstacle size judgment, obstacle movement state judgment, obstacle distance and movement direction judgment.

[0017] The present invention also provides a road perception system based on millimeter wave radar, including a vehicle radio frequency system, wherein the vehicle radio frequency system includes a radio frequency signal transceiver, a radio frequency signal power supply device and an embedded computer system; the radio frequency signal transceiver is used to output radio frequency signals and receive input radio frequency signals, and collect original data; the radio frequency signal power supply device is directly connected to the internal power supply of the vehicle; the embedded computer system analyzes the radio frequency signal, and then outputs the analysis and prediction results.

[0018] Preferably, the embedded computer system includes a signal receiving module, a signal preprocessing module, a detection and classification module, an analysis and prediction module and a screen prompt module;

[0019] The signal receiving module is used to receive radio frequency signals;

[0020] The signal preprocessing module is used to perform data preprocessing on the received signal;

[0021] The detection and classification module is used to build a model for the processed data, extract feature values ​​for object detection, and then classify the detected objects;

[0022] The analysis and prediction module performs different analyses according to different objects, makes predictions according to the analysis results, and feeds back the prediction results to the screen prompt module to give the driver a driving reminder;

[0023] The screen prompt module is used to remind the driver of the detected abnormal object information so that the driver can take risk avoidance operations.

[0024] Preferably, the data preprocessing includes performing mixing operations, filtering operations, ADC conversion, range Fourier transform and Doppler Fourier transform on the received data.

[0025] Therefore, the present invention adopts the above-mentioned road perception method and system based on millimeter wave radar, and the beneficial effects are as follows:

[0026] (1) The present invention deploys base stations on vehicles. When there are more vehicles, more data can be collected and the estimation results of road conditions can be more accurate. Deploying fixed base stations in specific areas can effectively solve the problems of data collection, result prediction and network establishment when there is only one vehicle.

[0027] (2) The present invention can be well applied to vehicle driving. It can not only detect road conditions, but also use radio frequency networking to communicate between vehicles, and then use multiple base stations to perform networking detection, and finally communicate and share the results, thereby improving the accuracy of detection and expanding the detection range.

[0028] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is an overall workflow diagram of an embodiment of a road perception method and system based on millimeter wave radar of the present invention;

[0030] Figure 2 It is a flow chart of establishing a self-organizing network between vehicles and between vehicles and ground base stations in a method and system embodiment of a road perception method based on millimeter wave radar according to the present invention;

[0031] Figure 3 It is a flow chart of a single vehicle forming a radio frequency signal self-organizing network diagram according to an embodiment of a road perception method and system based on millimeter wave radar of the present invention;

[0032] Figure 4 It is a working principle diagram of radio frequency signal reflection of a road perception method and system embodiment based on millimeter wave radar of the present invention;

[0033] Figure 5 It is a flow chart of motor vehicle detection based on radio frequency according to an embodiment of a road perception method and system based on millimeter wave radar of the present invention;

[0034] Figure 6 It is a flow chart of pedestrian detection based on radio frequency and character behavior analysis in an embodiment of a road perception method and system based on millimeter wave radar of the present invention;

[0035] Figure 7 It is an overall structural diagram of a vehicle radio frequency system of an embodiment of a road perception method and system based on millimeter wave radar of the present invention;

[0036] Figure 8 It is an overall architecture diagram of an embedded computer system of an embodiment of a road perception method and system based on millimeter wave radar of the present invention;

[0037] Fig. 9It is a model diagram of a radio frequency signal transmitting base station of a road perception method and system embodiment based on millimeter wave radar of the present invention;

[0038] Fig.10 This is a vehicle-to-vehicle radio frequency signal interconnection structure diagram of a road perception method and system embodiment based on millimeter wave radar of the present invention;

[0039] Fig.11 It is a schematic diagram of vehicle direction determination according to an embodiment of a road perception method and system based on millimeter wave radar of the present invention;

[0040] Fig.12 It is a schematic diagram of vehicle distance judgment according to an embodiment of a road perception method and system based on millimeter wave radar of the present invention;

[0041] Fig.13 It is a schematic diagram of pedestrian motion state analysis in an embodiment of a road perception method and system based on millimeter wave radar of the present invention;

[0042] Fig.14 It is a schematic diagram of pedestrian position and direction determination according to an embodiment of a road perception method and system based on millimeter wave radar of the present invention;

[0043] Fig.15 It is a schematic diagram of a top view of a road perception method and system embodiment based on millimeter wave radar of the present invention, showing a person detection diagram under vehicle-to-vehicle and vehicle-to-ground interconnection;

[0044] Fig.16 It is a schematic diagram of obstacle motion condition judgment in an embodiment of a road perception method and system based on millimeter wave radar of the present invention;

[0045] Fig.17 It is a schematic diagram of obstacle distance and movement direction judgment in an embodiment of a road perception method and system based on millimeter wave radar of the present invention;

[0046] Fig.18 It is a radar signal processing flow chart of an embodiment of a road perception method and system based on millimeter wave radar of the present invention;

[0047] Fig.19 It is a distance Fourier transform simulation diagram of a road perception method and system embodiment based on millimeter wave radar of the present invention;

[0048] Fig. 20 It is a simulation diagram after Doppler Fourier transformation of a road perception method and system embodiment based on millimeter wave radar of the present invention;

[0049] Fig.21 It is a schematic diagram of Lora communication between vehicles in an embodiment of a road perception method and system based on millimeter wave radar of the present invention;

[0050] Fig. 22 It is a Lora communication flow chart of a road perception method and system embodiment based on millimeter wave radar of the present invention. DETAILED DESCRIPTION

[0051] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0052] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0053] like Figure 1 As shown, a road perception method based on millimeter wave radar includes the following steps:

[0054] S1. The vehicle detects whether there is a base station in the surrounding environment;

[0055] S2. Determine whether networking is needed. The specific method is as follows: First, the vehicle detects whether there is a base station in the surrounding environment. If not, continue to detect. Figure 2 As shown, if there is a base station, the type of base station is determined. If it is a vehicle mobile base station, the vehicles establish a self-organizing network with each other. If there is a ground base station, the vehicle establishes a self-organizing network with the ground base station, such as Figure 3 As shown, if there is neither a car nor a ground base station nearby, a single-vehicle ad hoc network is established through radio frequency reflection. The establishment of a single-vehicle ad hoc network is specifically as follows: the vehicle generates radio frequency signals through a radio frequency signal transceiver. When there are no ground base stations or other vehicles around, an ad hoc network is formed based on the radio frequency signals reflected back by surrounding objects, and then network analysis and network detection are performed.

[0056] like Fig.21 As shown, the network is established through Lora technology for communication between vehicles, such as Fig. 22 As shown, Lora communication is specifically as follows: first, the vehicle detects whether there are any vehicles around that can be connected. If so, a connection is established for Lora communication, and then information is exchanged between vehicles, including basic information of the vehicle, the direction of the vehicle, the speed of the vehicle, and Beidou location information.

[0057] The basic principle of networking is to use radio frequency signal reflection. Figure 4As shown in the figure, base stations A, B and C all have the ability to transmit and receive radio frequency. Taking base station A as an example, when the radio frequency signal sent by base station A collides with an object, the signal will be reflected, scattered and absorbed. The reflected part will be received and utilized by base station A, and the scattered part will be absorbed and utilized by base stations B and C. The working principle of base stations B and C is the same as that of base station A, so that base stations A, B and C can detect objects respectively. When base stations A, B and C are within a certain range, anonymous networking will be automatically established to realize object detection data sharing, and finally the target vehicle will integrate the data to achieve the effect of data collection.

[0058] S3. Perform signal analysis on the network formed in step S2 to determine the networking status. If the networking is normal, perform target detection processing. If the networking is abnormal, continue to detect surrounding base stations that can be networked. Before target detection, it also includes information collection of all objects in the radio frequency signal field, and then perform data mining on the collected data, extract feature information for object recognition and classification, system prediction and estimation, and the system prompts the driver to complete the vehicle's estimation of road conditions.

[0059] Object detection processing includes RF-based vehicle detection, RF-based pedestrian detection, and RF-based obstacle detection, such as Figure 5 As shown, the radio frequency signal motor vehicle detection is based on the radio frequency signal field formed by the vehicle and the ground base station and the radio frequency signal field formed by the vehicle and the vehicle, and the vehicle in the signal field is detected, and then the vehicle status is analyzed. The vehicle status analysis includes the judgment of the vehicle movement direction, the distance from the vehicle, the vehicle type and the danger judgment; according to the analysis results, it is determined whether there will be a conflict with the vehicle. If there will be no conflict, the vehicle detection will continue. If there will be a conflict, timely feedback will be given to the driving system for driving operation prompts;

[0060] like Figure 6 As shown, the pedestrian detection and pedestrian behavior analysis based on radio frequency signals is based on the radio frequency signal field formed by the vehicle and the ground base station and the radio frequency signal field formed by the vehicle and the vehicle, and the pedestrians in the signal field are detected, and then the pedestrian status is analyzed. The pedestrian status analysis includes pedestrian movement status judgment, pedestrian position direction judgment, pedestrian behavior analysis, pedestrian danger judgment during vehicle driving, and person detection under vehicle-to-vehicle interconnection and vehicle-to-ground interconnection; according to the analysis results, it is determined whether it will affect driving. If it will not affect driving, pedestrian detection will continue. If it will affect driving, driving operation prompts will be given;

[0061] RF-based obstacle detection specifically detects all objects in range based on RF signals, then regards all objects except vehicles and pedestrians as road obstacles, and then analyzes the status of road obstacles. The road obstacle status analysis includes obstacle size judgment, obstacle movement status judgment, obstacle distance and movement direction judgment. Based on the analysis results, it is determined whether there will be a conflict with the vehicle. If there will be no conflict, obstacle detection will continue. If a conflict will occur, timely feedback will be given to the driving system for driving operation prompts.

[0062] S4, classifying the detected targets, and performing behavior analysis and status analysis on the classified targets;

[0063] S5. Determine whether the target will affect driving. If it will not affect normal driving, continue to test. If it will affect driving operation, feedback will be given to the driving system and operation prompts will be given to the driver.

[0064] like Figure 7 The present invention also provides a road perception system based on millimeter wave radar, including a vehicle radio frequency system, the vehicle radio frequency system includes a radio frequency signal transceiver, a radio frequency signal power supply device and an embedded computer system; the radio frequency signal transceiver is used to output radio frequency signals and receive radio frequency signal inputs, and collect raw data; the radio frequency signal power supply device does not require an external power supply, and is usually directly connected to the vehicle's internal power supply; the embedded computer system is a carrier of the algorithm software, analyzes the radio frequency signal, and then outputs the analysis and prediction results. This structural design is relatively simple and reasonable. In terms of appearance, it is embedded in the vehicle as a whole, which is highly concealed. It can collect various data inside the vehicle signal field without blind spots, and capture key information for analysis, processing, prediction and early warning.

[0065] like Figure 8 As shown, the embedded computer system includes a signal receiving module, a signal preprocessing module, a detection and classification module, an analysis and prediction module and a screen prompt module.

[0066] The signal receiving module is used to receive the radio frequency signal and store it inside the computer; the signal preprocessing module is used to perform data preprocessing on the received signal, such as Fig.18 As shown, data preprocessing includes mixing operations, filtering operations, ADC conversion, range Fourier transform, Doppler Fourier transform and other operations on the received data.

[0067] The distance, speed and angle of the target are detected by millimeter wave radar. Fig.18(a) shows the radar signal transmission flow chart, and (b) shows the radar signal reception process flow chart. First, the transmitted radar signal is mixed with the received radar signal by a mixing operation, and then the high-frequency signal after mixing is filtered out to obtain an intermediate frequency signal that is easy to analyze. Then, ADC conversion is performed to convert the analog signal into a digital signal for the subsequent Fourier transform processing. Fig.19 As shown in the figure, the distance peak spectrum is obtained by performing a close-range Fourier transform, which can intuitively show the distance of the target object. Fig. 20 As shown, Doppler Fourier transform is performed to obtain a multi-peak spectrum diagram so that multiple objects can be distinguished.

[0068] The detection and classification module uses a deep neural network method to build a model for the processed data, extract feature values ​​for object detection, and then classify and identify the detected objects.

[0069] The analysis and prediction module performs different analyses based on the different objects in the previous step, and makes predictions based on the analysis results. When an object is detected, it performs analysis and prediction. For example, if another vehicle is detected besides the vehicle, the vehicle's movement speed and direction are analyzed. Then, predictions are made based on the analysis results introduced above, such as whether the vehicle will change lanes, whether it will slow down significantly, and whether there is a risk of rear-end collision. If a pedestrian is detected, the pedestrian's movement trajectory is analyzed to analyze whether the pedestrian is at a safe distance from the vehicle, whether the pedestrian has suspicious behavior, etc. Feedback to the screen prompt module based on the prediction results gives the driver a driving reminder to achieve the role of assistance.

[0070] The screen prompt module is used to alert the driver of the detected abnormal object information so that the driver can take evasive action. For example, when a large object appears on the road and the driver can no longer drive, the driver will be reminded to brake to avoid danger. In addition, when various abnormal behaviors of pedestrians are detected, the driver will be prompted and given driving suggestions, so that the driver can take evasive actions at the first time and improve driving safety. Finally, it should be noted that the system will usually start when the car is started, and the screen display module will start after startup, and the screen will display system prompts.

[0071] Find a better RF signal transceiver on the market, install it on the car and build a system, install base stations at fixed locations on the road. The main equipment of the base station is RF signal generator, power supply device, etc.

[0072] Open the vehicle as Fig. 9 , Fig.10 As shown, the system establishes a radio frequency connection with surrounding vehicles or fixed base stations to form a signal field, automatically collects data on objects in the signal field, and performs subsequent processing through the embedded computer system in the vehicle.

[0073] Data is stored in the cloud and edge devices in a cloud-edge combined storage mode. The edge devices can process and respond to data in real time, while the cloud can provide long-term storage and advanced analysis capabilities. This storage method can improve the response speed and real-time performance of the system while taking into account the security and long-term accessibility of data. The local data storage time at the edge is greater than 30 days.

[0074] Taking the example of establishing radio frequency connections between vehicles to form a signal field, vehicle A establishes a connection with vehicle B, and collects data on all objects in the signal field. The more vehicles around, the more connections vehicle A can establish, the richer the data collected, the more accurate the analysis of surrounding road conditions, and the more effective the subsequent predictions. After collecting data, the system performs data mining on the data, extracts feature information for object recognition and classification, system prediction and estimation, and the system prompts the driver. According to this step, the vehicle completes the estimation of road conditions.

[0075] Embodiment 1

[0076] Vehicle status analysis mainly includes vehicle direction judgment, vehicle type judgment, distance judgment and danger judgment:

[0077] like Fig.11 As shown, vehicle direction determination is that target vehicle A establishes a coordinate axis with itself as the center, uses radio frequency positioning to detect vehicle B, and then uses an algorithm to compare the directions of the two vehicles to obtain the specific position of vehicle B relative to vehicle A, thereby achieving the function of vehicle direction identification. Then vehicles A and B communicate, such as Lora communication, which can transmit the location information obtained by vehicle A to vehicle B. Vehicle B confirms the location information to achieve two-way location notification. If there is a third vehicle detected by vehicle B, then vehicle A can also obtain the location of vehicle C through communication between vehicles, and so on, thereby expanding the judgment of vehicle position direction.

[0078] like Fig.12 As shown, the vehicle distance judgment is that the target vehicle A first detects vehicle B, and then determines the two distances between vehicles A and B based on the radio frequency positioning, which are the lateral distance d1 and the longitudinal distance d2. When the two vehicles are driving head-on, the distance d2 will continue to decrease until it is reduced to zero, and then slowly increase until it exceeds the range of the radio frequency signal field. If the two vehicles are getting closer during the driving process, the lateral distance d1 will be reflected quickly. When the vehicles are traveling in the same direction, there are two situations. The first is that the speed of vehicle A is faster than that of vehicle B, then the longitudinal distance between vehicles A and B will slowly decrease, and then slowly increase. In another situation, the speed of vehicle B is faster. The principle is similar to the previous situation, that is, if the two vehicles are getting closer during the driving process, the lateral distance d1 will be reflected quickly.

[0079] The vehicle model is determined by the target vehicle A first detecting vehicle B, and then calculating the size of vehicle B. The principle of calculating the size of the vehicle is to use an algorithm to calculate the volume of the two vehicles, then extract features for analysis, and finally compare with the database to determine the vehicle model.

[0080] Danger judgment is based on distance judgment and direction judgment. In distance judgment, when the lateral distance d1 and the longitudinal distance d2 are continuously reduced, a critical value will appear in the algorithm analysis. When the longitudinal distance d2 is less than half of the length of the car and the lateral distance d1 is less than half of the width of the car, an alarm will be issued, and it is judged to be a dangerous distance, otherwise it is a safe distance.

[0081] Embodiment 2

[0082] Pedestrian status analysis mainly includes pedestrian movement status judgment, pedestrian position and direction judgment, pedestrian behavior analysis, pedestrian danger judgment during vehicle driving, and person detection under vehicle-to-vehicle and vehicle-to-ground interconnection.

[0083] like Fig.13 As shown, the pedestrian motion state analysis is that the vehicle first uses radio frequency signals to detect pedestrians around it with itself as the center. When a pedestrian appears within the detection range, the pedestrian is tracked. If the pedestrian's position does not change within a certain period of time, it is judged that the pedestrian is not in a distance motion state, but whether he is in a behavioral motion is analyzed in the following steps. For example, a child can only play basketball in place. On the contrary, if the pedestrian's position changes, it can be judged that the pedestrian has a distance motion, such as a child playing basketball while walking. After that, a deeper analysis can be conducted to determine whether the pedestrian has performed behavioral motion while performing distance motion.

[0084] like Fig.14 As shown in the figure, the pedestrian position direction judgment, when the pedestrian position is detected to have changed, the pedestrian's direction can be analyzed. The principle used is similar to the judgment of the vehicle's movement direction. The vehicle establishes a coordinate axis with itself as the center, connects the line with the pedestrian to form an angle in the direction, and then makes a specific judgment on the task direction based on the algorithm.

[0085] Human behavior analysis is to individually judge pedestrian behavior after pedestrians are detected in the previous steps. It mainly uses deep learning models to detect and classify behaviors and predict behaviors.

[0086] Danger judgment is a combination of the above steps to make the final danger prediction. First, when the pedestrian's position and behavior have not changed, and the pedestrian's position and direction are far away from the moving vehicle, it is judged to be the safest. With the changes in movement status, direction status and behavior status, the danger index will continue to increase and an early warning will be issued.

[0087] like Fig.15As shown in the figure, human detection under vehicle-to-vehicle or vehicle-to-ground interconnection is to establish a network between vehicles or vehicles and ground base stations. When a vehicle detects a pedestrian and performs trajectory tracking prediction, it can communicate the pedestrian's motion information to another vehicle, or the ground base station detects the pedestrian and transmits the pedestrian's motion information to the vehicle, thereby expanding the perception range of the other vehicle and improving the safety warning capability.

[0088] Embodiment 3

[0089] Obstacle detection includes road obstacle detection except for vehicles and pedestrians. Road obstacle status analysis includes obstacle size judgment, obstacle movement status judgment, obstacle distance and movement direction judgment.

[0090] like Fig.16 As shown, the obstacle movement status judgment is that the vehicle first detects the obstacle. Similar to the principle of pedestrian movement status judgment, the vehicle establishes a coordinate axis with itself as the center, and tracks the obstacle after it appears within the detection range. If the obstacle does not change within a certain period of time, it means that the object has not changed, indicating that the object is stationary. If the object changes its position within a certain period of time, it means that the object is in motion. After making the above judgment, subsequent judgments are made.

[0091] like Fig.17 As shown in the figure, the obstacle distance and movement direction judgment is based on the previous step. When the object is detected to be stationary, the direction is confirmed. The lateral distance d1 and the longitudinal distance d2 between the vehicle and the object can be detected. When d1 is less than half the width of the vehicle and d2 is less than half the length of the vehicle, the system will issue an early warning, thereby changing the trajectory or making emergency braking operations. When the object is detected to be in motion, the deep learning algorithm is used to predict the future trajectory based on the historical motion trajectory to determine whether a dangerous accident will occur with the vehicle. If a safety accident occurs, an early warning is issued, so that the driver can change the trajectory or make emergency braking operations.

[0092] Therefore, the present invention adopts the above-mentioned road perception method and system based on millimeter wave radar, which can be well used in vehicle driving. It can not only detect road conditions, but also use radio frequency networking communication between vehicles, and then use multiple base stations to perform networking detection, and finally communicate and share results, thereby improving the accuracy of detection and expanding the detection range.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A road perception method based on millimeter wave radar, characterized in that: The following steps are involved: S1. The vehicle detects whether there is a base station in the surrounding environment; S2, determine whether networking is needed; S3, perform signal analysis on the network formed in step S2 to determine the networking status. If the networking is normal, perform target detection processing. If the networking is abnormal, continue to detect surrounding base stations that can be networked; S4, classifying the detected targets, and performing behavior analysis and status analysis on the classified targets; S5. Determine whether the target will affect driving. If it will not affect normal driving, continue to test. If it will affect driving operation, feedback will be given to the driving system and the driver will be prompted to operate. In step S2, the specific method for determining whether networking is required is as follows: first, the vehicle detects whether there is a base station in the surrounding environment. If not, continue to detect. If yes, determine the type of base station. If it is a vehicle mobile base station, establish an ad hoc network between vehicles. If there is a ground base station, establish an ad hoc network between vehicles. If there is neither a vehicle nor a ground base station nearby, establish a single vehicle ad hoc network through radio frequency reflection. The establishment of the single vehicle ad hoc network is specifically as follows: the vehicle generates a radio frequency signal through a radio frequency signal transceiver, and when there are no ground base stations and other vehicles around, an ad hoc network is formed according to the radio frequency signals reflected back by surrounding objects, and then network analysis and network detection are performed; The road sensing method based on millimeter wave radar includes a road sensing system based on millimeter wave radar, the road sensing system includes a vehicle radio frequency system, the vehicle radio frequency system includes a radio frequency signal transceiver, a radio frequency signal power supply device and an embedded computer system; the radio frequency signal transceiver is used to output radio frequency signals and receive radio frequency signal inputs, and collect raw data; the radio frequency signal power supply device is directly connected to the internal power supply of the vehicle; the embedded computer system analyzes the radio frequency signal, and then outputs the analysis and prediction results; The embedded computer system includes a signal receiving module, a signal preprocessing module, a detection and classification module, an analysis and prediction module and a screen prompt module; The signal receiving module is used to receive radio frequency signals; The signal preprocessing module is used to perform data preprocessing on the received signal; The detection and classification module is used to build a model for the processed data, extract feature values ​​for object detection, and then classify the detected objects; The analysis and prediction module performs different analyses according to different objects, makes predictions according to the analysis results, and feeds back the prediction results to the screen prompt module to give the driver a driving reminder; The screen prompt module is used to remind the driver of the detected abnormal object information so that the driver can take risk avoidance actions; The data preprocessing includes performing mixing operation, filtering operation, ADC conversion, range Fourier transform and Doppler Fourier transform on the received data.

2. The road perception method based on millimeter wave radar according to claim 1, characterized in that: In step S3, the target detection process includes RF-based motor vehicle detection, RF-based pedestrian detection and RF-based obstacle detection. The RF signal motor vehicle detection is based on the RF signal field formed by the vehicle and the ground base station and the RF signal field formed by the vehicle and the vehicle, and the vehicle in the signal field is detected, and then the vehicle status is analyzed. According to the analysis result, it is determined whether there will be a conflict with the vehicle. If there will be no conflict, the vehicle detection will continue. If there will be a conflict, timely feedback will be given to the driving system for driving operation prompts; The pedestrian detection and pedestrian behavior analysis based on radio frequency signals is based on the radio frequency signal field formed by the vehicle and the ground base station and the radio frequency signal field formed by the vehicle and the vehicle, and the pedestrians in the signal field are detected, and then the pedestrian status is analyzed. According to the analysis result, it is determined whether it will affect driving. If it will not affect driving, pedestrian detection is continued. If it will affect driving, driving operation prompts are given; The RF-based obstacle detection specifically detects all objects in the range according to the RF signal, then regards all objects except vehicles and pedestrians as road obstacles, and then analyzes the status of the road obstacles, and determines whether there will be a conflict with the vehicle based on the analysis results. If there will be no conflict, obstacle detection will continue. If there will be a conflict, timely feedback will be given to the driving system for driving operation prompts.

3. The road perception method based on millimeter wave radar according to claim 1, characterized in that: In step S3, before target detection, it also includes information collection of all objects in the RF signal field, and then data mining is performed on the collected data, feature information is extracted for object recognition and classification, system prediction and estimation, and the system prompts the driver to complete the vehicle's estimation of the road condition.

4. The road perception method based on millimeter wave radar according to claim 2, characterized in that: Vehicle status analysis includes the judgment of vehicle movement direction, distance from the vehicle, vehicle type and danger; pedestrian status analysis includes the judgment of pedestrian movement status, pedestrian position and direction, pedestrian behavior analysis, pedestrian danger judgment during vehicle driving, and person detection under vehicle-to-vehicle and vehicle-to-ground interconnection; the way Obstacle status analysis includes obstacle size judgment, obstacle movement status judgment, obstacle distance and movement direction judgment.

Images