Millimeter wave radar internal subsystem synchronization method and synchronization method between millimeter wave radar internal subsystem and different sensing systems
By introducing global timer module and CAN-FD interface into the internal subsystem of millimeter wave radar, synchronization between the subsystem and perception system is achieved, solving the perception shortcomings and high power consumption caused by the independent working of each perception system in the traditional intelligent assisted driving perception system, and achieving more efficient perception performance and low power consumption design.
Patent Information
- Application Number
- CN202411866051.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-18
- Publication Date
- 2025-04-11
AI Technical Summary
In traditional intelligent assisted driving perception systems, each perception system works independently, resulting in obvious perception shortcomings. The power consumption of millimeter-wave radar sensors is high, making it difficult to achieve a low-power design.
By introducing a global timer module into the internal subsystem of the millimeter wave radar, synchronization between subsystems is achieved and communication with the host controller through the CAN-FD interface is achieved, synchronization between different perception systems is achieved, thereby reducing power consumption and improving perception performance.
It realizes independence and synchronization between different subsystems, reduces the power consumption of millimeter-wave radar system, and improves the perception performance of intelligent assisted driving systems.
Smart Images
Figure CN119814209A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of millimeter wave radar sensors and intelligent assisted driving technology, and in particular to a method for synchronizing subsystems within a millimeter wave radar and between the subsystem and different perception systems. Background Art
[0002] Millimeter wave radar refers to a radar device that works in the 30-300GHz band (corresponding to a wavelength of 1-100mm). It is considered one of the most important sensor technologies in autonomous driving technology because it is not affected by climate and light conditions, has a long detection distance, is lightweight, requires little computing power, and has low manufacturing costs. It is used for high-precision, three-dimensional speed measurement, distance measurement, and obstacle avoidance.
[0003] At present, intelligent assisted driving technology uses a multi-sensory system fusion perception method, including cameras, millimeter-wave radars, lidars, and ultrasonic radars. The algorithm will fuse and analyze the data of multiple perception systems to obtain a more comprehensive and detailed driving situation, thereby helping the vehicle domain control to make effective decisions and ultimately achieve a higher level of autonomous driving.
[0004] The mainstream radar waveform of the millimeter-wave radar used is frequency modulated chirp wave (FMCW). In this waveform system, the radar transmits a series of chirp wave pulses, which constitute a radar frame. During the transmission of these chirp pulses, the radar needs to accurately detect the temperature, voltage and power of the analog RF circuit, and calibrate and compensate the RF circuit in the event of an abnormality, so that the millimeter-wave radar sensor can always be in a normal working state.
[0005] In traditional intelligent assisted driving perception solutions, different perception systems are independent of each other. The algorithms independently analyze the data of different perception systems and restore the physical world. Although the architecture is simple, different perception systems have obvious shortcomings: for example, cameras cannot accurately perceive at night, in heavy rain, or in foggy weather, millimeter-wave radars are not accurate enough, lidars are too large and expensive, and ultrasonic radars are too short. These obvious perception shortcomings have become bottlenecks restricting the improvement of intelligent assisted driving performance.
[0006] In traditional millimeter-wave radar sensor solutions, the auxadc circuit used to monitor the temperature, voltage and power of the RF circuit and to feedback the calibration compensation value is usually built into the subsystem inside the same chip to avoid the problem of auxadc and FM chirp wave being out of sync, and the problem of different auxadc monitoring being out of sync. However, this will bring another problem. If this subsystem only needs auxadc for monitoring, the ineffective working time of other modules will inevitably lead to an increase in power consumption, which will also become a bottleneck for the low-power design of millimeter-wave radar sensors.
[0007] One technical approach to break through the bottlenecks of these different problems is to place a global timer inside the perception system. When working, the internal modules of the system send time latch requests to the global timer to record the working moments, thereby achieving time synchronization of the modules inside the system; and different perception systems communicate with the common host controller through the CAN-FD bus and send their respective timestamps to the host controller, thereby realizing synchronization of different perception systems in the host controller. Summary of the invention
[0008] This application provides a method for synchronizing the internal subsystems of a millimeter-wave radar and different perception systems. The technical purpose is, first, to distribute the auxadcs in different systems so that the chip can be partially powered off to reduce the power consumption of the millimeter-wave radar system; second, different perception systems communicate with a common host controller through a CAN-FD interface to achieve synchronization, so as to achieve unified interface standardization and improve the perception performance of the intelligent assisted driving system.
[0009] The above technical objectives of this application are achieved through the following technical solutions:
[0010] A method for synchronizing a millimeter wave radar internal subsystem and a method for synchronizing the subsystem with different perception systems, the method being implemented by an smc module, a CAN-FD timer module and a global timer module, wherein the compression method comprises:
[0011] S1: The SMC module starts sending and receiving millimeter wave detection targets according to the software configuration, and sends to the global timer module
[0012] Time latch request;
[0013] S2: The CAN-FD timer module sends a time latch request signal to the global timer module according to the software configuration;
[0014] S3: After receiving the time latch request from the SMC module or the CAN_FD timer module, the global timer module will
[0015] Latch the current time and report the interrupt;
[0016] S4: The global timer module can start working according to the auxadc of the subsystem where the global timer is located according to the software configuration trigger, and keep synchronization with the auxadc of the subsystem where the smc module is located;
[0017] S5: The software synchronizes the CAN_FD time according to the time locked by the global timer, and sends the timestamp through the can
[0018] The interface is sent to the main controller outside the millimeter-wave radar to achieve synchronization with other perception systems.
[0019] Furthermore, the SMC module is configured according to software, which means that the software can freely configure the start time, chirp length, chirp interval, and end time of a frame of radar data, and send a time latch request signal to the global timer at the beginning.
[0020] Furthermore, the CAN-FD module is configured according to software, which means that the software can freely configure the CAN_FD timer to send a time latch request signal to the global timer at a specified time point.
[0021] Furthermore, the global timer will refresh the current time of the detected request into a read-only register according to the time latch request, and report an interrupt to inform the software to come and read it.
[0022] Furthermore, the global timer is configured according to the software, which means that the global timer can determine the state of the subsystem where the SMC module is located according to the latched time, and then according to the software configuration of a frame of radar data, it can infer the subsequent working state of the subsystem where the SMC module is located, and generate a trigger signal according to the inference result to trigger the AuxADC in the subsystem where the global timer is located, so as to achieve the purpose of synchronization of different subsystems in the millimeter-wave radar chip.
[0023] Furthermore, the software locks the time according to the global timer, which means that the software can synchronize the CAN_FD timer module with other subsystems inside the millimeter-wave radar chip according to the time when the smc module is latched on the global timer, the time when the CAN_FD timer module sends a request to the global timer, and the time when the CAN_FD timer module is latched on the global timer; further, the CAN_FD timer module will send a synchronized timestamp to the host controller outside the chip to synchronize with it, thereby achieving synchronization with other perception systems that have been synchronized with the host controller.
[0024] The beneficial effect of the present application is that the synchronization of the internal subsystems of the millimeter-wave radar described in the present application and the synchronization method between the millimeter-wave radar and different perception systems can ensure the independence and synchronization between different subsystems and different systems, and provide greater space for low-power design, unification of interface standards and improvement of intelligent assisted driving performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 A schematic diagram of the synchronization of the millimeter-wave radar's internal subsystems and its synchronization with different perception systems;
[0026] Figure 2 This is a state machine diagram of the SMC module;
[0027] Figure 3 Timing diagram for synchronizing the smc module smc_frame_start and the global timer;
[0028] Figure 4 Timing diagram for synchronization of CAN_FD timer and global timer. DETAILED DESCRIPTION
[0029] Example 1
[0030] The technical solution of the present application will be described in detail below with reference to the accompanying drawings.
[0031] The synchronization method of the millimeter wave radar internal subsystem and the synchronization method between different perception systems described in this application is as follows: Figure 1According to the schematic diagram shown in the figure, the SMC module triggers the sys1_auxadc operation according to the software configuration and sends a time latch request to the global timer; the can_timer module sends a time latch request to the global timer after a certain time is reached according to the software configuration, and sends a timestamp to the host controller; the radarsystem, camera system, and lidar system send timestamps to the host controller through the CAN interface; the host controller compares the received timestamp with its internal timer for synchronization.
[0032] like Figure 1 As shown, Figure 1 It includes three perception systems: 4D millimeter-wave radar, camera, and lidar, a host controller for synchronizing the timestamps of the three perception systems, and the targets in physical time that the three perception systems need to detect.
[0033] Each perception system synchronizes the timestamp of the detection operation initiated on the target object with the host controller through the CAN_FD interface. The host controller knows the time when each perception system acts on the absolute time axis, thereby achieving synchronization of different perception systems.
[0034] For the 4D millimeter-wave radar system, the sys1_auxadc module and the SMC module are in the same subsystem (both belong to rf_sys), which are synchronized. The sys0_auxadc module, the global timer module, and the can timer module are in the same subsystem (both belong to main sys):
[0035] SMC is a state machine that controls the analog RF circuit to generate FM chirp waves;
[0036] sys1_auxadc is a module that calibrates and monitors the RF transmission and receiving circuits (such as monitoring temperature, voltage, and power) during the transmission of FM chirp waves.
[0037] sys0_auxadc is a module that monitors the PLL circuit, XO circuit, and power supply LDO (such as monitoring temperature, voltage, and power) during the transmission of the FM chirp wave or when the FM chirp wave is not transmitted.
[0038] The can timer is a timer inside the CAN_FD interface that can send time latch requests to the global timer; it can also communicate synchronously with an external host controller;
[0039] The global timer is a free-run timer that latches the time according to the request of the SMC module and the can timer module, and triggers the sys0_auxadc module and the sys1_auxadc module to work synchronously;
[0040] Since the camera perception system and lidar system are third-party perception systems, the internal design details will not be elaborated here.
[0041] like Figure 2 As shown, Figure 2 It contains two state machines: frame FSM and chirp FSM. The chirp FSM is triggered by the CHIRP state in the frame FSM. When the chirp_start condition of the chirp FSM state machine is pulled high, the transmitter of the radio starts to send FM chirp waves, and the receiver of the radio starts to receive FM chirp waves reflected by the physical target; at the same time, the global timer module starts to latch and synchronize the time point when the above actions start:
[0042] IDLE state: the idle state of the frame FSM;
[0043] FRAM state: When the software enables frame_en_sync, it jumps from IDLE to this state;
[0044] DLY1 state: When the FRAM state lasts for a software-configurable time T0, it enters this state;
[0045] CHIRP state: When the DLY1 state lasts for a software-configurable time T1, it enters this state;
[0046] WAIT state: When the CHIRP state lasts for one clock cycle, it enters this state;
[0047] IDLE0 state: the idle state of the chirp FSM;
[0048] LOAD state: When the CHIRP state is detected in the IDLE0 state, this state is entered after the chirp_start signal is pulled high;
[0049] WAIT, STMP, DONE0 state: This state indicates that a software-configurable frequency-modulated chirp wave with a duration of T2 is being sent;
[0050] Other states: The other states are all the previous state that continues for a software-configurable time Tn or a software-configurable number of cycles, then enters the next state until it returns to the IDLE state, and the FM chirp wave is sent repeatedly in this way.
[0051] The specific implementation of the synchronization of the millimeter wave radar internal subsystem and the synchronization method between different perception systems described in this application is as follows: Figure 2 , Figure 3 , Figure 4 As shown, including:
[0052] S1: The SMC module starts sending and receiving millimeter wave detection targets according to the software configuration, and sends a time latch request to the global timer.
[0053] Specifically, Figure 2 As shown, the software will enable frame_en_sync, and then the SMC state machine will jump to a certain state, and generate a frame_start signal after a configurable period of time, which indicates that the SMC has started to send and receive FM chirp waves; after the SMC detects the frame_start signal, it will pull up the timer_latch request signal and keep it for a configurable period of time to ensure that the global timer module has enough time to synchronize this request signal to its own clock domain, as shown in Figure 3 shown.
[0054] S2: The CAN-FD timer module sends a time latch request signal to the global timer according to the software configuration;
[0055] Specifically, Figure 4 As shown in the figure, after FD-CAN is released from reset, FD_CAN timer starts timing; when FD_CAN receives SYNCmessage, SW can enable can_timer_16b_clr signal during this period to clear the 16-bit timer; while clearing the 16-bit timer, FD_CAN initiates a latch request can_timer_16b_latch_req to the global timer, and this request signal will be maintained for a configurable period of time to ensure that the global timer module has sufficient time to synchronize this request signal to its own clock domain;
[0056] S3: After receiving the time latch request from the smc module or the CAN_FD timer module, the global timer module will latch the current time and report an interrupt;
[0057] Specifically, in step S3, the process of the global timer module receiving the time latch request from the smc module or the CAN_FD timer module includes the following steps:
[0058] S31: Figure 3As shown in the figure, the global timer is a 64-bit free time timer, which starts timing after reset is released; and starts monitoring the time request from the SMC after the software configuration allows it to receive the time latch request from the SMC module; after the global timer detects timer_latch, it latches the time of the global timer at this moment; at the same time, the delay time for the auxadc to start working is refreshed from the configuration register to the operation register; when the global timer reaches the time of global_timer_latch0+auxadc_s_value, it pulls up auxadc_s_ptr; after auxadc detects that auxadc_s_ptr is enabled, it can start working; when the software detects the interrupt generated by latch0_ptr or auxadc_s_ptr, it can refresh the values of latch0_en and auxadc_s_value signal sources to reconfigure the global timer.
[0059] S32: Figure 4 As shown in the figure, the lobal timer is a 64-bit free time timer, which starts timing after reset is released. After the global timer detects the flip of can_timer_16b_latch_req, it starts latching the current time into global_timer_latch2. When the global timer latches the time, it reports a latch2_ptr interrupt to tell the software that the latch time is over.
[0060] S4: The global timer module can start working according to the auxadc of the subsystem where the global timer is located according to the software configuration trigger, and keep synchronization with the auxadc of the subsystem where the smc module is located;
[0061] Specifically, Figure 1 and Figure 3As shown, the SMC module will start to send FM chirp waves after the frame start. During this process, the SMC will trigger sys1_auxadc to monitor the RF circuit. This time point is determined by the software. At the same time, the SMC module will send a time latch request signal to the global timer at the frame start. The moment when the global timer latches and the time when the frame start is sent can be basically considered as a time point. In addition, the software will also configure the time from frame start to the start of sending FM chirp waves in the register bank of the global timer in advance. After receiving the time latch request from the SMC module, the globaltimer will refresh the configuration to the operation register auxadc_s_value. When the global timer reaches the time of global_timer_latch0+auxadc_s_value, it will pull up auxadc_s_ptr. After sys0_auxadc detects that auxadc_s_ptr is enabled, it can start working. This ensures that sys1_auxadc and sys0_auxadc work synchronously.
[0062] S5: The software synchronizes the CAN_FD time according to the time locked by the global timer, and sends the timestamp to the main controller outside the millimeter-wave radar through the CAN interface to achieve synchronization with other perception systems.
[0063] Specifically, there is a 16-bit time stamp field in the CAN_FD message structure. This value comes from the internal 16-bit Timer. When a completion message is sent or received, the value of TIME STAMP is updated; Figure 1 As shown, the hostcontroller sends synchronization information sync_message to the FD_CAN inside the millimeter wave radar sensor; Figure 4As shown in the figure, after FD_CAN receives the synchronization information, it will enable can_timer_16b_clr to clear the internal FD_CAN timer, and at the same time send a time latch request can_timer_16b_latch_req to the global timer; after the global timer detects the request signal, it will latch the current time and report the interrupt latch2_ptr; after receiving the interrupt, the software sends the latch time to the host controller in the form of a message; the host controller latches the time of its own internal timer at the same time after receiving the message; at this time, the host controller can calculate the subsequent working status of the millimeter-wave radar sensor according to the configuration of the frequency-modulated chirp wave; similarly, the host controller can also use this method to calculate the working status of the camera system and lidarsystem, so that the three independent perception systems can work synchronously.
[0064] Furthermore, the smc module is configured according to software, which means that the software can freely configure the start time of a frame of FM chirp wave, the length of the FM chirp wave, the interval of the FM chirp wave, the end time and the time point of sending a time latch request signal to the global timer at the beginning.
[0065] In summary, through software preset configuration, the timer latches the request latch moment according to the time latch of different systems, and reports the interrupt to inform the software. The software then calculates the working status of the asynchronous system in another system according to the preset configuration value, thereby ensuring that different systems are both independent and synchronized. It has application value and potential in low-power design and improvement of intelligent assisted driving perception performance.
[0066] As shown in Tables 1 and 2 below, Table 1 is a table of the detection performance of the perception system; Table 2 is a table of the back-end power consumption simulation data;
[0067] Table 1. Detection performance of perception system
[0068]
[0069]
[0070] Table 1 compares the advantages and disadvantages of different perception systems and the effects of multi-sensor fusion. It can be seen that different perception systems have different advantages and disadvantages. In order to make up for the disadvantages of different perception systems, multi-sensor data needs to be fused at the algorithm level. The first problem that data fusion solves is to ensure the synchronization of different independent sensor systems in detecting the same target.
[0071] Table 2 Back-end power consumption simulation data table
[0072] Overall power consumption of the digital part of the chip (mw) rf_sys power consumption (mw) Rf_sys power consumption ratio (%) 169 20 11.83
[0073] Table 2 compares the power consumption of the digital part of the millimeter wave radar chip without low power design and with low power design. It can be seen that during the transmission of the FM chirp wave, the chip must be fully powered on, and the digital part of the chip consumes 169mw, of which the rf_sys part consumes 20mw. During the entire chip and the internal algorithm module of the chip and when the FM chirp wave is not transmitted, the chip does not need to receive and calculate data, and the entire rf_sys subsystem can be turned off, thus saving 20mw of power consumption, accounting for 11.83%.
[0074] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited thereto, and various changes can be made within the knowledge scope of technicians in the relevant technical field without departing from the purpose of the present invention.
Claims
1. A method for synchronizing a millimeter wave radar internal subsystem and a method for synchronizing a millimeter wave radar internal subsystem and a subsystem with different sensing systems, characterized in that: The method is implemented by an SMC module, a CAN-FD timer module and a global timer module; the specific method includes the following steps: S1, the smc module starts sending and receiving millimeter wave detection targets according to the software configuration, and sends a time latch request to the global timer module at the same time; S2, the CAN-FD timer module sends a time latch request signal to the global timer module according to the software configuration; S3, after receiving the time latch request from the smc module or the CAN_FD timer module, the global timer module will latch the current time and report an interrupt; S4, the global timer module can trigger the auxadc of the subsystem where the global timer is located to start working according to the software configuration, and keep synchronization with the auxadc of the subsystem where the smc module is located; S5. The software synchronizes the CAN_FD time according to the time locked by the global timer, and sends the timestamp to the main controller outside the millimeter-wave radar through the CAN interface to achieve synchronization with other perception systems.
2. The synchronization method of the internal subsystem of the millimeter wave radar and the synchronization method between the internal subsystem and different perception systems according to claim 1, characterized in that: The SMC module is configured according to the software, which means that the software can freely configure the start time, chirp length, chirp interval, and end time of a frame of radar data, and send a time latch request signal to the global timer module at the beginning.
3. The synchronization method of the internal subsystem of the millimeter wave radar and the synchronization method between the internal subsystem and different perception systems according to claim 1, characterized in that: The CAN-FD module is configured according to the software, which means that the software can freely configure the CAN_FD timer to send a time latch request signal to the global timer module at a specified time point.
4. The synchronization method of the internal subsystem of the millimeter wave radar and the synchronization method between the internal subsystem and different perception systems according to claim 1, characterized in that: The global timer module will refresh the current time of the detected request into the read-only register according to the time latch request, and report an interrupt to inform the software to read it.
5. The synchronization method of the internal subsystem of the millimeter wave radar and the synchronization method between the internal subsystem and different perception systems according to claim 1, characterized in that: The global timer module, according to the software configuration, means that the global timer module can determine the state of the subsystem where the SMC module is located according to the latched time, and then calculate the subsequent working state of the subsystem where the SMC module is located according to the software configuration of a frame of radar data, and generate a trigger signal according to the calculation result to trigger the auxadc work in the subsystem where the global timer module is located, so as to achieve the purpose of synchronization of different subsystems in the millimeter-wave radar chip.
6. The synchronization method of the internal subsystem of the millimeter wave radar and the synchronization method between the internal subsystem and different perception systems according to claim 1, characterized in that: The software latches the time according to the global timer module, which means that the software can synchronize the CAN_FD timer module with other subsystems inside the millimeter-wave radar chip according to the time when the smc module latches the global timer module, the time when the CAN_FD timer module sends a request to the global timer module, and the time when the CAN_FD timer module latches the global timer module.
7. The synchronization method of the internal subsystem of the millimeter wave radar and the synchronization method between the internal subsystem and different perception systems according to claim 6, characterized in that: The CAN_FD timer module sends a synchronized timestamp to the off-chip host controller to synchronize with it, and thus synchronize with other sensing systems that have already synchronized with the host controller.