An automatic navigation driving system with multiple clocks

Abstract

The application provides an automatic navigation driving system with multiple clocks, which comprises: a first system on chip, which obtains a starting clock based on a first clock from a GNSS receiving module and / or a central gateway, generates a second clock according to a first local crystal oscillator based on the starting clock; a camera, which is triggered to expose according to the second clock; a laser radar; a millimeter wave radar; a micro control unit, which generates a third clock based on a second local crystal oscillator, and uses the third clock to time the millimeter wave radar, and stamps the second clock and the third clock time stamp in the millimeter wave radar sensing data according to the second clock and the third clock; a perception fusion module, which fuses the millimeter wave radar sensing data with the second clock time stamp and the third clock time stamp, the laser radar sensing data with the second clock time stamp, and the image data with the second clock time stamp.

Description

[Technical Field]

[0001] This invention relates to the field of driving assistance technology, and in particular to an automatic navigation driving system with multiple clocks. [Background Technology]

[0002] In recent years, with the development of Navigation On Auto-pilot (NOA) for automated driving assistance, vehicles need to achieve lane-level navigation functions, using GNSS (Global Navigation Satellite System) position coordinate information and high-precision map information for positioning. All information needs to be synchronized under the same clock. Furthermore, as sensor configurations become more abundant, the fusion of sensor data also places higher demands on time accuracy. Based on this, the original synchronization method based on a single clock can no longer meet the needs of today's increasingly complex autonomous driving control systems.

[0003] In existing single-clock systems, the time synchronization of each module has the following problems:

[0004] 1) For single-clock systems that only use UTC (Universal Time Coordinated), when a vehicle leaves an area without GNSS signals, such as an underground parking lot, time synchronization takes a certain amount of time. During this process, the sensor fusion algorithm may fail to accurately fuse or jump, affecting assisted driving and / or autonomous driving related functions.

[0005] 2) For a single-clock system that only uses a local clock and does not synchronize with an external clock in real time, lane-level positioning cannot be achieved using RTK (Real-time kinematic) technology, and the system cannot achieve more advanced driving assistance functions and / or autonomous driving functions.

[0006] Therefore, it is necessary to propose a new solution to overcome the above problems. [Summary of the Invention]

[0007] The purpose of this invention is to provide an automated navigation driving system with multiple clocks, which, while ensuring accurate lane-level positioning, achieves precise clock synchronization of sensor data, making data fusion unrestricted by location, resulting in more accurate algorithm results and improving the safety of assisted driving functions.

[0008] To achieve the objective of the invention, according to one aspect of the present invention, an autonomous navigation driving system with multiple clocks is provided, comprising: a GNSS receiving module for outputting recommended positioning information and a second pulse signal, the recommended positioning information including a navigation clock and positioning information; a central gateway for receiving an external first clock; a first on-chip system for receiving the recommended positioning information and the second pulse signal from the GNSS receiving module, using the second pulse signal to synchronize the navigation clock to an integer second to obtain a first clock, and / or receiving the first clock output by the central gateway, obtaining a start clock based on the first clock from the GNSS receiving module and / or the central gateway, and generating a second clock based on the start clock according to a first local crystal oscillator, the first on-chip system including a perception fusion module; a camera device for triggering exposure according to the second clock, thereby enabling the perception fusion module to obtain image data with a second clock timestamp T21; a lidar for timing by the second clock, acquiring lidar sensing data with a second clock timestamp T22, and transmitting it to the perception fusion module; a millimeter-wave radar; and a microcontroller unit. The system includes a second local crystal oscillator, which generates a third clock based on the second local crystal oscillator and uses the third clock to provide timing for the millimeter-wave radar. When the millimeter-wave radar generates millimeter-wave radar sensing data, it inserts a third clock timestamp T31 based on the third clock as the data generation time. The microcontroller receives millimeter-wave radar sensing data with the third clock timestamp T31 from the millimeter-wave radar and obtains the second clock and the third clock. Based on the second clock and the third clock, it inserts a second clock timestamp T23 and a third clock timestamp T32 into the millimeter-wave radar sensing data as the data reception time, thereby obtaining millimeter-wave radar sensing data with the second clock timestamp T23 and the third clock timestamps T31 and T32. The perception fusion module receives the millimeter-wave radar sensing data with the second clock timestamp T23 and the third clock timestamps T31 and T32, and fuses the millimeter-wave radar sensing data with the second clock timestamp T23 and the third clock timestamps T31 and T32, the lidar sensing data with the second clock timestamp T22, and the image data with the second clock timestamp T21.

[0009] Compared with existing technologies, this invention achieves precise clock synchronization of sensor data while ensuring accurate lane-level positioning, making data fusion unrestricted by location, resulting in more accurate algorithm results and improving the safety of assisted driving functions. [Attached Image Description]

[0010] The invention will be more readily understood in conjunction with the accompanying drawings and the following detailed description, wherein the same reference numerals correspond to the same structural components, wherein:

[0011] Figure 1This is a circuit diagram of an automated navigation driving system with multiple clocks in one embodiment of the present invention.

Detailed Implementation Methods

[0012] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0013] The term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic associated with that embodiment that is included in at least one implementation of the invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it necessarily a single or alternative embodiment that is mutually exclusive with other embodiments. "A plurality of" or "several" in this invention means two or more. "And / or" in this invention means "and" or "or".

[0014] Please refer to Figure 1 As shown, it is a circuit diagram of an automatic navigation driving system with multiple clocks in one embodiment of the present invention. Figure 1 The multi-clock autonomous navigation driving system shown includes a GNSS (Global Navigation Satellite System) receiver module 110, a central gateway (CGW) 120, a first system-on-a-chip (SoC) 130, a camera device 140, a lidar 150, a millimeter-wave radar 5*Radar 160, and a microcontroller unit (MCU) 170. The first SoC 130 includes a perception fusion module.

[0015] The GNSS receiver module 110 outputs recommended positioning information and a pulse-per-second (PPS) signal. The recommended positioning information includes a navigation clock and positioning information. The recommended positioning information is in GNSS PMC format. The central gateway 120 receives an external first clock, which may be a UTC clock. The central gateway 120 can be connected to the vehicle's central processing system, which typically has access to a UTC clock.

[0016] The first system-on-a-chip (SoC) receives recommended positioning information and a 1PPS (pixel per second) signal from the GNSS receiver module 110. It uses the 1PPS signal to synchronize the navigation clock to a whole second to obtain a first clock, and / or receives an external clock signal output from the central gateway CGW120 to obtain the first clock. Preferably, the first SoC obtains the first clock based on the GNSS receiver module 110. The first clock is a high-precision clock and can be used for high-precision positioning. Since the first clock is obtained based on satellite signals or a network, and satellite / network signals may experience interruptions / delays, the first clock may be lost or change.

[0017] A start clock is obtained based on a first clock from the GNSS receiver module 110 and / or the central gateway CGW120. A second clock is then generated based on this start clock using a first local crystal oscillator. Specifically, the first local crystal oscillator is located on the first on-chip system SoC. Each time the vehicle is powered on, a start clock is obtained based on the first clock from the GNSS receiver module 110 and / or the central gateway CGW120. The first local crystal oscillator generates a second clock based on this start clock (i.e., the second clock is synchronized with the first clock). After this power-on, the second clock is no longer synchronized with the first clock, thus implementing an internal monotonic clock.

[0018] The camera device Camera140 is triggered to expose based on the second clock of the first system-on-a-chip (SoC), thereby obtaining image data with the second clock timestamp T21 and transmitting it to the perception fusion module. The LiDAR device LiDAR 150 is timed by the second clock of the first SoC, obtains LiDAR sensing data with the second clock timestamp T22, and transmits it to the perception fusion module.

[0019] The microcontroller unit MCU170 includes a second local crystal oscillator, which generates a third clock. This third clock is not synchronized with the first and second clocks (i.e., the third clock is an internal monotonic clock of the MCU170). The MCU170 uses this third clock to provide timing information to the millimeter-wave radar 5*Radar160. When generating millimeter-wave radar sensing data, the MCU170 inputs a third clock timestamp T31 based on the third clock as the data generation time. The MCU170 receives millimeter-wave radar sensing data with the third clock timestamp T31 from the MCU170. The MCU170 obtains the second and third clocks and inputs second clock timestamps T23 and T32 into the millimeter-wave radar sensing data according to these timestamps, thus obtaining millimeter-wave radar sensing data with second clock timestamp T23 and third clock timestamps T31 and T32.

[0020] The perception fusion module receives millimeter-wave radar sensing data with second clock timestamp T23 and third clock timestamps T31 and T32, and fuses the millimeter-wave radar sensing data with second clock timestamp T23 and third clock timestamps T31 and T32, the lidar sensing data with second clock timestamp T22, and the image data with second clock timestamp T21.

[0021] Furthermore, in Figure 1 In the illustrated embodiment, the autonomous driving system includes a navigation and localization module 131, which is a software module of the first system-on-a-chip (SoC). The navigation and localization module 131 utilizes positioning information from the GNSS receiver module 110 and performs navigation and localization based on a first clock and real-time kinematic (RTK) technology to obtain lane-level positioning data. The perception fusion module fuses lane-level positioning data, millimeter-wave radar sensing data, lidar sensing data, and image data according to the time difference between the first and second clocks.

[0022] exist Figure 1 In the illustrated embodiment, the navigation and positioning module 131 is located on the first system-on-chip (SoC). For example... Figure 1 As shown, the autonomous navigation driving system with multiple clocks also includes a second system-on-a-chip (SoC) 180. The first SoC sends a second clock to the second SoC 180 so that the local clock on the second SoC 180 is synchronized with the second clock.

[0023] exist Figure 1In the specific embodiment shown, the first System-on-Chip (SoC) provides time synchronization to the LiDAR 150 via the gPTP (General Precise Time Protocol). The third clock is independent of the first and second clocks; that is, the third clock is not synchronized with the first and second clocks. After triggering exposure based on the second clock of the first SoC, the Camera 140 transmits image data to the perception fusion module, which adds a second clock timestamp T21 to the image data. The Microcontroller Unit (MCU) 170 provides time synchronization to the millimeter-wave radar 5*Radar 160 via the CANFD (CAN with Flexible Data Rate, an upgraded version of traditional CAN) protocol. The perception fusion module calculates the current second clock time of the radar target based on the time difference between the second clock timestamp T23 and the third clock timestamp T32. This achieves the alignment of the timestamps of the data from each sensor.

[0024] like Figure 1 As shown, the GNSS receiving module 110 transmits recommended positioning information (GPRMC) via the UART (Universal Asynchronous Receiver / Transmitter) protocol, and the central gateway CGW120 transmits the first clock via the Controller Area Network (CAN) protocol. The navigation and positioning module 130 selects one of the first clocks of the GNSS receiving module 110 and the central gateway CGW120 as the selected first clock.

[0025] Furthermore, compared to lidar and camera devices, the millimeter-wave radar 160 has higher requirements for timing accuracy and stability. The accuracy and stability of the second clock, which may meet the requirements of lidar and camera devices, may not be sufficient for the millimeter-wave radar. If the millimeter-wave radar 160 is timed using the second clock via the first system-on-a-chip (SoC) and the microcontroller unit (MCU) 170, the instability of the hardware and software in the clock signal transmission path will further reduce the accuracy and stability of the second clock signal. These factors make it impossible for the second clock to meet the precise and stable timing requirements of the millimeter-wave radar (instability of the millimeter-wave radar can have unacceptable impacts on many basic safety functions). Therefore, this invention uses the microcontroller unit (MCU) 170 to directly time the millimeter-wave radar 160 based on a third clock. This reduces the hardware and software in the timing data transmission path, reduces instability factors, and ensures the accuracy and stability of the third clock, thereby improving the stability and safety of the entire autonomous navigation and driving system. The radar data generated by the millimeter-wave radar is stamped with a timestamp T31 based on the third clock, which serves as the data generation time. Before transmitting millimeter-wave radar data to the perception fusion module, the microcontroller unit MCU170 simultaneously inputs a second clock timestamp T23 and a third clock timestamp T32. T23 / T32 corresponds to the reception time of the millimeter-wave radar data received by the microcontroller unit MCU170. The perception fusion module can determine the clock difference between the two clocks based on the time difference between (T32-T23). Combined with the third clock timestamp T31 (i.e., the radar data generation time) in the millimeter-wave radar data, it can calculate the second clock time corresponding to time T31 or identify abnormal second clock timestamps for subsequent fusion with signals from other modules, thereby ensuring the stability and accuracy of data fusion. It should be noted that, under normal circumstances, both the second and third clocks are internal monotonic clocks generated by the local crystal oscillator. The time difference between them is stable or changes slowly. Therefore, through the second clock timestamp T23 and the third clock timestamp T32, the perception fusion module can calculate the clock difference between the second and third clocks and identify the abnormal (i.e., abrupt change, the specific threshold of which can be set as needed) second clock timestamp T23. The method for identifying data abrupt changes is a conventional technique in this field and will not be elaborated here. For the abnormal second clock timestamp T23 and the corresponding data frame, the perception fusion module can calculate the second clock time corresponding to the data generation time based on the second clock / third clock time difference of several adjacent data frames. Alternatively, the perception fusion module can discard this frame of data, or, in the case of prolonged instability, report an error, and the corresponding upper-level function will also exit. The vehicle's instrument panel will prompt the driver to exit the function and be ready to take over.

[0026] In summary, this invention provides an automated navigation driving system with multiple clocks, which ensures accurate lane-level positioning while achieving precise clock synchronization of sensor data. This allows data fusion to be unrestricted by location, resulting in more accurate algorithm results and improved safety of assisted driving functions.

[0027] The foregoing description has fully disclosed the specific embodiments of the present invention. It should be noted that any modifications made to the specific embodiments of the present invention by those skilled in the art do not depart from the scope of the claims. Accordingly, the scope of the claims is not limited to the specific embodiments described.

Claims

1. An automatic navigation driving system with multiple clocks, characterized in that, It includes: GNSS receiver module, which is used to output recommended positioning information and second pulse signal, wherein the recommended positioning information includes navigation clock and positioning information; A central gateway, which is used to receive an external first clock; The first on-chip system receives recommended positioning information and a second pulse signal from the GNSS receiving module, uses the second pulse signal to synchronize the navigation clock to an integer second to obtain a first clock, and / or receives the first clock output by the central gateway, obtains a start clock based on the first clock from the GNSS receiving module and / or the central gateway, and generates a second clock based on the start clock according to a first local crystal oscillator. The first on-chip system includes a perception fusion module. The camera device triggers exposure according to a second clock, thereby enabling the perception fusion module to obtain image data with the second clock timestamp T21; The lidar is timed by the second clock, acquires lidar sensing data with the second clock timestamp T22, and transmits it to the perception fusion module; Millimeter-wave radar; The microcontroller unit includes a second local crystal oscillator, generates a third clock based on the second local crystal oscillator, and uses the third clock to provide timing for the millimeter-wave radar. When the millimeter-wave radar generates millimeter-wave radar sensing data, it inserts a third clock timestamp T31 based on the third clock as the data generation time. The microcontroller unit receives millimeter-wave radar sensing data with the third clock timestamp T31 from the millimeter-wave radar and obtains the second clock and the third clock. Based on the second clock and the third clock, it inserts a second clock timestamp T23 and a third clock timestamp T32 into the millimeter-wave radar sensing data as the data reception time, thereby obtaining millimeter-wave radar sensing data with the second clock timestamp T23 and the third clock timestamps T31 and T32. The perception fusion module receives millimeter-wave radar sensing data with second clock timestamp T23 and third clock timestamps T31 and T32, and fuses the millimeter-wave radar sensing data with second clock timestamp T23 and third clock timestamps T31 and T32, the lidar sensing data with second clock timestamp T22, and the image data with second clock timestamp T21.

2. The automatic navigation driving system as described in claim 1, characterized in that: The perception fusion module calculates the clock difference between the third clock and the second clock based on the time difference between the third clock timestamp T32 and the second clock timestamp T23 in the millimeter-wave radar data. Based on the third clock timestamp T31 in the millimeter-wave radar data and the clock difference, it calculates the second clock time corresponding to the millimeter-wave radar data generation time T31, which is used to fuse with the lidar sensing data with the second clock timestamp T22 and the image data with the second clock timestamp T21.

3. The automatic navigation driving system as described in claim 1, characterized in that, The first on-chip system also includes a navigation and positioning module, which uses the positioning information from the GNSS receiver module to perform navigation and positioning based on a first clock and real-time dynamic carrier phase differential technology to obtain lane-level positioning data.

4. The automatic navigation driving system as described in claim 3, characterized in that: The perception fusion module fuses lane-level positioning data, millimeter-wave radar sensing data, lidar sensing data, and image data based on the time difference between the first clock and the second clock.

5. The automatic navigation driving system as described in claim 3, characterized in that: The navigation and positioning module is located on the first on-chip system, and the first local crystal oscillator is located on the first on-chip system. The automatic navigation driving system also includes a second on-chip system. The first system-on-chip sends the second clock to the second system-on-chip so that the local clock on the second system-on-chip is synchronized with the second clock.

6. The automatic navigation driving system as described in claim 5, characterized in that, The first system-on-chip provides timing to the lidar via the gPTP protocol. The GNSS receiving module transmits the recommended positioning information via the UART protocol. The central gateway transmits the first clock via the CAN protocol. After the camera device triggers exposure according to the second clock, it transmits the image data to the perception fusion module, which then adds the second clock timestamp T21 to the image data. Each time the vehicle is powered on, a starting clock is obtained based on a first clock from the GNSS receiving module and / or the central gateway. A first local crystal oscillator generates a second clock based on this starting clock. After this power-on, the second clock is no longer synchronized with the first clock, thus achieving an internal monotonic clock. The first system-on-chip (SoC) selects one of the first clocks of the GNSS receiver module and the first clock of the central gateway as the selected first clock. The microcontroller unit provides timing for the millimeter-wave radar via the CANFD protocol.

Images