A vehicle radar recognition system based on image analysis principles

Abstract

The present invention relates to the field of automotive intelligent driving assistance technology, and discloses an on-board radar recognition system based on image analysis principles, comprising: an original vehicle on-board host, a 360-degree panoramic imaging system, a video switching and radar recognition integrated circuit, and a vehicle central control screen. The original vehicle on-board host is the host portion of the original vehicle, and the original vehicle video model collected by the original vehicle on-board host is transmitted to the video switching and radar recognition integrated circuit, which then transmits the video switching and radar recognition integrated circuit to the vehicle central control screen. This on-board radar recognition system based on image analysis principles, wherein the video switching and radar recognition integrated circuit provided in the system is a semi-custom circuit, has a large number of logic units, and by configuring logic units such as a coordinate mapping unit, a serial positioning unit, a radar recognition unit, and a radar transmission unit, a corresponding image processing circuit is constructed to perform more detailed classification of target objects.

Description

Technical Field

[0001] This invention relates to the field of automotive intelligent driving assistance technology, specifically to a vehicle radar recognition system based on image analysis principles. Background Technology

[0002] With the continuous increase in car ownership, road traffic safety issues are becoming increasingly prominent. As automotive intelligent technology continues to develop, traditional vehicle radar systems have limitations in target recognition, only providing basic information such as distance and speed, making it difficult to accurately identify the specific type and characteristics of a target. Therefore, 360° panoramic imaging systems have become a hot research area in the automotive field. This technology provides drivers with a panoramic view, helping them to better understand the road conditions and obstacles around the vehicle, thus improving driving safety. Currently, most high-end models from mainstream brands come standard with 360° panoramic imaging systems, but lower-end models, which sell more, require aftermarket installation, leading to some usage issues, such as:

[0003] In the aftermarket for low-end vehicles, 360-degree surround view systems obtain radar information by connecting to the vehicle's CAN bus. Since all vehicle communication uses this interface, the data volume is enormous and complex, making it prone to errors. Furthermore, installation requires damaging the original vehicle's wiring harness, posing certain safety hazards and affecting subsequent maintenance. The electronic switch for video switching in these aftermarket surround view systems can completely block the original vehicle's view when switching to the 360-degree surround view, causing information such as air conditioning and volume on the central control screen to be lost when reversing. Additionally, the original vehicle's radar information cannot be obtained during use, making it difficult to transmit the original radar information. When switching to the aftermarket surround view while reversing, there may be a situation where there is no radar, affecting driving safety and user experience.

[0004] To address the aforementioned issues, there is an urgent need for innovative design based on the 360-degree panoramic system already installed in the low-end models. Summary of the Invention

[0005] The purpose of this invention is to provide a vehicle radar recognition system based on image analysis principles. This system addresses the issues raised in the background section regarding the traditional method of acquiring radar information via the vehicle's CAN bus, which requires damaging the original vehicle's wiring harness during assembly, posing safety hazards and impacting subsequent maintenance. Furthermore, the electronic switch used in this method completely blocks the original vehicle view when switching to the 360-degree panoramic view, and the inability to acquire the original vehicle's radar information during use makes it difficult to transmit the original radar information. Additionally, switching to the added panoramic view while reversing results in a situation where there is no radar, affecting driving safety and user experience.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a vehicle radar recognition system based on image analysis principles, comprising an original vehicle host, a 360-degree panoramic imaging system, video switching, and a radar recognition integrated circuit and a vehicle central control screen;

[0007] The original vehicle host is the host unit in the original vehicle, and the original vehicle video model collected by the original vehicle host is transmitted to the video switching and radar recognition integrated circuit. The video switching and radar recognition integrated circuit sends the data to the vehicle's central control screen, and sends radar information to the 360-degree panoramic imaging system. The 360-degree panoramic imaging system sends the panoramic video signal to the video switching and radar recognition integrated circuit.

[0008] The video switching and radar recognition integrated circuit includes: an RX-8619 chip, a serial-to-parallel conversion unit, a host computer, a crosshair engine unit, a coordinate reordering management unit, a serial radar recognition processing unit, and a video switching unit. The RX-8619 chip receives GMS signals sent from the original vehicle's onboard host. The serial radar recognition processing unit, together with the radar recognition unit and the radar transmission unit, sends radar information to the 360-degree panoramic imaging system. The 360-degree panoramic imaging system sends the panoramic video stream to the video switching unit, and the video switching unit sends information data to the vehicle's central control screen.

[0009] By adopting the above technical solution, a serial processing channel is used and the ELVDS pin is used to encode and decode differential signals, so that the FPGA with relatively small resource usage can complete the function.

[0010] Preferably, the serial-to-parallel conversion unit receives dual LVDS signals from the RX-8619 chip, and the serial-to-parallel conversion unit is equipped with a BitFlip module and an LVDS parsing module, and the LVDS parsing module processes the dual LVDS signals.

[0011] Using the above technical solution, the RX-8619 chip is used to integrate and transmit dual LVDS signals.

[0012] Preferably, the BitFlip module processes the data stream received from the LVDS parsing module, and the first bit code input serial-to-parallel conversion unit of the BitFlip module decodes the serial data stream into parallel data.

[0013] Using the above technical solution, the BitFlip module uses the hypothetical method to infer the alignment relationship by analyzing the patterns.

[0014] Preferably, the serial radar identification processing unit is internally equipped with a coordinate mapping unit, a serial positioning unit, a radar identification unit, and a radar transmission unit.

[0015] The above technical solution facilitates the overlay of radar coordinate markers from the original vehicle RGB video stream onto the panoramic video stream.

[0016] Preferably, the original vehicle RGB video stream data processed by the serial-to-parallel conversion unit is sent to the coordinate mapping unit, and the coordinate mapping unit transmits the data to the serial radar recognition processing unit.

[0017] By adopting the above technical solution, the serial data stream is converted into parallel data through the serial-to-parallel conversion unit, which facilitates data transmission.

[0018] Preferably, the serial radar identification processing unit receives sorted coordinate information from the coordinate reordering management unit, and the information from the coordinate reordering management unit is transmitted by the host computer.

[0019] By adopting the above technical solution, the temporal order of feature points in each frame of the acquired video is rearranged to facilitate the mapping of feature points to a radar group.

[0020] Preferably, the host computer data information is transmitted to the crosshair cursor engine unit, and the cursor position of the crosshair cursor engine unit is controlled by the host computer.

[0021] By adopting the above technical solution, the cursor position and recording are controlled by the host computer, which makes it easy to load into the system in advance during normal programs.

[0022] Preferably, the radar identification unit uses eight feature points selected from the video stream for identification, and the feature point information processed by the radar identification unit is communicated to the radar transmission unit in real time.

[0023] By employing the above technical solution, the existence of the radar location can be confirmed when radar feature points in each frame of image are composited.

[0024] Preferably, the radar transmission unit is connected to the 360° panoramic imaging system for real-time communication, and the panoramic video stream data acquired by the 360° panoramic imaging system is transmitted to the video switching unit.

[0025] Using the above technical solution, the 360° panoramic imaging system receives radar data to provide video position for overlapping and synchronization.

[0026] Preferably, the video switching unit receives the original vehicle RGB video stream from the serial radar recognition processing unit, and the video switching unit simultaneously transmits the panoramic video stream and the original vehicle RGB video stream to the vehicle's central control screen.

[0027] By adopting the above technical solution, the video switching unit provides two video stream perspectives for the vehicle's central control screen, thereby improving the user experience.

[0028] Compared with the prior art, the beneficial effects of the present invention are: the vehicle radar recognition system based on image analysis principles:

[0029] 1. The video switching and radar recognition integrated circuit set in this system is a semi-custom circuit. It has a large number of logic units. By configuring the coordinate mapping unit, serial positioning unit, radar recognition unit and radar transmission unit, the corresponding image processing circuit is constructed to classify the target objects more finely. As the coordinate reordering management unit and the host computer work together to adjust the crosshair position of the crosshair engine unit, the position of the radar in the original vehicle video stream is recorded and the RGB value when the radar is present is recorded. This information is recorded and pre-loaded into the serial radar recognition processing unit, which then sends the processed radar information to the 360-degree panoramic imaging system. The 360-degree panoramic imaging system provides the original vehicle video and radar information to the vehicle's central control screen. The video switching unit provides synchronous switching between the original vehicle RGB video stream and the 360-degree panoramic video stream system to achieve the required smooth switching function between the original vehicle video stream and the panoramic video stream.

[0030] 2. The host computer in the system controls the crosshair cursor of the crosshair cursor engine unit. During use, it is superimposed on the original vehicle video screen. Each radar in the radar recognition unit selects 8 feature points. With the radar transmission unit communicating with the 360-degree panoramic motherboard in real time, the feature points of each radar are dispersed and the color component values ​​when they overlap and do not overlap are recorded. If a radar in each frame meets 5 / 8 or more of the feature points, it can be set to OPEN state. If it is continuously judged to be in OPEN state for 3-5 frames, it is considered that this radar exists. After coordinate reordering management, the corresponding order is rearranged, so that the processed radar information can be superimposed on the panoramic video of the vehicle's central control screen by the serial radar recognition processing unit. The serial processing channel is used, and the differential signal encoding and decoding is implemented using the ELVDS pin, so that this function can be completed with very small FPGA resources. Compared with other conventional FPGA solutions on the market, it has a great advantage in terms of cost, power consumption and size. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the framework of the vehicle-mounted radar identification system of the present invention;

[0032] Figure 2 This is a simplified flowchart of the vehicle-mounted radar identification system of the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] See also Figure 1-2 The present invention provides a technical solution: a vehicle radar recognition system based on image analysis principle, including the original vehicle host, a 360-degree panoramic imaging system, video switching, and radar recognition integrated circuit and vehicle central control screen;

[0035] The original vehicle host is the host unit in the original vehicle, and the original vehicle video model collected by the original vehicle host is transmitted to the video switching and radar recognition integrated circuit. The video switching and radar recognition integrated circuit sends the data to the vehicle's central control screen, and sends radar information to the 360-degree panoramic imaging system. The 360-degree panoramic imaging system sends the panoramic video signal to the video switching and radar recognition integrated circuit.

[0036] The video switching and radar recognition integrated circuit includes: an RX-8619 chip, a serial-to-parallel conversion unit, a host computer, a crosshair engine unit, a coordinate reordering management unit, a serial radar recognition processing unit, and a video switching unit. The RX-8619 chip receives GMS signals sent from the original vehicle's onboard host. The serial radar recognition processing unit, together with the radar recognition unit and the radar transmission unit, sends radar information to the 360-degree panoramic imaging system. The 360-degree panoramic imaging system sends the panoramic video stream to the video switching unit, and the video switching unit sends information data to the vehicle's central control screen. The serial-to-parallel conversion unit receives dual LVDS signals from the RX-8619 chip. The serial-to-parallel conversion unit internally includes a BitFlip module and an LVDS parsing module. The LVDS parsing module processes the dual LVDS signals, and the BitFlip module processes the data stream received from the LVDS parsing module. The first bit code of the BitFlip module is input to the serial-to-parallel conversion unit to decode the serial data stream into parallel data.

[0037] The original vehicle RGB video stream data processed by the serial-to-parallel conversion unit is sent to the coordinate mapping unit, and the coordinate mapping unit transmits the data to the serial radar recognition processing unit. The serial radar recognition processing unit receives the sorted coordinate information from the coordinate reordering management unit, and the information from the coordinate reordering management unit is transmitted by the host computer. The host computer data information is sent to the crosshair cursor engine unit, and the cursor position of the crosshair cursor engine unit is controlled by the host computer. The video switching unit receives the original vehicle RGB video stream from the serial radar recognition processing unit, and the video switching unit synchronously transmits the panoramic video stream and the original vehicle RGB video stream to the vehicle's central control screen.

[0038] The serial radar identification processing unit is equipped with a coordinate mapping unit, a serial positioning unit, a radar identification unit, and a radar transmission unit. The radar identification unit uses 8 feature points selected in the video stream for identification, and the feature point information processed by the radar identification unit is communicated to the radar transmission unit in real time. The radar transmission unit is connected to the 360 ​​panoramic imaging system for real-time communication, and the panoramic video stream data acquired by the 360 ​​panoramic imaging system is transmitted to the video switching unit.

[0039] Referring to the attached diagrams in the instruction manual Figure 1-Figure 2 As shown, during use, the original vehicle's onboard unit sends the acquired GMS signal to the RX-8619 chip. The RX-8619 chip, i.e., GMS-TO-LVDS, processes the signal to obtain dual LVDS signals, which are then sent to the BitFlip module in the serial-to-parallel conversion unit. Figure 1 As shown, the dual LVDS signals are processed in conjunction with the LVDS parsing module. The LVDS parsing module uses an LVDS-to-RGB888 data converter to convert LVDS to RGB for signal processing. Before parsing any serial data stream, the BitFlip module and the LVDS parsing module need to find the position of the first bit. Typically, LVDS parsing uses a clock pin pattern, such as 1100011, to confirm the alignment relationship. The FPGA selected in this system, to reduce production costs, requires numerous LVDS pins, with two sets of dual LVDS inputs and one set of dual LVDS outputs. During use, some ELVDS (pseudo-differential) pins are needed. Since pseudo-differential pins cannot be matched with the chip's IP core, an assumption method is used when obtaining the clock pattern. The alignment relationship is deduced from the parsed pattern. The process of deducing the alignment relationship from the parsed pattern is as follows:

[0040] LVDS is a 7:1 serial data stream, so the first bit can have 7 possible values;

[0041] First, assume that case 1 is correct, and then analyze the key video signals, namely VS, HS, and DE, in the parsed data stream;

[0042] The analysis is correct, meaning case 1 is the correct first code.

[0043] If an error occurs, continue assuming case 2 until all 7 cases have been analyzed, at which point the correct first code can be found.

[0044] After the original vehicle RGB video stream is sent to the serial-to-parallel conversion unit, the serial data stream is decoded into parallel data based on the first bit code obtained from the BitFlip module. The VS, HS, and DE of the horizontal and vertical axes are extracted and stored internally by the chip. The processed original vehicle RGB video stream is then sent to the serial radar recognition processing unit. Figure 1 As shown;

[0045] The serial radar identification and processing unit synchronously receives sorted coordinate information from the coordinate mapping unit. The sorted image source of the coordinate mapping unit is transmitted from the host computer. The host computer uses an STM32 chip to guide the position of the crosshair in the crosshair engine unit. The crosshair in the crosshair engine unit is superimposed on the original vehicle video. After being acquired by the host computer, it is added to the system in advance during the normal program. Since there are many radar scale lines in the vehicle system, there are usually four directions: front, rear, left, and right. Each direction has five channels, and each channel has a radar level of 6. Therefore, there are 4*5*6=120 radar lines to be identified, and each line segment requires 8 feature points for verification. Therefore, there are 120*8=960 features. In order to obtain the corresponding coordinate feature points more quickly, the coordinate reordering management unit reorders them according to the time order of the video signal. The coordinate mapping unit receives the reordered coordinates. Since each radar scale line has 8 feature points for identification, and they are all disordered after sorting, each feature point needs to be mapped to a radar group and cached in RAM for easy restoration later. In the restoration, the mapping relationship is stored in RAM through the coordinate mapping unit, and the RGB values ​​of all points are also processed in the serial positioning unit and cached in RAM. Therefore, the radar identification unit directly extracts the RGB values ​​of the corresponding points of each radar based on the mapping relationship for judgment.

[0046] The serial positioning unit is set to an idle state in the settings. Simultaneously, it locates the field synchronization video in the video feed, enters state 6, and then enters the timer synchronization state. It generates a row and column counter based on the valid video data signal. When the first coordinate matches the row and column counter, it acquires the current RGB (video color component) value. It then retrieves the next sorted coordinate and continues the comparison, repeating this process until all coordinates are processed before re-entering the idle state. The radar recognition unit selects eight feature points to identify each radar line. Since the vehicle's central control screen also displays turn indicators, which can overlap with the radar lines anywhere, the feature points of each radar line need to be dispersed, and the color component values ​​for overlapping and non-overlapping conditions need to be recorded. In each frame, if 5 / 8 or more of the feature points of a radar line match, it can be set to the OPEN state. If 3-5 consecutive frames are in the OPEN state, the radar is considered to exist. After determining the radar information, the radar transmission unit communicates in real time with the 360-degree panoramic motherboard of the 360-degree panoramic imaging system to ensure timeliness. The connection between the two uses a serial port protocol. After the protocol content is agreed upon with the software, the FF00DD01 information is obtained, where FF00 is a fixed header, DD is the radar line number, 01 indicates radar presence, and 00 indicates radar absence. When the 360-degree panoramic imaging system transmits the panoramic video stream to the vehicle's central control screen, the video switching unit receives the panoramic video stream from the sensor and the original vehicle RGB video stream to assist the driver in switching the environment. Both the original vehicle video and the 360 ​​video are input into the FPGA chip. Therefore, the system video switching is achieved by selecting the code within the FPGA chip.

[0047] This system accurately identifies radar signals by analyzing the original vehicle's image. It employs a serial processing channel and utilizes ELVDS pins for differential signal encoding and decoding, avoiding the need to disassemble and damage the original centerline. After fusing image analysis with onboard radar data, the system can better adapt to complex and changing environments. In adverse weather conditions, although image quality may be affected, radar data can still supplement it, providing distance and speed information for target objects. In foggy weather, the radar can detect the approximate location of objects ahead, while image analysis can utilize limited visual information, such as blurred object outlines, for further identification. The combination of these two approaches enhances overall perception capabilities and strengthens the system's adaptability to various environments.

[0048] Working Principle: When using this vehicle radar recognition system based on image analysis, the 360° panoramic imaging system and the original vehicle's onboard host first synchronously acquire the original vehicle video. The serial radar recognition processing unit, located inside the video switching and radar recognition integrated circuit, connects the host computer and the crosshair engine unit to overlay the panoramic video and control the position of the crosshair. After the coordinate reordering management unit marks the correct time sequence of the images, each frame of the panoramic video stream received from the serial radar recognition processing unit is compared with the radar status of the image coverage area displayed to the driver on the vehicle's central control screen. After image analysis and vehicle radar data fusion, the system can better adapt to complex and changing environments. By utilizing the dual advantages of image analysis and radar to identify target objects, it can flexibly cope with various traffic scenarios.

[0049] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention.

Claims

1. A vehicle radar recognition system based on image analysis principles, characterized in that, include: Original vehicle in-vehicle host, 360-degree panoramic imaging system, video switching and radar recognition integrated circuit and vehicle central control screen; The original vehicle host is the host unit in the original vehicle, and the original vehicle video model collected by the original vehicle host is transmitted to the video switching and radar recognition integrated circuit. The video switching and radar recognition integrated circuit sends the data to the vehicle's central control screen, and sends radar information to the 360-degree panoramic imaging system. The 360-degree panoramic imaging system sends the panoramic video signal to the video switching and radar recognition integrated circuit. The video switching and radar recognition integrated circuit includes: an RX-8619 chip, a serial-to-parallel conversion unit, a host computer, a crosshair engine unit, a coordinate reordering management unit, a serial radar recognition processing unit, and a video switching unit. The RX-8619 chip receives GMS signals sent from the original vehicle's onboard host. The serial radar recognition processing unit, together with the radar recognition unit and the radar transmission unit, sends radar information to the 360-degree panoramic imaging system. The 360-degree panoramic imaging system sends the panoramic video stream to the video switching unit, and the video switching unit sends information data to the vehicle's central control screen. The serial radar identification and processing unit is internally equipped with a coordinate mapping unit, a serial positioning unit, a radar identification unit, and a radar transmission unit. The radar identification unit uses eight feature points selected within the video stream for identification, and the feature point information processed by the radar identification unit is communicated to the radar transmission unit in real time. The radar transmission unit is connected to the 360-degree panoramic imaging system for real-time communication, and the panoramic video stream data acquired by the 360-degree panoramic imaging system is transmitted to the video switching unit.

2. The vehicle radar recognition system based on image analysis principle according to claim 1, characterized in that: The serial-to-parallel conversion unit receives dual LVDS signals from the RX-8619 chip, and the serial-to-parallel conversion unit is equipped with a BitFlip module and an LVDS parsing module, and the LVDS parsing module processes the dual LVDS signals.

3. The vehicle radar recognition system based on image analysis principle according to claim 2, characterized in that: The BitFlip module processes the data stream received from the LVDS parsing module, and the first bit code input serial-to-parallel conversion unit of the BitFlip module decodes the serial data stream into parallel data.

4. The vehicle radar recognition system based on image analysis principle according to claim 1, characterized in that: The original vehicle RGB video stream data processed by the serial-to-parallel conversion unit is sent to the coordinate mapping unit, and the coordinate mapping unit transmits the data to the serial radar recognition and processing unit.

5. The vehicle radar recognition system based on image analysis principle according to claim 1, characterized in that: The serial radar identification processing unit receives sorted coordinate information from the coordinate reordering management unit, and the information from the coordinate reordering management unit is transmitted by the host computer.

6. The vehicle radar recognition system based on image analysis principle according to claim 5, characterized in that: The host computer data information is transmitted to the crosshair cursor engine unit, and the cursor position of the crosshair cursor engine unit is controlled by the host computer.

7. The vehicle radar recognition system based on image analysis principle according to claim 1, characterized in that: The video switching unit receives the original vehicle RGB video stream from the serial radar recognition and processing unit, and the video switching unit simultaneously transmits the panoramic video stream and the original vehicle RGB video stream to the vehicle's central control screen.

Images