Vehicle-mounted radar recognition system based on image analysis principle

Abstract

The invention relates to the technical field of automobile intelligent driving assistance, and discloses a vehicle-mounted radar recognition system based on an image analysis principle, which comprises an original vehicle-mounted host, a 360-degree panoramic image system, a video switching and radar recognition integrated circuit and a vehicle central control screen, the original vehicle-mounted host is a host part in an original vehicle, an original vehicle video model collected by the original vehicle-mounted host is transmitted to the video switching and radar identification integrated circuit, and the video switching and radar identification integrated circuit transmits the video to the vehicle central control screen. According to the vehicle-mounted radar recognition system based on the image analysis principle, a video switching and radar recognition integrated circuit arranged in the system is a semi-customized circuit and has a large number of logic units, and the logic units such as a coordinate mapping unit, a serial positioning unit, a radar recognition unit and a radar transmission unit are configured; and constructing a corresponding image processing circuit to carry out more detailed classification on the target object.

Description

Technical Field

[0001] The present invention relates to the field of automobile intelligent driving assistance technology, and in particular to a vehicle-mounted radar recognition system based on image analysis principles. Background Art

[0002] With the continuous increase in the number of cars, road traffic safety issues are becoming increasingly prominent. With the continuous development of automobile intelligent technology, traditional on-board radar systems have certain limitations in target recognition. They can only provide basic information such as the distance and speed of the target, and it is difficult to accurately identify the specific type and characteristics of the target. Therefore, the car 360 panoramic imaging system has become one of the hot research directions in the automotive field. This technology can provide drivers with a panoramic view, helping drivers to have a more comprehensive understanding of the road conditions and obstacles around the vehicle, and improve driving safety. At present, most of the high-end models of mainstream brands have been equipped with 360 panoramic imaging systems as standard, but the low-end models with larger sales need to be installed in the aftermarket, which leads to some problems in use, such as:

[0003] The 360-degree panoramic system in the aftermarket for low-end models obtains radar information by connecting to the vehicle-based CAN bus. Since all vehicle information communications use this interface, the amount of data is very large and complex, and it is easy to make mistakes. In addition, the wiring harness of the original vehicle needs to be destroyed during assembly, which poses certain safety hazards and affects subsequent repairs and maintenance. The video switching of the additional panoramic system uses an electronic switch, which will completely block the original vehicle screen when switching to the 360-degree panoramic screen, causing the air conditioning, volume and other information on the central control screen to be lost when reversing, and the original vehicle radar information cannot be obtained during use, making it difficult to transmit the original vehicle radar information. When reversing and switching to the additional panoramic screen, there will be no radar, affecting driving safety and user experience.

[0004] In response to the above problems, it is urgently necessary to carry out innovative designs based on the 360 ​​panoramic system installed in the original low-end models. Summary of the invention

[0005] The purpose of the present invention is to provide a vehicle-mounted radar recognition system based on the principle of image analysis, so as to solve the problem proposed in the above background technology that the traditional method of obtaining radar information by connecting to the vehicle-based CAN bus requires the destruction of the original vehicle's wiring harness during assembly, which poses certain safety hazards and affects subsequent maintenance and repair. The electronic switch used in its work will cause the original vehicle image to be completely shielded when switching to the 360 ​​panoramic image, and the original vehicle radar information cannot be obtained during use, making it difficult to transmit the original vehicle radar information. When reversing and switching to the installed panoramic image, there will be a situation where there is no radar, affecting driving safety and user experience.

[0006] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a vehicle-mounted radar recognition system based on the principle of image analysis, including an original vehicle-mounted host, a 360-degree panoramic imaging system, a video switch, and a radar recognition integrated circuit and a vehicle central control screen;

[0007] The original vehicle onboard host is the host part in the original vehicle, and the original vehicle video model collected by the original vehicle onboard host is transmitted to the video switching and radar recognition integrated circuit, and the video switching and radar recognition integrated circuit transmits it to the vehicle central control screen, and the video switching and radar recognition integrated circuit transmits radar information to the 360 ​​panoramic imaging system, and the 360 ​​panoramic imaging system sends the panoramic video signal to the video switching and radar recognition integrated circuit;

[0008] The video switching and radar recognition integrated circuit includes: an RX-8619 chip, a serial-to-parallel conversion unit, a host computer, a cross cursor engine unit, a coordinate reordering management unit, a serial radar recognition processing unit and a video switching unit, and the RX-8619 chip receives the GMS signal sent from the original vehicle's on-board host, and the serial radar recognition processing unit jointly with the radar recognition unit and the radar transmission unit sends radar information to the 360 ​​panoramic imaging system, and the 360 ​​panoramic imaging system sends the panoramic video stream to the video switching unit, and the video switching unit sends information data into the vehicle's central control screen.

[0009] By adopting the above technical solution, a serial processing channel is adopted and the ELVDS pin is used to implement the encoding and decoding of the differential signal, so that an FPGA with a relatively small proportion of resources can complete the function.

[0010] Preferably, the serial-to-parallel conversion unit receives the dual LVDS signals transmitted from the RX-8619 chip, and a BitFlip module and an LVDS analysis module are provided inside the serial-to-parallel conversion unit, and the LVDS analysis module processes the dual LVDS signals.

[0011] Using the above technical solution, dual LVDS signals are integrated and transmitted through the RX-8619 chip.

[0012] Preferably, the BitFlip module processes the data stream received from the LVDS analysis module, and the first bit code of the BitFlip module is input into the serial-to-parallel conversion unit to decode the serial data stream into parallel data.

[0013] Using the above technical solution, the BitFlip module uses the analyzed rules to reversely infer the alignment relationship through the hypothesis method.

[0014] Preferably, the serial radar identification processing unit is internally provided with a coordinate mapping unit, a serial positioning unit, a radar identification unit and a radar transmission unit.

[0015] The above technical solution is adopted to facilitate superimposing the radar coordinate mark of the original vehicle RGB video stream on the panoramic video stream.

[0016] Preferably, the original vehicle RGB video stream data processed by the serial-to-parallel conversion unit is sent to the coordinate mapping unit, and the coordinate mapping unit transfers the data to the serial radar recognition processing unit.

[0017] By adopting the above technical solution, the serial data stream is converted into parallel data through the serial-to-parallel conversion unit, which facilitates data transmission.

[0018] Preferably, the serial radar identification processing unit receives sorted coordinate information from a coordinate reordering management unit, and the information of the coordinate reordering management unit is transmitted by a host computer.

[0019] By adopting the above technical solution, the time sequence of the feature points of each frame of the acquired video is rearranged to facilitate mapping of the feature points to a radar group.

[0020] Preferably, the host computer data information is transmitted to a cross cursor engine unit, and the cursor position of the cross cursor engine unit is controlled by the host computer.

[0021] By adopting the above technical solution, the cursor position and record are controlled by the host computer, so that it is easy to load into the system in advance in the normal program.

[0022] Preferably, the radar recognition unit uses 8 feature points selected from the video stream for recognition, and the feature point information processed by the radar recognition unit is communicated to the radar transmission unit in real time.

[0023] By adopting the above technical solution, when the radar feature points in each frame of image are compounded, the existence of the radar position is confirmed.

[0024] Preferably, the radar transmission unit is connected to the 360-degree panoramic imaging system for real-time communication, and the panoramic video stream data acquired by the 360-degree panoramic imaging system is transmitted to the video switching unit.

[0025] By adopting the above technical solution, the radar data is received by the 360-degree panoramic imaging system to provide overlapping and synchronization for the video position.

[0026] Preferably, the video switching unit receives the original vehicle RGB video stream from the serial radar recognition processing unit, and the video switching unit synchronously transmits the panoramic video stream and the original vehicle RGB video stream to the vehicle's central control screen.

[0027] By adopting the above technical solution, the video switching unit provides two video stream viewing angles for the vehicle's central control screen, thereby improving the user experience.

[0028] Compared with the prior art, the invention has the following beneficial effects: the vehicle-mounted radar recognition system based on the image analysis principle:

[0029] 1. The video switching and radar identification integrated circuit set in this system is a semi-customized circuit, which has a large number of logic units. By configuring the coordinate mapping unit, serial positioning unit, radar identification unit and radar transmission unit and other logic units, the corresponding image processing circuit is constructed to classify the target object more carefully. As the coordinate reordering management unit and the host computer jointly adjust the cross cursor position of the cross cursor engine unit, the position of the radar in the original vehicle video stream is obtained and the RGB value when the radar exists is recorded. It is recorded and loaded in advance into the serial radar identification processing unit, so that the serial radar identification processing unit can send the processed radar information to the 360 ​​panoramic imaging system. The 360 ​​panoramic imaging system provides the original vehicle video and radar information for the vehicle central control screen. The video switching unit provides synchronous switching for the original vehicle RGB video stream and the 360 ​​panoramic video stream system, so as to realize the required processing function of smoothly switching the original vehicle video stream and the panoramic video stream;

[0030] 2. The upper computer in the system controls the cross cursor of the cross cursor engine unit, which is superimposed on the original car video screen during use. After 8 feature points are selected from each radar in the radar identification unit, as the radar transmission unit communicates with the 360 ​​panoramic mainboard in real time, the feature points of each radar are dispersed and the color component values ​​of overlap and non-overlap are recorded. If 5 / 8 or more feature points of a radar meet in each frame of the image, it can be set to the OPEN state. If 3-5 frames are continuously judged to be in the OPEN state, it can be considered that this radar exists. After the coordinate reordering management, the corresponding order is rearranged, so that the serial radar identification processing unit can overlay the processed radar information on the panoramic video of the vehicle's central control screen. The serial processing channel is adopted, and the ELVDS pin is used to realize the encoding and decoding of differential signals, so that the FPGA resources used are very small to complete this function. Compared with other conventional FPGA solutions on the market, the cost, power consumption and volume have great advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a schematic diagram of the project framework of the vehicle-mounted radar identification system of the present invention;

[0032] Figure 2 It is a simplified flow chart of the vehicle-mounted radar recognition system of the present invention. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] See also Figure 1-2 , the present invention provides a technical solution: a vehicle-mounted radar recognition system based on the principle of image analysis, including an original vehicle-mounted host, a 360-degree panoramic imaging system, a video switch, and a radar recognition integrated circuit and a vehicle central control screen;

[0035] The original vehicle onboard host is the host part in the original vehicle, and the original vehicle video model collected by the original vehicle onboard host is transmitted to the video switching and radar recognition integrated circuit, and the video switching and radar recognition integrated circuit transmits it to the vehicle central control screen, and the video switching and radar recognition integrated circuit transmits radar information to the 360 ​​panoramic imaging system, and the 360 ​​panoramic imaging system sends the panoramic video signal to the video switching and radar recognition integrated circuit;

[0036] The video switching and radar identification integrated circuit includes: an RX-8619 chip, a serial-to-parallel conversion unit, a host computer, a cross cursor engine unit, a coordinate reordering management unit, a serial radar identification processing unit and a video switching unit, and the RX-8619 chip receives the GMS signal sent from the original vehicle's on-board host, the serial radar identification processing unit jointly with the radar identification unit and the radar transmission unit sends radar information to the 360 ​​panoramic imaging system, and the 360 ​​panoramic imaging system sends the panoramic video stream to the video switching unit, and the video switching unit sends information data to the vehicle's central control screen, the serial-to-parallel conversion unit receives the dual LVDS signal transmitted from the RX-8619 chip, and the serial-to-parallel conversion unit is internally provided with a BitFlip module and a LVDS analysis module, and the LVDS analysis module processes the dual LVDS signal, the BitFlip module processes the data stream received from the LVDS analysis module, and the first bit code of the BitFlip module is input into the serial-to-parallel conversion unit to decode the serial data stream into parallel data;

[0037] The original vehicle RGB video stream data processed by the serial-to-parallel conversion unit is sent to the coordinate mapping unit, and the coordinate mapping unit transmits the data to the serial radar recognition processing unit, the serial radar recognition processing unit receives the sorted coordinate information from the coordinate reordering management unit, and the information of the coordinate reordering management unit is transmitted by the host computer, the host computer data information is transmitted to the cross cursor engine unit, and the cursor position of the cross cursor engine unit is controlled by the host computer, the video switching unit receives the original vehicle RGB video stream from the serial radar recognition processing unit, and the video switching unit synchronously transmits the panoramic video stream and the original vehicle RGB video stream to the vehicle central control screen;

[0038] The serial radar recognition processing unit is internally provided with a coordinate mapping unit, a serial positioning unit, a radar recognition unit and a radar transmission unit. The radar recognition unit recognizes 8 feature points selected in the video stream, and the feature point information processed by the radar recognition unit is communicated to the radar transmission unit in real time. The radar transmission unit is connected to the 360-degree panoramic imaging system for real-time communication, and the panoramic video stream data acquired by the 360-degree panoramic imaging system is transmitted to the video switching unit.

[0039] In conjunction with the accompanying drawings Figure 1-Figure 2 As shown, when in use, the original vehicle host sends the collected GMS signal to the RX-8619 chip, and the RX-8619 chip, namely the GMS-TO-LVDS, processes the dual LVDS signal and sends it to the BitFlip module in the serial-to-parallel conversion unit, as shown Figure 1 As shown, the dual LVDS signal is processed with the LVDS analysis module, wherein the LVDS analysis module uses the LVDS-TO-RGB888 data converter to convert LVDS to RGB for signal processing, wherein the BitFlip module and the LVDS analysis module process any serial data stream. Before parsing, it is necessary to first find the position of the first Bit. The usual LVDS analysis is to confirm the alignment relationship through the clock pin rule such as 1100011. The FPGA selected in this system is used to reduce production costs, and the LVDS pins required are numerous, with 2 sets of dual LVDS inputs and 1 set of dual LVDS outputs. When using, some ELVDS, namely pseudo-differential pins, are required. Since the pseudo-differential pins cannot match the IP core of the chip, the assumption method is used when obtaining the clock rule processing, and the alignment relationship is reversed using the rules after analysis. The analysis and reverse alignment relationship processing is as follows:

[0040] LVDS is a 7:1 serial data stream, so there are 7 possible first bits.

[0041] Assume that case 1 is correct, and then analyze the key video signals, namely VS, HS, and DE, in the parsed data stream;

[0042] If the analysis is correct, that is, case 1 is the correct first digit;

[0043] If it is wrong, continue to assume case 2 until all 7 cases are analyzed and the correct first code is found;

[0044] After the original car RGB video stream is sent to the serial-to-parallel conversion unit, the serial data stream is decoded into parallel data according to the first bit code obtained by the BitFlip module, and the VS, HS, and DE of the row field are extracted to the chip. The video stream signal used is used to send the original car RGB video stream that has been parsed and processed to the serial radar recognition processing unit, such as Figure 1 As shown;

[0045] The serial radar recognition processing unit synchronously receives the sorted coordinate information transmitted from the coordinate mapping unit. The sorted image source of the coordinate mapping unit is transmitted to the host computer, wherein the host computer uses the STM32 chip to guide the cross cursor position in the cross cursor engine unit, and the cross cursor in the cross cursor engine unit is superimposed on the original vehicle video, obtained by the host computer, and then added to the system in advance in the normal program. Since there are many radar scale lines in the vehicle system, there are usually 4 directions of front, back, left and right, and each direction has 5 channels. The radar level of each channel is 6, so the radar that needs to be identified is 4*5*6=120 line segments, and each line segment requires 8 feature points to verify. , so there are 120*8=960 features. In order to obtain the corresponding coordinate feature points faster, the coordinate reordering management unit reorders them according to the time sequence of the video signal. The coordinate mapping unit receives the reordered coordinate model. Since each radar scale line has 8 feature points for identification, they are all disordered after sorting. Therefore, each feature point needs to be mapped to a radar group and cached in RAM for subsequent restoration. In the restoration, the mapping relationship is stored in RAM through the coordinate mapping unit, and the color RGB values ​​of all points are also processed in the serial positioning unit and cached in RAM. Therefore, the radar identification unit directly extracts the RGB value of each radar corresponding point according to the mapping relationship for judgment;

[0046] The serial positioning unit is set to an idle state in the settings, and at the same time, the field synchronization video in the video is found. After entering 6, the timer synchronization state is entered again. A row and column counter is generated according to the valid signal of the video data. When the first coordinate is equal to the row and column counter, the current RGB, that is, the video color component value, is collected, and then the next coordinate that has been sorted is taken out, and the comparison is continued, and so on, until all the coordinates are processed and the idle state is re-entered. The 8 feature points selected by the radar identification unit when in use are used to identify each radar separately. Since there is a steering scale line in the vehicle interface of the vehicle's central control screen, the steering scale line can overlap with the radar at any place, so the feature points of each radar need to be dispersed and the color component values ​​when overlapping and non-overlapping are recorded. If a radar reaches 5 / 8 or more feature points in each frame of the image, it can be set to the OPEN state. If 3-5 frames are judged to be in OPEN state continuously, it can be considered that the radar exists. After judging the radar information, the radar transmission unit communicates with the 360 ​​panoramic mainboard of the 360 ​​panoramic imaging system in real time to ensure timeliness. The two are connected using the serial port protocol. After the protocol content is determined by negotiation with the software, the FF00DD01 information is obtained, where FF00 is a fixed packet header, DD is the number of radar lines, 01 is the existence of the radar, and if it is 00, the radar does not exist. When the 360 ​​panoramic imaging system transmits the panoramic video stream to the vehicle's central control screen, the video switching unit receives the panoramic video stream received by the sensor and the original car RGB video stream to cooperate with the driver to switch the environment. Among them, the original car video and the 360 ​​video are both input into the FPGA chip, so the switching of the system video is the selection of the internal code of the FPGA chip, thereby realizing video switching;

[0047] This system accurately identifies the radar through the analysis of the original vehicle image recognition. The overall serial processing channel is adopted and the ELVDS pin is used to realize the encoding and decoding of the differential signal, while avoiding the removal of the vehicle and breaking of the original center line. After the image analysis is integrated with the vehicle-mounted radar data, the system can better adapt to the complex and changing environment. In severe weather conditions, although the image quality may be affected to a certain extent, the radar data can be used as a supplement and can still provide the distance and speed information of the target object. In foggy weather, the radar can detect the approximate position of the object in front. At the same time, the image analysis can try to use limited visual information, such as the blurred outline of the object, for further identification. The combination of the two improves the overall perception ability and enhances the ring adaptability of the system.

[0048] Working principle: When using the vehicle-mounted radar recognition system based on the image analysis principle, the 360 ​​panoramic imaging system first obtains the original vehicle video synchronously with the original vehicle's on-board host. The serial radar recognition processing unit located inside the video switching and radar recognition integrated circuit connects the host computer and the cross cursor engine unit to control the cross cursor position for the panoramic video superposition. The image with the correct time sequence is marked by the coordinate reordering management unit, and each frame of the panoramic video stream received from the serial radar recognition processing unit is compared. The vehicle's central control screen displays the radar status of the image coverage area to the owner. After the image analysis and vehicle-mounted radar data are fused, the system can better adapt to complex and changing environments. By utilizing the dual advantages of image analysis and radar to identify target objects, it can flexibly respond to various traffic scenarios.

[0049] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and alterations may be made to the embodiments without departing from the principles and spirit of the invention.

Claims

1. A vehicle-mounted radar recognition system based on image analysis principle, characterized in that: include: The original vehicle host, 360-degree panoramic imaging system, video switching, radar recognition integrated circuit and vehicle central control screen; The original vehicle onboard host is the host part in the original vehicle, and the original vehicle video model collected by the original vehicle onboard host is transmitted to the video switching and radar recognition integrated circuit, and the video switching and radar recognition integrated circuit transmits it to the vehicle central control screen, and the video switching and radar recognition integrated circuit transmits radar information to the 360 ​​panoramic imaging system, and the 360 ​​panoramic imaging system sends the panoramic video signal to the video switching and radar recognition integrated circuit; The video switching and radar recognition integrated circuit includes: an RX-8619 chip, a serial-to-parallel conversion unit, a host computer, a cross cursor engine unit, a coordinate reordering management unit, a serial radar recognition processing unit and a video switching unit, and the RX-8619 chip receives the GMS signal sent from the original vehicle's on-board host, and the serial radar recognition processing unit jointly with the radar recognition unit and the radar transmission unit sends radar information to the 360 ​​panoramic imaging system, and the 360 ​​panoramic imaging system sends the panoramic video stream to the video switching unit, and the video switching unit sends information data into the vehicle's central control screen.

2. The vehicle-mounted radar recognition system based on image analysis principle according to claim 1 is characterized by: The serial-to-parallel conversion unit receives the dual LVDS signals transmitted from the RX-8619 chip, and a BitFlip module and an LVDS analysis module are arranged inside the serial-to-parallel conversion unit, and the LVDS analysis module processes the dual LVDS signals.

3. The vehicle-mounted radar recognition system based on image analysis principle according to claim 2 is characterized by: The BitFlip module processes the data stream received from the LVDS analysis module, and the first bit code of the BitFlip module is input into the serial-to-parallel conversion unit to decode the serial data stream into parallel data.

4. The vehicle-mounted radar recognition system based on image analysis principle according to claim 1 is characterized by: The serial radar identification processing unit is internally provided with a coordinate mapping unit, a serial positioning unit, a radar identification unit and a radar transmission unit.

5. The vehicle-mounted radar recognition system based on image analysis principle according to claim 1 is characterized in that: The original vehicle RGB video stream data processed by the serial-to-parallel conversion unit is sent to the coordinate mapping unit, and the coordinate mapping unit transmits the data to the serial radar recognition processing unit.

6. The vehicle-mounted radar recognition system based on image analysis principle according to claim 1 is characterized by: The serial radar identification processing unit receives the sorted coordinate information from the coordinate reordering management unit, and the information of the coordinate reordering management unit is transmitted by the host computer.

7. The vehicle-mounted radar recognition system based on image analysis principle according to claim 6 is characterized by: The host computer data information is transmitted to the cross cursor engine unit, and the cursor position of the cross cursor engine unit is controlled by the host computer.

8. The vehicle-mounted radar recognition system based on image analysis principle according to claim 4 is characterized by: The radar recognition unit uses 8 feature points selected from the video stream for recognition, and the feature point information processed by the radar recognition unit is communicated to the radar transmission unit in real time.

9. The vehicle-mounted radar recognition system based on image analysis principle according to claim 4 is characterized by: The radar transmission unit is connected to the 360-degree panoramic imaging system for real-time communication, and the panoramic video stream data acquired by the 360-degree panoramic imaging system is transmitted to the video switching unit.

10. The vehicle-mounted radar recognition system based on image analysis principle according to claim 1, characterized in that: The video switching unit receives the original vehicle RGB video stream from the serial radar recognition processing unit, and the video switching unit synchronously transmits the panoramic video stream and the original vehicle RGB video stream to the vehicle central control screen.

Images