Vehicle-mounted ADB system architecture based on radar and vision fusion and use method

Abstract

The application provides a vehicle-mounted ADB system frame based on radar and vision fusion, which comprises a data collection module, an ADB intelligent decision module and an ADB lamp control module connected in sequence, the ADB intelligent decision module comprises a CAN communication module and a processor, the CAN communication module is connected with the processor, the CAN communication module is connected with the ADB lamp control module, the data collection module comprises a millimeter wave radar and a camera, the millimeter wave radar and the camera are arranged on the front side of a vehicle, the millimeter wave radar is connected with the camera, the camera is connected with the processor, and the left headlamp and the right headlamp of the vehicle are connected with the ADB lamp control module. The application realizes interactive fusion of radar and camera data, can ensure that the data source is accurate and reliable, and improves the robustness of the system. The method can utilize millimeter wave radar data to assist camera collected video stream data, perform accurate physical position information calculation, adapt to various severe weather, such as rain, snow, fog and the like, and increase the application scenarios of the ADB system.

Description

Technical Field

[0001] This invention relates to the technical field of vehicle lighting systems, and in particular to the architecture and usage method of an in-vehicle ADB system based on radar-visual fusion. Background Technology

[0002] Adaptive driving beam (ADB) is an intelligent adaptive high beam control system that adjusts the high beam projection according to driving scenarios. It ensures good visibility for the driver of the vehicle while avoiding glare for other road users. ADB introduces the concept of a "matrix beam," primarily relying on the high-resolution LED matrix bulbs in the headlight module to create the illumination beam. ADB intelligently turns the high and low beams on and off based on the vehicle's driving status, traffic conditions, and road traffic conditions. It identifies targets such as vehicles and pedestrians using camera data and individually controls each luminous unit, turning off certain illuminated areas and adaptively changing the high beam distribution to avoid glare for other road users, as shown in Figure 1. ADB can handle various scenarios including oncoming traffic, following other vehicles, single-target mode, and multi-target mode, achieving real-time dynamic adjustment.

[0003] Traditional in-vehicle ADB system architecture, such as Figure 2 As shown, the system uses a camera to collect road vehicle information as raw data, performs intrinsic and extrinsic parameter calibration on the camera, and realizes the conversion between real-world coordinates and camera pixel coordinates. The camera data is transmitted to the processor via the CAN bus. The processor calculates the real distance and corresponding angle between the detected vehicle and the self-driving vehicle, and obtains the rectangular coordinates of the two sides of the target relative to the LED matrix light group based on the position and angle data. Then, it obtains the spherical coordinates of the two sides of the target with the LED matrix light group as the origin and the minimum occlusion angle of the LED matrix light group (if the image data shows that there is no target vehicle in front of the self-driving vehicle, the LED matrix light group maintains the high beam state; if the image data shows that there is a target vehicle in front of the self-driving vehicle, the LED light in the LED matrix light group corresponding to the minimum occlusion angle is turned off). The processor sends the above lighting control information to the LED matrix light bead control switch via the CAN bus, and the matrix light bead performs the corresponding operation to realize the adaptive high and low beam switching of the headlight beam.

[0004] As can be seen from the above system architecture, since only the camera provides information flow data to the ADB system, the following problems may arise in the inaccurate control of the LEDs: 1. When the target vehicle or pedestrian in the image is far from the driving vehicle and the detection is inaccurate, the matrix LED shut-off area will not match; 2. When the target is close to the vehicle body, the target area in the image increases, which will cause the calculated occlusion angle to be too large, resulting in far more LEDs being shut off than should be. In addition, when facing severe weather, the camera's image data may become unreliable: for example, rain, snow, and fog may cause the camera to be blurred or obstructed, resulting in inaccurate data acquisition by the camera, such as blurred radial positions of vehicles and pedestrians, making it impossible to accurately locate vehicles and pedestrians, seriously affecting the accuracy of the matrix LED shut-off area. Summary of the Invention

[0005] To address the technical problem of inaccurate LED control in traditional systems, this invention proposes an in-vehicle ADB system architecture and implementation method based on radar-visual fusion. The intelligent scene recognition ADB system architecture, which uses video data and millimeter-wave radar data, uses millimeter-wave radar data to compensate for the shortcomings of video image data. This effectively solves the problem of ambiguity in the radial position of vehicles and pedestrians detected by video data, enabling precise positioning of vehicles and pedestrians, and based on this, accurate control of LEDs.

[0006] The beneficial effects of this invention are as follows:

[0007] (1) Data acquisition adds millimeter-wave radar to the traditional architecture, which can more accurately detect the specific physical location information of other vehicles and pedestrians on the road. At the same time, the high-resolution image data acquired by the camera is interactively fused with the data detected by the radar. The radar and camera assist each other to jointly determine the accurate location information of the object. The traditional ADB system architecture may have some misjudgments when using data acquired by a single camera, such as misidentification in certain specific situations. Using radar and camera data for interactive fusion can ensure the accuracy of the data source and improve the robustness of the system.

[0008] (2) It can adapt to various harsh weather conditions, such as rain, snow, and fog. Traditional cameras only use the camera for data collection, and in rain, snow, and fog, the images captured by the camera may be blurry or unclear, thus limiting their application scenarios. However, the vehicle-mounted ADB system architecture based on radar-visual fusion adds millimeter-wave radar data, which can be used to assist the video stream data collected by the camera to perform accurate physical location information calculation, thus increasing the application scenarios of the ADB system. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 For headlight shapes in scenarios with / without ADB (Adaptive Dashboard)

[0011] Figure 2 It is based on the traditional in-vehicle ADB system architecture.

[0012] Figure 3 This is an in-vehicle ADB system architecture based on radar-visual fusion.

[0013] Figure 4 This is a diagram showing the transformation relationships between the five coordinate systems.

[0014] Figure 5 This is a diagram showing the coordinate system relationship.

[0015] Figure 6 This is a diagram showing the relationship between radar coordinates and the world coordinate plane.

[0016] Figure 7 This is a schematic diagram of pinhole imaging in a camera.

[0017] Figure 8 This is a graph showing the relationship between the image and pixel coordinates.

[0018] Figure 9 This is a schematic diagram of radial distortion.

[0019] Figure 10 This is a schematic diagram of tangential distortion.

[0020] Figure 11 This is the YOLOv3 network structure.

[0021] Figure 12 This is a schematic diagram of the nearest neighbor matching process. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Example 1

[0024] like Figure 1As shown, a vehicle-mounted ADB system architecture based on radar-visual fusion includes a data collection module, an ADB intelligent decision-making module, and an ADB lighting control module connected in sequence. The ADB intelligent decision-making module includes a CAN communication module and a processor. The CAN communication module is connected to the processor and the ADB lighting control module. The data collection module includes a millimeter-wave radar and a camera. Both the millimeter-wave radar and the camera are located on the front of the vehicle. The millimeter-wave radar is connected to the camera, and the camera is connected to the processor via a coaxial cable. The left and right headlights of the vehicle are both connected to the ADB lighting control module. The millimeter-wave radar and camera are mainly used to collect radar data and image data from the environment to obtain the world coordinates of pedestrians and vehicles in front of the vehicle and their positions in the image. The ADB decision-making module is mainly used to determine how to control the ADB matrix LEDs. The ADB lighting control module is mainly used to control the ADB matrix LEDs in the left and right headlights according to the control scheme selected by the ADB decision-making module.

[0025] Specifically, the CAN communication module includes Channel I and Channel II, both of which are connected to the processor. The millimeter-wave radar is connected to Channel I via the CAN bus, and Channel II is connected to the ADB lighting control module via the CAN bus. The processor is an Nvidia Xavier processor, and the millimeter-wave radar is an ARS408 millimeter-wave radar.

[0026] Example 2

[0027] The specific steps for using the vehicle-mounted ADB system architecture based on radar-visual fusion are as follows:

[0028] S1: Data is collected using millimeter-wave radar and cameras, and the collected data is transmitted to the ADB decision module.

[0029] Specifically, millimeter-wave radar is used to collect radial and lateral distance data of vehicles and pedestrians on the road. The radar data obtained by the millimeter-wave radar is set to form world coordinates and transmitted to the ADB decision module via CAN bus. Cameras collect high-definition image data of the road conditions. The image data obtained by the cameras is set to image coordinates and transmitted to the ADB decision module via coaxial cable.

[0030] The camera transmits the collected image information to the processor in real time via a coaxial cable, and the processor uses... The algorithm detects pedestrians and vehicles in the video and marks people and vehicles with borders on each frame of the video image.

[0031] The millimeter-wave radar is the ARS408 millimeter-wave radar. This radar acquires radar data of the environment in front of the vehicle and returns the data in the form of CAN messages. The CAN messages returned by the millimeter-wave radar contain information such as the target sequence number, the relative radial distance (Range) between the target and the radar, the relative speed (Verl), the azimuth (Azimuth), the radar cross-section (RCS), and the signal-to-noise ratio (SNR). However, the fields in the CAN message do not directly represent the actual values ​​of the required information; each field needs to be parsed to obtain the true values.

[0032] The analytical formulas are shown in Table 1:

[0033] Table 1 CAN Message Parsing Formula

[0034]

[0035] S2: Perform camera calibration to ensure a one-to-one correspondence between world coordinates and image coordinates, thereby converting the world coordinates obtained from radar data parsing into image coordinates. The specific method is as follows:

[0036] S21: Using the established transformation relationship between the image coordinate system and the world coordinate system, perform camera calibration:

[0037] Image coordinate system With world coordinate system The correlation is also known as camera extrinsic parameter calibration, as shown in Figure 5. The image coordinate system and the world coordinate system have a correlation. Offset in direction (i.e., there is a height difference between the camera position and the bottom of the front of the car) and Offset in direction (That is, there is a certain distance between the camera position and the front bottom of the car).

[0038] At the same time, since cameras cannot be perfectly horizontal and vertical during actual installation, there will be a rotational distance between the image coordinate system and the world coordinate system. , , The relationship between the pitch, yaw, and roll of the shaft.

[0039] Define a three-dimensional translation vector and rotation matrix The transformation relationship between the image coordinate system and the world coordinate system is as follows:

[0040] ;

[0041] in, The x-axis coordinate of the image coordinate system. The y-axis coordinate of the image coordinate system. The z-axis coordinate of the image coordinate system. The x-axis coordinate in the world coordinate system. The y-axis coordinate in the world coordinate system. The z-axis coordinate in the world coordinate system.

[0042] For ease of subsequent calculations, it is converted into a fourth-order matrix form here:

[0043] ;

[0044] In the formula for The camera extrinsic matrix.

[0045] S22: Transform the world coordinate system to the pixel coordinate system using a matrix:

[0046] The essence of an image is its size. A two-dimensional matrix, where each element is a pixel, is a coordinate system for the image. With pixel coordinate system The relationship is shown in Figure 8. For being in the first Liede The pixel of the row, the origin of the image coordinate system The corresponding coordinates in the pixel coordinate system are And this point is located at the center of the pixel coordinate system.

[0047] set up and Each pixel is located in the image coordinate system. Axial direction and The unit length in the axial direction, then and The relationship is:

[0048] ;

[0049] The transformation relationship between the pixel coordinate system and the image coordinate system is as follows:

[0050] .

[0051] The transformation relationship between pixel coordinate system and image coordinate system is as follows:

[0052] ;

[0053] in for The camera intrinsic parameter matrix.

[0054] S23: Based on the transformation relationship between the image coordinate system and the world coordinate system, and the transformation relationship between the pixel coordinate system and the image coordinate system, the transformation relationship between the pixel coordinate system and the radar coordinate system is as follows:

[0055] ;

[0056] The transformation relationship between pixel coordinate system and radar coordinate system is used to convert the world coordinates collected by millimeter-wave radar into image coordinates.

[0057] S24: Use distortion correction formulas to correct distortion in images fused with information.

[0058] Actual camera imaging is not a simple, ideal pinhole imaging process; it exhibits a certain degree of distortion, including radial distortion caused by the lens shape and tangential distortion generated during camera assembly. Schematic diagrams of these two distortions are shown in Figures 9 and 9, respectively. Figure 10 As shown.

[0059] The distortion correction formula for a camera is:

[0060]

[0061] in, These are the corrected pixel coordinates. For the pixel coordinates of the ideal image, , This is the radial distortion correction factor. , This is the tangential distortion correction factor. The coordinates of the image pixels from the fused information are input into the distortion correction formula for correction, thus preventing subsequent failures in pedestrian and vehicle recognition due to image deformation.

[0062] S3: Use The algorithm compares the world coordinates converted to image coordinates with the positions of pedestrians and vehicles in the previously captured image data, thereby achieving the fusion and matching of the two sets of data.

[0063] It is a process , The improved regression-based network runs fast while maintaining accuracy, meeting the requirements for real-time detection, and also enhances the ability to identify and detect small objects. Prior detection systems repeatedly use classifiers or locators to perform detection tasks. Networks can be applied to different locations within images of varying sizes. Network structure such as Figure 11 As shown.

[0064] The specific method is as follows:

[0065] S31: The nearest neighbor frame matching algorithm is used to synchronize millimeter-wave radar data with visual data in time.

[0066] After spatial calibration of millimeter-wave radar and camera data, since they operate at different time frequencies and there is a delay in the data communication between the sensors, the sampling frame frequency of millimeter-wave radar is 20 frames / second, while that of the camera is 30 frames / second. Therefore, the inconsistent sampling frequencies mean that the data collected by the sensors cannot be effectively matched for every frame. Thus, synchronizing the radar coordinates and video detection data of the same target in time is the key to improving the accuracy of target fusion tracking.

[0067] Common time registration methods include interpolation and extrapolation. Interpolation and extrapolation primarily extrapolate data from a high-precision measurement timeline to a low-precision measurement timeline. First, time slices are defined, with the criteria based on the target's motion state: faster motion results in shorter fusion time slices. Then, sensor measurement data are sorted according to measurement time precision. Finally, each high-precision time data point is interpolated and extrapolated to the lowest precision time point to complete the time registration operation.

[0068] First, timestamped millimeter-wave radar data is obtained from radar data collected by the radar. Timestamped image frame data is obtained from image data collected by the camera. The timestamps of both the timestamped millimeter-wave radar data and the timestamped image frame data are extracted and subtracted from the corresponding average delay. Then, nearest neighbor matching is performed between the radar data and the image frame data. Figure 12 As shown.

[0069] Let the set of image data frames be defined as:

[0070] ;

[0071] The radar data frame set is defined as follows:

[0072] ;

[0073] The timestamps for the image data frames and the true set of radar point clouds are as follows:

[0074] ;

[0075] Because the camera clock and millimeter-wave radar clock are not perfectly synchronized with the host clock, there is a delay between them. Therefore, let the average delay between the camera clock and the host clock be... The average delay between the millimeter-wave radar clock and the host clock is Due to the image data frame rate and millimeter-wave radar frame rate Due to the difference, a lower frame rate sensor was selected, and its frame rate was denoted as . Set the frame time difference threshold The frame synchronization time accuracy is expressed by the formula:

[0076] ;

[0077] The sliding window method is used to fuse image frame data and millimeter-wave radar frame data to generate a result set. That is, for... :

[0078] ;

[0079] For the result set Each set of data These are all data groups whose synchronization time error is less than the frame time difference threshold.

[0080] S32: Fusion of millimeter-wave radar and visual information.

[0081] Since this invention requires fusing target category information obtained by the camera and target distance and velocity information obtained by the millimeter-wave radar, and the two data types are different, and considering the requirements of real-time detection, the computational load during data fusion cannot be too high, so decision-level fusion is adopted.

[0082] After completing the association of the five coordinate systems and the camera calibration and data time synchronization, the millimeter-wave radar scanning points can be projected into the pixel coordinate system. The camera detection results can output the pixel coordinate information of the four corner points of its bounding box. Therefore, the millimeter-wave radar detection results and camera detection results can be matched based on whether the projection point of the radar scanning point in the pixel coordinate system is within the target bounding box.

[0083] Suppose the pixel coordinates of the four corner points of a target bounding box are as follows: , , and The coordinates of the radar scan points projected onto the pixel coordinate system are: Since the radar is installed at a lower position than the camera, the projection of its scanning point in the pixel coordinate system is generally not located at the center of the target bounding box, but it is always located inside the bounding box. Therefore, the judgment rule can be set as follows: and For targets that meet the judgment rules, information fusion processing is performed, and their category, speed, and distance information are output simultaneously; for targets that do not meet the above judgment rules, no fusion processing is performed. In the pixel coordinate system, if no radar projection point is matched inside the target bounding box, the radar misses the target and only outputs its category information; if no target bounding box is matched around the radar projection point, the camera misses the target and only outputs its speed and distance information.

[0084] exist Figure 11 In the middle, a of The image undergoes operations such as convolution, pooling, upsampling, and tensor concatenation to generate feature maps at three scales, namely... : , : , : In the image above, Represents convolutional layer, Normalized layers and Activation layer. express , and indivual ,in Represents a residual structure, an original tensor and tensor After two Layers, then tensors operate. express , and combination, This indicates an upsampling operation. This indicates a convolution operation. This indicates a tensor concatenation operation. We use the YOLOv3 model to detect pedestrians / vehicles in the video and output the resulting bounding boxes on each frame.

[0085] S4: Use the fused result to send out lighting control information.

[0086] S41: Following step S3, the interaction and fusion of millimeter-wave radar and camera information streams have been achieved. For targets conforming to the judgment rules, their original radar data and video data are acquired to calculate the angle between the target and the vehicle. Let the coordinates of the camera device on the image pixels be... We use radar data as the standard, and the coordinates of the radar data on the image pixels are... included angle The angle between the millimeter-wave radar and the object matched in each frame of the camera video stream is calculated sequentially. Finally, the included angles of all objects Find the union of the sets to obtain the included angle corresponding to this frame of data. ,in To express finding the union, for example... It is between 10° and 20°. It is between 15° and 30°, then The range is from 10° to 30°.

[0087] S42: The angle between the detected vehicle and the self-vehicle is obtained through 1). Based on the one-to-one correspondence between the pre-set angle and the ADB matrix lamp control information, the corresponding ADB matrix lamp control information is selected and sent to the ADB matrix lamp control module through the CAN bus. The module performs the corresponding operation to realize the high beam switching control of the ADB matrix lamp.

[0088] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A vehicle-mounted ADB system architecture based on radar-visual fusion, characterized in that, The system includes a data collection module, an ADB intelligent decision-making module, and an ADB lighting control module connected in sequence. The ADB intelligent decision-making module includes a CAN communication module and a processor. The CAN communication module is connected to the processor. Both the CAN communication module and the processor are connected to the data collection module. The CAN communication module is connected to the ADB lighting control module. The data collection module includes a millimeter-wave radar and a camera. Both the millimeter-wave radar and the camera are located on the front side of the vehicle. The millimeter-wave radar is connected to the CAN communication module. The camera is connected to the processor. The left and right headlights of the vehicle are both connected to the ADB lighting control module. The steps for its use are as follows: S1: Data is collected using millimeter-wave radar and cameras, and the collected data is transmitted to the ADB decision module; S2: Perform camera calibration on the camera, converting the world coordinates obtained from radar data parsing to image coordinates; S3: Compare the world coordinates converted to image coordinates with the positions of pedestrians and vehicles in the previously captured image data to achieve the fusion and matching of radar data and camera data; S4: Use the fused results to send out lighting control information; The method for fusing and matching radar data and camera data information in step S3 is as follows: S31: Employ the nearest neighbor frame matching algorithm to synchronize millimeter-wave radar data with visual data in time; S32: Fusion of millimeter-wave radar with visual information; The method for synchronizing millimeter-wave radar data with visual data using the nearest neighbor frame matching algorithm described in step S31 is as follows: First, timestamped millimeter-wave radar data is obtained from the radar data collected by the radar. Timestamped image frame data is obtained from the image data collected by the camera. The timestamps of the timestamped millimeter-wave radar data and the timestamped image frame data are extracted and the corresponding average delay is subtracted. Then, nearest neighbor matching is performed between the radar data and the image frame data. Let the set of image data frames be defined as: ； The radar data frame set is defined as follows: ； The timestamps for the image data frames and the true set of radar point clouds are as follows: ； Assume the average delay between the camera clock and the host clock is... The average delay between the millimeter-wave radar clock and the host clock is ; Due to image data frame rate and millimeter-wave radar frame rate Due to the difference, a lower frame rate sensor was selected, and its frame rate was denoted as . Set the frame time difference threshold The frame synchronization time accuracy is expressed by the formula: ； The sliding window method is used to fuse image frame data and millimeter-wave radar frame data to generate a result set. : ； For the result set Each set of data These are all data groups whose synchronization time error is less than the frame time difference threshold.

2. The vehicle-mounted ADB system architecture based on radar-visual fusion as described in claim 1, characterized in that, The CAN communication module includes channel I and channel II, both of which are connected to the processor. The millimeter-wave radar is connected to channel I via a CAN bus, and channel II is connected to the ADB lighting control module via a CAN bus.

3. The vehicle-mounted ADB system architecture based on radar-visual fusion as described in claim 2, characterized in that, The processor is an Nvidia Xavier processor.

4. The vehicle-mounted ADB system architecture based on radar-visual fusion as described in claim 3, characterized in that, The millimeter-wave radar is the ARS408 millimeter-wave radar.

5. The vehicle-mounted ADB system architecture based on radar-visual fusion according to any one of claims 1-4, characterized in that, The method for converting the world coordinates obtained from radar data parsing to image coordinates as described in step S2 is as follows: S21: Using the established transformation relationship between the image coordinate system and the world coordinate system, perform camera calibration: Image coordinate system With world coordinate system The correlation, also known as camera extrinsic calibration, involves the existence of an image coordinate system and a world coordinate system. Offset in direction and Offset in direction Because cameras cannot be perfectly horizontal and vertical during actual installation, there will be a gap between the image coordinate system and the world coordinate system. , , The relationship between the pitch, yaw, and roll of the shaft; Define a three-dimensional translation vector and rotation matrix The transformation relationship between the image coordinate system and the world coordinate system is as follows: ； in, The x-axis coordinate of the image coordinate system. The y-axis coordinate of the image coordinate system. The z-axis coordinate of the image coordinate system. The x-axis coordinate in the world coordinate system. The y-axis coordinate in the world coordinate system. The z-axis coordinate in the world coordinate system; Here it is converted into a fourth-order matrix form: ； In the formula for The camera extrinsic matrix; S22: Transform the world coordinate system to the pixel coordinate system using a matrix: Let the image be of size . A two-dimensional matrix, where each element is a pixel. For being in the first Liede The pixel of the row, the origin of the image coordinate system The corresponding coordinates in the pixel coordinate system are And this point is located at the center of the pixel coordinate system; set up and Each pixel is located in the image coordinate system. Axial direction and The unit length in the axial direction, then and The relationship is: ； The transformation relationship between the pixel coordinate system and the image coordinate system is as follows: ； The transformation relationship between pixel coordinate system and image coordinate system is as follows: ； in for The camera intrinsic parameter matrix; S23: Utilize the transformation relationship between the pixel coordinate system and the radar coordinate system to convert the world coordinates collected by the millimeter-wave radar into image coordinates; S24: Perform distortion correction on the information-fused image using a distortion correction formula: Actual camera imaging is not a simple, ideal pinhole imaging process; it exhibits a certain degree of distortion, including radial distortion caused by the lens shape and tangential distortion generated during camera assembly. The distortion correction formula for a camera is: ； The corrected x-coordinate of the pixel. Here, x represents the corrected ordinate of the pixel, x represents the x-coordinate of the pixel in the ideal image, and y represents the ordinate of the pixel in the ideal image. , This is the radial distortion correction factor. , This is the tangential distortion correction factor. The coordinate information of the image pixels fused with information is input into the distortion correction formula for correction.

6. The vehicle-mounted ADB system architecture based on radar-visual fusion as described in claim 5, characterized in that, The specific method for fusing millimeter-wave radar and visual information as described in step S32 is as follows: Suppose the pixel coordinates of the four corner points of a target bounding box are as follows: , , and The coordinates of the radar scan points projected onto the pixel coordinate system are: The judgment rule is set as follows: and For targets that meet the judgment rules, information fusion processing is performed, and their category, speed, and distance information are output simultaneously; for targets that do not meet the above judgment rules, no fusion processing is performed. In the pixel coordinate system, if no radar projection point is matched inside the target bounding box, the radar will miss the target and only output its category information; if no target bounding box is matched around the radar projection point, the camera will miss the target and only output its speed and distance information.

7. The vehicle-mounted ADB system architecture based on radar-visual fusion according to claim 1 or 6, characterized in that, The method for issuing lighting control information using the fused result described in step S4 is as follows: S41: Let the coordinates of the camera device on the image pixels be... We use radar data as the standard, and the coordinates of the radar data on the image pixels are... included angle The angle between the millimeter-wave radar and the object matched in each frame of the camera video stream is calculated sequentially. Finally, the included angles of all objects Find the union of the sets to obtain the included angle corresponding to this frame of data. ,in This represents finding the union of sets; S42: The angle between the detected vehicle and the self-vehicle is obtained through S41. Based on the one-to-one correspondence between the pre-set angle and the ADB matrix lamp control information, the corresponding ADB matrix lamp control information is selected and sent to the ADB matrix lamp control module through the CAN bus. The module performs the corresponding operation to realize the high beam switching control of the ADB matrix lamp.

Images