ADB anti-interference control method and control system based on multi-modal feature fusion

Abstract

The invention relates to the technical field of intelligent automobile lighting control, in particular to an ADB anti-interference control method and system based on multi-modal feature fusion. The control method comprises the following steps: performing clock synchronization on a camera, a millimeter wave radar and a laser radar by adopting a PTP protocol, and unifying multi-source data to a vehicle coordinate system through a homogeneous transformation matrix; extracting an image feature vector Vcam through an improved YOLOv8 model integrated with a CBAM attention module, and extracting a radar point cloud feature vector Vradar by using a PointNet + + network; splicing the two feature vectors, generating a dynamic weight w through an embedded neural network, carrying out adaptive weighted fusion according to a fusion feature formula, and carrying out L2 normalization processing; according to the fusion result, the road curvature and the relative speed are combined, the light pattern rotation angle and the penumbra region gradient are dynamically calculated, shielding mask data are generated, the LED matrix partition brightness is controlled through PWM dimming, and high-precision and low-glare self-adaptive light beam regulation and control are achieved; and the anti-interference capability in a complex environment is effectively improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of intelligent automobile lighting control, and in particular to an ADB anti-interference control method and system based on multi-modal feature fusion. BACKGROUND

[0002] As a key technology for improving night driving safety and driving experience, the adaptive driving beam (ADB) system has become an important development direction of modern intelligent vehicle lamps. However, the current ADB system based on a mechanical light-shielding plate still has significant technical bottlenecks and performance defects in practical application, which seriously restricts the full play of its effectiveness.

[0003] In terms of environmental perception, the existing system relies on a single visual sensor, which has a lack of perception dimension and weak anti-interference ability. In rainy, foggy, snowy and other adverse weather conditions, the imaging quality of the camera deteriorates sharply, resulting in a high target false detection rate of up to 32%; and in the strong light backlight scene at the entrance and exit of the tunnel, the dynamic range of the camera is insufficient, causing a missed detection rate of more than 40%, and the system reliability is difficult to guarantee. In terms of execution response, the mechanical light-shielding plate has a response delay of more than 300 ms due to its inherent physical inertia. In high-speed meeting and other dynamic scenes with a relative speed of more than 80 km / h, such a high delay causes the shielding action to lag seriously, and the actual shielding failure probability is as high as 67%, which cannot achieve effective anti-dazzling of oncoming vehicles. In terms of light type quality, the mechanical light-shielding plate can only produce a simple rectangular shielding area, and the harsh edges will form a wide and bright half-shadow area. Tests show that the glare intensity of the half-shadow area exceeds 400 cd / m2, causing serious dazzling interference to other road participants, and there is a safety hazard. In terms of scene adaptability, the existing system lacks intelligent response capability to complex driving scenes, especially in the curve scene. When the road curvature is greater than 0.05 rad, due to the lack of effective dynamic compensation algorithm, the system shielding area deviates seriously from the actual oncoming vehicle position, with a deviation error of more than 1.5°, resulting in a basically ineffective shielding function, which exposes the serious limitations of its scene adaptability.

[0004] Therefore, the existing mechanical ADB system has systematic deficiencies in perception robustness, response real-time, light type accuracy and scene adaptability, and a new technical solution is urgently needed to solve the problem. SUMMARY

[0005] The technical problem to be solved by the application is to solve the problem of the existing technology in the background that the existing technology has deficiencies in perception robustness, response real-time, light type accuracy and scene adaptability. The application provides an ADB anti-interference control method based on multi-modal feature fusion, which fuses multiple sensor data, processes the data through a neural network, and finally controls an LED matrix to generate an adaptive anti-dazzling light type.

[0006] The technical solution adopted by this invention to solve its technical problem is: an ADB anti-interference control method based on multimodal feature fusion, comprising the following steps: S1. The camera, millimeter-wave radar and lidar are clocked through the PTP protocol, and the data collected by the above devices are unified into the vehicle coordinate system using the homogeneous transformation matrix. S2. Extract the first target feature vector V from the camera image by integrating the CBAM attention module into the improved YOLOv8 model. cam The second target feature vector V is extracted from radar point cloud data using the PointNet++ network. radar ; S3, transfer the first target feature vector V cam With the second target feature vector V radar Full-dimensional splicing is performed, and dynamic fusion weight w is generated by embedded neural network calculation. According to the formula: fusion feature = visual feature × w + radar feature × (1-w), weighted fusion is performed, and then the final fusion feature is output after L2 normalization. S4. Based on the final fusion characteristics, combined with road curvature k and relative speed v rel The optical rotation angle θ is calculated using a dynamic compensation algorithm. comp =arctan(k × target distance) and velocity compensation coefficient k v A penumbra suppression model is used to control the gradient G at the shading edge. edge Generate 1024-dimensional masking data for controlling a high-resolution LED matrix; S5. Based on the masking data, the brightness of each zone of the LED matrix is ​​controlled by the LED driver chip-level PWM dimming technology to achieve adaptive light projection.

[0007] According to one embodiment of the present invention, in step S1, clock synchronization is achieved by using the camera as the master clock, with a synchronization message period of 100ms, and a synchronization accuracy of ±1μs is achieved through the general-purpose timer module GTM of the Infineon TC397 chip; the homogeneous transformation matrix T lider_veh The installation offset t of the lidar in the vehicle coordinate system x t y t z The installation yaw angle θ around the z-axis is calibrated and determined; homogeneous transformation matrix. The calculation formula is: , Where, x lidar The x-axis represents the target's coordinate components in the lidar coordinate system, and the y-axis represents the target's coordinate components. lidar The z-axis represents the target's coordinate components along the y-axis in the lidar coordinate system. lidarThis represents the coordinate components of the target on the z-axis of the lidar coordinate system.

[0008] According to an embodiment of the present invention, in step S2, the first target feature vector V cam This includes the target bounding box coordinates, detection confidence, and target category probability distribution; the second target feature vector V. radar Including the target's lateral velocity component v x Longitudinal velocity component v y Acceleration a and radar cross section (RCS), denoted as V radar =[v x v y ,a,RCS].

[0009] According to an embodiment of the present invention, in step S3, the calculation formula for the dynamic fusion weight w is: , Where W is the weight matrix, σ is the Sigmoid function, ⊕ is the feature concatenation operation, and b represents the bias term in the cross-modal attention mechanism; the dynamic fusion weight w is generated by activating the Sigmoid function, and its value range is [0, 1].

[0010] According to an embodiment of the present invention, in step S4, the occlusion edge gradient G edge The calculation formula is: , Where α is an empirical coefficient and ε is a zero constant.

[0011] According to an embodiment of the present invention, in step S4, the dynamic compensation algorithm is further used to perform multi-target cooperative scheduling based on a multi-target priority strategy, using the Hungarian algorithm based on the Euclidean distance between the target location and the center of the occlusion area and a category conflict penalty term, so as to achieve synchronous occlusion control for no less than 5 targets. The calculation formula is as follows: Cost = Positional bias + γ × Category conflict exponential function, where γ is the penalty coefficient.

[0012] According to one embodiment of the present invention, a fault degradation step is also included. When the system diagnoses a sensor fault, it automatically switches to the fusion mode of the camera and millimeter-wave radar, reduces the LED matrix resolution from 32x32 partitions to 16x16 partitions, and disables the curve curvature compensation algorithm to ensure that the basic ADB function is maintained within an end-to-end latency of less than 120ms.

[0013] According to one embodiment of the present invention, a model training step is further included between steps S1 and S2, wherein extreme weather data containing rain, fog, and strong light interference is synthesized using a generative adversarial network CycleGAN, and a sensor noise model is superimposed to construct a training set; a gradient inversion layer GRL is introduced to perform domain adversarial training with a domain adversarial coefficient λ=0.5 to enhance the anti-interference generalization ability of the feature extraction model.

[0014] A control system based on the ADB anti-interference control method according to the above scheme is also provided, comprising: The clock synchronization module is used to perform hardware timestamp synchronization of multiple sensors via the PTP protocol and the GTM module of the TC397 chip. The data acquisition and coordinate transformation module is used to acquire data from cameras, millimeter-wave radar, and lidar, and to unify the data to the vehicle coordinate system using a homogeneous transformation matrix. The feature extraction and fusion module is used to extract visual and radar feature vectors respectively through the improved YOLOv8 model and PointNet++ network, and perform dynamic weighted fusion. The light pattern decision and control module is used to calculate the shading area and generate LED matrix control signals based on fused features, road curvature and vehicle dynamics data; The LED driver execution module is used to receive control signals and perform adaptive light pattern projection through a high-resolution LED matrix and driver chip. The fault degradation module is used to trigger a degradation process, switch the sensor fusion mode, and adjust the system configuration when a sensor fault is detected. Among them, the light pattern decision and control module also integrates a multi-target cooperative scheduling unit, which is configured to use the Hungarian algorithm to achieve priority allocation for multi-target occlusion.

[0015] According to one embodiment of the present invention, the LED driver execution module includes a 32x32 partitioned LED matrix, a TITLC6C5724 driver chip, and supports 12-bit PWM dimming and LVDS high-speed refresh interface, for realizing grayscale gradient dimming of 1024 partitions.

[0016] The beneficial effects of this invention are: By fusing heterogeneous information from cameras, millimeter-wave radar, and lidar, and leveraging their complementary characteristics, the problem of perception failure of a single vision sensor in extreme and harsh environments such as rain, fog, snow, strong light, and backlight is fundamentally overcome. Millimeter-wave radar is unaffected by weather, lidar provides accurate 3D information, and vision provides rich textures. The collaborative work of the three significantly reduces the false detection rate and false negative rate of target detection, ensuring the high availability and reliability of the ADB system in all weather and all scenarios. By replacing the slow mechanical light shield with a fully electronic LED matrix, the physical motion inertia is eliminated. Combined with a lightweight neural network model and optimized deployment on the high-performance embedded chip TC397, an end-to-end decision and control latency of less than 100ms is achieved. This enables the system to accurately predict and track high-speed moving targets in real time, solving the problem of masking failure caused by response latency and improving high-speed driving safety. With a high-resolution LED matrix of 1024 zones and 12-bit PWM grayscale control, it can generate precise masking spots of arbitrary shapes, rather than simple rectangles. Combined with a penumbra suppression model, it can control the gradient of the masking edge within 3° and strictly limit the penumbra glare intensity within the ECE R123 standard, completely avoiding glare for oncoming drivers, while providing a wider and more uniform lighting field of view. It can integrate high-precision maps and vehicle dynamics data to perform dynamic compensation for complex road conditions such as curves and slopes; it adopts a multi-objective collaborative scheduling algorithm to intelligently process the occlusion priority of multiple road participants at the same time, realizing intelligent and human-like beam management; The built-in fault diagnosis and degradation mechanism can seamlessly switch to a redundant fusion mode when any single sensor fails, ensuring a high level of driving safety. Attached Figure Description

[0017] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0018] Figure 1 This is a structural block diagram of the control system of the present invention.

[0019] Figure 2 This is a structural block diagram of the control method of the present invention.

[0020] Figure 3 This is a flowchart of feature fusion in the control method of the present invention.

[0021] Figure 4 This is a control flowchart of fault degradation in the control method of the present invention. Detailed Implementation

[0022] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0023] Example 1 like Figure 1As shown, a control system based on ADB anti-interference control method using multimodal feature fusion includes a clock synchronization module, a data acquisition and coordinate transformation module, a feature extraction and fusion module, a light pattern decision and control module, an LED drive execution module, and a fault degradation module. The clock synchronization module performs hardware timestamp synchronization of multiple sensors via the PTP protocol and the GTM module of the TC397 chip. The data acquisition and coordinate transformation module acquires data from cameras, millimeter-wave radar, and lidar, and unifies the data to the vehicle coordinate system using a homogeneous transformation matrix. The feature extraction and fusion module uses an improved YOLOv8 model and a Poin... The tNet++ network extracts visual and radar feature vectors respectively and performs dynamic weighted fusion; the light pattern decision and control module is used to calculate the occlusion area and generate LED matrix control signals based on fused features, road curvature and vehicle dynamics data; the LED drive execution module is used to receive control signals and perform adaptive light pattern projection through high-resolution LED matrix and driver chip; the fault degradation module is used to trigger the degradation process, switch sensor fusion mode and adjust system configuration when sensor fault is detected; the light pattern decision and control module also integrates a multi-target cooperative scheduling unit, configured to use the Hungarian algorithm to achieve priority allocation of multi-target occlusion.

[0024] The LED driver execution module includes a 32x32 partitioned LED matrix, a TI TLC6C5724 driver chip, supports 12-bit PWM dimming and an LVDS high-speed refresh interface, and is used to implement grayscale gradient dimming in 1024 partitions.

[0025] Example 2 like Figure 2 As shown, an ADB anti-interference control method based on multimodal feature fusion includes the following steps: Step 1: Multi-source data synchronization and acquisition. Starting with a 100ms synchronization cycle, ensure that all sensor data are consistent in time and space, providing an accurate benchmark for subsequent fusion.

[0026] Specifically, the system initiates the IEEE 1588 PTP protocol, with the camera serving as the default master clock, sending synchronization messages to the millimeter-wave radar and lidar at 100ms intervals. The GTM (General Purpose Timer Module) built into the Infineon TC397 chip adds a high-precision hardware timestamp to all incoming sensor data and compensates for clock skew.

[0027] The camera captures a frame of RGB image, the LiDAR acquires 3D point cloud data of the current environment, and the millimeter-wave radar acquires point cloud data of the target object. All sensor data are transformed through a pre-calibrated homogeneous matrix. The coordinates are uniformly converted to the vehicle coordinate system of the ISO 8855 standard.

[0028] Step 2: Anti-interference feature extraction and fusion. Robust features are extracted from visual and point cloud data respectively, and dynamically fused into a more reliable environmental perception result.

[0029] like Figure 3 As shown, the image data is input into an improved YOLOv8 model. The integrated CBAM attention module enhances the feature extraction capability under rain, fog, and strong light conditions. The output features include the target bounding box (x, y, w, h), detection confidence, and the feature vector V representing the target class probability distribution. cam Point cloud data is input into the PointNet++ network, and the output includes the target's lateral velocity v. x Longitudinal velocity v y The four-dimensional eigenvector V of acceleration a and radar cross section (RCS) radar ; The feature vector V cam and V radar A full-dimensional concatenation is performed, and the concatenated features are input into a lightweight embedded neural network. A dynamic fusion weight between 0 and 1 is generated using the sigmoid function. The formula used is: Fusion Feature = V cam ×w+V radar The final feature output is calculated using the multiplication factor (1-w). The fused features are then L2 normalized and transmitted to the next module. The entire fusion process is completed on the TC397 chip with an end-to-end latency of <100ms.

[0030] Step 3: Light pattern decision and generation. Based on the fused environmental perception features, the area and light pattern that need to be occluded are determined, and specific control signals are generated.

[0031] The system identifies multiple targets requiring anti-glare, such as oncoming vehicles and pedestrians. It employs the Hungarian algorithm to calculate the optimal shading allocation scheme based on target positional deviations and category conflicts, ensuring that the light spot can accurately cover ≥5 targets simultaneously. This is achieved by considering road curvature k and relative speed v. rel Real-time calculation of compensation, including curve compensation, and calculation of the beam rotation angle θ. comp =arctan(k×target distance) to preemptively block vehicles in the curve; speed compensation, introducing a speed compensation coefficient k v =0.003s / m to suppress vibration during high-speed driving; penumbra suppression, using parameters: α=0.35 cd·s / m² and β=0.15° to control the edge gradient of the shading area, ensuring that its sharpness meets the ECE R123 standard, that is, penumbra width < 3°; The calculated occlusion shape is mapped onto a 32x32 partitioned LED matrix with a total of 1024 pixels, generating a 1024-dimensional occlusion mask array, where each value represents the brightness of the corresponding LED partition, i.e., the PWM duty cycle.

[0032] Step 4: Real-time light pattern execution and control, converting digital mask signals into physical light spots and projecting them onto the road in real time.

[0033] The TC397 chip sends the generated mask array to LED driver chips such as the TITLC6C5724 via the LVDS high-speed interface (refresh rate ≥100Hz). The driver chip uses 12-bit PWM (4096 gray levels) to precisely control the brightness of each LED zone, achieving smooth gray-scale gradient dimming. The LED matrix physically projects an adaptive light pattern with curvature compensation and sharp edges according to the received control signal. The time from chip output signal to the completion of brightness update of the entire LED matrix is ​​<5ms.

[0034] Step 5: Continuous monitoring and fault degradation. Monitor the system status throughout the process to ensure functional safety under any fault conditions.

[0035] like Figure 4 As shown, the system continuously diagnoses the status of each sensor via the CAN bus. Once a fault is detected, such as a LiDAR signal loss DTC 0x0F01 or a data verification failure DTC 0x0F02, a degradation mechanism is immediately triggered. The system degrades from the LiDAR, camera, and millimeter-wave radar fusion mode to the camera and millimeter-wave radar fusion mode, reducing the LED matrix resolution from 32x32 to 16x16 and disabling curve curvature compensation. The entire fault response, decision-making, and mode switching process is completed within 0.3 seconds. The end-to-end latency of the degraded system is controlled to <120ms, which meets the ISO 26262 ASIL-B functional safety requirements, ensuring that the system can still provide basic anti-glare functionality.

[0036] In step one, the homogeneous transformation matrix The calculation formula is: , Where, x lidar The x-axis represents the target's coordinate components in the lidar coordinate system, and the y-axis represents the target's coordinate components. lidar The z-axis represents the target's coordinate components along the y-axis in the lidar coordinate system. lidar This represents the target's coordinate components along the z-axis in the lidar coordinate system. The homogeneous transformation matrix T lider_veh The installation offset t of the lidar in the vehicle coordinate system x t y t zThe installation yaw angle θ around the z-axis is calibrated to ensure that the lidar data can be accurately converted to a standard coordinate system with the vehicle's forward direction as the y-axis, the right side as the x-axis, and the top as the z-axis, thereby ensuring the precise alignment of multi-sensor data under a unified spatial reference.

[0037] In step two, the formula for calculating the dynamic fusion weight w is: , Where W is the weight matrix, σ is the Sigmoid function, ⊕ is the feature concatenation operation, and b represents the bias term in the cross-modal attention mechanism; the dynamic fusion weight w is generated by activating the Sigmoid function, and its value range is [0, 1].

[0038] In step three, the edge gradient G of the shading region edge The calculation formula is: , Where α is an empirical coefficient, α = 0.35 cd·s / m 2 ε is a constant that is reduced to zero.

[0039] Step two also includes a model training step, which uses the Generative Adversarial Network (CycleGAN) to convert real scene data into synthetic data containing rain, fog, and strong light interference, and superimposes a sensor noise model to build an extreme environment training set; by introducing a gradient inversion layer (GRL) in the feature extraction stage, the gradient direction is inverted at a ratio of λ=0.5, forcing the model to learn environment-invariant Lubang features, which improves the system's generalization ability in unseen scenes such as snow and backlight by more than 2 times, and finally forms a lightweight fusion model that can resist complex environmental interference.

[0040] The following examples will verify the relevant performance of Embodiment 2.

[0041] 1. Verify the robustness of the multimodal dynamic fusion mechanism in Example 2 for target detection under combined scenarios of extreme weather and sudden changes in lighting, demonstrating its significant advantages over traditional single-vision solutions.

[0042] A comprehensive environmental simulation platform was built in a closed test field. A rain system simulated heavy rainfall of 50 mm / h, creating severe water droplet interference on the camera lens and strong scattering interference on the lidar beam. High-intensity xenon lamps were installed at the tunnel entrance to simulate 120 klx of intense external lighting; the tunnel interior had an illumination of 20 klx, creating a sudden change in illumination of 100 klx, testing the camera's HDR performance and the adaptability of the fusion algorithm.

[0043] The camera uses an automotive-grade global shutter camera compliant with ISO 26262 ASIL-B, with HDR mode enabled. Its dynamic range is >140dB, supporting simultaneous capture of bright areas outside the tunnel and dark details inside the tunnel within a single frame. The millimeter-wave radar is a 77GHz forward-facing long-range radar with a maximum detection range >200m and a horizontal field of view >±60°. Its electromagnetic wave penetration characteristics in rain and fog are unaffected by weather. The lidar uses a 905nm wavelength, 128-line scanning lidar with a refresh rate of 50Hz. Its built-in anti-interference coding technology effectively filters out interference signals from other vehicles' LiDARs, providing sparse but accurate distance point clouds even in rain and fog.

[0044] Time synchronization is achieved via the IEEE 1588 PTP protocol, with the camera serving as the master clock. The GTM module of the Infineon TC397 chip ensures that the timestamp accuracy of all sensor data is within ±1μs. All sensor data undergoes a homogeneous transformation matrix T. lidar^veh Unified to the vehicle coordinate system; heavy rain caused image blurring, and strong light caused local overexposure. The improved YOLOv8 model with integrated CBAM attention module was activated. Its channel attention mechanism focuses on effective features (such as vehicle outlines) and suppresses noise features (such as rain streaks and water splashes), outputting the target feature vector V. cam The point clouds from millimeter-wave radar and lidar are processed by the PointNet++ network to extract the target's velocity, acceleration, and RCS information, generating a feature vector V. radar The fusion module calculates a dynamic weight w based on the current environmental signal-to-noise ratio. In this rainstorm and strong light scenario, the algorithm automatically determines that visual reliability is reduced and calculates a lower fusion weight w=0.3, significantly increasing the decision-making weight of radar features by 1-w=0.7. Finally, the fusion feature is calculated according to the formula: Fusion Feature = V cam ×0.3+V radar ×0.7 Outputs highly robust perception results.

[0045] Test results on the KITTI-C dataset's heavy rain and partial occlusion subsets show that the false detection rate is reduced to 5.2% and the false negative rate to 3.8%. The multimodal dynamic fusion mechanism in Example 2 reduces the perception error rate in extreme environments, demonstrating its superior anti-interference ability and environmental adaptability.

[0046] 2. Verify the ability of the dynamic compensation algorithm and high-resolution light pattern control technology in Example 2 to accurately compensate for the occluded area and suppress penumbra glare in a curved scene.

[0047] Given a road curvature k = 0.08 m⁻¹, a vehicle speed of 60 km / h, and an oncoming vehicle distance of 100 m, the system receives the road curvature signal and target distance from the CAN bus and calculates the beam rotation angle: θ. comp=arctan(k × target distance) ≈ 38.7°, this angle value is used to rotate and transform the basic shading light pattern, allowing the light spot to predict and cover the accurate position of vehicles approaching from behind in the curve. If there are multiple targets in the curve, the Hungarian algorithm will calculate the optimal shading allocation based on their position and category (pedestrian-triggered category conflict penalty term γ), ensuring that the light spot accurately covers vehicles while avoiding glare for pedestrians. The penumbra suppression model is activated, according to the formula... (where α = 0.35 cd·s / m², and ε is a zero constant) The gray-scale gradient of the occlusion edge is dynamically calculated and controlled, and a sharp-edged light spot is generated through 4096 levels of gray-scale adjustment of the high-resolution LED matrix.

[0048] Actual measurements showed that the offset error of the center point of the shaded area relative to the position of oncoming vehicles was reduced from 1.8° in the traditional solution to 0.4°, far exceeding the ≤1.0° requirement specified in the ECE R123 standard. Using a luminance meter to measure the penumbra at the edge of the shaded area, its width was controlled within 2.8°, and the maximum glare intensity was 210 cd / m², fully meeting the regulatory limit of ≤250 cd / m², effectively preventing glare. The end-to-end latency from perception to light pattern update was <95ms, ensuring real-time performance and effectiveness of control in dynamic cornering scenarios.

[0049] 3. Verify whether a smooth and seamless functional switch can be achieved based on the fault degradation mechanism when a critical sensor fails, while maintaining a performance level that meets functional safety requirements.

[0050] A short-circuit fault signal from the LiDAR power module was injected into the system bus using testing tools, causing an interruption in the LiDAR point cloud data output. The fault diagnosis module monitors the sensor status in real time, immediately detects the loss of LiDAR signal, and generates a diagnostic fault code (DTC) 0x0F01 (representing LiDAR communication loss) conforming to the UDS standard via the CAN bus. After the fault code is triggered, the degradation management unit within the TC397 chip completes fault confirmation and decision-making within <10ms, automatically downgrading from the LiDAR, camera, and millimeter-wave radar fusion mode to the camera and millimeter-wave radar fusion mode. The millimeter-wave radar provides the target's precise distance and velocity, while the camera provides target classification and lateral position information. Simultaneously, the curve curvature compensation, which highly depends on the accurate 3D point cloud of LiDAR, is disabled, and the control resolution of the LED matrix is ​​reduced from 32x32 (1024 partitions) to 16x16 (256 partitions) to reduce the computational complexity of the control algorithm. System resources are reallocated to ensure that the new fusion and control pipeline runs with the highest priority.

[0051] In actual testing under degraded mode, the end-to-end latency from camera and radar data acquisition to the final light pattern update was (108±5) ms, meeting the design requirement of less than 120ms and ensuring real-time performance after degrade. Even after degrade, the system can still effectively shield the main vehicles (≤3) ahead, maintaining the basic ADB anti-glare function. The entire fault response and mode switching process was smooth and rapid, without any functional interruptions or malfunctions, meeting the functional safety objectives of ISO 26262 ASIL-B level, demonstrating the system's high reliability.

[0052] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. An ADB anti-interference control method based on multimodal feature fusion, characterized in that, Includes the following steps: S1. The camera, millimeter-wave radar and lidar are clocked through the PTP protocol, and the data collected by the above devices are unified into the vehicle coordinate system using the homogeneous transformation matrix. S2. Extract the first target feature vector V from the camera image by integrating the CBAM attention module into the improved YOLOv8 model. cam The second target feature vector V is extracted from radar point cloud data using the PointNet++ network. radar ; S3, transfer the first target feature vector V cam With the second target feature vector V radar Full-dimensional splicing is performed, and dynamic fusion weight w is generated by embedded neural network calculation. According to the formula: fusion feature = visual feature × w + radar feature × (1-w), weighted fusion is performed, and then the final fusion feature is output after L2 normalization. S4. Based on the final fusion characteristics, combined with road curvature k and relative speed v rel The optical rotation angle θ is calculated using a dynamic compensation algorithm. comp =arctan(k × target distance) and velocity compensation coefficient k v A penumbra suppression model is used to control the gradient G at the shading edge. edge Generate 1024-dimensional masking data for controlling a high-resolution LED matrix; S5. Based on the masking data, the brightness of each zone of the LED matrix is ​​controlled by the LED driver chip-level PWM dimming technology to achieve adaptive light projection.

2. The ADB anti-interference control method based on multimodal feature fusion according to claim 1, characterized in that: In step S1, clock synchronization is achieved using the camera as the master clock, with a synchronization message period of 100ms, and a synchronization accuracy of ±1μs is achieved through the general-purpose timer module GTM of the Infineon TC397 chip; the homogeneous transformation matrix T lider_veh The installation offset t of the lidar in the vehicle coordinate system x t y t z The installation yaw angle θ around the z-axis is calibrated and determined; homogeneous transformation matrix. The calculation formula is: ， Where, x lidar The x-axis represents the target's coordinate components in the lidar coordinate system, and the y-axis represents the target's coordinate components. lidar The z-axis represents the target's coordinate components along the y-axis in the lidar coordinate system. lidar This represents the coordinate components of the target on the z-axis of the lidar coordinate system.

3. The ADB anti-interference control method based on multimodal feature fusion according to claim 1, characterized in that: In step S2, the first target feature vector V cam This includes the target bounding box coordinates, detection confidence, and target category probability distribution; the second target feature vector V. radar Including the target's lateral velocity component v x Longitudinal velocity component v y Acceleration a and radar cross section (RCS), denoted as V radar =[v x v y ,a,RCS].

4. The ADB anti-interference control method based on multimodal feature fusion according to claim 1, characterized in that: In step S3, the formula for calculating the dynamic fusion weight w is: ， Where W is the weight matrix, σ is the Sigmoid function, ⊕ is the feature concatenation operation, and b represents the bias term in the cross-modal attention mechanism; the dynamic fusion weight w is generated by activating the Sigmoid function, and its value range is [0, 1].

5. The ADB anti-interference control method based on multimodal feature fusion according to claim 1, characterized in that: In step S4, the occlusion edge gradient G edge The calculation formula is: ， Where α is an empirical coefficient and ε is a zero constant.

6. The ADB anti-interference control method based on multimodal feature fusion according to claim 1, characterized in that: In step S4, the dynamic compensation algorithm is also used to perform multi-target cooperative scheduling based on the multi-target priority strategy and the Hungarian algorithm, using the Euclidean distance between the target location and the center of the occlusion area and the category conflict penalty term, to achieve synchronous occlusion control for no less than 5 targets. The calculation formula is as follows: Cost = Positional bias + γ × Category conflict exponential function, where γ is the penalty coefficient.

7. The ADB anti-interference control method based on multimodal feature fusion according to claim 1, characterized in that: It also includes a fault degradation step. When the system diagnoses a sensor fault, it automatically switches to the fusion mode of the camera and millimeter-wave radar, reduces the LED matrix resolution from 32x32 partitions to 16x16 partitions, and disables the curve curvature compensation algorithm to ensure that the basic ADB function is maintained within an end-to-end latency of less than 120ms.

8. The ADB anti-interference control method based on multimodal feature fusion according to claim 1, characterized in that: Between steps S1 and S2, there is also a model training step, in which extreme weather data containing rain, fog and strong light interference are synthesized using the adversarial generative network CycleGAN, and a sensor noise model is superimposed to build a training set; a gradient inversion layer GRL is introduced to perform domain adversarial training with a domain adversarial coefficient λ=0.5 to enhance the anti-interference generalization ability of the feature extraction model.

9. A control system based on the ADB anti-interference control method according to any one of claims 1-8, characterized in that, include: The clock synchronization module is used to perform hardware timestamp synchronization of multiple sensors via the PTP protocol and the GTM module of the TC397 chip. The data acquisition and coordinate transformation module is used to acquire data from cameras, millimeter-wave radar, and lidar, and to unify the data to the vehicle coordinate system using a homogeneous transformation matrix. The feature extraction and fusion module is used to extract visual and radar feature vectors respectively through the improved YOLOv8 model and PointNet++ network, and perform dynamic weighted fusion. The light pattern decision and control module is used to calculate the shading area and generate LED matrix control signals based on fused features, road curvature and vehicle dynamics data; The LED driver execution module is used to receive control signals and perform adaptive light pattern projection through a high-resolution LED matrix and driver chip. The fault degradation module is used to trigger a degradation process, switch the sensor fusion mode, and adjust the system configuration when a sensor fault is detected. Among them, the light pattern decision and control module also integrates a multi-target cooperative scheduling unit, which is configured to use the Hungarian algorithm to achieve priority allocation for multi-target occlusion.

10. The control system of the ADB anti-interference control method based on multimodal feature fusion according to claim 9, characterized in that: The LED driver execution module includes a 32x32 partitioned LED matrix, a TI TLC6C5724 driver chip, supports 12-bit PWM dimming and LVDS high-speed refresh interface, and is used to realize grayscale gradient dimming of 1024 partitions.

Images