Fusion perception-based vehicle light control method, device and system

Abstract

This invention relates to the field of intelligent driving technology and discloses a vehicle headlight control method, device, and system based on fusion perception. The method includes: acquiring multi-sensor observation time-series data; the multi-sensor observation time-series data includes at least image data, point cloud data collected by millimeter-wave radar, and rainfall data; wherein, the target detection threshold of the millimeter-wave radar is adjusted according to the rainfall data at a preset frequency; spatiotemporal feature extraction is performed on the multi-sensor observation time-series data to generate fused features; the fused features include environmental context information and the motion state of each traffic target; the fused features are input into a scene classification model to identify the current driving scene and output the corresponding scene label; the corresponding vehicle headlight control strategy is matched according to the scene label to generate vehicle headlight control commands, which are then executed by the vehicle headlight control unit. This solution improves the environmental adaptability of vehicle headlight control.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, specifically to a vehicle lighting control method, device, and system based on fusion perception. Background Technology

[0002] In automotive active safety and intelligent driving assistance scenarios, achieving automated headlight control is crucial. Intelligent High-Beam Control (IHBC) can automatically switch between high and low beams under different traffic and environmental conditions, ensuring the driver's visibility while avoiding glare for other road users.

[0003] Intelligent high-beam control systems in related technologies typically use a single forward-facing camera as the sensing source, or combine simple parameters such as vehicle speed to set fixed rules for light control. However, these solutions have poor environmental adaptability and are prone to misjudgment in strong light interference, rainy or foggy weather, or non-standard light sources (such as low streetlights). Furthermore, they lack scene coverage and cannot dynamically identify complex traffic scenarios (such as meeting oncoming traffic on narrow roads, driving in tunnels, and sharp bends). Summary of the Invention

[0004] This invention provides a vehicle lighting control method, device, and system based on fusion perception to solve the problems of poor environmental adaptability and insufficient scene coverage in related vehicle lighting control schemes.

[0005] In a first aspect, the present invention provides a vehicle lighting control method based on fused perception, the method comprising:

[0006] Acquire multi-sensor observation time-series data; the multi-sensor observation time-series data includes at least image data, point cloud data collected by millimeter-wave radar, and rainfall data; wherein, the target detection threshold of the millimeter-wave radar is adjusted according to the rainfall data at a preset frequency;

[0007] Spatiotemporal features are extracted from the multi-sensor observation time-series data to generate fused features; the fused features include environmental context information and the motion state of each traffic target.

[0008] The fused features are input into a scene classification model to identify the current driving scene and output the corresponding scene label.

[0009] Based on the scene label, the corresponding headlight control strategy is matched to generate headlight control instructions, which are then used by the headlight control unit to perform headlight control operations.

[0010] In one optional implementation, adjusting the target detection threshold of the millimeter-wave radar based on rainfall data includes:

[0011] Acquire rainfall data; the rainfall data is collected in real time by a rainfall sensor;

[0012] The rainfall data is statistically analyzed using a sliding window to generate an environmental rainfall intensity index.

[0013] Based on the environmental rainfall intensity index, a threshold adjustment coefficient is determined according to a preset mapping relationship;

[0014] The target detection threshold of the millimeter-wave radar is adjusted according to the threshold adjustment coefficient.

[0015] In one optional implementation, the step of extracting spatiotemporal features from the multi-sensor observation time-series data to generate fused features includes:

[0016] The time-series data of the multi-sensor observations are synchronized in time and calibrated in space to obtain multi-sensor aligned data.

[0017] The multi-sensor aligned data is subjected to target association and weighted fusion processing to generate motion state sequences corresponding to each traffic target;

[0018] The motion state sequence corresponding to each traffic target is estimated by a state estimation algorithm to generate the continuous motion trajectory of each traffic target.

[0019] Environmental features are extracted from the multi-sensor observation time-series data to obtain environmental context features;

[0020] The continuous motion trajectory of each traffic target is combined with the environmental context features to form the fused feature.

[0021] In one optional implementation, the step of inputting the fused features into a scene classification model to identify the current driving scene and outputting the corresponding scene label includes:

[0022] Obtain fused features from multiple consecutive frames, including the current time.

[0023] The fused features of the consecutive multi-frames are input into the first recurrent neural network unit in chronological order, and the hidden state is calculated frame by frame to obtain the past context information.

[0024] The fused features of the consecutive multi-frames are input into the second recurrent neural network unit in reverse chronological order, and propagated back frame by frame to obtain future context information.

[0025] The past context information and future context information are fused to generate a bidirectional context joint feature;

[0026] The bidirectional contextual joint features are input into the classification layer, and the scene label at the current moment is output; the scene label includes at least meeting oncoming traffic, following oncoming traffic, sharp bends, and tunnel driving.

[0027] In an optional implementation, the method further includes:

[0028] Real-time monitoring of the operating status of multiple sensors; the multiple sensors include at least a forward-looking camera, a millimeter-wave radar, and a rain sensor;

[0029] When a fault is detected in any of the multiple sensors, the type of the faulty sensor is obtained;

[0030] If the faulty sensor is a preset sensor type, the headlight control strategy is switched to a conservative operation mode, and the target detection threshold of the forward-facing camera / millimeter-wave radar is increased; wherein, in the conservative operation mode, the critical distance for switching from high beam to low beam is shortened.

[0031] Secondly, the present invention provides a vehicle lighting control device based on fusion perception, the device comprising:

[0032] The acquisition module is used to acquire multi-sensor observation time-series data; the multi-sensor observation time-series data includes at least image data, point cloud data collected by millimeter-wave radar, and rainfall data; wherein, the target detection threshold of the millimeter-wave radar is adjusted according to the rainfall data at a preset frequency;

[0033] The feature extraction module is used to extract spatiotemporal features from the multi-sensor observation time-series data and generate fused features; the fused features include environmental context information and the motion state of each traffic target;

[0034] The scene classification module is used to input the fused features into the scene classification model, identify the current driving scene, and output the corresponding scene label.

[0035] The control command module is used to match the corresponding vehicle light control strategy according to the scene label and generate vehicle light control commands, so that the vehicle light control unit can execute vehicle light control operations according to the vehicle light control commands.

[0036] Thirdly, the present invention provides a vehicle lighting control system based on fusion perception, the system comprising:

[0037] A perception module is used to acquire time-series observation data from multiple sensors; the perception module includes at least a forward-looking camera, a millimeter-wave radar, and a rain sensor; the forward-looking camera is used to acquire image data; the millimeter-wave radar is used to acquire point cloud data; the rain sensor is used to acquire rainfall data; wherein, the target detection threshold of the millimeter-wave radar is adjusted according to the rainfall data at a preset frequency.

[0038] The control module is used to extract spatiotemporal features from the multi-sensor observation time-series data and generate fused features; the fused features include environmental context information and the motion state of each traffic target; the fused features are input into a scene classification model to identify the current driving scene and output the corresponding scene label; the corresponding headlight control strategy is matched according to the scene label to generate headlight control commands;

[0039] The execution module includes a headlight control unit; the headlight control unit is used to perform headlight control operations according to the headlight control command.

[0040] Fourthly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the vehicle lighting control method based on fusion perception described in the first aspect or any corresponding embodiment thereof.

[0041] Fifthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the vehicle lighting control method based on fusion perception described in the first aspect or any corresponding embodiment thereof.

[0042] In a sixth aspect, the present invention provides a computer program product, including computer instructions for causing a computer to execute the vehicle lighting control method based on fusion perception as described in the first aspect or any corresponding embodiment thereof.

[0043] The technical solution provided by this invention may include the following beneficial effects:

[0044] The vehicle headlight control method provided by this invention overcomes the problems of single sensors being susceptible to interference and having incomplete information in complex environments by acquiring multi-sensor observation time-series data, significantly improving the integrity and reliability of system input data. Adjusting the target detection threshold of millimeter-wave radar using rainfall data enhances the system's robustness under complex weather conditions, strengthening its anti-interference capability and operational stability. By extracting spatiotemporal features from multi-sensor observation time-series data, joint spatiotemporal modeling of multi-source time-series data is achieved, realizing the effective transformation from raw perception data to high-level semantic information, providing high-quality input for subsequent accurate scene recognition, thereby improving the accuracy of vehicle headlight control. Scene classification based on fused features accurately distinguishes different driving conditions, giving the system a stronger environmental understanding capability, ensuring that the selection of headlight control strategies is more aligned with actual driving needs, with good environmental adaptability and sufficient scene coverage. Matching corresponding headlight control strategies to scene labels enables the invocation of differentiated control strategies in different scenarios, further improving headlight control accuracy and enhancing user experience. Attached Figure Description

[0045] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0046] Figure 1 This is a schematic flowchart of a vehicle lighting control method based on fusion perception according to an embodiment of the present invention;

[0047] Figure 2 This is a second flowchart illustrating the vehicle lighting control method based on fusion perception according to an embodiment of the present invention;

[0048] Figure 3 This is a flowchart illustrating the three-level decision-making process according to an embodiment of the present invention;

[0049] Figure 4 This is a schematic diagram illustrating data acquisition by the sensing module and data-level fusion by the control module according to an embodiment of the present invention;

[0050] Figure 5 This is a schematic diagram of scene classification performed by the control module according to an embodiment of the present invention;

[0051] Figure 6 This is a schematic diagram of the system activation process according to an embodiment of the present invention;

[0052] Figure 7 This is a structural block diagram of a vehicle lighting control device based on fusion perception according to an embodiment of the present invention;

[0053] Figure 8 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0056] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0057] In automotive active safety and intelligent driving assistance scenarios, achieving automated headlight control is crucial. Intelligent High-Beam Control (IHBC) can automatically switch between high and low beams under different traffic and environmental conditions, ensuring the driver's visibility while avoiding glare for other road users.

[0058] Intelligent high-beam control systems in related technologies typically use a single forward-facing camera as the sensing source, or combine simple parameters such as vehicle speed to set fixed rules for light control. However, these solutions have the following drawbacks:

[0059] 1. Poor environmental adaptability. Relying solely on visual information for light source identification, it is susceptible to strong light reflection, backlight interference, water vapor scattering in rainy or foggy weather, and the influence of non-standard lighting facilities (such as low streetlights, billboards, and tunnel wall reflections). This can lead to frequent misjudgments of oncoming vehicles and frequent accidental switching off of high beams, or failure to promptly identify actual oncoming vehicles and continued use of high beams, posing safety hazards.

[0060] 2. Insufficient scene coverage. It lacks the ability to model and identify special driving conditions, making it unable to effectively handle complex scenarios such as meeting oncoming traffic on narrow roads, continuous sharp curves, and entering and exiting tunnels. For example, it fails to turn off high beams in time when entering a tunnel, or mistakenly turns them back on on sharp curves due to a temporary loss of sight of the target, affecting driving safety.

[0061] According to an embodiment of the present invention, a vehicle lighting control method based on fusion perception is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0062] This embodiment provides a vehicle lighting control method based on fusion perception, which is executed by the control module in the vehicle lighting control system based on fusion perception. Figure 1 This is a flowchart of a vehicle lighting control method based on fusion perception according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0063] Step S101: Obtain multi-sensor observation time series data.

[0064] The fusion-based vehicle lighting control system is deployed in the target vehicle, which includes an onboard computer, and the control module can be deployed within the onboard computer. The target vehicle also includes a vehicle lighting control unit, and the onboard computer is connected to the vehicle lighting control unit.

[0065] The multi-sensor observation time-series data is collected in real time by multiple sensors deployed on the target vehicle. Optionally, the multi-sensor observation time-series data includes at least image data, point cloud data collected by millimeter-wave radar, and rainfall data. The image data, point cloud data, and rainfall data are all time-series data. For example, the image data can be a continuous sequence of image frames, the point cloud data can be a continuous point cloud or a list of targets, and the rainfall data can be a time series of raindrop density / frequency.

[0066] The image data can be acquired by cameras deployed on the target vehicle. Different cameras can be deployed as needed, such as front-view cameras, side-view cameras, and fisheye cameras. When the controlled object is high / low beam headlights, the front-view camera is mainly used to acquire image data. The rainfall data can be acquired by a rain sensor deployed on the target vehicle. This rain sensor can be an optical rain sensor, a capacitive rain sensor, etc. The point cloud data is acquired by a millimeter-wave radar deployed on the target vehicle. The target detection threshold of the millimeter-wave radar is dynamically adjusted based on the rainfall data. Specifically, the target detection threshold of the millimeter-wave radar can be adjusted according to the rainfall data at a preset frequency. For example, the target detection threshold of the millimeter-wave radar can be adjusted every 5 minutes based on the rainfall data acquired at the current moment, so that the millimeter-wave radar performs subsequent acquisitions according to the adjusted target detection threshold. The target detection threshold of the millimeter-wave radar can be a received signal strength threshold (RSSIThreshold), a signal-to-noise ratio threshold (SNR Threshold), a point cloud density threshold, etc.

[0067] Step S102: Extract spatiotemporal features from the multi-sensor observation time series data to generate fused features.

[0068] Specifically, an algorithm model with spatiotemporal feature extraction capabilities can be used to extract spatiotemporal features from the observation time series data corresponding to each sensor in the multi-sensor observation time series data. This extracts spatial structure information (e.g., spatial clustering distribution of multiple targets in the radar point cloud, light and dark boundaries formed by the tunnel walls, the position, brightness, and shape of vehicle lights in the image, etc.) and temporal evolution patterns (e.g., the distance between oncoming vehicles and target vehicles is getting closer, the taillights ahead remain lit for more than 5 seconds, the ambient illumination is rapidly decreasing, etc.). These are then integrated into a fusion feature for subsequent scene recognition and decision-making. This fusion feature includes environmental context information and the motion state of each traffic target. The environmental context information indicates current weather conditions (e.g., sunny, rainy, foggy), lighting conditions (e.g., daytime, dusk, nighttime), whether a tunnel has been entered, and the current road type (e.g., urban, highway, mountain road). The traffic target information indicates objects in the observation time series data, such as oncoming vehicles, vehicles traveling in the same direction, stationary obstacles (e.g., streetlights), non-motorized vehicles, pedestrians, etc.

[0069] Step S103: Input the fused features into the scene classification model to identify the current driving scene and output the corresponding scene label.

[0070] This scene classification model identifies the current driving scenario of a target vehicle, such as meeting oncoming traffic, following oncoming traffic, or entering a tunnel, based on the environmental context information and the motion state of each traffic object contained in the fused features, and outputs the corresponding scene label as the scene classification result. This scene classification model can employ a neural network model with scene classification capabilities from related technologies.

[0071] Step S104: Match the corresponding headlight control strategy according to the scene label, generate headlight control instructions, and enable the headlight control unit to perform headlight control operations according to the headlight control instructions.

[0072] The control module pre-configures headlight control strategies. Different scene tags are pre-configured with different headlight control strategies, and the control parameters differ between these strategies, such as the required distance from oncoming vehicles for switching between high and low beams, and the desired headlight intensity. After acquiring a scene tag, the module matches it with a corresponding headlight control strategy and generates headlight control commands according to the matched strategy. These commands are then sent to the headlight control unit. The headlight control unit executes headlight control operations based on these commands, such as switching from high beams to low beams.

[0073] The vehicle headlight control method based on fusion perception provided in this embodiment overcomes the problems of single sensors being susceptible to interference and having incomplete information in complex environments by acquiring multi-sensor observation time-series data, significantly improving the integrity and reliability of system input data. Adjusting the target detection threshold of millimeter-wave radar using rainfall data enhances the system's robustness under complex weather conditions, strengthening its anti-interference capability and operational stability. Spatiotemporal feature extraction from multi-sensor observation time-series data enables joint spatiotemporal modeling of multi-source time-series data, achieving effective transformation from raw perception data to high-level semantic information, providing high-quality input for subsequent accurate scene recognition, and thus improving the accuracy of vehicle headlight control. Scene classification based on fusion features accurately distinguishes different driving conditions, giving the system a stronger environmental understanding capability, ensuring that the selection of lighting control strategies is more aligned with actual driving needs, with good environmental adaptability and sufficient scene coverage. Matching corresponding vehicle headlight control strategies to scene labels enables the invocation of differentiated control strategies in different scenarios, further improving the accuracy of vehicle headlight control and enhancing the user experience.

[0074] This embodiment provides a vehicle lighting control method based on fusion perception, which can be used in vehicle-mounted computers. Figure 2 This is a flowchart of a vehicle lighting control method based on fusion perception according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0075] Step S201: Obtain multi-sensor observation time series data.

[0076] The multi-sensor observation time-series data includes at least image data, point cloud data acquired by millimeter-wave radar, and rainfall data. Specifically, the target detection threshold of the millimeter-wave radar is adjusted based on the rainfall data at a preset frequency.

[0077] Optionally, when adjusting the target detection threshold of the millimeter-wave radar based on rainfall data, rainfall data is first acquired, which is collected in real time by a rain sensor. Next, the rainfall data is statistically analyzed using a sliding window to generate an environmental rainfall intensity index, which indicates the rainfall intensity level of the current vehicle driving environment. Then, based on this environmental rainfall intensity index, a threshold adjustment coefficient is determined according to a preset mapping relationship. For example, under a first environmental rainfall intensity index, the threshold adjustment coefficient is 1, and under a second environmental rainfall intensity index, the threshold adjustment coefficient is 1.2, to improve the system robustness against interference from severe weather conditions. Finally, the target detection threshold of the millimeter-wave radar is adjusted according to this threshold adjustment coefficient. For example, when the threshold adjustment coefficient is 1.2, the target detection threshold of the millimeter-wave radar is increased by 20% to further improve the system robustness against interference from severe weather conditions.

[0078] For example, a capacitive or optical rain sensor is used to monitor the density and velocity of raindrops on the windshield in real time, obtaining continuous signals (such as voltage values) or discrete levels (such as light rain / moderate rain / heavy rain) as rainfall data. The rainfall data is then preprocessed, for example, by filtering (such as Kalman filtering) to remove noise, and by using a sliding window to statistically average the data, generating a stable environmental rainfall intensity index, such as the Rainfall Scale Index (RSI), ranging from 0 to 100.

[0079] Next, the target detection threshold of the millimeter-wave radar is adjusted using a dynamic adjustment model for radar detection thresholds. This model pre-sets a basic threshold based on radar hardware parameters (such as transmit power and wavelength) and IHBC scenario requirements (such as oncoming vehicle detection distance ≥150m), setting an initial target detection threshold (e.g., -60dBm). The model also pre-sets a rainfall-threshold mapping function, establishing a nonlinear relationship model that maps the rainfall index (RSI) to a threshold adjustment coefficient (α). Specifically, if RSI < 30, the rainfall intensity level is light rain, the threshold adjustment coefficient (α) = 1.0, and the target detection threshold remains unchanged at -60dBm; if 30 ≤ RSI < 70, the rainfall intensity level is moderate rain, the threshold adjustment coefficient (α) = 1.1, and the target detection threshold increases by 10%, adjusting to -54dBm; if RSI ≥ 70, the rainfall intensity level is heavy rain, the threshold adjustment coefficient (α) = 1.2, and the target detection threshold increases by 20%, adjusting to -48dBm. The preset frequency is set to 100ms, meaning that the threshold adjustment coefficient α is updated every 100ms based on the latest rainfall index RSI value and applied to the millimeter-wave radar in real time.

[0080] During heavy rain, numerous water droplets in the air scatter radar waves, generating strong echo signals that can cause the system to mistakenly identify oncoming vehicles. Therefore, by combining the environmental perception of a rain sensor with target detection by a millimeter-wave radar, and adjusting the target detection threshold of the millimeter-wave radar based on rainfall data, deep coupling of rainfall data and point cloud data can be achieved, improving the stability and accuracy of the radar in adverse weather conditions. Furthermore, Kalman filtering can be used to process consecutive frames of point cloud data containing targets, performing cross-frame correlation to track target trajectories and avoid false targets caused by interference from raindrops, fog, etc., in single radar echoes, further filtering out momentary false detections caused by fluctuations in rainfall.

[0081] Optionally, the multi-sensor observation time-series data also includes illumination intensity data, which can be collected in real time by an ambient light sensor deployed on the target vehicle. The illumination intensity data, along with brightness statistics from the image data, can be used to determine the current ambient lighting conditions. The multi-sensor observation time-series data also includes GPS information, map information, the target vehicle's steering wheel angle, vehicle speed, etc.

[0082] Step S202: Extract spatiotemporal features from the multi-sensor observation time series data to generate fused features.

[0083] This fusion feature includes environmental context information and the motion state of each traffic target.

[0084] Specifically, the multi-sensor observation time-series data is synchronized in time and calibrated in space. The time synchronization operation uses a unified clock source (such as CAN timestamp or PTP protocol) to interpolate the data from each sensor to the same time point. The spatial calibration operation uses an extrinsic calibration matrix to transform all sensor data to the same reference coordinate system (usually the vehicle center coordinate system), ultimately obtaining multi-sensor aligned data. Data association is then performed on this multi-sensor aligned data to determine which data belong to the same traffic target (e.g., light source, vehicle, pedestrian, streetlight), and different weights are assigned to different sensors (e.g., millimeter-wave radar has a higher weight when measuring distance, camera has a higher weight when identifying targets, and the weight of millimeter-wave radar is reduced and the weight of camera is increased in rainy weather). Weighted fusion is then performed on the associated multi-sensor aligned data to generate motion state sequences corresponding to each traffic target. Finally, a state estimation algorithm is used to estimate the state of the motion state sequences corresponding to each traffic target, generating continuous motion trajectories for each traffic target. Simultaneously, environmental features are extracted from the multi-sensor observation time-series data. For example, the system determines whether it is day or night based on light intensity data and image data, assesses weather conditions based on rainfall data, analyzes the transition between light and dark at tunnel entrances, streetlight density, and visibility decline trends based on image data, and extracts curvature changes based on steering wheel angle and vehicle speed to determine if the vehicle is on a mountain curve, etc., thus obtaining environmental context features. Finally, the continuous motion trajectories of each traffic target are combined with these environmental context features to form the fused feature.

[0085] Optionally, the state estimation algorithm can employ the Kalman filter algorithm.

[0086] Step S203: Input the fused features into the scene classification model to identify the current driving scene and output the corresponding scene label.

[0087] Specifically, this scene classification model is based on bidirectional spatiotemporal modeling. First, it acquires fused features from multiple consecutive frames, including the current moment. Then, it inputs these fused features in ascending chronological order into a first recurrent neural network unit, calculating the hidden state frame by frame to obtain past context information. Simultaneously, it inputs these fused features in reverse chronological order into a second recurrent neural network unit, propagating backward frame by frame to obtain future context information. Next, it fuses the past and future context information to generate a bidirectional contextual joint feature. Finally, it inputs this bidirectional contextual joint feature into a classification layer, outputting the scene label for the current moment.

[0088] The scene tags should at least include passing oncoming traffic, following oncoming traffic, sharp bends, and tunnel driving. Other scene tags can also be set according to actual needs, such as additional tags for urban roads, mountain roads, etc., such as urban passing oncoming traffic, urban following oncoming traffic, and sharp bends in mountainous areas.

[0089] Optionally, the first recurrent neural network unit can be a BiLSTM, and the second recurrent neural network unit can be a ConvLSTM, forming a BiLSTM+ConvLSTM collaborative architecture. By combining the temporal context capture capability of BiLSTM with the spatial structure modeling advantage of ConvLSTM, the ability to extract spatiotemporal features from IHBC scenarios is significantly improved. Traditional unidirectional LSTM can only process time series data from front to back and cannot capture the impact of future time steps on current decisions (e.g., in sharp curve scenarios, the attitude adjustment of the vehicle after turning may affect the prediction of the current turning angle). BiLSTM, on the other hand, uses two parallel inverse LSTM layers, with the forward layer from the past to the future and the backward layer from the future to the past, concatenating the hidden states of the two directions as the output, thus utilizing both historical and future information simultaneously. For example, in a meeting scenario, the forward layer can capture the relative speed change when vehicles approach, and the backward layer can analyze the distance recovery trend after meeting, combining the two to improve classification confidence. Traditional standard LSTM processes input through fully connected layers, ignoring spatial local correlations in the data (such as the continuity of lane lines in camera images or the distribution of obstacles in radar point clouds). The core of ConvLSTM is to replace the fully connected operations in the input, forget, and output gates of LSTM with convolutional operations. This allows the model to extract local spatial features through convolutional kernels while processing temporal data. For example, in a tunnel scene, ConvLSTM can capture edge features of brightness changes at the entrance (detecting brightness gradients through convolutional kernels) and combine this with temporal information to determine whether to enter the tunnel.

[0090] Step S204: Match the corresponding vehicle lighting control strategy according to the scene label, generate vehicle lighting control instructions, and enable the vehicle lighting control unit to perform vehicle lighting control operations according to the vehicle lighting control instructions.

[0091] Optionally, a parameter learner can be set up, using the Q-Learning algorithm, with driver operation feedback as the reward signal, to optimize relevant parameters in the headlight control strategy, such as the high / low beam switching distance threshold.

[0092] Optionally, a multi-level headlight control strategy can be set, selecting different levels of headlight control strategies based on multi-sensor observation time-series data, scene labels, and driver operation feedback.

[0093] For example, a three-tier decision-making architecture is adopted to ensure system security and user-friendliness. Figure 3 This is a flowchart illustrating the three-level decision-making process according to an embodiment of the present invention. The first-level decision triggers a basic switch based on real-time perception data (such as oncoming distance DL1 and same-direction distance DL2); the second-level decision adjusts the switching threshold based on scene labels, for example, the switching threshold RL1 ranges from 450m to 600m when meeting oncoming traffic, and the switching threshold RL2 ranges from 150m to 300m when following another vehicle; in the third-level decision, manual operation by the driver takes priority, and the system records the operation and updates personalized parameters.

[0094] Optionally, headlight control commands can be generated via a decision actuator, compatible with CAN bus and LIN bus protocols.

[0095] Step S205: Monitor the working status of multiple sensors in real time. When a fault is detected in any of the multiple sensors, execute a fault alarm operation.

[0096] Specifically, the system monitors the operational status of multiple sensors in real time. These sensors include at least a forward-facing camera, millimeter-wave radar, and a rain sensor. When any of these sensors malfunctions, such as communication interruption (e.g., loss of CAN bus signal), data anomalies (e.g., blurred camera image, sudden change in radar target trajectory), or self-test failure (e.g., abnormal sensor power supply), the sensor is identified as faulty, and its type is determined. If the faulty sensor type is a preset sensor type, the headlight control strategy is switched to a conservative operating mode, and the target detection threshold of the forward-facing camera / millimeter-wave radar is increased. This reduces the detection sensitivity of the forward-facing camera / millimeter-wave radar, improving anti-interference capabilities and reducing false alarms. The system can also correlate the target detection threshold of the forward-facing camera / millimeter-wave radar, increasing the detection sensitivity of the millimeter-wave radar when the forward-facing camera malfunctions, and vice versa, allowing the forward-facing camera and millimeter-wave radar to complement each other and improve system robustness. In this conservative operating mode, the critical decision distance for switching from high beam to low beam is shortened to increase high beam utilization, prioritizing driving safety even when system reliability decreases. This preset sensor type indicates the core sensor type for headlight control, such as the forward-facing camera, millimeter-wave radar, and rain sensor. In conservative mode, the headlight control system is also downgraded, with the faulty sensor type displayed on the central control screen and a red indicator light on the instrument panel flashing to alert the driver that the headlight control system is malfunctioning.

[0097] This embodiment provides a vehicle lighting control system based on fusion perception, which includes a perception module, a control module, and an execution module. Figure 4 This is a schematic diagram illustrating the data acquisition performed by the sensing module and the data-level fusion performed by the control module according to an embodiment of the present invention. Figure 5 This is a schematic diagram of scene classification performed by the control module according to an embodiment of the present invention.

[0098] This perception module is used to acquire time-series data from multi-sensor observations. The module includes at least a forward-looking camera, a millimeter-wave radar, and a rain sensor. The forward-looking camera collects image data, which is then used to detect vehicle lights, taillights, pedestrians, etc. The millimeter-wave radar collects point cloud data, which is then used to determine distance, speed, angle, etc. The rain sensor collects rainfall data to determine the rainfall level. The target detection threshold of the millimeter-wave radar is adjusted according to the rainfall data at a preset frequency.

[0099] This control module extracts spatiotemporal features from the multi-sensor observation time-series data to generate fused features. These fused features include environmental context information and the motion states of various traffic targets. The control module also inputs these fused features into a scene classification model to identify the current driving scene and output a corresponding scene label. Furthermore, the control module matches the scene label with the corresponding headlight control strategy to generate headlight control commands.

[0100] The execution module includes a headlight control unit. This headlight control unit is used to perform headlight control operations according to the headlight control command.

[0101] Optionally, the sensing module also includes an ambient light sensor for real-time acquisition of light intensity data.

[0102] Optionally, the vehicle lighting control unit includes a headlight control unit for controlling the switching between high and low beams and supporting PWM dimming with a response time of ≤50ms. The vehicle lighting control unit also includes a hazard warning light control unit for activating hazard warning lights when a vehicle malfunction is detected.

[0103] For example, the forward-facing camera is mounted on the top of the windshield of the target vehicle, with a resolution of ≥2 million pixels and a frame rate of ≥30fps; the millimeter-wave radar operates in the 77GHz band, has a detection range of ≥200m, and an angular resolution of ≤1°; the rain sensor is an infrared reflective rain sensor capable of classifying rainfall into light, moderate, and heavy levels; and the ambient light sensor has a measurement range of 0.1 lux to 100 klux and a response time of ≤10 ms.

[0104] Optionally, the system also includes a human-machine interface for visually displaying relevant parameters of the target vehicle (such as vehicle speed), scene labels, and headlight control strategies. This human-machine interface also includes instrument panel indicator lights. When the instrument panel indicator light is white, it indicates that the headlight control system has entered automatic start mode, and activation can be determined based on factors such as light intensity. When the instrument panel indicator light is red, it indicates that the system has malfunctioned and the headlight control system needs to be manually switched on or off using the steering wheel buttons.

[0105] As one or more specific application embodiments of the present invention, the optimal implementation scheme or the scheme that the inventors most want to embody is described in combination with the specific application scenario.

[0106] This embodiment provides a vehicle lighting control method based on fusion perception, and the method flow is as follows:

[0107] a. System activation:

[0108] Conditions: Vehicle speed ≥ 40km / h, light intensity ≤ 10 lux, system enters automatic mode. Figure 6 This is a schematic diagram of the system activation process according to an embodiment of the present invention.

[0109] Status: The white indicator light on the dashboard is on.

[0110] b. Scene recognition and decision-making:

[0111] Meeting mode: When oncoming headlights are detected, if the distance to the oncoming vehicle DL1 ∈ the meeting mode switching threshold range RL1 (450m-600m), switch to low beam headlights;

[0112] Follow mode: When a taillight in the same direction is detected, if the distance between the following vehicles is DL2 ∈ Follow mode switching threshold range RL2 (150m-300m), switch to low beam headlights;

[0113] Special scenarios: Automatically turn off high beams when driving in tunnels, and switch them in advance on sharp curves (steering wheel angle > 30°).

[0114] c. Driver behavior learning:

[0115] Record the timing of manual switching (e.g., when DL1=500m, the driver actively turns off the high beams);

[0116] Update personalization parameters: Adjust the DL1 value from the default 550m to 500m to match driver habits.

[0117] d. Fault handling and redundancy:

[0118] In case of sensor failure: Switch to conservative mode (shorten the switching distance to 80% of the default value);

[0119] In case of system crash: Automatically switch to low beam headlights and issue an alarm via a red indicator light on the dashboard.

[0120] This embodiment also provides a vehicle lighting control device based on fusion perception, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0121] This embodiment provides a vehicle lighting control device based on fused perception, such as... Figure 7 As shown, it includes:

[0122] The acquisition module 701 is used to acquire multi-sensor observation time-series data; the multi-sensor observation time-series data includes at least image data, point cloud data collected by millimeter-wave radar, and rainfall data; wherein, the target detection threshold of the millimeter-wave radar is adjusted according to the rainfall data at a preset frequency;

[0123] The feature extraction module 702 is used to extract spatiotemporal features from the multi-sensor observation time-series data and generate fused features; the fused features include environmental context information and the motion state of each traffic target.

[0124] The scene classification module 703 is used to input the fused features into the scene classification model, identify the current driving scene, and output the corresponding scene label.

[0125] The control instruction module 704 is used to match the corresponding vehicle light control strategy according to the scene label and generate vehicle light control instructions, so that the vehicle light control unit can execute vehicle light control operations according to the vehicle light control instructions.

[0126] In an optional implementation, the device further includes a threshold adjustment module for:

[0127] Acquire rainfall data; this rainfall data is collected in real time by a rain sensor;

[0128] The rainfall data is statistically analyzed using a sliding window to generate an environmental rainfall intensity index.

[0129] Based on the environmental rainfall intensity index, the threshold adjustment coefficient is determined according to the preset mapping relationship;

[0130] The target detection threshold of the millimeter-wave radar is adjusted according to the threshold adjustment coefficient.

[0131] In an optional implementation, the feature extraction module is further configured to:

[0132] Time synchronization and spatial calibration are performed on the multi-sensor observation time series data to obtain multi-sensor aligned data;

[0133] The multi-sensor aligned data is subjected to target association and weighted fusion processing to generate motion state sequences corresponding to each traffic target.

[0134] The motion state sequence corresponding to each traffic target is estimated by a state estimation algorithm to generate the continuous motion trajectory of each traffic target.

[0135] Environmental features were extracted from the multi-sensor observation time-series data to obtain environmental context features;

[0136] The continuous motion trajectory of each traffic target is combined with the environmental context features to form the fused feature.

[0137] In one alternative implementation, the scene classification module is further used for:

[0138] Obtain fused features from multiple consecutive frames, including the current time.

[0139] The fused features from multiple consecutive frames are input into the first recurrent neural network unit in chronological order, and the hidden state is calculated frame by frame to obtain past context information.

[0140] The fused features from multiple consecutive frames are input into the second recurrent neural network unit in reverse chronological order, and propagated back frame by frame to obtain future context information.

[0141] The past and future context information are fused to generate bidirectional contextual joint features;

[0142] The bidirectional contextual joint feature is input into the classification layer, and the scene label for the current moment is output; the scene label includes at least meeting oncoming traffic, following oncoming traffic, sharp bends, and tunnel driving.

[0143] In an optional implementation, the device further includes a fault detection module for:

[0144] Real-time monitoring of the operating status of multiple sensors; these multiple sensors include at least a forward-looking camera, millimeter-wave radar, and a rain sensor.

[0145] When a fault is detected in any of the multiple sensors, the type of the faulty sensor is obtained;

[0146] If the faulty sensor is a preset sensor type, the headlight control strategy will be switched to a conservative operation mode, and the target detection threshold of the forward-facing camera / millimeter-wave radar will be increased; in this conservative operation mode, the critical distance for switching from high beam to low beam will be shortened.

[0147] The vehicle lighting control device based on fusion perception provided in this embodiment of the invention can execute the vehicle lighting control method based on fusion perception provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method. Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0148] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0149] The following is a detailed reference. Figure 8The diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 801, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 802 or a program loaded from memory 808 into random access memory (RAM) 803. The RAM 803 also stores various programs and data required for the operation of the electronic device. The processor 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0150] Typically, the following devices can be connected to I / O interface 805: input devices 806 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 807 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 808 including, for example, magnetic tapes, hard disks, etc.; and communication devices 809. Communication device 809 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 8 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0151] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 809, or installed from a memory 808, or installed from a ROM 802. When the computer program is executed by the processor 801, it performs the functions defined in the vehicle lighting control method based on fusion perception according to embodiments of the present invention.

[0152] Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0153] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the vehicle lighting control method based on fusion perception shown in the above embodiments is implemented.

[0154] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0155] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the invention.

Claims

1. A vehicle lighting control method based on fusion perception, characterized in that, The method includes: Acquire multi-sensor observation time-series data; the multi-sensor observation time-series data includes at least image data, point cloud data collected by millimeter-wave radar, and rainfall data; wherein, the target detection threshold of the millimeter-wave radar is adjusted according to the rainfall data at a preset frequency; Spatiotemporal features are extracted from the multi-sensor observation time-series data to generate fused features; the fused features include environmental context information and the motion state of each traffic target. The fused features are input into a scene classification model to identify the current driving scene and output the corresponding scene label. The corresponding vehicle lighting control strategy is matched according to the scene label, and a vehicle lighting control command is generated so that the vehicle lighting control unit can perform vehicle lighting control operations according to the vehicle lighting control command. The step of adjusting the target detection threshold of the millimeter-wave radar based on rainfall data includes: Acquire rainfall data; the rainfall data is collected in real time by a rainfall sensor; The rainfall data is statistically analyzed using a sliding window to generate an environmental rainfall intensity index. Based on the environmental rainfall intensity index, a threshold adjustment coefficient is determined according to a preset mapping relationship; The target detection threshold of the millimeter-wave radar is adjusted according to the threshold adjustment coefficient. The step of extracting spatiotemporal features from the multi-sensor observation time-series data to generate fused features includes: The time-series data of the multi-sensor observations are synchronized in time and calibrated in space to obtain multi-sensor aligned data. The multi-sensor aligned data is subjected to target association and weighted fusion processing to generate motion state sequences corresponding to each traffic target; The motion state sequence corresponding to each traffic target is estimated by a state estimation algorithm to generate the continuous motion trajectory of each traffic target. Environmental features are extracted from the multi-sensor observation time-series data to obtain environmental context features; The continuous motion trajectories of each traffic target are combined with the environmental context features to form the fused features; The step of inputting the fused features into a scene classification model to identify the current driving scene and output the corresponding scene label includes: Obtain fused features from multiple consecutive frames, including the current time. The fused features of the consecutive multi-frames are input into the first recurrent neural network unit in chronological order, and the hidden state is calculated frame by frame to obtain the past context information. The fused features of the consecutive multi-frames are input into the second recurrent neural network unit in reverse chronological order, and propagated back frame by frame to obtain future context information. The past context information and future context information are fused to generate a bidirectional context joint feature; The bidirectional contextual joint features are input into the classification layer, and the scene label at the current moment is output; the scene label includes at least meeting oncoming traffic, following oncoming traffic, sharp bends, and tunnel driving.

2. The method according to claim 1, characterized in that, The method further includes: Real-time monitoring of the operating status of multiple sensors; the multiple sensors include at least a forward-looking camera, a millimeter-wave radar, and a rain sensor; When a fault is detected in any of the multiple sensors, the type of the faulty sensor is obtained; If the faulty sensor is a preset sensor type, the headlight control strategy is switched to a conservative operation mode, and the target detection threshold of the forward-facing camera / millimeter-wave radar is increased; wherein, in the conservative operation mode, the critical distance for switching from high beam to low beam is shortened.

3. A vehicle lighting control device based on fusion perception, characterized in that, The device includes: The acquisition module is used to acquire multi-sensor observation time-series data; the multi-sensor observation time-series data includes at least image data, point cloud data collected by millimeter-wave radar, and rainfall data; wherein, the target detection threshold of the millimeter-wave radar is adjusted according to the rainfall data at a preset frequency; The feature extraction module is used to extract spatiotemporal features from the multi-sensor observation time-series data and generate fused features; the fused features include environmental context information and the motion state of each traffic target; The scene classification module is used to input the fused features into the scene classification model, identify the current driving scene, and output the corresponding scene label. The control command module is used to match the corresponding vehicle light control strategy according to the scene label and generate vehicle light control commands, so that the vehicle light control unit can execute vehicle light control operations according to the vehicle light control commands; The device further includes a threshold adjustment module for: Acquire rainfall data; the rainfall data is collected in real time by a rainfall sensor; The rainfall data is statistically analyzed using a sliding window to generate an environmental rainfall intensity index. Based on the environmental rainfall intensity index, a threshold adjustment coefficient is determined according to a preset mapping relationship; The target detection threshold of the millimeter-wave radar is adjusted according to the threshold adjustment coefficient. The feature extraction module is also used for: The time-series data of the multi-sensor observations are synchronized in time and calibrated in space to obtain multi-sensor aligned data. The multi-sensor aligned data is subjected to target association and weighted fusion processing to generate motion state sequences corresponding to each traffic target; The motion state sequence corresponding to each traffic target is estimated by a state estimation algorithm to generate the continuous motion trajectory of each traffic target. Environmental features are extracted from the multi-sensor observation time-series data to obtain environmental context features; The continuous motion trajectories of each traffic target are combined with the environmental context features to form the fused features; The scene classification module is also used for: Obtain fused features from multiple consecutive frames, including the current time. The fused features of the consecutive multi-frames are input into the first recurrent neural network unit in chronological order, and the hidden state is calculated frame by frame to obtain the past context information. The fused features of the consecutive multi-frames are input into the second recurrent neural network unit in reverse chronological order, and propagated back frame by frame to obtain future context information. The past context information and future context information are fused to generate a bidirectional context joint feature; The bidirectional contextual joint features are input into the classification layer, and the scene label at the current moment is output; the scene label includes at least meeting oncoming traffic, following oncoming traffic, sharp bends, and tunnel driving.

4. A vehicle lighting control system based on fusion perception, characterized in that, The system includes: A perception module is used to acquire time-series observation data from multiple sensors; the perception module includes at least a forward-looking camera, a millimeter-wave radar, and a rain sensor; the forward-looking camera is used to acquire image data; the millimeter-wave radar is used to acquire point cloud data; the rain sensor is used to acquire rainfall data; wherein, the target detection threshold of the millimeter-wave radar is adjusted according to the rainfall data at a preset frequency. The control module is used to extract spatiotemporal features from the multi-sensor observation time-series data and generate fused features; the fused features include environmental context information and the motion state of each traffic target; the fused features are input into a scene classification model to identify the current driving scene and output the corresponding scene label; the corresponding headlight control strategy is matched according to the scene label to generate headlight control commands; The execution module includes a vehicle lighting control unit; the vehicle lighting control unit is used to execute vehicle lighting control operations according to the vehicle lighting control command; The sensing module is also used for: Acquire rainfall data; the rainfall data is collected in real time by a rainfall sensor; The rainfall data is statistically analyzed using a sliding window to generate an environmental rainfall intensity index. Based on the environmental rainfall intensity index, a threshold adjustment coefficient is determined according to a preset mapping relationship; The target detection threshold of the millimeter-wave radar is adjusted according to the threshold adjustment coefficient. The control module is also used for: The time-series data of the multi-sensor observations are synchronized in time and calibrated in space to obtain multi-sensor aligned data. The multi-sensor aligned data is subjected to target association and weighted fusion processing to generate motion state sequences corresponding to each traffic target; The motion state sequence corresponding to each traffic target is estimated by a state estimation algorithm to generate the continuous motion trajectory of each traffic target. Environmental features are extracted from the multi-sensor observation time-series data to obtain environmental context features; The continuous motion trajectories of each traffic target are combined with the environmental context features to form the fused features; The control module is also used for: Obtain fused features from multiple consecutive frames, including the current time. The fused features of the consecutive multi-frames are input into the first recurrent neural network unit in chronological order, and the hidden state is calculated frame by frame to obtain the past context information. The fused features of the consecutive multi-frames are input into the second recurrent neural network unit in reverse chronological order, and propagated back frame by frame to obtain future context information. The past context information and future context information are fused to generate a bidirectional context joint feature; The bidirectional contextual joint features are input into the classification layer, and the scene label at the current moment is output; the scene label includes at least meeting oncoming traffic, following oncoming traffic, sharp bends, and tunnel driving.

5. An electronic device, characterized in that, include: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the vehicle lighting control method based on fusion perception as described in claim 1 or 2.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the vehicle lighting control method based on fusion perception as described in claim 1 or 2.

7. A computer program product, characterized in that, Includes computer instructions for causing a computer to execute the vehicle lighting control method based on fusion perception as described in claim 1 or 2.

Images