Scene adaptive projection vehicle lamp system based on deep reinforcement learning and control method

Abstract

The invention provides a scene adaptive projection vehicle lamp system based on deep reinforcement learning and a control method, and relates to the technical field of intelligent vehicle lamps and automatic driving perception systems. Comprising a multi-modal sensing module, a feature fusion module, a strategy generation module and an execution module. The multi-mode sensing module is used for collecting environment state data and performing primary processing to form an environment data information flow; the feature fusion module is used for generating a unified environment feature vector for the environment data information flow; the strategy generation module is used for receiving the environment feature vector and generating a vehicle lamp adjustment strategy through multi-layer neural network calculation; evaluating a result obtained by executing the vehicle lamp adjustment strategy based on the vehicle lamp, and optimizing strategy parameters of the strategy network based on a PPO algorithm; the execution module is used for controlling the vehicle lamp according to the vehicle lamp adjustment strategy output by the strategy network. The intelligent level of the vehicle lamp is remarkably improved, and the system is widely applied to night driving assistance, urban interaction prompt and low-visibility driving scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent vehicle lights and autonomous driving perception systems, and in particular to a scene-adaptive projection vehicle light system and control method based on deep reinforcement learning. Background Art

[0002] Existing intelligent vehicle lighting systems are mostly based on fixed logic or static sensor triggering control, which has the following technical problems:

[0003] (1) Poor scene adaptability: It is difficult to cope with changing driving environments, such as rainy and snowy weather, complex road conditions, and pedestrian interference at night.

[0004] (2) Weak autonomous learning ability: Lack of systematic learning mechanism, unable to self-evolve and optimize based on driving history and user behavior.

[0005] (3) Disconnection between perception and projection: The headlight control and perception systems are separated, lacking the overall perception and decision-making capabilities of the environmental semantics.

[0006] (4) Weak cross-model adaptability: The development of new models requires retraining parameters, resulting in insufficient generalization capabilities and low deployment efficiency.

[0007] The above problems are in urgent need of resolution. Summary of the Invention

[0008] The purpose of the present invention is to overcome at least one technical problem existing in the prior art and provide a scene-adaptive projection lighting system and control method based on deep reinforcement learning.

[0009] On the one hand, an embodiment of the present invention provides a scene-adaptive projection headlight system based on deep reinforcement learning, the projection headlight system comprising: a multimodal perception module, a feature fusion module, a strategy generation module and an execution module; the multimodal perception module is used to collect environmental status data and perform preliminary processing to form an environmental data information stream; the feature fusion module integrates a graph neural network, an LSTM network and a Transformer structure to generate a unified environmental feature vector for the environmental data information stream; the strategy generation module integrates a strategy network module and a value network module; the strategy network module is used to receive the environmental feature vector and generate a headlight adjustment strategy through a multi-layer neural network calculation, including: outputting a light spot projection range, brightness and shielding angle; the value network module is used to evaluate the results obtained by executing the headlight adjustment strategy based on the headlight, and optimize the strategy parameters of the strategy network based on the PPO algorithm; the execution module is used to control the light spot projection range, brightness and shielding angle of the headlight according to the headlight adjustment strategy output by the strategy network.

[0010] Furthermore, the projection headlight system also includes a knowledge transfer module, which is used to update strategy parameters through scene label recognition, prototype network matching and MAML algorithm optimization when the scene switches.

[0011] Furthermore, the knowledge transfer module integrates a scene label recognition unit, a prototype network matching unit and a MAML algorithm optimization unit; the scene label recognition unit is used to determine the scene switch when it detects a change in any dimension of the terrain type, time, weather, traffic density, driving style, vehicle model structure and environmental dynamic scene label; the prototype network matching unit is used to find the most similar prototype by matching the label of the new scene with the scene prototype pre-stored in the system after the scene switch, thereby obtaining the prior strategy knowledge corresponding to the prototype; the MAML algorithm optimization unit is used to use the MAML algorithm to quickly adjust the existing strategy network parameters based on the similar scene prototype strategy obtained by prototype network matching, combined with N groups of sample data of the new scene, to generate a headlight control strategy adapted to the current new scene, thereby realizing rapid migration and optimization of the strategy.

[0012] Furthermore, the multimodal perception module is integrated with a device collaborative acquisition unit and a data processing unit; the device collaborative acquisition unit is integrated with one or a combination of a forward camera, a millimeter-wave radar, a ToF depth camera and an ambient light sensor; the forward camera is used to capture road images, identify lane lines, road conditions and objects in front; the millimeter-wave radar is used to emit electromagnetic waves to detect the distance, speed and direction of obstacles; the ToF depth camera is used to obtain scene depth information by measuring the time of flight of light and locate the three-dimensional position of objects; the ambient light sensor is used to monitor the ambient brightness and / or weather conditions in real time; the data processing unit is used to perform preliminary filtering and noise reduction on the data collected in the device collaborative acquisition unit, eliminate obvious errors or interference data, and form a preliminary environmental data information flow.

[0013] Furthermore, the feature fusion module integrates a spatial topology modeling unit, a time series processing unit, an interactive feature extraction unit and a feature fusion and dimensionality reduction unit; the spatial topology modeling unit is used to model the spatial topological relationship between the vehicle and the obstacle based on a graph neural network; the time series processing unit is used to process the time series driving behavior based on the LSTM network; the interactive feature extraction unit is used to extract human-vehicle interaction features based on the Transformer structure; the feature fusion and dimensionality reduction unit is used to fuse the features extracted by the spatial topology modeling unit, the time series processing unit and the interactive feature extraction unit to generate a unified environmental feature vector.

[0014] Furthermore, the spatial topology modeling unit is used to regard the vehicle and each obstacle as nodes of the graph based on GNN, construct edges according to their relative positions and distances, and form a spatial topology graph to characterize the spatial relationship between the vehicle and the surrounding objects; the time series processing unit is used to receive time series data from the multimodal perception module based on the LSTM network, capture the changing trends and laws of driving behavior over time, and extract features in the time dimension; the interactive feature extraction unit is used to extract features of pedestrian postures and possible interactive behaviors based on the Transformer structure, focusing on the interaction patterns between people and vehicles; the feature fusion and dimensionality reduction unit is used to fuse the features extracted by GNN, LSTM and Transformer, and map the fused high-dimensional features into a 128-dimensional unified environmental feature vector through linear transformation operations.

[0015] Furthermore, the strategy network module integrates a strategy network calculation unit and an action space constraint unit; the strategy network calculation unit is used to output the light spot projection range ΔR, brightness ΔL and shielding angle ΔS based on the received environmental feature vector through a multi-layer neural network calculation based on a preset strategy network; the action space constraint unit is used to limit the action value output by the strategy network calculation unit to ensure that the headlight adjustment is within a safe and reasonable range.

[0016] Furthermore, the value network module integrates a reward function calculation unit, a value network evaluation unit and a PPO algorithm optimization unit; the reward function calculation unit is used to calculate the reward function value based on the result obtained by executing the current headlight adjustment strategy based on the headlights, based on a preset reward function calculation method; the value network evaluation unit is used to evaluate the pros and cons of the current strategy based on the reward function according to a pre-trained value network, and obtain a value estimate; the PPO algorithm optimization unit is used to update the parameters of the strategy network based on the PPO algorithm, using the evaluation results of the value network and the reward function, so as to optimize the strategy in the direction of obtaining higher rewards.

[0017] Furthermore, the reward function calculation method includes:

[0018] R=0.4·SSIM+0.2·Obstacle avoidance success rate+0.2·Energy efficiency

[0019] +0.2·Pedestrian gaze feedback;

[0020] Where SSIM represents the structural similarity index;

[0021] The PPO algorithm optimization unit integrates an advantage function calculation unit, an objective function design unit and a gradient descent update unit;

[0022] The advantage function calculation unit is programmed with an advantage function calculation formula, including:

[0023] A(s,a)=R+γV(s')-V(s);

[0024] The objective function calculation formula is burned into the objective function design unit, including:

[0025]

[0026] Where R is the reward function value, γ is the discount factor, V(s) is the value estimate in the current state, V(s') is the value estimate in the next state, and π θ (a|s) represents the probability of selecting action a in state s under the new policy parameters θ, Expressed as the old policy parameter θ old Under this condition, the probability of selecting action a in state s, ∈ is the clipping parameter;

[0027] The gradient descent update unit is used to optimize the policy network parameters through back propagation to maximize the objective function.

[0028] In the second aspect, an embodiment of the present invention provides a scene-adaptive projection headlight system control method based on deep reinforcement learning, which is applied to the above-mentioned scene-adaptive projection headlight system based on deep reinforcement learning. The control method includes: collecting environmental status data through a multimodal perception module and performing preliminary processing to form an environmental data information flow; generating a unified environmental feature vector for the environmental data information flow based on a graph neural network, an LSTM network and a Transformer structure; receiving the environmental feature vector through a policy network module, and generating a headlight adjustment strategy through a multi-layer neural network calculation, including: outputting the light spot projection range, brightness and shielding angle; evaluating the results obtained by executing the headlight adjustment strategy based on the headlight through a value network module, and optimizing the policy parameters of the policy network based on the PPO algorithm; controlling the light spot projection range, brightness and shielding angle of the headlight according to the headlight adjustment strategy output by the policy network.

[0029] On the other hand, the present invention also provides a computer-readable storage medium, which stores one or more instructions, and the computer instructions are used to enable the computer to execute the above-mentioned scene-adaptive projection vehicle light system control method based on deep reinforcement learning.

[0030] On the other hand, the present invention provides an electronic device comprising: a memory and a processor; the memory stores at least one program instruction; the processor implements the above-mentioned scene-adaptive projection vehicle light system control method based on deep reinforcement learning by loading and executing the at least one program instruction.

[0031] The beneficial effects of the present invention are:

[0032] Existing intelligent headlight systems suffer from problems such as poor scene adaptability, weak autonomous learning capabilities, disconnection between perception and projection, and poor cross-vehicle adaptability. This invention, through technological innovation, achieves breakthroughs in multimodal perception, dynamic strategy generation, knowledge transfer, and interactive functions, effectively improving the intelligence level of headlights and driving safety. Specific technical effects are as follows:

[0033] (1) Multimodal fusion perception improves environmental adaptability: Fusion of multi-sensor data such as forward-facing cameras and millimeter-wave radars enables accurate perception of roads, obstacles, pedestrians, and lighting environments, effectively responding to changing scenarios such as rainy and snowy weather and complex road conditions, and solving the problem of poor scene adaptability.

[0034] (2) Deep reinforcement learning driven to achieve autonomous optimization: The policy network based on the PPO algorithm is combined with the reward function (including SSIM, obstacle avoidance success rate, etc.), so that the system can optimize the projection strategy through continuous learning and self-evolve according to the driving history without human intervention, thus making up for the defect of weak autonomous learning ability.

[0035] (3) End-to-end strategy generation, connecting perception and control: The feature fusion module extracts spatiotemporal features through GNN, LSTM, etc. to generate environment vectors, directly driving the strategy network to output projection actions (such as spot range, brightness, etc.), achieving deep coupling of perception and projection, and solving the problem of disconnection between the two.

[0036] (4) Seven-dimensional label knowledge transfer to enhance cross-scenario adaptation: Based on the prototype network and MAML algorithm, using seven-dimensional scene labels (terrain, weather, etc.), only 5 sets of samples are needed to complete the strategy reconstruction across vehicle models and geographical environments, greatly improving the generalization ability and solving the problem of weak cross-vehicle adaptability.

[0037] (5) Dynamic shielding and interaction to improve safety and experience: When pedestrians or obstacles are detected, pixel-level dynamic shielding is achieved within 0.1 seconds to reduce interference; at the same time, it supports the projection of guide arrows, warning icons, etc., to improve driving safety and interactive experience in scenarios such as night lane guidance and tunnel brightness adjustment.

[0038] (6) Multi-dimensional reward optimization, balancing energy efficiency and safety: The reward function integrates factors such as image quality, obstacle avoidance efficiency, and energy consumption, optimizing the projection effect while reducing energy consumption, thus achieving a balance between safety and energy efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The present invention will be further described below with reference to the accompanying drawings and examples.

[0040] Figure 1This is a structural diagram of a scene-adaptive projection lighting system based on deep reinforcement learning provided in Example 1 of the present invention.

[0041] Figure 2 This is a schematic diagram of a multimodal perception process and spatiotemporal feature fusion process provided by Example 1 of the present invention.

[0042] Figure 3 This is a schematic diagram of a deep reinforcement learning strategy generation and control process provided by Example 1 of the present invention.

[0043] Figure 4 This is a schematic diagram of a cross-vehicle / scenario migration network structure provided by Example 1 of the present invention.

[0044] Figure 5 This is a flow chart of a scene-adaptive projection vehicle lighting system control method based on deep reinforcement learning provided in Example 2 of the present invention.

[0045] Figure 6 This is a partial block diagram of an electronic device provided in Example 4 of the present invention. DETAILED DESCRIPTION

[0046] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the various operations as sequential processes, many of the operations therein can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the various operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0047] It should be understood that although the terms "first," "second," and the like may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used solely to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. The term "and / or" as used herein includes any and all combinations of one or more of the listed associated items.

[0048] The present invention will now be described in detail with reference to the accompanying drawings. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore only shows the components related to the present invention.

[0049] Example 1

[0050] For ease of understanding, the invention concept is generally described before describing the embodiments of the present invention in detail: The present invention provides a scene-adaptive projection lighting system and control method based on deep reinforcement learning, focusing on the fields of smart lighting and autonomous driving assistance. The core is to build a projection lighting system with environmental adaptability through the integration of deep reinforcement learning and multiple technologies. The specific invention concept is as follows: Multimodal perception fusion: Integrate forward cameras, millimeter-wave radars, ToF sensors and light sensors to achieve multi-dimensional collection of roads, obstacles, pedestrians, lighting and meteorological data, providing comprehensive environmental information input for the system. Spatiotemporal feature extraction and fusion: Use graph neural networks (GNN) to model the spatial topological relationship between vehicles and obstacles, combine LSTM to process time series driving behaviors, and then use the Transformer structure to extract human-vehicle interaction features, and finally generate a 128-dimensional unified environmental state vector to achieve a deep understanding of environmental information. Deep reinforcement learning strategy generation: Construct a policy network (Actor) and a value network (Critic) based on the PPO algorithm. The policy network takes the environment state vector as input and outputs an action space containing the light spot projection range, brightness, and occlusion angle. The value network uses a reward function (Reward = 0.4·SSIM + 0.2·Obstacle Avoidance Success Rate + 0.2·Energy Efficiency + 0.2·Pedestrian Attention Feedback) to guide the system to continuously optimize its policy. Knowledge Transfer Mechanism: A prototype network is constructed using seven-dimensional scene labels (terrain type, time of day, weather, traffic density, driving style, vehicle model structure, and environmental dynamics). Combined with the MAML algorithm, this allows for rapid policy transfer across vehicle models and geographical environments, requiring only five sets of scene samples to complete policy reconstruction. Dynamic Occlusion and Interaction: When pedestrians are detected or objects enter the warning zone, the system can perform pixel-level dynamic occlusion within 0.1 seconds. It also supports projecting virtual guide arrows, warning icons, or interactive text, enabling route guidance, visual warnings, and human-vehicle interaction. Through the collaborative work of the above modules, this invention realizes the adaptive projection of vehicle lights in different environments, improves driving safety, energy saving and interactive experience at night and in low visibility scenarios, and is widely applicable to various scenarios such as night driving assistance and urban interactive prompts.

[0051] The specific implementation is as follows:

[0052] like Figure 1 As shown in the figure, it is a schematic structural diagram of a scene-adaptive projection car light system based on deep reinforcement learning provided by the present invention.

[0053] As an example, the projection headlight system includes: a multimodal perception module 1, a feature fusion module 2, a strategy generation module 3 and an execution module 4; the multimodal perception module 1 is used to collect environmental status data and perform preliminary processing to form an environmental data information flow; the feature fusion module 2 integrates a graph neural network, an LSTM network and a Transformer structure to generate a unified environmental feature vector for the environmental data information flow; the strategy generation module 3 integrates a strategy network module 30 and a value network module 31; the strategy network module 30 is used to receive the environmental feature vector and generate a headlight adjustment strategy through a multi-layer neural network calculation, including: outputting the light spot projection range, brightness and shielding angle; the value network module 31 is used to evaluate the results obtained by executing the headlight adjustment strategy based on the headlight, and optimize the strategy parameters of the strategy network based on the PPO algorithm; the execution module 4 is used to control the light spot projection range, brightness and shielding angle of the headlight according to the headlight adjustment strategy output by the strategy network.

[0054] In some feasible embodiments, the multimodal perception module 1 is integrated with a device collaborative acquisition unit 10 and a data processing unit 11; the device collaborative acquisition unit 10 is integrated with one or a combination of a forward camera 101, a millimeter-wave radar 102, a ToF depth camera 103 and an ambient light sensor 104; the forward camera 101 is used to capture road images, identify lane lines, road conditions and objects in front; the millimeter-wave radar 102 is used to emit electromagnetic waves to detect the distance, speed and direction of obstacles; the ToF depth camera 103 is used to obtain scene depth information by measuring the time of flight of light and locate the three-dimensional position of objects; the ambient light sensor 104 is used to monitor the ambient brightness and / or weather conditions in real time; the data processing unit 11 is used to perform preliminary filtering and noise reduction on the data collected in the device collaborative acquisition unit 10, eliminate obvious errors or interference data, and form a preliminary environmental data information flow.

[0055] Preferably, forward-facing camera 101 captures road images and identifies lane markings, road conditions, and objects ahead; millimeter-wave radar 102 emits electromagnetic waves to detect obstacle distance, speed, and direction; ToF depth camera 103 measures light time-of-flight to obtain scene depth information and accurately locate objects in three dimensions; and ambient light sensor 104 monitors ambient brightness, weather conditions (such as rain, snow, and haze) in real time. Data processing unit 11 performs preliminary filtering and noise reduction on the raw data collected by each sensor (such as image pixel values, radar point clouds, depth values, and light intensity values), eliminating obvious errors or interference, and forming a preliminary environmental data information stream.

[0056] In some possible implementations, combined Figure 2As shown, the feature fusion module 2 integrates a spatial topology modeling unit 20, a time series processing unit 21, an interactive feature extraction unit 22 and a feature fusion and dimensionality reduction unit 23; the spatial topology modeling unit 20 is used to model the spatial topological relationship between the vehicle and the obstacle based on a graph neural network; the time series processing unit 21 is used to process the time series driving behavior based on the LSTM network; the interactive feature extraction unit 22 is used to extract human-vehicle interaction features based on the Transformer structure; the feature fusion and dimensionality reduction unit 23 is used to fuse the features extracted by the spatial topology modeling unit, the time series processing unit and the interactive feature extraction unit to generate a unified environmental feature vector.

[0057] Preferably, the spatial topology modeling unit 20 is used to regard the vehicle and each obstacle as nodes of the graph based on GNN, construct edges according to their relative positions and distances, and form a spatial topology graph to characterize the spatial relationship between the vehicle and the surrounding objects; the time series processing unit 21 is used to receive the time series data from the multimodal perception module 1 based on the LSTM network, capture the changing trends and laws of driving behavior over time, and extract features in the time dimension; the interactive feature extraction unit 22 is used to extract features of pedestrian postures and possible interactive behaviors based on the Transformer structure, focusing on the interaction pattern between people and vehicles; the feature fusion and dimensionality reduction unit 23 is used to fuse the features extracted by GNN, LSTM and Transformer, and map the fused high-dimensional features into a 128-dimensional unified environmental feature vector through linear transformation operations.

[0058] In some feasible implementations, the policy network module 30 integrates a policy network calculation unit 301 and an action space constraint unit 302; the policy network calculation unit 301 is used to output the light spot projection range ΔR, brightness ΔL and shielding angle ΔS based on a preset policy network (ActorNetwork) for the received environmental feature vector through multi-layer neural network calculation; the action space constraint unit 302 is used to limit the action value output by the policy network calculation unit to ensure that the headlight adjustment is within a safe and reasonable range.

[0059] Preferably, combined Figure 3As shown, the policy network, built based on the PPO algorithm, takes a 128-dimensional environment state vector as input and generates a headlight adjustment policy through the following process: Environment state vector parsing: The 128-dimensional input vector incorporates multimodal perception data (such as road topology, obstacle locations, and pedestrian interaction characteristics). The policy network first decodes its features using a multi-layer perceptron (MLP), separating key information such as spatial position, light intensity, and dynamic obstacle trajectories. Action space mapping calculation: Projection direction angle ΔR: Based on information such as road curvature and lane offset, the angle by which the projected light needs to be deflected is calculated. For example, when a curve is detected ahead, a positive ΔR output shifts the light spot toward the inside of the curve to guide the driver's line of sight. Brightness ΔL: The system dynamically adjusts the brightness by combining ambient light sensor data with oncoming vehicle detection results. In the event of oncoming traffic, a negative ΔL output reduces local brightness to prevent glare. In low-light environments such as tunnels, a positive ΔL output enhances illumination. Blocking angle ΔS: When the ToF camera detects a pedestrian or obstacle entering the projection area, it calculates the angle range that needs to be blocked and outputs ΔS to turn off the projection in the corresponding area to reduce light interference (such as blocking the light spot in the direction of the pedestrian's gaze).

[0060] Preferably, the motion space constraint unit 302 limits the light spot projection range, brightness and shielding angle, such as ΔR∈[-15°, 15°], brightness ΔL∈[-300, 300] lumens, and shielding angle ΔS∈[-10°, 10°].

[0061] Preferably, the policy network receives environmental status updates at a high frequency (e.g., 100Hz), rapidly calculates and outputs the currently optimal headlight adjustment parameters, and drives the execution module 4 to perform real-time adjustments to the light spot shape, brightness, and shielding area. For example, if a pedestrian is detected approaching a zebra crossing on an urban road, the system outputs a ΔS signal within 0.1 seconds to turn off the light spot above the pedestrian and simultaneously projects a "Please Wait" warning graphic.

[0062] In some possible implementations, combined Figure 3 As shown, the value network module 31 integrates a reward function calculation unit 310, a value network evaluation unit 311 and a PPO algorithm optimization unit 312; the reward function calculation unit 310 is used to calculate the reward function value based on the result obtained by executing the current headlight adjustment strategy based on the headlights, based on a preset reward function calculation method; the value network evaluation unit 311 is used to evaluate the pros and cons of the current strategy based on the reward function according to a pre-trained value network, and obtain a value estimate; the PPO algorithm optimization unit 312 is used to update the parameters of the strategy network based on the PPO algorithm, using the evaluation results of the value network and the reward function, so as to optimize the strategy towards obtaining higher rewards.

[0063] Preferably, the reward function calculation method includes:

[0064] R = 0.4·SSIM+0.2·Obstacle avoidance success rate+0.2·Energy efficiency+0.2·Pedestrian gaze feedback;

[0065] Where SSIM represents the structural similarity index;

[0066] More specifically, SSIM (Structural Similarity Index) means: by comparing the structural similarity between the actual headlight projection image and the ideal projection image, the accuracy of the light spot coverage is measured. The acquisition methods include: Ideal image preset: according to the current environment (such as road curvature, obstacle position), pre-set the "ideal light spot projection effect" (for example, in a curve scene, the ideal image should be the light spot extending along the inside of the curve). Real-time image acquisition: Use a forward-facing camera or a dedicated projection monitoring camera to capture the projection image of the headlights on the road surface / obstacle in real time. Numerical calculation: Use the SSIM algorithm to calculate the brightness, contrast and structural similarity of the two images, and output a value between 0-1 (the closer to 1, the closer the projection effect is to the ideal state).

[0067] The obstacle avoidance success rate means: judging whether the vehicle has successfully avoided the obstacle due to the headlight adjustment strategy through sensor data. The acquisition methods include: Obstacle monitoring: millimeter wave radar and ToF depth camera track the position and movement trajectory of obstacles (such as pedestrians, vehicles, roadblocks) in real time. Trajectory prediction and verification: After the strategy is executed, the system predicts the collision risk of the obstacle with the vehicle based on the current trajectory of the obstacle (such as TTC collision time), and monitors in real time whether the vehicle actually avoids the obstacle. Result judgment: If the vehicle maintains a safe distance from the obstacle within the preset safety time (such as TTC>2 seconds), or the obstacle is successfully bypassed, the obstacle avoidance is judged to be successful, otherwise it fails. The obstacle avoidance success rate is obtained by dividing the number of successful obstacle avoidances by the total number of obstacles.

[0068] Energy efficiency measures the energy efficiency of a lighting adjustment strategy when achieving its intended effect. This data is collected through onboard current and voltage sensors, which record the power consumption (in watts) of the lighting (e.g., LED light sources) in real time. Energy efficiency is calculated as (baseline energy consumption - actual energy consumption) / baseline energy consumption.

[0069] Pedestrian gaze feedback measures whether the lights are providing interactive cues by detecting whether the pedestrian is gazing at the projected area of ​​the headlights. This information is obtained by using a forward-facing camera and computer vision algorithms (such as OpenPose) to identify the pedestrian's head orientation and eye gaze (using key points to detect eye posture). The actual projection area of ​​the headlights is marked in the image (e.g., the area of ​​the light spot on the road). The pedestrian gaze feedback score is calculated by dividing the number of frames in the projected area by the total number of detected frames.

[0070] As mentioned above, SSIM focuses on projection accuracy to ensure that lights are properly positioned; obstacle avoidance success rate verifies the safety of the strategy, ensuring that lights assist in avoiding danger; energy efficiency balances performance and power consumption to avoid resource waste; and pedestrian gaze feedback strengthens human-vehicle interaction and enhances the coordination of traffic participants. These four metrics together form a multi-dimensional reward function that drives the value network to comprehensively evaluate the strategy, thereby guiding the policy network to iterate towards the optimal direction of "precision, safety, energy saving, and interaction."

[0071] Preferably, the PPO algorithm optimization unit 312 integrates an advantage function calculation unit 3120, an objective function design unit 3121 and a gradient descent update unit 3122; the advantage function calculation unit 3120 is burned with an advantage function calculation formula, including:

[0072] A(s,a)=R+γV(s')-V(s);

[0073] The objective function calculation formula is burned into the objective function design unit 3121, including:

[0074]

[0075] Where R is the reward function value, γ is the discount factor, V(s) is the value estimate in the current state, V(s') is the value estimate in the next state, and π θ (a|s) represents the probability of selecting action a in state s under the new policy parameters θ, Expressed as the old policy parameter θ old Under this condition, the probability of selecting action a in state s, ∈ is the clipping parameter;

[0076] The gradient descent update unit 3122 is used to optimize the policy network parameters through back propagation to maximize the objective function.

[0077] Specifically, the advantage function calculation formula is burned into the advantage function calculation unit 3120, including:

[0078] A(s,a)=R+γV(s')-V(s);

[0079] The discount factor γ is preferably 0.99, V(s) = Critic(s t ), indicating that from state s t The long-term expected return of triggering the execution of the current strategy, that is, the quality of the strategy in this state. V(s')=Critic(s t+1 ), indicating that from state s t+1 The long-term expected benefit of triggering the execution of the current strategy, that is, the acquisition process includes: after executing action a, the sensor obtains new environmental data, and the feature fusion module generates a new state vector st+1 , will s t+1 Input and s t The value estimates obtained in the same value network.

[0080] Specifically, the objective function design unit 3121 is programmed with an objective function calculation formula, including:

[0081]

[0082] Here, ∈ is the clipping parameter, preferably 0.2.

[0083] For a specific example, assume that in the intelligent headlight control scenario, the vehicle is in a rural curve environment at night. The following is a specific calculation example of the new and old strategy probabilities: Environmental state s: The forward camera recognizes that the road ahead is a mountain road that bends to the left, and the millimeter-wave radar detects an obstacle 50 meters inside the curve; the ToF depth camera obtains the three-dimensional position of the obstacle; the ambient light sensor determines that there is no streetlight at night. Action a: The headlight adjustment parameters output by the strategy network are light spot projection range ΔR = -8° (deflected to the left), brightness ΔL = 250 lumens, and shielding angle ΔS = -8°. Old strategy Calculating the action probability includes: the old strategy network (the last round of iteration parameters θ old ) Based on the environment state s, the probability of action a is calculated through the neural network: Assume that the old policy network outputs Gaussian distribution parameter μ for ΔR old =-7°, σ old =2°, then the probability density at ΔR=-8° is:

[0084] According to the Gaussian distribution formula Calculated Similarly, the probability of brightness ΔL and shielding angle ΔS is calculated, assuming that they are Since action a is a multi-dimensional continuous value, the total probability is the product of the probabilities of each dimension:

[0085] Current strategy π θ Calculate the action probability: The current policy network (parameter θ) receives the same environment state s and outputs new Gaussian distribution parameters: output μ = -8.5°, σ = 1.8° for ΔR, and calculate π θ (ΔR=-8°|s)≈0.20. Calculate the probability of ΔL and ΔS, assuming they are π θ (ΔL=250|s)≈0.14 and π θ (ΔS=-8°|s)≈0.18. Total probability: π θ (a|s)=π θ (ΔR=-8°|s)·π θ(ΔL=250|s)·π θ (ΔS=-8°|s)≈0.00504.

[0086] Specifically, the gradient descent update unit 3122 is used to optimize the policy network parameters through back propagation to maximize the objective function. This includes: calculating the gradient of L with respect to θ through the back propagation algorithm. renew Where α is the learning rate, preferably 0.001, and the gradient Points to the direction that increases L, so the parameters are updated along the gradient direction. The updated parameters θ new This will make the policy network output the probability of action a in a similar state s closer to the direction that can obtain high rewards (for example, the light spot deflection angle may be adjusted from -8° to -8.5° to more accurately cover curved obstacles).

[0087] Through continuous learning, the strategy can automatically adjust headlights for different road conditions (such as curves and tunnels) and weather conditions (rain and fog), improving safety and energy efficiency. The PPO algorithm's pruning mechanism prevents the strategy from falling into the trap of "high rewards in the short term but suboptimal in the long term" (such as excessive energy consumption in pursuit of obstacle avoidance success rate), ensuring balanced optimization of multiple objectives.

[0088] In some possible implementations, combined Figure 4 As shown, the projection lamp system further includes a knowledge transfer module 5, which is used to update strategy parameters through scene label recognition, prototype network matching and MAML algorithm optimization when the scene is switched.

[0089] Preferably, the knowledge transfer module 5 integrates a scene label recognition unit 50, a prototype network matching unit 51 and a MAML algorithm optimization unit 52; the scene label recognition unit 50 is used to judge the scene switch when it detects a change in any dimension of the terrain type, time, weather, traffic density, driving style, vehicle model structure and environmental dynamic scene label; the prototype network matching unit 51 is used to find the most similar prototype by matching the label of the new scene with the scene prototype pre-stored in the system after the scene switch, thereby obtaining the priori policy knowledge corresponding to the prototype; the MAML algorithm optimization unit 52 is used to use the MAML algorithm to quickly adjust the existing policy network parameters based on the similar scene prototype strategy obtained by prototype network matching, combined with N groups of sample data of the new scene, to generate a headlight control strategy that adapts to the current new scene, thereby realizing rapid migration and optimization of the strategy.

[0090] Specifically, the core function of the scene label recognition unit 50 is to monitor changes in key characteristics of the vehicle's environment in real time, determine whether a scene switch has occurred, and provide trigger conditions for subsequent policy updates. This implementation includes: Defining seven-dimensional scene labels: The system pre-sets scene labels encompassing seven dimensions, including terrain type, time, weather, road type, traffic density, lighting conditions, and surrounding object types. Each dimension has specific classifications. For example, terrain type can be categorized as urban roads, mountain roads, highways, and rural roads; time can be categorized as daytime, nighttime, and dusk; and weather can be categorized as sunny, rainy, snowy, and foggy. Real-time data monitoring and comparison: The perception module continuously collects environmental data, such as determining whether it is daytime or nighttime using ambient light sensors and time information, identifying terrain and road types using camera and radar data, and determining weather conditions using meteorological sensors or image analysis. The system compares the real-time environmental information with the current scene label. If a change in any of the seven dimensions is detected, a scene switch is determined. For example, when a vehicle moves from a flat urban road to a winding mountain road, the terrain type label changes from "urban road" to "mountain road," triggering a scene switch.

[0091] The core function of the prototype network matching unit 51 is to match the new scene's label with pre-stored scene prototypes in the system after a scene switch, identifying the most similar prototype. This allows the system to obtain the prior policy knowledge corresponding to the prototype, providing a basis for policy adjustment in the new scene. Specific implementation methods include: Prototype network construction: During the system training phase, a prototype network is constructed by learning and summarizing a large amount of vehicle light control policy data from different scenarios. This network stores information such as various typical scene prototypes and their corresponding policy parameters, feature vectors, and other information. For example, the "Rainy Mountain Road at Night" scene prototype records the approximate value ranges and adjustment rules for parameters such as vehicle light spot projection range, brightness, and shading angle that have been trained and verified to be effective in this scenario. Label matching algorithm: Upon detecting a scene switch, the system inputs the new scene's seven-dimensional label into the prototype network. Using similarity calculation algorithms (such as cosine similarity and Euclidean distance), the system calculates the similarity between the new scene label and the pre-stored scene prototype labels. The scene prototype with the highest similarity is ultimately selected as the matching result. For example, the new scene label is "night, mountain road, light rain, narrow road, low traffic volume, weak lighting, and presence of pedestrians". Through calculation, it is found that it has the highest similarity with the pre-stored "night mountain road rainy day" scene prototype, and thus the prototype is determined to be the matching result.

[0092] The core function of the MAML algorithm optimization unit 52 is to rapidly adjust the parameters of the existing policy network based on similar scenario prototype policies obtained through prototype network matching, combined with a small amount of sample data from the new scenario, and generate a headlight control policy adapted to the current scenario, thereby achieving rapid policy migration and optimization. Specific implementation methods include: Initializing the policy: Using the policy parameters corresponding to the matched scenario prototype as the initial parameters for the new scenario policy. For example, if the "Rainy Mountain Road at Night" scenario prototype is matched, the parameters such as the light spot projection range and brightness recorded in the prototype are used as the initial output parameters of the policy network for the new scenario. Sample Data Collection and Training: In the new scenario, the system uses the perception module to rapidly collect five sets of sample data. These data contain information about the environmental state of the new scenario and corresponding feedback on the effectiveness of executing the headlight control policy (e.g., whether it effectively illuminates the road and whether it causes interference to other road users). Using these five sets of sample data as the basis, the MAML algorithm performs a gradient update on the original policy network parameters. The core concept of the MAML algorithm is to find a good parameter initialization point using a small number of samples, allowing rapid adaptation to the new environment with only a small number of gradient updates. For example, in a new mountain road scenario, by learning from five sets of data, the MAML algorithm may adjust the parameters of the light spot projection range to make it more in line with the lighting needs of the mountain road bend, while ensuring that energy consumption and the impact on pedestrians are within a reasonable range. Generate a new strategy: After optimization and adjustment by the MAML algorithm, the parameters of the headlight control strategy that adapt to the current new scenario are obtained. The strategy network outputs a new headlight adjustment strategy based on these new parameters, such as the adjusted light spot projection range, brightness, and shielding angle, thereby achieving rapid migration and effective application of the headlight control strategy between different scenarios. That is, when the scene switches, based on the existing strategies and parameters, combined with the new scene information, rapid optimization is performed, so that the strategy network can continuously output effective headlight control strategies that adapt to different scenarios.

[0093] In some feasible embodiments, when the system detects that a pedestrian is looking at or entering a warning zone, it can dynamically mask part of the headlights at the pixel level within 0.1 seconds to reduce interference and keep the lighting effects in other directions unchanged; virtual guide arrows, warning icons or interactive text can be projected during the interactive prompt process.

[0094] In the above implementation, multi-sensor fusion and deep learning algorithms enable accurate perception of complex and changing driving environments, enabling headlight control strategies to adapt to varying road conditions, weather conditions, and traffic scenarios, significantly improving driving safety. The Transformer architecture extracts human-vehicle interaction features, enabling effective interaction with pedestrians and other road users through headlight projection, reducing the probability of traffic accidents. Headlight adjustment strategies are optimized based on energy efficiency assessments, reducing energy consumption and improving energy efficiency while meeting lighting needs. Leveraging a knowledge transfer module, the system can rapidly adjust strategy parameters when switching between scenarios, avoiding retraining and improving system response speed and generalization capabilities.

[0095] It is worth noting that all modules involved in this embodiment are logical units. In actual applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovations of this invention, this embodiment does not include units that are not closely related to solving the technical problems proposed by this invention. However, this does not mean that other units do not exist in this embodiment.

[0096] Example 2

[0097] See also Figure 5 , this embodiment provides a flow chart of a scene-adaptive projection vehicle lighting system control method based on deep reinforcement learning.

[0098] As an example, the method is applied to the scene-adaptive projection lamp system based on deep reinforcement learning described in Example 1, and the control method includes:

[0099] S1. Collect environmental status data through the multimodal perception module and perform preliminary processing to form an environmental data information flow.

[0100] S2. Based on the graph neural network, LSTM network and Transformer structure, a unified environmental feature vector is generated for the environmental data information flow.

[0101] S3. Receive the environmental feature vector through the strategy network module, and generate a headlight adjustment strategy through multi-layer neural network calculation, including: outputting the light spot projection range, brightness, and shielding angle.

[0102] S4. Evaluate the results obtained by executing the headlight adjustment strategy based on the headlights through the value network module, and optimize the strategy parameters of the strategy network based on the PPO algorithm.

[0103] S5. Control the light spot projection range, brightness and shielding angle of the headlights according to the headlight adjustment strategy output by the strategy network.

[0104] It is not difficult to find that this embodiment is a method example corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the first embodiment.

[0105] Example 3

[0106] An embodiment of the present invention further provides a storage medium storing a method for controlling a scene-adaptive projection lighting system based on deep reinforcement learning. When executed by a processor, the method implements the steps of the method described above. Because this storage medium incorporates all the technical solutions of all the aforementioned embodiments, it possesses at least all the beneficial effects of the technical solutions of the aforementioned embodiments, and therefore will not be further elaborated upon here.

[0107] Example 4

[0108] See also Figure 6 An embodiment of the present invention also provides an electronic device, comprising: a memory and a processor; the memory stores at least one program instruction; the processor loads and executes the at least one program instruction to implement the scene-adaptive projection vehicle light system control method based on deep reinforcement learning provided in Example 2.

[0109] The memory 702 and processor 701 are connected using a bus. The bus can include any number of interconnected buses and bridges, connecting various circuits of one or more processors 701 and memory 702. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are all well known in the art and, therefore, are not described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor 701 is transmitted over a wireless medium via an antenna. Furthermore, the antenna receives data and transmits it to the processor 701.

[0110] The processor 701 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. The memory 702 can be used to store data used by the processor 701 when performing operations.

[0111] The above is only an embodiment of the present invention. Common knowledge such as the known specific structures and characteristics in the scheme is not described in detail here. Ordinary technicians in the field are aware of all common technical knowledge in the technical field of the invention before the application date or priority date, can obtain all existing technologies in the field, and have the ability to apply conventional experimental means before that date. Ordinary technicians in the field can improve and implement this scheme in combination with their own abilities under the inspiration given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several variations and improvements can be made, which should also be regarded as the scope of protection of the present invention. These will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A scene-adaptive projection lighting system based on deep reinforcement learning, characterized in that: The projection lamp system includes: a multimodal perception module, a feature fusion module, a strategy generation module and an execution module; The multimodal perception module is used to collect environmental status data and perform preliminary processing to form an environmental data information flow; The feature fusion module integrates a graph neural network, an LSTM network, and a Transformer structure to generate a unified environmental feature vector for the environmental data information flow; The strategy generation module integrates a strategy network module and a value network module; The strategy network module is used to receive the environmental feature vector and generate a headlight adjustment strategy through a multi-layer neural network calculation, including: outputting the light spot projection range, brightness and shielding angle; The value network module is used to evaluate the results obtained by executing the headlight adjustment strategy based on the headlights, and optimize the strategy parameters of the strategy network based on the PPO algorithm; The execution module is used to control the light spot projection range, brightness and shielding angle of the headlight according to the headlight adjustment strategy output by the strategy network.

2. The scene-adaptive projection lighting system based on deep reinforcement learning according to claim 1 is characterized in that: The projection vehicle lamp system further includes a knowledge migration module, which is used to update strategy parameters through scene label recognition, prototype network matching and MAML algorithm optimization when the scene switches.

3. The scene-adaptive projection lighting system based on deep reinforcement learning according to claim 2 is characterized in that: The knowledge transfer module integrates a scene label recognition unit, a prototype network matching unit and a MAML algorithm optimization unit; The scene tag recognition unit is used to determine the scene switching when a change is detected in any dimension of the terrain type, time, weather, traffic density, driving style, vehicle structure and environmental dynamic scene tags; The prototype network matching unit is used to find the most similar prototype by matching the label of the new scene with the scene prototypes pre-stored in the system after the scene is switched, thereby obtaining the prior strategy knowledge corresponding to the prototype; The MAML algorithm optimization unit is used to quickly adjust the existing strategy network parameters using the MAML algorithm based on the similar scenario prototype strategy obtained by prototype network matching, combined with N groups of sample data of the new scenario, to generate a headlight control strategy that adapts to the current new scenario, thereby achieving rapid migration and optimization of the strategy.

4. The scene-adaptive projection lighting system based on deep reinforcement learning according to claim 1, characterized in that: The multimodal perception module integrates a device collaborative acquisition unit and a data processing unit; the device collaborative acquisition unit integrates one or a combination of a forward camera, a millimeter wave radar, a ToF depth camera and an ambient light sensor; The forward-facing camera is used to capture road images and identify lane lines, road conditions, and objects ahead; The millimeter wave radar is used to emit electromagnetic waves to detect the distance, speed and direction of obstacles; The ToF depth camera is used to obtain scene depth information by measuring the time of flight of light and locate the three-dimensional position of the object; The ambient light sensor is used to monitor ambient brightness and / or weather conditions in real time; The data processing unit is used to perform preliminary filtering and noise reduction on the data collected by the equipment collaborative collection unit, eliminate obvious errors or interference data, and form a preliminary environmental data information flow.

5. The scene-adaptive projection lighting system based on deep reinforcement learning according to claim 1, characterized in that: The feature fusion module integrates a spatial topology modeling unit, a time series processing unit, an interactive feature extraction unit, and a feature fusion and dimensionality reduction unit; The spatial topology modeling unit is used to model the spatial topological relationship between the vehicle and the obstacle based on a graph neural network; The time series processing unit is used to process time series driving behavior based on the LSTM network; The interaction feature extraction unit is used to extract human-vehicle interaction features based on the Transformer structure; The feature fusion and dimension reduction unit is used to fuse the features extracted by the spatial topology modeling unit, the time series processing unit and the interactive feature extraction unit to generate a unified environment feature vector.

6. The scene-adaptive projection lighting system based on deep reinforcement learning according to claim 5 is characterized in that: The spatial topology modeling unit is used to treat the vehicle and each obstacle as nodes of a graph based on GNN, construct edges according to their relative positions and distances, and form a spatial topology graph to represent the spatial relationship between the vehicle and the surrounding objects; The time series processing unit is used to receive time series data from the multimodal perception module based on the LSTM network, capture the changing trends and patterns of driving behavior over time, and extract features in the time dimension; The interaction feature extraction unit is used to extract features of pedestrian postures and possible interaction behaviors based on the Transformer structure, focusing on the interaction pattern between pedestrians and vehicles; The feature fusion and dimensionality reduction unit is used to fuse the features extracted by GNN, LSTM and Transformer, and map the fused high-dimensional features into a 128-dimensional unified environment feature vector through linear transformation operations.

7. The scene-adaptive projection lighting system based on deep reinforcement learning according to claim 1, characterized in that: The strategy network module is integrated with a strategy network calculation unit and an action space constraint unit; The strategy network calculation unit is used to output the light spot projection range ΔR, brightness ΔL and shielding angle ΔS based on the received environment feature vector through a multi-layer neural network calculation based on a preset strategy network; The action space constraint unit is used to limit the action value output by the strategy network calculation unit to ensure that the headlight adjustment is within a safe and reasonable range.

8. The scene-adaptive projection lighting system based on deep reinforcement learning according to claim 7, characterized in that: The value network module integrates a reward function calculation unit, a value network evaluation unit and a PPO algorithm optimization unit; The reward function calculation unit is used to calculate the reward function value based on the result obtained by executing the current headlight adjustment strategy according to the headlight and based on the preset reward function calculation method; The value network evaluation unit is used to evaluate the pros and cons of the current strategy based on the reward function according to the pre-trained value network to obtain a value estimate; The PPO algorithm optimization unit is used to update the parameters of the policy network based on the PPO algorithm using the evaluation results of the value network and the reward function, so as to optimize the policy in a direction that can obtain higher rewards.

9. The scene-adaptive projection lighting system based on deep reinforcement learning according to claim 8, characterized in that: The reward function calculation method includes: R = 0.4·SSIM+0.2·Obstacle avoidance success rate+0.2·Energy efficiency+0.2·Pedestrian gaze feedback; Where SSIM represents the structural similarity index; The PPO algorithm optimization unit integrates an advantage function calculation unit, an objective function design unit and a gradient descent update unit; The advantage function calculation unit is programmed with an advantage function calculation formula, including: A(s,a)=R+γV(s ' )-V(s); The objective function calculation formula is burned into the objective function design unit, including: Where R is the reward function value, γ is the discount factor, V(s) is the value estimate in the current state, and V(s ' ) is the value estimate for the next state, π θ (a|s) represents the probability of selecting action a in state s under the new policy parameters θ, Expressed as the old policy parameter θ old Under this condition, the probability of selecting action a in state s, ∈ is the clipping parameter; The gradient descent update unit is used to optimize the policy network parameters through back propagation to maximize the objective function.

10. A method for controlling a scene-adaptive projection lighting system based on deep reinforcement learning, the method being applied to the scene-adaptive projection lighting system based on deep reinforcement learning according to any one of claims 1 to 9, characterized in that: The control method includes: Collect environmental status data through the multimodal perception module and perform preliminary processing to form an environmental data information flow; Based on graph neural networks, LSTM networks and Transformer structures, a unified environmental feature vector is generated for the environmental data information flow; The environmental feature vector is received by the strategy network module, and a headlight adjustment strategy is generated by a multi-layer neural network calculation, including: outputting the light spot projection range, brightness and shielding angle; evaluating, by a value network module, results obtained from executing the headlight adjustment strategy based on the headlights, and optimizing the strategy parameters of the strategy network based on a PPO algorithm; According to the headlight adjustment strategy output by the strategy network, the light spot projection range, brightness and shielding angle of the headlight are controlled.

Images