Artificial intelligence electric vehicle safe driving image recognition device and system

Abstract

The invention relates to the technical field of electric vehicle safe driving, and discloses an artificial intelligence electric vehicle safe driving image recognition device and system. A multi-source environment sensing module of the system obtains real-time road image data collected by a vehicle-mounted camera, and synchronously receives running speed, steering angle and obstacle distance information transmitted by a vehicle state sensor; the dynamic state construction module extracts traffic element space distribution characteristics based on real-time road image data, and generates a multi-dimensional driving state vector in combination with vehicle state information; and the feature fusion processing module inputs the multi-dimensional driving state vector and the historical hidden feature vector into a spatio-temporal evolution network, and outputs an updated hidden feature vector and an initial driving decision vector through a continuous feature evolution unit. The system can comprehensively perceive the driving environment, deeply fuse multi-source data, dynamically update features and generate a reasonable driving decision, improves the driving safety of the electric vehicle, and is suitable for various electric vehicle safe driving auxiliary scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electric vehicle safe driving, in particular to an artificial intelligence electric vehicle safe driving image recognition device and system. BACKGROUND

[0002] With the rapid development of new energy vehicle industry, electric vehicles have become an important part of the transportation field, and safe driving has always been one of the core directions of electric vehicle technology research and development. At present, in order to realize the function of safe driving assistance, various driving assistance systems based on image recognition are gradually applied to electric vehicles. These systems mostly rely on vehicle-mounted cameras to collect road image data, and through image recognition algorithms to extract traffic signal lights, pedestrians, other vehicles and other traffic element information, and then provide early warning or auxiliary decision support for the driver.

[0003] The existing electric vehicle safe driving system based on image recognition has obvious limitations. Most systems only rely on a single vehicle-mounted camera as the source of environmental perception. This single-source perception method is easily affected by external environmental factors. For example, in complex weather or environmental conditions such as heavy rain, heavy fog, and insufficient light at night, the image quality collected by the camera will decrease significantly, resulting in reduced accuracy of traffic element recognition, and thus affecting the reliability of the system's judgment of the road environment. At the same time, relying only on image data cannot fully obtain the dynamic relationship between the vehicle's own driving state and the surrounding obstacles, making it difficult to accurately grasp the potential risks in the driving process.

[0004] The existing system has deficiencies in data processing and fusion. Although some systems receive information such as driving speed and steering angle transmitted by vehicle state sensors, they fail to deeply integrate these vehicle state data with road image data. In most cases, the system only processes image data and vehicle state data separately, and then directly superimposes the processing results for decision-making. This processing method cannot build a multi-dimensional driving state model that reflects the overall driving scene, making the system's understanding of the driving scene one-sided. In addition, in the decision-making process, the existing system often only focuses on real-time data, ignoring the driving scene evolution rules contained in historical driving data, resulting in a lack of continuity and foresight in the generated driving decisions. For example, when the vehicle is driving on a continuous curve or a complex intersection, the system cannot predict possible road condition changes in advance based on historical driving state and road environment change trends, thus affecting the timeliness and effectiveness of the auxiliary decision-making.

[0005] In terms of feature processing, the existing system commonly uses a simple feature extraction and updating method, mostly adopting a static feature extraction method, which cannot capture the dynamic correlation of traffic elements and vehicle state changes over time in the driving scene. This static processing method is difficult to adapt to the dynamic changes of the road environment during driving. When sudden conditions occur on the road, such as pedestrians suddenly crossing the road, other vehicles changing lanes in an emergency, etc., the system cannot quickly update the feature information and generate a reasonable response decision, which may lead to delayed warning or decision errors, and cannot fully guarantee the safety of electric vehicle driving. SUMMARY

[0006] The purpose of the present application is to provide an artificial intelligence electric vehicle safe driving image recognition system to solve the problems raised in the above background.

[0007] To achieve the above purpose, the present application provides an artificial intelligence electric vehicle safe driving image recognition system, which comprises: A multi-source environment perception module acquires real-time road image data collected by a vehicle-mounted camera and synchronously receives driving speed, steering angle and obstacle distance information transmitted by a vehicle state sensor; A dynamic state construction module extracts traffic element spatial distribution features based on the real-time road image data, and generates a multi-dimensional driving state vector in combination with the driving speed, steering angle and obstacle distance information; A feature fusion processing module inputs the multi-dimensional driving state vector and historical hidden feature vector into a spatio-temporal evolution network, and outputs an updated hidden feature vector and an initial driving decision vector through a continuous feature evolution unit.

[0008] Preferably, the spatio-temporal evolution network comprises: A hierarchical feature extraction unit analyzes road topology features and dynamic obstacle trajectory features in the multi-dimensional driving state vector; A state evolution unit uses a time series feature transfer mechanism to model the spatio-temporal correlation of the road topology features, dynamic obstacle trajectory features and historical hidden feature vector; A decision output unit constrains the numerical range of the initial driving decision vector through a nonlinear mapping function.

[0009] Preferably, the feature fusion processing module further comprises: A dual-channel evaluation network receives the multi-dimensional driving state vector and initial driving decision vector, and generates a first safety evaluation value and a second safety evaluation value; A target decision network generates a target driving decision vector according to the multi-dimensional driving state vector and updated hidden feature vector at a subsequent time; The target evaluation network generates a first target evaluation value and a second target evaluation value based on the multi-dimensional driving state vector at the subsequent moment and the target driving decision vector.

[0010] Preferably, the system comprises: The risk quantification module calculates a scene risk evaluation value according to the initial driving decision vector and the multi-dimensional driving state vector at the subsequent moment. The experience storage module groups the multi-dimensional driving state vector, the historical hidden feature vector, the initial driving decision vector, the scene risk evaluation value, the multi-dimensional driving state vector at the subsequent moment and the updated hidden feature vector into a training data set.

[0011] Preferably, the system comprises: an evaluation network updating module configured to calculate a decision optimization target value according to the first target evaluation value, the second target evaluation value and the scene risk evaluation value, and update the evaluation network parameters based on the first safety evaluation value, the second safety evaluation value and the decision optimization target value.

[0012] Preferably, the system comprises: a decision network updating module configured to adjust the weight parameters of the spatio-temporal evolution network by using a gradient back propagation algorithm, and trigger parameter updating according to a preset updating period, so that the decision optimization target value output by the double-channel evaluation network is maximized.

[0013] Preferably, the scene risk evaluation value is calculated by the following logic: When the distance of the obstacle is lower than a safety threshold, a collision risk evaluation value is activated; when the traffic signal state is at a dangerous level, a signal violation evaluation value is activated; and for the rest of the scenarios, a weighted evaluation value of a path deviation coefficient and a comprehensive risk coefficient is calculated; wherein the comprehensive risk coefficient is the cumulative value of risk factors in each direction.

[0014] Preferably, the decision optimization target value is calculated by applying the scene risk evaluation value and a decay factor to the minimum value of the first target evaluation value and the second target evaluation value and then summing the results.

[0015] Preferably, the evaluation network updating module calculates the loss amount by the following method: randomly sampling a data set in the training data set, calculating a first mean square error amount of the first safety evaluation value and the decision optimization target value, and a second mean square error amount of the second safety evaluation value and the decision optimization target value.

[0016] Preferably, the system comprises: a driving execution module configured to convert the initial driving decision vector into a vehicle control instruction, wherein the vehicle control instruction comprises a longitudinal acceleration control amount and a lateral steering angle control amount, and drive a vehicle execution mechanism to complete a corresponding action.

[0017] Compared with the prior art, the present application has the following advantages: The artificial intelligence electric vehicle safe driving image recognition system effectively makes up for the deficiencies of existing electric vehicle safe driving systems in environmental perception, data processing and decision generation through the synergistic effect of the multi-source environmental perception module, dynamic state construction module and feature fusion processing module, and provides more comprehensive and reliable technical support for electric vehicle safe driving.

[0018] The multi-source environmental perception module simultaneously acquires real-time road image data collected by a vehicle-mounted camera and driving speed, steering angle and obstacle distance information transmitted by a vehicle state sensor, breaking the limitation of existing systems that rely on single image data for environmental perception. By synchronously collecting two types of data with different dimensions, the system can comprehensively acquire key information in the driving process from two levels of road environment and vehicle state. Road image data can intuitively reflect the distribution of traffic elements such as traffic lights, lane lines, pedestrians and other vehicles on the road, while driving speed and steering angle can reflect the current driving dynamics of the vehicle, and obstacle distance information can accurately reflect the spatial position relationship between the vehicle and the surrounding obstacles. The synchronous acquisition of the two types of data enables the system to comprehensively perceive the driving scene in complex environments, even when the quality of image data is affected by the environment. Vehicle state data and obstacle distance information can provide supplementary reference for the system, reducing the impact of insufficient single data source on the reliability of environmental perception, and making the system's judgment of the driving environment more comprehensive and objective.

[0019] The dynamic state construction module extracts traffic element spatial distribution features based on real-time road image data and generates a multi-dimensional driving state vector combining vehicle state information, realizing the deep integration of road environment data and vehicle state data. Traffic element spatial distribution features can clearly present the position and distribution of various traffic participants on the road, while driving speed, steering angle and obstacle distance information reflect the vehicle's own driving state and interaction with the surrounding environment. The fusion of the two types of features to generate a multi-dimensional driving state vector can construct a dynamic model that comprehensively reflects the driving scene, enabling the system not only to understand the static information of the road environment, but also to grasp the dynamic relationship between the vehicle and the environment, thereby more accurately understanding the overall situation of the current driving scene and providing a more comprehensive scene awareness basis for subsequent decision generation.

[0020] The feature fusion processing module inputs the multi-dimensional driving state vector and the historical hidden feature vector into the space-time evolution network, and outputs the updated hidden feature vector and the initial driving decision vector through the continuous feature evolution unit, which fully utilizes the value of the historical driving data and realizes the dynamic update of the features. The historical hidden feature vector contains the evolution law and feature change trend of the driving scene in the past period of time. Combining the current multi-dimensional driving state vector can enable the system to grasp the change process of the driving scene from the time dimension and avoid the one-sidedness of the decision caused by relying only on real-time data. The space-time evolution network can perform deep fusion of the two types of vectors in the space-time dimension, and mine the correlation information between the real-time data and the historical data. The continuous feature evolution unit can realize the dynamic update of the features, so as to ensure that the system can timely capture the change of the traffic elements and the vehicle state in the driving scene. This processing method makes the generated initial driving decision vector consider not only the actual situation of the current driving scene, but also the evolution law of the historical scene, which can more accurately predict the possible road condition changes in the driving process, make the driving decision more continuous and forward-looking, and thus better cope with the complex and changeable road environment and improve the safety of the electric vehicle driving process.

[0021] The modules of the whole system cooperate with each other to form a complete technical chain from environment perception, data processing to decision generation. The design of each module is optimized for the shortcomings of the existing system, the data flow transmission between each link is smooth, and the information processing efficiency is high. Compared with the existing system, the system has significant improvement in the comprehensiveness of environment perception, the depth of data fusion, the dynamic nature of feature processing, and the rationality of decision generation, and can better adapt to the driving needs in different weather and road conditions, provide stronger technical support for safe driving of electric vehicles, and help to promote the further development and application of electric vehicle safe driving assistance technology. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 The timing diagram of the artificial intelligence electric vehicle safe driving image recognition system described in the present application; Figure 2 The workflow diagram of the space-time evolution network; Figure 3 The workflow diagram of the network update module; Figure 4 The workflow diagram of the decision network update module. DETAILED DESCRIPTION

[0023] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work are within the protection scope of the present application.

[0024] With reference to Figure 1 The present application provides an artificial intelligence electric vehicle safe driving image recognition device and system. The system comprises: The multi-source environment perception module acquires real-time road image data collected by a vehicle-mounted camera, and receives driving speed, steering angle and obstacle distance information transmitted by a vehicle state sensor. The dynamic state construction module extracts the spatial distribution characteristics of traffic elements from the real-time road image data, and generates a multi-dimensional driving state vector in combination with the vehicle state information. The feature fusion processing module inputs the multi-dimensional driving state vector and the historical hidden feature vector into the spatio-temporal evolution network, and outputs the updated hidden feature vector and the initial driving decision vector through the continuous feature evolution unit. The system realizes dynamic modeling and decision optimization of the driving state through the collaborative work of multiple modules, and finally generates driving control instructions that adapt to complex road environments.

[0025] Embodiment 1: With reference to Figure 2 The hierarchical feature extraction unit of the spatio-temporal evolution network performs structured analysis on the multi-dimensional driving state vector. The vector contains road image features captured by the vehicle-mounted camera and real-time parameters transmitted by the vehicle sensor. The road topology feature extraction focuses on the geometric distribution of lane lines, the spatial position of traffic signs and the change of road curvature. For example, when the vehicle is driving on a curve, the unit identifies the curvature radius of the lane boundary line through an image segmentation algorithm, and locates the pixel coordinates of the curve warning sign. The dynamic obstacle trajectory feature analysis is aimed at the displacement mode of moving targets: the transverse movement speed of adjacent vehicles is calculated through continuous frame image difference, and the obstacle motion trajectory model is constructed in combination with the millimeter wave radar ranging data. If the front truck shows a left deviation trend, the unit will output the displacement direction angle change rate and the relative distance derivative with the vehicle.

[0026] The historical hidden feature vector stores the compressed information of driving states at previous time steps, such as the feature encoding of the vehicle's pedestrian avoidance behavior in the last 3 seconds. The spatio-temporal correlation modeling is achieved through a gated recurrent structure: the curvature data of the bend in the road topology feature and the truck offset data in the dynamic obstacle trajectory are weighted and fused, while establishing a correlation with the "pedestrian avoidance" state recorded in the previous hidden features. The spatial attention mechanism dynamically adjusts the feature weights, and when the truck offset increases, its trajectory feature weight increases to 0.85, while the road curvature feature weight decreases to 0.15. The residual connection structure is used in the time series transmission process to combine the longitudinal speed control mode in the previous hidden features with the current bend scene, generating updated features containing the "slow down to enter the bend" tendency.

[0027] The decision output unit maps the updated feature vector into the initial driving decision vector, which contains the longitudinal acceleration suggestion value [-5, 2] m / s² and the lateral steering angle suggestion value [-30°, 30°]. The nonlinear mapping function uses the hyperbolic tangent activation layer to constrain the original output to a reasonable range. For example, when the original calculation suggests steering -35°, the output after the tanh function compression is -28.7°. At the same time, the output layer bias term is set to calibrate the steering angle reference value to -0.5° in the straight driving state of the vehicle to compensate for sensor errors.

[0028] The first safety assessment channel builds a collision risk model: according to the relative speed 12 m / s and distance 38 m between the vehicle and the truck, the collision time threshold is calculated to be 3.17 seconds; combined with the steering decision angle -28.7°, the distance change between the two vehicles after 3 seconds is predicted. The second safety assessment channel performs path deviation analysis: based on high-precision map data, the lateral deviation of the vehicle trajectory from the planned path under the current steering decision is calculated. When the initial decision causes the predicted trajectory to deviate from the reference line by 1.2 meters, a path deviation coefficient of 0.42 is generated. The output results of the two channels are normalized to the [0, 1] interval, and in this example, the first safety assessment value is 0.35 (low collision risk), and the second safety assessment value is 0.68 (medium level path deviation).

[0029] When the vehicle executes the initial steering decision, the updated multi-dimensional state vector contains elements such as the truck has completed the lane change, a new motorcycle appears on the right, etc. Combined with the latest output hidden feature vector of the spatio-temporal evolution network (containing the "middle of the curve" state code), the target driving decision vector is generated: it is recommended to maintain a steering angle of -5° and reduce the acceleration to -1.2 m / s². The target evaluation network conducts a safety evaluation based on this decision: the first target evaluation value calculates the collision time between the vehicle and the motorcycle as 4.2 seconds (evaluation value 0.72), and the second target evaluation value calculates the path deviation as 0.3 meters (evaluation value 0.85). This target decision will be fed back to the evaluation network update process as an optimization benchmark. When the vehicle is traveling at 60 km / h, the system completes the entire process from image acquisition to decision output within a moving distance of 1.67 meters. Key intermediate states are preserved during feature evolution: the road topology feature matrix is stored as a 128-dimensional tensor, the obstacle trajectory feature code is encoded as a 64-dimensional vector, and the hidden feature vector dimension remains 256-dimensional to ensure the completeness of the state information. The decision vector conversion link sets an output delay detection, and when the processing time exceeds 80 milliseconds, a simplified calculation mode is automatically enabled to ensure real-time control requirements.

[0030] Example 2: see Figure 3 , the risk quantification module calculates the scene risk evaluation value based on the initial driving decision vector and the multi-dimensional driving state vector at the subsequent time. This module first detects potential risk factors in the current environment, including obstacle distance, traffic signal status, and vehicle travel path stability. When the obstacle distance is below a preset dynamic safety threshold, the collision risk evaluation value is activated. This threshold is not a fixed value, but is automatically adjusted according to the current speed of the vehicle. For example, when the vehicle is traveling at 60 km / h, the safety threshold is set to 30 meters; if the speed increases to 80 km / h, the threshold increases to 45 meters accordingly. The calculation of the collision risk evaluation value takes into account the relative speed, distance change rate, and obstacle type. For a truck that suddenly slows down in front, if its distance to the vehicle shortens from 50 meters to 28 meters while the relative speed is 20 km / h, the collision risk evaluation value increases to 0.75.

[0031] When the signal light is yellow or red, and the vehicle does not show a clear deceleration trend, the signal violation evaluation value is activated. This evaluation value is calculated in combination with the remaining time of the signal light and the vehicle stopping distance. For example, a red light is detected 15 meters away from the intersection, and the current speed is 50 kilometers per hour, which requires at least 25 meters of braking distance to completely stop, at which time the signal violation evaluation value reaches 0.9. For scenarios that do not involve collisions or signal violations, the risk quantification module calculates a weighted evaluation value of the path deviation coefficient and the comprehensive risk coefficient. The path deviation coefficient reflects the degree of deviation of the actual vehicle trajectory from the preset navigation path. In urban road environment, if the vehicle deviates from the planned lane center line by 0.8 meters due to obstacle avoidance, and the deviation lasts for more than 3 seconds, the path deviation coefficient rises to 0.6. The comprehensive risk coefficient assesses the potential threats in multiple directions, including lateral collision risk, longitudinal rear-end risk, and sudden obstacle risk. When the vehicle is driving on a highway, there is a fast approaching car on the left lane, the right lane is close to the guardrail, and there is a close following truck behind, the comprehensive risk coefficient may reach 0.7. The weights of each risk factor are dynamically adjusted according to the real-time environment, for example, in congested road sections, the weight of longitudinal rear-end risk is increased to 0.5, while in open road sections, the weight of lateral risk is higher.

[0032] The experience storage module uses a ring buffer structure to store the latest 1000 sets of driving state data. Each set of data contains a multi-dimensional driving state vector, a historical hidden feature vector, an initial driving decision vector, a scene risk evaluation value, a multi-dimensional driving state vector at the next time, and an updated hidden feature vector. Time stamp alignment is performed during data storage to ensure that sensor data, image features, and decision vectors at the same time are kept synchronized. For example, at time point T = 12.5 seconds, the stored data includes: the feature vector extracted from the 12.5 second road image captured by the camera, the speed and steering angle data transmitted by the vehicle CAN bus, the hidden feature vector output by the spatio-temporal evolution network, the scene risk evaluation value 0.45 calculated by the risk quantification module, and the state vector and decision vector updated at 13 seconds.

[0033] The evaluation network update module optimizes the network parameters by comparing the real-time evaluation value with the target value. The calculation of the decision optimization target value is divided into two steps: first, obtain the first target evaluation value and the second target evaluation value from the target evaluation network, and select the smaller one as the reference. For example, if the first target evaluation value is 0.72 and the second target evaluation value is 0.85, the reference value is 0.72. Then multiply the scene risk evaluation value by the decay factor, which decays exponentially with time step. For the risk evaluation value 0.45 at the current time, if the decay factor is 0.8, the adjusted risk contribution value is 0.36. The final decision optimization target value is the sum of the reference evaluation value and the adjusted risk value, which is 1.08 in this example.

[0034] In the parameter updating phase, the evaluation network updating module randomly selects 32 groups of data from the experience storage module as the training batch. For each group of data, the squared error of the first safety evaluation value and the decision optimization target value is calculated, as well as the squared error of the second safety evaluation value and the decision optimization target value. Assuming that the first safety evaluation value of a certain group of data is 0.65, the error between it and the target value 1.08 is 0.43, and the squared error is 0.1849; the second safety evaluation value is 0.78, and the squared error is 0.09. The total loss of batch training is the average of all data errors, and the weight parameters of the double-channel evaluation network are adjusted by the gradient descent algorithm. Each time the network parameters are adjusted in the direction of error reduction, so that the evaluation result gradually approaches the optimization target.

[0035] In a typical urban driving scenario, the vehicle completes a complete risk evaluation and network optimization cycle every 1.4 meters of travel (at 50 km / h). The evaluation value output by the risk quantification module is displayed in real time on the head-up display in the cockpit in the form of a 0-1 number for the driver to refer to. When the evaluation value exceeds 0.7, the system will trigger an audible and visual warning signal. The data retention period of the experience storage module is 30 minutes, and old data exceeding this time limit will be overwritten by new data to ensure that the training data set always reflects the characteristics of the recent driving environment. The parameter adjustment amplitude of the evaluation network updating module is constrained by the momentum factor to avoid dramatic fluctuations in network performance caused by single abnormal data.

[0036] Embodiment 3: refer to Figure 4 The decision network updating module adjusts the weight parameters of the spatio-temporal evolution network using the gradient backpropagation algorithm. This module randomly samples batches of data from the experience storage module during training, with each batch containing 32 driving state records. Each record contains a multi-dimensional driving state vector, a historical hidden feature vector, an initial driving decision vector, and a corresponding scene risk evaluation value. During backpropagation, the difference between the initial driving decision vector output by the spatio-temporal evolution network and the target driving decision vector is first calculated. The target driving decision vector is generated by the target decision network, and its value represents the optimization decision suggestion under the current environment. For example, when the initial decision suggestion is to turn -15° and the target decision suggestion is to turn -10°, the difference between the two in the steering angle dimension is 5°. The loss function uses a weighted mean squared error form, which is expressed as: where: represents the difference between the initial decision and the target decision in the longitudinal acceleration dimension in the th group of data, with the unit being m / s²; represents the difference in the steering angle dimension, with the unit being degrees; and are the weight coefficients of acceleration and steering angle, respectively, with default values of 0.7 and 0.3; L2 is the regularization coefficient, set to 0.01; represents the set of all trainable weight matrices in the spatio-temporal evolution network; is the batch size, set to 32. The loss function design makes the contribution of steering angle error to the total loss relatively small, reflecting the larger tolerance allowed for vehicle lateral control.

[0037] The error signal is first passed to the decision output unit, which computes the local gradient of the non-linear mapping function output layer. For neurons with hyperbolic tangent activation, the gradient value is , is the weighted input of the neuron. For example, when the steering angle output neuron , its gradient is . These gradient values are propagated layer by layer through the chain rule to the state evolution unit and the hierarchical feature extraction unit. In the state evolution unit, the gradient calculation of the gated recurrent structure involves the memory cell state transmission between time steps. Assuming that the memory cell state at the current time step contains 60% of the information of the previous time step state , the error signal needs to be allocated to the historical state in proportion. The parameter update uses the stochastic gradient descent algorithm with momentum. At each update, the weight adjustment amount depends not only on the current gradient , but also accumulates the influence of the direction of the previous update: where: is the learning rate, 0.001; is the momentum coefficient, 0.9; is the last weight update amount. This mechanism allows for faster convergence in flat error surface areas and suppresses oscillation in steep areas. For example, when the gradient direction of a certain convolution kernel weight is consistent in three consecutive iterations, its actual update amount will gradually increase, accelerating the approach to the optimal value.

[0038] The parameter update cycle of the spatio-temporal evolution network is set to trigger once every 200 new data processing. During the interval between two updates, the system continuously collects new driving state data and stores it in the experience storage module. During the update, 5 batches (160 groups) of data are randomly selected from the storage module for training. Each batch iterates the gradient calculation and accumulates the update amount, and finally applies the average update amount to adjust the network parameters. This batch accumulation mechanism can smooth the noise effect of a single batch of data. For example, a batch may contain rare extreme driving scenarios (such as emergency animal avoidance), but through multi-batch averaging, its impact on the overall model can be reduced.

[0039] The convolution kernel weight update of the hierarchical feature extraction unit needs to consider the receptive field alignment. For the 3x3 convolution kernel that analyzes the road topology features, the gradient of the 9 weight parameters reflects the importance of features at different spatial positions. When processing images of curved roads, the weights located at the edges of the convolution kernel may obtain larger gradients because they correspond to the key regions of the curvature change of the lane lines. In the LSTM unit of the dynamic obstacle trajectory feature, the weight update of the input gate and the forget gate follows the bidirectional propagation rule of the recurrent neural network. For example, when processing the moving data of an obstacle for 5 consecutive frames, the input gate weight adjustment amount of a certain LSTM unit needs to consider the time sequence association between the current frame feature and the previous and next frame states.

[0040] The fully connected layer of the decision output unit adopts a hierarchical learning rate strategy. The neurons in the output layer that control the longitudinal acceleration use a baseline learning rate of 0.001, while the learning rate of the steering angle control neurons is set to 0.0005, reflecting the need for more precise parameter adjustment for lateral control. The learning rate of the bias term is uniformly set to 10 times the learning rate of the corresponding layer, for example, the learning rate of the acceleration control bias term is 0.01, so that it can quickly adapt to the basic control needs of different driving scenarios. When the L2 norm of the weight update is detected to exceed the threshold value 1.0, all gradients are scaled proportionally to ensure that the total update amount does not exceed the threshold value. This mechanism prevents individual abnormal data from causing drastic fluctuations in parameters, for example, when a group of data contains sensor outliers (such as sudden zero distance measurements), the calculated gradient may be abnormally large, and the clipping operation can maintain the stability of the training.

[0041] The network weight parameters are periodically serialized into binary files and saved, retaining the parameter snapshots of the last 10 training cycles. When newly collected data causes the validation set loss to rise for 3 consecutive times, the system automatically rolls back to the previous version of the parameters to prevent model performance degradation. Each snapshot file contains a time range label for the training data, for example, "20230825_1430-1530" indicates that the model is trained based on data from this time period, which facilitates tracing the relationship between performance changes and driving environments.

[0042] For the output of the road topology feature extraction layer, the system maps the 128-dimensional feature tensor back to the image space to display the spatial distribution of the convolution kernel response. When the vehicle approaches an intersection, the heat map highlights the traffic signal and stop line areas, reflecting the high activation intensity of these elements in the feature space. The LSTM unit state of the dynamic obstacle trajectory feature is converted into a trajectory prediction map, showing the network's inference results for the future positions of surrounding vehicles. These visualized data assist engineers in understanding the basis for network decision-making.

[0043] In the example 4, the first target evaluation value and the second target evaluation value generated by the target evaluation network are compared first, and the value with lower value is selected as the reference evaluation value. For example, in the highway following scenario, the first target evaluation value (collision risk evaluation) is 0.63, and the second target evaluation value (path deviation evaluation) is 0.58, and the reference evaluation value is determined as 0.58. The scene risk evaluation value needs to be processed by a decay factor, and the factor is exponentially decayed with time step. If the current risk evaluation value is 0.42, and the interval between the decision time and the target time is 3 time steps (100 milliseconds per step), the decay factor is 0.794, and the adjusted risk contribution value is 0.42*0.794≈0.333. The final decision optimization target value is the sum of the reference evaluation value 0.58 and the adjusted risk value 0.333, that is, 0.913.

[0044] From the ring buffer of the experience storage module, 128 groups of data are randomly selected to form a training set, and each group of data contains the first safety evaluation value and the second safety evaluation value output by the double-channel evaluation network and the corresponding decision optimization target value. For each group of data, the square error of the two evaluation channels is calculated: the square of the difference between the first safety evaluation value and the decision optimization target value is the first mean square error, and the square of the difference between the second safety evaluation value and the decision optimization target value is the second mean square error. Assuming that in a certain group of data, the first safety evaluation value is 0.71, and the decision optimization target value is 0.913, the first mean square error is (0.71-0.913)²=0.041209; the second safety evaluation value is 0.68, and the second mean square error is (0.68-0.913)²=0.054289.

[0045] The weight coefficient is dynamically configured according to the driving scene: in the highway scene, the collision risk weight is set to 0.6, and the path deviation weight is set to 0.4; in the urban road scene, it is adjusted to 0.4 and 0.6. Continuing the previous example, if the current is a highway scene, the total loss amount=0.6*0.041209+0.4*0.054289=0.046385. The total loss amount of 128 groups of data is the arithmetic average of the partial loss of each group. The partial derivative of the total loss amount with respect to the parameters of the double-channel evaluation network is calculated, and the network weight is adjusted in the opposite direction of the gradient. Taking a certain hidden layer neuron of the collision risk evaluation channel as an example, its current weight value is 0.12, the gradient of the loss function with respect to the weight is-0.032, and the learning rate is set to 0.001, then the updated weight=0.12-0.001*(-0.032)=0.120032. The update step is constrained by the momentum factor, and if the previous update direction is consistent with the current gradient direction, the actual update amount will be appropriately increased.

[0046] The first three layers of the dual-channel evaluation network are shared feature extraction layers, and from the fourth layer onwards, it branches into two independent branches. During parameter updates, the gradient of the shared layer is derived from the weighted sum of the gradients of the two branches. For example, the gradient of a convolutional kernel in the shared layer is calculated as: collision risk branch gradient × 0.6 + path deviation branch gradient × 0.4. This design ensures that the basic features serve both types of safety assessments simultaneously.

[0047] When the decision optimization objective value of a set of data exceeds a preset threshold of 1.5, the data will be marked as an outlier. Outliers still participate in the loss calculation, but their gradient contribution is multiplied by a suppression coefficient of 0.2. For example, in extreme avoidance scenarios, the decision optimization objective value might reach 1.82; in this case, the gradient generated by such a sample will be weakened to avoid excessive interference to the network. Key metrics are recorded during the training process for iterative monitoring. Examples of network training data for different driving scenarios are shown in Table 1.

[0048] Table 1: Evaluation Network Training Data Sampling Table.

[0049] Scenario description First safety evaluation value Second safety evaluation value Decision optimization target value Total loss amount Highway following 0.71 0.68 0.913 0.0464 Urban intersection left turn 0.82 0.57 0.952 0.0381 Rain and fog weather straight 0.63 0.74 0.891 0.0512 Construction section lane change 0.69 0.61 0.934 0.0427 Night highway cruising 0.75 0.72 0.903 0.0298

[0050] The parameter update frequency is set to be executed every 500 sets of new data. Before the update, the data distribution of the experience storage module is analyzed. When it is detected that the proportion of a specific scenario is too low (such as less than 5% of the samples in rainy or foggy weather), the sampling weight of this type of data will be actively increased. After the update is completed, the new parameter configuration is immediately deployed to the online evaluation network, and the old parameter version is retained as a rollback backup.

[0051] The visual monitoring interface displays the learning status of the evaluation network in real time. The neuron activation patterns in the collision risk assessment channel are mapped as a heatmap. When the system processes a scenario where a vehicle ahead brakes suddenly, the heatmap highlights the area in the relative speed input dimension. The attention weight distribution in the path deviation assessment channel is displayed as a colored overlay on the lane feature map. In curve scenarios, the activation intensity of neurons related to road curvature is significantly increased. This visual data helps engineers understand the network's decision-making basis. The evaluation network update process includes a stability verification mechanism. After each parameter update, the change in loss is checked using a validation dataset. If the validation loss increases by more than 10% after three consecutive updates, the system automatically terminates the update and reverts to the previous stable version. The validation dataset is stored independently of the training data and contains 100 sets of benchmark samples covering typical scenarios, updated quarterly based on actual driving data.

[0052] The output layer of the dual-channel evaluation network incorporates numerical normalization. The original evaluation values ​​are constrained to the [0,1] interval using a sigmoid function and dynamically calibrated before output: when the median of evaluation values ​​for 20 consecutive data sets is below 0.3, the output bias is automatically adjusted to bring the median back to around 0.5. This mechanism maintains the semantic consistency of the evaluation scale and avoids benchmark drift due to environmental changes.

[0053] Training data preprocessing includes time alignment correction. Due to differences in sampling frequencies among sensors (30Hz for the camera, 20Hz for the radar), cubic spline interpolation is used to achieve time synchronization when generating the multi-dimensional driving state vector. For example, at t=12.345 seconds, the radar data is obtained by interpolating the measurements from the preceding and following frames (12.30s and 12.35s) to ensure accurate matching with the image data. Historical hidden feature vectors are also interpolated to maintain temporal consistency with the current state.

[0054] The network update log records detailed parameter changes, and a change report is generated for each update, including the adjusted weight parameter percentages, the maximum single parameter change magnitude, and changes in the correlation between channels. For example, in one update, 68% of the weights of the collision risk channel were adjusted, with a maximum change magnitude of 0.021; the output correlation coefficient between the two evaluation channels decreased from 0.38 to 0.31, reflecting that the network has learned more differentiated evaluation characteristics.

[0055] Example 5: The driving execution module receives the initial driving decision vector output by the feature fusion processing module. This vector contains suggested longitudinal acceleration and suggested lateral steering angle values. The conversion of the longitudinal acceleration control quantity is achieved through a proportional-integral-derivative controller. The controller uses the acceleration suggestion value in the decision vector as the target input and collects the actual vehicle acceleration in real time as the feedback signal. When the decision suggestion gradually changes from 0.2 m / s² to -0.8 m / s² within 3 seconds, the controller adjusts the drive motor torque and brake hydraulic pressure in stages: maintaining positive motor torque output for the first second but gradually reducing it; switching to energy recovery mode in the second second, the motor generates negative torque; and activating mechanical braking when the required deceleration exceeds -0.5 m / s² in the third second, the hydraulic system applies braking force at a pressure increasing by 5 bar every 0.1 seconds. The conversion of the steering angle control quantity relies on the steering motor control module, which has a built-in closed-loop control of the steering rack position. The -15° steering angle output by the decision is first converted into a target rack displacement of 8.7mm. The steering motor adjusts the three-phase current in real time based on the rotor angle feedback from the resolver, so that the rack moves from the current position to the target position within 0.3 seconds.

[0056] The electric drive system responds to longitudinal control commands: during acceleration, the traction motor receives torque commands and converts DC voltage into frequency-adjustable three-phase AC power via an inverter; during deceleration, the energy recovery system activates, with the motor acting as a generator to feed energy back to the battery pack, with the maximum recovery power limited to 85% of the battery charging power. The braking system employs an electro-hydraulic composite architecture: when the deceleration demand is below 0.3g, only motor energy recovery braking is activated; when this threshold is exceeded, the series brake master cylinder supplements the hydraulic braking force, and the front-to-rear axle braking force distribution ratio is dynamically adjusted to 60:40. The steering system executes lateral control commands: the permanent magnet synchronous motor in the steering column receives torque commands and converts rotational motion into linear displacement of the rack via a worm gear mechanism; every 1mm movement of the rack corresponds to a 0.58° change in the steering wheel angle.

[0057] The longitudinal acceleration limit is dynamically calculated based on real-time vehicle conditions: when the tire adhesion coefficient sensor detects a slippery road surface, the maximum acceleration decreases from 2.0 m / s² to 1.2 m / s², and the maximum deceleration is adjusted from -5.0 m / s² to -3.5 m / s². The steering angle limit is based on the physical limits of the steering mechanism: a steering range of ±30° is allowed at speeds below 30 km / h; when speeds exceed 100 km / h, it is limited to ±15°. All output commands undergo ramp filtering before transmission, ensuring that the rate of change of acceleration commands does not exceed 3 m / s³ and the rate of change of steering angle does not exceed 50° / s to avoid mechanical shock. The longitudinal control command is encapsulated as a CAN bus ID0x101 message, containing a 2-byte acceleration setpoint, a 1-byte brake pressure request, and a 1-byte drive torque flag. The lateral control command is transmitted periodically via the FlexRay bus, sending a 4-byte steering angle target value and a 2-byte steering rate parameter in time slot TS12. The execution status feedback is transmitted at a frequency of 100Hz: the electric drive system reports the actual torque output value and motor speed; the braking system provides feedback on the pressure values ​​of each wheel cylinder; and the steering system returns the rack displacement sensor readings and motor current values.

[0058] When the steering motor current exceeds 150% of the rated value for 200 milliseconds, the overheat protection program is triggered, gradually reducing the steering assist torque. The braking system monitors wheel speed difference in real time; when the wheel speed difference on the same axle exceeds 15 km / h, it automatically enters ABS mode, temporarily overriding longitudinal control commands. The drive motor is equipped with a torque gradient limit, with the maximum torque change rate set at 500 Nm / s to prevent transmission system shock. The status data of all actuators are integrated into the central diagnostic unit. If a steering angle tracking error greater than 3° is detected for 0.5 seconds, or an acceleration deviation greater than 0.4 m / s² for 1 second, a control anomaly flag is triggered, and a backup control strategy is activated.

[0059] During Level 1 downgrade, the steering system switches to a position-current hybrid control mode, and longitudinal control switches to open-loop acceleration control. Level 2 downgrade activates mechanical redundancy: the steering column clutch engages, allowing direct mechanical control by the driver; the braking system switches to pure hydraulic mode, interpreting driver intent via pedal travel sensors. During downgrade, the system continuously attempts to restore the main control link; if the execution deviation returns to the normal range after three consecutive control cycles (30 milliseconds), it automatically switches back to the main control mode.

[0060] The instrument panel features dual-color indicator lights: green indicates normal operation of the main control system; yellow represents level one degraded status; and red indicates level two degraded status is activated. The head-up display projects key control parameters: a vertical bar chart comparing target and actual acceleration; and a horizontal indicator showing the angle between the target and actual steering wheel positions. When the system enters degraded mode, a voice prompt module plays a specific warning tone, while haptic feedback applies a 3Hz vibration to the steering wheel as a reminder.

[0061] A new control command is generated every 10 milliseconds, with the command's effective time aligned with the inertial measurement unit's sampling time, with a deviation not exceeding 0.2 milliseconds. Actuator response delays are incorporated into feedforward compensation: the steering system has an inherent delay of 80 milliseconds, and the compensation module sends angle commands 80 milliseconds in advance; the braking system has a hydraulic build-up delay of 50 milliseconds, and the feedforward control increases the pressure request value in advance. Historical control commands are stored in a circular buffer, retaining data from the most recent 5 seconds for response analysis. For example, if 10 consecutive steering commands fail to reach the target, the integral gain parameter of the steering controller is automatically adjusted.

[0062] The zero steering angle reference is determined by the average value of the steering wheel angle sensor during electronic parking; the longitudinal acceleration zero-point calibration requires the vehicle to be stationary on a level road surface, and the median of IMU data is collected within 3 seconds. Dynamic parameters are continuously identified online: the steering system transmission ratio drift is calculated every 5 minutes, and the relationship between rack displacement and wheel angle is fitted using the least squares method; the braking system response curve is updated hourly, based on the pressure-deceleration characteristic table of a step braking test. Calibration results update control parameters in real time; for example, when a change in steering ratio from 15.8:1 to 16.2:1 is detected, the steering angle conversion coefficient is automatically adjusted from 0.58° / mm to 0.56° / mm.

[0063] Simulated commands are sent via the diagnostic interface to: force the steering motor to output excessive current to test overheat protection; and inject a brake pressure sensor offset fault to verify safety monitoring. Test results generate a diagnostic report, recording the response time and fault coverage of each subsystem. For example, in the steering overheat protection test, the delay from excessive current to torque limit activation must be less than 50 milliseconds; the brake pressure fault tolerance test requires the system to switch to redundant acquisition channels within 100 milliseconds when the sensor deviation is 20%. All test data is encrypted and stored in tamper-proof memory, supporting in-depth fault analysis by technical personnel.

[0064] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0065] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An artificial intelligence electric vehicle safe driving image recognition system, characterized in that, The system comprises: a multi-source environment perception module, which acquires real-time road image data collected by a vehicle-mounted camera and synchronously receives driving speed, steering angle and obstacle distance information transmitted by a vehicle state sensor; a dynamic state construction module, which extracts traffic element spatial distribution features based on the real-time road image data and generates a multi-dimensional driving state vector in combination with the driving speed, steering angle and obstacle distance information; a feature fusion processing module, which inputs the multi-dimensional driving state vector and a historical hidden feature vector into a spatio-temporal evolution network and outputs an updated hidden feature vector and an initial driving decision vector through a continuous feature evolution unit.

2. The artificial intelligence electric vehicle safe driving image recognition system according to claim 1, characterized in that, The spatio-temporal evolution network comprises: a hierarchical feature extraction unit, which analyzes road topology features and dynamic obstacle trajectory features in the multi-dimensional driving state vector; a state evolution unit, which models spatio-temporal correlations of the road topology features, dynamic obstacle trajectory features and historical hidden feature vector by using a time-series feature transfer mechanism; a decision output unit, which constrains a numerical range of the initial driving decision vector by using a nonlinear mapping function. 3.The artificial intelligence electric vehicle safe driving image recognition system of claim 2, wherein, The feature fusion processing module further comprises: a dual-channel evaluation network, which receives the multi-dimensional driving state vector and the initial driving decision vector, generates a first safety evaluation value and a second safety evaluation value; a target decision network, which generates a target driving decision vector according to a multi-dimensional driving state vector at a subsequent time and the updated hidden feature vector; a target evaluation network, which generates a first target evaluation value and a second target evaluation value based on the multi-dimensional driving state vector at the subsequent time and the target driving decision vector. 4.The artificial intelligence electric vehicle safe driving image recognition system of claim 3, wherein, The system comprises: a risk quantification module, which calculates a scene risk evaluation value according to the initial driving decision vector and the multi-dimensional driving state vector at the subsequent time; an experience storage module, which groups the multi-dimensional driving state vector, the historical hidden feature vector, the initial driving decision vector, the scene risk evaluation value, the multi-dimensional driving state vector at the subsequent time and the updated hidden feature vector into a training data set. 5.The artificial intelligence electric vehicle safe driving image recognition system of claim 4, wherein, The system comprises: an evaluation network update module, which calculates a decision optimization target value according to the first target evaluation value, the second target evaluation value and the scene risk evaluation value, and updates evaluation network parameters based on the first safety evaluation value, the second safety evaluation value and the decision optimization target value. 6.The artificial intelligence electric vehicle safe driving image recognition system of claim 5, wherein, The system comprises: a decision network update module, which adjusts weight parameters of the spatio-temporal evolution network by using a gradient backpropagation algorithm, triggers parameter update according to a preset update period, and maximizes the decision optimization target value output by the dual-channel evaluation network.

7. The artificial intelligence electric vehicle safe driving image recognition system according to claim 4, characterized in that, The scene risk evaluation value is calculated by the following logic: a collision risk evaluation value is activated when the obstacle distance is below a safety threshold, a signal violation evaluation value is activated when a traffic signal state is at a dangerous level, and a weighted evaluation value of a path deviation coefficient and a comprehensive risk coefficient is calculated for the rest of the scenarios; wherein the comprehensive risk coefficient is an accumulated value of risk factors in each direction. 8.The AI electric vehicle safe driving image recognition system of claim 7, wherein, The decision optimization target value is calculated by applying the scene risk evaluation value and a decay factor to the minimum value of the first target evaluation value and the second target evaluation value and then summing the results. 9.The AI electric vehicle safe driving image recognition system of claim 8, wherein, The evaluation network updating module calculates the loss amount by the following manner: randomly sampling a data set in the training data set, respectively calculating a first mean square error amount of the first safety evaluation value and the decision optimization target value, and a second mean square error amount of the second safety evaluation value and the decision optimization target value. 10.The artificial intelligence electric vehicle safe driving image recognition system of claim 1, wherein, Comprise: The driving execution module converts the initial driving decision vector into vehicle control instructions, the vehicle control instructions include longitudinal acceleration control amount and lateral steering angle control amount, and drives the vehicle execution mechanism to complete corresponding actions.

Images