Intelligent driving decision-making system based on intelligent camera

Abstract

The invention relates to the technical field of vehicle visual perception, and discloses an intelligent driving decision-making system based on an intelligent camera, which comprises the following steps: quantifying the dynamic stability of a scene in real time by calculating the structure entropy of an optical flow field in a foreground dynamic region, and combining with parallel monitoring of global illumination change to determine the dynamic stability of the scene; the risk early warning signal is generated only when the dynamic instability is judged to be from a real physical risk instead of an illumination artifact, the dynamic stability of a scene can be directly quantified, alertness is generated in the germination stage of risk occurrence, the limitation that the traditional technology depends on semantic recognition and responds to a tiny dynamic risk is delayed is avoided, and the risk early warning signal is generated only when the dynamic instability is judged to be from the real physical risk instead of the illumination artifact. And an endogenous sensing credibility self-calibration mechanism ensures the decision reliability of the system under a complex illumination condition, and improves the safety of the intelligent driving system.

Description

TECHNICAL FIELD

[0001] The present application relates to an intelligent driving decision system based on an intelligent camera, and belongs to the technical field of vehicle visual perception. BACKGROUND

[0002] The current method compresses the original visual information irreversibly at the front end of information processing, and the cost is that the system loses the rich dynamic texture representing how the target and its surrounding environment move while obtaining semantics. When this method is placed in a real and continuously changing driving scene, especially when facing early risks that have not yet developed into significant displacement or shape change, its limitations begin to appear. For example, when following a car on an urban road, the car may produce a small lateral drift due to the driver's distraction or road bumps. The risk nature lies in the destruction of the overall dynamic stability of the visual scene, rather than the change of the semantic label of a single object. The prior art is difficult to respond quickly to such risks due to the lack of effective representation means.

[0003] Further analysis shows that this limitation cannot be solved simply by improving the detection model accuracy or camera resolution. It is caused by a contradiction in the processing flow of the prior art, which is the trade-off between preserving semantic information and preserving dynamic information.

[0004] Specifically, the prior art mainly has the following deficiencies: 1. The original representation of scene dynamic stability is lost at the starting point of information purification, making the system insensitive to continuous dynamic abnormalities below a certain threshold; 2. Risk decision highly depends on prior and discrete semantic labels, and lacks effective perception ability for any unknown dynamic risk that has not been defined in advance; 3. The entire perception-understanding-decision chain is subject to the constraints of processing flow for sudden risks that require intuitive and rapid response. Therefore, how to break out of the framework of first identifying objects and then judging risks, and establish a new mechanism that directly extracts and quantifies new features representing the dynamic stability of the entire scene from continuous visual graph sequences, so as to achieve rapid alertness for unknown and non-obvious dynamic abnormalities before semantic understanding, has become a technical problem to be solved by the present application. SUMMARY

[0005] The present application provides an intelligent driving decision system based on an intelligent camera, which mainly aims to solve the problem of how to establish a new mechanism that bypasses semantic recognition and directly perceives unknown dynamic risks from visual graph sequences.

[0006] To achieve the above purpose, the intelligent driving decision system of the intelligent camera provided by the present application comprises: an image information processing unit configured to calculate a total optical flow field representing dynamic changes between consecutive video image frames acquired by a vehicle-mounted camera, and to calculate in parallel a luminance gradient divergence representing a degree of global illumination change in the video image frames; a foreground dynamic region identification unit connected to the image information processing unit and configured to identify a foreground dynamic region by subtracting a background optical flow field caused by the ego vehicle motion from the total optical flow field based on ego vehicle motion information; an optical flow field structure entropy calculation unit connected to the foreground dynamic region identification unit and configured to calculate in real time a single scalar value representing a stability of the optical flow field structure in the foreground dynamic region, i.e. the optical flow field structure entropy, by statistically analyzing a consistency of directions of a plurality of optical flow vectors in the foreground dynamic region; a risk decision unit connected to the image information processing unit and the optical flow field structure entropy calculation unit and configured to monitor a time change rate of the optical flow field structure entropy, and to generate a risk warning signal as a conservative bias input to adjust the main driving decision logic only when the time change rate of the optical flow field structure entropy exceeds a dynamic warning threshold and the luminance gradient divergence is below an illumination stability threshold.

[0007] Preferably, the optical flow field structure entropy calculation unit is specifically configured to randomly sample a plurality of optical flow vectors in the foreground dynamic region , calculate an average direction vector of the plurality of optical flow vectors , and approximately calculate the optical flow field structure entropy according to the following relationship , , wherein is a module length of the average direction vector, and is a sum of module lengths of the plurality of optical flow vectors.

[0008] Preferably, the foreground dynamic region identification unit is specifically configured to acquire ego vehicle motion information from a vehicle-mounted inertial measurement unit, calculate the background optical flow field based on the ego vehicle motion information, and identify the foreground dynamic region by subtracting the corresponding background optical flow field vector from all optical flow vectors in the total optical flow field.

[0009] Preferably, the risk decision unit is specifically configured to input the risk warning signal as a highest priority attention focusing instruction or conservative bias signal to the main driving decision logic, and the conservative bias signal is used to instantaneously increase a perception refresh rate of the main driving decision logic on the foreground dynamic region, or temporarily shorten a safety following distance threshold set by the system, or pre-charge the brake system of the vehicle.

[0010] Preferably, the system further comprises a driving style adaptive unit configured to: store and analyze time series data of the optical flow field structure entropy in the background; automatically cluster the time series data into one of a smooth driving style pattern, a normal driving style pattern and an aggressive driving style pattern by an unsupervised learning method; and dynamically adjust the dynamic warning threshold for the risk decision unit according to the currently identified driving style pattern, wherein the dynamic warning threshold set for the smooth driving style pattern is lower than the dynamic warning threshold set for the aggressive driving style pattern.

[0011] Preferably, the system further comprises a driver state awareness unit configured to: establish a baseline model for predicting the expected optical flow field structure entropy based on the vehicle driving state parameters; compare the optical flow field structure entropy calculated by the optical flow field structure entropy calculation unit in real time with the expected optical flow field structure entropy predicted by the baseline model to generate an entropy residual time series; and perform frequency domain analysis on the entropy residual time series, and when the low-frequency component energy of the entropy residual time series continuously exceeds a fatigue state threshold, determine that the driver is in an abnormal driving state and generate a corresponding cabin warning.

[0012] Preferably, the optical flow field structure entropy calculation unit is further configured to: before calculating the optical flow field structure entropy, perform local consistency verification on the optical flow vectors in the foreground dynamic region, and mark vectors whose direction or amplitude deviates significantly from their adjacent vectors as invalid vectors; calculate an invalid vector rate based on the number of invalid vectors marked; and adaptively adjust the time smoothing strength of the calculation result of the optical flow field structure entropy according to the value of the invalid vector rate, the time smoothing strength being positively correlated with the value of the invalid vector rate.

[0013] Preferably, the optical flow field structure entropy calculation unit is further configured to: record the long-term trend of the invalid vector rate in the background; and if the long-term trend of the invalid vector rate shows a continuous increase when the vehicle is in good lighting and weather conditions, generate a vehicle maintenance suggestion indicating that the camera lens is contaminated or the sensor is aging.

[0014] Preferably, the risk decision unit is configured to perform an inhibition and smoothing operation when the luminance gradient divergence is higher than a lighting stability threshold, which temporarily and substantially increases the dynamic warning threshold, and instructs the main driving decision logic to maintain the stability of the current motion state or perform a gentle deceleration within a preset time.

[0015] Preferably, the baseline model established by the driver state awareness unit is based on the current speed, current steering angular velocity and front safety distance of the vehicle to predict the expected optical flow field structure entropy.

[0016] Compared with the prior art, the present application has the following advantages: 1. By calculating the structural entropy of the optical flow field in the overall visual scene in front, the stability of the scene dynamics is directly quantified. When any potential risk, even those weak dynamic anomalies that do not conform to the preset object model, occurs, the system can generate an alert signal in the early stage of risk by capturing the instantaneous change rate of the structural entropy of the optical flow field. This approach avoids the limitations of traditional techniques that cause response delays to subtle dynamic risks due to premature compression of information in the semantic abstraction process.

[0017] 2. By coupling the calculation of the structural entropy of the optical flow field with the parallel monitoring of global light changes, a self-calibration mechanism for perceived reliability is constructed. The system can actively identify false signal caused by dramatic changes in light such as entering a tunnel or being irradiated by strong light at night. In such cases caused by imaging instability of the sensor itself rather than real physical world dynamic changes, the system will actively suppress the generation of risk warnings, thereby ensuring the decision robustness of the core risk perception mechanism in real and variable driving environments.

[0018] 3. The system introduces an unsupervised driving style adaptive capability through background analysis of the time series of the structural entropy of the optical flow field. It can automatically identify the habits of the current driver based on the fluctuation characteristics of the entropy value sequence during use, and dynamically adjust the sensitivity of triggering risk warnings based on this, so that the signal-to-noise ratio of risk warnings is improved, forming a more harmonious human-machine interaction between the system and the driver. The invention also utilizes the information byproducts in the optical flow calculation process. By statistically analyzing the invalid vectors in the original optical flow field, the system can diagnose the health status of the camera itself without increasing any additional hardware costs, and can identify problems such as lens contamination or sensor aging and provide maintenance recommendations. Thus, an instant risk perception scheme is formed into an intelligent perception system with environmental adaptability, user personalization, and hardware self-diagnosis capability. BRIEF DESCRIPTION OF DRAWINGS

[0019] Fig. 1 The flow chart for offline calibration and model initialization of the system of the present invention; Fig. 2 The flow chart for real-time risk decision logic of the present invention; Fig. 3 The overall architecture and data flow diagram of the system of the present invention. DETAILED DESCRIPTION

[0020] To make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be described in detail below. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present invention.

[0021] An intelligent driving decision system based on intelligent camera, the system architecture is configured to include an image information processing unit, a foreground dynamic region identification unit, an optical flow field structure entropy calculation unit and a risk decision unit, in some embodiments, the system can also integrate a driving style adaptive unit and a driver state perception unit, these units are coupled with each other through specific data flow and control logic, wherein the image information processing unit is the perception input end of the system, responsible for converting the continuous video image frames obtained by the camera into dynamic and illumination feature data, the foreground dynamic region identification unit processes the dynamic data using the vehicle's own motion information, the optical flow field structure entropy calculation unit quantifies the dynamic stability of the scene on this basis, and the risk decision unit fuses the dynamic stability and illumination reliability information to generate a risk warning signal for adjusting the main driving decision logic; in the application scenario of following a vehicle on urban roads, a technical challenge is how to effectively perceive the small lateral drift of the front vehicle caused by atypical reasons such as driver distraction or road bumps without touching the lane line, for this kind of risk mainly reflected in the non-steady state change of visual scene texture, the image information processing unit in the system is configured to calculate the dense total optical flow field representing the inter-frame pixel change based on continuous video image frames by Lucas-Kanade algorithm, providing dynamic data input without semantic compression for subsequent risk analysis, in order to identify the false interference in the total optical flow field caused by sudden changes in illumination such as vehicle entering or exiting a tunnel or night passing, the image information processing unit also calculates the brightness gradient map of the current image frame in parallel, and quantifies a single scalar value, i.e. the brightness gradient divergence, by the spatial distribution variance of all gradient vector amplitudes in the gradient map, which remains low when the illumination is uniform, but will instantaneously increase when the illumination changes suddenly, thus providing a synchronous basis for checking the reliability of the optical flow field data.

[0022] Considering that the total optical flow field generated by the image information processing unit mixes the background optical flow caused by the motion of the ego vehicle, the foreground dynamic region identification unit in the system is configured to perform a background optical flow stripping operation, this unit obtains the translation and rotation motion information of the vehicle from the vehicle controller area network bus through the vehicle-mounted inertial measurement unit, based on this motion parameter, the foreground dynamic region identification unit establishes a mathematical model to calculate the background optical flow field that the static background should generate, then the unit subtracts the background optical flow field vector corresponding in space position from all the optical flow vectors in the total optical flow field, and identifies the non-zero optical flow region remaining after operation as the foreground dynamic region, this way focuses the scope of risk analysis without the need for artificial intelligence model training; the foreground dynamic region identification unit calculates the background optical flow field, which is based on a calibrated pinhole camera model, which uses the focal length and other inherent internal parameters of the camera to calculate the vehicle translation speed and angular velocity , through the following kinematic equations, it is deterministically mapped to any pixel point in the image Background optical flow vector at : , , where the variable Z, that is, the scene depth corresponding to the pixel, is obtained by calling a pre-trained monocular depth estimation model that runs synchronously with the video stream; after identifying the foreground dynamic area, the optical flow field structure entropy calculation unit immediately executes a grid-based hierarchical uniform sampling procedure, which divides the bounding box of the foreground dynamic area into a 10×10 grid and selects a vector as a sample in each non-empty grid containing a valid optical flow vector. If the total number of samples is lower than the preset minimum sample number threshold , this threshold is the lower limit set to ensure the stability of subsequent statistics, then repeat the random selection in all non-empty grids until the total number of samples reaches .

[0023] In the identified foreground dynamic region, the optical flow field structural entropy calculation unit is configured to quantify the structural stability of the optical flow field in the region in real time. Specifically, the unit randomly samples multiple optical flow vectors in the foreground dynamic region. , calculate the average direction vector of these optical flow vectors , and according to the relationship Approximately calculate a single scalar value, namely the optical flow field structure entropy ,in, is the modulus of the mean direction vector, is the sum of the modulus lengths of the sampled optical flow vectors. According to this relationship, when the directions of all optical flow vectors in the area are highly consistent, such as when following a car normally, the vectors are almost parallel. The value approaches a stable low value. On the contrary, when abnormal motion occurs in the area and the direction of the optical flow vector is disordered, The value will increase significantly, thus representing the dynamic changes of the multi-dimensional scene as the monitoring of a single scalar value; in order to deal with the interference of isolated pseudo optical flow that may be generated by camera sensor noise or lens stains on the calculation of optical flow field structure entropy, the optical flow field structure entropy calculation unit is used in the calculation. Before calculating the value, a local consistency check is performed. For each optical flow vector, its direction and amplitude are checked to see if they deviate significantly from their spatial neighboring vectors. If the deviation exceeds the statistical threshold, the vector is marked as an invalid vector and removed from this calculation. At the same time, the unit calculates the invalid vector rate based on the number of vectors marked as invalid, and according to the rate value, adaptively adjusts the optical flow field structure entropy. The strength of temporal smoothing of the calculated results, i.e., the value of the invalid vector rate, is positively correlated with the strength of the temporal smoothing filter. This is intended to improve the stability of the output results under different sensor signal-to-noise ratio conditions. Furthermore, the unit also records the long-term trend of the invalid vector rate in the background. If the rate value shows a continuous increase under good lighting and weather conditions, the system will generate a vehicle maintenance recommendation indicating that the camera lens is damaged or the sensor is aging.

[0024] The risk decision unit integrates the optical flow field structure entropy and the illumination credibility information to make a decision. The unit monitors the optical flow field structure entropy. Time rate of change , and taking into account the driving habits of different drivers will affect The normal fluctuation baseline of the value is stored and analyzed in the background by the driving style adaptation unit in the system. The time series data of the values ​​of the vehicle are automatically clustered into stable, normal and aggressive driving style modes by unsupervised clustering methods such as K-Means algorithm in the non-driving state of the vehicle, and the corresponding basic entropy fluctuation range is determined for each mode. The risk decision unit dynamically adjusts the dynamic warning threshold for triggering the warning based on the currently identified driving style mode. The threshold set for the stable driving style is lower than the threshold set for the aggressive driving style. During driving, the risk decision unit only adjusts the time change rate of the optical flow field structure entropy. A risk warning signal is generated only when the dynamic warning threshold corresponding to the current driving style is exceeded and the brightness gradient divergence is lower than the lighting stability threshold. If the brightness gradient divergence is higher than the lighting stability threshold, the risk decision unit determines that the current perception uncertainty has increased due to a sudden change in lighting. At this time, suppression and smoothing operations are performed to temporarily increase the dynamic warning threshold, and instruct the main driving decision logic to maintain a stable motion state or perform a gentle deceleration in a short period of time. The generated risk warning signal is input into the main driving decision logic as a conservative bias signal to instantly increase the perception refresh rate of the foreground dynamic area, or temporarily shorten the safe following distance threshold set by the system, or pre-build pressure for the braking system, thereby providing reaction time for the main driving decision system.

[0025] When the risk decision unit determines the risk, the optical flow field structure entropy it monitors Time rate of change It is not obtained by direct differentiation, but is determined by a secondary processing procedure that first processes the raw entropy value of the real time The time series is filtered using an exponential moving average filter, which is a recursive filter that smoothes the current value by weighted averaging historical data. is set to 150 milliseconds, and then based on the smoothed entropy value sequence, at each time point, the window data of the previous 200 milliseconds is taken, and linear regression analysis is performed by the least square method, and the slope of the obtained straight line is determined as the estimated value; in parallel, to ensure the long-term effectiveness of the expected entropy reference model in the driver state perception unit, the model is configured to have an online adaptive calibration capability, which is activated in the continuous period when the system determines that the driver is in a non-fatigue state, and by calculating the mean value of the entropy residual sequence in this period, if the mean value deviates from zero by more than a calibration trigger threshold within a 10-minute observation window, the model will automatically fine-tune its constant term to compensate for the systematic deviation, so that the model baseline can adapt to the slow drift of the driver's driving style due to long driving; in some embodiments, to perceive the driver's state without increasing dedicated hardware, the system can configure the driver state perception unit to establish a reference model based on the current speed of the vehicle, the current steering angular velocity, and the front safety distance and other parameters to predict the expected optical flow field structure entropy that a standard driver should generate under the current driving state. The unit compares the actual value calculated by the optical flow field structure entropy calculation unit in real time with the expected value predicted by the reference model to generate an entropy residual time sequence. Given that fatigue driving behavior often exhibits a combination of long-term no correction and intermittent large correction in terms of control, it will manifest as a low-frequency high-amplitude oscillation in the residual signal. Therefore, the unit performs frequency domain analysis on the entropy residual time sequence, and when it detects that the energy of the low-frequency component continuously exceeds a preset fatigue state threshold, it determines that the driver is in an abnormal driving state and generates a corresponding cabin alarm.

[0026] Example 1: In a working condition of an automatic driving system driving along a suburban road, the vehicle travels at a medium speed on a shaded road section, then faces a short tunnel entrance, and the tunnel exit is followed by a backlight environment. In the shaded road section of this scenario, the intermittent light and shadow changes cause the brightness of the video image frames obtained by the camera to jump continuously. At the tunnel exit, an irregular tire fragment is stationary in the driving lane, its color is similar to the road surface and part of it is in the shadow. When the vehicle enters the shaded road section, the brightness gradient divergence value calculated by the image information processing unit in the system presents a high-frequency pulse fluctuation synchronized with the light spot flickering. At the same time, the optical flow field structure entropy output by the optical flow field structure entropy calculation unit also increases correspondingly due to the movement of light and shadow, but the decision logic of the risk decision unit is set to only consider the time rate of change of the optical flow field structure entropy as the optical flow field structure entropy time rate of change the system does not generate a risk warning signal, and the vehicle maintains its established steady driving state; at the moment when the vehicle enters the tunnel entrance, the camera's field of view suddenly changes from bright to dark, and the luminance gradient divergence value calculated by the image information processing unit instantaneously rises to a peak value; at this time, the invalid optical flow caused by the imaging of the sensor changes dramatically, resulting in a significant jump in the structure entropy value of the optical flow field; the risk decision unit determines that the perception system reliability has decreased according to the coupled verification logic, and performs suppression and smoothing operations, temporarily increasing the dynamic warning threshold, thereby avoiding false decisions caused by the imaging fluctuations of the sensor itself.

[0027] When the vehicle exits the tunnel, the camera's field of view returns to the bright environment under the backlight, and the luminance gradient divergence value quickly falls below the stable light threshold after experiencing a short pulse, indicating that the lighting conditions are tending to be stable; at this moment, the tire debris on the driving path causes local changes in the optical flow field vector in the foreground dynamic area as the vehicle approaches, and the optical flow field structure entropy calculation unit detects this local optical flow field structure instability caused by physical obstacles and calculates a time variation rate of the structure entropy value of the optical flow field that exceeds the current dynamic warning threshold ; since the luminance gradient divergence value is low at this time, the decision condition of the risk decision unit is met, and the system immediately generates a risk warning signal as a conservative bias input; after the risk warning signal is input into the main driving decision logic, it does not trigger emergency braking or large-scale steering, but instructs the vehicle's braking system to pre-establish pressure, and the main driving decision logic temporarily shortens the preset safe following distance threshold, which makes the vehicle reduce the driving speed in a smooth manner, provides time for higher-level perception and planning modules to analyze the source of the local dynamic anomaly, and ultimately avoids the unknown obstacle; in this operation sequence, risk perception is realized as a direct measurement of scene dynamics, and this measurement is always coupled with a synchronously generated perception channel reliability evaluation for decision-making.

[0028] ​Example 2: To verify the performance of the technical solution in response to dynamic risks and light interference, a hardware-in-the-loop test platform was built. The platform uses driving simulation software to generate a video stream with a resolution of 1920x1080 pixels and a frame rate of 30 frames per second, and inputs it into an embedded processing unit running the decision system algorithm. The response of the unit is recorded in real time, forming a closed-loop test. The test sets up a control group and a test group. The decision logic of the control group only relies on the change of the structure entropy of the optical flow field, while the test group is fully configured with the decision logic of the present application that couples and checks the structure entropy of the optical flow field with the divergence of the brightness gradient. The dynamic warning threshold used in the test is set based on a receiver operating characteristic curve analysis of a database containing diversified road conditions. The threshold point that maximizes the suppression of false positives caused by regular driving operations is selected under a specific false alarm rate control target. A baseline value of 0.75 is set.

[0029] In the first light interference test, the simulated vehicle enters the tunnel at a speed of 80 kilometers per hour. At this time, both the control group and the test group detect that the peak value of the structure entropy of the optical flow field reaches 0.92, which exceeds the set dynamic warning threshold. However, the test group correspondingly calculates the peak value of the divergence of the brightness gradient to be 0.95, which is much higher than the preset light stability threshold. Therefore, its coupling and checking logic suppresses the generation of the warning signal. The control group produces a false warning due to the lack of this checking mechanism. In the second weak dynamic risk test that follows, the simulated vehicle follows at a speed of 120 kilometers per hour on the highway. The front vehicle produces a lateral yaw for avoidance. This action causes the peak value of the structure entropy of the optical flow field to rise to 0.79 in both groups. Since the light is stable in this scenario, the value of the divergence of the brightness gradient is only 0.15, which is lower than the light stability threshold. Therefore, the coupling and checking logic of the test group confirms the authenticity of the dynamic change, and generates an effective risk warning signal like the control group. The test results show that the technical solution maintains the ability to perceive weak physical risks while avoiding false risk signals caused by dramatic changes in light at the sensor level by introducing parallel monitoring and coupled decision-making of the stability of the light environment, solving the problem of insufficient reliability of single risk measurement in complex light conditions.

[0030] Example 3: This example combines Figs. 1 to 3 to implement an intelligent driving decision system based on an intelligent camera. For example, Fig. 1As shown, the procedure starts from the data acquisition process, and the collected video, vehicle parameters and driver state labels are stored in the synchronous data stream database, and the data of the database is used for three parallel initialization tasks: first, the driving style adaptive unit initialization, by extracting the time series of the structure entropy S of the optical flow field, applying unsupervised clustering algorithm, and calculating the statistical features of each mode, finally output the dynamic warning threshold function; second, the driver state perception unit initialization, by extracting the vehicle driving parameters and the actual entropy value under the state of concentration to establish the prediction model, and calibrating the entropy residual sequence to determine the fatigue threshold, finally output the expected entropy reference model and the fatigue state threshold; third, key threshold calibration, by extracting the segment of the brightness gradient divergence pulse caused by the sudden change of light, statistical distribution analysis is carried out, and the 95th percentile of the peak set is set as the final light stability threshold; the outputs of the three initialization tasks together constitute a complete initial state decision system, which provides all the decision parameters and models for subsequent deployment to the vehicle platform.

[0031] As shown in Fig. 2 The figure starts with receiving the structure entropy of optical flow field and the divergence of brightness gradient data, first judges the interference of sudden change of light, if the divergence of brightness gradient is higher than the light stability threshold, it is judged that the perception uncertainty is increased, the inhibition and smoothing operation is executed, so that the warning is inhibited, to avoid false alarm caused by light artifact; if the divergence of brightness gradient is lower than the light stability threshold, it indicates that the light condition is stable, the system further calculates the entropy value time change rate to evaluate the dynamic risk; then, the system judges whether there is real dynamic risk, that is, compares the entropy value time change rate with the dynamic warning threshold, if it does not exceed the threshold, it is judged that there is no significant risk, if it exceeds the threshold, a risk warning signal is generated as a conservative bias input to the main driving logic, and finally the warning process is completed.

[0032] As shown in Fig. 3As shown in the figure, the vehicle-mounted camera is used as the main sensor to input continuous video frames to the 1.0 image information processing module, which in parallel outputs the total optical flow field to the 2.0 quantization scene dynamics module and the brightness gradient divergence to the 3.0 fusion information decision module; the 2.0 quantization scene dynamics module further integrates the vehicle motion information from the on-board inertial measurement unit to calculate and output the optical flow field structure entropy to the 3.0 module, output the entropy value time series to the D1 entropy value time series database, and output the invalid vector rate to the 6.0 diagnostic camera health module and the D4 invalid vector rate history database respectively; 4.0 The adaptive driving style module updates the D2 driving style pattern library based on historical data in the D1 database and provides the current style threshold to the 3.0 module; the 3.0 module integrates all input information, ultimately generates a risk warning signal and transmits it to the main driving decision logic; in parallel, the 5.0 driver status perception module compares the real-time entropy value from the 2.0 module with the D3 expected entropy benchmark model to perceive the driver's status and may output a cabin alert, while the 6.0 diagnostic camera health module issues maintenance recommendations to the driver when necessary based on the real-time invalid vector rate and historical records in the D4 database.

[0033] Example 4: To solve the engineering problem of the lack of deterministic setting basis for the internal decision model and key thresholds of the autonomous driving system before being deployed on a specific vehicle platform, a set of offline system calibration and model initialization procedures is adopted. The procedure begins with a data acquisition process, that is, using a test vehicle equipped with a camera, inertial measurement unit and controller area network interface with the same specifications as the target vehicle model, a cumulative synchronous data stream of no less than 500 hours is collected under working conditions covering a variety of roads and environments. The data stream contains continuous video images, vehicle dynamic parameters, and driver status and driving style labels annotated by background safety personnel; for the initialization of the driving style adaptive unit, the system extracts the optical flow field structure entropy from the collected data The time series is obtained by K-Means unsupervised clustering algorithm, and the time series data is classified into three driving style modes: stable, normal and aggressive. For each classified mode, the system calculates the corresponding optical flow field structure entropy. Mean of the values and standard deviation ; Subsequently, the dynamic warning threshold used to adjust the risk warning sensitivity is determined as a function related to the baseline threshold, that is, the dynamic warning threshold under a specific driving style is set as the sum of the baseline threshold and an adaptive coefficient, where the value of the adaptive coefficient is linearly determined by the ratio of the standard deviation of the entropy value of the current driving style to the standard deviation of the entropy value of the ordinary driving style, so that the aggressive driving style corresponds to a threshold with a higher tolerance.

[0034] For the establishment of the expected optical flow field structure entropy benchmark model in the driver state perception unit, the system uses the vehicle driving state parameters in the collected data as inputs to obtain the actual optical flow field structure entropy generated by the driver labeled as the focused state within the corresponding time period As output, a prediction model is established by multiple linear regression analysis, and the relationship is determined as wherein, is the absolute value of the current vehicle speed, is the absolute value of the current steering angle speed, is the model coefficient for the specific vehicle platform obtained by regression analysis; for the determination of the fatigue state, the system applies fast Fourier transform to the entropy residual time series and calculates the total energy in the 0.1 Hz to 0.5 Hz frequency band, and the fatigue state threshold is set by statistical distribution analysis of the frequency band energy of the focused and fatigue driving segments in the labeled data, and the energy value that can distinguish 99.5% of the two types of samples on the distribution is selected as the decision boundary; for the calibration of the light stability threshold, the system automatically identifies all segments in the data set where the brightness gradient divergence value appears instantaneous pulse due to the vehicle entering or exiting the tunnel or encountering strong light, and records the peak values of these pulses, and the light stability threshold is set as the 95th percentile of the peak value set, which is used as a reference benchmark that can identify sudden changes in light and avoid misjudgment of normal scene changes; by performing the above procedures, the multiple decision parameters and models in the system are determined to have specific numerical values or functional relationships with traceable sources, so that the entire decision system reaches an initial state of verified and complete functions before deployment.

[0035] Example 5: When the vehicle-mounted camera is temporarily blocked by mud or snow melting, causing changes in the quality of the data output, the risk decision unit in the system is configured to use the invalid vector rate calculated in real time as a basis for checking the integrity of the perception channel. If the value of the invalid vector rate jumps from a stable baseline and exceeds a preset data integrity threshold within a 100 millisecond monitoring window, the system determines that the camera is physically blocked, suspends the risk judgment logic based on the optical flow field structure entropy of the system, and generates a perception-limited state signal, which is used to prompt the driver that the camera function is limited and instruct the main driving decision logic to maintain the current driving state of the vehicle within a preset time.

[0036] In the risk warning suppression logic of the system, to deal with the oscillation of the luminance gradient divergence value around the stable illumination threshold, the control is set to a double threshold strategy, which sets a suppression activation threshold and a suppression release threshold lower than the activation threshold. The suppression operation of the risk warning is started when the luminance gradient divergence value exceeds the suppression activation threshold. The suppression state will continue to be maintained until the luminance gradient divergence value is lower than the suppression release threshold and continuously maintained for 500 milliseconds, so that the state switching of the decision module is not triggered due to signal oscillation under critical illumination conditions.

[0037] In an offline optimization process aimed at optimizing the internal parameters of the optical flow field data verification link, the objective function is set to maximize a comprehensive data quality index composed of the rejection rate of known sensor noise data and the retention rate of valid optical flow data belonging to the edge of a real moving object. To achieve this goal, the system performs a parameter traversal search on a pre-labeled database containing a large number of static scene noise samples and dynamic scene edge samples. In this search process, the direction consistency angle threshold and the amplitude consistency proportion range in the local consistency verification step are systematically adjusted. By calculating the comprehensive data quality index under each parameter combination, the optimization process determines a set of parameters that maximize the objective function, i.e., the direction consistency angle threshold is determined to be 30 degrees, and the amplitude consistency proportion range is determined to be 0.5 to 2.0. All subsequent optical flow field structure entropy calculations use this set of parameters determined by the optimization process.

[0038] To determine the data integrity threshold for judging physical occlusion of the camera, the system performs another offline optimization, which aims to minimize the false positive rate caused by adverse weather such as heavy rain but not occlusion conditions, while ensuring that the detection rate of occlusion events exceeding 10% of the lens area is not less than 99%. By testing thousands of virtual occlusions of different areas and transparency on a hardware-in-the-loop platform and recording the corresponding invalid vector rates, and analyzing the statistical distribution of invalid vector rates in interference scenes such as heavy rain, a decision threshold is finally determined that meets the above optimization goal. This threshold is fixed in the logic of the risk decision unit.

[0039] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0040] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the present application.

Claims

1. An intelligent driving decision system based on an intelligent camera, characterized in that: The system includes: an image information processing unit configured to calculate, based on continuous video image frames acquired by the vehicle-mounted camera, a total optical flow field representing dynamic changes between video image frames, and in parallel calculate a brightness gradient divergence representing a degree of global illumination change in the video image frames; a foreground dynamic region identification unit connected to the image information processing unit and configured to subtract the background optical flow field caused by the vehicle's own motion from the total optical flow field based on the vehicle's own motion information, thereby identifying the foreground dynamic region; an optical flow field structure entropy calculation unit connected to the foreground dynamic region recognition unit and configured to calculate in real time a single scalar value representing the stability of the optical flow field structure in the foreground dynamic region by statistically analyzing the directional consistency of multiple optical flow vectors; A risk decision unit is connected to the image information processing unit and the optical flow field structure entropy calculation unit, and is configured to monitor the time change rate of the optical flow field structure entropy, and only when the time change rate of the optical flow field structure entropy exceeds a dynamic warning threshold and the brightness gradient divergence is lower than an illumination stability threshold, generate a risk warning signal as a conservative bias input to adjust the main driving decision logic.

2. The intelligent driving decision-making system based on an intelligent camera according to claim 1, characterized in that: The optical flow field structure entropy calculation unit is specifically configured to: randomly sample multiple optical flow vectors in the foreground dynamic area ; Calculate the average direction vector of multiple optical flow vectors ; And the optical flow field structural entropy is approximately calculated according to the following relationship ,in, , is the modulus of the mean direction vector, It is the sum of the modulus lengths of multiple optical flow vectors.

3. The intelligent driving decision system based on an intelligent camera according to claim 1, characterized in that: The foreground dynamic area identification unit is specifically configured to: obtain the vehicle's own motion information from a vehicle-mounted inertial measurement unit; infer the background optical flow field based on the vehicle's own motion information; and subtract the corresponding background optical flow field vector from all optical flow vectors in the total optical flow field, and identify the non-zero optical flow area remaining after the operation as the foreground dynamic area.

4. The intelligent driving decision system based on an intelligent camera according to claim 1, characterized in that: The risk decision unit is specifically configured to: input the risk warning signal into the main driving decision logic as a highest priority attention focus instruction or conservative bias signal; the conservative bias signal is used to instantaneously increase the main driving decision logic's perception refresh rate of the foreground dynamic area, or temporarily shorten the system-set safe following distance threshold, or pre-build pressure for the vehicle's braking system.

5. The intelligent driving decision system based on an intelligent camera according to claim 1, characterized in that: The system also includes a driving style adaptation unit configured to: store and analyze time series data of optical flow field structural entropy in the background; automatically cluster the time series data into one of a stable driving style mode, a normal driving style mode, and an aggressive driving style mode through an unsupervised learning method; and dynamically adjust a dynamic warning threshold for the risk decision unit based on the currently identified driving style mode, wherein the dynamic warning threshold set for the stable driving style mode is lower than the dynamic warning threshold set for the aggressive driving style mode.

6. The intelligent driving decision system based on an intelligent camera according to claim 1, characterized in that: The system also includes a driver state perception unit configured to: establish a baseline model for predicting expected optical flow field structural entropy based on vehicle driving state parameters; compare the optical flow field structural entropy calculated in real time by the optical flow field structural entropy calculation unit with the expected optical flow field structural entropy predicted by the baseline model to generate an entropy residual time series; The entropy residual time series is analyzed in the frequency domain. When the energy of the low-frequency component of the entropy residual time series continuously exceeds a fatigue state threshold, the driver is judged to be in an abnormal driving state and a corresponding cockpit alarm is generated.

7. The intelligent driving decision-making system based on an intelligent camera according to claim 1, characterized in that: The optical flow field structural entropy calculation unit is further configured to: before calculating the optical flow field structural entropy, perform a local consistency check on the optical flow vectors in the foreground dynamic area, and mark the vectors whose directions or amplitudes deviate significantly from their neighboring vectors as invalid vectors; calculate the invalid vector rate based on the number of vectors marked as invalid; and adaptively adjust the temporal smoothing intensity of the optical flow field structural entropy calculation results according to the value of the invalid vector rate, and the temporal smoothing intensity is positively correlated with the value of the invalid vector rate.

8. The intelligent driving decision-making system based on an intelligent camera according to claim 7, characterized in that: The optical flow field structural entropy calculation unit is further configured to: record the long-term change trend of the invalid vector rate in the background; and when the vehicle is in good lighting and weather conditions, if the long-term change trend of the invalid vector rate shows a continuous increase, generate a vehicle maintenance recommendation indicating that the camera lens is damaged or the sensor is aging.

9. The intelligent driving decision-making system based on an intelligent camera according to claim 1, characterized in that: The risk decision unit is configured to perform a suppression and smoothing operation when the brightness gradient divergence is higher than the lighting stability threshold. This operation temporarily and significantly increases the dynamic warning threshold and instructs the main driving decision logic to maintain the stability of the current motion state or perform a gentle deceleration within the next preset time.

Images