Light field display safe driving assistance system based on vehicle machine platform

Abstract

The invention relates to the technical field of intelligent driving, in particular to a light field display safe driving assistance system based on a vehicle machine platform, which comprises an environment perception and data processing module, a light field display and augmented reality module, a multi-scene recognition and adaptive adjustment module and an intelligent monitoring and fault early warning module, the traffic environment feature vectors are obtained by processing the traffic environment data, so that the accuracy and reliability of the data are improved; through accurate presentation and reality enhancement of the traffic environment feature vector, a driver can visually perceive the surrounding driving environment. The display position of the light field is automatically adjusted by intelligently identifying various scenes of the driving environment so as to adapt to different driving scenes and driver requirements; through quantitative analysis of system operation state data and establishment of a fault early warning mechanism, potential safety hazards can be found and processed in time. The method is used for solving the technical problems that current intelligent driving is not high in safety and not timely in response.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a light field display safety driving assistance system based on a vehicle computer platform. Background Art

[0002] Intelligent driving technology, also known as autonomous driving or adaptive driving, refers to the use of advanced sensors (such as radar, lidar, cameras, etc.), computer systems and artificial intelligence algorithms to enable vehicles to autonomously complete driving tasks with no or only a small amount of human intervention.

[0003] In the current intelligent driving management, there are still the following problems: driving safety and reaction speed need to be improved. The existing intelligent driving technology has room for improvement in safety, especially in complex and changeable driving environments. The system may not be able to perceive and respond to potential dangers and obstacles in a timely and accurate manner, thereby increasing the risk of traffic accidents. Drivers need to respond quickly when facing emergencies, but the existing system may not be able to provide sufficient information support or warnings, resulting in limited driver reaction speed and affecting overall driving safety; environmental perception and data processing capabilities are limited. The current intelligent driving system has limitations in environmental perception, especially at night or under low light conditions, which directly affects the accuracy and reliability of driving. The system's processing and analysis capabilities for captured data are also limited, and it may not be able to fully extract and utilize data, thereby affecting the judgment and warning of potential dangers; the system's operating status monitoring and fault warning mechanism is imperfect. The existing intelligent driving system lacks a comprehensive operating status monitoring and fault warning mechanism, which makes it impossible for drivers or system maintenance personnel to take measures in advance to prevent the occurrence or escalation of faults. Therefore, a series of targeted methods are urgently needed to deal with these problems in order to improve the monitoring and optimization management level of intelligent driving. Summary of the invention

[0004] The purpose of the present invention is to solve the problems in the background technology and to propose a light field display safety driving assistance system based on a vehicle computer platform.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The light field display safety driving assistance system based on the vehicle computer platform includes: environmental perception and data processing module, light field display and augmented reality module, multi-scene recognition and adaptive adjustment module, and intelligent monitoring and fault warning module;

[0007] Environmental perception and data processing module: This module integrates a sensor group (such as infrared cameras, laser radars, millimeter-wave radars, etc.) to capture traffic environment data at night or in low light conditions, including road information, vehicles, pedestrians and other obstacles, and processes and analyzes the captured traffic environment data to obtain traffic environment feature vectors. At the same time, it determines potential dangers and obstacles and generates risk obstacle signals.

[0008] Light field display and augmented reality module: obtains risk obstacle signals from the environment perception and data processing module, and accurately presents and augments the traffic environment feature vectors corresponding to the risk obstacle signals to embed them into the driving field of view;

[0009] Multi-scenario recognition and adaptive adjustment module: Based on the data processing results of the light field display and augmented reality modules, it intelligently recognizes multiple scenarios of the driving environment, including road types (such as highways, urban roads, rural roads, etc.), weather conditions (such as sunny days, rainy days, foggy days, etc.) and vehicle conditions (involving vehicle speed, position, etc.); according to the driving scene recognition results, it automatically adjusts the light field display position to adapt to different driving scenes;

[0010] Intelligent monitoring and fault warning module: After completing multi-scenario recognition and adaptive adjustment, continue to monitor the overall operating status of the safe driving assistance system, obtain the operating status data of the safe driving assistance system, and perform quantitative analysis to obtain the operating status index value; combine the operating status index value to establish a fault warning mechanism to detect whether there is a fault or abnormality in the current operating status.

[0011] It should be noted that the application object of the light field display safety driving assistance system based on the vehicle-mounted platform in the present invention can be the on-board system management in the field of automobile intelligent driving. Specifically, it can be through an integrated sensor group (such as infrared cameras, lidar, millimeter-wave radar, etc.) to comprehensively analyze the traffic environment data under night or low-light conditions, and effectively improve the driving safety and comfort through multiple core links such as environmental perception and data processing, light field display and augmented reality, multi-scene recognition and adaptive adjustment; at the same time, the system also promotes the further development and application of intelligent driving, intelligent transportation and advanced driver assistance technologies, and provides strong support for building a more intelligent and safe transportation environment.

[0012] Furthermore, the environment perception and data processing module processes and analyzes the captured traffic environment data to obtain the traffic environment feature vector and determine potential dangers and obstacles. The process of generating risk obstacle signals includes:

[0013] The traffic environment data set captured by the sensor is marked as D = {d ij}; where d represents the traffic environment data captured by the sensor, that is, the specific value of the traffic environment data; i represents different types of sensors, which are used to distinguish different types of sensor sources, where i = 1, 2, ..., k, k represents the total number of sensor types, for example, when k = 3, it corresponds to infrared cameras, laser radars, and millimeter-wave radars respectively; j represents the index of data points captured by each sensor within a preset time period (for example, the preset time period is within the past hour), which is used to identify a single data point in the data captured by each sensor, where j = 1, 2, ..., n i , n represents the number of data points captured by the sensor, n i Represents the number of data points captured by the i-th sensor within a preset time period. For example, for an infrared camera (assuming i = 1), j = 1, 2, ..., n 1 Indicates the first, second, ... nth image captured by the infrared camera within the preset time period 1 data points;

[0014] Combined with the traffic environment data captured by the sensor, the traffic environment data preprocessing function ψ i =d ij , preprocess the traffic environment data of each sensor, such as removing noise, normalizing, etc.; where ψ represents the symbol of the preprocessing function, which is a function relationship that converts the original sensor data into a form more suitable for subsequent processing;

[0015] The traffic environment data after preprocessing is D'={ψ i (d ij )};

[0016] Through the traffic environment feature extraction function Φ(D') = {f m}, extract M traffic environment feature vectors f from the preprocessed traffic environment data D' m ; Among them, these traffic environment feature vectors contain relevant features of road information, pedestrians and other obstacles. Specifically, road information includes road width, curvature, traffic signs and signals, etc., pedestrians include pedestrians' positions, speeds, walking directions, etc., and other obstacles may include road construction, parked vehicles, animals, etc.; Φ represents the symbol of the traffic environment feature extraction function, which is a data conversion relationship that converts the pre-processed traffic environment data into a vector form with representative features; m represents the traffic environment feature vector index, m = 1, 2, ..., M, M represents the number of traffic environment feature vectors;

[0017] The extracted traffic environment feature vectors are subjected to data fusion; the specific formula is as follows: In the formula, F f Represents the fused traffic environment feature vector; αi represents the fusion weight of the traffic environment feature vector of the i-th sensor data, and Fusion weight α i It can be determined based on factors such as the accuracy and reliability of the sensor and the importance of different driving scenarios. For example, if the accuracy of the lidar is higher in a certain driving scenario, the corresponding α i The value is relatively large;

[0018] According to the risk obstacle determination formula, determine whether there are potential dangers and obstacles; the specific formula is as follows: In the formula, RO represents the traffic environment risk obstacle value; g represents the index of the fused traffic environment feature vector; I represents the number of fused traffic environment feature vectors; w g is the fused traffic environment feature vector F f The corresponding weight indicates the importance of each feature in determining the risk obstacle of the traffic environment; b is the bias term, which is used to adjust the benchmark for determination;

[0019] When analyzing the calculated traffic environment risk obstacle value, if RO>0, it is determined that there are potential dangers and obstacles, and a risk obstacle signal is generated; if RO≤0, it is determined to be a safe state.

[0020] Furthermore, the light field display and augmented reality module obtains the risk obstacle signal of the environment perception and data processing module, and accurately presents and augments the traffic environment feature vector corresponding to the risk obstacle signal to embed it into the driving field of view, and the process includes:

[0021] Obtain the risk obstacle signal output by the environment perception and data processing module and mark it as S; when S=1, it means there is a risk, and when S=0, it means there is no risk;

[0022] The traffic environment feature vector set corresponding to the risk obstacle signal is defined as TE = {t l},l=1,2,...,L; where t represents the traffic environment feature vector, which is the basic element constituting the traffic environment feature vector set; l is the index, representing the lth traffic environment feature vector in the traffic environment feature vector set; L represents the number of traffic environment feature vectors;

[0023] The light field display function is used to optimize and present each traffic environment feature vector; the specific formula is as follows: Where G(TE) is the value calculated based on the traffic environment feature vector set TE; l is the adjustment factor corresponding to the lth traffic environment feature vector, which determines the influence of each traffic environment feature vector on the light field display effect; Π represents the product symbol, and all (1+λl t l ) term, that is, (1+λ 1 t 1 )×(1+λ 2 t 2 )×…×(1+λ L t L );

[0024] Assume W(x,y,z) is the light field intensity at the point (x,y,z) in the three-dimensional space, then the light field display is expressed as W(x,y,z)=G(TE)·(TE,(x,y,z)); where x, y, z represent the x-axis, y-axis, and z-axis of the three-dimensional space coordinates, respectively;

[0025] It can be understood that by optimizing and presenting each traffic environment feature vector, the traffic environment feature vector set E is presented in a three-dimensional manner on the road in front of the driver;

[0026] Get driving assistance safety information A = {a pq}, including lane departure warning, blind spot monitoring, pedestrian detection, etc.; where p represents different attributes of driving assistance safety information, p = 1, 2, ..., P, where P represents the number of attributes of driving assistance safety information, q represents the parameters corresponding to the driving assistance safety information, q = 1, 2, ..., Q, where Q represents the number of parameters corresponding to the driving assistance safety information;

[0027] An augmented reality embedding function is established; wherein the augmented reality embedding function formula is: In the formula, ζ(A,S) is the function symbol, β pq is the embedding weight corresponding to the driving assistance safety information parameter, indicating the importance of the driving assistance safety information parameter when embedded into the driving field of view;

[0028] When S=1, the driving assistance safety information A is embedded into the driving field of view W(x, y, z) through the augmented reality embedding function to facilitate subsequent multi-scene recognition and adaptive adjustment.

[0029] Furthermore, the multi-scene recognition and adaptive adjustment module intelligently recognizes multiple scenes of the driving environment based on the data processing results of the light field display and augmented reality modules, including:

[0030] Assume that the feature vector set of the driving environment is X = (X 1 ,X 2 ,...,X C ), where C represents the number of driving environment feature vectors, X 1 Indicates the road type (such as highway, urban road, rural road, etc.), X 2Indicates weather conditions (such as sunny, rainy, foggy, etc.), X 3 Indicates vehicle status (involving vehicle speed, position, etc.);

[0031] A driving scene recognition formula is established to analyze the characteristic vector X of the driving environment and identify the current driving scene; wherein the driving scene recognition formula is:

[0032] Where SR(X) represents the driving scene recognition result, c represents the index of the driving environment feature vector, and c = 1, 2, ..., C; w c It represents the weight corresponding to the c-th driving environment feature vector, that is, the importance of each driving environment feature vector in scene recognition.

[0033] Furthermore, the multi-scenario recognition and adaptive adjustment module automatically adjusts the light field display position according to the driving scene recognition result to adapt to different driving scenes, including:

[0034] Combined with the driving scene recognition result SR(X), the light field display position is optimized to obtain the driver's line of sight direction θ=(θ x ,θ y );

[0035] Use the formula to optimize the light field display position: In the formula, (x 0 ,y 0 ) represents the initial light field display position, △x, △y represent the offset adjusted according to the line of sight direction θ, and the matrix It is used to rotate and adjust according to the angle of the line of sight, and △x and △y are used to further fine-tune the position, so as to dynamically adjust the light field display position according to the driver's line of sight direction θ, which is beneficial to reduce line of sight distraction and reduce the impact of glare, facilitate real-time acquisition of driving environment information, and improve interaction efficiency.

[0036] Furthermore, the process in which the intelligent monitoring and fault warning module obtains the operating status data of the safe driving assistance system and performs quantitative analysis to obtain the operating status index value includes:

[0037] Monitor the operating status of the safe driving assistance system in real time and obtain the operating status data of the safe driving assistance system Y=(E (1) ,E (2) ,H (1) ,H (2) ), where E 1 、E 2 Respectively represent the speed and accuracy of data receiving and processing, H 1 , H 2 They represent the stability and clarity of light field display respectively;

[0038] Perform quantitative analysis on the operating status data; the quantitative analysis formula is as follows:

[0039]

[0040] Where, RSR(Y) represents the operating status index value; N 1 Indicates the number of data receiving and processing tasks; N 2 The number of evaluation items that represent the accuracy of data reception and processing tasks; E (1)min 、E (1)max They respectively represent the minimum and maximum speeds allowed for data receiving and processing tasks; u represents the index of the data receiving and processing task; v represents the evaluation item index of the accuracy of the data receiving and processing task; μ, γ, and η are different preset proportional coefficients used to adjust the degree of influence of different factors in the operating status indicator value; it can be understood that the speed and accuracy of data receiving and processing are obtained by using professional data processing software (such as AutoCAD, ArcGIS, etc.) and precise data acquisition methods, and the stability and clarity of light field display depend on light field camera technology and advanced image processing algorithms.

[0041] Furthermore, the intelligent monitoring and fault warning module combines the operating status indicator value to establish a fault warning mechanism, detects whether there is a fault or abnormality in the current operating status, and issues a warning prompt, including the following process:

[0042] The fault warning index is calculated using a formula; the specific formula is as follows:

[0043] In the formula, T(Y) represents the fault warning index; r represents the index of the operating status indicator; R represents the number of operating status indicators; δ r represents the adjustment factor corresponding to the rth operating status indicator, which is used to adjust the influence of the operating status indicator on the fault warning;

[0044] Set the fault warning threshold T 0 、T 1 、T 2 , and T 0 <T 1 <T 2 ;

[0045] When analyzing based on the calculated fault warning index, if the fault warning index T(Y)≤T 0 , it is determined that there is no fault or abnormality at present, and no warning prompt is issued; if the fault warning index T 0 <T(Y)≤T 1 , a low-level warning is issued, indicating that there may be a minor fault or abnormality and attention should be paid; if the fault warning index T1 <T(Y)≤T 2 , a medium-level warning is issued, indicating that there may be a moderate fault or abnormality and corresponding measures need to be taken; if the fault warning index T(Y)>T 2 , a high-level warning is issued, indicating that there is a serious fault or abnormality and immediate action is required.

[0046] Compared with the existing technology, the advantages of the light field display safety driving assistance system and method provided by the present invention based on the vehicle platform are:

[0047] 1. The present invention integrates multiple sensors such as infrared cameras, laser radars, millimeter-wave radars, etc. to capture driving environment data, analyze the captured environmental data, determine potential dangers and obstacles, and generate risk obstacle signals to improve the accuracy and reliability of environmental perception; by accurately presenting and enhancing the reality of the traffic environment feature vectors corresponding to the risk obstacle signals, not only the readability of the information is improved, but also the driver can obtain key information without distracting his attention, which helps to improve the driver's reaction speed;

[0048] 2. The present invention intelligently identifies various scenes of the driving environment, including road types, weather conditions, and vehicle conditions; according to the driving scene recognition results, the light field display position is automatically adjusted to adapt to different driving scenes, which can reduce the driver's visual distraction and glare, facilitate real-time acquisition of driving environment information, and improve interaction efficiency and driving safety;

[0049] 3. The present invention obtains the operating status data of the safe driving assistance system and performs quantitative analysis to obtain the operating status index value. At the same time, a fault warning mechanism is established in combination with the operating status index value to detect whether there is a fault or abnormality in the current operating status. This helps the driver or system maintenance personnel to better understand the system status and take more appropriate operations, and provides data support for system maintenance and optimization, thereby improving the reliability and durability of the system.

[0050] In summary, through intelligent perception, processing, presentation and monitoring methods, the system of the present invention can significantly improve the safety, comfort and convenience of driving, bring a more pleasant and safe driving experience to the driver, and ensure the efficient and stable operation of the subsequent light field display safety driving assistance system based on the vehicle platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a module diagram of the light field display safety driving assistance system based on the vehicle-mounted computer platform proposed by the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the implementation regulations described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] Reference Figure 1 , based on the light field display safety driving assistance system under the vehicle platform, the system includes an environmental perception and data processing module, a light field display and augmented reality module, a multi-scene recognition and adaptive adjustment module, and an intelligent monitoring and fault warning module;

[0054] Environmental perception and data processing module: This module integrates a sensor group (such as infrared cameras, laser radars, millimeter-wave radars, etc.) to capture traffic environment data at night or in low light conditions, including road information, vehicles, pedestrians and other obstacles, and processes and analyzes the captured traffic environment data to obtain traffic environment feature vectors. At the same time, it determines potential dangers and obstacles and generates risk obstacle signals.

[0055] Light field display and augmented reality module: obtains risk obstacle signals from the environment perception and data processing module, and accurately presents and augments the traffic environment feature vectors corresponding to the risk obstacle signals to embed them into the driving field of view;

[0056] Multi-scenario recognition and adaptive adjustment module: Based on the data processing results of the light field display and augmented reality modules, it intelligently recognizes multiple scenarios of the driving environment, including road types (such as highways, urban roads, rural roads, etc.), weather conditions (such as sunny days, rainy days, foggy days, etc.) and vehicle conditions (involving vehicle speed, position, etc.); according to the driving scene recognition results, it automatically adjusts the light field display position to adapt to different driving scenes;

[0057] Intelligent monitoring and fault warning module: After completing multi-scenario recognition and adaptive adjustment, continue to monitor the overall operating status of the safe driving assistance system, obtain the operating status data of the safe driving assistance system, and perform quantitative analysis to obtain the operating status index value; combine the operating status index value to establish a fault warning mechanism to detect whether there is a fault or abnormality in the current operating status.

[0058] It should be noted that the application object of the light field display safety driving assistance system based on the vehicle-mounted platform in the embodiment of the present invention can be the on-board system management in the field of automobile intelligent driving. Specifically, it can be through an integrated sensor group (such as infrared camera, lidar, millimeter wave radar, etc.) to comprehensively analyze the traffic environment data under night or low light conditions, and effectively improve the driving safety and comfort through multiple core links such as environmental perception and data processing, light field display and augmented reality, multi-scene recognition and adaptive adjustment; at the same time, the system also promotes the further development and application of intelligent driving, intelligent transportation and advanced driving assistance technologies, and provides strong support for building a more intelligent and safe transportation environment.

[0059] The environment perception and data processing module processes and analyzes the captured traffic environment data to obtain the traffic environment feature vector and determine potential dangers and obstacles. The steps of generating risk obstacle signals include:

[0060] Step 101: mark the traffic environment data set captured by the sensor as D = {d ij}; where d represents the traffic environment data captured by the sensor, that is, the specific value of the traffic environment data; i represents different types of sensors, which are used to distinguish different types of sensor sources, where i = 1, 2, ..., k, k represents the total number of sensor types, for example, when k = 3, it corresponds to infrared cameras, laser radars, and millimeter-wave radars respectively; j represents the index of data points captured by each sensor within a preset time period (for example, the preset time period is within the past hour), which is used to identify a single data point in the data captured by each sensor, where j = 1, 2, ..., n i , n represents the number of data points captured by the sensor, n i Represents the number of data points captured by the i-th sensor within a preset time period. For example, for an infrared camera (assuming i = 1), j = 1, 2, ..., n 1 Indicates the first, second, ... nth image captured by the infrared camera within the preset time period 1 data points;

[0061] Step 102: Combine the traffic environment data captured by the sensor and use the traffic environment data preprocessing function ψ i =d ij , preprocess the traffic environment data of each sensor, such as removing noise, normalizing, etc.; where ψ represents the symbol of the preprocessing function, which is a function relationship that converts the original sensor data into a form more suitable for subsequent processing;

[0062] Step 103: The traffic environment data after preprocessing is D'={ψ i (d ij )};

[0063] Step 104: Extract the traffic environment feature function Φ(D')={f m}, extract M traffic environment feature vectors f from the preprocessed traffic environment data D' m ; Among them, these traffic environment feature vectors contain relevant features of road information, pedestrians and other obstacles. Specifically, road information includes road width, curvature, traffic signs and signals, etc., pedestrians include pedestrian position, speed, walking direction, etc., and other obstacles may include road construction, parked vehicles, animals, etc.; Φ represents the symbol of the traffic environment feature extraction function, which is a data conversion relationship that converts the pre-processed traffic environment data into a vector form with representative features; m represents the traffic environment feature vector index, m = 1, 2, ..., M, M represents the number of traffic environment feature vectors;

[0064] Step 105: perform data fusion on the extracted traffic environment feature vectors; wherein the specific formula is as follows: In the formula, F f Represents the fused traffic environment feature vector; α i represents the fusion weight of the traffic environment feature vector of the i-th sensor data, and Fusion weight α i It can be determined based on factors such as the accuracy and reliability of the sensor and the importance of different driving scenarios. For example, if the accuracy of the lidar is higher in a certain driving scenario, the corresponding α i The value is relatively large;

[0065] Step 106: Determine whether there are potential dangers and obstacles based on a risk obstacle determination formula; the specific formula is as follows: In the formula, RO represents the traffic environment risk obstacle value; g represents the index of the fused traffic environment feature vector; I represents the number of fused traffic environment feature vectors; w g is the fused traffic environment feature vector F f The corresponding weight indicates the importance of each feature in determining the risk obstacle of the traffic environment; b is the bias term, which is used to adjust the benchmark for determination;

[0066] Step 107: When analyzing the calculated traffic environment risk obstacle value, if RO>0, it is determined that there are potential dangers and obstacles, and a risk obstacle signal is generated; if RO≤0, it is determined to be a safe state.

[0067] The light field display and augmented reality module obtains the risk obstacle signal of the environment perception and data processing module, and accurately presents and augments the traffic environment feature vector corresponding to the risk obstacle signal to embed it into the driving field of view, including the following steps:

[0068] Step 201: Obtain the risk obstacle signal output by the environment perception and data processing module and mark it as S; wherein, when S=1, it indicates that there is a risk, and when S=0, it indicates that there is no risk;

[0069] Step 202: define the traffic environment feature vector set corresponding to the risk obstacle signal as TE = {t l},l=1,2,...,L; where t represents the traffic environment feature vector, which is the basic element constituting the traffic environment feature vector set; l is the index, representing the lth traffic environment feature vector in the traffic environment feature vector set; L represents the number of traffic environment feature vectors;

[0070] Step 203: Use the light field display function to optimize and present each traffic environment feature vector; wherein the specific formula is as follows: Where G(TE) is the value calculated based on the traffic environment feature vector set TE; l is the adjustment factor corresponding to the lth traffic environment feature vector, which determines the influence of each traffic environment feature vector on the light field display effect; Π represents the product symbol, and all (1+λ l t l ) term, that is, (1+λ 1 t 1 )×(1+λ 2 t 2 )×…×(1+λ L t L );

[0071] Step 204: Let W(x,y,z) be the light field intensity at the point (x,y,z) in the three-dimensional space, then the light field display is represented by W(x,y,z)=G(TE)·(TE,(x,y,z)); wherein x, y, z represent the x-axis, y-axis, and z-axis of the three-dimensional space coordinates, respectively;

[0072] In steps 203-204, by optimizing and presenting each traffic environment feature vector, the traffic environment feature vector set E is presented in a three-dimensional manner on the road in front of the driver;

[0073] Step 205: Obtain driving assistance safety information A = {a pq}, including lane departure warning, blind spot monitoring, pedestrian detection, etc.; where p represents different attributes of driving assistance safety information, p = 1, 2, ..., P, where P represents the number of attributes of driving assistance safety information, q represents the parameters corresponding to the driving assistance safety information, q = 1, 2, ..., Q, where Q represents the number of parameters corresponding to the driving assistance safety information;

[0074] Step 206: Establish an augmented reality embedding function; wherein the augmented reality embedding function formula is: In the formula, ζ(A,S) is the function symbol, β pq is the embedding weight corresponding to the driving assistance safety information parameter, indicating the importance of the driving assistance safety information parameter when embedded into the driving field of view;

[0075] Step 207: When S=1, the driving assistance safety information A is embedded into the driving field of view W(x, y, z) through the augmented reality embedding function to facilitate subsequent multi-scene recognition and adaptive adjustment.

[0076] The multi-scene recognition and adaptive adjustment module intelligently recognizes multiple scenes of the driving environment based on the data processing results of the light field display and augmented reality modules, and automatically adjusts the light field display position to adapt to different driving scenes according to the driving scene recognition results. The steps include:

[0077] Step 301: Assume that the feature vector set of the driving environment is X=(X 1 ,X 2 ,...,X C ), where C represents the number of driving environment feature vectors, X 1 Indicates the road type (such as highway, urban road, rural road, etc.), X 2 Indicates weather conditions (such as sunny, rainy, foggy, etc.), X 3 Indicates vehicle status (involving vehicle speed, position, etc.);

[0078] Step 302: Establish a driving scene recognition formula to analyze the characteristic vector X of the driving environment and identify the current driving scene; wherein the driving scene recognition formula is:

[0079] Where SR(X) represents the driving scene recognition result, c represents the index of the driving environment feature vector, and c = 1, 2, ..., C; w c represents the weight corresponding to the c-th driving environment feature vector, that is, the importance of each driving environment feature vector in identifying the scene;

[0080] Step 303: Combine the driving scene recognition result SR(X) to optimize the light field display position and obtain the driver's line of sight direction θ=(θ x ,θ y );

[0081] Step 304: Optimize the light field display position using the formula:

[0082] In the formula, (x 0 ,y 0) represents the initial light field display position, △x, △y represent the offset adjusted according to the line of sight direction θ, and the matrix It is used to rotate and adjust according to the angle of the line of sight, and △x and △y are used to further fine-tune the position, so as to dynamically adjust the light field display position according to the driver's line of sight direction θ, which is beneficial to reduce line of sight distraction and reduce the impact of glare, facilitate real-time acquisition of driving environment information, and improve interaction efficiency.

[0083] The intelligent monitoring and fault warning module obtains the operating status data of the safe driving assistance system, and performs quantitative analysis to obtain the operating status index value. The steps of establishing a fault warning mechanism based on the operating status index value to detect whether there is a fault or abnormality in the current operating state include:

[0084] Step 401: monitor the operating status of the safe driving assistance system in real time and obtain the operating status data Y of the safe driving assistance system. (1) ,E (2) ,H (1) ,H (2) ), where E 1 、E 2 Respectively represent the speed and accuracy of data receiving and processing, H 1 , H 2 They represent the stability and clarity of light field display respectively;

[0085] Step 402: Perform quantitative analysis on the running status data; wherein the quantitative analysis formula is as follows:

[0086]

[0087] Where, RSR(Y) represents the operating status index value; N 1 Indicates the number of data receiving and processing tasks; N 2 The number of evaluation items that represent the accuracy of data reception and processing tasks; E (1)min 、E (1)max They represent the minimum and maximum speeds allowed for the data receiving and processing tasks respectively; u represents the index of the data receiving and processing task; v represents the evaluation item index of the accuracy of the data receiving and processing task; μ, γ, η are different preset proportional coefficients used to adjust the influence of different factors on the operating status index value;

[0088] In step 402, the speed and accuracy of data reception and processing are obtained by using professional data processing software (such as AutoCAD, ArcGIS, etc.) and precise data acquisition methods, and the stability and clarity of light field display depend on light field camera technology and advanced image processing algorithms;

[0089] Step 403: Calculate the fault warning index using a formula; wherein the specific formula is as follows:

[0090] In the formula, T(Y) represents the fault warning index; r represents the index of the operating status indicator; R represents the number of operating status indicators; δ r represents the adjustment factor corresponding to the rth operating status indicator, which is used to adjust the influence of the operating status indicator on the fault warning;

[0091] Step 404: Setting the fault warning threshold T 0 、T 1 、T 2 , and T 0 <T 1 <T 2 ;

[0092] Step 405: When analyzing the fault warning index obtained by calculation, if the fault warning index T(Y)≤T 0 , it is determined that there is no fault or abnormality at present, and no warning prompt is issued; if the fault warning index T 0 <T(Y)≤T 1 , a low-level warning is issued, indicating that there may be a minor fault or abnormality and attention should be paid; if the fault warning index T 1 <T(Y)≤T 2 , a medium-level warning is issued, indicating that there may be a moderate fault or abnormality and corresponding measures need to be taken; if the fault warning index T(Y)>T 2 , a high-level warning is issued, indicating that there is a serious fault or abnormality and immediate action is required.

[0093] In the embodiment of the present invention, by integrating multiple sensors (such as infrared cameras, laser radars, millimeter wave radars, etc.), traffic environment data, including road information, vehicles, pedestrians and other obstacles, are fully captured. By processing and analyzing these data, traffic environment feature vectors are obtained, and potential dangers and obstacles are determined, and risk obstacle signals are generated, so as to provide timely and accurate traffic information to the driver in the future. By presenting the traffic environment feature vectors in a three-dimensional manner, the driver can understand the surrounding traffic environment more intuitively, improve the immersion and safety of driving, and at the same time obtain driving assistance safety information, such as lane departure warning, blind spot monitoring, pedestrian detection, etc., and embed it into the driving field of view, further enhancing the driver's perception ability, and automatically adjusting the light field display position by identifying the current driving scene. , to adapt to different driving environments, this adaptive adjustment can reduce the driver's visual distraction and glare, facilitate the driver to obtain driving environment information in real time, improve interaction efficiency and driving safety, and monitor the operating status of the safe driving assistance system in real time, and obtain the operating status data of the safe driving assistance system, and quantitatively analyze the operating status index value, so as to achieve a comprehensive understanding and evaluation of the system performance. By combining the operating status index value, a fault warning mechanism is established, which can detect whether there is a fault or abnormality in the current operating state, which helps to find problems in time and avoid the occurrence or escalation of faults. By issuing different levels of warning prompts according to the size of the fault warning index, it is ensured that the driver or system maintenance personnel can take corresponding measures according to the severity of the warning, so as to effectively deal with potential risks. In summary, the example of the present invention solves the problems of low safety and untimely response of current intelligent driving. In actual situations, more data and contextual information may be needed to make specific decisions and optimization plans.

[0094] In addition, the formulas involved in the above are all calculated by removing dimensions and taking their numerical values. They are a formula that is closest to the actual situation obtained by collecting a large amount of data and performing software simulation. The proportional coefficient in the formula and the various preset thresholds in the analysis process are set by technical personnel in this field according to actual conditions or obtained by simulating a large amount of data; the size of the proportional coefficient is to quantify each parameter to obtain a specific value for subsequent comparison. The size of the proportional coefficient depends on the amount of sample data and the preliminary setting of the corresponding processing coefficient for each group of sample data by technical personnel in this field; as long as it does not affect the proportional relationship between the parameter and the quantized value.

[0095] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically based on the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0096] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0097] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0098] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0099] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0100] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0101] Secondly: In the drawings of the embodiments disclosed in the present invention, only the structures related to the embodiments disclosed in the present invention are involved, and other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of the present invention can be combined with each other;

[0102] Finally: The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A light field display safety driving assistance system based on a vehicle computer platform, characterized by: It includes environment perception and data processing module, light field display and augmented reality module, multi-scene recognition and adaptive adjustment module, and intelligent monitoring and fault warning module; Environmental perception and data processing module: This module integrates a sensor group to capture traffic environment data at night or in low light conditions, including road information, vehicles, pedestrians and other obstacles, and processes and analyzes the captured traffic environment data to obtain traffic environment feature vectors. It also determines potential dangers and obstacles and generates risk obstacle signals. Light field display and augmented reality module: obtains risk obstacle signals from the environment perception and data processing module, and accurately presents and augments the traffic environment feature vectors corresponding to the risk obstacle signals to embed them into the driving field of view; Multi-scenario recognition and adaptive adjustment module: Based on the data processing results of the light field display and augmented reality modules, it intelligently recognizes multiple scenarios of the driving environment, including road types, weather conditions, and vehicle conditions; according to the driving scene recognition results, it automatically adjusts the light field display position to adapt to different driving scenarios; Intelligent monitoring and fault warning module: After completing multi-scenario recognition and adaptive adjustment, it continues to monitor the overall operating status of the safe driving assistance system, obtains the operating status data of the safe driving assistance system, and performs quantitative analysis to obtain the operating status index value; A fault warning mechanism is established in combination with the operating status indicator values ​​to detect whether there are faults or abnormalities in the current operating status.

2. The light field display safety driving assistance system based on the vehicle computer platform according to claim 1 is characterized in that: The environment perception and data processing module processes and analyzes the captured traffic environment data to obtain the traffic environment feature vector and determine potential dangers and obstacles. The process of generating risk obstacle signals includes: The traffic environment data set captured by the sensor is marked as D = {d ij }; where d represents the traffic environment data captured by the sensor; i represents different types of sensors, which are used to distinguish different types of sensor sources, where i = 1, 2, ..., k, and k represents the total number of sensor types; j represents the index of the data point captured by each sensor within a preset time period, which is used to identify a single data point in the data captured by each sensor, where j = 1, 2, ..., n i , n represents the number of data points captured by the sensor, n i represents the number of data points captured by the i-th sensor within the preset time period; Combined with the traffic environment data captured by the sensor, the traffic environment data preprocessing function ψ i =d ij , preprocess the traffic environment data of each sensor; where ψ represents the symbol of the preprocessing function; The traffic environment data after preprocessing is D'={ψ i (d ij )}; Through the traffic environment feature extraction function Φ(D') = {f m }, extract M traffic environment feature vectors f from the preprocessed traffic environment data D' m ; Wherein, Φ represents the symbol of the traffic environment feature extraction function; m represents the traffic environment feature vector index, m=1,2,...,M, M represents the number of traffic environment feature vectors; The extracted traffic environment feature vectors are subjected to data fusion; the specific formula is as follows: In the formula, F f Represents the fused traffic environment feature vector; α i represents the fusion weight of the traffic environment feature vector of the i-th sensor data, and According to the risk obstacle determination formula, determine whether there are potential dangers and obstacles; the specific formula is as follows: In the formula, RO represents the traffic environment risk obstacle value; g represents the index of the fused traffic environment feature vector; I represents the number of fused traffic environment feature vectors; w g is the fused traffic environment feature vector F f The corresponding weight; b is the bias term, which is used to adjust the judgment benchmark; When analyzing the calculated traffic environment risk obstacle value, if RO>0, it is determined that there are potential dangers and obstacles, and a risk obstacle signal is generated; if RO≤0, it is determined to be a safe state.

3. The light field display safety driving assistance system based on the vehicle computer platform according to claim 1 is characterized in that: The light field display and augmented reality module obtains the risk obstacle signal of the environment perception and data processing module, and accurately presents and augments the traffic environment feature vector corresponding to the risk obstacle signal to embed it into the driving field of view. The process includes: Obtain the risk obstacle signal output by the environment perception and data processing module and mark it as S; when S=1, it means there is a risk, and when S=0, it means there is no risk; The traffic environment feature vector set corresponding to the risk obstacle signal is defined as TE = {t l },l=1,2,...,L; where t represents the traffic environment feature vector, which is the basic element constituting the traffic environment feature vector set; l is the index, representing the lth traffic environment feature vector in the traffic environment feature vector set; L represents the number of traffic environment feature vectors; The light field display function is used to optimize and present each traffic environment feature vector; the specific formula is as follows: In the formula, λ l is the adjustment factor corresponding to the lth traffic environment characteristic vector; Π represents the product symbol, and all (1+λ l t l ) term, that is, (1+λ1t1)×(1+λ2t2)×…×(1+λ L t L ); Assume W(x,y,z) is the light field intensity at the point (x,y,z) in the three-dimensional space, then the light field display is expressed as W(x,y,z)=G(TE)·(TE,(x,y,z)); where x, y, z represent the x-axis, y-axis, and z-axis of the three-dimensional space coordinates, respectively; Get driving assistance safety information A = {a pq }; where p represents different attributes of driving assistance safety information, p=1,2,...,P, where P represents the number of attributes of driving assistance safety information, q represents the parameters corresponding to the driving assistance safety information, q=1,2,...,Q, where Q represents the number of parameters corresponding to the driving assistance safety information; An augmented reality embedding function is established; wherein the augmented reality embedding function formula is: In the formula, ζ(A,S) is the function symbol, β pq is the embedding weight corresponding to the driving assistance safety information parameter, indicating the importance of the driving assistance safety information parameter when embedded into the driving field of view; When S=1, the driving assistance safety information A is embedded into the driving field of view W(x, y, z) through the augmented reality embedding function.

4. The light field display safety driving assistance system based on the vehicle computer platform according to claim 1 is characterized in that: The multi-scene recognition and adaptive adjustment module intelligently recognizes various scenes of the driving environment based on the data processing results of the light field display and augmented reality modules. The process includes: Assume that the feature vector set of the driving environment is X = (X1, X2, ..., X C ), where C represents the number of driving environment feature vectors, X1 represents the road type, X2 represents the weather conditions, and X3 represents the vehicle condition; A driving scene recognition formula is established to analyze the characteristic vector X of the driving environment and identify the current driving scene; wherein the driving scene recognition formula is: Where SR(X) represents the driving scene recognition result, c represents the index of the driving environment feature vector, and c = 1, 2, ..., C; w c It represents the weight corresponding to the c-th driving environment feature vector, that is, the importance of each driving environment feature vector in scene recognition.

5. The light field display safety driving assistance system based on the vehicle computer platform according to claim 4 is characterized in that: The process by which the multi-scenario recognition and adaptive adjustment module automatically adjusts the light field display position according to the driving scenario recognition result to adapt to different driving scenarios includes: Combined with the driving scene recognition result SR(X), the light field display position is optimized to obtain the driver's line of sight direction θ=(θ x ,θ y ); Use the formula to optimize the light field display position: In the formula, (x0, y0) represents the initial light field display position, △x, △y represent the offset adjusted according to the line of sight direction θ, and the matrix It is used to perform rotation adjustment according to the angle of the line of sight, and △x and △y are used to further fine-tune the position, thereby dynamically adjusting the light field display position according to the driver's line of sight direction θ.

6. The light field display safety driving assistance system based on the vehicle computer platform according to claim 1 is characterized in that: The process by which the intelligent monitoring and fault warning module obtains the operation status data of the safe driving assistance system and conducts quantitative analysis to obtain the operation status index value includes: Monitor the operating status of the safe driving assistance system in real time and obtain the operating status data of the safe driving assistance system Y=(E (1) ,E (2) ,H (1) ,H (2) ); E1 and E2 represent the speed and accuracy of data reception and processing, respectively; H1 and H2 represent the stability and clarity of light field display, respectively; Conduct quantitative analysis on the operation status data; among them, the quantitative analysis formula is as follows: Where RSR(Y) represents the operating status indicator value; N1 represents the number of data receiving and processing tasks; N2 represents the number of evaluation items for the accuracy of data receiving and processing tasks; E (1)min 、E (1)max They respectively represent the minimum and maximum speeds allowed for the data receiving and processing tasks; u represents the index of the data receiving and processing tasks; v represents the evaluation item index of the accuracy of the data receiving and processing tasks; μ, γ, and η are different preset proportional coefficients used to adjust the degree of influence of different factors on the operating status index value.

7. The light field display safety driving assistance system based on the vehicle computer platform according to claim 6 is characterized in that: The process by which the intelligent monitoring and fault warning module combines the operation status index value to establish a fault warning mechanism, detects whether there is a fault or abnormality in the current operation status, and issues a warning prompt includes: Calculate the fault warning index using the formula; among them, the specific formula is as follows: In the formula, T(Y) represents the fault warning index; r represents the index of the operating status indicator; R represents the number of operating status indicators; δ r represents the adjustment factor corresponding to the rth operating status indicator, which is used to adjust the influence of the operating status indicator on the fault warning; Set the fault warning thresholds T0, T1, and T2, and T0 < T1 < T2; When analyzing according to the calculated fault warning index, if the fault warning index T(Y) ≤ T0, it is determined that there is no fault or abnormality at present, and no warning prompt is issued; if the fault warning index T0 < T(Y) ≤ T1, a low-level warning prompt is issued; if the fault warning index T1 < T(Y) ≤ T2, a medium-level warning prompt is issued; if the fault warning index T(Y) > T2, a high-level warning prompt is issued.

Images